Speakers: Jade Xin (TensorPRO)
Track: AI Engineering
Long context solves within-session coherence, not cross-session persistence. Once an assistant starts storing memories, a quieter failure appears: its own inferences can be written back, retrieved in later sessions, and presented as user facts. Hallucinations then compound over time. We encountered this while building EDN, TensorPRO's external memory middleware. Our response was intentionally less "smart": make provenance structural, and keep generative models out of the retrieval path. Every record carries one of seven provenance classes, including user-stated, document-verified, and AI-generated. A single write chokepoint enforces those labels. At session start, EDN assembles a provenance-gated Session Initialization Package (SIP) using cosine-similarity retrieval. AI-generated records cannot enter factual sections; if one crosses that boundary, the context builder fails loudly. On LongMemEval-S (sealed evaluation, three runs, GPT-4o judge), this design increased answer accuracy from 45.5% with full-history context to 85.4%, while using 9.7× fewer input tokens. SIP assembly p95 was 30–35 ms. Ablations were equally instructive: storing complete turn pairs (round-level records) was the single largest gain, at +22.5 points, while diversity re-ranking (MMR) added no significant gain, so we removed it from the read path. Those results changed what we optimised: not ever more sophisticated retrieval, but governance over what is allowed to become memory.
This talk walks through the failure mode, architecture, evaluation, and trade-offs, including what we would change next. Attendees will leave with a practical framework for deciding when provenance over prediction and determinism over "smart" retrieval are worth the rigidity, and when a probabilistic pipeline is the better engineering choice.
https://data.webdirections.org/ai-engineer-sydney/sessions.json#672b2bfb-1045-4f51-9a27-88693840dff9
Full text on webdirections.org
.