Join us for a hands on evening with engineers from Arize and Qdrant exploring how top performing teams build reliable agents.
Brought to you by the AI Collective, Code & Coffee and Superoptimal.
Meet fellow engineers tackling the most common failure modes in high-stakes domain, like Legal, and learn to build agents that can be trusted.
🗓️ Event Agenda5:30pm - Doors open | Food, drinks & networking 6:00pm - Introductory statements 6:10pm - Demo: Voice Agents in Legal 6:15pm - Talk by Arize: Evals in Practice 6:30pm - Workshop by Qdrant: The Reliability Lab 8:00pm - Breakout for networking 📊 Arize Kickoff Talk: Evals in PracticeArize will kick things off with how teams building AI for high-stakes domains actually evaluate it: what to measure, what "good" looks like when there's no single right answer, and how to tell whether your system is improving or just changing.
We'll then bring those evaluation skills straight into a workshop setting.
🔎 Qdrant Workshop: Reliability Lab
Everybody knows legal AI can draft, summarize, and cite. So what's the problem?
Well, what happens when the right authority was never pulled? When a superseded version outranks the current one? Or when the search returns five documents that agree, and misses the one that didn't?
In this hands-on lab, engineers from Qdrant will work in a real retrieval stack and investigate realistic breakdowns in a legal AI system. For each one, your job is to figure out what went wrong, trace the answer back to the evidence, and decide whether you'd actually trust it.
You'll investigate:• Missing authority, key documents not being cited • Outdated sources, maintaining source freshness• Weak and conflicting evidence, broken source ranking• Upstream fixes, see how what you feed a model determines what it can possibly get right
By the end you'll have a working mental model for where high-stakes AI breaks, language for describing it to your team, and a starting point for evaluating the systems you're building.
💻 What You Need
A laptop. That's it. We provide the environment, the data, and the exercises: no setup, no coding environment, no prior machine learning experience. If you'd rather watch than work through it yourself, that's completely fine.
👋 Who Should Join?
Engineers building in high-stakes environments, legal engineers building legal products, search and relevance teams, and anyone responsible for deciding whether an AI system is good enough to put in front of users will enjoy this!
Come for the workshop. Stay for the food, drinks, and people. 🍕🍻
Space is extremely limited, so register early.
The AI Collective is a global non-profit building the human layer for the AI era. We unite 200,000+ leaders, builders, and stakeholders across 100+ forums worldwide to democratize the frontier, build trust, and coordinate how society navigates the rapid acceleration of technological progress.
