Conference In 5 weeks
December
Monday
to Tue 8 December 2026 · Sydney time
You sign up with the host. foundercal only links to it.
Speakers: Khali Kalpa-Young (Alchymie)
Track: Research / Deep Tech
An eval is a definition of "good" written down well enough for a machine to grade against. In a real business, "good" isn't written down anywhere. It lives in someone's reaction, in what past work actually did, and in how a person talks. So the eval comes second. First you get "good" out of wherever it lives, and each place needs its own method.
The first place is the person's reaction. Two domain experts corrected a coaching agent in a Telegram group, and each correction became a commit to the agent's own instruction file, routed by whether the agent could apply it alone, had to ask me first, or should stay a note. Then a panel of critics I built to grade runs lied to me, rating a shallow run "demo-ready and close to trial-ready." The fix was not a better judge prompt. It was turning that failure into a permanent human-labelled case and refusing to trust any critic that couldn't reproduce my verdict.
The second place is the system's residue. For a second agent, the owner couldn't describe his own process, so I reconstructed it from past campaigns and tags on old lists and proposed it back for correction. The third place is his conversation history. I mined it for every correction and acceptance and built a simulated version of him, so new versions could run against him without him present.
A business can only hand an agent the work it can check, so the eval is the edge of what you can delegate. I'll cover the infrastructure: an agent that commits to its own instructions under authority rules, hot reload into live sessions, simulated users built from transcripts, a gate-first rubric, and critics that must pass a human-labelled test before they're trusted.
https://data.webdirections.org/ai-engineer-sydney/sessions.json#3d87474a-6760-4fe7-ada5-1d09ae1c7017
Full text on webdirections.org .
488 George Street, 2000, Sydney
Open in Maps ↗Founders pitch
Investors in the room
One of the year's big ones