Workshop In 4 weeks
December
Monday
to Tue 8 December 2026 · Sydney time
You sign up with the host. foundercal only links to it.
Your event? Manage this listing Report a change
Speakers: Simon Harloff (Dam Secure)
Track: Workshops
Can an AI security reviewer find vulnerabilities it has never seen—or is it simply repeating familiar benchmarks? In this three-hour hands-on workshop, you’ll build a security-review agent and the evaluations needed to tell whether it actually works.
Starting with a supplied TypeScript agent, you’ll examine how its context, tools, prompts and structured outputs shape the result. You’ll run a baseline evaluation, inspect the complete execution trace and investigate benchmark contamination: whether apparently strong performance comes from genuine reasoning or familiarity with public vulnerability data.
You’ll then create replayable test fixtures derived from real CVEs across several vulnerability classes, compare model and harness configurations using recall, runtime and cost, diagnose failed or incomplete results and improve the system against evidence.
You’ll leave with a working security-review agent, CVE-backed evaluation fixtures, a method for separating model failures from harness failures, example vulnerability cases and workshop notes you can adapt to your own language and security work.
Who it’s for
Intermediate AI engineers, application-security engineers and software engineers. You should be comfortable reading TypeScript and working in a terminal. Familiarity with application-security concepts or an agent framework such as the Vercel AI SDK will help, but is not required.
What you’ll need
Bring a laptop with Docker configured, GitHub access and a coding-agent subscription such as Claude Code, ChatGPT or Cursor. Setup and repository instructions will be provided before the workshop.
https://data.webdirections.org/ai-engineer-sydney/sessions.json#f11fc6f6-c3a4-4aac-857d-404ef83a3077
Full text on webdirections.org .
488 George Street, 2000, Sydney
Open in Maps ↗
Investors in the room
Founders pitch
Investors in the room
One of the year's big ones