Designing Scalable and Rigorous Evaluations for AI Systems
Picture: Lorong AI

Workshop In 2 days

November
Friday
14:00–17:00 2026 · Singapore time

Designing Scalable and Rigorous Evaluations for AI Systems

Where
Lorong AI @ One-North 69 Ayer Rajah Cres., Singapore 139961
Host
Lorong AI
Entry
Free
Sign-up
Apply, the host picks
Apply on Luma

All Singapore events

You sign up with the host. foundercal only links to it.

Your event? Manage this listing Report a change

Other datesThe same event, on its next dates.

Who you meetThe people on the listing, and what the host tells us about the room.

  • Lorong AI Host
  • Mehul Shah Host
Investors Not stated The listing does not say
The format Hands-on, small group Workshop
Sign-up Apply, the host picks Free

AboutIn the host's words.

Note: For Government Officers (Technical Level 200 and above) Building an AI application is just the first step. Equally important is knowing whether it performs as expected, produces reliable results, and can be used safely in real-world settings.Evaluating AI agents brings another layer of complexity. Their behaviour unfolds across multi-step trajectories, where decisions depend on intermediate outputs, tool calls, changing state, and interactions with external systems. A system may produce a reasonable final answer while still taking an incorrect, unsafe, or inefficient path to get there.Meaningful agent Evals therefore need to look beyond final outputs to the trajectory as a whole: how the agent makes decisions, uses tools, responds to intermediate results, and whether errors compound along the way. This becomes especially important as agents take more autonomous actions and interact with external systems.This hands-on technical workshop introduces METIS, a specifications-driven approach to functionality evaluations that brings together best practices into a guided testing workflow, including assessing functional performance within agentic traces.Participants will leave with a practical framework and recommendations for designing repeatable evaluations for their own agentic systems, including how to build testing specifications, when to evaluate final outcomes, individual actions, or the session as a whole.

Who Should Attend

This workshop is primarily intended for developers, engineers, and technical product teams building or evaluating agentic AI systems. It will also be accessible to technology enthusiasts, researchers, and innovators interested in understanding how agent evaluations can be designed and carried out in practice.Some familiarity with LLM applications or AI agents would be useful, but no prior experience with evaluation frameworks is required.

Prerequisites

This is a hands-on, code-along workshop. To get the most out of the session, participants should:

  • Be comfortable with the command line: Running basic Bash CLI commands and navigating terminal environments.

  • Have Python proficiency: Practical experience with Python fundamentals and package management tooling (e.g., uv, pip, or virtual environments).

  • Understand AI basics: Foundational knowledge of LLM applications, prompt flows, or chatbots.

  • Bring a laptop for hands-on exercises:- Web: You will require a web browser to read the walkthrough documentation and interact with METIS' web UI

  • Local Setup (Optional): If you prefer to code along locally on your own machine, follow the METIS Installation Guide and ensure your local development environment has the necessary dependencies installed prior to the session.

Important Notes

  • Please bring a laptop for this hands-on session.

  • Registration begins at 2.00pm, with the workshop starting at 2.30pm.

  • This workshop is strictly open to Singapore Government officers only.

  • Please register using your government email address.

This workshop is part of GovTech's AI Practice division's efforts to uplift Whole of Government capabilities. Read more about how GovTech is building AI for public good.

Get involved: Learn more about Lorong AI | Speaker Sign-up | WhatsApp Community | LinkedIn | X

Full text on Luma .

Where

Lorong AI @ One-North

69 Ayer Rajah Cres., Singapore 139961

Open in Maps ↗

Same week in Singapore All 100 events

Nov
Wed 19:00
Meetup

SFF After Hours With The Founders ROCK on stage

Investors in the room

Singapore EXPO
Elevandi & MAS
Nov
Thu
Meetup

French Pavilion Networking at Singapore Fintech Festival 2026

Singapore Expo
French Chamber of Commerce in Singapore
Nov
to 21 Nov
Conference

Meet Taipei Startup Festival 2026

Investors in the room

Action Community for Entrepreneurship (ACE) Ltd
Nov
Thu
Meetup

Seoul Fintech Day

Invest Seoul & Seoul Fintech Lab
Nov
Thu 12:30
Meetup

SFF MeetUp

2 dates

MeetUp lounges in Hall 4B and Hall 5A
Elevandi & MAS
Nov
Thu 15:00
Workshop

RoboNexus Supply Chain Workshop

69 Ayer Rajah Cres.
Lorong AI
Free
Nov
Thu 18:00
Meetup

GenAI Builders Meetup Singapore

BeyondPilot Free
Nov
Thu 18:00
Meetup

Voice AI Night

Amazon Web Services Singapore Pte Ltd
Lionel Sim
Free
Nov
Thu 18:30
Meetup

How to Build With AI (and Stay Lean)

ACE Ideation Centre
The Runway Ventures
Free
Nov
Thu 18:30
Meetup

Omarchy Singapore

Omarchy Singapore Free
Apply on Luma Fri 20 Nov ↗