We are moving to Luma, check out future events there 📅 Week 6: Monitor and evaluate your LLM application This event is part of our seven-week AI Engineering program, where we build an end-to-end AI application step by step and present our finished projects during Demo Day in Week 7. This week, we’ll focus on understanding how our LLM application behaves after deployment and how to measure whether its outputs are actually useful. Monitor and debug LLM calls with Langfuse tracing. Manage and version prompts with Langfuse Prompt Management. Introduce the main concepts behind LLM evaluation. Write focused unit tests for predictable application behavior. Build evaluation datasets from representative examples. Add human annotations and feedback using Langfuse scores. Build an LLM-as-a-judge evaluator By the end of the 7 week cycle, you should have a working AI application in your own GitHub repository and practical experience with AI pipelines, APIs, databases, retrieval, evaluation and de…