Back to Module 1.11: LLM EvaluationComing Soon
AI Lesson & Submodule
Regression Testing in CI/CD
Integrate evaluation checks into automated deployment pipelines.
Why This Matters
Automated evaluations running on git commits block prompt updates that degrade generation quality.
What You Will Learn
- •Write CLI eval scripts
- •Integrate evals with GitHub Actions
- •Block regressive commits
Concepts Covered
Vitest eval runnersGitHub Actions test runsCommit checker blocks
Mapped Foundation Project: LLM Evaluation Lab
Regression evaluation dashboard testing prompt templates against golden datasets, checking faithfulness and schema compliance.
Architecture Preview
CI/CD testing pipeline running prompt variants against tests, compiling scores, and flagging failures.
Prompt Template CommitGolden Dataset CollectionModel Completion Loop
Tech Stack Planned
TypeScriptVitestReact
GitHub: Coming SoonLive Demo: Coming Soon
Coming SoonTechnical Interview Value
- ?How do you set up an evaluation step in a GitHub Actions pipeline that blocks merging if accuracy drops?