Lesson 5 of 6 · 0%Evaluate output and enforce review gatesNext
Course map

Multi-Agent Workflows, Goals and Production Operations

0 of 6 complete0 of 6

Lesson 5.1 · 45 minutes

Evaluate output and enforce review gates

Separate generation from verification and require evidence-based acceptance before a workflow can affect an audience or system.

Skip course map
Current lessonEvaluate output and enforce review gates0% complete · 0/6 lessons

Multi-Agent Workflows, Goals and Production Operations

0% complete · Current: Evaluate output and enforce review gates

Verifiedon 2026.7.1-2

Action boundary

Before you act

Expected result
A reviewer can accept or reject a draft using artifact-linked criteria and can prove that publication remained human-gated.
Failure mode
The same role generates and grades a claim, quality is measured by confidence or style alone, and publishing is implicitly coupled to a successful draft.
Rollback
Mark the draft held, retain the review findings, withdraw any queued delivery, and reopen only the failed dependency or criterion.

Quality is evidence, not enthusiasm

A generated summary can be fluent and still be wrong. For the capstone, the reviewer checks each claim against a primary URL, version or date, source excerpt location, and plain-language uncertainty label. The editor cannot turn a rejected claim into an accepted one by rewriting it.

Use two gates:

  1. Technical acceptance: the verifier confirms that every factual claim has a primary receipt and the output meets scope, format, and freshness rules.
  2. Consequential release: the publication owner independently decides whether the accepted draft may be sent or published. A green technical review never auto-publishes.

Lab: make an acceptance checklist

Write binary checks before drafting:

  • Every claim links to an approved primary source and retrieval date.
  • The source version/revision is included when the claim is version-sensitive.
  • Unsupported recommendations are labelled as recommendations, not facts.
  • The draft is within the stated audience and word limit.
  • Private data, credentials, and internal URLs are absent.
  • A named reviewer accepted the technical pack.
  • A named publication owner made a separate release decision.

Check: Reject a draft containing one compelling but unreferenced claim. Record the finding and return only that claim to research; do not discard verified work.

Checkpoint: separate the gates

A cron job finishes successfully and delivers a polished bulletin to its output location. Is it ready to publish? No. Job completion means the run reached a terminal status. It does not prove factual correctness, audience safety, or approval to create an external effect.

Learner artifact

Sketch an annotated review screen with three accepted claims, one rejected claim, source links, a technical-review signature field, and a distinct human publication button in a disabled state until review is complete.

Source provenanceVerification and sources

Review receipt rr_multi_agent_evaluation_review_gates

Outcome
approved
Method
source-review
Reviewer
scout-independent-review
Reviewed

Evidence

Limitations

  • Review gates are original course controls, not automatic correctness proof.

Open the public evidence snapshot

Lesson checkpoint

Ready to move on?

Mark this lesson complete when you can apply its outcome without relying on the examples above.