Multi-Pass Review
Independent Claude instances review output without generation bias, and splitting large reviews into per-file then cross-file passes overcomes attention dilution.
Asking Claude to review its own output in the same conversation produces shallow critique. The model still holds the reasoning it used to generate the output, so it tends to confirm its decisions rather than challenge them. A robust review architecture uses independent instances and multiple focused passes.
Independent instances beat self-review
When a model reviews within the same session, it knows *why* it chose each approach and classified each finding, so it defends those choices. An independent instance — a separate invocation given only the output to review, with no generation context — judges the work on what it sees alone. That fresh perspective is what makes the critique genuinely evaluative.
Asking the model to review its own output in the same conversation lets it keep its generation reasoning, so it confirms its choices instead of challenging them.
messages += [{"role": "user",
"content": "Now review your answer above."}]
# still holds why it decided each thingSpin up a fresh invocation given only the output — no generation context — so it judges the work on its merits alone.
review = client.messages.create(messages=[
{"role": "user",
"content": f"Review this output:\n{output}"}])Review in a separate Claude invocation that receives only the output — not the conversation that produced it. Same-session self-review confirms; an independent instance challenges.
Multi-pass architecture for large outputs
A single pass over a large output — a multi-file pull request or a complex extraction — suffers attention dilution: inconsistent depth across files, bugs missed in the middle, and the same pattern flagged differently in different places. The fix is a two-phase design.
- 1Pass 1 — per-file local analysis. Review each file in its own focused invocation for bugs, security issues, and logic errors. Isolating one file guarantees consistent attention depth.
- 2Pass 2 — cross-file integration. Feed all per-file findings to a separate invocation that checks data-flow inconsistencies, contradictory patterns across files, and API-contract violations at service boundaries.
A bigger context window does not fix attention dilution — it still spreads attention unevenly. Only focused per-file passes ensure consistent depth. Larger models and majority-vote reruns miss this too.
Confidence-based routing
Attach a confidence score (0.0-1.0) to each finding and route on it: high-confidence findings go straight to the developer, low-confidence findings go to a human-review queue. The catch is calibration — raw, uncalibrated self-reported confidence is unreliable. Validate your thresholds against a labelled set where the correct answers are known before trusting them for automated routing.
The complete production pipeline
| Stage | Role |
|---|---|
| Generation | Initial instance produces the output |
| Per-file review | Independent instances analyse each component |
| Integration review | Separate instance checks cross-component consistency |
| Confidence routing | Uncertain findings sent to human review |
| Calibration loop | Labelled sets continuously refine thresholds |
This architecture costs more than a single pass. Reserve it for cases where missed issues have real consequences — CI/CD gates, financial extraction, and compliance workflows.
How the exam will try to trick you
The distractors below look right under time pressure — learn the tell.
- The trap
Have Claude review its own output in the same conversation that produced it.
Correct answerUse an independent instance given only the output, with no generation context.
Why: In-session, the model retains its generation reasoning and confirms its choices rather than challenging them.
- The trap
Review a large multi-file output in a single pass.
Correct answerSplit into per-file local passes, then a separate cross-file integration pass.
Why: One pass suffers attention dilution — inconsistent depth, missed bugs, and contradictory findings.
- The trap
Fix inconsistent multi-file review by switching to a bigger context window / higher-tier model.
Correct answerUse focused per-file passes for consistent attention depth.
Why: A larger window holds more text but still spreads attention unevenly — it is an architecture problem, not a capacity one.
- The trap
Route findings using raw self-reported confidence scores.
Correct answerCalibrate thresholds against a labelled validation set before trusting them for routing.
Why: Uncalibrated self-reported confidence is unreliable.
Key takeaways
- Same-session self-review is biased; an independent instance reviews without generation reasoning.
- Give the review instance only the output, never the conversation that produced it.
- Single-pass review of large outputs suffers attention dilution — inconsistent depth and contradictions.
- Split large reviews into per-file passes, then a cross-file integration pass.
- A larger context window or bigger model does not solve attention dilution; architecture does.
- Calibrate confidence thresholds against labelled sets before using them to route findings.
Frequently asked questions
Why is an independent Claude instance better than same-session self-review?+
In the same session, the model retains the reasoning it used to generate the output and tends to confirm those decisions. An independent instance sees only the output with no generation context, so it evaluates the work on its own merits and challenges rather than defends it.
Does a larger context window fix inconsistent multi-file reviews?+
No. Attention dilution is a quality-of-attention problem, not a capacity problem — a bigger window still spreads attention unevenly across many files. The fix is architectural: focused per-file passes for consistent depth, then a separate cross-file integration pass.