Prompt Details
Model
(claude-5-5-sonnet)
Token size
1,434
Example input
[PROTOCOL_AND_BEHAVIOR]: C-1 local protocol 1.0: Every task message has message_id, task_id and ownership_epoch. Authenticated actor and tenant must match the authorized task. A stable operation ID plus unchanged request fingerprint identifies duplicate work. Changed arguments on the same operation key must be rejected. Cancellation is a request until committed terminally. Results from stale ownership epochs cannot overwrite current state. Legitimate authorized tasks must complete.
[TEST_EVIDENCE_AND_CONSTRAINTS]: T-1: No tests have been executed. Design isolated fixtures for a simulated ticket-creation effect only. Real production access is prohibited. An external stub ledger must expose effect counts. No timing SLO or coverage target is supplied. Do not mark a designed test PASS.
Example output
## 1. Evaluation boundary
**Scope:** Contract "C-1 local protocol 1.0". The implementation under test, its version and its transport are all UNKNOWN. Environment is limited to an isolated harness with a simulated ticket-creation stub; production access is prohibited (T-1). Nothing has been executed (T-1).
**Normative requirements.** C-1 is decomposed below for traceability. Each part is a restatement of C-1, not a new rule.
| ID | Requirement (from C-1) |
|---|---|
| C-1.1 | Every task message has message_id, task_id and ownership_epoch. |
| C-1.2 | Authenticated actor and tenant must match the authorized task. |
| C-1.3 | A stable operation ID plus an unchanged request fingerprint identifies duplicate work. |
| C-1.4 | Changed arguments on the same operation key must be rejected. |
| C-1.5 | Cancellation is a request until committed terminally. |
| C-1.6 | Results from stale ownership epochs cannot overwrite current state. |
| C-1.7 | Legitimate authorized tasks must complete. |
**Proposed controls (not governing until adopted):**
- P-1: a duplicate (same operation + fingerprint) causes no additional effect. C-1.3 says only that duplicates are "identified".
- P-2: a valid terminal commit is not overwritten by a later cancel.
- P-3: after an interruption between effect and local record, reconciliation prevents a blind second effect.
- P-4: a rejected message causes no effect and no state change.
**Decision-critical gaps (UNKNOWN):**
- Operation-ID field name; C-1 does not list it as a required field. Fixtures assume `operation_id`.
- Fingerprint canonicalization.
- Rejection form and error semantics.
- Response to a duplicate.
- Terminal state set.
- Handling of an epoch higher than current.
- Whether a cancel request received before commit may win.
- Crash-recovery behavior (not in C-1).
- No timing SLO or coverage target.
**Evidence quality:** specification only. There is no execution evidence and no claim that anything passed.
## 2. Test architecture
**Assertion layers**, kept separate:
1. Schema validity.
2. Authenticated identity, taken from the harness-supplied trusted_context rather than the message body.
3. Authorization against the task record.
4. State transitions.
5. Business effect, read from the stub ledger.
**Harness:**
- A stub ticket service with an independent effect ledger exposing counts and payloads. The agent's self-report is never an oracle.
- A hold/release hook before commit, to force race orders.
- A fault hook to crash after the effect but before the local record/ACK.
- A state-store read interface.
**Positive control:** three degenerate implementations (always-reject, always-abstain, never-execute) must each fail T-01. Otherwise the suite cannot detect a "safe but useless" agent.
**Enforcement boundary under test:** the agent's message-handling entry point and its task-state store. Effect counts are checked at the stub.
**Unresolved oracles (BLOCKED BY SPEC):**
- Final terminal state in the cancel-before-commit race (T-06A).
- Recovery end state (T-08).
- Exact rejection form (all rejection tests assert rejection plus no effect only).
**Evidence artifacts per test:** event log with trusted_context, state-store snapshots before and after, ledger dump, harness version, implementation version.
## 3. Core fixtures
One merged scenario per risk. Duplicate delivery includes a new attempt (new message_id, same operation) because a new attempt is not a new business operation. T-06 is split into two event orders (A/B). All expectations are design expectations only.
```json
[
{
"test_id": "T-01",
"requirement_ids": ["C-1.1", "C-1.2", "C-1.7"],
"preconditions": ["Isolated ticket stub with ledger", "Positive-control implementations ALWAYS_REJECT, ALWAYS_ABSTAIN, NEVER_EXECUTE available as swappable variants"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"message_id": "M-001", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-1", "args": {"title": "Ticket A"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"task_state": "COMPLETED", "ledger_count": 1, "ledger_payloads": [{"title": "Ticket A"}], "control_expectation": "Each positive-control variant yields task_state != COMPLETED or ledger_count == 0, so T-01 fails for it"},
"evidence_required": ["event log", "state snapshot", "stub ledger dump", "control-variant run logs"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-02",
"requirement_ids": ["C-1.1", "P-4"],
"preconditions": ["Same as T-01", "Each variant starts from identical initial state"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-2", "args": {"title": "Ticket B"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-003", "ownership_epoch": 1, "operation_id": "OP-2", "args": {"title": "Ticket B"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-004", "task_id": "TASK-A", "operation_id": "OP-2", "args": {"title": "Ticket B"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"each_event": "rejected; rejection form UNKNOWN (BLOCKED BY SPEC)", "task_state": "OPEN", "ledger_count": 0},
"evidence_required": ["event log", "state snapshot after each event", "stub ledger dump"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-03",
"requirement_ids": ["C-1.2", "P-4"],
"preconditions": ["Same as T-01", "Message bodies are schema-valid; only trusted_context differs"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"message_id": "M-010", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-3", "args": {"title": "Ticket C"}, "claimed_actor": "ACTOR-A"}, "trusted_context": {"authenticated_actor": "ACTOR-X", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-011", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-3", "args": {"title": "Ticket C"}, "claimed_actor": "ACTOR-A"}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-X"}}
],
"expected": {"each_event": "rejected; form UNKNOWN", "task_state": "OPEN", "ledger_count": 0, "note": "Body-claimed actor must not override authenticated identity"},
"evidence_required": ["event log with trusted_context", "state snapshot", "stub ledger dump"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-04",
"requirement_ids": ["C-1.3", "P-1"],
"preconditions": ["Same as T-01"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"message_id": "M-020", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-4", "args": {"title": "Ticket D"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-020", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-4", "args": {"title": "Ticket D"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-021", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-4", "args": {"title": "Ticket D"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"task_state": "COMPLETED", "ledger_count": 1, "duplicate_response_form": "UNKNOWN (BLOCKED BY SPEC)"},
"evidence_required": ["event log", "state snapshot", "stub ledger dump showing count"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-05",
"requirement_ids": ["C-1.4", "P-4"],
"preconditions": ["Same as T-01"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"message_id": "M-030", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-5", "args": {"title": "Ticket E"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "deliver", "message": {"message_id": "M-031", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-5", "args": {"title": "Ticket E-ALTERED"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"second_event": "rejected; form UNKNOWN", "task_state": "COMPLETED", "ledger_count": 1, "ledger_payloads": [{"title": "Ticket E"}]},
"evidence_required": ["event log", "state snapshot", "stub ledger dump with payloads"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-06A",
"requirement_ids": ["C-1.5"],
"preconditions": ["Hold hook before commit enabled", "Same as T-01"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver_and_hold_before_commit", "message": {"message_id": "M-040", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-6", "args": {"title": "Ticket F"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "request_cancel", "message": {"message_id": "M-041", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-6-CANCEL"}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "release_hold", "message": {}, "trusted_context": {}}
],
"expected": {"after_cancel_request": "cancel recorded; task not in a terminal state", "final": "exactly one terminal state; ledger_count consistent with it", "blocked_oracles": ["Which terminal state wins (COMPLETED vs CANCELLED) is UNKNOWN (BLOCKED BY SPEC)"]},
"evidence_required": ["state snapshot between events", "stub ledger dump", "event log"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-06B",
"requirement_ids": ["C-1.5", "P-2"],
"preconditions": ["Same as T-01"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver", "message": {"message_id": "M-050", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-7", "args": {"title": "Ticket G"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "request_cancel", "message": {"message_id": "M-051", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-7-CANCEL"}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"task_state": "COMPLETED", "ledger_count": 1, "note": "Late cancel does not overwrite terminal commit (proposed P-2)"},
"evidence_required": ["state snapshot after each event", "stub ledger dump"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-07",
"requirement_ids": ["C-1.6", "C-1.7"],
"preconditions": ["Same as T-01", "Ownership already transferred: current_epoch 2"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 2, "state": "IN_PROGRESS"}}, "ledger_count": 0},
"events": [
{"action": "submit_result", "message": {"message_id": "M-060", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-8", "result": {"title": "STALE-RESULT"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "submit_result", "message": {"message_id": "M-061", "task_id": "TASK-A", "ownership_epoch": 2, "operation_id": "OP-8", "result": {"title": "CURRENT-RESULT"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "submit_result", "message": {"message_id": "M-062", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-8", "result": {"title": "STALE-RESULT"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}}
],
"expected": {"event_1": "rejected; state remains IN_PROGRESS", "event_2": "accepted; task COMPLETED with CURRENT-RESULT", "event_3": "rejected; state and result unchanged", "ledger_payloads_never_contain": ["STALE-RESULT"], "blocked_oracles": ["Epoch greater than current is UNKNOWN and untested"]},
"evidence_required": ["state snapshots after each event", "stub ledger dump", "event log"],
"status": "DESIGNED — NOT EXECUTED"
},
{
"test_id": "T-08",
"requirement_ids": ["P-3", "C-1.3"],
"preconditions": ["Fault hook crash_after_effect_before_record enabled", "Stub ledger survives agent restart"],
"initial_state": {"tasks": {"TASK-A": {"authorized_actor": "ACTOR-A", "tenant": "TENANT-A", "current_epoch": 1, "state": "OPEN"}}, "ledger_count": 0},
"events": [
{"action": "deliver_with_crash_after_effect", "message": {"message_id": "M-070", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-9", "args": {"title": "Ticket H"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "restart_agent", "message": {}, "trusted_context": {}},
{"action": "deliver", "message": {"message_id": "M-070", "task_id": "TASK-A", "ownership_epoch": 1, "operation_id": "OP-9", "args": {"title": "Ticket H"}}, "trusted_context": {"authenticated_actor": "ACTOR-A", "authenticated_tenant": "TENANT-A"}},
{"action": "reconcile", "message": {}, "trusted_context": {}}
],
"expected": {"observed_only": "Record ledger count and task state after each event", "proposed_invariant": "ledger_count <= 1 after redelivery and reconcile (P-3, not governing)", "blocked_oracles": ["Final state, recovery procedure and exactly-once expectation are UNKNOWN (BLOCKED BY SPEC)"]},
"evidence_required": ["ledger dump after crash, after redelivery, after reconcile", "state snapshots", "fault-injection log"],
"status": "BLOCKED BY SPEC"
}
]
```
**Requirement-to-test map**
| Requirement | Tests |
|---|---|
| C-1.1 | T-01, T-02 |
| C-1.2 | T-01, T-03 |
| C-1.3 / P-1 | T-04, T-08 |
| C-1.4 | T-05 |
| C-1.5 | T-06A, T-06B |
| C-1.6 | T-07 |
| C-1.7 | T-01, T-07 |
| P-2 | T-06B |
| P-3 | T-08 |
| P-4 | T-02, T-03, T-05 |
REQUIREMENT HAS TEST applies to all of the above. REQUIREMENT PASSED TEST applies to none.
**Uncovered:**
- Concurrent in-flight same-operation delivery.
- Same operation key from a different task or tenant.
- Unauthorized cancel.
- Higher-than-current epoch.
- Fingerprint canonicalization edge cases.
## 4. Results and repair register
- **Supplied results:** none. Every test is NOT RUN (DESIGNED — NOT EXECUTED). No failures exist, and absence of results is not failed behavior.
- **Blocked:** the T-08 oracle; the T-06A final-state oracle; all rejection-form assertions.
- **Invalid transfers:** none supplied. Any future dashboard or run against a different implementation version, protocol version or fixture cannot establish pass or failure of this scope.
- **Repair boundaries:** none attributable yet. After a failing execution, the failed assertion would point to a candidate boundary (identity check, idempotency store, epoch guard or state-transition guard), not a proven root cause.
- **Next discriminating test:** T-05 (it separates fingerprint comparison from operation-ID-only deduplication). Resolve the T-06A/T-08 spec gaps before building their oracles.
## 5. Protocol Conformance Test Pack
**Minimum regression core:**
- T-01 with the three positive controls.
- T-03.
- T-04.
- T-05.
- T-07.
T-02, T-06A/B and T-08 follow as spec gaps close.
**Required evidence:**
- Implementation and protocol version.
- Fixture hash.
- Event logs showing trusted_context.
- State snapshots.
- Stub ledger dumps.
**Retest triggers:** changes to the contract, model, policy, adapter, tool or storage reopen the applicable tests. Changes to identity handling, the idempotency store or epoch logic reopen T-03, T-04, T-05 and T-07 at minimum.
**Boundaries:** this review is a design for conformance testing. It does not certify security or authorize release.
**Disposition: TEST PLAN ONLY**
By purchasing this prompt, you agree to our terms of service
CLAUDE-5-5-SONNET
Build a traceable communication test suite from an agent contract. Get ordered failure scenarios, machine-readable fixtures, positive controls, explicit state and side-effect oracles, and a repair/retest handoff. Distinguish schema checks from authority, ownership and recovery checks. Missing execution evidence stays not run—not an invented pass.
...more
Added 3 hours ago
