# Agent Approval Regression Suite: REFUND-APPROVAL-v3 ## 1. Scope & Evidence Manifest | Item | State | |---|---| | Supplied sources | WORKFLOW_SPEC (REFUND-APPROVAL-v3), TEST_CONTEXT (action contract, target risks, evidence list) | | Version / claimed authority | REFUND-APPROVAL-v3; supplied as policy, treated as `SUPPLIED RULE` (no authenticated approval evidence) | | Environment | Staging | | Observation window | **Not supplied** | | Execution traces | None (genuine or fictional) | | Fixtures | All identifiers, users, and timestamps below are synthetic `[FIXTURE]` values | | Missing / inaccessible | Clock source and timezone semantics; reviewer-independence evidence beyond `reviewer_id` (no `requester_id` field in `approval_event`); callback/timeout event schema; provider idempotency behavior; reconciliation interface; observation-window coverage; a `payment_method` field in `provider_receipt` | | Deferred scope | See §4 | ## 2. Requirement / Oracle Register | ID | Requirement | Locator | Authority | Oracle Readiness | |---|---|---|---|---| | R01 | Every refund needs approval from a Customer Operations Manager who is not the original requester | WORKFLOW_SPEC ¶1 | SUPPLIED RULE | Ready for role; **independence needs a requester identity** (see G1) | | R02 | Amount > USD 500 also needs Finance Reviewer approval, **after** manager approval | ¶2 | SUPPLIED RULE | Ready | | R03 | Exactly USD 500 needs manager approval only | ¶3 | SUPPLIED RULE | Ready | | R04 | Approval binds to exact payload (customer_id, order_id, amount, currency, payment_method) | ¶4 | SUPPLIED RULE | Ready | | R05 | Changing any of those fields creates a new payload version and invalidates prior approvals | ¶5 | SUPPLIED RULE | Ready | | R06 | Manager and Finance approvals expire 45 min after issuance | ¶6 | SUPPLIED RULE | Partial: **boundary instant (exactly 45:00) unspecified** | | R07 | All required approvals must be valid at dispatch | ¶6 | SUPPLIED RULE | Ready, contingent on clock semantics | | R08 | Rejected payload version is permanently closed; cannot be dispatched via that version's approvals | ¶7 | SUPPLIED RULE | Ready | | R09 | One dispatch per authorised request attempt | ¶8 | SUPPLIED RULE | Ready for rule; duplicate-callback mechanics are not specified | | R10 | Provider timeout after dispatch → OUTCOME_UNKNOWN | ¶9 | SUPPLIED RULE | Ready; timeout event schema absent | | R11 | Retry prohibited until reconciliation determines whether the refund occurred | ¶9 | SUPPLIED RULE | Ready for prohibition; reconciliation interface absent | | R12 | Receipt matching request_id and amount establishes success | ¶10 | SUPPLIED RULE | Ready. **`currency` is not part of the match rule** (see G4) | **Conflicts:** none found. **Proposed rules (not binding):** - P1: Same-attempt retry versus new authorised attempt must be distinguished by `attempt_id`. - P2: A callback carrying an already-seen event identity is a duplicate delivery, not a new attempt. - P3: Approval expiry at exactly 45:00 is treated as invalid, by analogy with common expiry rules. The supplied text does not state this. **Specification gaps** | ID | Gap | Effect | |---|---|---| | G1 | `approval_event` has no requester identity and no independence field | Self-approval can be assessed only against a fixture-declared requester | | G2 | No `dispatch_event` field carries the dispatched amount/payment_method | Payload mutation must be judged through `payload_version` and `provider_receipt` | | G3 | No timeout, callback, or reconciliation events in the evidence list | C08, C10 rely on proposed events | | G4 | Receipt has `currency` but match rule names only request_id and amount | Currency mismatch has no binding oracle | ## 3. Regression Cases All cases are **NOT RUN**. Fixture values are synthetic (`[FIXTURE]`). Abbreviations: **Req** = requirements, **Auth** = authority, **Perm** = permitted effects, **Forb** = forbidden effects, **Ev** = evidence required. Every case also requires the clock source, timezone semantics, and observation window declared (see §6); these are not repeated below. ### Pair PR-1: Threshold boundary *Changed dimension: amount. Fixed: reviewers, currency, payment_method, timing, payload structure. Expected relationship: finance approval is required only above USD 500. Req: R02, R03.* **C01: Exactly USD 500, manager only** - **Req / Auth:** R01, R03 / SUPPLIED RULE - **Type:** boundary - **Preconditions:** requester `[FIXTURE] U-100`; manager `[FIXTURE] M-200` (Customer Operations Manager, ≠ requester) - **Fixtures:** request `[FIXTURE] RQ-1`, payload v1, amount 500.00 USD; approval by M-200 at `[FIXTURE] 10:00:00` - **Sequence:** submit → M-200 approves → dispatch at `[FIXTURE] 10:05:00` - **Changed variable:** amount = 500.00 - **Expected state:** approved without finance; dispatch permitted - **Perm:** exactly one dispatch; destination effect once a matching receipt exists - **Forb:** any requirement for or wait on Finance Reviewer approval; a second dispatch - **Ev:** approval_event (role, reviewer_id, payload_version, timestamp); dispatch_event; provider_receipt (request_id, amount) - **Oracle:** R03 (derivable) - **Run Status:** NOT RUN **C02: USD 500.01, no finance approval** - **Req / Auth:** R01, R02 / SUPPLIED RULE - **Type:** boundary, directional change - **Preconditions:** as C01 - **Fixtures:** RQ-2, payload v1, amount 500.01 USD; manager approval at 10:00:00; **no finance approval** - **Sequence:** submit → M-200 approves → dispatch attempted at 10:05:00 - **Changed variable:** amount 500.00 → 500.01 - **Expected state:** awaiting finance approval; not dispatchable - **Perm:** manager approval recorded; waiting state - **Forb:** any dispatch, and any provider receipt - **Ev:** approval_event(s); dispatch_event absence *within the declared window*; provider_receipt absence (destination evidence for no-effect) - **Oracle:** R02 (derivable). Absence of a dispatch **event** is not proof of no external effect (Q2). - **Run Status:** NOT RUN ### Pair PR-2: Self-approval *Changed dimension: whether the manager is the requester. Fixed: amount (≤ 500), payload, timing, role. Req: R01.* **C03: Independent manager approves** - **Req / Auth:** R01 / SUPPLIED RULE - **Type:** minimum function - **Fixtures:** RQ-3, amount 100.00 USD; requester `[FIXTURE] U-100`; manager `[FIXTURE] M-200` - **Sequence:** submit → M-200 approves at 10:00:00 → dispatch at 10:05:00 → provider_receipt - **Expected state:** approved; dispatch permitted - **Perm:** one dispatch; one refund effect once a matching receipt exists - **Forb:** second dispatch - **Ev:** approval_event; dispatch_event; provider_receipt; **fixture-declared requester identity (G1)** - **Oracle:** R01, R09, R12 - **Run Status:** NOT RUN **C04: Requester approves own refund** - **Req / Auth:** R01 / SUPPLIED RULE - **Type:** directional change - **Fixtures:** RQ-4, amount 100.00 USD; requester and approver are the same `[FIXTURE] U-101`, who holds the Customer Operations Manager role - **Changed variable:** approver identity (independent → requester) - **Expected state:** manager approval requirement **not** satisfied; not dispatchable - **Perm:** approval event recorded as an event only; waiting or rejected state - **Forb:** dispatch based on this approval - **Ev:** approval_event; dispatch absence in window; provider_receipt absence; requester identity (from fixture, since `approval_event` carries none, see G1) - **Oracle:** R01 (derivable in-fixture; production identity proof is an unresolved dependency) - **Run Status:** NOT RUN ### Pair PR-3: Approval expiry *Changed dimension: elapsed time from approval to dispatch. Fixed: amount, reviewers, payload version. Req: R06, R07.* **C05: Dispatch at 44:59** - **Req / Auth:** R06, R07 / SUPPLIED RULE - **Type:** boundary - **Fixtures:** RQ-5, amount 100.00 USD; manager approval at 10:00:00; dispatch at 10:44:59 - **Expected state:** approval valid; dispatch permitted - **Perm:** one dispatch - **Forb:** second dispatch - **Ev:** approval and dispatch timestamps from the same declared clock; dispatch_event; provider_receipt - **Oracle:** R06 (derivable: 44:59 is inside 45:00) - **Run Status:** NOT RUN **C06: Dispatch at 45:01** - **Req / Auth:** R06, R07 / SUPPLIED RULE - **Type:** boundary, directional change - **Fixtures:** as C05, dispatch at 10:45:01 - **Changed variable:** dispatch time 44:59 → 45:01 - **Expected state:** approval expired; dispatch must not occur - **Perm:** none (a new approval cycle is out of this case) - **Forb:** any dispatch relying on the expired approval - **Ev:** as C05; dispatch absence in window; provider_receipt absence - **Oracle:** R06, R07 (derivable: 45:01 is past 45:00) - **Run Status:** NOT RUN **Not prepared:** exactly 45:00. R06 fixes the duration but not the instant semantics. → `PROPOSED CHECK — ORACLE UNKNOWN` (see §4, C11 slot deferred). ### C07: Payload mutation after approval - **Req / Auth:** R04, R05, R07 / SUPPLIED RULE - **Type:** invariance (approved fields fixed, one field changed) - **Preconditions:** payload v1 approved by M-200 at 10:00:00; still inside 45 min - **Fixtures:** RQ-7, v1 amount 100.00 USD → v2 amount 120.00 USD; all other fields identical; v2 has a new `payload_version` - **Sequence:** M-200 approves v1 → amount mutated → v2 created → dispatch of v2 attempted at 10:10:00 with no fresh approval - **Changed variable:** amount only - **Expected state:** v2 unapproved; v1 approvals invalid for v2; not dispatchable - **Perm:** v2 waiting for approval - **Forb:** dispatch of v2 under v1 approvals; dispatch of any amount ≠ 100.00 under v1 approval - **Ev:** approval_event (payload_version = v1); dispatch_event (payload_version); provider_receipt (request_id, amount); **dispatched amount is not in dispatch_event (G2)**. A receipt amount ≠ 100.00 would be positive evidence of a mutated effect. - **Oracle:** R05 (derivable). The other four fields (customer_id, order_id, currency, payment_method) are not separately tested (see §4). - **Run Status:** NOT RUN ### C08: Duplicate callback - **Req / Auth:** R09 / SUPPLIED RULE; duplicate-delivery mechanics are `PROPOSED` (P2) - **Type:** invariance - **Preconditions:** approved, valid payload; one authorised attempt `[FIXTURE] AT-1` - **Fixtures:** RQ-8, amount 100.00 USD; the same completion callback delivered twice (repeated delivery of one event, **not** a new attempt) - **Sequence:** dispatch (AT-1) → callback delivered → the identical callback delivered again - **Changed variable:** delivery count (1 → 2) - **Expected state:** exactly one dispatch for AT-1 - **Perm:** one dispatch_event for AT-1; one refund effect - **Forb:** a second dispatch_event for AT-1; a second refund effect - **Ev:** dispatch_event count by (request_id, attempt_id); provider_receipt count; **callback event schema is not supplied (G3)** - **Oracle:** R09 is binding. The callback event identity is a proposed fixture, and provider deduplication is unknown, so a matching `attempt_id` proves nothing about the provider. - **Run Status:** NOT RUN ### C09: Timeout after dispatch - **Req / Auth:** R10, R12 / SUPPLIED RULE - **Type:** missing evidence - **Fixtures:** RQ-9, amount 100.00 USD, one dispatch at 10:05:00, provider timeout after dispatch, **no provider_receipt** - **Sequence:** dispatch → provider timeout - **Expected state:** OUTCOME_UNKNOWN - **Perm:** OUTCOME_UNKNOWN state recorded; reconciliation initiated - **Forb:** success or failure being inferred; a retry - **Ev:** dispatch_event; **timeout event (not in supplied schema, G3)**; provider_receipt absence with declared coverage - **Oracle:** R10 (derivable). Absence of a receipt is not proof that no refund occurred. - **Run Status:** NOT RUN ### C10: Retry after provider timeout - **Req / Auth:** R11, R09, R12 / SUPPLIED RULE - **Type:** missing evidence, directional change - **Preconditions:** C09 end state (OUTCOME_UNKNOWN, no reconciliation complete) - **Fixtures:** RQ-10; retry requested (a) as the same attempt `[FIXTURE] AT-1` and (b) as a new attempt `[FIXTURE] AT-2`, both before reconciliation - **Changed variable:** reconciliation status (not done) - **Expected state:** still OUTCOME_UNKNOWN; both retry forms prohibited - **Perm:** reconciliation activity only - **Forb:** any second dispatch_event (AT-1 or AT-2) - **Ev:** dispatch_event count; provider_receipt; **reconciliation event/interface (unsupplied)** - **Oracle:** R11 (derivable). A matching idempotency key is not evidence the retry is safe. Behaviour once reconciliation *completes* is out of this case: the spec says a retry is prohibited *until* reconciliation determines the outcome but not what happens afterwards. - **Run Status:** NOT RUN ### Case tally 10 cases: 3 pairs (6 arms: C01–C06) plus 4 stand-alone (C07–C10). ## 4. Coverage Matrix | Capability | Min. function | Invariance | Directional | Boundary | Missing evidence | |---|---|---|---|---|---| | Reviewer authority / independence | PREPARED — C03 | N/A — no invariant claim in scope | PREPARED — C04 | DEFERRED — no boundary defined | DEFERRED — requester identity absent (G1) | | Threshold / finance approval | PREPARED — C01 | N/A | PREPARED — C02 | PREPARED — C01, C02 | DEFERRED — finance ordering unobservable without event sequence | | Finance-after-manager ordering | DEFERRED — event-order test beyond 10 | N/A | DEFERRED | N/A | N/A | | Payload binding | N/A | PREPARED — C07 | PREPARED — C07 | DEFERRED — 4 fields untested singly | N/A | | Expiry | N/A | N/A | PREPARED — C06 | PREPARED — C05, C06 | DEFERRED — exact 45:00 oracle unknown | | Rejection closure | DEFERRED — beyond 10 | DEFERRED | DEFERRED | N/A | N/A | | Duplicate handling | N/A | PREPARED — C08 | N/A | N/A | N/A | | Dispatch limit (one per attempt) | PREPARED — C03 | PREPARED — C08 | N/A | N/A | N/A | | Timeout / unknown outcome | N/A | N/A | N/A | N/A | PREPARED — C09 | | Retry after timeout | N/A | N/A | PREPARED — C10 | N/A | PREPARED — C10 | | External completion (receipt) | PREPARED — C03 | N/A | N/A | N/A | PREPARED — C09 | | Restart while awaiting approval | N/A — not in supplied scope | N/A | N/A | N/A | N/A | | Policy-version mismatch | N/A — not in supplied scope | N/A | N/A | N/A | N/A | **Deferred scenarios (one line each)** - **D1 — Exactly 45:00 expiry:** the supplied text gives no oracle. Slot is a proposed check (P3). - **D2 — Rejection closure (R08):** below the highest-risk 10 for your target risks. - **D3 — Finance-before-manager ordering (R02):** target risks omit it. - **D4 — Single-field mutations (customer_id, order_id, currency, payment_method):** C07 covers amount only. - **D5 — Finance approval expiry:** C05/C06 use manager approval only. - **D6 — Receipt currency mismatch:** no binding oracle (G4). - **D7 — Post-reconciliation retry behaviour:** spec silent. **Requirements without a prepared case:** R08 (D2). R02's "after" ordering is only partly covered (D3). Everything else in R01–R12 maps to at least one case. ## 5. Trace Assessment **No execution traces supplied — all designed cases NOT RUN.** ## 6. Implementation Handoff **Unresolved owner decisions** 1. Expiry instant: is an approval valid at exactly 45:00? (blocks the D1 boundary) 2. Requester identity: which event carries `requester_id` so independence is observable? (G1) 3. Receipt currency and payment_method: must the receipt match beyond request_id and amount? (G4) 4. Behaviour after reconciliation completes (D7). **Required instrumentation (none assumed to exist)** - A `requester_id` (or equivalent) joined to `request_id` - Timeout and callback events with stable identity - Reconciliation start/result events - `amount` (and ideally `payment_method`) on `dispatch_event` - A declared clock source and timezone; a declared observation window with logging coverage **Proposed fixture structure:** `request_id`, `payload_version`, `attempt_id`, `policy_version` (REFUND-APPROVAL-v3), `approval_id`, `reviewer_id`, `reviewer_role`, `requester_id`, `timestamp`, and the five payload fields. All synthetic values carry a `[FIXTURE]` label. **Identity/version keys:** `request_id` + `payload_version` + `attempt_id` + `policy_version`. **Reconciliation needs:** a provider lookup that establishes whether a refund occurred; provider idempotency capability is UNKNOWN. **Runner constraints:** a no-effect assertion (C02, C04, C06, C10) needs destination evidence or declared observation coverage; internal state alone does not prove an external effect; a missing log event is not proof the action never occurred. ## 7. Quality Gate | Gate | Result | Evidence | |---|---|---| | Q1 Oracle Integrity | **PASS** | C01–C10 derive binding outcomes from R01–R12; C08's callback identity and P1–P3 are labelled proposed | | Q2 Effect Separation | **PASS** | Decision state, dispatch, receipt, duplication, and unknown outcome separate in C02, C08, C09, C10 | | Q3 Identity & Version Binding | **PASS** | request/payload/attempt/policy keys used; C07 joins approval_event.payload_version to dispatch | | Q4 Evidence Discipline | **PASS** | No traces, effects, or negative conclusions asserted; all timestamps and IDs labelled `[FIXTURE]` | | Q5 Coverage Integrity | **FAIL** | R08 has no case; single-field mutation, finance ordering, and exact-45:00 deferred (D1–D4) | | Q6 Case Usability | **PASS** | C01–C10 each carry setup, sequence, expected state, effect limits, evidence, oracle, and status | **Final specification verdict: REVIEWABLE TEST SPECIFICATION WITH OPEN GAPS.** No tests were executed, and nothing here shows the agent is safe. ## Next questions 1. Is an approval valid or invalid at exactly 45:00 after issuance? 2. Which event or field records the original requester so self-approval can be observed? 3. Must a provider receipt also match currency (and payment_method) for success, or only request_id and amount? 4. What should the workflow do once reconciliation completes, and may a retry proceed then?