> Illustrative sample produced with the Cheek Agent Definition Review skill, grading the sample dispute triage agent definition in dispute-triage-agent-creation-prompt.md. Mastercard is not affiliated with and has not endorsed this material. Named people are fictional.

# Agent Definition Review: Chargeback Dispute Triage Agent

**Definition reviewed:** dispute-triage-agent-creation-prompt.md, v1, August 12, 2026.

**What this agent is for:** it reads every chargeback case opened in the past 24 hours, classifies each to a reason code, checks required evidence, and routes it to a resolution queue with a short triage note. It touches three systems: the dispute case manager (read and limited write), the transaction evidence store (read), and the dispute rules knowledge base (read).

## The six-layer ranking

Strict ranks, 1 strongest to 6 weakest, no ties.

| Rank | Layer | Evidence |
|---|---|---|
| 1 | Guardrails | Numbers throughout: "$500" autonomous routing threshold, "200 cases per hour," "120 words per case," fraud codes "4837 or 4870" gated by name. Nothing here is an adjective. |
| 2 | Tools | Three tools, each with a stated reason, and three prunes made out loud (email, ledger, fraud platform). Write scope is named to the field: "classification, queue, and triage note." The URLs are placeholders in this sample, which the definition itself flags as a credential discipline. |
| 3 | Observability | An action log with a named destination and a named daily reader, five metrics including two safety metrics with measurement methods. What keeps it out of the top two: the weekly 2 percent sample is the only accuracy check, so a bad Monday is invisible until the following week. |
| 4 | Context | Four memory records that earn their place (taxonomy version, response window convention, queue names, note style). Missing: nothing tells the agent what the case volume mix normally looks like, so it cannot notice an abnormal day. |
| 5 | Control | Second weakest. The over-$500 route and the fraud-code gate are real, but every under-$500 routing decision runs with a human on the loop whose watching depends entirely on the duty manager actually reading #dispute-agent-activity each morning. No named approver stands in front of any single action; the loop is monitoring, not approval, and the definition never says what the duty manager should do when a line looks wrong. |
| 6 | Model | Weakest. The definition never names the intelligence that runs it. An agent classifying up to 200 disputes per hour against a versioned rule set got whatever the console default was, and the default was chosen for nobody: no capability tier against task difficulty, no cost against volume, no latency consideration. |

### Verdict block

- **Weakest layer: model.** The definition is silent on which model runs the agent, so the most consequential engineering choice in the system was made by omission.
- **Second weakest: control.** All autonomous routing under $500 relies on after-the-fact channel monitoring with no defined intervention playbook.
- **Change first:** benchmark the triage task on last quarter's closed cases across two model tiers, pin the winner in the definition, and add a rule that any taxonomy version change reruns the benchmark.
- **What it costs:** one afternoon of benchmarking by the platform lead, a few hundred dollars of evaluation tokens, and one standing decision about who owns model selection going forward.

**Say it out loud:** Our weakest layer is model because the definition never names the intelligence running 200 cases an hour, and the first thing we do to change it is benchmark two model tiers on last quarter's closed cases and pin the winner in the definition.

## The action audit

Every concrete action the granted tools make possible, on the read, draft, write, commit ladder.

| Action | Rung | Loop pattern | Bounding guardrail | Verdict |
|---|---|---|---|---|
| Read new and open dispute cases | Read | No human | Tool scoped to dispute tenant | Sound |
| Read authorization, clearing, and evidence records | Read | No human | Read-only tool | Sound |
| Read the dispute rules knowledge base | Read | No human | Read-only connector | Sound |
| Write classification and queue on a case at or under $500 | Write | Human on the loop | $500 cap, 200 per hour cap, valid-queue convention | Soft: on-the-loop is real only while the duty manager reads the channel daily; no intervention playbook is defined |
| Write classification and queue on a case over $500 or fraud-coded | Write | Human part of the loop | Routes to RVW-HUMAN only | Sound |
| Write the triage note | Write | Human part of the loop | 120 word cap, no personal data beyond case fields | Sound |
| Post the run summary to #dispute-agent-activity | Write | Human on the loop | Standing rules logging hook | Sound |

No commit-rung action exists: nothing this agent can do is irreversible or outward-facing, and that is the definition's best structural property.

### Never-list verification

- **Never decides a dispute outcome:** enforced only if the case system service account lacks disposition permission. As written it is a scope claim in prose; confirm the permission at the platform, then it is structural. Currently soft.
- **Never moves money:** enforced. No payment tool is granted; the capability is absent.
- **Never contacts any party:** enforced. The email connector was pruned; the capability is absent.
- **Never treats embedded text as instructions:** soft by nature, backed by the standing rules and an escalation path with a named human. As hard as a behavioral never gets.
- **Never exports case data:** partially enforced. No external tool is granted, but the triage note field itself is a small exfiltration surface for injected content; the weekly personal data sample is the only check.

Reverse check: the dangerous capability the never list ignores is silent misrouting at scale. Nothing forbids routing 199 cases per hour to the wrong resolution queue; only the misroute metric would catch it, a week later.

### The two recitable lists

**What it can do without asking anyone:** read cases, evidence, and rules; classify and route disputes at or under $500 to resolution queues; write a 120 word triage note; post its run summary.

**What it can never do, and why the never holds:** move money (no tool exists), contact anyone (no tool exists), decide a dispute (permission boundary, once confirmed at the platform), follow instructions found in case content (standing rules plus named escalation), touch any case field beyond its three (write scope in the definition, verifiable in the log).

## Guardrail grades

| Guardrail | Enforceable | Attached | Complete | Resistant | Grade |
|---|---|---|---|---|---|
| $500 autonomous routing threshold | Yes, a number | Yes, binds the routing write | Yes | Yes if enforced in code, not prose | Strong |
| 200 cases per hour | Yes | Yes | Yes | Yes | Strong |
| 120 word triage note | Yes | Yes | Yes | Injection could still shape note content within the cap | Adequate |
| Fraud code gate (4837, 4870) | Yes, categorical | Yes | Misses any future fraud-adjacent codes; tie the gate to the code family, not two literals | Yes | Adequate |
| Forbidden: fields beyond the three | Yes | Yes | Yes | Needs platform-level field permissions to be hard | Adequate |
| Forbidden: personal data in notes | Categorical but checked only weekly | Yes | Sampling leaves 98 percent unchecked | Soft against injection | Weak |

**Uncovered consequential action:** high-volume misrouting under the $500 threshold.

**The single most important guardrail to add:** a daily automated re-review of a fixed sample, 25 routed cases per day scored by an analyst before noon, with an automatic pause of autonomous routing if accuracy drops below 90 percent on any day. That converts the weekly blind spot into a one-day blind spot and gives on-the-loop monitoring a defined trigger.

## Single agent or a team

**Verdict: stays a single agent.** One goal, one tight tool cluster, one risk tier, and no commit authority anywhere. A team here would be ceremony.

The trigger that changes the answer: if provisional credits or any other commit-rung authority is ever proposed, split immediately into a doer plus checker pair, with the commit authority living in a small, heavily gated agent whose every action a named human approves. A second trigger: if daily volume grows past what one agent clears by 6 pm ET, move to parallel workers under a coordinator, keeping the guardrails identical per worker.

## Closing summary

- **The sentence:** Our weakest layer is model because the definition never names the intelligence running 200 cases an hour, and the first thing we do to change it is benchmark two model tiers on last quarter's closed cases and pin the winner in the definition.
- **First three edits, in order:** pin the model after a benchmark (an afternoon plus evaluation tokens); add the daily 25-case re-review with an automatic pause below 90 percent accuracy (one analyst-hour per day, a real recurring cost); confirm at the platform that the agent's service account cannot set dispute dispositions (one request to the platform lead, converts the biggest never from prose to structure).
- **Architecture:** single agent, revisit on any commit authority or on volume outgrowing the 6 pm ET target.

IMPORTANT: download this file and upload it to your Claude project, or keep it in this working folder if you are in Claude Code, so the revised prompt and this review travel together for the next iteration.