A customer is paid twice for one return. The support agent transcript shows a single
issue_refund call and a clean RESOLVED ending, so nothing in the run looks wrong.
The payment gateway statement shows two debits, three seconds apart, for the same order
and the same amount. Nobody notices for eleven days, until the finance reconciliation
for the week does not balance. By then the same pattern has repeated on nine more orders,
and the total is a five-figure rupee number that has already left the account.
Any agent or workflow that calls a payment, ticket, email or write API over a network. Fires hardest on desks where a retry was added for reliability — a gateway timeout, a 504, a dropped socket — and where the retry sits in the harness that also drives an agent loop that can decide to call the same tool again. The P4 reference desk plants the shape directly: 37 of its 173 returns are already marked refunded, and 10 of those arrive in the inbox again as fresh refund requests. Linked modules: P4 (Tool-using agent workflow), 3.9 (Agents), 4.9 (Scaling and cost control), 4.5 (Testing ML code).
There are two different double-refund bugs wearing the same costume, and telling them apart is the whole job.
The first is the network retry. The refund actually succeeded at the gateway, but the
response never came back inside the timeout, so the harness retried a call it believed had
failed. The tool call count in your own logs says one; the gateway says two. You can
confirm this in one query: compare the count of issue_refund rows in your audit log
against the count of debits in the gateway statement for the same window. If the gateway
number is higher, the duplicate was created below your code and no amount of agent
tuning will touch it.
The second is the agent repeating itself. A small model that gets an unclear tool result — an empty dictionary, an error string it does not understand, a success message with no confirmation id — will often call the same tool again with the same arguments. Here the audit log shows two rows, and the second carries a justification along the lines of "the earlier call did not confirm". Both bugs pay twice, and the second one is visible to you while the first is not, which is why teams fix the wrong one first.
The third possibility is the one to rule out before either: two different messages about the same order, processed minutes apart, each correctly deciding a refund is due, because neither run could see what the other did. This is not a retry bug at all. It is a missing check of current state, and it looks identical in the bank statement.
Immediate: stop the automated refund path. Do not "add a check" while it is running. Pull the audit log and the gateway statement for the full period the path has been live, join them on order id, and list every order with more than one debit. That list is a money question and it goes to whoever owns the accounts on day one, not after the code is fixed.
Structural, in this order. Build an idempotency key on the payment itself — a hash of
order id, return id and amount — sent with the call and stored before the call. A repeat
with the same key returns the first result, records replayed=true, and moves nothing.
This fixes the network retry, which is the bug you cannot see. Then add the repeated-call
detector in the harness: the same tool with the same arguments twice in one run returns
the cached result with a warning, and a third attempt ends the run and escalates. Then
make the refund tool re-read live state before it acts, so a return already marked
refunded is refused whatever the agent believes. On the P4 desk that single refusal blocks
10 of the 310 messages outright.
Last, the evidence layer. Write the audit row before the tool executes, never after, carrying run id, step, message id, tool, arguments, result and the model's justification. A row written after the call is missing exactly the case you are investigating: the call that started and did not return.
Treat every irreversible tool as if the network will lie to you, because sometimes it does. Design the idempotency key at the same moment you design the tool signature, not after the first incident — it is a five-line change on day one and a reconciliation project later. Write the behaviour test before the tool: process the same message twice and assert one payment. That test is worth more than any prompt.
Then make the duplicate visible without a human looking. A daily check on refunds per order, alarming above one, costs nothing and would have caught this on day one instead of day eleven. Keep the audit log queryable and keep it for as long as your finance team keeps its books, because a payment dispute is answered with a row and a timestamp, not with a transcript. Under India's DPDP Act the log is also personal data, so redact the customer's phone and address in it while keeping the order id that makes it useful.
Q: "Your agent refunded one customer twice. Walk me through it." Strong answer: separate the three causes in the first thirty seconds — a network retry above your code, the agent repeating a call inside one run, and two runs on the same order that could not see each other — and say which evidence tells them apart (audit rows against gateway debits). Give the fix in layers: an idempotency key on the payment, a repeated-call detector in the harness, a live state check inside the tool, and the audit row written before the call. Strongest answers add the operational half: the incident is a money question before it is an engineering one, so the duplicate list goes to finance while the path stays stopped, and a daily refunds-per-order check makes the next one visible in hours instead of weeks.