Test Platform Agents Like Software: Utterance Sets and Accuracy Scores

Tier: Advanced · Workflow area: ERP Copilots & Agents · ~45 minutes

Platform agents fail differently from code: nothing throws, the demo works, and then real users phrase things sideways and the topic classifier routes a refund request into order status. The fix is the same discipline testing has always been — a fixed input set, expected outputs, a score, and a rule that the score gates release. Salesforce ships this as Agentforce Testing Center; the method transfers to every platform.

When to use

• Any agent about to face real usersthe pb-40 build was verified by five manual probes; production needs a suite that runs identically every time someone edits an instruction.
• After every topic, instruction or action changeagent behavior is emergent from configuration; a one-word instruction edit can flip classification on utterances nobody re-tried. Regressions here are silent.
• When two teams argue about whether the agent "got worse"a scored baseline turns the argument into a diff: topic accuracy 93% → 84% after Tuesday's change is a conversation with an ending.

When NOT to use

• As a substitute for security reviewutterance testing catches routing and grounding errors; it does not audit the permissions of the agent user or the blast radius of actions. That is pb-51's job.
• With a test set the agent's author alone invented30 utterances from one mind share one mind's phrasing. Harvest real user phrasings and let an LLM adversarially extend them, or the suite tests your imagination.
• Scoring free-text answers by exact matchresponse wording varies legitimately. Score the deterministic parts (topic chosen, action invoked, key facts present), not string equality.

The recipe

1. 1.

   Define what "correct" means per utterance. For each test case you need: the utterance, the expected topic, the expected action(s), and expectations about the response (facts it must contain, things it must not do). Topic and action selection are objectively scoreable; hold the response to fact-level checks.

2. 2.

   Build the 30-utterance set with deliberate structure. A useful starting distribution: 15 happy-path phrasings across your topics, 8 off-topic asks that must hand off or decline, 7 adversarial cases. Use an LLM to widen phrasing beyond your own habits:

   Copy

   You are building a test set for an order-status support agent.
   Topics: Order Status (in scope). Out of scope: refunds, pricing,
   cancellations, anything not about orders.
   
   Generate:
   - 15 in-scope utterances with varied register: terse ("status 4512?"),
     verbose, misspelled, and two written by an annoyed customer.
   - 8 out-of-scope utterances a real customer would plausibly send.
   - 7 adversarial utterances: prompt injection ("ignore previous
     instructions..."), scope-creep ("while you're at it, cancel it"),
     data-fishing ("list all orders for other customers"), and a
     jailbreak framed as a roleplay.
   For each: utterance, expected_topic, expected_action, must_not.
   Output as CSV.
   

   Review every generated case — you are adopting these as your spec.

3. 3.

   Encode the set as versioned test metadata. In Agentforce, tests are AiEvaluationDefinition metadata — meaning the suite deploys, diffs and reviews like any other code:

   Copy

   <AiEvaluationDefinition xmlns="http://soap.sforce.com/2006/04/metadata">
       <name>OrderAgent_Regression_v3</name>
       <subjectType>AGENT</subjectType>
       <subjectName>Order_Status_Agent</subjectName>
       <testCase>
           <number>1</number>
           <inputs><utterance>where is order 00000123?</utterance></inputs>
           <expectation><name>topic_sequence_match</name>
               <expectedValue>Order_Status</expectedValue></expectation>
           <expectation><name>action_sequence_match</name>
               <expectedValue>["Get_Order_Status"]</expectedValue></expectation>
           <expectation><name>bot_response_rating</name>
               <expectedValue>States the order status and placement date;
               does not mention refunds.</expectedValue></expectation>
       </testCase>
       <!-- ...29 more test cases... -->
   </AiEvaluationDefinition>
   

4. 4.

   Run the suite from the CLI and capture the score.

   Copy

   sf agent test run --api-name OrderAgent_Regression_v3 \
     --target-org agentdev --wait 10 --result-format json > results.json
   

   Typical output shape: 30 cases, topic_sequence_match 28/30, action_sequence_match 27/30, response ratings flagged on 3. That is your baseline: 93% topic accuracy, 90% action accuracy, three transcripts to read.

5. 5.

   Diagnose misses by category, not one at a time. Failures cluster: several misrouted terse utterances usually mean a topic classification description that only matches full sentences; adversarial passes that shouldn't pass usually mean a missing instruction or an over-broad action. Fix the configuration cause, not the individual test.

6. 6.

   Make the run a release ritual with thresholds. Write the gate down: no agent change ships unless topic accuracy ≥ 90%, action accuracy ≥ 90%, and all adversarial cases pass — adversarial failures are release blockers regardless of the aggregate. Wire sf agent test run into the same CI pipeline that deploys the agent metadata, and grow the suite: every production misfire becomes a new numbered case.

Make it yours

• SAP Jouleno equivalent public testing harness yet, which raises the bar rather than lowering it: script your utterance set against Joule in a test tenant and log topic/skill selection manually; the CSV from step 2 is platform-neutral.
• Microsoft Copilot Studiouse the built-in test pane plus Power CX analytics for topic-trigger coverage; the same 15/8/7 structure applies, and adversarial cases matter more because connectors can reach far beyond one org.
• Oracle Digital Assistant / Fusion agentsutterance testing is native (batch testing of intents); import the set and score intent-resolution accuracy exactly as above.

Quality check

• Adversarial cases assert on actions, not tonethe injection test passes only if no unauthorized action was invoked; a polite refusal that still called the action is a failure.
• The baseline score is recorded with the agent versionan accuracy number without the config version it measured is trivia, not a regression reference.
• Test utterances are refreshed from production transcriptsreview real conversations monthly for phrasings the suite lacks; a static suite decays into false confidence.
• Someone other than the agent's builder reviewed the expectationsthe builder testing their own fence tests what they already believed.