Sandbox Testing for AI Agent Payments: What to Validate Before Go-Live

TL;DR

• Before flipping to production, sandbox-test five categories: happy path, decline scenarios, async events (refunds + chargebacks + dispute), policy edge cases, error handling.
• Don't only test green paths. Most production incidents come from the ~10% of paths that aren't the happy one.
• Total time on a working keyboard: 2-3 days for a thorough sandbox suite. Production pain you avoid: incalculable.

A sandbox-tested integration ships to production cleanly. A green-path-only tested integration ships and breaks within the first week on something the sandbox would've caught. Use this checklist to spend 2-3 days now to save 2-3 weeks of production firefighting later.

What categories to test

1. Happy path

Validate the basic flow works:

• ✅ Issue a card successfully.
• ✅ Issue a delegation against the card.
• ✅ Run an authorization that the policy approves.
• ✅ Receive the auth.created webhook.
• ✅ Verify the ledger entry is created with proper tags.
• ✅ Receive auth.settled webhook on settlement.
• ✅ Confirm the agent's available budget reflects the spend.

This is the bare minimum. The other 4 categories matter more.

2. Decline scenarios

Test every common decline path:

• ✅ Amount exceeds per-transaction cap.
• ✅ Amount + prior spend exceeds daily/weekly/monthly cap.
• ✅ Merchant not on whitelist.
• ✅ MCC blocked.
• ✅ Geo not allowed.
• ✅ Outside business hours (if rule is set).
• ✅ Velocity exceeded (per minute / per hour / per day).
• ✅ Card frozen.
• ✅ Delegation expired.
• ✅ Delegation revoked.

For each: verify decline reason code matches expected, agent-side handler can interpret it, audit trail entry is created.

3. Async events

Refunds, chargebacks, settlement-after-auth-mismatch, partial captures:

• ✅ Refund a successful auth — verify ledger reversal.
• ✅ Partial refund — verify partial reversal.
• ✅ Chargeback initiated — verify dispute event fires.
• ✅ Chargeback resolved (won) — verify proper accounting.
• ✅ Chargeback resolved (lost) — verify ledger adjustment.
• ✅ Auth that doesn't settle (auth-only, never captured) — verify hold expiration after timeout.
• ✅ Partial capture — auth $100, merchant captures $80, $20 hold released.
• ✅ Settlement amount differs from auth amount (FX, network adjustment) — verify ledger handles delta.

Sandbox supports forced-refund, simulated-chargeback, and simulated-settlement endpoints for these tests.

4. Policy edge cases

The cases that break things in production:

• ✅ Transaction at exactly the per-transaction cap (boundary).
• ✅ Transaction one cent over the cap.
• ✅ Multiple authorizations landing simultaneously, total exceeding daily cap.
• ✅ Policy update mid-flight — verify in-flight tx evaluates against original version.
• ✅ Multi-currency transaction with FX.
• ✅ Agent at zero budget — every transaction should decline.
• ✅ Agent attempts charge against a delegated user with revoked underlying authorization.

5. Error handling

Failure modes you don't normally see:

• ✅ Webhook endpoint down — verify retry happens, eventual delivery + replay-ability via the inspector.
• ✅ Webhook signature wrong — verify the request returns 401 and platform marks it for retry.
• ✅ Idempotency key reused — verify duplicate request returns the original response, not a new one.
• ✅ Network error mid-charge — verify your retry logic doesn't double-charge.
• ✅ Auth approval webhook timeout — verify proper fallback (your backup behavior).
• ✅ Card token expired — verify proper error.
• ✅ Invalid card token — verify error.

What's the sandbox limitations?

Sandbox doesn't fully simulate every production behavior. Known gaps:

• Network latency variability. Sandbox is more predictable than production. Don't tune timeouts based purely on sandbox.
• Real fraud signals. Fraud scoring is mocked in sandbox. You can simulate flagged transactions but not the gradual fraud-score evolution.
• Cross-acquirer behavior. If you're using multi-acquirer routing, sandbox uses a single test acquirer.
• Geo + regulatory variations. Sandbox is region-agnostic. Real production may have region-specific behaviors that sandbox doesn't replicate.

For these gaps, do a small production smoke test before full launch.

What's the typical sandbox test plan look like?

A 3-day breakdown for a typical agent product:

Day 1: Happy path + basic declines.

Walk through the 5-minute quickstart. Then reproduce 5-10 decline scenarios. Confirm webhook handler works for each.

Day 2: Async events + idempotency.

Run refund scenarios. Run chargeback scenarios. Test idempotency with deliberate retries. Check ledger consistency.

Day 3: Edge cases + production smoke test.

Walk through policy edge cases. Boundary tests. Then flip to production with strict caps + monitoring + 5 test transactions on real cards.

What's the production smoke test?

Once sandbox passes:

Switch to production credentials (one env var).

Issue a real card to a single test agent.

Set tight policy: $5/transaction, $20/day, single allowed merchant.

Run 3-5 small real transactions.

Verify each in dashboard + ledger + webhook.

Expand limits gradually as confidence grows.

Total cost of production smoke test: ~$25-50 in real charges. Saves an order of magnitude in time.

FAQ

How long does the full sandbox suite take?

2-3 days for a thorough test plan. Faster if you skip async events or policy edges (don't).

Do I need a separate sandbox account from production?

Yes — entirely separate environments. Sandbox API keys (sk_sandbox_) only work in sandbox; production keys only in production.

Can I import production data into sandbox for testing?

No — sandbox is its own environment. You can replicate production policies and card configurations, but actual transactions are simulated.

What about load testing?

Sandbox supports moderate load (100s of TPS sustained). For higher load, coordinate with the platform's support team for dedicated load test windows.

What's the most common production incident?

Webhook handling. ~60% of post-launch incidents in our experience are: webhook signature mismatch, idempotency-key reuse, or timeout handling. Test all three thoroughly.

Related reading

• [Get Your First Agent Payment in 5 Minutes](/blog/shatale-publisher-quickstart) — the base quickstart
• [How to Inspect Webhooks for Agent Payment Integrations](/blog/inspect-webhooks-agent-payments) — webhook debugging deep-dive
• [The Publisher Developer Portal](/blog/publisher-developer-portal) — Sandbox tab reference

External references

• [Shatale API reference](https://docs.shatale.com/api) — full sandbox + production endpoint docs
• [Stripe testing guide (general fintech context)](https://docs.stripe.com/testing) — broader fintech sandbox testing reference

---

By Ted L.. Last updated 2026-04-29.