Test Disruption Software on the Carrier Response

Key Takeaways

Test a disruption or exception tool on what happens with the carrier after the alert, not on how early or how clearly the alert appears.
Script the carrier side of the test: a clear reply, a vague reply, no reply, a reply that conflicts with tracking, and a cancellation.
A scenario passes when the carrier's answer changes the shipment record, the next action follows your approval rules and the right people are told.
Grade each scenario pass, fail or incomplete on a recovery test scorecard, using one of your own recent shipments.
Wilson, by Cartage, contacts carriers, follows up, rebooks from approved carriers under company rules and escalates when a person needs to decide.

What should you test after a disruption alert fires?

Test whether the software gets a usable answer from the carrier and turns that answer into the next shipment action. An alert says something may be wrong; only the carrier can say whether the load will still move, and when.

Disruption tools are often evaluated on detection: how early they flag a late truck, how well they predict delays, how the map looks. Those signals start the work but don't finish it. A prediction says the load might be late. The dispatcher says the driver is two hours behind, the truck broke down, or the load was never dispatched. Those are different questions, as the difference between a predicted delay and a carrier reply shows, and only the second one tells you what to do next.

So the test begins where the alert ends. Does the tool contact the carrier itself? Does it understand the reply? Does it update the shipment, notify the right people and, if the load can't move, start a recovery from carriers you have approved? Or does it hand a coordinator a red flag and wait?

Five carrier responses to script into the test

Real carriers rarely reply cleanly, so build the test around the replies your team actually gets. Have a teammate play the carrier dispatcher from a separate mailbox and phone, and run all five scenarios on the same shipment.

The clear answer. "Driver is delayed, new ETA tomorrow at 9:00." The tool should put the new ETA on the shipment and notify the team and, if configured, the customer, without anyone retyping the email.
The vague answer. "We're checking with the driver, will update." This is not a resolution. The tool should keep the exception open, follow up on a defined schedule and not mark the load as handled.
No answer. The carrier ignores the first message. Watch for a second attempt, a switch from email to a phone call where configured, and a clear escalation point if the silence continues.
The conflicting answer. The carrier says the load is on time while the tracking source shows the truck hasn't moved. The tool should flag the conflict and ask again rather than silently picking one version.
The cancellation. "We can't cover this load." The tool should start rebooking from the approved carrier list, respect the approval limit and escalate if no approved carrier accepts.

Run the cancellation twice: once with a replacement rate inside your approval limit and once above it. The first should proceed under your rules; the second should stop and ask a named person for a specific decision.

How to set up the test so the results mean something

Preparation decides whether the test proves anything.

Use your own shipment. Pick a real load from last month with its actual lane, carrier contacts, pickup window and delivery appointment. Sample loads hide the details that break workflows, such as a carrier who only answers by phone.
Load your rules first. Give the vendor your approved carrier list for the lane, the approval limit for rebooking, and who gets notified internally and externally. Without them, you only see default behavior.
Keep the carrier role on your side. Your teammate decides when and how to reply. If the vendor plays the carrier, the replies will be the ones the product handles best.
Reply through real channels. Email from a real mailbox, a phone call where the tool claims phone support, and EDI or API messages only if your carriers actually use them.
Write the expected outcome before you start. Agree in advance what the shipment record, the notifications and the next action should look like after each reply.

Note when each action happens, but don't turn this into a timing exercise. The question is whether the right action happens at all. Broader failure cases, such as missing order data, belong in a separate session; make the freight AI fail in the demo covers those. This test stays on the carrier.

How to judge the results: pass, fail and warning signs

A scenario passes when three things are true: the carrier's reply changed the shipment record correctly, the next action followed your rules, and the people who needed to know were told. It fails if any of those depends on a person reading the reply and doing the work by hand.

Watch for these failure patterns:

The tool stores the carrier's email, but the shipment status and ETA don't change.
"Will update" is treated as a resolved exception.
After no reply, the tool keeps resending the same email and never escalates.
In the cancellation scenario, it suggests or books a carrier that isn't on your approved list because the load is urgent.
The over-limit rebooking goes through anyway, or it stops with a generic alert that doesn't say what decision is needed.
The customer is notified of a new plan before the carrier has confirmed it.

Some results are judgment calls. If the vendor says a scenario "would work in production" but can't show it, mark it incomplete, not passed. If a step depends on integration work you haven't scoped, write down the dependency and who owns it. For the cancellation case, compare the escalation with what should happen when no approved carrier accepts the load: the person who picks it up should see each tender attempt and each response without asking around.

The recovery test scorecard

Fill in one row per scenario and grade each pass, fail or incomplete.

Scenario	What the tester does	Pass looks like
Clear answer	Replies by email with a new ETA	ETA updated on the shipment; team and customer (if configured) notified
Vague answer	Replies "checking, will update"	Exception stays open; follow-up sent on schedule
No answer	Ignores the first two messages	Second attempt, phone call where configured, escalation with history
Conflicting answer	Says "on time" while tracking shows no movement	Conflict flagged; carrier asked to confirm
Cancellation within limit	Cancels; replacement rate is under the approval limit	Rebooking from approved carriers proceeds under the rules
Cancellation over limit	Cancels; replacement rate exceeds the limit	Rebooking waits for approval, with options and cost shown
No approved capacity	Every approved carrier declines	Escalation to the team; no unapproved carrier used

Read the pattern, not just the count. A tool that passes the clear answer but fails the vague and no-answer rows is reading emails, not running recovery. A tool that passes every row except the over-limit cancellation has a control problem, which matters more than any single miss elsewhere.

How Wilson, by Cartage, handles the carrier side of a disruption

Wilson is an AI logistics agent that replaces the TMS for freight execution and connects to the company's ERP or existing order data flow. It tracks each load from the best available source (carrier API, GPS/ELD data where available, or the carrier website by PRO number) and contacts the carrier by email or phone when those sources don't give an answer.

When a disruption appears, Wilson contacts the carrier by email, API, EDI where supported, or phone where configured, follows up when a reply is missing or incomplete, and updates the shipment with what the carrier confirms. It notifies the team in Slack or Microsoft Teams and the customer if configured. If the load can't move, Wilson initiates rebooking from the approved carrier list: inside the company's rules it proceeds, and above a configured limit it waits for approval. If no approved carrier accepts, Wilson escalates. Messages, confirmations and changes are recorded in the shipment's change history, so each row of the scorecard can be checked against the record.

Wilson starts with assisted execution, and the approval boundaries are set by the customer. Most teams start on a few lanes and expand the scope as the workflow holds up.

To run these scenarios on your own freight, bring one recent disruption and your approved carrier list to Cartage and grade the carrier response yourself.