Tony Brooks started in a law firm about twenty years ago and has been responsible for some form of automation ever since. That much experience should mean walking into a discovery meeting with an open mind, and it doesn't always work out that way. As technologists we bring our own biases into the room. We've usually done the maths before the process owner has finished their first sentence: what the business case will say, what the agent will cost and how accurate it needs to be. Quite often we're wrong on all of it.
The process
This story came near the end of our second AI wave talk, where the technology met a real client. It's a national law firm, so we won't name them, and it's everyday work: contract variation and renewal requests.
The requests land in a shared mailbox. A lawyer reads each one and creates a matter in the practice management system (PMS), where the firm tracks time and billing. They run a title search on the property, check that everything passes, generate the documents, file them in the document management system (DMS), then review the lot and email the client.
By hand, that's hours per request, frequent rework, and more than a week from the request arriving to the client getting their documents. The firm handles thousands of these a year. On paper it's a perfect candidate for automation, which is exactly when the biases kick in.
Bias one: counting the hours
Our first instinct is to count effort. Say, for the sake of example, it's three hours a request and hundreds of requests a week. Multiply it out and you're saving hundreds of hours a week, and the slide writes itself.
When we sat down with the process owner, the hours mattered, but the lead time mattered more: seven days from the request coming in to the client getting what they asked for. If the firm can turn a renewal around in four days instead of seven, that's a much better story to tell their client.
It changes where you look, too. Chasing hours points you at the steps that take longest. Chasing elapsed time points you at the waiting: the request sitting in the mailbox, and the handoffs where nothing happens at all.
Bias two: the price per run looks scary
The cost of AI is visible now. Copilot Credits have put a meter on agent work that everyone can watch tick over, and our piece on credits explains how it works. Technologists now walk into discovery worried about the number on it.
Say we'd run this process straight through an unoptimised agent and it came out at $38 a run. Our reaction would be alarm, and we'd be reaching for optimisation work before anyone had asked for it.
We asked the partner early on how much they'd pay per run to have this automated. For illustration, call it eighty dollars a run. The number that would have frightened us was already under their ceiling, and we'd been about to solve a problem the client didn't have.
Cost still matters. An unoptimised agent is wasteful, and there's plenty you can do to bring the cost per run down and make it more predictable. But a number can only be judged against a ceiling, and the ceiling belongs to the business. The price is rarely the most interesting figure on the page.
Bias three: expecting near perfection
Because we're automating something, we assume it has to go right 95 or 99 per cent of the time, and anything less feels like failure.
Ask the process owner how often the work needs reworking today. If it's one in five, the current process gets it right about 80 per cent of the time, which sets the bar. An automated version that does better than 80 per cent is meeting it.
Sloppy work is still sloppy work. The point is an honest baseline. Hold an agent to 99 per cent when the job is done at 80 per cent today, and a good project can sit in testing chasing the last few per cent that nobody asked for. This process also already ends with a person reviewing the work before it goes to the client.
Three questions, one success test
Each bias is a number we came up with on our own, and each has a question that fixes it. These are the three we ask any process owner before designing anything:
- What is the real value driver? It might be time saved, elapsed time, managing risk or something else, so don't assume it's hours.
- What is your cost ceiling per run? Ask how much they'd be willing to pay, each time, to have it done for them, and get their number before you show them yours.
- How often does it need reworking today? The answer is your real quality bar, in place of a figure picked because it sounds responsible.
Then use the answers to write the success test for the proof of concept. This is the step people skip. On the contract renewal work, the test came down to one question: if we can prove the technical pieces and get the lead time from seven days down to four or fewer, is going ahead a no-brainer?
A partner can say yes or no to that before anything is built. A proof of concept proves the concept, time-boxed, against a test agreed up front, and our two-week PoC piece lists what comes out of one.
Your test can fold in the other two answers as well: a cost per run under the ceiling they named, and rework no worse than it is today. From there the project ran with what we'd now call a forward deployed team.
Try it on your next process
Before your next automation conversation, write the three questions on a sticky note. Ask them before anyone opens a spreadsheet, and write the answers down in the process owner's own words.
Then turn those answers into a single sentence the process owner can say yes or no to. If you can't write that sentence, you're not ready to build.
If you're not sure which process to start with, look in a shared mailbox. This one started in one, and there are usually plenty more in there.