What to Ask an AI Consultant Before You Sign

Most AI proposals a UK SME receives are impossible to compare. One is four pages of methodology, one is a day rate, one is a fixed price with a diagram. None of them answers the question the owner is actually asking, which is: if I sign this, what will be different in six months and what will it cost me to keep?

The questions below are the ones that separate a consultant who has run this before from one who has read about it. They are deliberately awkward. A good supplier will have crisp answers to all of them and will not mind being asked.

What does it cost to run, and who absorbs a price rise?

Almost every proposal prices the build. Far fewer price the year after it.

Ask for the monthly running cost at your expected volume — model usage, hosting, licences, and the support time to keep it working. Then ask what happens when the model provider changes their prices or retires the version you are on. Both happen, and neither is hypothetical.

The answer you want is a number with the assumptions written next to it: at roughly this many documents a month, expect this much, and here is what moves it. The answer that should worry you is that running costs are "minimal" or will be "reviewed later".

What happens on the day it is wrong?

Not "how accurate is it". Accuracy figures are a lab measure and your firm does not operate in a lab.

The real question is about the Tuesday it produces a wrong figure in a client letter and somebody has already sent it. Who notices? How? What is the correction path, and who is named as accountable for that class of output?

If a supplier treats this as a testing question, they are building a demo. If they treat it as a process question — who checks what, before which outputs leave the building — they have deployed something into a firm that has customers.

Which of our processes are you not going to automate?

A consultant who says everything is a candidate has not looked properly.

The useful answer names things and explains why: this one because the exceptions outnumber the rule, this one because it is a judgement call your professional body expects a human to make. That list is worth as much as the list of things they will build. It is also the clearest sign that someone has understood how you work rather than pattern-matched your sector.

What data leaves our systems, and where does it go?

You are asking three things at once: which provider processes it, in which jurisdiction, and whether anything you send can be used to train a model.

The answer should be checkable — a named provider, a named region, a link to the terms that say so. "It's secure" is not an answer, and neither is "enterprise-grade". Under UK GDPR you remain the controller of client data whoever built the tool, so this is your exposure, not the supplier's.

If you handle health, care, or financial records, ask the same question again about logs and error reporting. That is where data ends up when nobody thought to ask.

What do we own at the end, and can somebody else maintain it?

Ask what happens if you stop working with them in a year. Which parts are yours — prompts, configuration, integration code, documentation — and which parts stop working the day the relationship does.

Then ask the sharper version: could a competent developer who has never met you pick this up from the documentation? If the honest answer is no, you are not buying a system. You are renting a person, which is fine as long as you priced it that way.

How will we know if it worked?

Get one number agreed before anything is built, measured before and after. Minutes per job, error rate, days to turn something round, cost per case.

Watch for the substitution, because it is the most common one in this market: the metric becomes usage. Seats filled, documents processed. Those measure adoption rather than value, and a tool can be busy and worthless at the same time.

What changes as the model changes?

This is the question that separates a project from a programme.

The system you sign for is built against a model that will be replaced, on a schedule you do not control. Ask what the supplier does when that happens — whether behaviour is monitored after a version change, who re-tests the outputs that matter, and whether that work sits inside the contract or becomes a new quote. The board-level version of this argument is in static controls, live models.

The shape of a good answer

None of these tests technical depth. They test whether someone has been through the unglamorous middle part — the corrections, the price change, the handover, the version that behaved differently on a Monday.

If the answers arrive as a diagram, keep looking. If they arrive as specifics, with the occasional honest "we don't know yet, here's how we'd find out", you are talking to someone who has done it.

Worth doing before any of these conversations: work out what you already have. Most firms cannot say what AI tools are in use, on whose accounts, against what data, and a proposal written on top of that fog gets priced for the fog. The AI readiness scorecard takes about five minutes and scores you on ownership, process clarity and measurement. Result straight away, no email required — and a supplier worth hiring will be glad you arrived with it.