Put AI Where Checking Is Cheap

BAS CRM, the appraisal platform I’m on the team building, has three AI features: fee quotes, an assistant and a voice agent. The feature list describes the first with one word that I think is the most important design decision in all three: the quotes are drafted.

The Wrong First Question

Most conversations about an AI feature start with capability: is the model good enough to do this? It’s the wrong first question, because the answer is nearly always “mostly”. Models are right often enough to be useful and wrong often enough to matter, and that won’t change by next quarter.

The better first question is about the person on the other end: when the model is wrong, what does it cost to notice?

Why Quotes Fit

Fee quotes sit close to the ideal. Producing one means weighing the property, the kind of assignment, the turnaround and how the firm prices its work, then writing it up. Checking one means an experienced appraiser reading a number and a paragraph and knowing within seconds whether it’s right. That gap between the cost of producing and the cost of checking is exactly where AI earns its place.

It’s also why “drafted” matters. A draft has an owner, and the owner isn’t the model. The quote that reaches a client is the appraiser’s quote, so approving it should be a deliberate act, not a default someone clicks through on a busy afternoon.

Model Output Is User Input

Inside the system, model output deserves exactly the trust we give a form submission from a stranger: none by default. A model can be confidently wrong in ways a schema catches instantly, so structured output goes through the same validation as any other request:

// $output is the model's JSON, decoded. Treat it like a request from a stranger.
$draft = Validator::make($output, [
    'fee_cents' => ['required', 'integer', 'min:1'],
    'turnaround_days' => ['required', 'integer', 'between:1,60'],
    'summary' => ['required', 'string', 'max:2000'],
])->validate();

Validation catches the malformed. Authorization catches the overreaching: any record the output refers to must belong to the firm asking, checked on the server like every other request. The model is just another client, and the law still applies to it.

The Voice Agent Is the Hard Case

A voice agent breaks the draft pattern, because there’s no moment to review a sentence before the caller hears it. Speech is published the instant it’s generated. When you can’t check the output, the safety has to move somewhere else: into what the agent is allowed to do.

That suggests a rule of thumb. The less you can review an AI’s output, the narrower its permissions should be. A drafting tool can be ambitious, because a person stands between it and the world. An agent that speaks or acts directly should be able to inform, collect details and hand off, and should need a person for anything that commits the business: a price, a promise, a change to an order.

Judgment Stays Human

None of this makes AI less useful. It makes it useful in the right places: absorbing the slow, mechanical part of the work and leaving judgment where it already lives. An appraiser’s expertise is knowing whether a number is right. The best AI feature hands them more numbers worth judging, faster, and never takes the judging away.