Taking the Clinical Decision Out of the LLM
An LLM asked to do therapy improvises. It's trained to be agreeable, and the decisions that matter most in a clinical context — which technique, how deep, is this safe right now — are exactly the ones you don't want an agreeable model making.
We tried doing this with prompts first. Prompts don't hold. A sufficiently emotional message talks the model past its instructions, and the failure is silent: you get a fluent, warm, confident reply that happens to be clinically wrong. There's no exception to catch.
So we took the decision out of the model.
The split
Every turn runs the same fixed pipeline:
SCORE → select bucket → admission gate → select action → micro-practice → crisis pre-screen → GENERATE
Two of those steps are the LLM. The rest is Python.
- SCORE — the model reads the message and emits a state assessment across five dimensions
- GENERATE — the model writes the words, given an action that has already been chosen for it
Everything in between is deterministic code. The model never decides whether a clinical step is appropriate. It doesn't see the admission table and isn't asked to reason about it — by the time it's called to generate, the decision has already been computed.
This is the whole idea, and it's not specific to therapy: if a decision has consequences you can enumerate, don't ask the model to make it. Ask the model to characterize the input, decide in code, then ask the model to render the output.
The admission table
Nine therapeutic schools × four client states. The cell says what's admissible.
Concretely: CBT with a regulated client admits a thought record, evidence-checking, a behavioral experiment. The same CBT, the same Avatar, a client who's decompensating — grounding only. The technique set doesn't narrow because the model judged the moment and felt cautious. It narrows because the state score indexed a different row.
Underneath the table sit 58 evidence-based micro-practices, each carrying its own contraindications. Depth is earned across a session and resets on decompensation. That's a state machine, not a disposition.
The property this buys: the gate can't be talked out of. A client who is very articulate about why they're fine does not get deeper work than a client who isn't. Their state score does that, and the score reads what they said, not how persuasively they said it.
Crisis detection runs before generation, not after
The ordering is the design decision.
The common pattern is to filter after: the model writes something, a classifier checks it, you block or regenerate. That's fail-open. The model's output is the default and the filter is an exception — so every gap in the filter is a shipped reply.
Ours runs before any therapeutic reply is composed:
- Stage 1 — deterministic keyword pre-screen. Pure code, no model. Target ≤50ms, measured typically under 5ms. Cheap enough to sit in front of every message.
- Stage 2 — zero-temperature classifier. Deterministic by configuration, so the same message gets the same verdict every time. Target ≤3s, measured 0.8–2.0s.
- Stage 3 — continuous re-evaluation. Risk isn't assessed once and filed.
On trigger, generation is skipped entirely. A fixed message goes out — not a generated one. The named supervising clinician gets an email and a live console push, and an immutable row lands in the alert table. The safety flag doesn't decay: a risk signal from three weeks ago still constrains what's admissible today.
The fixed message is deliberate. Acute risk is the last moment you want a language model being creative.
We're also deliberate about what does not trigger it: metaphor ("this is killing me"), reflective talk about past events, general sadness or burnout without stated intent, outward-directed anger. False positives erode trust faster than false negatives get caught downstream — a system that panics at sadness is one people stop talking to honestly.
What it cost
The honest part. Determinism isn't free, and most of the price is paid in places that don't show up in a demo.
Conversational flow. The gate sometimes produces a turn that's stiffer than the model "wanted." The model has a better-sounding sentence available and the table won't allow it. That's the trade working as designed, and it's still a real cost — you can feel it in transcripts.
Iteration speed. Adding a school isn't a prompt edit. It's a table change plus clinical review. The 58 micro-practices and their contraindications were authored, not few-shotted. A prompt-only competitor ships a new modality in an afternoon; we don't.
It can't do general conversation. By construction. The same machinery that stops it from improvising a technique stops it from being a good all-purpose chatbot. That's intentional, and it is nonetheless a capability we don't have.
Engineering surface. A deterministic spine is a lot of code that a prompt would have replaced. It has its own bugs, and they're our bugs rather than the model's.
What isn't solved
The crisis protocol catches explicit disclosure and routes it to a human quickly. It does not detect risk someone is deliberately concealing, and it can't intervene physically. What it buys is minutes and a named clinician — we don't claim more.
The outcome-slope metric — scoring whether a conversation moved someone toward equilibrium rather than just scoring each turn — is still roadmap, not shipped.
The full crisis protocol is published in detail so institutional partners can audit it rather than take our word for it. The nine methods and the shared spine are documented separately.
Frequently Asked Questions
Why doesn't Hamo AI let the LLM choose the therapeutic technique?
Because prompts don't hold. An LLM is trained to be agreeable, and a sufficiently emotional message talks it past its instructions — the failure is a fluent, warm reply that happens to be clinically wrong, with no exception to catch. So the decision was moved out of the model entirely: pure Python adjudicates every turn.
What does the LLM actually do in Hamo's pipeline?
Exactly two steps: SCORE, where it reads the message and emits a state assessment across five dimensions, and GENERATE, where it writes the words for an action that has already been chosen for it. Bucket selection, the admission gate, action and micro-practice choice, and the crisis pre-screen are all deterministic code.
What is the admission table?
A grid of nine therapeutic schools by four client states; each cell says which techniques are admissible right now. Underneath sit 58 evidence-based micro-practices, each with its own contraindications. Depth is earned across a session and resets on decompensation — the gate can't be talked out of, because it reads the state score, not the client's persuasiveness.
How does Hamo AI's crisis detection work, and how fast is it?
It runs before generation, not after. Stage 1 is a deterministic keyword pre-screen measured typically under 5ms; stage 2 is a zero-temperature classifier measured at 0.8–2.0 seconds; stage 3 is continuous re-evaluation. On trigger, generation is skipped and a fixed message goes out while the named supervising clinician is alerted by email and console push.
What does the deterministic design cost?
Four real prices: turns are sometimes stiffer than the model could have produced; adding a school is a table change plus clinical review, not a prompt edit; the system can't do general-purpose conversation by construction; and the deterministic spine is a large engineering surface with its own bugs.
What doesn't this system solve?
It catches explicit crisis disclosure and routes it to a human in minutes, but it cannot detect risk someone is deliberately concealing and can't intervene physically. The outcome-slope metric — scoring whether a whole conversation moved someone toward equilibrium — is still roadmap, not shipped.