Alignment in Plain Language: Helpfulness, Honesty and Caution

When a model refuses a reasonable request, or answers a risky one too readily, it is showing the results of choices made during its training. Alignment is the name for the work of shaping a model's behavior toward being helpful, honest and careful. Those three goals are useful, but they do not always agree. This article explains what that work tries to do, why the goals pull against each other, and what it means for anyone who builds a product on top of a model.

What alignment training is trying to do

After a model learns to predict text, a further stage adjusts its behavior. That stage uses examples of good responses, comparisons between candidate responses judged by people or by written rules, and guidance about which kinds of answer are preferred. The aim is a model that follows reasonable instructions, tells the truth as far as it can, and declines to help with clearly harmful goals.

The result is a tendency, not a rulebook. The model does not look up a policy and apply it. It produces responses that resemble the ones it was rewarded for. This distinction explains why behavior can shift with small changes in wording. A request phrased one way may be answered, and the same request phrased another way may be declined, even though a human would see no difference in intent.

Why helpful, honest and careful pull apart

Helpfulness says answer the question as it was asked. Honesty says do not claim certainty you lack. Caution says avoid anything that could cause harm. Usually all three are satisfied together, but not always. A question about the maximum safe dose of a common medicine can be answered helpfully, but the answer has to be accurate and has to mention that individual circumstances vary. A request for information that is harmless for most askers but occasionally misused presents a real trade-off. Refusing fails the many people with legitimate reasons. Answering gives some uplift to the few who would misuse it.

Honesty adds its own friction. A careful answer carries qualifications, and some readers hear qualifications as evasion. Every choice moves a little along one axis at the cost of another, and no setting removes the tension entirely.

Over-refusal and under-refusal

Two errors are common. Over-refusal happens when a legitimate request is declined because it resembles a risky one on the surface. A nurse asking about overdose thresholds for a safety protocol, a security engineer asking how a known attack works so they can defend against it, or a novelist writing a crime scene may receive a refusal or a heavily diluted answer. Under-refusal is the opposite. A request is answered that should not have been, often because it was framed innocently or split across several turns of conversation.

Neither error is rare in practice, and neither can be eliminated. What changes is the balance, and the right balance depends on who is asking and what the product is for. A model that is cautious by default will over-refuse some users. A model that is permissive will under-refuse others. The useful question for a builder is which of those errors costs more in the context they serve.

The form of a refusal matters too. A flat refusal with no alternative is a cost even when the decision behind it is right. A better refusal says what the model can help with, points to a more suitable resource where one exists, and avoids implying that the person asking is up to something. Refusal style is an area where builders can improve the experience without changing the underlying decision.

Policy inside a product

Builders control part of the behavior. Instructions in a system prompt, the kind of user the product serves, the topics in scope and the tone expected can all narrow or widen what an assistant does. A children's education tool and an internal engineering assistant should not behave the same way, and they can be configured to differ. But there is a floor set by the model provider, and some behaviors cannot be switched on by configuration. Read the usage terms and the documented limits before assuming that a behavior can be changed in your product. If a behavior matters to your users, test it, because the documentation describes intent and the tests show what happens.

Context supplied by a product can also shift what is appropriate. A statement that users are verified professionals may reasonably change how detailed an answer should be. But the model cannot check that statement, so a product should not rely on it alone for high-stakes actions. Pair any context with controls that do not depend on the model believing it.

Practice: measure your own false refusals

Write ten requests from your own domain that a careful human expert would handle without hesitation, but that contain words or topics that sound sensitive on first read. Keep the wording realistic, the way a real user would type it. Run each one through the model and mark the response as answered, partly answered or refused. Then add a few requests that are clearly outside what your product should do, and run those too.

The first set tells you how often the system over-refuses for the people you serve. The second tells you whether the boundary sits where your product needs it. If the first set shows many refusals, look at the wording that triggered them before changing anything, because a small rewording in the system prompt often has a larger effect than a change in the user's request. Keep the results with the date and the model version, so you can see whether later changes improved the balance or simply moved it.