Measuring Jailbreak Resistance Honestly

A claim that a model resists jailbreaks is not a measurement. Resistance is a property of a system under a particular set of attempts, and the result depends on how those attempts were chosen, how failure was defined and how the system was configured when tested. One prompt that failed to produce a bad answer says almost nothing. A careful measurement takes more work, and it produces claims others can check. This article describes the structure of such a measurement at a general level. It leaves out attempt text on purpose.

Define failure before testing

Decide what counts as a failure before running anything, or the results will be shaped by what looks bad afterward. Failures differ. The model may produce content the product forbids. The system may take an action without the authority it needs. It may reveal a system instruction it was told to keep private. It may follow instructions from a retrieved document instead of the user's request. Each has a different severity and a different owner. A single pass rate that blends them hides the most serious failure inside an average. Severity should also set the order of reporting. A failure that exposes restricted data outranks one that produces an unpleasant sentence, and the report should say so plainly.

Write each failure as an observable condition, so a grader can decide yes or no from the output and the system's logs rather than from the tester's impression.

Organize attempts into categories

Group attempts by the mechanism they probe, described at a level that does not teach the attack. Common categories include reframing a request as a role or a hypothetical, trying to override earlier instructions, hiding instructions inside retrieved or user-supplied content, escalating across several turns, changing language or encoding, and claiming authority the user does not hold. Each category needs enough variation that a single wording change cannot make it pass or fail by accident. Balance each category with attempts that should succeed, such as legitimate requests that look unusual. A test set made only of hostile-looking requests pushes a system toward over-refusal, and a measurement built that way will never show the cost.

The test set is itself sensitive. Store it under access control, keep it out of public repositories and share results rather than raw attempts with anyone who does not need them.

Report results with their conditions and limits

A failure rate means little without context. Report the model and configuration version, the system instructions in use, the sampling settings, the number of attempts in each category, the date of the run and the grader used. Model outputs vary between runs, so repeat each attempt and report the spread rather than one figure.

Sample size matters. If a category has forty attempts and three failures, the honest reading is three in forty, with a wide range around it. A single rate without its denominator hides both the sample size and the uncertainty. Where your tooling can calculate an interval, include it so readers can judge how much weight the number can bear. Then state what was not tested. An honest statement lists its gaps, and that list is what invites checking. Words such as unbreakable or safe under any prompt cannot be supported by any finite test, and a claim that uses them will not survive the first person who tries.

Combine automated grading with human review

Automated graders are cheap and fast, which makes them the default. They also make errors, and the errors are not random. A grader may count a polite refusal as a pass in one case and as a failure in another, or miss a failure stated indirectly. Sample results for human review on each run, measure how often the grader and the reviewers agree, and examine the disagreements. When agreement is low for a category, the grader needs work before its numbers mean much. Give reviewers the same failure definitions, so their judgments are consistent with each other. Disagreement between graders is itself a signal. When two automated graders disagree on a result, route it to a person rather than averaging the verdicts.

Retest after every change

Resistance is not a fixed property. Changes to the system prompt, the model version, the retrieval sources or the available tools can alter behavior in ways nobody predicted. Run the full suite after each change, compare results by category, and treat a new failure in a previously clean category as a regression even when the overall rate looks stable. Overall rates can hide movement inside categories, and the category is usually where the cause shows up. Keep a history of runs as well, so that a change in results can be traced to the configuration that produced it.

Practice with a measurement plan

Write a one-page measurement plan for a system you know. Name the failure definitions, the categories, the number of repeats per category, the grader and review sample, and the template for reporting results. The plan should also say who may run the suite, where its results are stored and who receives a report of failures. Give the plan to a colleague with instructions to find its weakest assumption. The weakest assumption is often in the failure definitions, because those decide what the numbers mean. Revise the plan around that assumption and keep the earlier version, so the next review can show how the thinking changed.