Debugging as a Way to Learn

Debugging feels like an interruption until it is treated as an experiment. A bug is the program showing a behavior that was not expected, and each step of finding the cause teaches something about the system, about one's own assumptions, and about reasoning under uncertainty. The same habits carry over to other fields, including the behavior of models.

Reproduce before reasoning

Start by making the failure happen on demand. A bug that appears sometimes is harder to study than one that appears every time, so find the smallest input and the simplest sequence of steps that produces the problem. Write them down exactly. If the failure cannot be reproduced, that is itself a result, and it points toward timing, leftover state or environment differences.

Reasoning before reproduction is a common mistake. It produces plausible explanations that the evidence has not tested. A reliable reproduction gives a fixed point that every later step can be checked against. When the failure depends on a large input, strip the program or the data down until removing any piece makes the bug disappear, then keep what remains as the reproduction.

Narrow the search

Once the failure is reproducible, cut the search space in half repeatedly. In code, bypass a section and see whether the bug persists. In data, split the input and test each half. Version control can do this across history: bisecting the commit history narrows the search to the first change that breaks the behavior, which is often quicker than reading every change.

Many bugs sit at boundaries: the first or last element, a zero, an empty input, a value just above or below a threshold, or a point where two components exchange data. When narrowing, check the values at each boundary before examining the middle. The cause is more often at an edge than in the middle of a long calculation. Suppose a function returns the wrong total only for some orders. Print the running total after each item and compare the first wrong value with the item that produced it. Usually the cause is an item with an unusual type, a missing field or a boundary condition the function never tested.

State a falsifiable hypothesis

A useful hypothesis is one that a single test could contradict. "Maybe it is the cache" is too vague to test, because almost any result can be explained by a cache. "The cache returns stale values when the input list is reordered" can be tested directly. Write the hypothesis in one sentence, then write the result that would show it is wrong.

This discipline protects against confirming what was already believed. If a test would have passed whether or not the hypothesis was right, the test tells you nothing, and a different one is needed.

Use the debugger and the logs for what they show

A debugger stops execution at a point and lets you inspect state: the values of variables, the call stack and the path taken. It is excellent for understanding the state at one moment. It can change timing, so it is less useful for bugs that depend on concurrency or real-time behavior.

Logs record a timeline of values across a run. They are cheap to add and noisy to read, and they reveal patterns a single stop cannot. Use logs when the question is what happened over time, and the debugger when the question is what is true at one point. Choosing the right tool is part of the skill. Conditional breakpoints, which stop only when a chosen condition holds, combine the two approaches. They are useful when a failure happens after thousands of normal iterations.

Apply the same habits to model behavior

Debugging a model follows the same structure. Reproduce first: fix the input, the settings and any randomness that can be controlled, and see whether the bad output repeats. Models can vary between runs, so repeat the input several times to see how often the failure occurs rather than trusting one result.

Then narrow. Change one part of a prompt at a time, or test subsets of the data, to find which part produces the behavior. Form a falsifiable hypothesis, such as "the answer changes when the order of the examples changes," and design a test that could contradict it. The methods are the same even though the system is probabilistic. The main difference is that rates must be measured rather than inferred from a single output. Before blaming the model, check the path around it. A bad answer can come from a malformed input, a truncated document or a preprocessing step that changed the text, and those are ordinary bugs that the same method finds.

Keep a record, then drill

After a bug is fixed, write a short record: the symptom, the cause, the signal that revealed it, and the habit that would have caught it sooner. Over several months these records show patterns. Most of one person's bugs may come from off-by-one errors in loops, from assumptions about input order, or from forgetting how a library handles missing values. A bug that seemed unique often matches one already written down, so the log becomes a guide to where to look first.

Choose a bug from the past month and rebuild it on paper. Write the reproduction steps, the hypothesis held at the time, the test that separated the hypotheses, and the cause. Compare the reconstruction with what actually happened, and note where the first guess was wrong and what led to the right answer.

For practice without a real bug, ask a colleague to plant one in a small program you have not seen before. Set a thirty-minute limit and keep a log of each step: the reproduction, each narrowing move, each hypothesis. When time runs out or the cause is found, write one sentence on what to check first next time. Reading the log afterward shows where the process was efficient and where it was guessing.