From Analyst to Data Scientist: Which Gap Actually Matters

Analysts moving toward data science often assume the missing piece is more statistics. Statistics matters, but in most of these moves the gap sits somewhere else: in how questions get framed, how models are judged against simple alternatives, and how results reach someone who will act on them. An analyst already has more of the path than might be expected, and it helps to know which parts to keep and which to build.

What an analyst already brings

Analysts know the data. They can query it, notice when two sources disagree, and tell a metric that measures what it claims from one that merely has a convenient name. They understand the business well enough to question whether a number is plausible before anyone else notices a problem. Most analysts have presented results to people who were not technical, and they have learned that a chart is useful only when the question behind it is clear.

These skills carry directly into modeling. Many model problems begin with data issues that an experienced analyst is well placed to spot: missing values that are not random, a definition that changed partway through the year, or a label recorded after the outcome it is meant to predict. Those strengths are worth naming in interviews. They are evidence of judgment, not preliminary work to get through on the way to the real job.

Modeling judgment, not only modeling technique

The first real gap is judgment about when a model is worth building. Technique can be learned from a course, and much of it is documented well. Judgment comes from comparing many attempts against a sensible baseline. A baseline is the simplest reasonable prediction, such as the value from the same week last year or the average for a comparable group. A model that beats the baseline by a small margin may not justify its maintenance cost, while a model that looks excellent may be exploiting a leak.

Leakage is the mechanism to understand. It happens when information from the future, or from the outcome itself, enters the training data. Check the timeline of every input and ask whether it could have been known at the moment the prediction would be made. When order matters, split the data by time rather than at random, so the test set reflects the future instead of a shuffled version of the past. Write down the baseline, the split and the metric before looking at results. That habit is most of what makes a result believable.

Experiment design and what results can prove

Analysts often report what changed. Data scientists are more often asked whether a change caused it, which requires a design. A randomized experiment assigns people or units to versions at random, so that differences between the groups can be attributed to the change. When randomization is impossible, quasi-experimental methods try to approximate it, but each depends on assumptions that must be stated and checked.

Before any experiment starts, define the primary metric, the smallest change that would matter, and how long the test will run. Decide in advance how the result will be read. Stopping early because a result looks good is one of the most common ways to be fooled. Add guardrail metrics, measures that must not get worse, such as error rates or complaints. Sample size depends on how small an effect needs to be detected and how variable the outcome is. A formula is not needed to begin, but the team should be able to say whether a test is large enough to show the change it cares about.

Deployment and the step after the notebook

A model in a notebook affects no one. The gap between a notebook and a working system is where many analysts first meet engineering. The work includes making the code runnable by someone else, recording which data and code produced each prediction, scheduling refreshes, and checking that the model still works after the world changes.

A small concrete target helps. A scheduled job that reads a table, writes predictions to another table, and checks that the input columns still have the expected types and ranges is a complete system at this scale. Add an alert for inputs that drift outside their normal range, and write a short note on what a person should do when the alert fires. This work is less glamorous than tuning a model, and it is what most teams need first.

Practice: a three-project sequence

Choose three projects, each building on the last. The first is a forecast of something already tracked, compared against a simple baseline and tested on data split by time. Write down the baseline chosen and why it was fair. The second is an experiment designed on paper and then simulated, including the sample size reasoning and the guardrail metrics. Show what would happen if the test were stopped early. The third is a small pipeline that reruns weekly and writes a short log of its own failures.

After each project, write a page that a non-specialist could read. State the question, the baseline, the assumptions, and the main limit of the result in plain language, and name who would act on it and how. Communication runs through all three projects, because a result that no one can act on has not finished its work. Then test each project against one question: can the author explain the baseline, the leak that was checked for and the decision the result supports in two minutes, without opening the code? If not, the project is unfinished, whatever the accuracy score says.