Databricks Machine Learning

Tuning is where cluster time is won or wasted. You run Optuna trials in parallel, move training itself onto Ray, and use pandas UDFs to fit many small models at once, with budgets and pruning that keep the bill in check.

• Search spaces, samplers and pruners in Optuna
• Running trials in parallel across a cluster
• Distributed training and tuning with Ray
• Fitting one model per group with pandas UDFs
• Trial budgets and early stopping
• Reading a finished search and picking the winner

You build A parallel hyperparameter search that logs every trial and stops weak ones early.

Basic tracking logs one run. Real tuning and cross-validation produce hundreds, so you structure them as parent and child runs, log what matters beyond metrics, and compare results against a baseline through the API instead of by eye.

• Parent and child runs for tuning and cross-validation
• Logging custom metrics, plots and artifacts
• Autologging and where it falls short
• Model signatures and input examples
• Querying and comparing runs programmatically
• Evaluating a candidate against a baseline

You build A tuning experiment with nested runs and an automatic comparison report.

Leakage is the quiet way models fail in production: a feature that looked fine in training used information from the future. You build feature tables with timestamp keys and learn lookups that return only what was known at each moment.

• Feature tables in Unity Catalog and their keys
• Timestamp keys for time series features
• Point-in-time lookups that prevent leakage
• Building a training set from feature lookups
• On-demand features computed at request time
• Streaming updates to a feature table
• Keeping training and serving features identical

You build A leakage-free training set, with one feature computed the same way in training and at the endpoint.

ML code breaks in ways a notebook run never shows. You write unit tests for feature logic and model code, integration tests for whole pipelines, and data checks that fail early, so that a change can be merged with evidence behind it.

• What to test in an ML project and what not to
• Unit tests for feature and transformation functions
• Testing training code on small fixtures
• Integration tests for an end-to-end pipeline run
• Data and schema checks as tests
• Running tests automatically on every change

You build A test suite that covers feature code, training code and one full pipeline run.

A project held together by clicks in a workspace cannot be reviewed or repeated. You describe jobs, endpoints and settings as code in Databricks Asset Bundles, then deploy one definition to development, staging and production.

• Why ML assets belong in version control
• Bundle structure: resources, targets and variables
• Declaring training and batch inference jobs
• Deploying one bundle to several environments
• Validating, deploying and running from the CLI
• Plugging bundles into a CI/CD pipeline

You build A bundle that deploys training and inference jobs to development and production targets.

Models decay as the world changes. You learn the kinds of drift, the statistical tests that detect them, and how Lakehouse Monitoring watches an inference table, so that degraded performance starts a retraining run rather than a support ticket.

• Data drift, label drift and concept drift
• Statistical tests for drift and when each applies
• Lakehouse Monitoring on an inference table
• Alerts that trigger a retraining job
• Champion and challenger comparison before promotion
• Guarding retraining against bad incoming data

You build A monitor on an inference table that starts a retraining job when drift crosses a threshold.

Not every model is a plain scikit-learn object. You wrap preprocessing, a model and postprocessing into one PyFunc, declare its dependencies, and register it in Unity Catalog so that anyone who loads it gets the same behavior.

• The PythonModel interface and its lifecycle
• Bundling artifacts and preprocessing with the model
• Declaring dependencies and environments
• Signatures that catch bad inputs early
• Registering versions and managing aliases
• Lineage from a registered model back to its data

You build A custom PyFunc model with its own preprocessing, registered and loadable by alias.

Replacing a model in one step bets everything on an offline metric. You compare shadow, canary and blue-green releases, decide what evidence promotes a version, and prepare the way back before the first request is routed.

• Shadow, canary and blue-green compared
• Splitting traffic between model versions
• Choosing the metrics that decide a promotion
• Comparing versions without fooling yourself
• Rolling back fast and cleanly
• Using aliases as the release switch

You build A canary rollout plan with a traffic split, success metrics and a rollback trigger.

A registered model delivers nothing until something can call it. You create endpoints, size them for the load you expect, split traffic across versions, log requests and responses, and read latency and cost to judge whether the setup is right.

• Creating an endpoint from a registered model
• Served entities, traffic configuration and updates
• Scaling, scale to zero and cold-start trade-offs
• Authentication and querying with REST and the SDK
• Logging requests and responses to inference tables
• Load testing and reading latency numbers

You build A serving endpoint with a traffic split, request logging and a load-test result.

Everything meets in one project. You take a model from tuned training through features, tests, bundles, a staged rollout and drift monitoring, then explain the decisions behind each stage the way you would to a reviewing team.

• Scoping a modelling problem and its constraints
• Training and tuning at the scale the data needs
• Wiring features, tests and bundles together
• Releasing with a canary and watching it
• Closing the loop with automated retraining
• Presenting the design and its trade-offs

You build A complete ML system: trained, tested, bundled, served, monitored and documented.