Linux Foundation PyTorch Certified Associate

Tensors are the data type everything else is built on. You create them, control shape and type, index and slice them, and combine them with broadcasting and matrix operations, moving between PyTorch and NumPy.

• Creating tensors: from data, zeros, ones, random
• Shape, dtype and the rules for changing them
• Indexing, slicing and masks
• Reshape, view, transpose and permute
• Broadcasting and element-wise operations
• Matrix multiplication and reductions
• Sharing memory with NumPy

You build A tensor workbook that solves a series of data-manipulation problems without writing loops.

The same tensor code can run on a processor, an NVIDIA GPU or an Apple GPU. You learn to select a device, move data to it, and avoid the errors that come from mixing devices.

• CPU, CUDA and MPS compared
• Checking which devices are available
• Moving tensors and models between devices
• Writing device-agnostic code
• Device mismatch errors and how to read them
• Memory on the device and how to watch it

You build A script that picks the best available device and runs one computation on every device you have.

Training depends on gradients, and autograd computes them for you. You see how PyTorch builds a graph as you compute, call backward, control when tracking happens, and avoid the usual gradient mistakes.

• Gradients and why training needs them
• requires_grad and the computation graph
• Calling backward and reading .grad
• Accumulation and why gradients must be zeroed
• no_grad, detach and inference mode
• Checking gradients numerically

You build A gradient check that compares autograd with a hand-derived derivative on a small function.

Modules are how PyTorch packages parameters and behavior. You assemble networks from layers, activations and regularization, write your own module, and inspect parameters and saved weights.

• nn.Module, forward and parameters
• Linear and convolutional layers
• Activation functions and when each is used
• Normalization and dropout
• Sequential models and custom modules
• Initialization, state_dict and parameter counts

You build A small convolutional network written as a custom module, with its parameters inspected.

A model is only as good as the data pipeline that feeds it. You write a Dataset, batch and shuffle with a DataLoader, apply transforms, and split data so that testing is honest.

• The Dataset contract: length and item access
• Built-in datasets and custom datasets
• DataLoader: batch size, shuffling and workers
• Transforms for images and tensors
• Train, validation and test splits
• Collate functions for uneven data

You build A custom Dataset and DataLoader for an image folder, with transforms and a clean split.

A training loop is a few lines that have to be exactly right. You choose a loss and an optimizer, write the loop, schedule the learning rate, log progress, and save checkpoints you can resume from.

• Loss functions for regression and classification
• Optimizers: SGD, momentum and Adam
• The loop: forward, loss, backward, step
• Learning rate schedules
• Logging loss and accuracy per epoch
• Checkpoints: saving and resuming

You build A training loop that logs its metrics and can resume from a saved checkpoint.

A model that scores well on its training data proves little. You evaluate on held-out data, spot overfitting, switch between training and evaluation modes, and load a saved model to make predictions.

• Train and eval modes and what changes
• Evaluating accuracy and other metrics
• Overfitting and how to recognize it
• Saving and loading weights safely
• Inference on a single input and on batches
• Reading a confusion matrix

You build An evaluation script that loads a saved model, reports its metrics and predicts on new images.

Speed matters once a model works. You use mixed precision, compile models, tune data loading, and measure where time goes with the profiler, so that each optimization rests on evidence.

• Mixed precision with autocast and gradient scaling
• Compiling a model with torch.compile
• Keeping the GPU busy: workers and pinned memory
• Timing GPU code correctly
• Profiling with torch.profiler
• Reading memory use and finding bottlenecks

You build A before-and-after profile of one training job, with each optimization measured on its own.

One GPU is eventually not enough. You learn how data parallel training splits batches across processes, how gradients are shared, and where sharding of very large models comes in, so you can read and reason about distributed code.

• Why and when to distribute training
• Processes, ranks and process groups
• DistributedDataParallel and gradient synchronization
• Distributed samplers and uneven data
• Sharding large models with FSDP
• Launching and debugging multi-process jobs

You build A single-node, multi-process training script with a distributed sampler and checkpoints written from one rank.

Build a classifier end to end: load and transform data, define and train a network, evaluate it, speed it up and keep a model you can reuse. You explain each choice as you would when reviewing someone else’s code.

• Choosing a dataset and defining success
• Building the data pipeline
• Defining, training and evaluating the network
• Speeding it up and measuring the gain
• Saving, loading and serving predictions
• Reviewing the code for common mistakes

You build A trained image classifier with a reproducible script, a metrics report and a saved model.