NVIDIA Generative AI LLMs

A model sees tokens, not words, and tokenization quietly decides cost, multilingual quality and many failure modes. You learn how vocabularies are built, what breaks when text and tokenizer disagree, and how to measure the damage.

• Byte-pair encoding and related tokenizers
• Choosing a vocabulary size
• Special tokens, chat templates and padding
• Tokenization across languages and code
• Extending a vocabulary for a new domain
• Counting tokens to estimate cost

You build A tokenizer comparison report on your own domain text.

Data quality sets the ceiling on model quality. You clean, filter, deduplicate and format text for each stage of a model’s life, and see what changes between a pretraining corpus, an instruction set and the inputs sent at inference time.

• Sourcing and licensing text corpora
• Quality filtering and language identification
• Deduplication and contamination checks
• Removing personal data and toxic content
• Formatting instruction and conversation data
• Mixing sources and balancing a dataset
• Synthetic data and where it helps

You build A curation pipeline that turns raw text into a tuning-ready dataset.

Prompting is the cheapest way to adapt a model, and often enough. You write zero-shot, few-shot and chain-of-thought prompts, steer a model into a new domain, and control format and randomness so that software can use what comes back.

• Zero-shot and few-shot prompting
• Chain-of-thought and self-consistency
• Steering a model into a new domain
• Sampling controls: temperature, top-p and top-k
• Structured output and constrained decoding
• Prompt templates and versioning
• Knowing when prompting is not enough

You build A prompt suite for one domain task, tested against output-format rules.

When prompts fall short, you change the weights. This module covers full fine-tuning, supervised and instruction tuning, and parameter-efficient methods such as LoRA, plus how to keep what the model already knew while it learns something new.

• Full fine-tuning against parameter-efficient methods
• Supervised and instruction fine-tuning
• LoRA, QLoRA and adapters in practice
• Preference tuning with RLHF and DPO
• Catastrophic forgetting and how to limit it
• Choosing hyperparameters and stopping points
• Merging and versioning tuned weights

You build A LoRA-tuned model for a new domain, compared with its base model.

A score is only useful if you know what it measures. You use standard benchmarks and task metrics, read a model’s mistakes by hand, and design evaluations that keep working as the data and the models grow.

• Perplexity, accuracy and overlap metrics
• Public benchmarks and what they miss
• Model-based judging and its biases
• Systematic error analysis
• Building a task-specific evaluation set
• Automating evaluations at scale
• Spotting benchmark contamination

You build An evaluation suite with an error taxonomy for a tuned model.

A large model does not fit on one device. You split work by data, tensor, pipeline and sequence, shard optimizer state, and see how each choice trades memory against communication between GPUs.

• Data parallelism and gradient synchronization
• Tensor and pipeline parallelism
• Sequence and context parallelism
• Sharded optimizers and parameter sharding
• Mixed precision and loss scaling
• Activation checkpointing
• Training with NeMo and Megatron-LM

You build A multi-GPU fine-tuning run that combines two forms of parallelism.

Most slow or crashing jobs come down to memory and idle hardware. You budget memory for weights, gradients, optimizer state and activations, find stalls with profilers, and tune batch size to the hardware you actually have.

• Memory budget for weights, gradients and activations
• Reading GPU utilization with nvidia-smi
• Timelines and kernels in Nsight Systems
• Finding data-loading and communication stalls
• Choosing batch size and gradient accumulation
• Diagnosing out-of-memory errors
• Interconnect bandwidth and NCCL behavior

You build A profiling report that finds and fixes the main bottleneck in a training job.

Serving cost is set by latency and throughput choices. You compress models with quantization, pruning and distillation, tune batching and caching, and plan how to roll out an updated model without breaking the people who use it.

• Latency, throughput and the trade between them
• Quantization to lower precision
• Pruning and distillation
• KV caching and continuous batching
• Speculative decoding
• Compiling models with TensorRT-LLM
• Rolling out model updates safely

You build An optimized model with a latency and throughput comparison against the original.

A model is useful once it answers requests reliably. You package models into containers, serve them behind an inference server, and let an orchestrator scale replicas and keep the service healthy as load changes.

• Building inference containers
• Serving with Triton Inference Server and NIM
• Model repositories and configuration
• Kubernetes deployments, GPUs and scheduling
• Autoscaling on load and queue depth
• Health checks and real-time metrics
• Load testing a serving endpoint

You build A containerized inference service that scales on a Kubernetes cluster.

Models degrade quietly. You build dashboards and alerts, trace anomalies to their cause, and run the loop of retraining and versioning that keeps a live model accurate and recoverable.

• Dashboards for quality, latency and cost
• Structured logs and request tracing
• Detecting drift and anomalies
• Root-cause analysis on a bad release
• Triggering retraining from production data
• Model registries and versioning
• Rolling back safely

You build A monitoring and retraining loop with alerts and a rollback path.

Responsible AI is engineering work. You audit a model for bias, add guardrails around its inputs and outputs, test them with adversarial prompts, and set up the monitoring that shows compliance holding over time.

• Sources of bias in data and models
• Running a bias audit
• Input and output guardrails with NeMo Guardrails
• Red-teaming and adversarial prompts
• Privacy, consent and data retention
• Documenting models and decisions
• Compliance monitoring after release

You build A bias audit and a guardrail layer tested against adversarial prompts.

Take one model from a business problem to a monitored service. You choose between prompting and tuning, adapt and evaluate the model, optimize and deploy it, then document the safeguards and defend your decisions.

• Scoping the problem and the success measures
• Choosing prompting, tuning or both
• Adapting and evaluating the model
• Optimizing and deploying it
• Monitoring and governing the result
• Presenting the decisions and their trade-offs

You build A tuned, optimized and monitored LLM service, with its evaluation and audit records.