AWS Machine Learning Engineer

Models inherit every flaw in their data pipelines. You gather data from batch and streaming sources, choose formats and storage that training can read fast, and store reusable features in one place.

• Ingestion from S3, databases and streams
• File formats for training: CSV, Parquet and RecordIO
• Moving and joining data with Glue
• Streaming data with Kinesis
• SageMaker Feature Store for shared features
• Cost and throughput of training data access

You build A data collection pipeline that lands batch and streaming data in training-ready form.

Features often matter more than algorithms. You clean missing and noisy values, encode and scale features correctly, handle imbalanced classes, and build transformations you can apply the same way in production.

• Handling missing values and outliers
• Encoding categorical data and scaling numeric data
• Feature creation, selection and dimensionality reduction
• Imbalanced data: resampling and class weights
• Transformations with Data Wrangler and Glue DataBrew
• Applying identical transforms at training and inference

You build A reusable feature engineering pipeline tested on a messy dataset.

A model trained on skewed or leaking data looks good and fails later. You validate data, find leakage, measure bias before training, protect personal data, and set up labeling you can trust.

• Data validation and quality checks
• Train, validation and test splits without leakage
• Detecting bias in data with SageMaker Clarify
• Finding and masking personal data
• Labeling workflows and label quality
• Data versioning and lineage

You build A data quality and bias report for one training dataset, with fixes applied.

Training is a set of decisions about algorithms, resources and time. You pick an approach for each problem, run training jobs at the right scale, tune hyperparameters and recognize overfitting early.

• Choosing algorithms for tabular, text and image data
• Built-in algorithms, frameworks and your own containers
• Training jobs, instance types and Spot training
• Distributed training basics
• Hyperparameter tuning with automatic model tuning
• Overfitting, regularization and early stopping

You build A tuned model trained on managed infrastructure with a documented experiment history.

Foundation models change how many ML problems are solved. You decide between prompting, retrieval and fine-tuning, then adapt a model with your own data and check what the adaptation changed.

• Prompting, RAG and fine-tuning compared
• Preparing datasets for fine-tuning
• Parameter-efficient fine-tuning ideas
• Customizing models in Bedrock and SageMaker JumpStart
• Continued pre-training versus instruction tuning
• Cost and risk of customized models

You build A fine-tuned foundation model with a before-and-after comparison on one task.

A single accuracy number hides most failures. You choose metrics that fit the cost of each error, validate properly, explain individual predictions, and evaluate generative output with methods built for text.

• Precision, recall, F1, AUC and error metrics for regression
• Cross-validation and holdout testing
• Explaining predictions with SHAP and Clarify
• Evaluating generative output with automatic and human methods
• Comparing models and choosing a winner
• Model cards and documented limits

You build An evaluation report that compares two models and explains their errors.

How a model is served decides its cost and latency. You compare real-time, serverless, asynchronous and batch inference, package models in containers and size infrastructure for the traffic you expect.

• Real-time, serverless, asynchronous and batch inference
• Multi-model and multi-container endpoints
• Packaging models in containers with ECR
• Autoscaling endpoints
• Choosing instance types and accelerators
• Deploying to edge devices

You build A model deployed as an autoscaling endpoint with a batch option for bulk scoring.

A model you cannot retrain on demand becomes stale. You automate data processing, training, evaluation and registration into pipelines, then promote versions through reviewed releases.

• SageMaker Pipelines and Step Functions
• Model Registry and approval workflows
• CI/CD for models with CodePipeline
• Triggers for retraining
• Infrastructure as code for ML environments
• Testing pipelines and data in automation

You build An automated pipeline that retrains, evaluates and registers a model.

Production data drifts away from training data, quietly. You monitor inputs, predictions and quality over time, compare versions in live traffic and set alarms that start retraining before users notice.

• Data quality and model quality monitoring
• Detecting data drift and concept drift
• Bias and explainability drift
• A/B testing and shadow deployments
• Logging predictions for later analysis
• Alarms and retraining triggers

You build A monitoring setup that detects drift on a deployed endpoint and raises an alarm.

ML workloads hold sensitive data and use expensive hardware. You apply network isolation, encryption and least-privilege roles to training and endpoints, and cut spend with right-sized instances and commitments.

• IAM roles for training, pipelines and endpoints
• VPC isolation and private endpoints
• Encryption of data, models and artifacts
• Protecting model endpoints and generative applications
• Spot, Savings Plans and right-sizing
• Tracking cost per experiment and per prediction

You build A security and cost review of one ML workload with concrete changes.

Build a complete ML solution. You prepare data, train and evaluate a model, deploy it, automate retraining, and monitor it, then explain every decision to a reviewer.

• Choosing the problem and success metric
• Preparing data and training the model
• Deploying and automating the pipeline
• Monitoring and responding to drift
• Presenting the system

You build A working ML system with data pipeline, model, endpoint, pipeline and monitor.