CompTIA DataAI

Experiments settle questions that observation cannot. You design tests with adequate power, handle many comparisons honestly, run online experiments and reason about cause when randomization is not possible.

• Hypothesis tests, power and sample size
• Multiple comparisons and false discovery
• A/B testing and sequential analysis
• Bootstrap and resampling methods
• Confounding and causal inference basics
• Observational methods when you cannot randomize

You build An experiment design with power analysis and a pre-registered analysis plan.

Most models are matrices and gradients underneath. You work with decompositions, projections and eigenstructure, and the calculus of optimization, so that model behavior has an explanation rather than a mystery.

• Vector spaces, projections and least squares
• Eigenvalues, singular value decomposition and PCA
• Matrix calculus and gradients
• Convexity and why it matters
• Gradient descent and its variants
• Numerical stability and conditioning

You build A from-scratch implementation of regression and PCA that matches a library result.

Most failures come before the algorithm. You turn a business question into a target and a metric, set baselines, prevent leakage, engineer features and choose model families on evidence.

• From business question to modeling target
• Baselines and simple models first
• Data leakage and how to detect it
• Feature engineering and selection
• Generalized linear models and their uses
• Process models for data science projects

You build A problem framing document with target, metric, baseline and leakage review for a real question.

A model’s output is only useful if its meaning is understood. You analyze errors, quantify uncertainty, connect predictions to decisions and costs, and explain results to people who do not read code.

• Error analysis and slicing by segment
• Uncertainty, intervals and calibration
• Cost-sensitive decisions and thresholds
• Interpretability and explanation methods
• Fairness and bias analysis
• Presenting results to decision makers

You build A decision memo that turns model output and uncertainty into a recommended action.

Supervised methods carry most applied work. You compare regularized linear models, trees, ensembles and kernel methods, tune them properly and deal with imbalanced or noisy labels.

• Regularized regression and classification
• Decision trees and random forests
• Gradient boosting in practice
• Support vector machines and kernels
• Hyperparameter search without overfitting the search
• Imbalanced classes and noisy labels

You build A benchmark of five model families on one problem, with tuned results and a justified choice.

Without labels, structure has to be found. You cluster, reduce dimensions, detect anomalies and model data that arrives in order, and judge results that have no ground truth.

• Clustering: k-means, hierarchical and density-based
• Mixture models
• Dimensionality reduction and visualization
• Anomaly detection methods
• Forecasting with classical and machine learning models
• Judging output without a ground truth

You build A segmentation and an anomaly detector on one dataset, with validation of both.

Deep models handle data other methods cannot. You train networks with backpropagation, regularize them, use convolutional and attention-based architectures, and adapt pretrained models.

• Backpropagation and optimizers
• Initialization, normalization and regularization
• Convolutional and recurrent architectures
• Attention and transformers
• Transfer learning and fine-tuning
• Diagnosing training that will not converge

You build A trained network with a training log that diagnoses and fixes an unstable first run.

Evaluation must resemble use. You choose metrics that fit the cost of errors, split data to match deployment, compare models with statistics and find the gap between test results and real performance.

• Cross-validation and time-aware splits
• Classification metrics and curves
• Regression and ranking metrics
• Calibration and learning curves
• Bias, variance and model complexity
• Comparing models with statistical tests

You build An evaluation protocol for one deployment scenario, with metrics, splits and a comparison.

Many decisions are optimization problems with a model inside. You formulate linear, integer and nonlinear problems, solve them, and combine prediction with optimization where the best action depends on a forecast.

• Formulating objectives and constraints
• Linear and integer programming
• Convex and nonlinear optimization
• Heuristics and metaheuristics
• Simulation-based optimization
• Combining forecasts with decisions

You build An optimization model for scheduling or allocation, driven by a forecast, with sensitivity analysis.

Text carries much of a business’s data. You represent language with embeddings, use transformer and language models for classification and extraction, and decide when to prompt, retrieve or fine-tune.

• Tokenization and text representation
• Embeddings and similarity search
• Text classification and entity extraction
• Prompting, retrieval and fine-tuning compared
• Evaluating generative outputs
• Failure modes: hallucination, bias and leakage

You build A text classification and retrieval system with an evaluation of its failures.

Images require their own methods and data practices. You train classification, detection and segmentation models, build augmentation and labeling plans, and evaluate vision systems with the right measures.

• Image classification with convolutional networks
• Object detection and segmentation
• Vision transformers and pretrained models
• Augmentation and labeling strategy
• Evaluation: intersection over union and average precision
• Edge cases, lighting and dataset shift

You build An image model with a labeling plan and an evaluation across difficult conditions.

A model has value only when it runs reliably. You build data and feature pipelines, serve models in batch and real time, version data and code, and monitor for drift, decay and silent failure.

• Data and feature pipelines
• Batch and real-time serving
• Reproducibility and versioning of data and models
• Continuous integration for machine learning
• Drift detection and retraining triggers
• Cost, latency and incident handling

You build A deployed model with a versioned pipeline, monitoring and a retraining trigger.

One project covers the full practice. You frame a problem, build and validate a model with sound statistics, deploy it, monitor it and defend the design and its limits to a skeptical review panel.

• Framing and scoping the problem
• Building, validating and explaining the model
• Deploying and monitoring it
• Estimating business value and risk
• Defending the work in review

You build A complete data science project: framing, model, evaluation, deployment and review notes.