Linux Foundation Secure AI Associate

Risk enters at every stage, from data collection to retirement. You map controls to each stage of the lifecycle and see who is responsible for them in a team that mixes data scientists, engineers and security staff.

• The stages of the ML lifecycle
• Where each risk enters the pipeline
• Controls at each stage
• Roles: data science, MLOps and security
• Secure-by-default project templates
• Retiring models and data safely

You build A lifecycle map that lists the risks and controls at each stage for one real ML project.

A model follows instructions from anyone who can reach its input. You defend against direct and indirect prompt injection, handle model output as untrusted, and limit cost abuse and data disclosure.

• Direct and indirect prompt injection
• Treating model output as untrusted input
• Insecure output handling: injection into downstream systems
• Input and output filtering and its limits
• Rate limits and cost controls against abuse
• Preventing disclosure of system prompts and data

You build A set of attack prompts and the defenses that stop or contain each one in a sample application.

An agent that can act can be tricked into acting. You limit what tools can do, confirm risky steps, separate identities, and keep an injected instruction from turning into an action with real consequences.

• Excessive agency and how it arises
• Least privilege for tools and credentials
• Human confirmation for high-impact actions
• Sandboxing code and browsing tools
• Separating agent identity from user identity
• Testing an agent with hostile inputs

You build An agent with scoped tools and approval gates, tested against injected instructions.

Retrieval puts your documents in front of a model, and an attacker’s too. You enforce access control at retrieval time, defend against poisoned documents, and protect embeddings and vector stores like any other data.

• How retrieval changes the attack surface
• Access control at query time, not index time
• Poisoned and malicious documents
• Embeddings as sensitive data
• Tenant isolation in a shared vector store
• Keeping secrets out of indexed content

You build A retrieval pipeline that filters results by caller permissions, with tests that try to cross tenants.

Training data decides model behavior and creates legal duties. You trace where data came from, control who can use it, protect personal information, check quality and bias, and respect licensing.

• Data lineage and provenance records
• Access control for datasets
• Personal data, consent and minimization
• Quality and bias checks before training
• Licensing and terms of use
• Retention and deletion

You build A dataset record with its lineage, access rules, privacy review and license terms.

A few bad examples can plant a backdoor. You learn how poisoning and label attacks work, verify the integrity of data sources, and use statistical checks and review to catch tampering before training.

• How poisoning and backdoor attacks work
• Verifying data sources and integrity with hashes
• Statistical checks for outliers and drift
• Label review and trusted subsets
• Controlling crowdsourced and scraped data
• Responding when poisoning is suspected

You build A data validation stage that checks integrity and flags suspicious samples, tested with injected poison.

Models and their dependencies are software, and attackers treat them that way. You protect registries and CI/CD, avoid unsafe model file formats, sign artifacts, and keep a bill of materials for what you ship.

• Model files, serialization risks and safer formats
• Registry access control and promotion rules
• CI/CD pipeline credentials and isolation
• Dependencies and third-party model risk
• Signing models and verifying them before use
• An AI bill of materials

You build A model release process that scans, signs and records each artifact, with a bill of materials.

A deployed model is an exposed service. You authenticate and rate-limit endpoints, defend against model extraction and abuse, isolate compute, and manage the secrets and networks around serving.

• Authenticating and authorizing model endpoints
• Rate limiting and quota design
• Model extraction and how to slow it
• Isolating GPU and compute workloads
• Secrets and network controls for serving
• Hardening the serving container

You build A hardened model endpoint with authentication, quotas and network limits, tested from outside.

Safety work belongs in development, not after release. You document models, test robustness against adversarial inputs, check for privacy leakage, and keep versions you can roll back.

• Model cards and documentation that auditors can use
• Adversarial robustness testing
• Measuring memorization and privacy leakage
• Red teaming a model or application
• Versioning, release gates and rollback
• Recording evaluation results

You build A model card and a test report covering adversarial and privacy checks, with a rollback plan.

Controls decay unless someone watches them. You inventory AI assets, collect telemetry, detect abuse and drift, respond to AI incidents, and apply governance and trustworthy AI principles across the lifecycle.

• Inventory of models, data and prompts
• Telemetry for abuse, drift and anomalies
• Alerting on AI-specific signals
• AI incident response: contain, investigate, recover
• Governance, accountability and review
• Trustworthy AI principles in practice

You build An AI incident runbook and a monitoring plan for one deployed model.

Take one AI product through a full security review: threat model, data, training pipeline, application, serving and operations. You decide controls, test them with attacks and record what remains risky.

• Scoping the product and its threat model
• Reviewing data and training controls
• Testing the application and agent layers
• Hardening release and serving
• Planning monitoring and response
• Writing the security case and residual risks

You build A security case for an AI product, with its threat model, controls, test results and residual risks.