About AI Security & Governance
AI security and governance is the discipline of attacking, defending, and
overseeing AI systems so they can be trusted in production, and this course walks
through it in the order most practitioners learn it. It starts with
AI Risk Landscape (the OWASP Top 10 for LLMs, MITRE ATLAS, and the NIST
AI RMF), then Threat Modeling AI Systems using trust boundaries,
STRIDE, and MAESTRO, so you know what you are defending before you test it.
Next come the attacks themselves. Adversarial Machine Learning
covers evasion, data poisoning, model extraction, and membership inference, while
LLM & Agent Attacks covers prompt injection, jailbreaks, and
the abuse of tools and permissions in agentic systems.
AI Red Teaming turns those techniques into a repeatable practice:
scoping, threat-led test plans, tooling such as PyRIT and garak, and findings a
product team can act on. AI Supply Chain & Data Security and
Guardrails & Defensive Controls then cover the fixes, from
model provenance and safe serialization to input and output filtering, sandboxing,
and least-privilege agents.
Finally, the governance half: AI Risk Management Frameworks
(NIST AI RMF, ISO/IEC 42001), AI Regulation & Compliance
(the EU AI Act, GDPR, and sector rules), and
Governance, Privacy & Incident Response, covering policies,
documentation, audits, and what to do when an AI system fails: the skills that
separate a clever demo from a system an organisation can safely deploy.
Content
10 modules
01
AI Risk Landscape
The main categories of AI risk and the frameworks used to describe them: the OWASP Top 10 for LLMs, MITRE ATLAS, and the NIST AI RMF.
OWASP LLM Top 10MITRE ATLASNIST AI RMF
02
Threat Modeling AI Systems
Mapping assets, trust boundaries, and adversaries for models, data pipelines, and LLM applications before any testing starts.
STRIDEMAESTROTrust Boundaries
03
Adversarial Machine Learning
Evasion, data poisoning, backdoors, model extraction, and privacy attacks such as membership inference, with the defences that blunt them.
EvasionData PoisoningModel Extraction
04
LLM & Agent Attacks
Prompt injection, jailbreaks, insecure output handling, and how excessive permissions turn a chatbot flaw into a real incident in agentic systems.
Prompt InjectionJailbreaksExcessive Agency
05
AI Red Teaming
Scoping engagements, building threat-led test plans, automating attacks at scale, and writing findings that product teams can fix.
PyRITgarakAttack Reporting
06
AI Supply Chain & Data Security
Verifying models, datasets, and dependencies, spotting unsafe serialization, and protecting training and retrieval data from tampering and leakage.
Model ProvenanceAI-BOMSafetensors
07
Guardrails & Defensive Controls
Layering input and output filters, sandboxed tool use, least-privilege access, rate limiting, and runtime monitoring around a model.
Input/Output FilteringSandboxingRuntime Monitoring
08
AI Risk Management Frameworks
Identifying, measuring, and treating AI risk with recognised frameworks, risk registers, and impact assessments.
NIST AI RMFISO/IEC 42001Risk Register
09
AI Regulation & Compliance
Reading the rules that apply to AI systems: risk tiers, obligations for providers and deployers, data protection law, and sector requirements.
EU AI ActGDPRConformity Assessment
10
Governance, Privacy & Incident Response
Setting AI policy, documenting models, auditing for bias and safety, handling privacy, and responding when an AI system causes harm.
Model CardsAI AuditsIncident Response
What You Can Do After This Course
By the end of the curriculum you'll have worked through 5 hands-on projects and
covered the core of what an AI security, red team, or AI governance role expects.
Concretely, you'll be able to:
Map the AI risk landscape: explain the main categories of AI risk and where data, models, pipelines, and applications can be attacked.
Threat model AI systems: use trust boundaries, STRIDE, and MAESTRO to identify and prioritise risks before launch.
Attack models and LLM apps: demonstrate evasion, poisoning, prompt injection, and agent abuse in a controlled, authorised setting.
Run an AI red team engagement: scope it, automate testing, and deliver clear, reproducible findings with fixes.
Harden AI systems: secure the supply chain, add layered guardrails, and apply least privilege to models and agents.
Manage AI risk and compliance: apply NIST AI RMF and ISO/IEC 42001, classify systems under the EU AI Act, and keep evidence audit-ready.
Govern AI responsibly: write policy, document models, handle privacy, and respond to AI incidents when they happen.