FDA AI Cybersecurity Threats: 7 Attacks

Published June 29, 2026

Table of Contents

• Why This Matters
• At-a-Glance: 7 Threats → Test Types → Standards
• 1. Data Poisoning
• 2. Model Inversion and Model Stealing
• 3. Model Evasion
• 4. Data Leakage
• 5. Overfitting
• 6. Model Bias
• 7. Performance Drift
• Mapping the 7 Threats to Your SPDF and Threat Model
• How Blue Goat Cyber Approaches AI Threat Modeling
• FAQ
• Sources & References

Why This Matters

The FDA's final premarket cybersecurity guidance dated February 3, 2026 makes one point unmistakable: if your device meets the definition of a "cyber device" under section 524B(c) of the FD&C Act and incorporates AI, the AI is not a separate compliance track. It is part of your cybersecurity submission. The guidance lists seven categories of AI risk that "can be impacted by cybersecurity threats", and the FDA expects each one to be analyzed in your threat model, controlled in your Secure Product Development Framework, and verified before clearance.

This is a meaningful shift. Until recently, manufacturers could treat model quality as a data-science concern and cybersecurity as an IT concern. The 2026 guidance, read with AAMI CR34971 on AI risk management and the FDA's predetermined change control plan (PCCP) framework, fuses them. A reviewer can, and will, issue a cybersecurity deficiency for an AI-enabled device whose threat model does not address data poisoning or adversarial evasion. Standards including IEC 81001-5-1, AAMI TIR57 / ANSI/AAMI SW96:2023, AAMI TIR97, and ISO/IEC 27001 provide the surrounding controls, but it is the threat-modeling and SPDF evidence that the FDA reads first.

The seven threats below are taken directly from the guidance text. Each section explains what the threat is, why the FDA treats it as a cybersecurity issue (not only a model-quality issue), and what control evidence a reviewer expects to see.

At-a-Glance: 7 Threats → Test Types → Standards

This matrix is the fastest way to line up FDA's seven AI cyber threats with the tests you should run and the standards a reviewer will recognize. Every technique ID is from MITRE ATLAS; attack classes map to NIST AI 100-2 E2023, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations; practitioner test lists are from OWASP ML Top 10 and, for LLM-backed devices, OWASP LLM Top 10.

#	FDA threat	Test types to run	NIST AI 100-2 class	MITRE ATLAS	OWASP
1	Data Poisoning	Training-data provenance verification; signed-manifest negative tests; backdoor / trigger scans on imported weights; canary-sample detection; red-team poisoning injection against drift/outlier detector	Poisoning (targeted, backdoor, availability)	AML.T0020 Poison Training Data; AML.T0018 Backdoor ML Model	ML02 Data Poisoning; LLM03 Training Data Poisoning
2	Model Inversion & Stealing	Membership-inference tests; attribute-inference tests; model-extraction / surrogate-fidelity tests under a defined query budget; rate-limit and auth pen tests against inference endpoint; output-minimization verification	Privacy (membership inference, model extraction, attribute inference)	AML.T0024 Exfiltration via ML Inference API; AML.T0044 Full ML Model Access; AML.T0057 LLM Data Leakage	ML05 Model Theft; ML03 Model Inversion; LLM10 Model Theft; LLM06 Sensitive Information Disclosure
3	Model Evasion	Adversarial robustness testing with FGSM, PGD, AutoAttack, C&W, adversarial patches; modality-specific attacks (audio, waveform, tabular); OOD-detector validation; confidence-threshold fallback tests; certified-robustness bounds where warranted	Evasion (white-box, black-box, transferability)	AML.T0015 Evade ML Model; AML.T0043 Craft Adversarial Data	ML01 Input Manipulation Attack; LLM01 Prompt Injection (for LLM-backed devices)
4	Data Leakage	Authenticated + unauthenticated pipeline pen test (object storage, feature store, model registry, logs, caches); IAM and encryption config scans; inference-endpoint verbose-output tests; debug-log PHI-exfiltration tests	Privacy (data reconstruction, side-channel)	AML.T0035 ML Artifact Collection; AML.T0025 Exfiltration via Cyber Means	ML06 AI Supply Chain Attacks (storage/logs); LLM02 Insecure Output Handling; LLM06 Sensitive Information Disclosure
5	Overfitting	Held-out and external-cohort validation; train/val gap thresholds gated in CI; decision-boundary sharpness tests; adversarial-overfitting simulations after training-data refresh; reproducible-training pipeline verification	Poisoning (availability) + robustness generalization	AML.T0018 Backdoor ML Model (as exploit path); AML.T0031 Erode ML Model Integrity	ML04 Membership Inference (indirect indicator); ML08 Model Skewing
6	Model Bias	Stratified subgroup performance tests (sensitivity, specificity, calibration by demographic and clinical subgroup); disparity-threshold gates; backdoor / trojan scanning on imported checkpoints; ML-SBOM provenance verification; postmarket subgroup-drift monitoring validation	Poisoning (backdoor); abuse (bias amplification)	AML.T0019 Publish Poisoned Datasets; AML.T0010 ML Supply Chain Compromise	ML07 Transfer Learning Attack; ML08 Model Skewing; LLM05 Supply Chain
7	Performance Drift	Drift-detector validation against benign vs. adversarial shifts; reference-distribution regression tests; signed-update negative tests (unsigned rejected); rollback tests; postmarket telemetry-integrity tests	Availability / integrity (adversarial drift)	AML.T0031 Erode ML Model Integrity; AML.T0034 Cost Harvesting	ML09 Output Integrity Attack; LLM04 Model Denial of Service

GenAI / LLM-backed devices should additionally run: prompt-injection suites (direct and indirect), jailbreak suites, insecure-output-handling tests, and tool-abuse tests. Tooling: IBM Adversarial Robustness Toolbox (ART), Foolbox, CleverHans, Microsoft Counterfit, NVIDIA Garak for LLMs, Microsoft PyRIT for GenAI red-teaming, and Giskard for ML testing. Each threat's per-section table below drills into the specific SPDF artifact and submission evidence expected.

1. Data Poisoning

What it is. Cyber attackers deliberately inject inauthentic or maliciously modified records into training, fine-tuning, or feedback data so the model learns the wrong thing, for example, labeling malignant lesions as benign or shifting decision boundaries in a sepsis predictor.

Why it is a 524B issue. Data poisoning is not just bad data hygiene; it is an integrity attack on the model's training pipeline. The FDA treats the training pipeline as part of the device's attack surface when training artifacts or update channels affect the deployed model.

Mitigations.

• Cryptographically signed dataset manifests and per-record hashes; reject any training run whose corpus does not match the manifest.
• Role separation between data engineers, label reviewers, and model owners; least-privilege access to the training bucket and feature store.
• Statistical drift and outlier detection on inbound training and feedback data, gated before the next training run.
• A documented retrain/revert procedure with a known-good model baseline if poisoning is suspected post-deployment.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Training-data provenance policy + signed manifest schema	Manifest-signature validation test (negative case rejected)	Data-integrity test report referenced in eSTAR Security Architecture section
Secure implementation	Access-control matrix for training infra and feature store	IAM/least-privilege review + log review	Access-review record and IAM export attached to threat model
Secure verification	Statistical drift / outlier detector on inbound data	Red-team poisoning injection against detector	Adversarial-test report; STRIDE/ATLAS row "Tampering, AML.T0020 Poison Training Data"
Secure release & response	Retrain-from-known-good runbook	Tabletop on poisoning incident	Postmarket response plan + runbook version in submission

For a deeper treatment, see our dedicated post on AI data poisoning in medical devices.

2. Model Inversion and Model Stealing

What it is. An attacker queries the deployed model with crafted inputs to either (a) reconstruct sensitive attributes of the training data, model inversion, or (b) replicate the model's behavior in a surrogate, model stealing. Both attacks turn the inference endpoint into a side channel.

Why it is a 524B issue. Inversion can leak PHI used during training, which is a confidentiality breach under 524B. Stealing exposes proprietary weights and creates a parallel model an attacker can then attack offline at no cost, accelerating every other threat on this list.

Mitigations.

• Authentication, authorization, and per-user/per-device rate limiting on every inference endpoint.
• Output minimization: return the class or score the clinician needs, not full logits, embeddings, or attention maps.
• Differential-privacy or noise-injection techniques where training data is sensitive.
• Query-pattern anomaly detection for high-volume probing or adversarial query distributions.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Inference-API threat model entry (Information Disclosure)	Architectural review against ATLAS "AML.T0024 Exfiltration via ML Inference API"	Threat-model row + data-flow diagram
Secure implementation	AuthN/AuthZ, rate-limit, and output-minimization config	Endpoint pen test with stolen credentials and probing harness	Pen-test report (AI scope)
Secure verification	Model-stealing simulation (query budget, fidelity score)	Surrogate-model fidelity test below documented threshold	Verification report with fidelity metrics
Secure release & response	Inference-log retention + anomaly alert rules	Incident tabletop on detected probing campaign	Postmarket monitoring plan section

See Medical Device AI Model Inversion for the full breakdown.

3. Model Evasion

What it is. An attacker crafts or perturbs an input, often imperceptibly to a clinician, so the model misclassifies it. A pixel-level perturbation on a dermatology image, an adversarial patch on an ECG waveform, or carefully chosen noise in a radiology scan can flip a prediction.

Why it is a 524B issue. Evasion attacks degrade the reliability of clinical output, which the FDA treats as patient-safety-relevant. The 2026 guidance is explicit that cyber controls must protect the integrity of model predictions, not only the system around the model.

Mitigations.

• Adversarial robustness testing against published attack libraries (FGSM, PGD, patch attacks, modality-specific attacks).
• Input validation, range checks, and out-of-distribution (OOD) detection on raw inputs and intermediate features.
• Confidence thresholds with human-in-the-loop fallback when confidence is low or OOD detection fires.
• Adversarial training or certified-robustness techniques where the harm rating warrants it.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Adversarial-robustness requirement tied to hazard analysis (ISO 14971)	Threat-model row "Tampering, AML.T0015 Evade ML Model"	Hazard-to-control traceability table
Secure implementation	OOD detector + confidence-threshold logic in inference path	Unit + integration tests for OOD fallback	Code-review record + test results
Secure verification	Adversarial test plan (FGSM, PGD, patch, modality-specific)	Robustness benchmark vs. clean accuracy delta	Adversarial-test report appended to risk-based testing summary
Secure release & response	Field-collected misprediction triage workflow	Tabletop on adversarial input cluster in production	Postmarket surveillance plan

See AI Model Evasion in Medical Devices for examples and test patterns.

4. Data Leakage

What it is. Cyber threats exploit vulnerabilities to access sensitive training data, fine-tuning data, or inference-time data, for example, through misconfigured object storage, over-broad model endpoints, debug logs, or cached intermediate tensors.

Why it is a 524B issue. Data leakage is a direct confidentiality failure under 524B's expectations for protecting electronic information. It is also a regulatory issue under HIPAA and applicable international privacy regimes, but the FDA cybersecurity reviewer will judge it on 524B grounds.

Mitigations.

• Encryption in transit and at rest for every store that holds training, validation, or inference data.
• Strict separation of training infrastructure from production patient data; documented data-minimization at each stage.
• Tamper-evident access logging that captures who read which dataset and when.
• Pen testing that explicitly probes the data pipeline and the inference endpoint, not only the device firmware.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Data-flow diagram with classification labels (PHI, derived, model artifact)	Architecture review against 524B "protect electronic information"	Architecture section of submission
Secure implementation	Encryption + IAM config for every data store and bucket	Config scan and IAM review	Cryptographic-controls summary
Secure verification	Pipeline pen test (storage, logs, caches, inference)	Authenticated and unauthenticated tests with leakage findings	AI penetration testing report
Secure release & response	Incident-response runbook covering PHI exposure	Tabletop with disclosure and notification path	IR plan + breach-notification SOP

5. Overfitting

What it is. A model is trained too closely to its training set and generalizes poorly. A cyber attacker can deliberately push a model toward overfitting by injecting narrow patterns into training data, then exploit the brittle decision surface with adversarial inputs at inference time.

Why it is a 524B issue. Overfitting is normally a model-quality concern, but the FDA includes it here because attackers can cause it deliberately and then exploit it. That makes overfitting an indicator of compromise as well as a quality defect.

Mitigations.

• Held-out validation across demographically and clinically diverse cohorts; cross-validation results in the submission.
• Train/validation gap and decision-boundary sharpness treated as security-relevant alerts, not only data-science metrics.
• Regularization, dropout, and ensemble strategies documented in the model card.
• Re-evaluation of overfitting indicators after every training-data refresh or PCCP-authorized update.

See also: Infusion Pump Cybersecurity: FDA, Medical Device Pen Testing: FDA vs EU MDR 2026, and Mining FDA Databases for Cybersecurity Precedent.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Generalization requirement linked to clinical performance acceptance criteria	Review against intended-use population	Performance requirements traceability
Secure implementation	Regularization + ensemble configuration captured in model card	Reproducible training pipeline test	Model card + training-pipeline artifact
Secure verification	Subgroup + external-dataset validation; train/val gap threshold	Validation report with diverse cohorts	Clinical/non-clinical performance section
Secure release & response	Overfitting-alert rule tied to retraining triggers	Tabletop on suspected adversarial overfitting	Postmarket monitoring plan

See AI Overfitting Cybersecurity Threats in Medical Devices for detection patterns.

6. Model Bias

What it is. Bias enters when training data under-represents populations, when labels carry historical inequities, or when an attacker deliberately skews fine-tuning data, embeds backdoors, or distributes pre-trained models that already carry bias to be amplified downstream.

Why it is a 524B issue. The 2026 guidance treats deliberate bias manipulation, including backdoor triggers and adversarial bias amplification, as a cybersecurity attack on the model. Reviewers expect bias monitoring to be integrated with security monitoring, not run as a separate fairness exercise.

Mitigations.

• Provenance and integrity of any third-party datasets or pre-trained checkpoints; SBOM-style accounting for ML artifacts.
• Subgroup performance reporting (sensitivity, specificity, calibration by demographic and clinical subgroup) in the premarket submission.
• Backdoor scanning on imported model weights and a procedure for re-training if a backdoor is found.
• Postmarket bias monitoring tied to the device's postmarket cybersecurity program.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	ML-SBOM listing datasets, checkpoints, licenses, versions	Review of artifact provenance against SBOM/supply chain requirements	ML-SBOM in submission
Secure implementation	Backdoor / trojan scan in CI on imported weights	Negative test with planted trigger	CI scan report
Secure verification	Subgroup performance + fairness metrics	Subgroup test report with predefined thresholds	Performance section of submission
Secure release & response	Bias-monitoring rules in postmarket telemetry	Tabletop on detected subgroup drift	Postmarket plan

7. Performance Drift

What it is. Cyber attackers shift the input data distribution over time, through compromised upstream sensors, manipulated EHR fields, or slow injection of skewed feedback, so the model's accuracy decays. Drift can also arise naturally; the threat is that an attacker can cause it on purpose and hide it as ordinary degradation.

Why it is a 524B issue. Performance drift connects directly to postmarket cybersecurity obligations. The FDA expects manufacturers to detect drift, decide whether it is benign or adversarial, and respond within their PCCP or change-control process.

Mitigations.

• Continuous performance monitoring against a held-out reference distribution; alarm thresholds documented in the SPDF.
• Distinction between data drift, concept drift, and adversarial drift, with investigation playbooks for each.
• Integrity controls on the update channel so legitimate retraining cannot be hijacked.
• Reporting paths into the FDA when drift indicates a cybersecurity issue affecting safety or effectiveness.

SPDF control / test mapping.

SPDF activity	Control artifact	Verification test	Submission evidence
Secure design	Drift-monitoring requirement + reference distribution definition	Architectural review of telemetry path	Postmarket monitoring architecture
Secure implementation	Signed update channel + rollback control	Negative test: unsigned update rejected	CI/CD security gates evidence
Secure verification	Drift-detector validation against benign and adversarial shifts	Adversarial-drift simulation	Verification report
Secure release & response	Drift triage playbook + FDA reporting decision tree	Tabletop covering benign vs. adversarial drift	Postmarket plan + reporting SOP

See AI Performance Drift in Medical Devices for the monitoring architecture.

Mapping the 7 Threats to Your SPDF and Threat Model

For a 524B submission, each of the seven threats above should appear in three places in your documentation:

1. Threat model. Use STRIDE or a STRIDE-plus-MITRE-ATLAS overlay so AI-specific threats sit alongside conventional ones. ATLAS gives you the adversarial ML tactics and techniques; STRIDE keeps the framing consistent with the rest of the device. See our step-by-step threat modeling guide.
2. SPDF risk and control matrix. Each threat ties to one or more controls, each control ties to one or more verification activities, and each verification activity produces an artifact that goes into the submission.
3. Postmarket monitoring plan. Drift, bias, and anomalous query patterns need monitoring rules and response playbooks, not just dashboards.

When a reviewer asks "how does your device address AI cybersecurity threats," the answer is not a paragraph. It is a traceable chain from threat to control to test to monitoring rule, for all seven categories.

Need help? Our team supports manufacturers with FDA cybersecurity submissions end-to-end. Explore our medical device cybersecurity services or book a discovery call.

How Blue Goat Cyber Approaches AI Threat Modeling

Our ThreatGoat workflow treats the seven FDA-listed AI threats as first-class entries in the threat model, not appendices. We map each one to STRIDE plus the relevant MITRE ATLAS techniques, generate the control list and test plan from that mapping, and produce the SPDF artifacts a 524B reviewer expects to see. Our AI penetration testing service then exercises the controls against the actual deployed model, so the evidence in your submission reflects how the system behaves under attack, not only how it is designed.

If you are preparing a 510(k), De Novo, or PMA for an AI-enabled device, the fastest path through a clean cybersecurity review is a threat model and SPDF that already speak the language of the Feb 3, 2026 guidance.

FAQ

Does the 2026 guidance apply if my device only uses a locked AI model?

Do I need ATLAS in my submission, or is STRIDE enough?

How does this interact with a Predetermined Change Control Plan (PCCP)?

Are these threats only relevant to imaging or diagnostic AI?

Where do AAMI CR34971 and IEC 81001-5-1 fit?

Related Deep-Dives on the 7 AI Threats

Each of the threats above has a dedicated post with detection patterns, controls, and test evidence:

• Data Poisoning, Safeguarding Medical Device AI from Data Poisoning
• Model Inversion & Stealing, Medical Device AI Model Inversion: Threats and Solutions
• Model Evasion, Protecting Medical Devices from AI Model Evasion
• Data Leakage, AI Penetration Testing for Medical Devices (covers data-pipeline leakage paths)
• Overfitting, AI Overfitting Cybersecurity Threats in Medical Devices
• Model Bias, How to Respond to an FDA Cybersecurity AI Request (includes bias-monitoring evidence the FDA expects)
• Performance Drift, Understanding AI Performance Drift in Medical Devices

Supporting reading: Threat Modeling Connected and Implantable Devices · FDA Section 524B Subsections Explained

Sources & References

FDA and statute

• FDA, Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions (Final Guidance, February 3, 2026).
• FDA, Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations (Draft, January 2025).
• FDA, Predetermined Change Control Plans for Machine Learning-Enabled Device Software Functions (Final, December 2024).
• Section 524B of the Federal Food, Drug, and Cosmetic Act (added by the Consolidated Appropriations Act, 2023).

AI-attack taxonomies and testing frameworks

• NIST, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2023).
• NIST, AI Risk Management Framework (AI RMF 1.0) and the Generative AI Profile (NIST AI 600-1).
• MITRE ATLAS (Adversarial Threat Landscape for AI Systems) - tactic and technique catalog cited in the matrix above.
• OWASP Machine Learning Security Top 10.
• OWASP Top 10 for Large Language Model Applications.

Medical-device standards

• AAMI CR34971:2023, Application of ISO 14971 to Machine Learning in Artificial Intelligence. See our CR34971 explainer.
• AAMI TIR57:2016 (R2023), Principles for medical device security - Risk management.
• ANSI/AAMI SW96:2023, Standard for medical device security - Security risk management for device manufacturers.
• IEC 81001-5-1:2021, Health software and health IT systems safety, effectiveness and security - Part 5-1: Security - Activities in the product life cycle.
• ISO 14971:2019, Medical devices - Application of risk management to medical devices.
• ISO/IEC 23894:2023, Artificial intelligence - Guidance on risk management.
• ISO/IEC 27090 (in development), Cybersecurity - Artificial Intelligence - Guidance for addressing security threats to artificial intelligence systems.

Testing tooling referenced above

• IBM Adversarial Robustness Toolbox (ART), Foolbox, CleverHans, Microsoft Counterfit, NVIDIA Garak, Microsoft PyRIT, Giskard.

---

Need help building an SPDF and threat model that covers all seven of these AI threats? Talk to Blue Goat Cyber about a 524B-ready AI cybersecurity engagement, or start with our ThreatGoat threat-modeling workflow.

About the author, The Blue Goat Cyber team specializes in cybersecurity for FDA-regulated medical devices, including AI/ML-enabled systems subject to section 524B of the FD&C Act. We help manufacturers ship clean premarket submissions and run defensible postmarket programs.