Training Data Poisoning in Medical AI: Controls and Evidence

Part of our Medical device vulnerability, threat, and attack-technique catalogue. For the full overview, start with The Top 50 Cybersecurity Issues with Medical Devices.

An AI-enabled device that learns after clearance carries a risk that a fixed-function device does not: the thing being cleared can keep changing after it ships. If the data feeding that change is corrupted, the model drifts toward wrong answers without any code change to flag in a diff. A poisoned imaging classifier or dosing recommender does not crash. It keeps running and keeps producing confident, wrong output, which is worse for patient safety than an obvious failure. Reviewers now expect manufacturers to show exactly how they would catch that before it reaches a patient, not just describe the risk in prose.

Why This Matters

The FDA's final guidance Cybersecurity in Medical Devices: Quality Management System Considerations and Content of Premarket Submissions, issued February 3, 2026, is the current end of a chain that ran through the September 27, 2023 final guidance and the June 27, 2025 update, and it treats AI-enabled devices as software that carries the same Section 524B cybersecurity obligations as any other cyber device. It does not use the word poisoning, but its expectations for data integrity, threat modeling, and change control apply directly to how a model learns.

A device that retrains on field data has an attack surface that a static device does not: the data pipeline itself. AAMI CR515:2025 is the consensus report that names this attack surface explicitly, covering training data poisoning alongside adversarial evasion, model inversion, and pipeline compromise, and reviewers increasingly expect a threat model that reflects it rather than a generic STRIDE pass. Good Machine Learning Practice principles, jointly issued by the FDA and its counterparts, describe how to build and monitor a model responsibly, but they assume the data feeding it is trustworthy. Provenance and integrity controls are what make that assumption true.

What actually counts as data poisoning in a medical device?

Data poisoning is deliberate manipulation of the data a model learns from, so the model's behavior shifts in a way the attacker wants. It can happen at three points: in the original training set, in a vendor-supplied or licensed dataset, or in the stream of field data used for retraining after clearance. Each entry point needs a different control, because the trust boundary is different at each one.

A mislabeled subset of training images, a small number of shifted labels, or fabricated samples blended into a real dataset are all forms of the same attack. The device keeps functioning and keeps producing plausible-looking outputs, so the corruption is not obvious in a demo. It shows up later as a pattern of wrong classifications that nobody connects to a specific batch of training data until someone goes looking.

How does poisoning get in during retraining or continuous learning?

Retraining is the highest-risk path because it happens after the device is already in clinical use and the data source is field data rather than a curated dataset. A device that adapts to hospital-specific patterns, or that periodically retrains on aggregated field data, is trusting every contributing site and every device instance that fed that pipeline.

An attacker does not need to breach the device itself. Poisoning the data upstream, at a connected system, a labeling workflow, or a federated learning contributor, gets the same result with less effort. That is why the threat model for a continuously learning device has to extend past the device boundary to every system that touches the training data before it reaches the model.

How does this interact with a Predetermined Change Control Plan and GMLP?

A Predetermined Change Control Plan is the mechanism that lets a manufacturer make bounded changes to a cleared AI model without a new submission for every update. It has to state exactly what the retraining process is allowed to change, the data sources it is allowed to draw from, and the performance bounds that trigger a hold before a new model version goes live. A poisoning event that pushes the model outside those bounds should trip the same gate that a legitimate performance drift would trip.

Good Machine Learning Practice guides model development, training methodology, and performance monitoring, but it assumes the underlying data is what it claims to be. A PCCP that only checks output accuracy against a validation set can still pass a poisoned model if the poisoning is subtle enough to stay inside the accuracy tolerance while shifting behavior on a specific input class. The PCCP's monitoring plan should include data-level checks, not only output-level accuracy checks, so a change in the input distribution gets flagged even when aggregate accuracy still looks fine.

What does an AIBOM add that an SBOM does not cover?

An SBOM inventories software components: libraries, frameworks, and their versions. It does not tell a reviewer what data trained the model, what version of the model is deployed, or which third-party or hosted models the device calls at inference time. An AIBOM, built in CycloneDX ML-BOM or the SPDX 3.0 AI profile, closes that gap by inventorying the model artifacts, their weights and versions, the training and evaluation datasets, and any external model dependencies.

See also: Medical Device AI Performance Drift, FDA AI Cybersecurity Threats: 7 Attacks, and FDA & AI Pen Testing for Medical Devices.

That inventory matters directly for poisoning response. If a poisoning event is discovered in a specific dataset version, the AIBOM is what tells you which model versions used that dataset and need to be pulled or retrained. Our AIBOM guide walks through the fields reviewers expect and how to generate one from an existing training pipeline.

How do you verify model integrity and roll back a bad update?

Model integrity verification means cryptographically confirming that the model running on the device matches the version that passed validation, every time it loads or updates. A signed model artifact with hash verification at load time catches both a poisoned retrain that slipped through validation and a tampered model pushed through a compromised update channel. Without this check, a device has no way to know that what it is running is what was actually validated.

Rollback is the other half. If post-deployment monitoring flags a model version behaving outside its validated bounds, the device needs a tested path back to the last known-good version, not a manual patch process invented after the fact. That path should be exercised in verification testing before launch, the same way a firmware rollback would be, because the first time you need it should not be the first time you try it.

What evidence does a reviewer expect to see?

A reviewer expects to see the poisoning-relevant threat model, the data provenance controls for each training data source, the PCCP's monitoring and hold criteria, and a demonstrated rollback, not just a paragraph asserting that data governance exists. AAMI CR515:2025 is a consensus report rather than a certifiable standard, so you cannot claim conformity to it, but citing it as the method behind your ML threat model gives the reviewer a recognizable structure to evaluate against.

Our CR515 crosswalk tool maps CR515's threat catalog against the controls a submission needs to show, which is useful for confirming nothing in the poisoning threat model got left out before you submit.

How Blue Goat Cyber Approaches This

We build the ML-specific threat model before we look at the rest of the security architecture, because a device that learns after clearance has an attack surface a static device does not, and generic threat modeling misses it. Our engineers map every data source that feeds training or retraining as a trust boundary, then verify that the PCCP's monitoring criteria would actually catch a poisoning event rather than only a benign performance drift.

We build and validate the AIBOM alongside the SBOM, test the model integrity and rollback mechanisms as part of the same penetration test that covers the rest of the device, and document the result so the submission traces the threat to a control to a verified test outcome. If the FDA raises cybersecurity deficiencies after our submission, we resolve them at no additional cost.

Frequently Asked Questions

Is data poisoning different from an adversarial evasion attack?

Does a device that never retrains still need these controls?

Can a PCCP cover data provenance controls, or only model performance bounds?

What is the difference between AAMI CR515 and Good Machine Learning Practice?

Do I need a separate AIBOM if I already maintain an SBOM?

CTA

If your device retrains after clearance, the poisoning threat model and the PCCP monitoring plan need to work together, not sit in separate documents. Book a discovery session to walk through your training pipeline and submission scope.