On this page
Key Takeaways
- ISO/TR 80002-2:2017 covers validation of quality system software (your toolchain), not medical device software. IEC 62304 and ISO 14971 cover the device.
- 21 CFR 820.70(i) is the binding requirement; ISO/TR 80002-2 is the informative method for meeting it without over-documenting low-risk tools.
- SBOM generators, SCA scanners, threat modeling tools, fuzzers, and signing pipelines all automate part of the quality system and fall in scope.
- Risk-tier each tool by what a silent failure would do to a submission or to patient safety, then scale evidence to that tier.
- The FDA's September 2022 Computer Software Assurance guidance aligns with the same critical-thinking model and supersedes checkbox-style validation thinking.
- Tool validation records live in QMS records, while the outputs those tools produce (SBOM, threat model, test reports) live in the design history file.
ISO/TR 80002-2:2017 is a technical report on validating software used in a medical device quality system, not software in the device. It applies directly to cybersecurity because your SBOM generator, software composition analysis scanner, threat modeling platform, fuzzer, and code signing pipeline all produce regulatory evidence. If a tool silently fails, the submission artifact it generated is wrong and no one notices. ISO/TR 80002-2 gives you a risk-tiered, critical-thinking method for validating those tools proportionately, which is what 21 CFR 820.70(i) requires.
Most security teams have never heard of 80002-2, and most quality teams have never applied it to security tooling. That gap is where a specific class of deficiency lives: a submission SBOM that missed a whole dependency tree because the scanner was pointed at the wrong build artifact, or a vulnerability assessment built on a scanner whose feed had been silently stale for four months.
Why this matters
Section 524B of the FD&C Act requires a cyber device manufacturer to submit a software bill of materials and evidence of reasonable assurance that the device is cybersecure. The February 3, 2026 final premarket cybersecurity guidance sets out what that evidence looks like: a threat model, a security risk analysis, vulnerability testing including fuzzing and penetration testing, and an SBOM with vulnerability mapping.
Every one of those artifacts is machine-produced. The reviewer is not reading your scanner's source code; they are reading its output and treating it as a factual claim about your device. When the tool is wrong, your factual claim is wrong.
Meanwhile 21 CFR 820.70(i) has said for decades that when computers or automated data processing systems are used as part of production or the quality system, the manufacturer must validate the software for its intended use. Security tooling is now unambiguously part of the quality system, because it generates the records that support a design control deliverable. The requirement has not changed; the surface it applies to has grown enormously since 2017.
What ISO/TR 80002-2 actually is
| Attribute | Detail |
|---|---|
| Full title | Medical device software, Part 2: Validation of software for medical device quality systems |
| Edition and date | Edition 1, June 2017 |
| Document type | Technical report (informative). It sets no conformity requirements and produces no certificate |
| Scope | Software used in the quality system, including production equipment software and QMS tooling |
| Out of scope | Software in the medical device itself, which is IEC 62304 and ISO 14971 territory |
| Core idea | Critical thinking and risk-based effort rather than uniform, exhaustive scripted testing for every tool |
| Related binding rule | 21 CFR 820.70(i) in the United States, and QMSR alignment to ISO 13485 clause 4.1.6 |
21 CFR 820.70(i) is the obligation. ISO/TR 80002-2 is a method for meeting it. You cannot cite the technical report as conformity evidence, but you can and should cite it as the rationale for how much validation effort each tool received.
Which cybersecurity tools are in scope
A tool is in scope when its output becomes, or materially shapes, a record in your quality system or a submission deliverable. That test catches more of the security stack than most teams expect.
| Tool category | Typical examples | Output that becomes regulatory evidence | Failure mode if unvalidated |
|---|---|---|---|
| SBOM generation | Syft, cdxgen, build-integrated generators, commercial platforms | The submitted SBOM itself | Missing transitive dependencies, wrong versions, incomplete supplier fields |
| Software composition analysis | Black Duck, Snyk, Dependency-Track, OSV-based scanners | Vulnerability assessment and CVE mapping | Stale feed, unmatched components, false negatives on known CVEs |
| Threat modeling | Threat modeling platforms, structured STRIDE tooling, diagram-derived analysis | Threat model submitted under the 2026 guidance | Dropped data flows, trust boundaries silently unmodeled |
| Static and dynamic analysis | SAST, DAST, secret scanners | Testing evidence supporting reasonable assurance | Rules disabled by a config change, scans passing on zero files |
| Fuzzing | Protocol and interface fuzzers, coverage-guided harnesses | Fuzz testing report | Harness not reaching target code, crash triage dropping findings |
| Penetration test support | Scanners and exploit frameworks used inside a test | Penetration test report | Misreported coverage of interfaces |
| Build and signing | CI pipelines, signing services, artifact repositories | Integrity of the released binary and its provenance | Unsigned or wrongly signed artifact shipped as validated |
| Vulnerability intake and triage | Ticketing and PSIRT workflow tools | Postmarket monitoring and patch records | Reports lost, disclosure clocks missed |
| Document generation | Report exporters, template automation, spreadsheet macros | Submission documents themselves | Silent truncation, wrong data merged into a report |
Tools that only inform an engineer's judgment, without their output being carried into a record, sit at the low end of the scale. A local linter that never leaves a developer's machine is not the same as the scanner whose JSON is attached to the eSTAR.
Risk-tiering with critical thinking
ISO/TR 80002-2's central contribution is the instruction to think, then scale effort. The question is not "how do we test everything" but "what happens if this tool is wrong and no one catches it before the record is used?"
Three questions drive the tier:
- Impact. If the tool produces a wrong result, what is affected? Patient safety, product quality, or a regulatory record?
- Detectability. Would a normal downstream process catch the error? A human reviews the threat model diagram; nobody manually re-derives 900 SBOM entries.
- Frequency and change rate. How often is the tool used, and how often does it, its configuration, or its data feed change?
| Tier | Description | Typical security tools | Validation evidence expected |
|---|---|---|---|
| High | Wrong output could reach a submission or the field undetected and touches safety or a required 524B deliverable | SBOM generator, SCA scanner, code signing pipeline | Documented intended use, installation and configuration record, scripted test cases against a known-answer reference project, ongoing periodic re-verification, change control on version and feed updates |
| Medium | Wrong output shapes a deliverable but a competent reviewer would probably catch it | Threat modeling platform, SAST, fuzz harness framework, report generator | Intended use statement, vendor assessment, a limited set of test cases exercising the paths you rely on, revalidation on major version change |
| Low | Output is advisory, transient, or fully re-checked downstream | Local linters, exploratory scanners, developer utilities | Statement of intended use and rationale for low tier. No scripted testing needed |
The rationale is the deliverable. An auditor who disagrees with a low tier will ask why, and a written justification that engages with impact and detectability is a far stronger answer than a thin test script for every tool on the estate.
How to validate the high-tier tools
SBOM generators
The most useful technique is a known-answer reference build. Construct a small project whose dependency tree you have enumerated by hand, including at least one transitive dependency several layers deep, one vendored component that is not in a package manager, and one component with a known version-string quirk. Run the generator against it and compare output to the hand-derived truth.
Record what the tool did and did not find. Vendored and statically linked components are the classic blind spot, and documenting that limitation in your validation record is what lets you defend the manual supplement you add for those components. Re-run the reference build whenever the generator version changes, and confirm the output still parses as valid CycloneDX or SPDX at the version you submit.
Software composition analysis scanners
Two properties matter: matching accuracy and feed freshness. For matching, seed a test project with components that have well-known CVEs and confirm the scanner reports them. Include a component whose name collides across ecosystems, since that is where false matching shows up.
For freshness, capture the feed timestamp with every scan run and put a threshold on it. A scanner that silently stops updating produces clean reports, which looks like good news right up until a reviewer runs the same SBOM through their own tooling and finds what you missed.
Code signing and build pipelines
Validate that the artifact you tested is the artifact you shipped. That means verifying signature checks actually fail on tampered input, not just that they pass on good input. A negative test is the whole point; a signature verification step that returns success unconditionally will never be caught by positive testing.
Threat modeling tools
Where a tool auto-derives threats from a diagram, validate that a deliberately incomplete diagram produces the coverage gap you expect. Tools that silently ignore an unconnected element or an unlabeled trust boundary will give you a clean threat model for an incomplete system.
Where the FDA's Computer Software Assurance guidance fits
The FDA's September 2022 final guidance on Computer Software Assurance for Production and Quality System Software moves in the same direction as ISO/TR 80002-2: think about risk, spend assurance effort where failure matters, and accept unscripted and exploratory testing as legitimate evidence for lower-risk uses.
For security tooling, CSA is helpful in two concrete ways. It legitimizes ad hoc testing with a record of what was done, rather than requiring a formal protocol for every utility. And it treats vendor-supplied evidence as usable input, so a mature commercial scanner with published validation material lets you scope your own testing to the configuration and integration you added, rather than re-testing the vendor's product.
CSA does not lower the bar for the tools that produce submission deliverables. It raises it, by asking you to concentrate effort there instead of spreading it evenly.
Where the records live
A recurring source of confusion in audits is which file the paperwork belongs in.
| Record | Home | Reason |
|---|---|---|
| Tool intended use statement and risk tier rationale | QMS records | It is a quality system software validation record |
| Tool installation, configuration, and version history | QMS records | Same |
| Tool test cases and results | QMS records | Same |
| Tool revalidation on version or feed change | QMS records under change control | Same |
| The SBOM the tool produced | Design history file and the submission | It is a device design output |
| Vulnerability assessment and CVE mapping | Design history file and the submission | Device design output |
| Threat model | Design history file and the submission | Device design output |
| Penetration test and fuzz reports | Design history file and the submission | Device design output |
Put simply: how you know the tool works is a QMS record, and what the tool said about your device is a DHF record. Reviewers ask for the second. Auditors ask for the first.
Common gaps we find
- No tool inventory. Teams cannot list the tools that produced their submission artifacts, which makes tiering impossible.
- Feed staleness unmonitored. Nobody is watching whether the vulnerability database behind the scanner is current at the time of each scan.
- Version drift between test and release. The SBOM in the submission was generated by a different tool version than the one now in the pipeline, with no record of the change.
- Cloud tools treated as out of scope. A hosted scanner still needs an intended use statement and a rationale; the vendor's SOC 2 report is not a validation record for your configuration.
- Spreadsheets with macros. Traceability matrices and risk registers built in spreadsheets with formulas or macros are quality system software, and they fail silently more often than any commercial tool.
- No negative tests. Everything is tested for the happy path, so a check that never fails looks identical to a check that always passes.
How Blue Goat approaches this
We start by inventorying every tool that touched a submission artifact, then tier each one against impact, detectability, and change rate rather than applying a uniform protocol. For the high tier, we build known-answer reference projects, run the client's actual toolchain against them, and document both what the tool found and what it structurally cannot find, because the documented limitation is what justifies the manual supplement alongside it.
We keep the validation record in the quality system and the tool output in the design history file, so an audit and a submission review each find what they expect. Where a client already has CSA-aligned procedures, we map the security tooling into those procedures rather than creating a parallel process. Where they do not, we scope the tiering so the effort lands on the three or four tools that actually generate 524B deliverables.
FAQ
CTA
If your SBOM, vulnerability assessment, and threat model come out of tools nobody has validated, we can inventory the toolchain, tier it, and build the validation evidence before an auditor or a reviewer asks. Talk to a MedTech cybersecurity expert.
