On this page
Published: · Updated:
Key Takeaways
- ISO/TR 80002-2:2017 covers validation of quality system software (your toolchain), not medical device software. IEC 62304 and ISO 14971 cover the device.
- 21 CFR 820.10 and ISO 13485:2016 clause 4.1.6 is the binding requirement; ISO/TR 80002-2 is the informative method for meeting it without over-documenting low-risk tools.
- SBOM generators, SCA scanners, threat modeling tools, fuzzers, and signing pipelines may fall in scope depending on intended use in the quality system.
- Risk-tier each tool by what a silent failure would do to a submission or to patient safety, then scale evidence to that tier.
- The FDA's February 3, 2026 final Computer Software Assurance guidance recommends a risk-based approach for production and QMS software.
- Tool validation records live in QMS records, while the outputs those tools produce (SBOM, threat model, test reports) live in the design and development file.
What is ISO/TR 80002-2:2017?
ISO/TR 80002-2:2017 is a technical report on validating software used in a medical device quality system, not software in the device. It applies directly to cybersecurity because your SBOM generator, software composition analysis scanner, threat modeling platform, fuzzer, and code signing pipeline all produce regulatory evidence. If a tool silently fails, the submission artifact it generated is wrong and no one notices. ISO/TR 80002-2 gives you a risk-tiered, critical-thinking method for validating those tools proportionately, which is what 21 CFR 820.10 and ISO 13485:2016 clause 4.1.6 requires.
Most security teams have never heard of 80002-2, and most quality teams have never applied it to security tooling. That gap is where a specific class of deficiency lives: a submission SBOM that missed a whole dependency tree because the scanner was pointed at the wrong build artifact, or a vulnerability assessment built on a scanner whose feed had been silently stale for four months.
Why this matters
Section 524B of the FD&C Act requires a cyber device manufacturer to submit a software bill of materials and evidence of reasonable assurance that the device is cybersecure. The February 3, 2026 final premarket cybersecurity guidance sets out what that evidence looks like: a threat model, a security risk analysis, vulnerability testing including fuzzing and penetration testing, and an SBOM with vulnerability mapping.
These artifacts may depend on tool outputs as well as expert analysis. The reviewer is not reading your scanner's source code; they are reading its output and treating it as a factual claim about your device. When the tool is wrong, your factual claim is wrong.
Since February 2, 2026, the QMSR incorporates ISO 13485:2016. Clause 4.1.6 requires validation of software used in the quality management system before initial use and, as appropriate, after changes, with effort proportionate to risk. Determine cybersecurity tool scope from its intended use in the QMS, rather than assuming every security utility is in scope.
What ISO/TR 80002-2 actually is
| Attribute | Detail |
|---|---|
| Full title | Medical device software, Part 2: Validation of software for medical device quality systems |
| Edition and date | Edition 1, June 2017 |
| Document type | Technical report (informative). It sets no conformity requirements and produces no certificate |
| Scope | Software used in the quality system, including production equipment software and QMS tooling |
| Out of scope | Software in the medical device itself, which is IEC 62304 and ISO 14971 territory |
| Core idea | Critical thinking and risk-based effort rather than uniform, exhaustive scripted testing for every tool |
| Related binding rule | 21 CFR 820.10 and ISO 13485:2016 clause 4.1.6 in the United States, and QMSR alignment to ISO 13485 clause 4.1.6 |
21 CFR 820.10 and ISO 13485:2016 clause 4.1.6 is the obligation. ISO/TR 80002-2 is a method for meeting it. You cannot cite the technical report as conformity evidence, but you can and should cite it as the rationale for how much validation effort each tool received.
Which cybersecurity tools are in scope
A tool is in scope when its output becomes, or materially shapes, a record in your quality system or a submission deliverable. That test catches more of the security stack than most teams expect.
| Tool category | Typical examples | Output that becomes regulatory evidence | Failure mode if unvalidated |
|---|---|---|---|
| SBOM generation | Syft, cdxgen, build-integrated generators, commercial platforms | The submitted SBOM itself | Missing transitive dependencies, wrong versions, incomplete supplier fields |
| Software composition analysis | Black Duck, Snyk, Dependency-Track, OSV-based scanners | Vulnerability assessment and CVE mapping | Stale feed, unmatched components, false negatives on known CVEs |
| Threat modeling | Threat modeling platforms, structured STRIDE tooling, diagram-derived analysis | Threat model submitted under the 2026 guidance | Dropped data flows, trust boundaries silently unmodeled |
| Static and dynamic analysis | SAST, DAST, secret scanners | Testing evidence supporting reasonable assurance | Rules disabled by a config change, scans passing on zero files |
| Fuzzing | Protocol and interface fuzzers, coverage-guided harnesses | Fuzz testing report | Harness not reaching target code, crash triage dropping findings |
| Penetration test support | Scanners and exploit frameworks used inside a test | Penetration test report | Misreported coverage of interfaces |
| Build and signing | CI pipelines, signing services, artifact repositories | Integrity of the released binary and its provenance | Unsigned or wrongly signed artifact shipped as validated |
| Vulnerability intake and triage | Ticketing and PSIRT workflow tools | Postmarket monitoring and patch records | Reports lost, disclosure clocks missed |
| Document generation | Report exporters, template automation, spreadsheet macros | Submission documents themselves | Silent truncation, wrong data merged into a report |
Tools that only inform an engineer's judgment, without their output being carried into a record, sit at the low end of the scale. A local linter that never leaves a developer's machine is not the same as the scanner whose JSON is attached to the eSTAR.
Risk-tiering with critical thinking
ISO/TR 80002-2's central contribution is the instruction to think, then scale effort. The question is not "how do we test everything" but "what happens if this tool is wrong and no one catches it before the record is used?"
Three questions drive the tier:
- Impact. If the tool produces a wrong result, what is affected? Patient safety, product quality, or a regulatory record?
- Detectability. Would a normal downstream process catch the error? A human reviews the threat model diagram; nobody manually re-derives 900 SBOM entries.
- Frequency and change rate. How often is the tool used, and how often does it, its configuration, or its data feed change?
| Tier | Description | Typical security tools | Validation evidence expected |
|---|---|---|---|
| High | Wrong output could reach a submission or the field undetected and touches safety or a required 524B deliverable | SBOM generator, SCA scanner, code signing pipeline | Documented intended use, installation and configuration record, scripted test cases against a known-answer reference project, ongoing periodic re-verification, change control on version and feed updates |
| Medium | Wrong output shapes a deliverable but a competent reviewer would probably catch it | Threat modeling platform, SAST, fuzz harness framework, report generator | Intended use statement, vendor assessment, a limited set of test cases exercising the paths you rely on, revalidation on major version change |
| Low | Output is advisory, transient, or fully re-checked downstream | Local linters, exploratory scanners, developer utilities | Statement of intended use and rationale for low tier. Use justified assurance activities; scripted testing is not always needed |
The rationale is the deliverable. An auditor who disagrees with a low tier will ask why, and a written justification that engages with impact and detectability is a far stronger answer than a thin test script for every tool on the estate.
How to validate the high-tier tools
SBOM generators
The most useful technique is a known-answer reference build. Construct a small project whose dependency tree you have enumerated by hand, including at least one transitive dependency several layers deep, one vendored component that is not in a package manager, and one component with a known version-string quirk. Run the generator against it and compare output to the hand-derived truth.
Record what the tool did and did not find. Vendored and statically linked components are the classic blind spot, and documenting that limitation in your validation record is what lets you defend the manual supplement you add for those components. Re-run the reference build whenever the generator version changes, and confirm the output still parses as valid CycloneDX or SPDX at the version you submit.
Software composition analysis scanners
Two properties matter: matching accuracy and feed freshness. For matching, seed a test project with components that have well-known CVEs and confirm the scanner reports them. Include a component whose name collides across ecosystems, since that is where false matching shows up.
For freshness, capture the feed timestamp with every scan run and put a threshold on it. A scanner that silently stops updating produces clean reports, which looks like good news right up until a reviewer runs the same SBOM through their own tooling and finds what you missed.
Code signing and build pipelines
Validate that the artifact you tested is the artifact you shipped. That means verifying signature checks actually fail on tampered input, not just that they pass on good input. A negative test is the whole point; a signature verification step that returns success unconditionally will never be caught by positive testing.
Threat modeling tools
Where a tool auto-derives threats from a diagram, validate that a deliberately incomplete diagram produces the coverage gap you expect. Tools that silently ignore an unconnected element or an unlabeled trust boundary will give you a clean threat model for an incomplete system.
Where the FDA's Computer Software Assurance guidance fits
The FDA's Computer Software Assurance guidance (drafted in September 2022, finalized in September 2025, and updated in February 2026 for the QMSR) moves in the same direction as ISO/TR 80002-2: think about risk, spend assurance effort where failure matters, and accept unscripted and exploratory testing as legitimate evidence for lower-risk uses.
For security tooling, CSA is helpful in two concrete ways. It legitimizes ad hoc testing with a record of what was done, rather than requiring a formal protocol for every utility. And it treats vendor-supplied evidence as usable input, so a mature commercial scanner with published validation material lets you scope your own testing to the configuration and integration you added, rather than re-testing the vendor's product.
CSA does not lower the bar for the tools that produce submission deliverables. It raises it, by asking you to concentrate effort there instead of spreading it evenly.
Where the records live
A recurring source of confusion in audits is which file the paperwork belongs in.
| Record | Home | Reason |
|---|---|---|
| Tool intended use statement and risk tier rationale | QMS records | It is a quality system software validation record |
| Tool installation, configuration, and version history | QMS records | Same |
| Tool test cases and results | QMS records | Same |
| Tool revalidation on version or feed change | QMS records under change control | Same |
| The SBOM the tool produced | Design history file and the submission | It is a device design output |
| Vulnerability assessment and CVE mapping | Design history file and the submission | Device design output |
| Threat model | Design history file and the submission | Device design output |
| Penetration test and fuzz reports | Design history file and the submission | Device design output |
Put simply: how you know the tool works is a QMS record, and what the tool said about your device is a design and development record. Reviewers ask for the second. Auditors ask for the first.
Common gaps we find
- No tool inventory. Teams cannot list the tools that produced their submission artifacts, which makes tiering impossible.
- Feed staleness unmonitored. Nobody is watching whether the vulnerability database behind the scanner is current at the time of each scan.
- Version drift between test and release. The SBOM in the submission was generated by a different tool version than the one now in the pipeline, with no record of the change.
- Cloud tools treated as out of scope. A hosted scanner still needs an intended use statement and a rationale; the vendor's SOC 2 report is not a validation record for your configuration.
- Spreadsheets with macros. Traceability matrices and risk registers built in spreadsheets with formulas or macros are quality system software, and errors can go undetected.
- No negative tests. Everything is tested for the happy path, so a check that never fails looks identical to a check that always passes.
How Blue Goat approaches this
Blue Goat Cyber provides penetration and security testing and helps clients understand the scope and limitations of cybersecurity tool outputs. The manufacturer and its quality-system validation specialists remain responsible for tool validation, risk-tiering, and validation records. Blue Goat Cyber does not provide general software verification and validation or QMS tool-validation services.
FAQ
CTA
If you need security testing or help interpreting cybersecurity tool outputs, talk to a MedTech cybersecurity expert; coordinate tool validation with your quality-system specialists.
Related
Sources & references
Primary sources cited in this article. Links open in a new tab.
