Listen now
Key takeaways
- Measures are direct, quantifiable attributes (e.g., time to patch), while metrics are calculations derived from measures (e.g., percentage of systems patched).
- The FDA requires manufacturers to report specific measures and metrics related to vulnerability management in premarket submissions.
- Key data points for the FDA include the percentage of identified vulnerabilities patched, the duration from identification to patch availability, and the duration from patch availability to deployment across devices.
- For new devices without a predicate, a comprehensive plan for post-market data collection is necessary, while historical data is expected for predicate devices or PMA annual reporting.
- A device's risk profile and security requirements vary based on its environment of use, such as a hospital network versus a home setting.
- Beyond vulnerability patching, measures like device downtime and recovery time are critical, especially for life-supporting or critical care systems.
- The goal extends beyond mere compliance; collected data must be actionable to effectively reduce risk and enhance patient safety throughout the device lifecycle.
- Meeting the FDA's minimum requirements should be viewed as a baseline, not the finish line, for robust medical device cybersecurity.
How do measures and metrics differ, and why is this distinction crucial for FDA submissions?
In this episode, Christian and Trevor demystify the difference between cybersecurity measures and metrics in the context of FDA guidance. They explore what the FDA expects in submissions, emphasizing patch timelines, vulnerability tracking, and post-market data collection. They also discuss the importance of actionability over mere compliance and include real-world challenges like device downtime and risk in different environments.
Key points:
(0:30) Measures vs Metrics Defined
- Measures are raw figures like time or count; metrics are calculated from measures.
(4:06) FDA Guidance and Patch Timelines
- FDA expects metrics like percentage of patched vulnerabilities and two patch-related durations.
(7:49) Real-Time Alerts
- Devices should notify users immediately of anomalies to compensate for lack of SOC monitoring.
(14:01) When to Include Metrics in Submissions
- Metrics aren’t always required during initial submission unless data is available.
(18:07) Downtime, Rebooting, and Risk Profiles
-
Reboot times and system recovery durations should be treated as key measures.
-
Risk profiles shift based on device use environment.
Notable quotes
“A measure is just like a tape measure, right? It's like a quantifiable attribute of something, like how long does it take to apply a patch? How many incidents occurred? That's a measure. Whereas a metric is some sort of calculation derived from a couple of measures, typically usually a percentage.”
“The FDA wants to see as one of the first points what your percentage of identified vulnerabilities that are updated or patched is.”
“Just because the FDA says these are the minimum things you should do, that doesn't mean that's all you should do, because I'm a firm believer that cybersecurity goes much further than just compliance or checking a box.”
“While this acts as the minimum baseline, it shouldn't be thought of as the finish line for how you're addressing security.”
Frequently asked questions
Bring this work to your device
Need help with fda premarket cybersecurity?
Blue Goat Cyber delivers fda premarket cybersecurity services for medical device manufacturers - from threat modeling to FDA-ready reports.
FDA Premarket Cybersecurity ServicesMore on FDA Premarket Cybersecurity
Keep listening
-
Episode 70
Why MedTech Needs Specialists with Zoltan Kevei and Saby Toth of Bishop & Co
With Zoltan Kevei
-
Episode 69
Science Before Hype in MedTech Investing with Varun Turlapati of Chaanakya Capital
With Varun Turlapati
-
Episode 67
De-Risking Product Decisions in MedTech Startups with Brent Lavin of Ironwood MedTech Partners
With Brent Lavin
-
Episode 65
Who Owns Patient Data Security in Trials with Rob Bedford, CEO of Franklyn Health
With Rob Bedford