Overcoming AI and Data Security Challenges in MedTech with May Lee

Hello and welcome back to another episode of the Med Device Cyber podcast. Today we're going to go over some of the global regulations and requirements as well as talk about some pretty interesting things as far as what the future of medical device cybersecurity looks like in a post quantum computing encryption breaking era. I'm your co-host Trevor Slattery joined by our co-host Christian Espinosa and we have a special guest from CS Life Sciences, May. How are you doing today, May? May: Really good. Thank you for the invite. Christian: No problem. Where are you coming from today, May? May: So, I am based in London, um, but I'm actually originally from Arizona. Um, so I grew up in Arizona, Chandler area. I know you guys are based in Tempe. Is that right? Christian: Yeah, I'm in Tempe looking at the Tempe Town Lake right now. Trevor is actually coming to us from Belize, I believe. I think he's at the lobster fest, eating lobster or something. May: Exciting. Jealous. Trevor: It's, it's been a hard week. May: But yeah. Yeah. Um, yeah, originally from Arizona. Um, I did my engineering degree at Arizona State. Um, and then went on to basically working for medical device companies. I've always been in medical device companies for the past, I think, 10, 11 years now. And, um, from startups to like big corporations, like Philips, uh, Cochlear for cochlear implants, and now I'm in consulting. Christian: Awesome. So, I know we have quite a bit to cover today, so we'll, uh, we'll kind of jump into it. Uh, I know, uh, well maybe you can describe a little bit your role with, uh, CS Life Sciences before we jump into our discussion. May: Yeah, definitely. So, my role at CS Life Sciences is quite unique in the sense that we work in the hardware, software team, um, but my specific expertise is related to AI, so artificial intelligence and machine learning. So we are seeing a lot of companies nowadays where they want to incorporate AI, they want to incorporate machine learning into their medical devices. So that's a lot of different competing regulations. So what I do is I help them kind of parse out the requirements that they need to meet, what are the absolute musts, what are nice to have, what are future considerations. So really, um, we work with any level of companies from startups out of a university spinout or, you know, a bigger corporation that just needs a little bit extra guidance, um, in terms of the technical support. Christian: Cool. Uh, so let's, um, jump into a little bit here. Like from your experience, how do you feel the industry is shifting in terms of cybersecurity and like quantum computing and just like in general? Because I, you, it sounds like you've been in the industry for a while. Like, how do you see things shifting? May: Yeah, so that's a really interesting topic and I think one of the things that people are more aware now is definitely, um, the cybersecurity aspect of compliance. So traditionally, of course, you know, there's hardware, software and you're worried about design control, you're worried about, you know, safety and performance of the device. Um, I think when I started in the industry, I don't think cybersecurity was quite that big of a topic, but now all I'm seeing is, you know, one of the first few questions clients ask us is, you know, what do we do? How do we, how do we make our device more secure? Um, and you're seeing that from the regulators as well, you know, they're coming out with more guidance, uh, guidances, more regulations, standards of secure by design essentially. So it's moving out of thinking about compliance maybe at a later stage or like post-launch security compliance, but now it's really weaving into, weaving the security requirements into design control itself, thinking about those security aspects right from the very start. Christian: You think the industry is actually shifting that direction? Uh, because, I don't know, what do you think, Trevor? I, from our perspective, it feels like maybe it's a shift, but it's like so slow I can't even recognize it. It's like a snail crawling across the sidewalk or something. I don't know. What do you think, Trevor? Trevor: It's pretty gradual for sure. I think that it's, there is a little bit of a shift, but like you said, it's pretty slow. We're seeing some companies try to come in at this earlier stage, but we're seeing still far too many companies, unfortunately, coming to us at the last minute saying, hey, we just weren't even aware this was something we needed to address. We just submitted to the FDA and they rejected us. What do we do now? We're lost. We're not sure where to go from here. And I feel like unfortunately that's still the majority of what we're seeing, but maybe it's down to 80% instead of 90% like it was in the past. May: Yeah, no, that's fair. I think, um, a lot of what I've seen in terms of the clients CS gets, I think they're more, very much like software development heavy, AI development heavy. And when you incorporate the AI side of things, you know, data, privacy protection, security related concerns might be more at the forefront of their mind. That could be it. Christian: Yeah, maybe so. I guess they're a little bit more on top of things if they're developing software with AI. They have to look a little bit more towards the future. And I know the FDA just finally finalized their cybersecurity guidance in, uh, I think it was June 27th. Is that right, Trevor? They think they... Trevor: Yeah. June 27, I believe. Christian: It came out with a, like they, they released it a day before and, uh, it was like future dated sort of. So with that guidance, um, what have you seen from your perspective, May? Are you actually seen it change anything or, I know it's only been, uh, you know, a few weeks or whatever, but what, what's your perspective on the, the new guidance? May: Yeah, I think traditionally with the FDA's, you know, cybersecurity requirements, it's quite burdensome, isn't it? I don't know if that's what you've seen, um, in your day-to-day as well. Um, the, the level of detail and the level of documentation that needs to be generated is really hefty. I think one of the important things that I've seen in the guidance is being able to, I guess, incorporate a lot of more total product lifecycle requirements, um, and being able, like I said earlier, to weave it into your design control, um, requirements. I think right now, one of the important things is like the updatability of the device, right? Um, being able to adapt, um, being able to be updated to, you know, meet the latest security requirements, plus address any out-coming or in-going, that was wrong. Upcoming, um, or foreseeable, non-foreseeable, uh, security risks and having kind of a really robust medication plan to handle all of that. Christian: Yeah, and what is your perspective, and I, I'll ask Trevor after this. Uh, from like the US requirements from the FDA versus the EU MDR? What, which one do you think is more stringent and kind of, which one has more regular, regulatory standards and just like, I guess your perspective on both of them, a comparison. May: Yeah, I think, um, it's, it's different in a sense where I feel like, from what I've seen at least, the FDA does a pretty good job at like spelling out, um, their expectation in what they want to see in a submission, for instance, um, or in a design of a document. I think with the EU outside of the EU MDR, um, of course, there's NIS2, there's the Cyber Security Act, that is more, I guess, um, generic in a sense where it doesn't just cover healthcare. So there's more complication in trying to translate a lot of the requirements to align with EU MDR requirements where, you know, the cybersecurity, um, kind of call out is in the GSPRs, so the general safety and performance, uh, requirements, you know, requiring state of the art cybersecurity and requiring software lifecycle controls. I think fundamentally the philosophy is the same, but I think the guidance we have might just be a different level. Christian: So it sounds like you think the FDA guidance has a little more clarity and the EU guidance is a little more, trying to put all these pieces together. Is that a decent summary? May: Yeah. Christian: What are your thoughts, Trevor? Trevor: Yeah, I totally agree. The FDA guidance is very mature from what we can see from a lot of different countries' regulations. EU MDR and EU cybersecurity regulations try to lean on certain other standards. Like we look at IEC 62304, 81001-5-1 as some of the main points of conversation for talking about how we're designing medical software safely and securely. While the FDA, of course, utilizes those same standards, they have a little bit more of a prescriptive approach in my mind than the MDR, then the European guidance, which is a bit more of a clear and I guess easy forward path for a medical device manufacturer. Having said that, I feel like the FDA requirements can be a little bit more burdensome in that light. Christian: And and what is, uh, I know we, we talked about FDA, EU, and I know China, uh, the NMPA. Are they, are these like the top three you think in terms of, uh, medical device, uh, regulations that need to be followed? May: Um, I'm going to give you like a very lawyerly answer and say it depends. It depends of course on like, you know, your business strategy, but I think, you know, a lot of countries are looking towards FDA and EU and China as, you know, that driver of regulation, of policy, um, of providing guidance, um, to what they should do within their own countries when developing regulations. Like, you know, with different new regulations like the EU AI Act, a lot of countries are looking to this regulation to frame their own regulatory framework. So I think it's very dependent, country dependent, but under the underlying theme is, you know, risk-based approach, ensuring appropriate, you know, management of the safety and performance of your device. Christian: And which one of these three are the most stringent? Uh, I'll throw that one to Trevor, I think, I think I know what your answer is going to be, but I, maybe you have changed your mind. Trevor: Typically, I think that the FDA is going to be the most strict as far as their requirements and the NMPA is going to be the most unique. If you're trying to submit every, into every regulatory authority in the world, you're going to have to change the most for NMPA just to comply with Chinese cyber law. And, let's say you're using top of the line, state-of-the-art, FIPS 140-3 encryption in America, you can't use that in China if you're handling sensitive information. You need to use cyber law approved encryption, which is going to be SM 2, 3, 4. And using those are not FIPS 140-3 approved, so they won't be accepted in America. So you're likely going to have to effectively make two separate devices for this. And, you know, this is usually the prime example that I bring up, but the Chinese NMPA regulations are pretty far detached from what we're seeing in the U. S. But I think in general, the U. S. has the most strict controls. Christian: Hmm. I was just in Hong Kong and, uh, Shenzhen, uh, China, I think last week. And, uh, one of the loopholes I, I was discussing with some people is if a device is FDA approved, you can bring it to Hong Kong, which is a special authorized region of China. And then once it's been in Hong Kong for a while, it can then be brought to China and get NMPA approved without having to go through the entire process. I don't know if you've heard that before, but there, there are some ways around this. And I think, uh, the FDA has set the bar as you said, but China has different encryption standards, so it's a whole different, um, ballgame there. But it, it's one of the challenges I think with a medical device manufacturer, if you don't understand your regulatory strategy and what markets you're trying to sell to, uh, it could be very cumbersome to try to meet all these different requirements later on if you haven't designed them in properly. Because I was talking to a manufacturer while I was over in Hong Kong and they wanted to sell to China, but they decided to put their, um, deployment on, I think it was Google Cloud platform, which I don't think is allowed in China. Uh, so they deployed like their software as a medical device in a cloud infrastructure that is not supported in China. So now they have to, like you said, Trevor, have to come up with two different versions of their product in order to sell to both markets. Trevor: And I know often times when I'm talking to our clients in their early phase, and they say, okay, lay out every market you're looking at going for. If they haven't decided on a cloud service provider, and they say, well, we want to use AWS, but we're looking at selling to China at some point. I say, all right, scrap everything, start over. You're moving to Azure now. That's the way you're going to get through this. May: Yeah. Yeah, I mean, that's, it's really interesting you brought up that loophole, Christian, because that is exactly what we did in a previous company that I worked for. We wanted to launch in China and that was the strategy that was the strategy recommended by our Chinese lawyers. That's the strategy to go to Hong Kong and then to China. Exactly. Um, and I think also, it's quite like culturally different as well. You know, the way, um, the Chinese businesses and the Chinese government conducts their business. I think there's that difference in culture that I find, you know, with my experience with Chinese regulators is very different to how you deal with, you know, FDA regulators or your notified bodies. There's almost a sense of like ceremony to it, if that's the right word. Christian: Yeah, I agree with that. Uh, when I was in, uh, Shanghai part of Shenzhen, uh, we went, did a tour of a hospital and it was interesting, we couldn't go to one of the floors because there was a celebrity there. So, I don't know if they would do the same thing in the United States, maybe so, I guess it depends on how popular the celebrity is, but I was talking to some people there as well and I guess there's a private hospital in the United States where only celebrities go because they actually have super high security and they don't want to leak, you know, what disease or what ailment a celebrity has. So it's interesting like, I guess if you have more money, you can pay for better security almost, uh, in our, in the United States healthcare system and also in other countries as well. I don't know how true that is, but they, they give the perception there's better security. Cool. Let's let's jump back to quantum computing because I, I we talked about different encryption with China versus the United States and with quantum computing, I know, Trevor, you were talking about future-proof encryption, but the, I thought the whole idea was there is no such thing as future-proof encryption with quantum computing threats. Um, maybe you can expand upon that a little bit, because I know in the past, like if you use AES 256, it's like DOD grade encryption, but the whole idea is if you harvest now, we can decrypt that later. So is it really going to be a type of encryption that will be future-proof or not really? Trevor: The same way that you can use quantum computers for decrypting, you can use them for encrypting as well. And it's fundamentally different from how we look at encryption right now. And again, I don't know the full details of exactly how it works, but essentially current encryption is based on you run it through a whole bunch of math which gives you a one-way function to get an output. And they compare the output of those functions. With quantum computing, you're running it through quantum encryption as well, which on a reg- on standard computing, you have any bit that you're assigning for these mathematical operations or for the output that can be zero or one. For quantum computing, it can be both at once. And so you're relying on physics as opposed to math, which I do not know the details at all. That's about as far as I'm going to go on the explanation, but it's a fundamentally different process. So the quantum encryption is not as easily broken by quantum computing, the same way that traditional encryption is not very easily broken by, um, traditional computing. Having said that, it doesn't work in reverse. So there isn't going to be a very easy way for us to have encryption from standard, like one-way hashing or one-way mathematical operations that can't be broken by quantum computing at a much higher rate. It's going to be a difficult problem to solve. Christian: So the solution is to use quantum computing for the encryption as well. And I don't, I don't know enough about it to know if it's more resource intensive than traditional encryption. Um, but yeah, it sounds like this is something we definitely need to keep an eye on and figure out a strategy for. And I think it's interesting, the Q-bit, uh, can be a zero and a one simultaneously versus just a bit. Trevor: And it's interesting with trying to break quantum encryption, it depends on if you're actively trying to break it. Since the whole concept behind quantum computing and quantum encryption is the visibility of it changes the state of it. So the visibility of encrypted information is going to alter whether or not it can be broken through that encryption. Christian: So we're going to have to develop these quantum-safe devices and quantum-safe environments to deploy them on, it sounds like, in the future. Trevor: The quantum safe devices will be the quantum devices, so we'll s- we'll see when we get there. I think it's... Christian: I think we're like, I think we're going to be like 20 years away based on how the legacy devices out there right now. May: Oh, yeah. Yeah, the legacy device issue is definitely a big concern. Yeah. Christian: Yeah, and it seems like at some point the line needs to be drawn to remove those legacy devices, but it's a bigger problem than, uh, just removing the device because you've got to retrain physicians on how to use a new device and you've got to deploy these new devices and test these new devices. So, you know, this is a much bigger challenge than, uh, a lot of people realize. Because I, I know a lot of people think, yeah, let's just say, by the end of this year, we're going to remove all legacy devices that are less than, you know, more than five years old from a hospital environment, but it's not that simple of a problem. So, in terms of total product life cycle, and we sort of started talking about this, uh, I know Trevor and I just released a podcast episode on the total product life cycle. Um, is there anything that, if I'm a manufacturer listening to this episode, that I should think about from a T- P- L- C perspective that may not be, I may not have heard of or something that would be a big tip for me? May: Yeah, I think one of the other things that we didn't quite touch on, which is equally as important is, you know, your supply chain, right? Like, what type of software components are you using? SOUP items, um, off the shelf software components, what are you using? And, you know, how much visibility you have on the developers, the tech, um, what are you actually integrating in a device that's under your responsibility? So, I think third party risk management is going to become more important, not that it's ever less important, but really like something to think about and highlight as you're developing, um, a software as a medical device, AI as a medical device. You know, you're taking, for instance, an L. L. M. model from a third party and you're integrating it into your device, you need to be, you're the one responsible for ensuring that it's performing in the way that you intended it to and that it's safe for use, it's secure, um, and all the other considerations in design. Christian: When do people typically come to you? May: Um, we have a range of, uh, inquiries actually. So, a lot of our clients, um, especially with like the software AI clients, they're quite, um, new. They're like university spin outs, right? It's someone's PhD project and suddenly it's like, you know what, we could do this as a company. Um, so that's when they reach out to us. So we work with a lot of, um, university like accelerator programs, um, to kind of get the word out there that we're here to help from a regulatory perspective. But we also work with more mature companies as well. Um, so if they're planning on introducing AI into their device, that is where we'll be like, you know, this is what you need to supplement in your technical documentation to meet the AI specific requirements. Christian: Okay, I think we advocate sooner than later. Uh, what, what is, um, the overall view, you think of our clients, Trevor, like when do the people typically come to us? I know we're trying to shift it to sooner by raising the awareness. Uh, do you think that that's working and when do people typically come to us from a cybersecurity lens? Trevor: I think it is getting better, but it's still a little bit of a slow progression. A lot of times we have people come to us, um, the worst time to come to us is after you've already submitted to the FDA, which is unfortunately a lot of what we see. They submit to the FDA, they weren't aware of the cybersecurity requirements, they get rejected. And redesigning a device with security in mind in a review cycle window is very difficult and very stressful. So coming to us as early as possible in the process, right when you're starting to design that software and before you get your hands on the keyboard and start writing any code, we want to talk about what standards are you writing the code to, how are you ensuring you aren't introducing any vulnerabilities through your coding practices, through your development practices, through your environment. I think that's the ideal time to come to us, but for the regulatory documentation and testing, I think the sweet spot is between six and four months out from the submission. That's when we want to prepare everything, do all the penetration testing. At that point, it's assumed we're going to have a close to final product and that's where we like to do the actual testing. Christian: Yeah, so it seems like we're both advocating from a regulatory perspective and a cybersecurity perspective, which cybersecurity kind of falls under regulatory, but it's also falls under design as well. Uh, the sooner they can, a client, um, a manufacturer reach out to us just for some consulting basically to get them on the right path, the better and the more cost effective typically. Uh, so we're kind of coming up on time here. So I'd like to go around and ask for last words of wisdom for our listeners. So, I I'll start with you, Trevor, and then I'll go over to May. Trevor: I'm going to go back to the classic here. Um, we want to see you considering cybersecurity early and often. It's not something that you should leave towards the end. Make sure that it's at the front of mind as you're going through the design process and don't leave it to the last minute. It's going to lead to cost overruns and time overruns. Christian: All right, early and often. The it's a t-shirt you want to get made, I know. All right, May, what about your last, uh, departing words of wisdom here? May: What really resonated with me within our conversation is the idea of, you know, reaching out early. Um, so before like Trevor said, you know, keep, fingers to keyboard, get yourself a good regulatory consultant, a technical consultant, someone that can really help you develop that roadmap. Um, and that is more cost effective. That will save you time, get you to market faster, and really remember that the regulations aren't here to stop you from getting to market, to bringing your ideas out into the world. They really are just to ensure safety and, um, effectiveness, right, of the device, of the solutions. Um, regulators do want these solutions. They want to see innovation. So working with them instead of against them will be so beneficial in the long run. Christian: Yeah, I agree with that. Uh, my parting words of wisdom are to begin with the end in mind. That's my new tagline, Trevor. Kind of like early and often, I like to say begin with the end in mind, figure out what your markets you want to go to ahead of time and then you can plan your strategy and work backwards from there. And you can also look at how I'm going to get reimbursed for this thing and how I'm actually going to make money because a lot of people don't think about that and they come up with a cool product, but there's no one, no adoption of the product because there's no reimbursement category for it. So there's a lot of things to think about, but you have to think about the end first and work backwards from there. So, thanks everyone for tuning into this episode. I hope you found it valuable and we hope to see you on the next one.