242: Are Foundation Models Really Better for Digital Pathology? Podcast with Panu Kauppila

What good is a powerful foundation model if it slows the pathologist down, can’t explain its result, or doesn’t fit the clinical workflow?

Foundation models are gaining attention across digital pathology. But they’re not finished clinical tools by themselves.

In this episode of the Digital Pathology Podcast, I speak with Panu Kauppila, Chief Product Officer at Aiforia, about what foundation models are, how they differ from traditional convolutional neural networks, and what it takes to make them useful for pathologists.

Panu describes a foundation model as a large, context-aware building block. To perform a specific pathology task - such as grading, segmentation, or mitotic counting - it must be combined with an adapter, a task-specific head, curated annotations, a usable interface, and integration with the laboratory workflow.

We also discuss one of the biggest practical constraints: speed.

A pathologist shouldn’t have to click a button and wait for an analysis. Panu explains why AI should run automatically in the background so the results are already available when the case reaches the pathologist’s worklist.

The conversation also examines the tradeoff between model size, computational cost, and clinical performance. Larger models may improve robustness and generalizability, but they can also require more processing power. For clinical applications, Aiforia focuses on smaller and medium-sized foundation models that provide the necessary quality without making the workflow slower or unnecessarily expensive.

Annotated data remains central. The foundation model supplies the underlying image understanding, while the task-specific head and controlled annotations determine how the model performs on a particular pathology problem. This structure also raises important questions about bias, data provenance, ownership, regulatory documentation, and explainability.

Finally, we look at multimodal AI models that combine pathology images with reports, genomic data, molecular information, and clinical outcomes. These tools could support more interactive, predictive, and prognostic applications - but only if they’re introduced through secure, controlled workflows with clear audit trails.

Episode Highlights

• 00:00 — Where does bias enter a foundation model workflow?
  Panu distinguishes the underlying foundation model from the annotated dataset used to build the task-specific application.
• 00:27 — Meet Panu Kauppila
  An introduction to Aiforia’s Chief Product Officer and the episode’s focus on foundation models in digital pathology.
• 01:05 — From radiology AI to digital pathology
  Panu describes his background in medical device development, radiology, oncology solutions, and clinical AI implementation.
• 06:29 — Foundation models versus convolutional neural networks
  What makes foundation models more context-aware, robust, and generalizable across image datasets.
• 07:07 — Image-only and multimodal foundation models
  Why these two categories offer different capabilities and potential clinical uses.
• 07:50 — A foundation model is a platform, not a finished solution
  The underlying model may understand image features, but it still needs a task, interface, and clinical workflow.
• 08:34 — Foundation model, adapter, and task-specific head
  How these components work together to create an application for grading, mitotic counting, or another pathology task.
• 09:57 — The cost of larger models
  Why increased robustness must be balanced against computational demands, inference speed, and affordability.
• 11:12 — Pathologists won’t wait for AI
  Why even short delays can interrupt the clinical workflow.
• 11:46 — Running AI in the background
  A workflow in which slides are scanned, analyzed automatically, and added to the worklist with results ready for review.
• 12:16 — Combining foundation models with curated annotations
  How smaller task-specific datasets and adapter technology can produce practical pathology models.
• 15:24 — Generalizability across scanners, laboratories, and populations
  How foundation models may make adaptation to new domains more manageable.
• 16:36 — Two datasets, two sources of potential bias
  The regulatory questions created by an underlying foundation model and a separate controlled annotated dataset.
• 20:46 — Making foundation models accessible
  Why a platform and user interface are necessary for pathologists and researchers who don’t work directly with code.
• 21:57 — Testing foundation models in Aiforia Create
  How researchers can compare a CNN with supported foundation models in the same no-code environment.
• 25:45 — How foundation models are selected
  Quality, licensing, model size, annotated data, and the requirements of the intended use.
• 26:57 — Training cost versus inference cost
  Why a more expensive training iteration may still reduce total development costs if fewer iterations are needed.
• 32:23 — Pathologists are visual reviewers
  The importance of segmentation quality and showing exactly what the model identified.
• 36:26 — The potential of multimodal AI
  Combining pathology images with text, genomic information, molecular data, and clinical outcomes.
• 37:35 — Keeping multimodal AI inside a controlled environment
  Privacy, security, regulatory oversight, and the risks of moving clinical information into consumer AI tools.
• 43:24 — Explainability in clinical pathology AI
  Using semantic segmentation, object detection, instance segmentation, and annotated ground truth to show how results were calculated.
• 47:43 — Moving toward predictive and prognostic models
  How established digital workflows could allow pathologists to contribute more information about likely outcomes.
• 49:45 — Research and clinical collaboration with Aiforia
  How interested researchers and laboratories can connect through the Aiforia website.

Resources Mentioned

• Aiforia
• Aiforia Create no-code model-development environment
• PathChat

Listen to the full discussion to understand what foundation models can add to digital pathology—and what still has to happen before they become practical, trusted clinical tools.

Transcript

TRANSCRIPT:

00:00:00

 the interesting [music] question of bias. What is your data made of? So now you're going to have two data sets. You have the annotated [music] data set and then you have the foundation model. So where do you tackle your bias question? I mean technically with the foundation model, we're going to be robust generalized. [music] As of now, I'm saying the annotated data set that we [music] own, we control, we know it precisely, we have all the rights to it. That's what I'm going to explain to my

00:00:27

 regulatory body. I'm going to say [music] this is at works. The underlying foundation model for me is more like a library. It's like a platform [music] underneath but I expect to have a lot of interesting discussions around that topic. Welcome my digital pathology trailblazers. Today my guest is Panu Copila. He's the chief product officer at Aiforia and we're going to be talking about something pretty popular in the digital pathology space. There was some coverage on this on the podcast already. We're going to

00:01:05

 be talking about foundation models. Welcome Panu to the show. How are you today? >> I'm doing fine, thank you Alex. Thank you for having me. >> I'm super excited because I want to hear from an expert, technical expert on this topic. But before we dive in, we always start with the guest. Can you tell the digital pathology trailblazers about you a little bit? >> Okay. Yeah. So, uh, I was, uh, I'm a medical device sort of development professional. That's what I've been for

00:01:41

 quite a long time. And I ended up you know ended up with digital pathology through the previous company which was Phillips where a lot of people got >> in famous space >> but my path was years years ago I started writing algorithms for radiology for the longest part of my career was actually radiology MR devices then you got towards uh multimodal oncology solutions. Then I ended up being at Phillips the head of R&D for what we called oncology solutions. And there as part of that

00:02:25

 then we had digital pathology. I was this was a while ago this eight years ago or so I got into digital pathology for a while. I was leading the R&D at at Phillips uh at the time when they uh we were squeezing out the the second generation scanner. So that's where I got involved heavily on what's happening there. And then as a continent for that, I ended up at Aphoria because you know I'm sitting in the headquarters of Aphoria in Helsinki and this is where I was born. This is where I live and I

00:03:02

 was so thrilled with through my times with Phillips on on what's happening here and a bit of perspective on the on the AI models and techniques and that for me all this started to come alive around 2014 before digital pathology. We worked on we worked on uh how to leverage uh MR images in radi the radiation therapy planning and we built models and 2014 was the moment when for the first time we said oh it looks like convolutional neural networks are going to be the way of doing this and we actually squeezed

00:03:47

 out the first diagnostic product for uh for radiation therapy planning uh 2015 we got through the FDA with an AI model diagnostic and this was uh or helping diagnosis and and planning. This was 200 16. So that's 10 years ago. So for in my world 10 years ago is when seriously things started happening and then we've been on that uh convolutional neural network journey in a lot of modalities for a long time and then things started changing and now where we are we are with vision transformers and we are in

00:04:34

 foundation models and figuring out where does all this go and where I'm sitting at Boria now is is uh yeah okay we we transitioned to this technology uh a while ago uh and now everything we do is basically vision transformerbased be it for the research customer or be it for the clinical workflow but that's my path how I ended up here but I have this big bag of experience of building medical device products for different modalities the whole thing about regulatory about you know customer requirements,

00:05:14

 releasing, maintenance, configuration, all that. That is the same things that we do now for digital pathology. And really when I came here, that was four years ago to Euphoria. My primary mission has been that we put these things into clinical practice seriously at scale. Uh make things accessible for the pathologist, simple workflows. That's that's the thing I do. And then I speak with our pathologist. I speak with the bioscientist and the software people and UX people, the regulator people,

00:05:50

 everybody. So that's my job like figure out which models could help the pathologist and how do you actually make it available for them in a nice way. So that's that's what I do. Mhm. So you say 2016 that was more or less when I joined the digital pathology space and that like you say that coincided with the introduction of convolutional neural networks and I I was like when I was beginning it was the transition from handcrafted features to annotating different uh subanotical structures as

00:06:29

 examples and I was like wow this is going to be the next new thing it's fantastic and now I feel and then you learn about it you learn about limit the potential and the limitations and now I feel kind of similarly to the foundation models so let's take a step back what are they and how do they differ from these convolutional neural networks what are they good for what are they not good for >> first is this thing about uh how broad and lose those that definition is which I think believe you said

00:07:07

 somewhere >> yeah of course people speak about they speak about foundation models and they it can be a lot of things in my head first we split it into are we talking about image only or are we talking about multimodal and I think that's two diff quite different baths uh different promises although the underlying technology might be the name uh how you how you create embedded uh out of uh out of a context be it images or be sentences. So our work uh for now is primarily focused on the image only models

00:07:50

 take virgamma or whatever. So image only and then figure out how we can make models better for the users and there for me first foundation model is just a big model. It it's just a big big model >> u and it does for us it it's more like a platform component. It's like a building block. It's it's a it's a big big model that understands features but it doesn't really it doesn't have a UI. It doesn't know what what is what and then then you come to this question of uh well how do you make

00:08:34

 use of it and then we have this concepts of here's a foundation model then is a thing we call an adapter and then there's a thing we call a head and the head >> plus adapter plus foundation model constitutes the task specific model that helps you with grading or breast grading or mitoic counting or whatever the task is. Uh so that's the the approach we have as of right now for this topic. Then we have the vast topic of uh let's have multimodal models first text and and uh pathology images and then you can even

00:09:20

 combine uh gimmonic genomic uh and molecular information. So then that opens up a totally different set of use cases which to me is more like okay now we're going to have something where I could in a workflow discuss with a model I could ask questions I could get the task specific uh calculation of cells or or whatever so if we now stick first to the image only because that's in the heart of what we do today. So in that space foundation model well people have big discussions about what is how big do

00:09:57

 you have to be to be a foundation model 300 million parameters is what some people throw in the game for me it's just big and with big comes the good side of it which is well it knows a lot uh it understands the images as a whole so which is a different thing from how convolutional neural networks work and and it can really bring you a lot of robustness and generalizability over data sets. >> The other thing is it is big. Big means costly, heavy, uh potentially slow. So what we've been focusing on is okay, so

00:10:40

 how do you make this cost effective? Because I've been telling my team for for several years now that you know forget don't don't bring this to me if analysis becomes three times five times 10 times 20 times slower in the first tries we had a few years back and we we became like 30 times slower 50 times slower. I said forget it. I mean this is costing money. You can't I mean you can't do this. >> And >> I love your attitude. Don't even bring it to me if it's slower. I think that

00:11:12

 resonates with the pathologists a lot because if you say that anything is going to be slower than the manual or than what they knew before, it's a no-go. >> Yeah. and and then you come of course you know the way we when [clears throat] we put together workflow for clinical practice we always I always say what happens is put the slides in the scanner uh get them scanned less than a minute put them in automatically for an analysis and then just put an indication to the pathologist that the analysis is

00:11:46

 done. Never have the pathologist wait for, you know, click a button and wait because because because I've learned that if it takes more than a second, your pathologist will be pretty upset. So don't even dream about, you know, having them wait for 15 seconds. They won't. Uh so that's why all the workflows are designed as put it in the background, get it analyzed, get it on your worklist, and says results ready, and then you can do okay with this. You could say, well, okay, I can run a heavy

00:12:16

 foundation memo, but the thing is if they are a billion parameters, they are they're going to be super expensive and slow to run. So, what we've done is we did some innovation on the adapting part. So how we interface with those models so that we can kind of extract the essential out of the foundation model and we train together with our annotated data set. So we put together limited size uh image repository with annotations and this this is not going to be millions this is going to be thousands tens of thousands uh kind of

00:13:00

 numbers and you combine that with smart adapting technology with the foundation model and then you're going to get an outcome which basically for the pathologist looks exactly like what you had with a convolutional neuronet network. It's just that the underlying technology is different. Now the models will perform better and for me when when our R&D guys started to say okay now we are like twice as you know we take twice as much resources with the convolutional neuronet networks. That's when I said

00:13:38

 okay this is now it sounds good. Let's put this into into real world uh practice because you because we do have better processors. I mean the technology does evolve. So you can you can put a little bit more load there but if you're 10 times slower more more GPU uh needed then u it won't work. And essentially what we can do now is um we we don't use the billion parameter versions of anything. we we just don't uh we we're looking at 100 million parameters and below that kind of models

00:14:14

 and for comparison if we have convolution neural networks the ballpark was more like 10 million parameters or something like this so so we're getting to the same ballpark and I can accept that it's a little bit uh little bit more uh computational power needed but but with this it's manageable and because because I can't go to uh pathologist and say well we're going to charge you twice as much as you know because even getting any you know getting your AI analysis into the budgets of the pathology department in

00:14:50

 itself is a challenge because you got to show the productivity and you got to show the health the outcomes improving which is more tricky but you have to show the productivity. So then you come to the question of well what does it look like? Uh well we're going to have come to come to that in the in the future but but this is our philosophy about you know how to get these foundation models image only foundation models into the hands of the pathologist. It would say well we just plug them in into our platform

00:15:24

 underlying platform. We're going to we're still going to be able to show you the ground truth of the annotated data. We can still we can even go to a let's say we go to a new country completely. Let's imagine we go to um somewhere where uh let's say Japan where you know the racial distribution is kind of a bit extreme and you would ask well I need to have data from this country dominating the models. I can do it pretty easily. We we just combine these two worlds to each other and and keep it

00:15:59

 effective. And then the generalizability comes from there. And this is our experience. And we get we get high quality faster. We get robust models faster. And it's basically delivering to the promise that's been there for 10 years. I mean, it's uh but people have learned the the hard way in a lot of labs where you were the front runners of started to use algorithms. everything was smooth and then oh whoops we buy a new scanner and oh whoops this doesn't work anymore. So >> I mean this this dilemma should should

00:16:36

 be uh if not go away but it's going to be so much easier that the robustness comes from the foundation model but still you're going to have that for the particular task we're still going to have annotations >> and then and then you have Alex this I was thinking we we we might want to discuss this then you have the interesting question of bias what is your data made of so now you're going to have two data sets. You have the annotated data set here and then you have the foundation model that was

00:17:07

 trained with something. >> Uhhuh. >> So where is where where do you tackle your bias question? I mean technically with the foundation model we're going to be robust generalized as of now I'm saying the annotated data set that we own we control. We know it precisely we have all the rights to it. That's what I'm going to explain. to my regulatory body. I'm going to say this is that works. The underlying foundation model for me is more like a it's like a library. It's like a

00:17:42

 platform uh underneath. But I expect to have a lot of interesting discussions around that that topic. I am super curious what these discussions are going to be when you have them and if we get to meet again then let's talk about that as well because making this like taking this through the FDA is going to be a totally different game. I want to highlight a couple of things that you said that I think sum it up very nicely because it kind of mirrors the discussion or like the definitions of AI

00:18:16

 when there was not so much AI now everything is AI and people don't even ask oh what do you mean do you mean like you started saying okay is it vision only is it multimodel here basically there is kind of an understanding that AI is computers doing is in the background and like you said foundation model is just a big model. Why is it different from what we had before before? It understands the image. So foundation model would be a big model that un understands the image like with this context and is more robust to the

00:18:55

 domain shift >> and then whatever these models are going to be probably people are going to be coming up with new versions more par parameters less parameters more efficient less efficient but like you said the architecture the framework is the same and the differences is the generalizability understanding of the image age and that it's big and and that you actually this so I'm going to just take one more step back for those who don't know Aiforia yet actually a stands for AI for image analysis so it's in our

00:19:33

 case it's going to be analysis of pathology images and you said something >> super logical very understandable for pathologist but profound for digital pathology, do they even show it to the pathologist if they still have to wait >> because they will not. >> And I can totally attest to that. Um I had like instances where I was working where there was an option to look at image analysis masks for a project. If I had to wait for them, I just skipped that step. >> Yeah. Exactly. And this is by the way

00:20:11

 this is one of those things where where you had that question of access that okay people >> that's where I want to go now because >> mhm because so so you have this platform for image analysis and uh for the particular use case that we're talking about it's going to be vision transformers because everybody knows about the names that you mentioned vero uh path chat they kind of became famous in the pathology But if I wanted to use one, like there is no user interface. It's

00:20:46

 assume they're on GitHub or wherever these open source things are. If you're not a computer scientist, you know about them. You read the publication about them. You think, hm, would be cool to use it, but how do I use it? And this is where the platform comes in, right? My understanding is that with this platform you give people access to using them in a userfriendly and efficient way. So let's dive into that. >> Yeah. Yeah. So so so we we've focused uh for many years now on of course for

00:21:21

 those that don't know us there's there's really two things. is one is you make AI models viewers around them reporting structures and put them into clinical practice. You work out the integration to whatever they have whatever systems they have work out the integration so that images come in analyze is done you can either see the results confirm the report and put it and then it goes to whatever LIS or IMS or wherever you need to have it. So that's one thing we do and the other thing is is Ioria create which is the

00:21:57

 environment where we train models and we sell that as a product as well. So now there's two different use cases but on both of these the foundation models are available. So, and it's super exciting on the research side where you use this I for create because you can basically with two clicks you can jump from CNN to using one of the supported foundation models. So, let's imagine that you worked on a model for uh for a year. Uh you were training it, you created a convolutional neuronet network you and

00:22:33

 it it performed fine. Now, if you want to try what happens to this model if I if I put uh any of those foundation models underneath it, it's two clicks and you retrain and then you can compare and that's super exciting for the research guy. This is this is friendly. uh you're still in a no coding environment >> because you can and I got this interestingly from one of our big big customers uh in Rochester that is they where they use for create uh then one of the leading research guys when I spoke to

00:23:15

 him about foundation models he was worried and he said does this mean my team needs to start using Python or something else. No, absolutely not. You're still going to be in the same environment. Uh you're just going to look at the results and you have the choice of picking which foundation model you want to put underneath. Uh and obviously uh so so this is what we accepted long ago was that oh this is going to be an if not the tsunami but a big wave of foundation models coming from open

00:23:50

 source companies. So let's just ride that wave and make it make make it possible through the platform to get access to these models because because who can make the bet what's going to be the best and most powerful so for the image only I think that's how it works you can you can train it easily uh and then on the other other hand we pick we handpick some and we use small and mediumsiz foundation models we marry them with our annotated data and we put out clinical models and that that for

00:24:27

 the user it's just a question of quality they they just you know >> quality >> and I want to very much highlight that because you mentioned okay we have the this layer of annotated data and I experienced this firsthand because uh with my team we were working in Aphoria create and we had projects where uh the convolution convolutional neural networks were used and we wanted to know, hey, are these foundation models actually like better? Can we use what we already did and check it against our annotations? And we were

00:25:03

 able to. And that um is what you just said, okay, there is something underneath that's powering the image analysis, but you still compare to what was used as ground truth. Yes. >> And then it's a um question of quality. If the quality doesn't change or increases, there lies your decision what to pick. >> Right? But do you need to pick? How do you pick? Uh like is it too overwhelming or is it being guided by the quality of data? What if the data uh or like like your quality layer doesn't change? If

00:25:45

 the accuracy, F1 scores and everything is the same between the old and new, like how do you decide? >> For our customer, we we pick of course we experiment with a number of models and when whatever we put in clinical practice, we got to make sure that the the terms of the the models are are correct. But it really is no different from traditional sort of developing software. People use a lot of libraries. People use a lot of open source. It's like an established way of working. You just got to you use what is available

00:26:21

 and what is what is where the license terms are, right? Of course, you can strike a deal with another company and say, "Let's let's use your foundation model, but then you have to see great benefits for it." And I have to say, we everything that we've tried is is helping. There's no like super dramatic difference between what you have there. Still at the end of the day, the performance of your task specific model is very much depending on the head that is the annotated data. Is that clean? Is

00:26:57

 it curated? It is is it you know that's still going to dominate uh how the model performs. But the underlying model and here's here's one one thing to take notice of in all of this AI training is expensive inference running the model is not that expensive. So when you talk about cost you better look at how expensive is it to train how heavy is that? So after a while having worked on this uh first of all we we started picking smaller models looking for that balance uh of of what is enough uh not not too

00:27:36

 heavy but still helps us with the quality and we learned after a while that okay now the development cost is actually at the same level even though one training costs more the number of iterations became less. >> Mhm. because you had that you had that uh foundation model on the bottom. So actually it started to make sense. I don't know whether in your team you experienced that saying but uh the number of it sometimes you do a lot of iterations before you're happy with your model but now it's good enough and we

00:28:10

 just see the number of iterations goes down which is good news because it's not like we develop a model we put it in clinical practice that sits there forever. we're we're going to we're updating those models continuously. So this you know cost of training and retraining models and getting them uh to the to the clinical practices is super important uh how that works out but then on the research side how do people pick um we're going to recommend uh some models and we just make them readily

00:28:45

 available so you can just go and click and select that model. Then next to that people can come with I want to use this and we said okay we make it available for you. We've been delivering uh the platform for research use for years and years and years. So basically people do what they do. You know if they want to work on some crazy big model they will. Uh we're we're going to be interested. Sometimes we partner but you never know. I mean, if you have a big fat budget in your research department,

00:29:16

 you're going to try something crazy. Uh that that's how it goes. But we just recommend some models that are uh you know a a right balance. And and of course, if you look at some of those uh companies that work on foundation model, you'll find out they have the large model and mediumsiz model and small model. So everybody knows this that for research purposes, you want to reach for the big and uh but then when when you read about okay million slides two million slides we're not talking about clinical

00:29:54

 practice >> when we are in those numbers not not today uh and and I'm pretty sure every every company they have the cost optimized sort of uh usable version um that can be put into clinical practice but it's um I'll just say We we chose to go like this. Let's open the door. Uh make everything available. See what what happens because this field is moving so fast. >> Mhm. Yes. You know what it reminds me like you having this incorporated in the platform right now. And when when did

00:30:33

 this like appear the transformers? So the 2021 22 >> GPT for the first time or maybe 2023 I don't know but like several years ago right and then people see it people are excited about it and like how do they get their hands on it and that was the same case with CNN's where like you said first you work with your research group who can figure out the way how to get access to them you see if it's good enough and then somebody implements it where other people can like no non-developers can get access to and I

00:31:10

 know I'm like digressing from from the flow of our conversation but I wanted to highlight that that there's always this lag uh between new exciting technology and implementing it in non-ressearch reality into a software with user interface and I always see it as a funnel like you said the The top of the funnel is going to be research super heavy models and you figure out okay what's good enough >> and then one comment on what you said okay what is my experience with how many iterations to get to a good enough

00:31:48

 version for a pathologist at the end it's a lot and you see little improvement but you still want to have it a little better so you run it again and again and again so um when we looked at the comparison between CNN's and foundation ation models in Aphoria. Um it was nice to see that it was less iterations to get the same or better results. M >> and you know pathologist visually you're going to look at the image um and you guys have uh in the platform a special module to validate it against different

00:32:23

 annotations but for the pathologist it's always going to be the quality of segmentation right we don't even like bother with the calculations cognitively and I remember I had one of your collaborators on the podcast who was developing very sophisticated model for colon cancer and it was all bas based on segmentation on segmenting their visual features that pathologists are are describing in the model and then calculating predicting things from it. But the pathologist role was to confirm

00:32:58

 that segmentation. >> It is it is interesting that that where you said like our medical director Richard Dowy who's been with you as well on >> Yeah. He was my guest as well. Mhm. >> Yeah. Yeah. So he keeps telling me he tells me like every second day that pathologists are visual people. They're visual people. Remember they're visual people and and everything that how we visualize things is the is the big thing >> and and even in the development tool we've started to go to that direction

00:33:24

 that you know support that. But I do though wonder when we say how do you get access if you're a researcher uh pathologist I would though argue that you have a lot of young pathologists typically that are super handy in coding in Python. So so this is shifting you know that once we've moved away a little from the microscope and you have digital data all that it is changing. So there's the visual side of course, but there's also uh you know I've worked with a lot of modalities, a

00:34:02

 lot of medical doctors and and there you know quite in my head like the surgeons are mechanical people and there there's there's yeah we have a lot of uh people that are quite capable of scripting things and if they want to try out uh some some open source foundation model and they want to they want use the API of that model and just extract some things they will that that will happen but but but for the most part in my head I have this what's happening in the clinical practice >> what can you do in a subsecond

00:34:40

 [laughter] uh >> exactly in a very fastpac workflow >> where you don't want to like stop for any longer than necessary that's like the reality check to me but but you're totally right. Suddenly since the pathology became digital or since there was the option for this to be digital since this option was introduced um like computational pathology fellowship started appearing this became a lot cooler specialty than just you know the stereotype of a person sitting in the basement and looking at slides and not

00:35:19

 talking to anybody. Now it's fun. Now you can experiment and you see it in the literature. So when I do uh the journal clubs there is a lot of these like experiments which makes me super happy because that means pathologists are interested in this. they collaborate, you know, even if they are not going to do this themselves, they're going to find a person who can collaborate with them in the research institution. And really cool things are happening for people who are just starting their career.

00:35:50

 >> And and then you have the clinical reality which is the other end of the spectrum where it has to be super efficient, logistically, visually easy for somebody who is working very fast and there's no option for them to work slower. >> Yeah. Then we then we do have the I mean that I mean that that's the like the primary focus area for me and and our team has been the images only and and clinical workflow next to that that no code environment where you can train models and these two are in the same

00:36:26

 platform so you can easily work across but then you have the in the context of foundation models then you have the multimodal models uh and this is uh this is a very different game and we haven't announced big product launches in that field but but this is uh this is so interesting of course in the research community a ton of work very significant work is now going on in with with multimodal models uh doing longitudinal studies looking at outcomes and coming back which is in in my mind more

00:37:02

 interesting than the image only foundation models like pumping more and more millions of slice into the thing. I don't know whether that's going to help that's going to >> maybe something unexpected comes out but it's more like the marrying of this with the text and that shows tremendous promise and I've thought a lot about this this workflows of how what how do you actually make this available for the pathologist and of course and you can tell me your insight on this but you do

00:37:35

 see a bit of worrying signs is that instead of a controlled uh path, you know, way of managing your data and taking care of security and privacy and everything in an integrated workflow within the lab, all of a sudden somebody might just take your image and pop it into a path chat type of a window than what is here. It's it's s such a different uh use case and and of those famous models pathjhat and vow like different worlds uh they can be combined but but different worlds and I've I've

00:38:18

 worked on how to how to make this controlled and supported. Now if you imagine you have this uh this multimodel foundation model available you can basically ask it anything about your workflow is that medical device in your workflow is it the supporting device how do you get that how do you discuss that with the FDA and I know there's a there's a process going on with them about that and that is that is going to be uh really important how we make this happen. I think for me it's a reality. I

00:38:57

 need to have very tightly controlled image only models that actually they're validated. They're you have everything tightly under control and you have your performance numbers and you're going to be challenged by the regulatory all all this is going to have to happen. But then next to that there will be oh what is this? And now the power of the large models comes into the game because then you want to have you just put a box there and say what is this? Have I seen this before? And you're going to be told

00:39:32

 that you have seen this before and you saw it you know seven years ago in this case or in the literature there's this that shows an enormous promise. I think I think the regulatory path of that is obscure. It's going to be tricky. Uh but in my head, those are supporting tools. Uh and we just have to for the greater good of everybody, we have to get them into the platforms. Um so that you you have it under control that where did the image come from, where did it go, who asked what and from

00:40:08

 where you have an audit log of everything that happened. So then it be it then it's safe and controlled for all the >> all the hospitals and and the labs but enormous enormous opportunity in that field. Uh and then of course going towards predictive and prognostic models which is something that we work on quite a bit as well. >> That that's a different promise of the that's where the large data becomes pretty relevant. >> Yes. And exciting. And I agree with you that it should be controlled

00:40:51

 with an audit log. And people think like there are different opinions on that. But why do I think that that's the way to do it in healthcare? I mean probably in healthcare people will agree because it's sensitive information and uh there are regulations but my point here is because it is available in consumer AI people are going to be using it. There's >> if you look at u you know LinkedIn posts for professionals you're going to find a plethora of oh I pasted this image into

00:41:28

 chat GPT and it was right but with this other image it was wrong and like a lot of these so basically people are already using it you can you can take a screenshot with your phone whatever and always this gap between what's available in your real life and what's available for work people will try to figure out to bridge this gap if they don't get the same capabilities in a secure way to use for work. >> So that is another um motivation for me to push it into the tools that are being

00:42:05

 responsibly used in the clinic. So it's uh just resonates with me what you're saying not just for the safety but also to alleviate this pressure of not doing it the right way. um when it's like already being done for fun out of curiosity. But where is the border between fun and curiosity and actually using it for something it should not be used >> if you know that the value is there and could benefit your work. >> Yes. >> So I am a champion of doing that. I saw a couple of early demos couple of years

00:42:45

 ago and the first time I heard about the um image search that was a Google effort that I heard about in 2019 and I was like where is it? I want it never appeared but now it's reappearing. So thanks for mentioning that as well. Um, one thing that I know we wanted to talk about is, and it's kind of kind of derivative from, okay, how are you going to be talking about it to the FDA about the explanability aspect of this all >> the architecture is kind of explainable, the results, maybe there are ways to

00:43:24

 explain them, but can you tell us a little bit uh about the topic of explanability of this particular or new development of AI which uh we're talking about the foundation models. >> Well, yeah. So, so, so our approach has been all along with with the specifically on the clinical models if we were on that side is that you visualize to the detail and it's it's quite challenging if you want to do it. uh in contrast to let's say you uh if you would uh train with just feed pathology reports in here and

00:44:08

 images there and then you train a model then don't really know what's happening between and then then you visualize that there might be something here and you show a heat map of look looking here there might be something there I think there's a lot to explain what what's going on here so where did this come probe and and we've been in the game of okay you do your semantic segmentation you do the object detection and you instance segmentation you pinpoint we think this this here here is a positive

00:44:42

 cell and that's a negative one so but that's challenging it's uh it bumps up your quality level to have but for we always that this is how I can we can explain to the pathologist this is where when we when we calculate the the number of cells and we said on this slide you have this and this many cells then you can actually pinpoint and explain this is how it works. So so we're going to stay on that and next to that we can we can show them what is our ground truth. That's one of the most uh that's the

00:45:17

 >> tricky questions in in this domain that you go to a new lab and and you start talking by you show how your model performs and then the first thing the pathologist can I don't agree with that and then we then we have to go back to well this is a this is our this is the ground truth we used uh for the annotated data. So that makes the discussion pretty hard. It's not always easy because you know you may end up with well look in this country we do this differently. Uh but you can explain how it is. Now if you have the

00:45:51

 the combination of a foundation model in the bottom and then you have still a limited amount of annotated data you can explain how what what this head what this this this model is doing. Uh and and yeah, I've always found those discussions challenging but mandatory like absolutely mandatory that how come you claim that this is grade three or grade four and you know and then of course we presume that what's going to happen and what has happened with a lot of customers is that okay with time

00:46:30

 you've you've gained that trust that you can trust that but in the foreseeable future I think every pathologist I'm just going to going to want to see where where do these numbers come from. And maybe at some point in time you you have a system that you know eats slides and outputs reports and you only sample every now and then but in the meantime in the foreseeable future you have to be able to explain how did I calculate uh uh where where did I come up with this result and but that combines with uh if

00:47:03

 if you have uh if you don't have that if then then it gets tricky like uh uh a foundation model alone can't can't do that that kind of uh >> visualization of the detailed results. >> Yes, that's true. So, anything else you're excited about? Anything else that you guys are working on that you can already reveal >> about the >> And if not, that's okay as well. [laughter] No, I'm excited about just I mean across the board for for us >> our whole portfolio is is going to run

00:47:43

 and runs on the on the new AI engines as we call them uh >> vision transformed base. So everything goes to that space. I'm super excited about the predictive and prognostic work uh the promise of that and how we can get that into the same workflow because because everyone that's been implementing in their lab digital pathology knows that there's this this weight of getting things integrated and talk to each other and all that. Once you're done with that then be it the diagnostic clinical AI model

00:48:27

 or uh a model that is doing a prognosis like like for example what we did with Mayo Clinic the quantcorectile cancer that kind of applications it's the same workflow it's the you just you just put it in there you have your breast case if you have the H& you have the IHC All done. You can trigger uh a calculation of of an estimate that's an outcome outcome based training trained model. You you you just plug it in there. I mean once the whole community gets over this that let's first get the

00:49:07

 labs clinical then you have these workflows you have the AM models nicely as integral part of the workflow. Then this whole field where pathologist can actually step a little bit further into towards the oncologist and start making predictions. It's it's like it's so it's within reach. It will happen. >> So that if you ask me what I'm mostly excited about, I'm pretty excited about that. >> Sounds amazing. So if somebody would be interested in trying it be be it for

00:49:45

 research work or um to deploy any clinical tools what's the best way to engage with you and I'm going to leave if there is any you know I'm going to leave the website link in the description um but how does that work if somebody's interested in doing it actually in A4 >> well easy is it of course through the IOA website and it just post there or or you put an email to you just you just send an email to me and that's going to be my name panel.comforia.com uh just just write to me and we'll take

00:50:24

 it from there. >> Sounds great. Thank you so much for joining me today. I enjoyed the discussion a lot. Thank you so much. Have a great day and I talk to you in the next episode. >> All right. Thank you. Take care. Bye bye.