123: Generative AI: Deeper Dive

In this episode of the Digital Pathology Podcast, I take a deeper dive into Generative AI in Pathology, following the AI in Pathology series published by USCAP. AI has already begun transforming medical diagnostics, but what does Generative AI mean for digital pathology? From synthetic data generation to multimodal AI models, this episode explores the cutting edge of AI’s role in pathology and how it’s evolving to enhance efficiency, accuracy, and patient care.

🔬 Key Topics Covered:

• [00:00:00] Introduction – The Evolution of AI in Pathology
• [00:02:00] Acknowledging the Authors Behind the AI Review Series
• [00:04:00] What Is Generative AI and Why Is It a Game-Changer?
• [00:06:00] The Cambrian Explosion of Generative AI in Medicine
• [00:08:00] Understanding Transformer Architectures and Cloud Computing
• [00:12:00] ChatGPT, DALLE, and Multi-Modal AI: What’s Next?
• [00:18:00] Synthetic Data Generation: A New Era for Pathology Training
• [00:24:00] How AI Can Pass Medical Exams and Assist Pathologists
• [00:30:00] Reducing Bias and Ethical Concerns in AI-Based Diagnostics
• [00:38:00] Real-World Use Cases of AI-Generated Pathology Reports
• [00:45:00] The Future of Generative AI in Digital Pathology

🩺 Why This Episode Matters:
Generative AI is no longer just a concept—it’s already being used to train models, generate high-fidelity pathology images, and assist with diagnostic decision-making. However, challenges remain, from bias in AI models to the need for domain-specific training data. Understanding these factors is essential for pathologists and medical professionals who want to leverage AI responsibly in clinical practice.

🚀 What’s Next?
This episode discusses not just what Generative AI is but how it can reshape pathology workflows and where we’re headed next. If you’re interested in how AI can improve efficiency and accuracy in diagnostics, this is a conversation you don’t want to miss.

🎧 Listen now to explore the future of Generative AI in pathology!

👉 Watch or listen here: https://www.youtube.com/live/hRv9GmMWSjk?si=OEg8gafqRA2M_zlx

#DigitalPathology #AIinHealthcare #PathologyInnovation #GenerativeAI

Transcript

[00:00:00] Good morning. Welcome. Welcome. My digital pathology trailblazers. And today we have the second part of our AI in Pathology series published by USCAP. And we're going to be doing a deeper dive into generative AI, generative artificial intelligence in pathology. Learn about the newest digital pathology trends in science and industry.

Meet the most interesting people in the niche and gain insights relevant to your own projects. Here is where pathology meets computer science. You are listening to the Digital Pathology Podcast with your host, Dr. Alexandra Zhurav. Let's start with acknowledging our authors because I am so beyond grateful that they wrote this series.

This is our Number two today and our authors here human [00:01:00] Rashidi, Joshua Pantanovitz, Ali Reza Chamanzar, Brandon Fennell, Yashang Wang, Rama R. Gulapali, Ahmed Tafti, Mustafa D. Baja, Samer Albara, Eric Glassie, Matthew Hanna, and Liran Pantanovic, thank you so much for writing this paper. And today we're going to be diving deep into the generative AI, which Who is very famous for chat GPT and all the other large language models that we already know.

So they call it the Cambrian explosion of generative AI that, that was the evolution from early rule based to sophisticated new ways of doing AI and generating novel realistic data. So we already said that last time, but I'm going to repeat it. Generative AI is. AI that generates something, [00:02:00] makes something out of, you know, training data, makes something that did not exist before.

And we're going to be diving into what it can be, how it's generating it, but basically generative AI, and they call it Gen AI. I'm, I'm like Gen AI, was there no Gen Z or whatever gens we are? Millennials? Anyway, now I guess everybody's Gen AI. But in the context of this paper, generative AI. And we're going to be diving deep.

So what happened that, that we were even able to have generative AI? Two things, transformer architecture, which I think we mentioned last time that the famous paper that describes this architecture is attention is all you need, and there is a mechanism of self attention there and cloud computing infrastructure.

So, because these models. Up until recently when DeepSeq came out, we're requiring a lot of computational resources. We're going to mention DeepSeq today, so [00:03:00] stay tuned. So, November 2022, the release of ChatGPT, and I remember I gave a workshop on ChatGPT, and I was trying to do literature research then, and there was no literature.

I'm like, oh, if I was motivated enough, I would write a paper, just like what it is and what. could do. Of course, I was not motivated enough, but I gave a webinar. I was motivated enough to give a webinar. So anyway, ChatGPT, the most famous example. There are many examples. Llama, including Llama, my favorite, that I don't use.

I just like the name. And we have Anthropic Claw that I use, and these were text based, right? But also we have DALY, Stable Diffusion, Mid Journey, or FLUX, which are images. And they can generate images and the bleeding edge. I love this. It's not cutting edge. It's the bleeding edge of development are multi modal models.

And this is going to be [00:04:00] basically the theme here today. A generative AI can generate what can generate different types of data. And if it's multi modal, it's super powerful. It's, it's, it's like on steroids. So examples of models that can use different media is. chat, uh, chat GPT 4. 0 or yeah, 4. 0, GPT 0. 1.

And the next one coming out is 0. 3. And then we have Gemini, Claude. And at some point I'm going to start giving you examples because these are all the same examples of things of models of those AIs that can do different things. And within the medical field, what. What did it already demonstrate? It demonstrated it can pass medical exams.

And also I think the board certification for pathology, there were publications on that. And for medical exams, the newer model, the CHAT GPT 4. 0, 3. 5 didn't make it, didn't make the cut, but 4. 0 did. [00:05:00] So this is fantastic, but what can happen there? We can generate high fidelity images and they can assimilate complex biological system.

These models can summarize, synthesize clinical documentation and can do different, different things and basically make the data that we are using. And we have images today, so we're going to be working more with the images than with text. So, these general models, let me call general models, they have great potential within medical domain as well.

But, at the moment they are limited because the training sets, well, like the framework is super powerful, but the application at the moment of the general public information needs domain specific data, right? And there are already models that are being fine tuned or trained for those specific data types.

[00:06:00] But one thing is the, this retrieval augmented generation. So, so you can, when you train these models and It's described in this paper. It's not magic, my friends. This AI is not magic. What do I mean by that? It's something, it is like magic. But basically, these transformative concepts still rely on the basic concepts of machine learning, right?

Fundamentals of machine learning that we talked about last. So it's going to be unsupervised learning, then it's going to be supervised learning and at the end it's going to be reinforcement learning approach. So it's not magic. Concepts are not magic. They rely on those fundamentals. Unsupervised self attention mechanism.

Then we have a supervised, this is the fine tuning phase. And then we have the feedback reinforcement learning, often feedback from [00:07:00] humans. So it's a process to train one of those models. So how can you leverage what you already have without having to go through the process. So for example, for the fine tuning, you would need to start with step two supervised learning fine tuning, which people are doing as well, but like for immediate use, right?

How can you leverage it? You can leverage it with retrieval augmented generation. Which is basically retrieval from documentation, and then it generates outputs, it generates whatever you're asking it for either an image or mostly text, because it's still the texts. Text chatbots are more popular based on what you've given it.

So I use it a lot for literature search. There are even apps that are specifically, you can already install them as Google or, or a browser. What is it called? Plugin, plugin. A plugin that whenever there is text, whenever there is papers, [00:08:00] paper, it shows up and lets you chat with this paper. So for example, you want to cite something, uh, you see the title matches, but is it really in the text what you're trying to cite?

You can ask this little chatbot to do it. So let's go to images today. We are here, my trailblazers. We are already in stage two. So cute that they give us this, uh, educational journey. Uh, I like it a lot. Still, a lot ahead of us, and some we may be splitting, especially, this one, statistics. So. The, this figure two is going to take us to the basics, hierarchical structure of AI technologies.

So we have machine learning. This is the broadest one. And often you will see this diagram as, as a circle as well, that one circle and another circle. Okay. So we have our [00:09:00] machine learning, everything. And it's like everything that you will see here comes. like propagates from the stuff that was invented at the beginning, like in every funnel, right?

So machine learning, then deep learning. And we have foundation models, which are the huge models, chat, GPT, and all the other models that are trained on large amounts of data. And then Based on this, we have LLM and vision models and et cetera, and we're talking today about generative AI, and all this can be leveraged for non generative AI as well.

And what can this generative AI give us, like conceptually, right? Let's start with this. We read from left to right. I'm gonna start with syn synthetic data generation, [00:10:00] and last week I told you the story, how I learned that this series is gonna be created. I talked to Huma Rashidi at the A CVP in 2023. We were talking at the same.

at the same stage. And he was talking about synthetic data, but he was mostly talking about tabular synthetic data and AI. And this generative AI can give us not only tabular data, but it can give us synthetic tests, a custom medical chatbots, which I don't know, maybe there is already something. Like out there, but I have never seen it when I go to the doctor.

I don't go to the doctor too often So that's good last time. Okay. I need to tell you the story last time I went to the foreign For a knee exam, my knee was hurting because I was running too much and then I told the doctor Well, when I stop running it stops hurting. He's like looking at me Yeah, how about you stop running?

I'm like, yeah, I do when it hurts Anyway, but the story [00:11:00] here is I go in and there is a lady sitting next to a machine for electronic health records So I go in there and like type my name. It's a touchscreen and I'm like, oh hi tech fantastic. And I'm like Why are you sitting here? Oh, I'm sitting here because I need to encourage the people who come in to actually put it here in Electronically, they're not used to it.

Especially older people and I'm like, oh good. That's like change management good change management and Then I type everything I click submit and she's like, oh and now go to that lady They're at the counter take a paper and put that again on paper because she needs a paper record as well. I'm like What's the point?

Yes. So I don't see those chat bots entering. Let's continue with synthetic data. So it can be text. It can be the tabular data that I mentioned. It can be of course, video and whatever other type of data. So in, in, and [00:12:00] the cool thing is that you can generate data to train models. On more data than it actually has so last time we were talking about Data augmentation.

So this is data augmentation on steroids You have one image and you can generate multiple images and then train another model to recognize things in those images So this is fantastic. Then we have the virtual reality content It can help create immersive vr experience for medical education and training, which is not to be underestimated.

And there's always this discussion here, medical education and training. Hey, are we gonna lose our abilities because now we have this AI or what are we going to do with our profession? And it's. across the board. That's, you know, every profession, then every profession is asking themselves, Oh, how is it going to change?

Uh, recently I had a discussion on LinkedIn, uh, with a histotechnologist with a [00:13:00] lot of experience. Uh, when I published an, um, article about direct to digital pathology, where you don't need to do histology anymore, right? You just have fresh tissue and you image it. And that sparked discussion. So here, um, the, the danger is, okay, we're going to get complacent and we're going to rely on those.

tools. The advantage here is, um, we can have these immersive VR, a lot more like fake practical training on data that is being like, As, as educational as real data, but you don't have to get this real data and we all know that getting real data and now I'm thinking about training image analysis models, training the, the vision stuff for pathology, you don't have as much data as Google or whoever was training those image models in the medical domain, regardless whether it's pathology or ideology, like there is not as much data in a subdomain as a [00:14:00] natural images, right?

So if you want to train on different appearances of tumors of rare on rare disease Now you can do it with a synthetic and with virtual reality So you can do virtual reality for surgery like can you practice surgery other than doing surgery? And now you can with virtual reality. So this is fantastic Audio and video processing, generating and converting audio from, for clinical notes and visual content for medical training.

And this is so, it sounds so simple, but it's profound for the efficiency. If you look at literature, I will not cite exact numbers, but there are numbers like how much, I think it's like 30 percent of medical professional time is spent. Doing admin tasks. And one of those admin tasks is making, making clinical notes.

And when you look on social media on [00:15:00] LinkedIn specifically, there's going to be like, Oh, writing a clinical note two weeks, Hmm, what did I do during that consult here, audio and video processing. And you can dictate the note and you can have it real time. It's going to structure unstructured data, which is also a super important leverage of this AI, because that was.

It still is a bottleneck, and we are not getting rid of preparing data for AI. And we need to do that as well, but not to that extent. And individual users in the medical profession and across the board can basically use unstructured data and make it structured, including audio, including, like, me. What I'm gonna do with this, and that's gonna be part of the, um, of the surprise that I have, uh, for you at the end.

What I'm gonna do with these live streams, I'm gonna Update my book because there is an AI chapter from I published it in 2023. Hello. It's old in digital pathology Um world, but [00:16:00] if you download this one from my website, you're gonna get the updated version Of course whenever it's out But what i'm gonna do and that's why i'm so super grateful for those publications because I can now cite them I'm gonna take these the transcript of those live streams and prepare it in a structured way so that I can update my AI chapter in the book.

And whenever it's out, you're going to let, I will let you know. So yeah, this audio and video processing, of course, natural language processing, enhancing and automating the drafting of documents, research of educational papers, and we already know that the journals. Like, I don't know if there was ever a ban, it's more in institutions because they are afraid of data leakages, which happened when JAD GPT was released.

Even the whole country banned it, Italy banned it for some time. Anyway, but the journals, scientific journals, they basically require you to disclose, okay, what did you use [00:17:00] AI for, for drafting, ideation, whatever, right? And there are even frameworks, um, a veterinary pathologist colleague of mine who I had, uh, on the podcast.

Candice Chu, she actually trained a clinical pathology for vet, a clinical pathology chatbot for veterinary pathology and she Publishes a lot and uses AI and has like a specific framework that she teaches at workshops how to streamline and speed up your publication creation of publication, right? And then we have also image to text conversion.

Here we are entering the multi model. way of doing AI, and we can convert medical, convert medical images into structured report using multi model AI, for example, using a pathology image to predict disease followed by generation of diagnostic pathology report. I don't know how good those models are with that, but definitely there is the potential to do that, [00:18:00] because if you have the option to do it with natural Images with additional training, you can do it with medical images because it has already been proven well by that.

It can be interpreted even though they are super complex. It's not going to be easy. It's not going to be straightforward, and it's not going to just require rag that I love so much that you like prepare data and it goes searches because that's just text, but it's possible. So, um, it yeah. there is an option for creating a report from an image, which I think is fantastic and scary, both at the same time.

So here is an example of a synthetic image from this paper. So here we have A, and this is real H and E, and the diagnosis here is A. An HNU recut of a BCL6 compressor gene B Core rearranged sarcoma case. And you can see that it's a [00:19:00] sarcoma because of its architecture and something typical for sarcoma are these wavy cells.

And there are different other diagnostic criteria. But I would know that this is, even though maybe I would not be able to diagnose it as a pathologist, I look at it and I know, okay, it's mesenchymal. And then I look at this one, at the B, And if I didn't know, I probably wouldn't know it was generated with AI, right?

So, that's impressive that, and it's a publication, so we know that publications get the most beautiful parts of an image. But even if At the point of this publication, this is like just a little good corner of some image, which I'm just basically assuming that the worst, worst case scenario. That's amazing.

That, like, I wouldn't be able to tell if I didn't know. That B is the synthetically [00:20:00] generated HNE recut. Uh, uh, Of the same created through infusion model and any type of infusion models for GAN lines here and the same, well not the same, but this is already implemented for virtual stains. There has been commercial efforts to generate virtual stains that are indistinguishable from, from the non virtual.

This is being used for generalization across different domains. And by domains here, I mean, scanner, different colors, different labs have different colors of H and E, right? And if you want to train a model, you need to somehow homogenize them. And this image creation is something that is being leveraged here.

So impressive, right? And it doesn't have to be call site images. This is something I wanted to emphasize here because call site images And we talked about it, and the authors here [00:21:00] talk about bias and ethical considerations. That's going to be an ongoing discussion as new tools come out. But one of the discussion points when we talk about countries with resources or places with resources, even within, you know, one city, and places with less resources, Oh, wholesale imaging is a limited technology because You may not have research.

Well, you can do it with static images it and this is a lot less limiting technology. Take your phone if you want and take a static image. High quality static diagnostic image, right? Um, and by diagnostic, I mean, it has enough diagnostic features for a pathologist to tell you what's in that image. I'm not talking about regulated diagnostics, but I'm talking about diagnostic quality.

This is something you can totally do with a phone. And we also have options for modality. I have a special [00:22:00] star for path chat. I only knew what BD was. Not business development. I meant something else. BDE, in my case, was Breakthrough Designation. So, Breakthrough Designation is like a category or like a speed lane from the FDA to go and develop something and put it in the regulated Environment breakthrough designation.

So past chat from the company. Well, it's, uh, spun out of a research lab, but now it's a company called Modella dot AI, and they got a breakthrough designation from the FDA to go and develop this chat. And it's actually, it's multimodal and it's, it has adjantic capabilities. We're going to talk about adjantic in a second, but what it does.

And I met them at the conference. It's basically, and what I did, I had, um, histology earrings, uh, I took [00:23:00] a picture with the phone and we pasted it in past chat and asked, what do you see? And it actually recognized what my earrings had. I don't have the earrings today. Next time I'll bring them. And then you could chat with, with the image, like you could chat.

Okay. So it was a multinucleated giant cell. And is this, which diseases do I need to consider as my differential diagnosis when I see this in an image? And you basically can have a discussion and they have another agent that does image analysis, but basically they are very much trying to leverage AI to improve patient care through pathologist workflow and through support of pathologists and also pathology researchers.

So that this path chat and and this can bridge the gap between various medical disciplines and various scientific disciplines. So what they said that they had this version that people could download and [00:24:00] a lot of image analysis researchers that do image analysis for pathology and in the medical domain, well, they are pathology specific, would download it because they don't have access to a pathologist.

So they wanted something to tell them what's in the images. So this is figure three and we have categories of generative AI models. And there are four main categories and four main categories, as you can see, are related to the type of data that is going to be generating, right? We're going to have text.

We're going to have text. We're going to have image. We're going to have multi model models and we're going to have multi agent frameworks. So let's go through this on this image. Text generators. Well, it generates text, right? And it's so cool because You look at this image and you basically know what's happening in the generative AI space and this generate text [00:25:00] Generators generate images and use case generating diagnostic reports.

I am waiting for this to be mainstream for reports where It should be in the diagnostic space, and I would be very disappointed if nobody did it in the veterinary diagnostic space, which is unregulated, where you dictate and have speech recognition anyway. So I hope they're leveraging it. I need to ask some diagnostic people.

What are the advantages, pros and cons, advantages and disadvantages of this fast, efficient and effective can be customized to disciplines, and it is easy to use it. Like, who has used EverChartGPT? What did you think, um, fantastic image? Yes, well, thank you authors of the paper. They did a great job in making it.

Making it very reader friendly. It's not the [00:26:00] normal paper speech that we are used to from our scientific papers. It's, it's a very reader friendly review AI speech. So this is fantastic. Okay. So the interface of ChatGPT, right? How difficult was it? Not difficult at all. You don't have to learn to use it.

You just open it and you type and that's the user interface. Ain't that brilliant? I think it's brilliant. So, and issues, of course, all the minuses, right? Bias is gonna be showing up often. So bias. Many of the best ones are still closed source. There is a discussion on open versus closed source, prone to hallucinations.

And our examples are Chajipiti Lama, Claude Mistral, Gemini, but also the newest kid on the blog that's resource efficient, DeepSeek, right? I need to find the text to talk a little bit more about hallucinations. So once we finish the image, I'm going to find that. Generating image. I love this. A picture is worth A thousand words.

[00:27:00] You have one picture, you have this thing that generates images, and you have another picture times infinity. Very comprehensive. Um, so, this one, the image generators generate images, and use cases is Synthetic image data for clinical research and educational needs and creation of synthetic biobanks, um, less red tape.

This is so cool. I read about it. That was the other applications are kind of like logical, but I didn't think about the synthetic biobanks. You can basically generate a bunch of images and let people use it for testing their solutions, uh, image. Image analysis. I was talking about image analysis.

Different, right? Different things. You can have synthetic biobanks of tabular data. You can have synthetic biobanks of image data. Yeah. And I think this paper with the applications, when you think about the applications, [00:28:00] then you will know how to use it in a sense that, right, I will tell you how I use it for the non medical space.

Well, educational space, right? If you're in the educational space, you can use it as well, but just a quick digression, right? How do you use it? You have now this capability. You can, uh, have chat GPT for free. This deep seek probably has a free option. I didn't sign up yet. I will to check it and You have a paper, you can create the paper, you can summarize the paper, you could dictate something, and get it transcribed for you and put it in a structured way in a table.

So these are like quick hacks. What can you do with the tool? And actually your imagination and sky's the limit because you just get this tool, this leverage, and you can figure out, okay, when, uh, where can you apply it, uh, in your workflow? And obviously if it's behind regulations, then you're less free to use to [00:29:00] use those tools or you need to wait for the regulated tools.

But if it's outside of the regulated space, education, veterinary medicine, you know, with some workflow efficiencies, Prose can synthesize many images for different use cases and enables open source options. And this is a visual, well, visual medium that these are the minuses may be limiting use of which may be less user friendly than LLMs, LLMs are the large language models.

So this is text that's easy. You just type, right? The images may be a little bit difficult and we have the black box element of AI. We talked about explainable AI. Last time, so this black box, there is a whole science disciplines trying to explain the AI black box in different ways. So that's still is a limitation.

And synthetic images are only as good as the real, as the real images. The [00:30:00] Consumer synthetic images that I see like when you look on linkedin or even on my thumbnails and different things You will recognize when people were using ai image generators. It doesn't mean that this is like the best it gets but this is the easiest thing to generate and you like give it a prompt and it generates an image and I can immediately recognize that something was image, uh, generated.

So then we have multi modal models, uh, which use combination of data. And here, the, for example, integrating clinical notes and medical images, they. The, the plus is going to be it can consider different types of inputs and produce several types of outputs and the multimodal inputs allow for more advanced and nuanced pattern recognition.

The problem is going to be evaluating their performance in real world application. So [00:31:00] evaluating performance is of those AMLs is. A challenge in general, and then the more modalities you have, the more difficult it's going to be and need to plan how to effectively integrate multiple output modalities, as well as large computational resources and greater technical knowledge is required.

Yes. Yes, at the moment, yes. And then the, the fourth level or the fourth way of doing it is multi agent frameworks where AIs are working with AIs to do different tasks. So the whole is greater than the sum of its part. I love the descriptions. So text simplicity is often the key to success. So here, this is crosstalk between different generative AI models.

And it can analyze different aspects of the patient data. For example, uh, images, notes, another laboratory data. You can add, basically have agents, have different AIs for [00:32:00] different things. And it can compile it, it can summarize it for you. And then you look at it and think, okay, does it make sense or not?

And then it can come to a consensus enhanced diagnosis based on all the evidence. So the pros is it maximizes model strengths and minimizes their weaknesses. So they like work together, the robots work together, but difficult to ensure desired interactions. So model incentive to work with others may be limiting, that would require some, some special design, and also computational resources, greater technical knowledge.

Okay, I'm gonna be super vulnerable here. Here, a vulnerable moment. I wanted to have an agent for for digital pathology plays for creating content for like I'm telling you all having audio and then making structured posts out of it and then [00:33:00] social media posts and things like that. And I found this company, I'm not going to mention this company.

And I'm like, this is it. I need to have it right now. First. I didn't really think through my use cases. I'm like, okay, what is the workflow that I want? And I gave, and they offered like custom agent design and I, I gave some like simple use case and, and like, I had to have it. I'm like, this is so fantastic.

I must have it. It's going to solve all my problems. I spent 1, 500 on this. And never used it because I didn't check if it connects with my Outlook. No, it was working with Google and I don't run my business on Google tools. I run my business on Microsoft tools at the moment. So yes, I spent this money, didn't use it because I didn't check the basic freaking thing.

If it's going to integrate with my current workflow. So please do that and like, just don't spend money. Before you [00:34:00] do your research like yours truly did and they lost it. That's okay. This is how that's the price of education. I think when you consider that this is what I paid on my gigantic AI education, probably there are courses that cost more than that.

So that's okay. Okay. Hallucination and bias. Okay, we're gonna, we're gonna get there. But first, I want to mention this, this mixture of experts framework. So the multi agent frameworks also have enhanced flexibility and robustness. But I do, if I were diligent enough to do my due diligence before I bought my agent, that I then discontinued, I would probably get it to be flexible and robust.

And because they use different frameworks. And one of the frameworks is mixture of experts framework seen in the MIS trial by Misra where. model that intrinsically [00:35:00] incorporates a dynamic and collaborative approach for multiple LLMs. So it leverages multiple LLMs and it like knows which LLM has which strength and then deploys the one that has the strength.

And this is also the framework. I don't know if like this exact one, but it also is called mixture of experts. That is the framework behind the DeepSeq, which is so computationally light, so it only deploys a certain numbers of parameters depending on the query. It just activates like this narrow, um, narrow set of whatever it needs computationally to answer this particular, uh, question.

So it's not going to like activate everything and use a lot of resources. It just, and the number was 600 something of, I don't know what, I think parameters, just. parameters. But anyway, it deploys what it needs [00:36:00] to deploy to answer a particular question, rather than, like, go this huge blanket approach, activating everything when it doesn't need to activate everything.

So, mixture of experts framework and Let's talk about hallucinations and biases, also known as confabulations. I love this name, confabulations. So it's basically inaccuracies. When you ask a question and you get an answer that is not correct. In the example they give here, I have a It was supposed to be a heart here that the, the sky is purple, which grammatically is correct, but factually is not true.

Well, maybe when it's a sunset or sunrise, it is true, but most of the cases, it's not true, right? But they say, That you can minimize these things with different approaches. One of them is the RAG, [00:37:00] the Returnal Augmented Generation, which we already mentioned, but where do they come from? So because of drawing from poor quality training data, um, and also a model drawing on its semantic structure of language without a complete contextual understanding of the world we live in, which it will never have, right?

But these are the sources, and it is influenced by so called model's temperature. It's one of the hyperparameters. It's a hyperparameter set. that is defining the creativity, creativity of the model. And if you set it low, it's not going to be creative. And if you set it high, it's going to be more creative.

So when the temperature approaches zero, then we are gonna get just one possible, only a single possible outcome if predicted. So if you have used ChatGPT [00:38:00] you know that for the same question you will not always get the same answer in terms of like literally in words. It's not going to be the same output.

It can vary in terms of different sentence structure, different lengths, and things like that. So this is not set to zero for ChatGPT then it's just one possible. Option for query and when the temperature is of one, it's going to give you multiple options and and if you increase this temperature, it's going to reach a level of gibberish, so it's not like it's going to be so creative that it's not going to be language anymore.

It's not. It's going to be outside of the framework, right? So. You can work with this particular parameter, but of course temperature is not the only factor that influences models output diversity. There are different things and they are giving just examples. Something called top k and top p. I love it. I love the names.

TopK is the predictions. So the models topK parameter [00:39:00] limits the model prediction to that topK most probable tokens at each generation steps. And tokens are like these sub words or parts of language that are usually smaller than words where the model predicts with with these are like the building blocks of the predictions.

Top k, uh, it limits to the most probable tokens and then top p is gonna regulate the cumulative probability of the generated tokens. As you see, regardless whether you understand it at the moment or not, you have already been introduced to the concept. And I think that's the value of being exposed to this information.

Even like, we're not going to be learning this by heart right now, but we're exposed to this concept. And the next time we hear it, we can build on top of this concept. I remember when I was learning about GANs, I'm like, Ooh, it's very abstract. And now I can. explain, uh, what the generative adversarial network [00:40:00] is, uh, without any problem to somebody who have not heard about it.

And that's going to be you as well, because you're right here. So in this case of top P, the model continues generating tokens until, uh, the probability surpasses that selected threshold P.

I will skip the open source versus closed source discussion. We will, let's look at this image. So, obviously, open source, the pros are it's collaborative and transparent, community driven, customizable, flexible integration, usually free to download, right? Free? We like free, because it's free. And even if sometimes we can pay for things, and there are pros of paying for things.

Not everybody has the resources to pay for all the things, right? The cons are community based support may be slow, incomplete. Like there's no accountability. Nobody's paying for this. So whenever the community [00:41:00] decides or gets inspired to do something, they will do it. Right. Uh, less security protocols and may require users to provide their own computing power or servers.

And whereas closed source, um, they are, they are building a product, uh, it's optimized for performance and there is option for proprietary data integration and dedicated support, which in any kind of healthcare institution and basically for profit, not well. Let's separate healthcare and other institution that has like an objective even if it's not Not for profit.

You need support, right? You're not gonna be fumbling with it on your own. You're gonna call support and That's what you pay for when you pay for these things less transparency due to proprietary nature less adaptability solar slower in play implementation obviously less affordable more red tape less flexibility, so you know, choose which one [00:42:00] works for you at the moment.

Um, uh, one more word that we took. Okay, so these, these were biases and hallucinations that we addressed, but going back to biases, I have a little star here because And there is a publication by Manavi et al demonstrated that LLM's bias, demonstrated that LLM's bias manifesting in the form of discrimination of prediction were based on geographically predicted socioeconomic status.

Does this surprise me? No. And why does it not surprise me? And why is there a conscious effort needed to. Incorporate like smart anti bias gateways. What do I mean by smart? When you look at the probability of somebody, just the probability right of somebody [00:43:00] from a certain country or from some geographic area being of a certain socioeconomic status, like think of U.

S. And think of Poland. Let me take Poland. Where do you think people have higher socioeconomic status? Like, and we're not answering this. Just think in your mind, right? And that was one of the trainings. I'm digressing again, but there was a training, like anti bias training that I took for work. And it was like, imagine the, the CEO of the company talking to the secretary and then the secretary gives some documents.

And then the questions were. Which gender in your mind was the CEO? Which gender was the secretary? And like, you don't even have to like click multiple choice. You already know in your head that you have an image of a certain position in a company being certain gender, and then they ask, okay, what, uh, [00:44:00] race or like different questions like that, right?

You have this in your head based on experience. And in, in case of here, this geography, it's also based on probabilities. When you look, it doesn't mean that all people from. Poland are going to be lower socioeconomic status than people in the US, but when you look like at percentages, maybe that's not even the case, but basically, probabilistically, mathematically, when you check those parameters, that's what it's going to be.

And. That means that this data, this socio economic data, goes into the training of the model. So, is the model going to be biased? Yes, because it has biased, like, that's the data, right? So, probabilistically, statistically, these are going to be the numbers. So, as the saying goes, when you hear hoofs, think horses, not zebras.

That's what the model is going to do. And then, in a healthcare [00:45:00] setting, or like in any setting, there need to be a gate to be able to check for that, right? And sometimes this prediction is right, sometimes it's wrong, but basically this bias is also statistics from the data. And We are biased by our experience, right?

We only know what we have learned. We only have seen what we have seen. And other things we have not seen. So our, like, work, our job is to keep the mind open, right? But we will filter stuff through our experience. Here is a heart for retrieval augmented generation. So let's quickly Go through that because this is something that could minimize hallucinations and retrieval augmented generation is retrieving information from a database that later and later generating output like summaries, something based on transcripts, something based on.

Paper like whenever there is a database the thing with this rag that of course when I heard [00:46:00] about it I thought it's going to solve all the world's problems with house nations. You have to prepare the data for rag, right? So if you don't have the resources or options and this is uh more not computational, but technically advanced thing.

It's not the chat bot. It's preparing a full database and there, there are different like libraries, different things you need to do here. For example, the, the, there are four parts. There's the LLM models. There is the delivery platform of the LLM models. There is a vector database such as Chrome app or FICE or other to stores the vectors.

beddings of the local targeted data. So here we already are getting into the technical territory vector embeddings of some data, right? And, and then there are certain libraries, uh, such as long chain or Yama index that facilitate the construction and interactions of these components. So this is something, this is a framework you have to build.

And, and it's going to be [00:47:00] complex because you need to prepare the data to. be retrievable and mostly it's text. Um, we don't have a good image retrieval yet. Common tools for image generation is going to be the diffusion based models and generative adversarial networks based models. So what are the benefits and concerns of today and tomorrow?

Um, let's start with the concerns. Well, Navigating complex regulatory requirements and data security and privacy of course, cost and scalability, transparency and accountability, consent issues, right? New ethical dilemmas that we're going to be inventing as we create new tools and avoiding bias and ensuring model validity.

What are the benefits? Improved diagnostic accuracy. I don't know, [00:48:00] when I read this sentence, I always think, so, we're like, so inaccurate? We're, like, we're good, right? But we can be better. So, we can improve diagnostic accuracy, and definitely improve treatment effectiveness and patient care. We can reduce with this healthcare cost, accelerate research and discoveries, and more time for the humans to refocus on important tasks, machines, cannot do.

So I have published a podcast recently with Dr. Leah Joseph. And I think part of our discussion was, okay, when we can leverage AI to free up us, free us up from, for example, the admin tasks that I mentioned, then we can actually talk to the patient. And she's a pathologist, even though we know what the stereotype, um, of a pathologist is, of a doctor sitting in the basement looking at slides.

She is a very patient centric pathologist. And there is another one that I interviewed for the Digital Pathology Podcast, Dr. Marilyn Bui. She's a cytopathologist, so [00:49:00] she actually, um, sees patients for the procedures. But both of them were emphasizing how important it is to reconnect with the patient, to put the pathologist as part of the diagnostic and not clinical team, but basically give patients access to the pathologist so that pathologists can explain their disease because often this is something that they need to watch this podcast if you have not already, and We want to thank the Computational Pathology and AI Centers of Excellence at the University of Pittsburgh, and I promised you the Valentine Surprise, which is basically a course that you can get.

There's a QR code for you that will take you to a course about AI in pathology. How is this course built and what is it going to [00:50:00] give you? It's going to give you a framework to build your new knowledge base. on new AI knowledge that you will acquire through reading papers, going to conferences, showing up.

And this is a framework and what is going to be in there as we go through those live streams. But basically all the recordings of the full series with those papers is going to be in the course. And this course is currently costs, I think 99, but I have a code for you. And the code is 50 percent off if you use The code file 50 should give you 50 percent discount.

This is something we are building out I know I was kind of promising. Well, there's gonna be a shop with different things and this course is going to be in the shop and Today because it's valentine's there is a 50 percent discount for the course So, go ahead, check it out, uh, see what's in [00:51:00] there. So, if you, like, don't want to pay for this, this information is available on YouTube.

What I am giving you here is a structured way of accessing it, and this thing is going to be updated on a regular basis, so I'm going to be, whoever gets this course gets it forever for whatever price you get it for, and at this moment it's a discounted price, but The content of the course is going to be updated.

So, like I told you from, from the book, right? The AI chapter is outdated. I need to update it. So, it's going to be the same with those videos. And it's going to be structured, it is structured in a logical way, so that you can follow the The progression of concepts that you need to learn to be able to be AI literate and the papers that we are discussing in those free live streams are going to be part of it for whenever you need them.

There are real life examples from [00:52:00] papers. from different, from different publications about eosinophilic esophagitis. And if you look at these images, they have something in common. One of them is the tool that they were AI generated. So we have real life examples about different applications of AI. We have also deeper dive in different things like whole site repositories, is decentralized AI the future of pathology?

What are foundation models? And my favorite information retrieval, this is actually, so it's not only my videos, it's also videos that are very valuable from the internet. I curated this for you. And we have something about adjantic AI and you will see, you can see the earring test that I mentioned to you and a little bonus about glassless pathology.

Currently we have 18 lessons here regardless. That's what you decide to do. [00:53:00] I am super grateful that you are here and I appreciate you so much. Have a fantastic day and let's keep pushing the boundaries of digital pathology. Have a wonderful day and I talk to you in the next episode.