Welcome and thank you for joining this presentation. My name is Siri Khalsa. I'm a fifth year neurosurgery resident at the University of Michigan. I'm here to present our work on using machine learning to automatically provide a diagnosis for intraoperative frozen sections during brain tumor surgery. This research was supported by an NIH T32 grant. I'm going to start with our research problem and overall objective. Consider this patient who presents with a rim-enhancing brain mass. The histologic identity of this mass will be important for surgical decision making. For example, if the lesion appears to be normal or reactive brain tissue, then the neurosurgeon may not yet have accessed the actual lesion or the lesion may not be neoplastic at all. However, if lymphoma is encountered, then further resection is typically not indicated as lymphomas are treated with chemotherapy and radiation. However, if a high-grade glioma or a large isolated metastatic adenocarcinoma are encountered, then typically maximal safe resection would be pursued. So overall, intraoperative diagnosis is important for surgical decision making. Now, one of the problems we face during brain tumor surgery is access to a board certified neuropathologist for intraoperative consultation. In the United States, there are only approximately 56% as many neuropathologists as there are hospitals performing brain tumor surgery. This has led several groups to ask the question as to whether there is a way that we can use computer algorithms and machine learning to automatically provide a diagnosis. That brings us to our overall objective, which is illustrated here. That is to be able to take a standard intraoperative frozen section, which are prepared by technicians at essentially any hospital, and have a computer algorithm analyze the scanned slides to provide a whole slide diagnosis during surgery. To address related work on this issue, Nature Medicine recently published this work by our lab, a project led by my colleague Todd Holland, who used deep machine learning to automatically provide a diagnosis on intraoperative specimens that were analyzed by a specialized microscope that harnessed a stimulated Raman scattering to produce permanent histology-grade images as shown here. This machine learning algorithm was able to correctly classify 13 different tissue types with an overall accuracy of 94.6%. The work in this presentation builds on these methods, but instead of analyzing slides generated with a specialized microscope, our objective again is to analyze standard traditional intraoperative frozen sections, which can be generated by technicians at any hospital. Now to describe our method for training a computer algorithm to automatically diagnose whole frozen section slides. The general framework for the method is illustrated here, but I will describe this in parts. The first step is to train a deep machine learning algorithm called a convolutional neural network. The convolutional neural network is an interconnected network of image filters that's able to adapt and learn how to diagnose an image simply by being provided a large number of training examples. Here are three images that are labeled as normal brain, glioblastoma, and metastasis, but in reality, hundreds of thousands or millions of images are typically needed to train a convolutional neural network adequately. Now, once that training of the network is completed, which is a one-time step, the network is now ready to automatically classify new images that it has never seen before. Here we have a very large whole frozen slide, which is obtained during brain tumor surgery, and the diagnosis of this slide is unknown at this time. This slide is then divided into hundreds or thousands of smaller fields of view, each of which is small enough to be analyzed by the neural network. The neural network then produces a diagnosis for each of these small fields of view, and then these classified fields of view are summated into a single image. Now, to go into this method in more detail, the first aim is to train this convolutional neural network to classify individual small fields of view. The reason this is necessary is because whole slides themselves are tens of thousands of times too large at full resolution to be analyzed by modern computers. Therefore, they have to be broken up into smaller pieces for analysis. We generated approximately 2 million of these smaller fields of view from a large collection of different frozen sections. Our aim at this stage is to focus on four different diagnoses for supertentorial brain tumors in adults. So, consider this whole frozen section slide on the left, which is obtained during a brain tumor operation. Our algorithm divides it into four parts. The first part is called the frozen section. The second part is called the frozen section. The third part is called the frozen section. The fourth part is called the frozen section. The fifth part is called the frozen section. This whole section slide on the left, which is obtained during a brain tumor operation. Our algorithm divides it into thousands of smaller fields of view, each of which is classified by the convolutional neural network. So, you can see on the right that the algorithm thinks most of these fields of view are carcinoma, which is blue on the legend. Now, the second aim is to take one of these class maps that is generated by the algorithm and synthesize it into a single whole slide diagnosis. The way that this whole slide diagnosis is produced is essentially by a voting mechanism, where the most encountered field of view wins. In this case, metastatic carcinoma is the most plentiful, and the whole slide diagnosis will be metastatic carcinoma. I described the method for training a convolutional neural network and using that network to automatically diagnose whole frozen slide images obtained during brain tumor surgery. Here are our current results. To start with some examples, the left shows a large frozen section. You can see some artifacts, including a fiber and a bubble. An example field of view looks like this. As you can see, there are a malignant appearing cells and also astrocytic processes. The algorithm produced this class map. Most of the fields of view are light blue, which corresponds to high-grade glioma. The summation yielded high-grade glioma as the automated whole slide diagnosis. The board-certified neuropathologists agreed. This was a high-grade glioma on frozen and confirmed to be glioblastoma on permanent. Another example. At the top is a whole frozen section, and at full resolution, you can see a dense, round, malignant-appearing nuclei. The automated class map produced by the algorithm is shown here. This is summated to be lymphoma automatically by the algorithm. At the time of frozen, the neuropathologist said favor lymphoma, and lymphoma was later confirmed on permanent section. In order to test the algorithm, we applied it to 40 different whole slides, none of which were used as training images. The ground truth diagnosis is on the horizontal axis, and the program's output is on the vertical axis. So any cases that are along the green diagonal are correct. The algorithm correctly diagnoses 11 out of 12 carcinoma cases, 9 out of 10 high-grade glioma cases, 6 out of 7 lymphomas, and 11 meningiomas, yielding an overall accuracy of 92.5% for these frozen whole slides. In summary, interoperative histologic diagnosis is crucial for decision-making in the operating room during brain tumor surgery. However, many centers lack access to adequate interoperative neuropathology consultation. Our proposed solution to this was to develop a deep machine learning algorithm to automatically provide a diagnosis for whole, frozen section slides during brain tumor surgery. At this time, limited to high-grade glioma, lymphoma, metastatic carcinoma, meningioma, and non-neoplastic tissues. Our next steps are to complete a prospective multicenter validation, which is currently underway. Thank you again for listening to this presentation. I would welcome an email at this address with any questions or concerns you may have. Thank you very much.

machine learning

brain tumor

neurosurgery

diagnosis

convolutional neural network