Combining the proficiency of AI with the capabilities of the Nvidia Jetson set resulted in building an edge system that uses Artificial Intelligence (AI) in histopathology image validation.
With microscope image processing performed by AI, many manual tasks could be eliminated or made much easier.
Healthcare and medical sciences have been a field of data correlations and statistics since the very beginning, even if researchers had not identified the matter as such. Modern medical sciences generate tremendous amounts of data in various forms – from numeric to images to 3D scans.
According to IBM estimations, up to 90% of modern medical data takes the form of images, usually gathered by radiology and histopathology. With the growing need for healthcare in an aging population and the development of new diagnostic technologies, this already massive amount of data will only grow.
This creates fertile ground for Artificial Intelligence, where image recognition and image processing solutions have witnessed unprecedented development. The trend of using image recognition technology in healthcare has already been recognized. Accenture estimates that the savings from using AI-powered solutions in healthcare will accumulate to $150 billion annually in the US alone by 2025. The trend has already been seen in healthcare startup acquisitions, which have risen from $600m in 2014 to an estimated $6.6 bln in 2021.
This text covers:
- What is edge AI
- How edge AI and healthcare go well together
- AI on edge use case using Nvidia Jetson Nano
What is edge AI
Running an AI-based solution in a powerful data center is currently the most common way of implementing this type of technology. But it is neither possible nor reasonable to use this model in every situation, especially in healthcare. Running AI in a cloud environment comes with some limitations, including:
- Access to the internet – while in developed countries it is not that challenging, access to the internet can be limited in more remote locations or during natural disasters, when the need for field hospitals rises.
- The cost of transfer – another challenge related to transferring data to the cloud. Images used in healthcare are saved in non-loss formats to keep as much detail as possible. Running them there and back again would create costs that would be easy to avoid if the image was processed on-site.
- The delays – transferring large files through the internet two-ways results in a significant delay. When analyzing a single case, the delay can be skipped, but the delay stacks when processing every patient, causing the analyst to just wait in front of the computer for many hours. And the patient just needs to be, well… patient.
Transferring the AI system to the end device is a reasonable solution to this problem. This approach, called edge computing AI, focuses on implementing neural networks on the end devices (on the edge of the network). This is but one reason behind the dynamic healthcare mobile app development – a lot of computing and analysis is done on-device rather than in the cloud.
According to Statista data, the market for edge AI processors is growing rapidly, from $0.34 billion in 2019 to $1.5 billion in 2023. This growth is fueled mostly by the fact that neural networks can deliver superior work while using a relatively small amount of computing power. The network is trained and developed off-site, for example, in the vendor’s data center. More about edge AI can be found in our AI trends 2020 report.
Why healthcare and edge AI go well together
Running neural network-based solutions on edge devices bring numerous benefits, including:
Transferring The Dull Work From Doctors To Machines
Artificial intelligence can support doctors and medical staff in more or less mundane tasks that require dull and repetitive work. An AI solution shines in automating workflows in administrative tasks, bringing savings in both working hours and money alike.
The average US nurse spends up to 25% of her work time on administrative tasks. This can be reduced by applying Robotic Process Automation. Natural Language Processing (NLP)-based assistants like chatbots used to perform initial patient interviews is an off-the- top-of-the-head example. Building a comprehensive Electronic Health Record system to be used by AI is the first step toward building a next-gen AI-powered healthcare solution.
Providing Early Diagnostic Services
The challenge with early diagnosis is mainly in access to a specialist. The problem is seen in both developing countries and developed ones alike. In less developed countries the problem is in basic access to medical infrastructure and doctor services. More developed countries struggle with queues and resource consumption in a healthcare system that needs to serve an aging population.
Early diagnosis is one of the key strategies to fight diseases, including the most severe, like cancer. With the ability to perform multiple tests swiftly and on a large scale, AI solutions can deliver early diagnosis at scale. Also, implementing AI-powered diagnostic tools can bring early screening to a higher level.
A good example comes from Google supporting the early diagnosis of diabetic retinopathy in India with AI-powered tools.
Improving Existing Processes By Providing Robotic Support
As with every institution, the healthcare system is prone to time and resource leakage. This can take the form of multiple hidden inefficiencies that stack up with the scale to tremendous amounts.
A good example comes from mislabelled data coming from histopathology and radiology diagnosis. Usually, these come from human error, when the specialist brings the best he or she can and the mistake is later verified by a doctor. Usually, the best one can do is either repeat the diagnosis process or return to the lab and ask for an update. This takes time.
A Tooploox-designed tool that uses an edge-AI system aims to tackle this type of challenge with a focus on the microscopic slides used in histopathology.
Source: Virtum web interface for internal usage.
Zoomed in image of human tissue as seen in the Virtum interface. There is a foreground mask overlaid which allows for extracting the most informative parts of the image for further analysis. The mask can originate from manual annotation or be automatically created by using an AI solution.
AI at the edge use case – histopathology analysis on Nvidia Jetson
Microscope slides can be defective in various ways, it could be a mistake during the staining procedure as performed to make specific tissues more visible, it could be the improper collection of the sample, an issue with the sample itself, or the sample could have been marked with a pen by a pathologist.
Such artifacts decrease the reliability of both the analysis made by AI models and diagnoses made by humans.
The delivered technology was derived from the Virtum platform, a comprehensive toolset to analyze and process large-scale images with a focus on the microscopic. Virtum is a cloud-based tool, but some functionalities can be transferred to an edge device by using the Nvidia Jetson platform.
Professional microscopy slide scanner by Zeiss. It can scan up to 100 slides in one go. It’s a laborious task to analyse their quality manually.
What Is Nvidia Jetson Nano
Nvidia Jetson is a small computer used to design, develop and prototype embedded systems for IoT and edge healthcare devices alike. The device can be compared to Arduino or Raspberry Pi solutions and is backed by Nvidia. The computer is comparable in size to a package of coffee, so it can be easily added to existing setups or transported. The technology is repeatable, so there is no problem adding this component anywhere it is required.
Why Scan Quality Validation Matters
Histopathology scans require high proficiency to work with. But despite the skills of the people working with them, there are multiple types of errors which can occur in slides, including:
- out of focus regions in the slide
- regions containing ink
- tiling effect due to stitching
- folded or torn regions
- completely missed regions during scanning
The development of AI for health requires the digitalization and building of accessible and trustworthy datasets. Any of the mistakes listed above can hamper the effectiveness of algorithms and AI-based automation. Also, a non-perfect slide in the dataset can significantly influence the whole network’s effectiveness.
Last but not least, this solution has the potential to be further developed and optimized to analyze slides in real-time. By this, the solution would aid physicians in delivering a more accurate diagnosis or provide them with more information right when they need it.
Automated quality control right after the scanning has finished helps to prevent cluttering in a database and gives the scanner’s operator a chance to redo the scan or take other adequate actions if and when possible.
In our solution, we perform the segmentation of microscope slide scans using a fully convolutional neural network converted to TensorRT. As we’re using a sliding window approach, we can run inference on arbitrarily large images even on Jetson Nano 2GB. The largest image presented in the demo has ~3MP, inference on a single-window (256x256px) takes on average only 140 ms.
Here’s the demo:
Gains
This device can be used to support the work of histopathology workers who annotate slides. When looking for a swift answer on if their work is done according to requirements, they don’t waste time on exchanging slides with physicians and returning to a job that was already done.
It is possible to add the device to nearly every existing setup in order to aid in either daily work or in building new datasets. After a slight bit of tuning, the same solution can be used in radiology or similar tasks requiring markings on large images.
The solution is not a core part of the Virtum platform but can bring significant synergies when used in conjunction. Building a collection of properly marked annotated slides is fundamental when building an AI-based tool or automating image-related workflows.
Summary
The Nvidia edge AI-based solution shows how implementing single-step automation can bring potentially immense savings in time and resources. Also, it is proof that edge AI is the go-to strategy for healthcare, especially when aiming to save on bandwidth and transfer times as well as prepare better input for cloud-based solutions.
If you wish to talk more about this solution or the possibilities that edge AI brings to your institution, don’t hesitate to contact us now.
Read more our healthcare-related content:
