AI in Medical Imaging
Artificial Intelligence is now a word that is present in our day-to-day lives, perhaps due to the incredible advances in language processing accomplished by the famous ChatGPT. In the field of medical imaging, AI has increasingly been used to segment medical images as a way to aid healthcare decisions and diagnosis, namely in modalities such as CT or MRI. Overall, deep architectures such as Convolutional Neural Networks are applied to extract meaningful information from medical images, oriented for different objectives such as classification or segmentation. The former deals with identifying conditions, while in the latter, inference results show segmentation masks that once applied on top of the images denote or delimit important organs/sections. But how can we bring these segmentation masks to be visualized in a PACS setup?
At BMD Software we use an approach that combines DICOM SEG with Overlay Injection in order to show segmentation results, supporting an AI information flow from Dicoogle until the image is visualized in Viewer, and standardizing inference results for different AI providers. A contribution from BMD Software in the MONAI Label open-source project further accentuates this idea by adding support for DICOM SEG retrieval in inference (https://github.com/Project-MONAI/MONAILabel/releases/tag/0.8.0).
DICOM SEG
DICOM SEG is a somewhat recent extension of the DICOM protocol that given its nature it effectively leverages the application of Machine Learning or Artificial Intelligence in medical images. It specifies how information about a segmentation is stored, including attributes such as:
- The number of segments encoded.
- The associated label name, the algorithm that executed the segmentation and its type for each segment.
- The referenced medical image IDs, that associates a segmentation with an original DICOM image. All the masks of every segment are stored contiguously in the file in the Pixel Data field, working as a type of multiframe object. Note that the original DICOM image pixel data is not available in the SEG format.
By using DICOM SEG, we can use Dicoogle as a common infrastructure to store DICOM and segmentation files. This way, we leverage normal DICOM operations such as search, retrieve or store that can also be applied for DICOM SEG files. As in the era of Big Data, and specifically for the medical imaging area where large quantities of images are being collected every day, this is especially important to help to find the information that effectively helps diagnosis in the most efficient way. If you are using common research image formats to store segmentations, such as NIFTI or NRRD, there are several tools that can be used to convert these types into the DICOM SEG standard, such as Pydicom (https://pydicom.github.io) or dcmqi (https://github.com/QIICR/dcmqi).
PACScenter Viewer and Dicoogle are designed in a way that enables different AI providers to be easily integrated. This is why it is important to converge to a single standard format, as otherwise it would be difficult to deal with different formats in the frontend.
Overlays Injection
When a DICOM SEG arrives at the Viewer, it is necessary to process the file metadata and pixel data to be shown on top of their corresponding images. Here, a wrapper is applied that converts from DICOM SEG to DICOM Overlays. PACScenter Viewer currently supports the visualization of DICOM Overlays, which allows overlapping binary masks, encoded in the original DICOM files, on top of the original image data. Although DICOM Overlays have their own associated groups of metadata, the goal of adding relevant information on top of the original image is maintained across them and DICOM SEG’s, which led to an approach that combined the two of them.
Figure 1: Application of the deepedit model to a given image.
In the figure above, we see an example of the application of one of the available models in MONAI Label, the deepedit model, to an image, which shows the full flow of information that occurs in PACScenter until the segmentations are visualized in Viewer, with Dicoogle as a powerful tool that mediates all these communications.
If you are trying to leverage AI for medical images, we at BMD Software have available components that are easily prepared to integrate any model and show their segmentations/annotations, which allows us to take advantage of the multiple image processing capabilities in Viewer and, therefore, effectively using AI as a tool to increase productivity in diagnosis.