What is NVIDIA Clara?


With the recent advances of Artificial Intelligence and software solutions, the healthcare industry is seeing a dramatic change in the medical imaging workflow. Artificial intelligence and deep learning are proving to be very effective tools that can assist with disease detection, localization and classification.  These features can lead to healthcare providers saving time and money on diagnosing patients and spending more time on resources to save lives.  One of the more effective tools that we would like to address today is Clara – a platform created by NVIDIA, designed to be implemented seamlessly into the healthcare providers workflow and bring deep learning research to the field of medical imaging.

One of the key takeaways from researching the impact Clara Medical Imaging can have on the healthcare industry is the disconnect between AI research and applications in the medical field.  Much of the research on AI tools to enhance healthcare services is being done in isolation with limited datasets.  NVIDIA has created a tool to expedite data preprocessing which can then enable companies like VasoGnosis to deliver a product in a significantly faster timetable.

Cutting down the time and effort required to process and format datasets will benefit healthcare providers by getting the diagnostic information quicker and enabling the doctors and healthcare providers to spend more time focusing on the patient and treatment.  Not only will Clara speed up the process of properly formatting and labeling the data in order to train the algorithm, but the benefit of having an AI system to process large amounts of data very quickly will also broaden the variability that the AI predictor algorithm will see.  The ability to process large amounts of data will give the predictor algorithm a better representation of the population to be able to determine successfully, patients that have uncommon brain scans or diseases. 

The Clara technology includes various types of data processing software that can work with different formats of data as medical images can be created in many different formats. Clara also includes a development environment for the researcher to bring their own model architectures and run workflows. In addition to this, Clara offers many pre-trained models packaged as complete 2D/3D model applications for organ-based segmentation, classification and annotation.  Based on research done by NVIDIA, AI-Assisted Annotation increased data processing speeds by ten times current manual annotation.  These capabilities are provided by the MMAR or Medical Model Archive, which provides the environment for developing new models, storing and organizing artifacts produced during model development.

An interesting feature of the Clara technology is transfer learning.  Due to different variances in patient data, including differences in equipment used, and age of the patient, these factors can create a difficult environment for successfully training a predictive algorithm on a single dataset.  With transfer learning, some of these road-bumps can be navigated to successfully train the model.  Using a pre-trained convolutional neural network that has been trained on a large dataset can quickly overcome some of the issues that might occur if the preprocessing was done manually.  Clara Train SDK for Medical Imaging contains many different models that are pre-trained that can be used as a starting point for processing data.


  • Image Source
  • When looking at the Clara Deploy SDK service, the Clara Core Services includes the following features; DICOM Adapter, Results Service, Tensor RT Inference Server, Render Server, and Clara I/O Model.  I would like to give a brief summary of some of these features, as one or all of these would be a great enhancement to the workflow process between the healthcare provider and the software provider. Starting with the DICOM Adapter, this feature acts as the bridge between the Healthcare provider and service provider.  This service will enable the user to configure a method for sending the medical images directly to the Clara Deploy platform to initialize the pipeline within Clara.  Once the pipeline has completed its execution the output is sent back to the healthcare provider via the DICOM adapter.  This creates a seamless effect of data transfer from healthcare provider to software provider and back again. 

Results services is a feature of the Clara Deploy SDK that can monitor the process of data transferring from the healthcare provider to software provider. This feature will track results generated by the pipeline of the Clara Deploy SDK to streamline the output sent to the healthcare provider or use Clara’s built in visualization server for further processing.  The Tensor RT Inference Server is a containerized inference server that can enable features such as multi-GPU support, concurrent model execution, managed model repository in order to speed up the process of analyzing the data.  The Render Server allows for Clara Deploy utilize a 3D visualization of the medical imaging data, before and after it has been processed.  This feature would enable Clara users to give a clearer picture for the program to better understand the provided medical image.  

One of the hindrances of training a robust AI algorithm in the medical field is obtaining enough data from many different sources.  Hospitals and medical institutions are often reluctant to share data, the main reason being because patient data is private data that hospitals do not want to share.  The newest development of Clara Train SDK with the Federated Learning feature can now combat this issue.  Federated Learning weights data from each source which is then shared with the model, ensuring that privacy is preserved, and inversion exposure is reduced.  When using the Clara Train SDK, the centralized server acts as a facilitator for the overall federated training when using multiple clients.  This feature makes scalability increasingly more seamless when using Clara Train SDK.


The server manages the model training progress and distributes the model to each of the different clients.  Then the model training will happen locally at each client, so the server does not need to access the training data.  This will keep the private data for each client from ever leaving the client but enable to model to come to the client in order to learn and then the local model will be sent to the server. The model then aggregates weights from each local client model to update the general model. This federated learning model provides benefits for each participant, a robust centralized model that has learned from various clients, and a more accurate local model, which has trained just with one client.  The federated learning model was able to achieve similar performance results to a model trained on a centralized data set. 

Both the federated learning model and the Data-centralized model achieved an 82% correct prediction score. While the federated model was able to achieve that score over the same number of epochs as the centralized data, the federated model did increase its rating at a slower rate.  To ensure the privacy and security of the patient data, the client must first submit intention to participate in the federated learning to the server, when the server receives the request it will authorize the client and send the client the model along with a token.  The client will train the model locally and send the updated model back to the server using the token.  If the federated learning was successful, the contribution will be accepted by the server and the model will be updated.


Based on the features that are included with Clara, this service would be of great value to organizations providing AI technologies for medical image processing.  Clara greatly enhances the speed and capability of obtaining and training models based off the data.  Many pretrained models are available to users to speed up the process of training a model.  The federated learning feature can greatly speed up the process of training a model, and create an easier approach for obtaining data from healthcare institutions by providing secure methods of sending the model to the client to train as opposed to obtaining the data from the client in order for the model to train. This can give a healthcare institution the peace of mind that they would not have to share private data, and at the same time give them the benefit of having an AI assisting them with diagnosis and treatments of their patients.



This blog was written based off of information provided from  More blog posts on Clara and features associated with Clara can be found at NVIDIA’s website