Development and Validation of Fully Automatic Deep Learning-Based Algorithms for Immunohistochemistry Reporting of Invasive Breast Ductal Carcinoma
Summary
Paper digest
What problem does the paper attempt to solve? Is this a new problem?
The paper aims to address the challenge of accurately assessing IHC-stained tissue samples in breast cancer diagnosis through the development and validation of fully automatic deep learning-based algorithms . This problem is not new, as manual interpretation of IHC stains is known to be laborious, time-consuming, and subject to inter-observer variability, especially with the increasing number of breast cancer cases and the limited availability of pathologists . The paper proposes a novel semantic segmentation-based training model to improve scoring accuracy and robustness in breast cancer IHC analysis, demonstrating superior performance compared to conventional methods .
What scientific hypothesis does this paper seek to validate?
This paper aims to validate the hypothesis that the developed deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma can accurately predict HER2 scores for each region of ROI and the entire slide using K-fold cross-validation with a target Kappa Score (Quadratic) . The study focuses on training a Random Forest model on whole-slide images (WSI) data to predict HER2 scores and validate the model's performance against other WSI images . The algorithms are designed to provide accurate predictions for HER2 scores based on the analysis of stained tissue regions and nuclei segmentation . The performance baseline set for the algorithm on validation slides was 85 +/- 5%, and the nuclei segmentation and classification algorithm successfully met this performance baseline .
What new ideas, methods, or models does the paper propose? What are the characteristics and advantages compared to previous methods?
The paper proposes several innovative ideas, methods, and models for the automatic reporting of invasive breast ductal carcinoma using deep learning-based algorithms :
- Deep Learning-Based Algorithms: The study involves the development and validation of fully automatic deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma. These algorithms utilize advanced techniques such as semantic segmentation for membrane and nuclei detection .
- Model Architectures: The paper introduces various model architectures such as EfficientNet-UNET for semantic segmentation in HER2 image patches. This model combines EfficientNet-B4 architecture with the UNet architecture to accurately segment membrane and nuclei structures in HER2 images .
- Training Data and Cyclic Training: The models are trained using de-identified, digitized whole-slide images of IHC breast tissue obtained from multiple hospitals/laboratories. The training data includes manually annotated patches from expert pathologists. The second model is trained using predictions from the first model in a cyclic manner, with six cycles of training performed .
- Post-Processing Modules: To address scoring issues caused by artifacts and defects in tissue, the algorithms incorporate post-processing modules. One approach involves clustering tumorous nuclei based on their spatial location in tissue to improve performance .
- Performance Evaluation: The paper evaluates the performance of the models by comparing predicted results with ground-truth images. The accuracy and F1 scores for nuclei detection are reported to be in the range of 75-85%, indicating satisfactory model performance .
- Feature Extraction and Classification: The models extract morphological features at each region of interest (ROI) level and for the entire slide to classify HER2 scores. Features such as color histogram, skewness of the histogram curve, and nuclei to membrane ratio are considered for classification .
- Knowledge Learning: The study incorporates a teacher-student architecture for knowledge learning, which involves a survey on this approach to enhance the model's learning capabilities .
- Data Availability: The datasets used in the study were collected through internal agreements with hospitals/laboratories. While the data is not publicly available due to data-sharing restrictions, it can be accessed from the authors upon reasonable requests and permissions .
- Institute Collaboration: The study involves collaboration with multiple institutes for data collection and validation. The development and learning phase were conducted in collaboration with Kasturba Medical College (KMC), Manipal, and the validation phase included de-identified slides from various centers .
These innovative ideas, methods, and models collectively contribute to the advancement of automated immunohistochemistry reporting for invasive breast ductal carcinoma, aiming to improve diagnostic accuracy and efficiency in pathology analysis. The paper introduces a novel semantic segmentation-based training model for breast cancer IHC scoring, offering several characteristics and advantages compared to previous methods :
- Training Approach: Unlike transfer learning methods that utilize pre-trained models, the proposed approach involves training a Convolutional Neural Network (CNN) from scratch specifically for IHC staining analysis. This training strategy aims to capture fine-grained details and spatial relationships within stained regions, enhancing scoring accuracy and robustness.
- Performance Evaluation: The study conducted a rigorous multi-centric trial involving multiple institutions, laboratories, and pathologists to evaluate the model's performance and robustness. The results were compared against manual scoring by experienced pathologists, demonstrating the superiority of the semantic segmentation-based training model over conventional image processing techniques, SVM/RF-based approaches, and transfer learning methods.
- Accuracy and Reliability: The proposed model achieved higher accuracy, reduced inter-observer variability, and better adaptability to variations in staining patterns, slide preparation techniques, and scanner characteristics. This indicates improved accuracy and robustness in breast cancer diagnosis and treatment decision-making.
- Generalizability: The multi-centric trial ensured the algorithm's reliability and generalizability across different settings, showcasing its potential for widespread clinical implementation in breast cancer diagnosis.
- Model Stability: The model's stability and robustness were validated on the test set, with an Area Under the Curve (AUC) of 0.85 for tumors and 0.92 for other classes, indicating well-ranked predictions and good prediction quality.
- Data Availability: While the datasets used in the study are not publicly available due to data-sharing restrictions, they can be accessed from the authors upon reasonable requests and permissions, ensuring transparency and reproducibility in research .
Overall, the semantic segmentation-based training model offers improved accuracy, reliability, and generalizability in breast cancer IHC scoring compared to conventional methods, showcasing its potential for enhancing diagnostic outcomes in pathology analysis.
Do any related researches exist? Who are the noteworthy researchers on this topic in this field?What is the key to the solution mentioned in the paper?
Several related researches exist in the field of developing and validating deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma. Noteworthy researchers in this field include S.J., A.M., G.S., S.M., R.K., Dr. Kanthilatha Pai, Dr. Brij Mohan Kumar Singh, Dr. Anil Betigeri, Dr. Vani Verma, and Dr. Madhavi Pai . The key to the solution mentioned in the paper involves the development and validation of fully automatic deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma. The study involved the acquisition of de-identified, digitized whole-slide images of IHC breast tissue for ER, PR, HER2, and Ki67 from multiple hospitals/laboratories, followed by the training and validation of AI-based decision support systems for IHC breast tissue analysis . The algorithms developed in the study demonstrated high accuracy and performance in nuclei detection, stain classification, and clinical evaluation for ER, PR, and Ki67 markers in invasive ductal breast carcinoma grading .
How were the experiments in the paper designed?
The experiments in the paper were designed in a systematic manner involving two main phases:
- Phase 1: Development and Learning Phase:
- This phase was conducted in collaboration with Kasturba Medical College (KMC), Manipal.
- Retrospective samples previously prepared by KMC, Manipal were collected for research purposes.
- Approximately 238 slides each of ER, PR, HER2, and Ki67 were de-identified and converted to digital TIFF images.
- The Institutional Ethics Committee approved the use of these samples.
- A total of 920 slides were available for phase 1 after rejecting 32 slides due to poor quality.
- Phase 2: Multi-centric Validation Study Phase:
- The developed DSS was validated on de-identified slides from four different centers: KMC (Manipal), KMC (Mangalore), Sikkim Manipal University (SMU), and Neuberg Anand Labs (NAALM).
- About 569 cases were targeted in this study, with 663 cases received in total.
- 94 cases were discarded due to bad slide quality.
- All samples were anonymized and the study was approved by each institute's Ethical Committee.
The experiments involved the validation of the developed algorithms on a diverse set of samples from multiple institutes, ensuring a robust evaluation of the deep learning-based models for immunohistochemistry reporting of invasive breast ductal carcinoma.
What is the dataset used for quantitative evaluation? Is the code open source?
The dataset used for quantitative evaluation in the study on deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma consists of 152 manually annotated patches from two expert pathologists . The code for the algorithms developed in this study is not mentioned to be open source in the provided context .
Do the experiments and results in the paper provide good support for the scientific hypotheses that need to be verified? Please analyze.
The experiments and results presented in the paper provide strong support for the scientific hypotheses that needed to be verified. The study involved the development and validation of fully automatic deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma . The algorithms were trained and validated using a dataset of whole-slide images (WSI) to predict HER2 scores for different regions of interest (ROI) and entire slides . The study utilized K-fold cross-validation with Kappa Score to assess the model's performance .
The paper details the methodology used for data acquisition, which involved obtaining de-identified, digitized whole-slide images of IHC breast tissue from multiple hospitals and laboratories for ER, PR, HER2, and Ki67 analysis . The study was conducted in two phases: a development and learning phase, followed by a multi-centric validation study phase . The experiments included the collection of retrospective samples, training the models, and validating them on de-identified slides from different centers .
The results of the experiments demonstrated the effectiveness of the deep learning-based algorithms in nuclei detection, stain classification, and overall performance evaluation . The algorithms showed high accuracy, precision, recall, and F1 scores for ER, PR, and Ki67 stained slides . The models achieved good agreement with ground truth scores and pathologists' annotations, indicating reliable performance . Additionally, the algorithms exhibited satisfactory performance in terms of nuclei segmentation and stain classification, meeting predefined performance baselines .
Overall, the experiments conducted in the study, along with the detailed results and performance metrics, provide robust evidence supporting the scientific hypotheses and the effectiveness of the deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma.
What are the contributions of this paper?
The paper acknowledges the contributions of various individuals and entities:
- The management at Applied Materials supported the work in this domain .
- Pathologists like Dr. Kanthilatha Pai, Dr. Brij Mohan Kumar Singh, Dr. Anil Betigeri, Dr. Vani Verma, and Dr. Madhavi Pai provided IHC scores annotations for the study .
- Authors S.J., A.M., G.S., S.M., and R.K. were involved in planning, designing experiments, data acquisition, code writing, validation, results collection, and performance analysis .
- The study involved collaboration with Kasturba Medical College (KMC), Manipal, for the collection of retrospective samples and the development of a diagnostic solution .
- The paper highlights the development and validation of fully automatic deep learning-based algorithms for immunohistochemistry reporting of invasive breast ductal carcinoma .
- The AI-based algorithm aimed to assist pathologists in well-informed and evidence-based reporting, improving the overall turn-around time for better patient outcomes .
What work can be continued in depth?
To further enhance the model's performance in accurately segmenting membrane and nuclei structures in HER2 images, further work can be done to maximize the Intersection over Union (IoU) values for membrane and nucleus detection. Currently, the IoU values obtained are 38.75% for membrane detection and 57.26% for nucleus detection . By conducting additional experiments and optimizations, these values can be improved to achieve more precise segmentation results .
Moreover, the model can be refined by introducing more non-linear activation-based depth in the UNet model to enhance the understanding of features and make the model more generic, especially in dealing with variations in color and width of the membrane and visibility of nuclei in the image . This approach can help the model better capture the intricate details of the membrane and nuclei structures, leading to improved segmentation accuracy.
Additionally, to address challenges related to ROI segmentation, a generic multi-level multi-tissue segmentation model inspired by the Hook-Net architecture can be further developed. This model should work effectively with all four stain types (ER, PR, Ki67, and HER2) and be robust across multiple data sources and scanners, ensuring accurate classification of different tissue regions . By refining this segmentation model, the algorithm can achieve more precise and reliable segmentation of various tissue structures in breast tissue images.