The rise of deep learning (DL) framework and its application in object recognition could benefit image-based medical diagnosis. Since eye is believed to be a window into human health, the application of DL on differentiating abnormal ophthalmic photography (OP) will greatly empower ophthalmologists to relieve their workload for disease screening. In our previous work, we employed ResNet-50 to construct classification model for diabetic retinopathy(DR) within the PACS. In this study, we implemented latest DL object detection and semantic segmentation framework to empower the eye-PACS. Mask R-CNN framework was selected for object detection and instance segmentation of the optic disc (OD) and the macula. Furthermore, Unet framework was utilized for semantic segmentation of retinal vessel pixels from OP. The performance of the segmented results by two frameworks achieved state-of-art efficiency and the segmented results were transmitted to PACS as grayscale softcopy presentation state (GSPS) file. We also developed a prototype for OP quantitative analysis. It’s believed that the implementation of DL framework into the object recognition and analysis on OPs is meaningful and worth further investigation.
Retinal changes on a fundus image have been found to be related to a series of diseases. The traditional retinal image quantitative features are usually collected by various standalone and proprietary software which results in variabilities in feature extraction and data collection. Based on our previously established web-based imaging informatics platform to view DICOMized and de-identified fundus images, we developed a computer aided detection structured report (CADe SR) to capture some of the quantitative features on fundus images such as arteriole/venule diameter ratio, cup/disc diameter ratio and to record several lesions such as aneurysms, hemorrhages, neovascularization and exudates into different regions based on known research and clinically related templates such as Early Treatment Diabetic Retinopathy Study (ETDRS) 9 Region Map and four Region Map. In this way, the location patterns of the above lesions as well as morphological changes of anatomy structures could be saved in SR for further radiomics research. In addition, an on-line consultation tool was developed to facilitate further discussion among clinicians and researchers regarding any uncertainty of measurements. Compared with the present workflow of utilizing standalone software to obtain quantitative results, qualitative and quantitative data was acquired by the CADe SR directly, which will provide researchers and clinicians the ability to capture findings and will foster future image-based knowledge discovery researches.