We propose and demonstrate a new phase retrieval method based on a deep neural network (DNN) structure. By inputting only one sample interferogram, measured from an off-axis holography based quantitative phase microscope (QPM), the DNN can output an accurate quantitative phase image of the sample without using a calibration interferogram, therefore significantly simplifying the measurements procedure. Importantly, our method can eliminate the need of performing phase unwrapping, therefore making it easy to achieve real-time phase retrieval in different program platforms. We used different types of cells as test samples to characterize the performance of our method, and we found that the accuracy of our DNNbased phase retrieval method is similar compared with the standard Fourier transform based phase method, while the background phase noise is reduced. Considering the experimental procedures and image processing steps are significantly simplified, we envision this new phase retrieval method will make QPM more easily accessible in bioimaging and material metrology applications in the future.