Histopathology of breast tissue typically involves labelling of a tissue section with the Hematoxylin and Eosin (H&E) counter stain. The contrast generated by the stain allows pathologists to extract both diagnostic and prognostic information (such as malignancy, grade and stage) during microscopic examination. Despite being the most frequent assessment method in use today, this procedure provides limited information and is subject to observer bias due to its qualitative nature. While commercial tissue slide scanners promise to improve the throughput of this method in the coming years, quantitative evaluation of tissue remains a challenge. In this work, we propose a method for simultaneously extracting color bright field and phase images of stained breast tissue biopsies using Spatial Light Interference Microscopy (SLIM) coupled with an RGB camera. The amplitude information allows standard qualitative histopathology while the quantitative phase information can be used, after normalizing for the staining, to extract physical markers characterizing patient health. We demonstrate this by imaging an H&E stained tissue microarray and showing that the normalized phase values can be exploited to classify benign and malignant tissue. Furthermore, we demonstrate that these images allow quantitation of tumor-adjacent collagen structure - an important prognostic marker for breast cancer. Extraction of these biomarkers requires measurement of the tissue optical path-length map which is not available in standard tissue evaluation. Our method, therefore, expands on the current diagnostic pipeline by complementing standard histopathology with quantitative tissue biomarkers, all obtainable in a single scan.