In this study, we proposed a multi-space-enabled deep learning modeling method for predicting Oncotype DX recurrence risk categories from digital mammogram images on breast cancer patients. Our study included 189 estrogen receptor-positive (ER+) and node-negative invasive breast cancer patients, who all have Oncotype DX recurrence risk score available. Breast tumors were segmented manually by an expert radiologist. We built a 3- channel convolutional neural network (CNN) model that accepts three-space tumor data: the spatial intensity information and the phase and amplitude components in the frequency domain. We compared this multi-space model to a baseline model that is based on sorely the intensity information. Classification accuracy is based on 5- fold cross-validation and average area-under the receiver operating characteristics curve (AUC). Our results showed that the 3-channel multi-space CNN model achieved a statistically significant improvement than the baseline model.