Breast cancer is one of the most common malignant tumor in females. Adjuvant chemotherapy is a common method of breast cancer treatment, while not all patients will benefit from the treatment. The purpose of this study is to predict prognosis of breast cancer and stratify patients with high risk of recurrence by radiomic analysis based on dynamicenhanced magnetic resonance imaging (DCE-MRI) and gene expression data. We performed this study in three steps. First, a retrospective single-institution cohort of 61 patients with invasive breast cancer was enrolled. We extracted quantitative imaging features depicting tumor enhancement patterns and screened for those that were potentially prognostic for survival. Multivariate Cox regression analysis showed that image feature of inverse difference was independently associated with recurrence-free survival (RFS) with P value of 0.0371. Second, we built a regression model with 74-gene signature from 87 patients whose MRI and gene expression data were available for associating image features that is related with breast cancer prognosis identified in the first dataset. Finally, we validated the prognostic value of the established signature by applying it on public available genomic data sets. Using the 74-gene signature in the independent validation set, 1010 patients were divided into two groups to stratify patients for RFS (log-rank P=0.011) and overall survival (OS, log-rank P=0.029) among different survival risk levels. Our results showed that imaging features representing tumor biological characteristics would be valuable in predicting prognosis of breast cancer.