Even though the clinical consequences of a missed colorectal cancer far outweigh those of a missed polyp, there has been little work on computer-aided detection (CADe) for colorectal masses in CT colonography (CTC). One of the problems is that it is not clear how to manually design mathematical image-based features that could be used to differentiate effectively between masses and certain types of normal colon anatomy such as ileocecal valves (ICVs). Deep learning has demonstrated ability to automatically determine effective discriminating features in many image-based problems. Recently, residual networks (ResNets) were developed to address the practical problems of constructing deep network architectures for optimizing the performance of deep learning. In this pilot study, we compared the classification performance of a conventional 2D-convolutional ResNet (2D-ResNet) with that of a volumetric 3D-convolutional ResNet (3D-ResNet) in differentiating masses from normal colon anatomy in CTC. For the development and evaluation of the ResNets, 695 volumetric images of biopsy-proven colorectal masses, ICVs, haustral folds, and rectal tubes were sampled from 196 clinical CTC cases and divided randomly into independent training, validation, and test datasets. The training set was expanded by use of volumetric data augmentation. Our preliminary results on the 140 test samples indicate that it is feasible to train a deep volumetric 3D-ResNet for performing effective image-based discriminations in CTC. The 3D-ResNet slightly outperformed the 2D-ResNet in the discrimination of masses and normal colon anatomy, but the statistical difference between their very high classification accuracies was not significant. The highest classification accuracy was obtained by combining the mass-likelihood estimates of the 2D- and 3D-ResNets, which enabled correct classification of all of the masses.