Specimen x-ray imaging provides important information on the margin of surgically excised tissue as well as radiologic and pathologic correlation of the lesion. Similar to breast imaging, where mammograms are digitally processed to enhance readability and lesion conspicuity, specimen images are also processed and enhanced. However, specimen image processing is made challenging by the diversity of specimen containers that are commercially available, compounded by variations in specimen size. In this work, we demonstrate our specimen container and size classification system based on a simple convolutional neural network (CNN), trained to identify the container type. This system allows for automated image processing of the supported container types. A dataset consisting of 1428 HIPAA and IRB-complaint anonymized specimen images were collected. We prepared a simple CNN for image classification with 3 convolutional and 3 fully connected layers, and evaluated the performance based on three comparison metrics. Each network was analyzed in terms of accuracy, multi-class AUC, and via a confusion matrix. The best performing classifier, determined via cross validation, was then used for testing, and evaluated with the same three metrics. The results of training and tuning within cross validation showed that the specimen classes are easily differentiable with this simple convolutional neural network structure. During testing, the network was able to achieve an accuracy of 95.8±4.0%, and an AUC of 0.9763±0.0001.