In vehicle monitoring, recognizing graphic vehicle identification number (VIN) on the car frame is a particularly important step. While text recognition methods have made great progress, automatic graphic vehicle VIN recognition is still challenging. In VIN images, the VIN text is engraved on the car frame, with complex background and arbitrary orientation, which make it extremely difficult for recognition. We propose an efficient framework for recognizing rotational VIN. First, combining lightweight convolutional neural network and per-pixel segmentation in the output layer, we achieve fast and excellent VIN detection. Second, we take the VIN recognition task as a sequential position-dependent classification problem. By attaching sequential classifiers, we predict VIN text without character segmentation. Finally, we introduce a VIN dataset, which contains 2000 raw rotational VIN images and 90,000 horizontal VIN images for validating our framework. Experiments results show that the framework we proposed achieves good performance in VIN detection and recognition. By automatically identifying the VIN, we can quickly confirm the vehicle’s identity and help vehicle monitoring and tracking.