The detection of product discoloration is very important to many industrial applications, such as the inspection of vegetables, fruits, grains, pills, etc. There are currently two methods for detecting the color change. The first method is using an optical sensor to measure the amount of light energy that is reflected by the color surface. The other method determines the change in product color with the black and white image. The performance of these two methods suffers, due to the lack of the color information and poor resolution. A high-speed automated color sorting vision system is designed and developed using the IDASTM (imaging development and application system) real-time image processing system to provide true discoloration detection. A color line-scan camera, which detects the amount of light in the red, green, and blue color spectrum, is used to obtain color information from the product surface. The conbinations of information from these three color channels cover almost all of the color variations that human eyes can distinguish. The information used by this vision system is at least three times of that by the black and white system and the defect resolution required can be achieved by selecting the proper optics and the camera sensor size. The use of the pipeline real-time image processing hardware and the implementation of the image processing algorithms enable the system to detect the discoloration at very high speed. The detail descriptions of the configuration of the vision system, optics selection, image processing algorithms, hardware control, and communication will be introduced in this paper.