Diffraction-based optical correlators working under broadband illumination, in contrast to their coherent counterparts, allow us to exploit color information. However, the use of the wavelength as an additional parameter requires to take into account the chromatic dispersion inherent to the diffraction process. In this contribution, we describe a novel family of dispersion-compensated broadband optical correlators that operate some in the Fourier and some in the Fresnel region. In both cases, the chromatic compensation is achieved by a proper combination of a small number of commercially available optical elements, conventional diffractive and refractive lenses. In all cases, and with a single matched filter, the chromatic content of the correlation peak provides the spectral composition of the detecting color signal. On top of that, some of our optical solutions work with point-source illumination and others with spatially-incoherent light. In this way, on the one hand, our spatially coherent optical designs permit to perform the color correlation in amplitude for each spectral channel. Accordingly, working in the Fresnel domain, we achieve a space-variant color pattern recognition setup. On the other hand, totally incoherent optical correlators, which are linear in irradiance, provide important practical advantages as they employ natural light and allow us to deal with diffuse, reflecting or self-luminous color objects.