Optical isolators are important devices in photonic circuits. To reduce the unwanted reflection in a robust manner, several setups have been realized using nonreciprocal schemes. In this study, we show that the propagating modes in a strongly-guided chiral photonic crystal (no breaking of the reciprocity) are not backscattering-immune even though they are indeed insensitive to many types of scatters. Without the protection from the nonreciprocity, the backscattering occurs under certain circumstances. We present a perturbative method to calculate the backscattering of chiral photonic crystals in the presence of chiral/achiral scatters. The model is, essentially, a simplified analogy to the first–order Born approximation. Under reasonable assumptions based on the behaviors of chiral photonic modes, we obtained the expression of reflection coefficients which provides criteria for the prominent backscattering in such chiral structures. Numerical examinations using the finite-element method were also performed and the results agree well with the theoretical prediction. From both our theory and numerical calculations, we find that the amount of backscattering critically depends on the symmetry of scatter cross sections. Strong reflection takes place when the azimuthal Fourier components of scatter cross sections have an order l of 2. Chiral scatters without these Fourier components would not efficiently reflect the chiral photonic modes. In addition, for these chiral propagating modes, disturbances at the most significant parts of field profiles do not necessarily result in the most effective backscattering. The observation also reveals what types of scatters or defects should be avoided in one-way applications of chiral structures in order to minimize the backscattering.