The development of sensitive fiber-optic accelerometers is a subject of continuing interest. To acquire high resolution, Michelson phase interferometric techniques are widely adopted. Among the variety structures, the compliant cylinder approach is particularly attractive due to its high sensitivity that is defined as the induced phase shift per applied acceleration. While the two arms of Michelson interferometer should be at the same optical path, it is inconvenient to adjust the two arms’ length to equal, also the polarization instability and phase random drift will cause a signal decline. To overcome these limitations, a novel optical fiber accelerometer based on differential interferometric techniques is proposed and investigated. The interferometer is a Sagnac-like white light interferometer, which means the bandwidth of laser spectrum can be as wide as tens nanometers. This interferometer was firstly reported by Levin in 1990s. Lights are divided to two paths before entering the coupler. To induce time difference, one passes through a delay arm and another goes a direct arm. After modulated by the sensing component, they reflect to opposite arm. The sensing part is formed by a seismic mass that is held to only one compliant cylinder, where the single-mode optical fiber is wrapped tightly. When sticking to vibrations, the cylinder compresses or stretches as a spring. The corresponding changes in cylinder circumference lead to strain in the sensing fibers, which is detected as an optical phase shift by the interferometer. The lights from two arms reach the vibration source at different time, sensing a different accelerate speed; produce a different optic path difference. Integrating the dissimilarity of the accelerated speed by time can obtain the total acceleration graph. A shaker’s vibration has been tested by the proposed accelerometer referring to a standard piezoelectric accelerometer. A 99.8% linearity of the optical phase shift to the ground acceleration is achieved. The acceleration sensitivity is 300 rad/g. It proves to have a simple structure, good practice, reliable measurement and stable performance.