Due to the finite size of the hologram aperture in digital holography, high frequency intensity and phase fluctuations along the edges of the images, which reduce the precision of phase measurement. In this paper, the apodization filters are applied to improve the phase measurement in the digital holography. Firstly, the experimental setup of the lensless Fourier transform digital holography is built, where the sample is a standard phase grating with the grating constant of 300μm and the depth of 150nm. Then, apodization filters are applied to phase measurement of the sample with three kinds of the window functions: Tukey window, Hanning window and Blackman window, respectively. Finally, the results were compared to the detection data given by the commercial white-light interferometer. It is shown that aperture diffraction effects can be reduced by the digital apodization, and the phase measurement with the apodization is more accurate than in the unapodized case. Meanwhile, the Blackman window function produces better effect than the other two window functions in the measurement of the standard phase grating.