Ion beam finishing techniques are commonly used for improvement of surface error topography of optical devices. Optical aluminum surfaces after manufacturing by single-point diamond turning meet the requirements for applications in the infrared spectral range. However, optics used for applications in the short-wavelength visible and ultraviolet spectral range demand improved surface qualities. To overcome the limitations mainly caused by structural and compositional inhomogeneities of aluminum alloys, a reactive ion beam machining process using oxygen and nitrogen operating gas is applied. This technology enables direct surface machining while preserving the initial roughness up to a 1-μm etching depth using low-energy ion beams. Moreover, the use of oxygen allows us to smooth the surface in the microroughness regime. Based on Monte-Carlo simulations and roughness evolution measured by atomic force microscopy, a more detailed discussion of the ion beam process is presented. Hence, a model scheme for direct smoothing of high-frequency surface features is suggested.