We developed a new virtual reality (VR) system that enables direct interaction between a 3D image and a finger without wearing special 3D glasses and without attaching any marker or detector to the finger. Moreover, it frees the user from visual fatigue. The system consists of a 128-directional 3D display, a PC cluster, and a fingertip detection system. The 128-directional display provides a natural 3D image which does not have the accommodation-vergence conflict and has very smooth motion parallax. It differs from conventional multi-view displays in that it precisely reconstructs rays from a 3D object. It contains 128 LCD panels, and the 128 images displayed on them are projected in different horizontal directions with a horizontal angle pitch of 0.23°. The PC cluster consists of 16 PCs. Each PC generates eight video signals. The fingertip detection system employs a stereo infrared (IR) camera. The 3D position of the fingertip is estimated by triangulation. We also made three application programs. The first one enables the fingertip manipulation of a 3D image in VRML format. The second one is a 3D drawing program that allows users to draw lines in the air. The last one enables the rotation of a 3D image of a 360° directional image by the finger.