This course address three basic questions related to vision enabled autonomous flight: 1. Why are vision systems fundamental and critical to vision enabled autonomous flight? 2. What are the vision system tasks required for autonomous flight? 3. How can those tasks be approached? Autonomous vehicle operations depend on developing a temporally evolving world model of the vehicle environment including items such as terrain, fixed and moving obstacles, other aircraft, landing sites and taxiways, and ownship location. Much of this information can be obtained from databases such as digital terrain elevation databases, cultural features databases, maps, air traffic control, ADS-B, and GPS antennas. However, this information can be incomplete, due to database errors and omissions, GPS failure, jamming, or spoofing, new construction or temporary closures not reflected in current databases, moving objects such as ground vehicles, personnel, and wildlife, and equipment failures such as ADS-B transponders. Vision systems provide up to the moment input to the world model and form a critical component of the safety case for autonomous operations, including both fixed wing and vertical lift aircraft. This course addresses the role of vision systems for autonomous operations and discusses the critical tasks required of a vision system including taxi, take-off, enroute navigation, detect and avoid, and landing, as well as formation flight or approach and docking at a terminal or with other vehicles. These tasks are then analyzed to develop field of view, resolution, latency, and other sensing requirements, and to understand when one sensor can be used for multiple applications. Along the way, the reader is introduced to the various airspace classifications, landing visibility categories and decision height criteria, and typical runway dimensions. Following this analysis of basic functional requirements, the book provides an overview of sensors and phenomenology from visible through infrared, extending into the radar bands, including both passive and active systems. Human visual system performance is discussed as a comparison benchmark. System architectures are discussed including distributed aperture sensor systems and multi-use sensors. Finally various algorithms for extracting information from sensor data are examined, including moving target detection for detect and avoid, shape from motion, multi-sensor triangulation, model based pose estimation, wire and cable detection, and geo-location techniques.

Image based motion analysis is a key technology that can be used to analyze missile flight performance, aircraft stores separation, aircraft safety and crash worthiness, ejection seat dynamics, artillery and small arms projectile performance and scoring, and explosive projectile distribution and velocity. Other applications include biological motion analysis and automotive crash test analysis. Imagery may be visible (standard or high speed), IR, or digitized film. This course describes techniques for extracting quantitative information from a time sequence of imagery. The primary focus is on position versus time of multiple objects, but intensity, separation distances, velocities, shape, angle of attack, and other features can be determined as well. The course covers basic single camera motion analysis, multiple camera three-dimensional motion analysis, rigid body 6 Degree Of Freedom analysis, and the use of non-image data (such as mount pointing data) to provide additional information during analysis. The course also provides a basic understanding of the image formation sequence from target radiance to image pixels, in order to understand how various effects in the image formation process may affect the final motion analysis results, and how these effects can be compensated for during analysis.