It is well-known that a pattern recognition system is only as good as the features it is built upon. In the fields of image processing and computer vision, we have numerous spatial domain and spatial-frequency domain features to extract characteristics of imagery according to its color, shape and texture. However, these approaches extract information across a local neighborhood, or region of interest, which for target detection contains both object(s) of interest and background (surrounding context). A goal of this research is to filter out as much task irrelevant information as possible, e.g., tire tracks, surface texture, etc., to allow a system to place more emphasis on image features in spatial regions that likely belong to the object(s) of interest. Herein, we outline a procedure coined soft feature extraction to refine the focus of spatial domain features. This idea is demonstrated in the context of an explosive hazards detection system using forward looking infrared imagery. We also investigate different ways to spatially contextualize and calculate mathematical features from shearlet filtered candidate image chips. Furthermore, we investigate localization strategies in relation to different ways of grouping image features to reduce the false alarm rate. Performance is explored in the context of receiver operating characteristic curves on data from a U.S. Army test site that contains multiple target and clutter types, burial depths, and times of day.