Object detection has gained considerable interest in remote sensing community with a broad range of applications including the remote monitoring of building development in rural areas. Many earlier studies on this task performed their analysis using either multispectral satellite imagery or color images obtained via an aerial vehicle. In recent years, hyperspectral imaging (HSI) has emerged as an alternative technique for remote monitoring of building developments. Unlike other imaging techniques, HSI provides a continuous spectral signature of the objects in the field of view (FOV) which facilitates the separation among different objects. In general, spectral signature similarity between objects often causes a significant amount of false alarm (FA) rate that adversely effects the overall accuracy of these systems. In order to reduce the high rate of FA posed by the pixel-wise classification, we propose a novel rural building detection method that utilizes both spatial information and spectral signature of the pixels. Proposed technique consists of three parts; a spectral signature classifier, watershed based superpixel map and an oriented-gradient filters based object detector. In our analysis, we have evaluated the performance of proposed approach using hyperspectral image dataset obtained at various elevation levels, namely 500 meters and 3000 meters. NEO HySpex VNIR-1800 camera is used for 182 band hyperspectral data acquisition. First 155 band is used due to the atmospheric effects on the last bands. Performance comparison between the proposed technique and the pixel-wise spectral classifier indicates a reduction in sensitivity rate but a notable increase in specificity and overall accuracy rates. Proposed method yields sensitivity, specificity, accuracy rate of 0.690, 0.997 and 0.992, respectively, whereas pixel-wise classification yields sensitivity, specificity, and accuracy rate of 0.758, 0.983, 0.977, respectively. Note that the sensitivity reduction is due to sparseness of buildings in rural areas, however, increase in overall accuracy is considered more important in our study.