There are affordable houses that violation of construction like construction disorder and less construction in China, which will cause wasting and misappropriating the government investment. Thus, it needs to be verified. In view of the large number and wide distribution of residential housing, as well as traditional artificial verification methods are too time-consuming and laborious. In this paper, based on the characteristics of high-width and wide-width of GF-1 satellite, a fast verification method of affordable house from GF-1 panchromatic image by geographic constraint is proposed. First, use the Morphological Building Index (MBI) method to extract the building features of the entire study region, and crop out image blocks by using the affordable house’s vector data that measured by GPS for geographic constraints. Secondly, for the local features of each image block, combined with the Canny operator and the adaptive Mean-shift image segmentation algorithm to extract the buildings within the image block. Finally, based on the overlap rate between the building extraction result and the vector data, it is judged whether the location of the affordable house exists. Experiments show that the building extraction module of this paper can effectively extract the buildings in the image blocks on the GF-1 panchromatic image and is better than the MBI method, which can effectively realize the verification of the affordable houses.