Aimed at the slow matching speed of SIFT feature matching method in searching the whole database of feature points for matching points, an improved matching method of searching between the layers of the pyramid is proposed for improving the speed of SIFT matching. First, according to the feature points in different layers of the pyramid, the feature points are divided into different sets. Then a layer set in the input image pyramid is chosen to search for the similar layer in the template image pyramids, and then the similarity between input image pyramid and the template pyramid is determined. Finally, the matches are found between the similar layers. The chosen layer in the input image pyramid is from the bottom of the pyramid to the top of the pyramid. By so doing, the time of searching for similar layer is shorten. Compared with the original algorithm, this algorithm has the same rotational stability. This algorithm and the original algorithm are applied to the actual image registration respectively, the results show that the matching speed is 3.2 times as fast as the original matching speed and the correct matching rate is increased by 10.3% in visual image registration. In infrared image registration the matching speed is 1.4 times as fast as the original matching speed and the correct matching rate reaches 100%.