The accurate geomagnetic field value is a prerequisite and foundation for the geomagnetic navigation. To solve this problem caused by magnetometer in actual magnetic field measurement influenced by environmental magnetic field, the influence of hard iron and soft iron errors on magnetic measurement is analyzed and the parameterized geomagnetic measurement compensation model is constructed firstly. The geomagnetic field is simulated by using three pairs of Helmholtz coils to generate uniform magnetic field with the software of ANSYS. By setting the different parameters of interference source and rotating the magnetometers in 360 degrees in the simulation model, the three dimensional magnetic field values with and without interference are got. Those parameters of the model are estimated using least squares based algorithm and geomagnetic measurement errors are calibrated. The simulation results show that this compensation method is simple, and easy to use, and has higher error compensation ability, and geomagnetic measurement errors can be reduced from 20000nT to several nanoteslas.