The downward continuation algorithm of geomagnetic potential field is a typical ill-posed problem in mathematics. Aimed at the problems that the traditional downward continuation of geomagnetic data is susceptible to the effects of noise and continuation distance, this paper explores the downward continuation of magnetic anomalies by using the U-Net neural network. Magnetic anomalies obtained at different heightsvia forward modeling of simple prisms and complex combined prisms, are taken to be labels of machine learning with 5 000 label groups being constructed for each dataset. The dataset is divided into training, test, and validation sets at a ratio of 8∶1∶1. The construction of machine learning labels is in comprehensive consideration information such as geometric position of the model center, the length, width, and height of the model, as well as magnetic inclination, magnetic declination, and magnetization intensity. Under condition of interference-free experiments, the average relative prediction accuracy is 97.0% for the single prism model and 95.2% for the three-combined prism model. Under the interference of 5% Gaussian noise, the average relative prediction accuracy of the combined model remains 94.9%, showing excellent anti-interference performance. And simultaneously, the model exhibits strong distance robustness within the range of continuation distances involved in the experiments. This study verifies that the UNet network can effectively realize downward continuation and has broad application prospects in the construction of 3D magnetic reference maps.