Aimed at the problems that the degree of freedom in the current nested MIMO array (Multiple-Input Multiple-Input Multiple Output Multiple Array) is limited, an optimized nested array is proposed, i.e. to optimize the nested MIMO array design. This method not only retains the original advantage of nested MIMO array design-closed expression with array element position and degree of freedom, but also greatly improves the array aperture and degree of freedom (DOF). Firstly, the optimized nested array is used as the transmitting and receiving arrays, and then the difference between the MIMO arrays is processed to obtain the difference array of array elements. By properly designing the array spacing of the transmitting and receiving arrays, a non-porous difference matrix can be obtained. When the total number of array elements is given, by analyzing the characteristics of the array structure, the best array element number of the transmitting and receiving arrays can be obtained. The simulation experiments show that compared with the nested MIMO array design, the proposed method can effectively expand the array aperture, increase the degree of freedom, and increase the DOA estimation accuracy of the MIMO radar under condition of non increasing the number of actual array elements.