Aimed at the problems that varieties of feature vectors extracted from the image is stacked into a highdimensional feature vector for image semantic segmentation, and these lead to the weakening or loss of the classification ability of some feature vectors, an image semantic segmentation method based on deep convolution neural network AlexNet and conditional random fields is proposed. The pretrained AlexNet model is utilized for extracting image features, and then the semantic segmentation of the image is achieved through the efficient use of conditional random fields for multiple features and context information. The experimental results compared with the methods using the traditional classical features show that Conv5 is the most effective feature extraction layer when AlexNet model is used to extract features for image semantic segmentation. The recognition accuracy in the Stanford background and Weizmann horse datasets is respectively 81.0% and 91.7%, and both the accuracy rates are higher than that of the two comparison methods, indicating that the deep convolution neural network can extract more effective features and obtain higher semantic segmentation accuracy.