Abstract: With continuous development of pattern recognition，image recognition，as a typical application in this field，had attracted attention of many scholars，especially in the area of fast image recognition.Convolution neural network algorithm was applied to fruit image recognition based on deep learning.By analyzing network structure of convolution neural network algorithm and training process of convolution neural network，a fruit image recognition system based on convolution neural network was constructed by using convolution neural network architecture design and classifier design.Recognition results were compared with traditional fruit image recognition results，which provided a reference for field of fruit image recognition.