Abstract: Due to the problems of dimensionality disaster caused by excessive extraction of feature parameters and the need to improve the classification accuracy in speech/music classification process,this paper proposes a speech/music classification algorithm based on zero-crossing rate and spectrum. After endpoint detection and segmented preprocessing of speech and music signals, the algorithm classifies and recognizes each audio segment by combining the zero-crossing rate and spectral amplitude characteristics, and finally realizes the classification by calculating the probability of being identified as speech or music.Experimental results show that the accuracy of this algorithm in audio classification is about 7.9% higher on average than that of the same algorithms which only mention two audio features at most and do not use the classifier , and about 5.7% higher than that of the algorithms which extract multiple audio features and use the classifier.It proves that this algorithm not only has a small amount of calculation,but also improves the classification accuracy.