Abstract: In order to fully explore the physiological and pathological information of heart sound signals and improve the accuracy of automatic classification of heart sound, a new algorithm for automatic classification of heart sound was proposed which did rely on segmentation and denoising. Firstly, the time-frequency features of the heart sound signal Bark domain fractional Fourier transform were extracted. Then, the deep residual contraction network was introduced into the convolutional neural network to construct a new classification model, and the feature information irrelevant to the current task was automatically removed. The feature information irrelevant to the current task was automatically removed to improve the accuracy and stability of the model prediction. The institute used 5 000 heart sound samples, 1 000 of which were tested. Experimental results showed that the accuracy, sensitivity and specificity of the proposed algorithm were 0.925, 0.902 and 0.948, respectively, and the F1 value was 0.923. Compared with the previous methods, the overall performance of this method had been significantly improved, and it had strong robustness and generalization ability, which was expected to be applied to the clinical screening of congenital heart disease.