Abstract: The Gradient Structure Similarity (GSSIM) cannot make a good judgment on the near-threshold distortion images, which leads to non consistent with the Human Visual System (HVS). In order to improve the accuracy of this index and its consistency with HVS, an image quality evaluation index based on gradient masking and visual saliency (VS-GSSIM) is proposed. The index first optimizes the GSSIM model according to the masking effect between the gradient signals, then combines with the saliency model to improve the consistency of the index and the HVS evaluation, and finally, it adjusts the relative importance of the perceived quality level of the local area of the image and pooling the final score. The experimental results show that the proposed model surpasses GSSIM in terms of consistency and monotonic evaluations, and is better than most current algorithms.