Visual information fidelity


Visual information fidelity is a full reference image quality assessment index based on natural scene statistics and the notion of image information extracted by the human visual system. It was developed by Hamid R Sheikh and Alan Bovik at the Laboratory for Image and Video Engineering at the University of Texas at Austin in 2006. It is deployed in the core of the Netflix VMAF video quality monitoring system, which controls the picture quality of all encoded videos streamed by Netflix.

System model

Distortion model

The distortion process is modeled using a combination of signal attenuation and additive noise in the wavelet domain. Mathematically, if denotes the random field from a given subband of the distorted image, is a deterministic scalar field and, where is a zero mean Gaussian vector with co-variance, then
Further, is modeled to be independent of and.

HVS model

The duality of HVS models and NSS implies that several aspects of the HVS have already been accounted for in the source model. Here, the HVS is additionally modeled based on the hypothesis that the uncertainty in the perception of visual signals limits the amount of information that can be extracted from the source and distorted image. This source of uncertainty can be modeled as visual noise in the HVS model. In particular, the HVS noise in a given subband of the wavelet decomposition is modeled as additive white Gaussian noise. Let and be random fields, where and are zero mean Gaussian vectors with co-variance and. Further, let and denote the visual signal at the output of the HVS. Mathematically, we have and. Note that and are random fields that are independent of, and.

VIF index

Let denote the vector of all blocks from a given subband. Let and be similarly defined. Let denote the maximum likelihood estimate of given and. The amount of information extracted from the reference is obtained as
while the amount of information extracted from the test image is given as
Denoting the blocks in subband of the wavelet decomposition by, and similarly for the other variables, the VIF index is defined as

Performance

The Spearman's rank-order correlation coefficient between the VIF index scores of distorted images on the LIVE Image Quality Assessment Database and the corresponding human opinion scores is evaluated to be 0.96.