vp_suite.measure.image_wise

This module hosts several popular image-wise measures, such as Mean-Square Error (MSE), Structural Similarity Index (SSIM) or Learned Perceptual Image Patch Similarity (LPIPS).

APPLIES TO ALL LOSSES:

• expected data type: torch.Tensor (data type: torch.float)

• expected shape: [b, t, c, h, w] ([b, t, 3, h, w] for LPIPS and SSIM)

class L1(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements the pixel-wise Mean Absolute Error (MAE/L1).

NAME: str = 'Mean Absolute Error (MAE) / L1 Loss'

The clear-text name of the measure.

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

training: bool

class LPIPS(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements the “Learned Perceptual Image Patch Similarity (LPIPS)” from Zhang et al. (https://arxiv.org/abs/1801.03924), a perceptual measure that uses a pre-trained CNN to obtain features from the given prediction and ground truth. These perceptual features are then compared to yield the actual measurement value.

NAME: str = 'Learned Perceptual Image Patch Similarity (LPIPS)'

The clear-text name of the measure.

REFERENCE: str = 'https://arxiv.org/abs/1801.03924'

The reference publication where this measure is originally introduced (represented as string)

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

forward(pred, target)

The module’s forward pass takes the predicted frame sequence and the ground truth, compares them based on the deriving measure’s criterion and logic and outputs a numerical assessment of the prediction quality.

The base measure’s forward method can be used by deriving classes and simply applies the criterion to the input tensors, sums up over all entries of an image and finally averages over frames and then batches.

Parameters

• pred (torch.Tensor) – The predicted frame sequence as a 5D tensor (batch, frames, c, h, w).

• target (torch.Tensor) – The ground truth frame sequence as a 5D tensor (batch, frames, c, h, w)

Returns: The calculated numerical quality assessment.

training: bool

class MSE(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements the pixel-wise Mean-Square Error (MSE/L2).

NAME: str = 'Mean Squared Error (MSE) / L2 Loss'

The clear-text name of the measure.

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

training: bool

class PSNR(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements the Peak Signal-to-Noise Ratio, which is related to the MSE.

BIGGER_IS_BETTER = True

Specifies whether bigger values are better.

NAME: str = 'Peak Signal to Noise Ratio (PSNR)'

The clear-text name of the measure.

OPT_VALUE = inf

Specifies the best value attainable (e.g. when input tensors are equal).

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

forward(pred, target)

The module’s forward pass takes the predicted frame sequence and the ground truth, compares them based on the deriving measure’s criterion and logic and outputs a numerical assessment of the prediction quality.

The base measure’s forward method can be used by deriving classes and simply applies the criterion to the input tensors, sums up over all entries of an image and finally averages over frames and then batches.

Parameters

• pred (torch.Tensor) – The predicted frame sequence as a 5D tensor (batch, frames, c, h, w).

• target (torch.Tensor) – The ground truth frame sequence as a 5D tensor (batch, frames, c, h, w)

Returns: The calculated numerical quality assessment.

classmethod to_display(x)

Converts a measurement value from the lower-is-better representation returned by the forward() method to the actual representation of the measure (e.g. SSIM having its best value at 1.0). If the measure did not get inverted in the forward(), this method just returns the input value.

Parameters

x (float) – The value to be converted.

Returns: The converted value.

training: bool

class SSIM(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements the structural similarity index (SSIM), as introduced in Zhou et al. (https://ieeexplore.ieee.org/document/1284395).

BIGGER_IS_BETTER = True

Specifies whether bigger values are better.

NAME: str = 'Structural Similarity (SSIM)'

The clear-text name of the measure.

OPT_VALUE = 1

Specifies the best value attainable (e.g. when input tensors are equal).

REFERENCE: str = 'https://ieeexplore.ieee.org/document/1284395'

The reference publication where this measure is originally introduced (represented as string)

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

forward(pred, target)

The module’s forward pass takes the predicted frame sequence and the ground truth, compares them based on the deriving measure’s criterion and logic and outputs a numerical assessment of the prediction quality.

The base measure’s forward method can be used by deriving classes and simply applies the criterion to the input tensors, sums up over all entries of an image and finally averages over frames and then batches.

Parameters

• pred (torch.Tensor) – The predicted frame sequence as a 5D tensor (batch, frames, c, h, w).

• target (torch.Tensor) – The ground truth frame sequence as a 5D tensor (batch, frames, c, h, w)

Returns: The calculated numerical quality assessment.

classmethod to_display(x)

Converts a measurement value from the lower-is-better representation returned by the forward() method to the actual representation of the measure (e.g. SSIM having its best value at 1.0). If the measure did not get inverted in the forward(), this method just returns the input value.

Parameters

x (float) – The value to be converted.

Returns: The converted value.

training: bool

class SmoothL1(device)

Bases: vp_suite.base.base_measure.VPMeasure

This class implements a smoothed L1 Loss, resembling the MSE/L2 loss for smaller discrepancies and transitioning to the MAE/L1 loss for larger discrepancies.

NAME: str = 'Smooth L1 Loss'

The clear-text name of the measure.

__init__(device)

Instantiates the measure class by setting the device. Additionally, for the derived measure classes, instantiates the criterion that is used to calculate the measure value.

Parameters

device (str) – A string specifying whether to use the GPU for calculations (cuda) or the CPU (cpu).

training: bool