QUALCOMM Incorporated
EXTENDED LOW-FREQUENCY NON-SEPARABLE TRANSFORM (LFNST) DESIGNS WITH WORST-CASE COMPLEXITY HANDLING

Abstract:

A video decoder can be configured to determine a number of allowed non-zero coefficients for a block of video data based on a size of the block; obtain a set of dequantized coefficients for the block, wherein the set of dequantized coefficients comprises a first subset of dequantized coefficients that includes non-zero dequantized coefficients and a second subset of dequantized coefficients that includes all zero coefficients, wherein a number of coefficients in the first subset of dequantized coefficients is equal to the number of allowed non-zero coefficients for the block of video data; apply an inverse low-frequency non-separable transform (LFNST) to the first subset of dequantized coefficients to determine a first intermediate subset of coefficients; and apply an inverse separable transform to the first intermediate subset of coefficients and at least a portion of the second subset of coefficients to determine a block of reconstructed residual values.