Microsoft Corporation
Compressing and Decompressing Data for Language Models

Abstract:

Embodiments of the present disclosure include systems and methods for compressing and decompressing data generated by sub-blocks in a neural network. In some embodiment, an input matrix is received at a compression block in the neural network. The compression block compresses the input matrix into a compressed matrix and outputs the compressed matrix. The compressed matrix has a reduced dimensionality relative to a dimensionality of the input matrix. A decompression block retrieves the compressed matrix. The decompression block decompresses compressed matrix into a decompressed matrix and outputs the decompressed matrix. The decompressed matrix has a same dimensionality as the dimensionality of the input matrix. The compression and decompression blocks are optimized based on feedback received from the neural network.