International Business Machines Corporation
Optimized hierarchical scratchpads for enhanced artificial intelligence accelerator core utilization

Abstract:

Various embodiments are provided for optimized placement of data structures in a hierarchy of memory in a computing environment. One or more data structures may be placed in a first scratchpad memory, a second scratchpad memory, an external memory, or a combination thereof in the hierarchy of memory according to a total memory capacity and bandwidth, a level of reuse of the one or more data structures, a number of operations that use each of the one or more data structures, a required duration each the one or more data structures are required to be placed a first scratchpad or a second scratchpad, and characteristics of those of the one or more data structures competing for placement in the hierarchy of memory that are able to co-exist at a same time step. The second scratchpad memory is positioned between the external memory and the first scratchpad memory at one or more intermediary layers.