MIT Researchers Customizing Chip Cache Memory Allotments for Programs

Researchers at MIT have found a method of allocating individualized portions of chip cache memory to multiple computer programs at once, which makes better use of the cache memory and improves performance for all running applications.

The research, being conducted at MIT’s Computer Science and Artificial Intelligence Laboratory, shows that memory cache access can be reallocated on the fly so it can be directly tied to programs as needed, rather than being provided in its entirety to applications. By allocating the memory more directly using new cache hierarchies, significant gains in energy savings and performance can be found, according to a July 10 announcement by the school.

The system, called Jenga, has been co-developed by Daniel Sanchez, an assistant professor in MIT's Department of Electrical Engineering and Computer Science and co-authors Po-An Tsai, a graduate student in the department, and Nathan Beckmann, who was an MIT graduate student when the work was done and is now an assistant professor of computer science at Carnegie Mellon University. The team presented its research with the Jenga project recently at the International Symposium on Computer Architecture.

The idea, co-author Po-An Tsai told ITPro, is "to make the software able to control what the cache hierarchy is for each application. In that case, each application could have its own cache suitable for its performance."

The work involves multi-core processors using L1, L2, L3 and L4 cache. In modern multi-core processors, the L1 and L2 cache areas are "tiny," while L3 and L4 cache each represent about 50 percent of the cache on the chip. All the cores share those sections of L3 and L4 similarly for all applications, he said.

"Our research is trying to say since you have, for example, 8MB of cache memory resources, we can give it some kind of budget" to operate well for each application. "Because if you don't do that, all applications will use those resources in [total] and that rigid hierarchy is seldom right for every application."

The group's research, which involves hardware and software, means they are pooling together the L3 and L4 cache as a single resource, allowing applications to take the memory resources they need without affecting other running applications, he said.

The motivation for the research has been the wide range of memory technologies on the market, which today are very specialized for chip cache use. "The traditional wisdom was to throw them all at layers and layers of caches to make things react faster if they are used well," said Tsai. "But the fact is you cannot just throw new technologies at those rigid hierarchies."

The researchers have conducted simulations of this process to proves it works, he said.

So far, they are not talking with any chip makers about their research, but that could happen in the future, he said.

"To build this into chips I think companies would want to look more at how it works with applications and how it could be done in production at some point," said Tsai. "Chip companies would want to see how this worked with their test array of applications."

Computer chips have been increasing their efficiency for decades by using caches, which are small, local memory banks that store frequently used data and cut down on time- and energy-consuming communication with off-chip memory. Modern chip development began with one level of cache memory but today includes as many as four levels of integrated cache. Each cache level is larger but slower than the level before it and each also is rarely suited exactly to the needs of the applications that use it, which led to the research.

The Jenga research builds on an earlier system from the MIT group, called Jigsaw, which also allocated cache access on the fly. Jigsaw, however, didn't build cache hierarchies, which made the memory allocation problem more complex, according to the research.