, as itis common for a page to convert from being frequently accessedto not being accessed at all on two consecutive epochs, thus theprediction MAE can be significantly high.

Overall, we prove that the accuracy of the RNN predictions issuch that it can deliver application performance similar to whatwould be possible with oracular knowledge of the access frequency.

lso, we assume dedicated DMA enginesthat allow seamless page migration, which is overlapped with thecomputation, as explored in [14, 19].

Concerning the memory footprint of these applications,

Number of accesses

Identifies the subset of application pages that are important toperformance, through its page selector component, described indetail later on

we flip the problem and explore the case of pre-dicting when a page is going to be accessed next

However, whennormalizing hundred thousand values in such a way (total numberof pages according to Table 1), there will be vast information loss.

This approach has several limitations

More specifically, itis the sequence of the page accesses that were serviced from mainmemory and not the processor’s hardware caches, as they happenedthroughout the application run time.

, so as to aggregate the accesses on an application pagegranularity and then determine an ordering of heavily accessedpages. These predictions need to happen periodically, when thepage scheduler is invoked, so that the appropriate page migrationsare determined and executed.

Figure 2:

History page scheduler can reduce performanceup to 55% (in the case of lulesh) and 13% on average.

Purely history-based pagescheduling methods are limited in the performance opportunitiesthey can provide to applications running on hybrid memory systems.Instead, they must be augmented with more intelligent, predictivemethods

commontheme among these approaches is that they rely exclusively onhistoric information about page accesses. Specifically, the state-of-the-art [ 20 , 27 , 28 ] in system-level dynamic page managementsolutions for HMS utilize the immediate observed behavior to makedecisions on the best future page placement.

An effective page scheduler is responsible for ensur-ing that hot pages – the ones that are accessed frequently

Kleio reduces on average 80% of the performance gapbetween the existing solutions and an oracle with knowledge offuture access pattern

Shorter thresholds thus make the task of the ML predictormore tractable. While longer thresholds move the byte missratio of relaxed Belady closer to Belady’s MIN. It is thusimportant to find the proper threshold

Distribution of tier2 access ratio and tier2 resi-dency ratio under different promotion policies.

However, a newly referenced page is likely to be referenced againsoon and frequently.

Tier2 performance impact and overheads.

the large variation means that population studies are nec-essary to determine the actual impact.

that this metric is relatively high especially compared toswap based solutions [32 ],

. The base policy promotes pagestouched in two or more consecutive hot scan periods.

The hardware deployed with TMTS supportsprecise events filtered to loads sourced from tier2. Unfortunately,the hardware does not support such filtering for memory stores,so stores are not sampled

hence, rely on swap-backed memory(anonymous and tmpfs) for more than 98% of their pages. Thus,only swap-backed pages are treated as demotion candidates.

. This system design point is quitedistinct from virtual memory

The diversity and scale of its applicationsmotivates an application-transparent solution in the general case,adaptable to specific workload demands.

at a much higher train-ing time and memory footprint

variable temporal dependencies

general

Figure 5

measured to be up to 15x the training times

Emulating Online Learning:

This reduces the output size and the neuralnetwork is trained to predict binary outputs that are laterconverted to decimal.

000x compression

n ≈ 50, 000

may lead to slowing down of inference due to largenumber of output labels.

Figure 1: Autocorrelation coefficients for each trace for various lags.

for various lags

We propose to use a hybrid offline+online training approach wherea base model is trained offline first. At runtime, in case of low ac-curacy, a more specialized model is trained in real-time with thehypotheses that high accuracy can be obtained by only few trainingsamples with few epochs, and that this high accuracy can be sus-tained for a long period of time before another round of retrainingis required

the approach of training offline and testingonline for individual application is not a practical prefetcher,

Art

Second, it is not worth learning correlations for addresses that occurinfrequently.

illustrates the page-aware offset embedding

the entire model is trained online

second embedding layer

Voyager is trained to predict the mostpredictable address from multiple possible labels

P(Addrt +1 |Addrt )

For example, a prefetchis considered correct if any one of the ten predictions by the modelmatch the next address, thus ignoring practical considerations ofaccuracy and timeliness.

Unfortunately, regression-based models are trained to arrive close to the ground truth label,but since a small error in a cache line address will prefetch thewrong line, being close is not useful for prefetching.

While the RL formulation is conceptu-ally powerful, the use of tables is insufficient for RL because tablesare sample inefficient and sensitive to noise in contexts.

data prefetchers have no known ground truth labelsfrom which to learn.

, in the presence of data-dependent correlations acrossmultiple PCs.

STMS

To address this issue, weuse a novel attention-based embedding layer that allows the pageprediction to provide context for the offset prediction.

Only PC

t-SN

K highest-probability deltasare chosen for prefetching

hierarchicalsoftmax

the size of the vocabulary required in order toobtain at best 50% accuracy is usually O(1000) or less

Figure 1

LSTM

We relate contemporary prefetchingstrategies to n-gram models in natural languageprocessing

However, the space ofmachine learning for computer hardware archi-tecture is only lightly explored.