Near Data Processing: Impact and Optimization of 3DMemory System Architecture on the UncoreSyed Minhaj Hassan, Sudhakar Yalamanchili & Saibal MukhopadhyaySchool of Electrical and Computer EngineeringGeorgia Institute of Technology, Atlanta, Georgiaminhaj@gatech.edu, sudha@gatech.edu & saibal@ece.gatech.eduABSTRACTA promising recent development that can provide contin-ued scaling of performance is the ability to stack multipleDRAM layers on a multi-core processor die. This paper an-alyzes the interaction between the interconnection networkand the memory hierarchy in such systems, and its impacton system performance. We explore the design consider-ations of a 3D system with DRAM-on-processor stackingand note that full advantages of 3D can only be achievedby conf i guring the memory with high number of channels.This signif i cantly increases memory level parallelism whichresults in decreasing the traf f i c per DRAM bank, reducingtheir queuing delays, but increasing it on the interconnectionnetwork, making remote accesses expensive. To reduce thelatency and traf f i c on the network, we propose restructuringthe memory hierarchy to a memory-side cache organizationand also explore the ef f ects of various address translationsand OS page allocation strategies. Our results indicate thata carefully designed 3D memory system can already improveperformance by 25-35% without looking towards new sophis-ticated techniques.Categories and Subject DescriptorsB.3.1 [Hardware-Dynamic memory]: Computer systemsorganization Multicore architecturesKeywords3D memory system, Near data computing, Interconnectionnetwork, Address mapping, HMC1. INTRODUCTION3D packaging has emerged as a vehicle for scaling sys-tem densities and performance due to i) increased inter-tierbandwidth, ii) reduced inter-tier latencies, and iii) abilityto integrate dies from dif f erent process technologies as ameans of customization and hence performance improve-ment. Moving forward, continued scaling of Through Sil-Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prof i t or commercial advantage and that copies bear this notice and the full cita-tion on the f i rst page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specif i c permissionand/or a fee. Request permissions from permissions@acm.org.MEMSYS ’15, October 05-08, 2015, Washington DC, DC, USAc ? 2015 ACM. ISBN 978-1-4503-3604-8/15/10...$15.00DOI: http://dx.doi.org/10.1145/2818950.2818952icon Vias (TSVs) is increasing the inter-die bandwidth andreducing inter-die latency. Large numbers of f i ne grainedTSVs across a 2D cross section can support a large num-ber of memory channels. Hence, the memory bandwidthcan be increased signif i cantly. e.g. a Micron die stackedDRAM increases bandwidth up to 128GB/s as opposed to21.34GB/s and 10.66GB/s of DDR4-2667 and DDR3-1333respectively [17]. It is important to understand features ofarchitectural organizations that can make use of this tech-nology capacity.This paper addresses the problem of improving executionperformance in 3D chip multiprocessors with DRAM-on-processor stacking. The specif i c problem of interest is therole of the interconnection network in limiting the memorybandwidth utilization. We seek to understand how best toorganize the memory hierarchy so as to maximize the band-width and latency advantages of 3D technology. The workaddresses two key issues, that is, 1) the extreme parallelismof a 3D memory system makes address translations a key de-terminant of locality and parallelism, which can be used toreshape the memory traf f i c maximizing performance, and 2)reduced 3D DRAM delay leads to refactoring of the memorylatency path, which increases pressure on the interconnec-tion network between the memory and the cache hierarchyand requires architectural modif i cations that either reducethe network latency and traf f i c or adapt to this re-factoredmemory-latency path.We f i rst discuss the importance of increasing the num-ber of memory channels in a 3D system and show that 2Dnetwork latency becomes a more critical problem in such asystem. We then propose a memory-side cache organiza-tion in which distributed L2 banks are placed next to theDRAM channels which reduces the traf f i c in the network,simultaneously distributing it to various memory channels.With coordinated address space mappings across the cachesand DRAM channels, this organization increases TSV uti-lization and eliminates unnecessary traf f i c on the network,with net improvements in performance. We also evaluateref i nements of the address space mapping to distribute re-quests across dif f erent DRAM channels. Our load distribu-tion mechanisms signif i cantly reduces queuing delays in theMC queues, thus resulting in reduced round trip memorylatency and improved performance. Lastly, we explore theimpact of OS page allocation in keeping the network traf f i cminimal. Our combined approach indicates that a carefullydesigned 3D memory system can already improve perfor-mance by 25-35% without the overhead of any sophisticatedtechnique.
