A mix of emerging technologies promises qualitatively new memory system capabilities. However, today's memory controllers and channels constrain heterogeneity. Today's integration of controllers on processor dies prevents systems from accommodating diverse, technology-specific protocols and schedulers in a modular manner; memory design decisions must be made during processor design. Moreover, today's channel architectures are not flexible enough to accommodate diverse demands for bandwidth and capacity. To address these challenges, we present strategies for scalability and heterogeneity, which include (i) disintegrating memory controllers to support heterogeneous command protocols in a modular manner; (ii) adding buffers to ensure signal integrity; and (iii) organizing buffers hierarchically to reduce latency. We apply these strategies to architect a novel heterogeneous DRAM / PCM system. Finally, we present mechanisms for power-efficient data movement. © 2013 IEEE.