Boost HPC workloads with Micron DDR5 and 4th Gen AMD EPYC processors 

The goal of the AMD and Micron collaboration is to deliver best-in-class user experiences across client and data center platforms. To that end, the two companies have a joint server lab in Austin, working to ensure we are reducing time to validate server memory and performing joint workload testing throughout validation and launch. In this blog, we look at some common HPC-workload benchmark results that use Micron DDR5 data center memory with 4^th Gen AMD EPYC^TM Processors as both these products are shipping now.

High-performance computing (HPC) workloads have historically been the domain of some of the world’s fastest supercomputers. These are often large-scale, data-intensive workloads split into millions of operations that are run in parallel and use terabytes of data. These complex workloads are dedicated to solving some of humankind’s most challenging problems — weather and climate simulations; seismic modeling; chemical, physics and biological analysis; and more.

With advances in computer architectures, these workloads have increasingly been hosted in very large “scale-out” clusters of high-performance servers. These clusters require the latest and greatest compute, fabric, memory and storage infrastructure to address the scalability, low latency and performance needs of such critical workloads. While server CPUs have grown in performance and throughput, the past several years have seen the bandwidth provided by DDR4 memory become a bottleneck. There is just not enough memory bandwidth to supply the growing number of high-performance cores.

Micron DDR5 memory and the new AMD Zen 4 server architectures featuring 4^th Gen AMD EPYC Processors change that. Now, server CPUs and memory can be in much better balance to unlock performance and efficiency for the most demanding workloads. DDR5 memory helps organizations reach those insights faster, whether on premises or in the cloud. Consider some of the following proof points generated while testing Micron DDR5 with the latest AMD Zen 4 96-core CPU with an industry-standard HPC workload benchmark. All of our test results have shown two times the performance improvement. 

Double the memory bandwidth with Micron DDR5 + 4th Gen AMD EPYC Processors using STREAM

STREAM¹ is a simple, well-known benchmark that measures memory bandwidth in HPC computers. It captures peak memory bandwidth for HPC systems

Software stack used for this workload

• Alma 9 Linux kernel 5.14
• STREAM.f  11-29-2021 release

Test setup

• DDR4 system 3^rd Gen AMD EPYC Processors with 64 cores and 3.7 GHz; DDR4 3200 MHz system2 is fully populated with 64GB RDIMM
• DDR5 system 4^th Gen AMD EPYC Processors with 96 cores and 3.7 GHz; DDR5 4800 MHz system3 is fully populated with 64GB RDIMM

Test results

• Double the memory bandwidth of 378 GB/s for a single-socket DDR5 system
• This result means that customers can run larger artificial intelligence/machine learning (AI/ML) projects or do more HPC computations with increased memory bandwidth from DDR5.

Weather research and forecasting (WRF)⁴ runs two times faster with Micron DDR5

This HPC workload code is used by the weather and climate community, and the model is widely used for meteorological applications. WRF typically performs well on traditional HPC architectures that support high floating-point processing, high memory bandwidth and a low-latency network. For this effort, the Continental United States (CONUS) at 2.5-km lateral resolution was chosen.

Software stack used for this workload 

• Alma 9 Linux kernel 5.14 
• WRF 2.3.5 & 4.3.3 
• Open MPI v4.1.1

Test setup

• DDR4 system 3^rdGen AMD EPYC Processors with 64 cores and 3.7 GHz; DDR4 3200 MHz system2 is fully populated with 64GB RDIMM
• DDR5 system 4^th Gen AMD EPYC Processors with 96 cores and 3.7 GHz; DDR5 4800 MHz system3 is fully populated with 64GB RDIMM

Test results

• We were able to execute 1.3567 time steps per second using Micron DDR5 and 4th Gen AMD EPYC Processors as compared to 2.8533 time steps per second.
• Faster execution time means weather forecasters can either choose bigger datasets or run more models. Both efforts lead to improved forecasts.

OpenFOAM⁵ with Micron DDR5 runs two times faster

OpenFOAM is an open-source HPC workload for computation fluid dynamics (CFD), used in a wide variety of industries to reduce development time and costs. It simulates physical interactions in applications ranging from consumer product design to aerospace design. One of the simulations included in the data set features a motorbike turbulence simulation. For this model, OpenFOAM calculates steady air flow around a motorcycle and rider. OpenFOAM load balances calculations according to the number of processes specified by the user, and then decomposes the mesh into parts for each process to solve. After the solve is complete, the mesh and solution is recomposed into a single domain.

Software stack used for this workload

• OpenFOAM CFD Software (v8) with motorBike mesh size of 600 x 240 x 240
• Alma 9 Linux kernel 5.14 
• Open MPI v4.1.1

Test setup

• DDR4 system 3^rd Gen AMD EPYC Processors with 64 cores and 3.7 GHz; DDR4 3200 MHz system2 is fully populated with 64GB RDIMM
  
• DDR5 system 4^th Gen AMD EPYC Processors with 96 cores and 3.7 GHz; DDR5 4800 MHz system3 is fully populated with 64GB RDIMM

Test results

Our tests demonstrated a 2.4 times relative gain for OpenFOAM, which is seen as among the top 5 HPC software platforms with a large open-source community. Used widely in universities and R&D centers, the high parallelization nature of the software takes advantage of both memory (increased bandwidth) and CPU features like denser cores.

Molecular dynamics⁶ with Micron DDR5 run two times faster

CP2K is an open-source quantum chemistry tool that can be used for a number of applications, including simulations of solid-state biological systems. CP2K provides a general framework for different modeling methods such as DFT using the mixed Gaussian and plane wave approaches GPW and GAPW. The example that we looked at was linear-scaling density functional theory (DFT) of water (H2O) consisting of 6144 atoms in a 39-cubic-angstrom box (2048 water molecules in total).

Software stack used for this workload

• H₂O-DFT-LS.NREP4 & H2O-DFT-LS
• Alma 9 Linux kernel 5.14

Test setup

• DDR4 system 3^rd Gen AMD EPYC Processors with 64 cores and 3.7 GHz; DDR4 3200 MHz system2 is fully populated with 64GB RDIMM
  
• DDR5 system 4^th Gen AMD EPYC Processors with 96 cores and 3.7 GHz; DDR5 4800 MHz system3 is fully populated with 64GB RDIMM

Test results

Our tests demonstrated a 2.1 times relative gain for molecular dynamics, and this scales well with more cores and more memory bandwidth.

Summary

The results above are just the start — and just a few examples of HPC workloads. The ability to better match high-performance, high-bandwidth memory with the incredible performance offered by new server processors such as the 4th Gen AMD EPYC Processors stands to be a watershed moment for HPC customers. We can expect to see many more such proof points that demonstrate how enterprise data center and cloud operators can use Micron DDR5 on these new platforms to unlock new levels of performance and efficiency. We look forward to sharing these with you in the coming months. To learn more about Micron DDR5 and data center workload benefits, visit Micron.com/ddr5.