- US - English
- China - 简体中文
- India - English
- Japan - 日本語
- Malaysia - English
- Singapore - English
- Taiwan – 繁體中文
We characterized the generational uplift for 6th Gen AMD EPYC™ on an agentic application-server workload. Live at AMD Advancing AI, July 22, 2026.
"You're gonna need a bigger boat." — Chief Brody
As AI use shifts to an agentic model, the requirements on your data center infrastructure are going to increase. This drives CPU, memory, storage performance, and density targets up on a per-server basis, but what does the right agentic application server look like? How do you test it?
My team built a benchmark that models a tool-call workload that agentic AI can throw at your infrastructure and worked with AMD's Data Center Ecosystem Applications engineering team to demo it at AMD Advancing AI on July 22. We tested a 5th Gen AMD EPYC 128-core CPU and AMD tested their new 6th Gen AMD EPYC 256-core CPU. The generational uplift of the 6th Gen AMD EPYC CPU with Micron® DDR5-8000 and 9650 PCIe® Gen6 NVMe™ SSDs gains 3.8× the performance at 2.9× better CPU power efficiency.
Thankfully, AMD built us a bigger boat.
The agentic application server
An agentic AI deployment runs on two distinct compute tiers. Inference lives on the AI cluster with accelerator-heavy hosts running the language model and the agent framework. Every time an agent invokes a tool, that request leaves the AI cluster and lands on an application server. The application server does the work, hands the results back, and the agent decides what to do next.
Figure 1. Agentic AI data flow. This benchmark characterizes the application server, highlighted in orange.
The application server is what we set out to characterize. When there are hundreds of tool calls per host, the CPU, memory, and NVMe subsystems are what determine the response time. It's a holistic storage, memory, and compute problem.
Agentic benchmark design
We profiled production agentic workflows and designed a test that uses real applications to model operations common to an application server.
- Coding agents pull assets, look up build metadata, and grep the source tree.
- Document-analysis agents fetch documents, query policy databases, and search corpora.
- Operations agents pull log files, query metrics stores, and grep runbooks.
- Data-analyst agents pull raw exports, query catalog databases, and search prior report definitions.
The specifics differ but the workload profile is similar.
Figure 2. Four representative agentic workflows mapped to the three benchmark applications: S3 GET for asset and corpus fetch, MariaDB® for metadata and session lookup, and ripgrep for text and code search.
- Object storage: S3 GET traffic against a local MinIO™ endpoint. Agents fetch documents, images, vector shards, and RAG corpus chunks. We used a mixed object-size distribution (500 × 1 MiB, 500 × 10 MiB, 190 × 50 MiB per container) to stress the throughput of the NVMe layer.
- Structured queries against MariaDB: 50 queries per cycle across five query shapes (point lookup, range scan, aggregation, ordered scan, distinct). This shows how agents hit their metadata, session, and long-term memory stores.
- Code and text search with ripgrep: 100 ripgrep searches per cycle across the full Linux kernel and CPython source trees. Text search is CPU-heavy and exercises host memory, and it shows up in coding agents, log-analysis agents, and workflows that find references across a corpus.
Each agent runs in its own Docker™ container with a 4GB memory limit. Each container runs one cycle: 1,190 S3 GETs, 50 MariaDB queries, and 100 ripgrep searches.
AMD Advancing AI and 6th Gen EPYC CPUs
At AMD Advancing AI 2026, AMD launched the 6th Gen AMD EPYC Server CPU, built to deliver leadership performance for the agentic AI era. AMD pairs leadership core density, up to 256 cores and 512 threads per socket, with the highest frequency AI host node in the industry at 5GHz and an industry-leading PCIe Gen 6 I/O subsystem that doubles the bandwidth per link versus the previous generation. AMD positions it as an ideal AI host node for scaling agents with dense compute, and as a broad, workload-optimized portfolio for the general-purpose workloads, databases, storage, and CPU inference, that keep organizations running every day.
The new 6th Gen AMD EPYC system enables Micron DDR5-8000 DRAM, raises the data rate 25% over the DDR5-6400 used on the 5th Gen AMD EPYC system and increases memory channels from 12 to 16. Together that lifts peak memory bandwidth from 614 GB/s on the 5th Gen EPYC system to 1,024 GB/s on the 6th Gen EPYC system, a 1.7× increase, with more capacity per socket to hold the working set for hundreds of concurrent agents.
Storage also makes a generational leap. The Micron 9650 SSD is among the first production PCIe Gen6 data center SSDs, and it uses the doubled interface bandwidth to return object and corpus reads faster under the mixed I/O an agent host generates. The 6th Gen AMD EPYC system leveraged PCIe Gen6 Micron 9650 drives while the 5th Gen AMD EPYC system used the PCIe Gen 5 Micron 9550.
| Platform axis | 6th Gen AMD EPYC processor | 5th Gen AMD EPYC processor |
|---|---|---|
| CPU | 6th Gen AMD EPYC CPU 9996 256 cores | 5th Gen AMD EPYC CPU 9745 128 cores |
| Host RAM | 2 TB DDR5-8000 128GB x 16 channels | 1.5 TB DDR5-6400 128GB x 12 channels |
| Concurrent containers | 256 | 128 |
| Container memory limit | 4 GB | 4 GB |
| Storage | Micron 9650 PCIe Gen 6 NVMe SSD 2x 7.68TB | Micron 9550 PCIe Gen 5 NVMe SSD 2x 7.68TB |
| CPU topology | NPS2 (2 IO die); SMT off | NPS1 (1 IO die); SMT off |
Performance results
The key metric is Agentic Operations Per Second (AOPS), the aggregate of completed operations from all the running workloads within the containers divided by the time it took to complete, showing how quickly operations are processed across the entire system. The 6th Gen EPYC CPU sustained 2,185 AOPS versus 581.4 AOPS for Turin, a 3.8× increase. This is with 2× the number of containers, meaning the 6th Gen EPYC CPU completed twice as much work in almost half the time.
Aggregate agentic performance
Figure 3. Aggregate AOPS: 2,185 for the 6th Gen platform and 581.4 for the 5th Gen platform. Figure 4. AOPS per physical core: 8.5 for the 6th Gen platform and 4.5 for the 5th Gen platform.
Our results show 3.8× more aggregate AOPS on the 6th Gen platform. This effectively measures how quickly a system completes 1,190 S3 GETs, 50 MariaDB queries, and 100 ripgrep searches multiplied by the number of containers deployed. The 6th Gen EPYC system completed 343,040 operations in 157 seconds while the 5th Gen EPYC system completed half as many in 299 seconds. On a per-core basis, the 6th Gen EPYC system completed 1.9× more AOPS per core while running twice as many containers on twice as many cores.
Application latency by workload
Even with double the number of running containers, the 6th Gen AMD EPYC CPU delivered lower application latency across all workloads. S3 heavily leverages the mixed I/O performance of the Micron 9650 at PCIe Gen 6, MariaDB benefits from the entire system uplift, and ripgrep is heavily CPU dependent.
Figure 5. Container-average latency for each workload.
CPU performance
The 6th Gen EPYC CPU averaged 33% utilization across all cores, and the 5th Gen EPYC CPU averaged 87%, but that does not tell the full story. Across testing both CPUs frequently burst to 100% utilization.
Figure 6. CPU utilization during each active window: average and peak measured values.
Storage performance
Figure 7. Measured average and peak aggregate storage read throughput. Figure 8. Measured average and peak aggregate storage read IOPS.
The storage workload is complex, with S3 driving large random I/O, MariaDB using 16KB I/O, and ripgrep issuing 4KB I/O and smaller. With two drives the Micron 9650 burst up to 53 GB/s (1.7M IOPS), very near the theoretical max of 28 GB/s per drive. This proves the 9650's capability of not only absorbing extremely high throughput but doing so in difficult mixed I/O workloads.
CPU package power and energy efficiency
Figure 9. Measured average and peak CPU package RAPL power. Figure 10. Measured AOPS per package watt: 5.2 for the 6th Gen platform and 1.8 for the 5th Gen platform.
CPU package RAPL (Running Average Power Limit) power averaged 320.6 W and peaked at 456.4 W on the 5th Gen EPYC CPU. The 6th Gen EPYC CPU averaged 422.8W and peaked at 604.7W. With a higher TDP and twice the core count the 6th Gen EPYC CPU did draw more power, but it also completed more work with that power. Looking at efficiency, AOPS per CPU package watt was 1.8 on the 5th Gen EPYC CPU and 5.2 on the 6th Gen EPYC CPU, a 2.9× increase in CPU power efficiency.
Advancing AI across the data center
AMD and Micron developed this benchmark to demonstrate system performance as the number and diversity of agents per host climbs. IDC projects more than one billion actively deployed AI agents worldwide by 2029, executing 217 billion actions a day and consuming 3.7 trillion tokens and API calls a day to serve that load1. Gartner puts agentic AI spending at $201.9B in 2026 on a trajectory to $753B by 20292. At the rack level, that means more agents per host, more concurrent I/O per agent, and more load on every subsystem inside the server at once.
The 6th Gen AMD EPYC system sustained 2,185 AOPS with 256 containers, besting the previous generation by 3.8× aggregate AOPS and 1.9× AOPS per core, at 2.9× better power efficiency.
Check out our demo at AMD Advancing AI on July 22. We're happy to walk you through the workload design, the telemetry, and what these numbers mean for the servers you're specifying for 2027.
Recognition
It took a cross functional team from Micron and AMD to execute this testing. Thank you.
AMD
Milind Damle, Sr. Director, Software and Solutions
Anre Kashyap, Senior Manager Cloud Engineering
Abimanyu Shah, Senior Software System Designer
Gigi Khmaladze, Software Systems Designer
Apostolos Kotsiolis, AMD EPYC Product Marketing Manager
Ketan Sanghrajka, IHV Solutions Architect
Chandana Ambekar, AI Inference Performance Engineer
Micron
John Mazzie, Member of Technical Staff
Dilim Nwobu, Principal System Performance Engineer
Amit Bodas, Partner Engineering
References
1. IDC (Dec 10, 2025). "Agent Adoption: The IT Industry's Next Great Inflection Point." More than 1 billion actively deployed AI agents worldwide by 2029, executing 217 billion actions per day. https://www.idc.com/resource-center/blog/agent-adoption-the-it-industrys-next-great-inflection-point/
2. Gartner (Jan 15, 2026, via Software Strategies Blog roundup, Feb 26, 2026). Agentic AI spending projected at $201.9B in 2026, expanding to $753B by 2029. https://softwarestrategiesblog.com/2026/02/26/roundup-of-agentic-ai-forecasts-and-market-estimates-2026/
AMD, the AMD arrow logo, EPYC, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Micron and the Micron logo are trademarks or registered trademarks of Micron Technology, Inc. PCIe is a registered trademark of PCI-SIG. NVMe is a trademark of NVM Express, Inc. Docker is a trademark or registered trademark of Docker, Inc. MariaDB is a trademark or registered trademark of MariaDB plc or its subsidiaries. MinIO is a trademark of MinIO, Inc. All other trademarks are the property of their respective owners.
1P AMD EPYC™ 9745 HPE ProLiant DL345 Gen11 production system (1 x 128C) 1.5TB 12x128GB DDR5-6400, 2xMicron 9550 Pro 7.68TB, NPS=1, Ubuntu 24.04.4 LTS, Kernel Linux 6.8.0-134-generic, BIOS 2.2 (determinism enable=power), Testing Date: July 14, 2026.
1P AMD EPYC™ 9996 reference system (1 x 256C) 2.0TB 16x128GB DDR5-8000, 2xMicron 9650 7.68TB, NPS=2, Ubuntu 24.04.4, Kernel Linux nigeria-4370-os 6.18.2-amdsos-build70-ubuntu-24.04+, (determinism enable=power), Testing Date: July 10, 2026.
