5 things you need to know about AI at the edge

In today’s tech-driven world, AI is revolutionizing industries. Whether you’re into mobile phones, PCs or cars, you need to understand the crucial role of memory and storage in AI workloads on edge devices. At Micron Technology, we know that, while compute like GPUs, tensor cores and NPUs are vital, AI workloads often hit a memory wall before computing resources are maxed out. Our industry-leading memory and storage solutions are designed to meet these extreme data demands at the edge, ensuring performance, efficiency and reliability.

The rise of AI workloads on edge devices is a massive opportunity for memory and storage makers. Gartner projects that spending for generative AI-capable edge device hardware will increase by 99.5% to $3.9 billion in 2025. Micron, a leader in DRAM and NAND technology, is a key player in the AI ecosystem and perfectly positioned to capitalize on this market.

When you consider how AI will grow in the cloud and on edge devices — whether they’re AI PCs, autonomous robots, cars or mobile phones — you need to know these five key things.

1. When inference moves to edge devices, efficiencies will be realized and AI adoption will increase

AI inference on edge devices brings several benefits — reduced latency, improved privacy and less reliance on a network connection. By processing data locally instead of sending it to the cloud, edge devices offer faster and more responsive AI applications. For example, in autonomous driving, vehicles used to rely heavily on cloud servers for processing sensor data, which could introduce delays. Now, with edge AI, vehicles can process data in real-time on-board, making immediate driving decisions without waiting for cloud responses. This shift is expected to drive greater adoption of AI technologies across industries, enabling real-time decision-making and enhancing user experiences. When consumers enjoy valuable AI experiences on their devices, adoption will increase.

2. The cloud is not going away — a distributed model for AI is the most likely outcome

While edge computing is on the rise, the cloud remains a key part of the AI ecosystem. A distributed hybrid model, combining edge and cloud computing, is emerging as the best solution for AI workloads. The cloud will continue to handle large-scale data processing, model training and centralized management while edge devices manage real-time inference and localized processing.

This hybrid approach uses the strengths of both edge and cloud, offering flexibility, efficiency and scalability. Additionally, agentic AI — that is, autonomous AI systems making intelligent decisions without human intervention — helps seamlessly integrate edge and cloud environments, optimizing performance, enhancing security and ensuring efficient resource allocation. In simple terms, an AI agent can reside on your device — whether it’s your phone, PC or car — and when it comes up against a question it can’t fully answer, it will automatically reach out to a more complex or specialized AI model in the cloud or data center to get the answer. Then it will return a more precise response to you.

In short, edge devices will still rely on AI data centers and cloud computing for certain tasks, but they won’t need the cloud for all AI inferencing.

3. AI at the edge and in the cloud is the ultimate data challenge

Managing AI workloads in both edge and cloud environments presents unique data challenges. The sheer volume and variety of data — coupled with the need for real-time processing — require innovative solutions. Micron’s advanced memory and storage technologies are engineered to address these challenges, offering the performance, reliability and efficiency needed for complex AI data workloads.

Memory bottlenecks are a significant issue, especially during training and inference phases. High-bandwidth memory (HBM3E) helps to alleviate these bottlenecks in the cloud, while LPDDR5X offers high bandwidth and power efficiency for edge devices. These technologies ensure smooth and efficient AI applications, whether on edge devices or in the cloud.

Our customers, across all products, rely on our leadership and expertise to navigate these data challenges effectively.

4. Memory and storage are more important than ever on edge devices and in the cloud

As AI models grow in complexity, the demand for memory and storage capacity increases. Both edge devices and cloud infrastructure need to support these expanding models without compromising performance. Micron’s memory and storage solutions are designed to meet these demands, providing the necessary capacity and speed for AI.

Our products are built on industry-leading process nodes and offer superior power efficiency, the most advanced being Micron’s 1γ (1-gamma) process node, which leads against all competitors. For AI data centers, high-bandwidth memory (HBM3E and HBM4) breaks down the memory wall for AI acceleration. AI data centers need a full hierarchy of memory and storage for best performance, including high-density DDR5 modules, LPDDR5X, CXL-based expansion memory pools using Micron CZ122, local SSD data cache using Micron 9650 NVMe™ SSDs, and networked data lakes using Micron 6600 ION.

Similarly, edge devices need specialized memory and storage, like low-power DRAM (LPDDR5X), designed for mobile phones and expanding to AI PC and automotive applications for its low-power capabilities and high-bandwidth performance. Universal flash storage (UFS 4.0), PCIe Gen5 SSDs like the Micron 4600 NVMe SSD, and 4150AT centralized storage SSDs solutions ensure that AI applications run smoothly and efficiently, whether on edge devices or in the cloud.

5. You don’t have to solve all your data challenges on your own — Micron is here to help

With over 45 years of experience, Micron is the trusted partner for mobile phone makers, PC OEMs and automakers. Our expertise in memory and storage solutions is unparalleled, and we are committed to helping our customers achieve their AI goals. Our industry knowledge and innovative products make us the ideal collaborator for solving the most difficult AI data challenges and pursuing new opportunities to accelerate the AI evolution. Together, we can navigate the complexities of AI and drive forward intelligent technology.

As AI continues to evolve, the importance of memory and storage in edge applications and devices cannot be overstated. Companies in the mobile, PC and automotive sectors — as well as those in industrial AI and robotics — must prioritize these components to ensure the success of their AI workloads. Micron is here to support these companies, with solutions that are fast, efficient and reliable.

Our technology doesn't just store data; it accelerates the transformation of data into actionable intelligence.