Intel's Nervana, a neural network chip for inference-based workloads, will lack a standard cache hierarchy, and software will directly manage on-chip memory
At a press event at the 2019 Consumer Electronics Show, Intel announced the Nervana Neural Network Processor (NNP-I) …
Context & Ripple Effects
The NNP-I is the inference half of a plan Intel has been assembling since it detailed its Nervana-based AI chip vision in late 2016, with Facebook supplying technical input since 2017 and Baidu later signed on for the training-oriented sibling. What is new at CES 2019 is an architectural admission: for inference workloads, Intel is willing to abandon the standard cache hierarchy entirely and hand memory management to software.
That choice matters because it defines who can actually use the chip — only customers with the compiler and runtime muscle to schedule on-chip memory themselves, which is precisely the profile of the hyperscalers Intel has been co-designing with. It also foreshadows the cost-optimized inference hardware Intel would still be shipping years later in Crescent Island, built around LPDDR5X rather than expensive HBM.
First-order effects
- Facebook and Baidu, as named technical contributors, get an inference part shaped to their own workloads — but their software teams inherit direct responsibility for managing on-chip memory that a cache hierarchy would normally handle.
Second-order effects
- By stripping out general-purpose caching, Intel ties the NNP-I's performance to its software stack, making the compiler and tools layer — not the silicon alone — the competitive surface against rival inference accelerators.
Third-order effects
- If the co-design pattern holds, data center AI silicon consolidates around a handful of hyperscalers able to fund bespoke architectures, while the memory hierarchy itself becomes a per-workload software decision rather than a fixed hardware convention — a trajectory visible from the NNP-I through Crescent Island's LPDDR5X cost trade-off.
The trend: AI chips are splitting off from general-purpose CPU design into workload-specialized accelerators co-engineered with large customers, where software, not hardware caches, governs memory.