AMD EPYC "Milan" CPU Brief

Updated: Mar 9

So one of the bright spots of 2020 is going to be when AMD release the new EPYC Milan CPU fare, which is going to be sometime in September or October of 2020.

It would have been August but with the Great Corona Caper currently going on, it, like everything else on our planet, it has been impacted.

Some of our tuned in to the core EPYC evangelistas noted the latest revision to AMD's Programmer Reference Manual points to PCID and MPK/PKEY support coming on future AMD EPYC CPUs.

New Linux patches now confirm that the MPK support is on the way with next-gen EPYC processors.

The Memory Protection Key (PKE) support has been supported by Intel processors since Xeon Skylake and allow for page-based memory protections with the RDPKRU and WRPKRU instructions.

As noted in the PRM, the feature "provides a way for applications to impose page-based data access protections (read/write, read-only or no access), without requiring modification of page tables and subsequent TLB invalidation's when the application changes protection domains."

Sent out recently wuz a set of patches reusing the Linux kernel's X86_INTEL_MEMORY_PROTECTION_KEYS code path and supporting MPK within the AMD SVM KVM virtualization code.

The patch comment spells it out quite clearly that this is for next-gen EPYC CPUs, "AMD's next generation of EPYC processors support the MPK (Memory Protection Keys) feature."

The patches are quite straight-forward in re-using the kernel's existing MPK code plumbed in previously by Intel and just two dozen or so lines of code for the AMD SVM code within the Kernel-based Virtual Machine (KVM).

It's possible this AMD MPK support will be mainlined for Linux 5.8.

The patches may be revised as an upstream Intel developer has asked not to have the Kconfig X86_INTEL_MEMORY_PROTECTION_KEYS option to the proposed X86_MEMORY_PROTECTION_KEYS to avoid configuration file woes when transitioning to new kernel versions, so we'll see, but in any case it's a small enough change that there still is time to get this material queued for the forthcoming 5.8 cycle kicking off in June and should be out and stable in August sometime.

The AMD EPYC Milan processors will succeed the current EPYC Rome lineup.

The fundamental change for the EPYC Milan lineup would be the new Zen 3 core architecture which will be based upon a 7nm+ process node as per ZEN 2.

From what I know and what AMD has officially shown, the AMD Zen 3 based EPYC Milan processors would focus primarily on performance per watt enhancements but that doesn't mean we won't be looking at core updates.

While performance per watt is at the core of Zen 3 cores, they would also deliver an uplift to overall performance.

EPYC Milan main features:

  • 7nm+ Zen 3 cores (~64 core / 128 thread)

  • Pin Compatible With SP3 Socket

  • 120W-225W TDP SKUs

  • PCIe 4.0 Support

  • DDR4 Memory Support

  • Launch in 2020

AMD has shown that unlike Zen 2 which has 16 MB of L3 cache per CCX within a CCD, Zen 3 would feature a shared cache (32 MB+) for each die.

This would allow all cores to share the entirety of the L3 cache available on the die rather than each CCX having its smaller and separate cache shared among the cores.

This is confirmation of Milan offering 8 Zen 3 cores within a single CCX.

AMD also confirmed that they are currently sampling the first Milan CPUs which is fantastic news for customers awaiting the chips in 2020.

These are still 7nm by the way the Genoa stuff coming after these ones will be 5nm.

These ZEN 3 architectures are brand new though, they are not an extension of the ZEN 2 stuff that the EPYC Rome 7002 series is based on.

I will be getting my Milan sample set in July.

As AMD goes to smaller process nodes (5nm), we should also start seeing the company further ramp core counts.

Current Rome-based CPUs top out at 64 cores/128 threads, and it's expected that Milan will hold the line on core counts.

However, Zen 4 (Genoa) will double that core count to 128/256.

SMT4 is 4 threads per core!! This will be here with Genoa apparently.

There's a number of application areas that just continue to benefit from increasing core counts and increasing compute density, Nutanix AOS is going to be in Operating system heaven.

AMD has experienced stronger than expected demand for its 48- and 64-core Rome processors, with particularly strong interest from supercomputer/HPC and virtualized HCI systems that Nutanix focuses on.

New Nutanix nodes are certified for the single Socket EPYC systems and the Dual Socket systems will be making an arrival in 90 days.

Right now HPe and Lenovo have certified AMD platforms on the Nutanix HCL.

This is great news for Nutanix because these nodes that are EPYC Rome or Milan based will have HUGE increases in performance over the Intel XEON fare with no hint of DIMM challenges either.

It will also bring Intel Optane based NVMe SSD to the table for even more explosive and radical HPC compute solutions running Nutanix AOS.

AMD and nVidia have also been working on various GPU pairings that Nutanix also embrace with the T4 series GPU cards and now there is a platform that can take full advantage of PCIe 4.0 based Optane SSD and nVidia T4 pairings in the same node and push all 6 cores per T4.

We could have 4 x T4 cards per node in the right configuration but the use case for that is very niche.

The Nutanix AMD EPYC powered nodes are going to be like comparing Light beam powered UFO's to 1930's roller skates.

Oh and FYI, The Nutanix FRAME protocol that competes with RDP, PCoIP and MechDyne's TGX makes them all look like Neanderthals in comparison.

FRAME FGP is H.264 based and can drive multiple 12K monitors easily.

Customers running nVidia T4 GPU's in Nutanix AMD based nodes using AHV and FRAME will be able to drive two 8K monitors or 4 x 4K Monitors with relative ease.

Interesting times from the HPC angle for HCI consumers awaits all users of Nutanix AMD EPYC powered platforms.

I have been getting into the ZEN 5 Raphael stuff as well chatting about nice to have and must have features.

By the time Raphael hits us in 2022 the server Chip world will be mostly AMD centered and so far advanced compared to anything Intel pulls out the box that they will not be a player in this space going forward.

Intel have long put off the inevitable that people like myself have been bleating into their ear-holes to no avail since 2005.

They can't say they did not get fair warnings from the Itanic (Itanium) experience....

A ZEN moment if ever there was one....

Intel Itanic going down with Xeon DIMM Issues onboard