Re: [RFC] Memory tiering kernel alignment

[Matthew Wilcox <willy@infradead.org>]

On Thu, Jan 25, 2024 at 10:26:19AM -0800, David Rientjes wrote:
> There is a lot of excitement around upcoming CXL type 3 memory expansion
> devices and their cost savings potential.  As the industry starts to
> adopt this technology, one of the key components in strategic planning is
> how the upstream Linux kernel will support various tiered configurations
> to meet various user needs.  I think it goes without saying that this is
> quite interesting to cloud providers as well as other hyperscalers :)

I'm not excited.  I'm disappointed that people are falling for this scam.
CXL is the ATM of this decade.  The protocol is not fit for the purpose
of accessing remote memory, adding 10ns just for an encode/decode cycle.
Hands up everybody who's excited about memory latency increasing by 17%.

Then there are the lies from the vendors who want you to buy switches.
Not one of them are willing to guarantee you the worst case latency
through their switches.

The concept is wrong.  Nobody wants to tie all of their machines together
into a giant single failure domain.  There's no possible redundancy
here.  Availability is diminished; how do you upgrade firmware on a
switch without taking it down?  Nobody can answer my contentions about
contention either; preventing a single machine from hogging access to
a single CXL endpoint seems like an unsolved problem.

CXL is great for its real purpose of attaching GPUs and migrating memory
back and forth in a software-transparent way.  We should support that,
and nothing more.

We should reject this technology before it harms our kernel and the
entire industry.  There's a reason that SGI died.  Nobody wants to buy
single image machines the size of a data centre.