Re: [BUG] DAX access of Memory Expander on RCH topology fires BUG on page_table_check

[Gregory Price <gregory.price@memverge.com>]

On Mon, Apr 17, 2023 at 11:43:27PM -0700, Dan Williams wrote:
> Gregory Price wrote:
> > Why NUMA-mode works under these conditions without crashing the system
> > is escaping me at the moment,
> 
> Why would it crash? That range is valid within
> 0x1050000000-0x304fffffff.
> 

Basically I was expecting a page-fault in NUMA to produce the same
effects as a fault fault in DAX, clearly this is not the case and either
the switch from numa to dax is causing 

> >  given that the page faulting system goes
> > through the same driver.  But my guess is that pfn-to-page mappings are
> > off in some way when placed in devdax mode, whereas they're correct
> > under numa mode.
> 
> pfn-to-page is pretty simple, its the pfn to page_ext that's concerning
> for CONFIG_PAGE_TABLE_CHECK.
> 

Testing CONFIG_PAGE_TABLE_CHECK=n now, will report back when done.

> > Note that the above code chops off the first 768MB of the dax region and
> > the last 1.25GB of the dax region.
> 
> Yes, if the core-mm picks 2GB for the block size (which it does for
> systems with more the 64GB of memory, then it will align hot-added
> ranges.
> 
> > The CFWM is required to be 256MB aligned, but this code will force
> > anything mapped into that area to be 2GB aligned.  I don't think it's
> > safe to safe the BIOS is wrong.
> 
> The *minimum* alignment of the CFMWS window is 256M, but if they don't
> want to waste memory on Linux they had better make it 2GB aligned.
> 
> BIOS looks ok here.
>

FWIW i have a QEMU instance with 64GB that puts CXL devices on 256MB
alignment as well, so QEMU instances over a certain amount of DRAM
produce the same effect as hardware - lost memory.

~Gregory