Re: [PATCH 0/3] x86/platform/UV: Update Memory Block Size Setting

[Mike Travis <mike.travis@hpe.com>]

[re-including everyone for any other comments about these patches...]

On 5/10/2018 11:48 PM, Michal Hocko wrote:
> On Thu 10-05-18 18:18:32, mike.travis@hpe.com wrote:
>>
>> Update support for the UV kernel to accommodate Intel BIOS changes in
>> NVDIMM alignment, which caused UV BIOS to align the memory boundaries
>> on different blocks than the previous UV standard of 2GB.
> 
> Please elaborate (much) more. What is the actual problem and how is the
> patchset addressing it.
> 

On 5/15/2018 1:55 AM, Michal Hocko wrote:
 > On Fri 11-05-18 08:08:14, Mike Travis wrote:
 > [...]
 >> If you think I need this more detailed explanation in the patch 
descriptions
 >> themselves, I'll add it.
 >
 > Yes we definitely want this information along with a high level
 > description of how this got fixed. Incomplete memblocks need to be
 > handled gracefully (especially when blocks are 2GB in size).
 >
 > Thanks!
 >

Hi Michal,

I will add more info but this patch does not address anything about 
incomplete memblocks.  They have existed in 2GB mem block size form 
since 2009 (v2.6) with the first UV1 system release.  I am not changing 
any of that handling.

The fixing part was to adapt to what Intel BIOS was using as a different 
boundary to align the PMEM NVDIMMs.  This is explained in both patch 1 
and patch 2:

"Add a new function to "adjust" the current fixed UV memory block size 
of 2GB so it can be changed to a different physical boundary.  This is 
out of necessity so UV BIOS can accommodate Intel BIOS changes for 
NVDIMM's, which can align these new PMEM modules at other than 2GB 
boundaries."

"Add a call to the new function to "adjust" the current fixed UV memory 
block size of 2GB so it can be changed to a different physical boundary. 
This accommodates changes in the Intel BIOS, and therefore UV BIOS, 
which now can align boundaries different than the previous UV standard 
of 2GB. It also flags any UV Mem boundaries that cause a change in the 
mem block size boundary."

I can't explain why the Intel BIOS changed the boundaries, it probably 
has something to do with accommodating other areas of the NVDIMMs 
physical address space.  This caused a boundary alignment to be less 
than 2GB which UV had been using.  From one of the UV BIOS engineers I 
found out that Intel really only guarantees a 64MB boundary (which is 
actually less than the current Linux minimum of 128MB.)  Our own UVHUB 
requirements dictated that 2GB was an acceptable boundary up until now.

So let me know how much more info is needed on how I need to explain 
_this_ change.

Thanks,
Mike