Re: E820 memory allocation issue on Threadripper platforms

["Roger Pau Monné" <roger.pau@citrix.com>]

On Wed, Jan 17, 2024 at 11:40:20AM +0100, Jan Beulich wrote:
> On 17.01.2024 11:13, Roger Pau Monné wrote:
> > On Wed, Jan 17, 2024 at 09:46:27AM +0100, Jan Beulich wrote:
> >> Whereas I assume the native kernel can deal with that as long as
> >> it's built with CONFIG_RELOCATABLE=y. I don't think we want to
> >> get into the business of interpreting the kernel's internal
> >> representation of the relocations needed, so it's not really
> >> clear to me what we might do in such a case. Perhaps the only way
> >> is to signal to the kernel that it needs to apply relocations
> >> itself (which in turn would require the kernel to signal to us
> >> that it's capable of doing so). Cc-ing Roger in case he has any
> >> neat idea.
> > 
> > Hm, no, not really.
> > 
> > We could do like multiboot2: the kernel provides us with some
> > placement data (min/max addresses, alignment), and Xen let's the
> > kernel deal with relocations itself.
> 
> Requiring the kernel's entry point to take a sufficiently different
> flow then compared to how it's today, I expect.

Indeed, I would expect that.

> > Additionally we could support the kernel providing a section with the
> > relocations and apply them from Xen, but that's likely hm, complicated
> > at best, as I don't even know which kinds of relocations we would have
> > to support.
> 
> If the kernel was properly linked to a PIE, there'd generally be only
> one kind of relocation (per arch) that ought to need dealing with -
> for x86-64 that's R_X86_64_RELATIVE iirc. Hence why (I suppose) they
> don't use ELF relocation structures (for being wastefully large), but
> rather a more compact custom representation. Even without building PIE
> (presumably in part not possible because of how per-CPU data needs
> dealing with), they get away with handling just very few relocs (and
> from looking at the reloc processing code I'm getting the impression
> they mistreat R_X86_64_32 as being the same as R_X86_64_32S, when it
> isn't; needing to get such quirks right is one more aspect of why I
> think we should leave relocation handling to the kernel).

Would have to look into more detail, but I think leaving any relocs
for the OS to perform would be my initial approach.

> > I'm not sure how Linux deals with this in the bare metal case, are
> > relocations done after decompressing and before jumping into the entry
> > point?
> 
> That's how it was last time I looked, yes.

I've created a gitlab ticket for it:

https://gitlab.com/xen-project/xen/-/issues/180

So that we don't forget, as I don't have time to work into this right
now, but I think it's important enough that we don't forget.

For PV it's a bit more unclear how we want to deal with it, as it's
IMO a specific Linux behavior that makes it fail to boot.

Roger.