Re: Debian linux-image-2.6.32-4-xen-amd64 2.6.32-11 doesn't boot with > 4 GiB; resets immediatelly, no log messages

[Thomas Goirand <thomas@goirand.fr>]

Jeremy Fitzhardinge wrote:
> Yes, you system has 3990MB of RAM; the hardware/BIOS is hiding, using or
> otherwise disabling 106MB of memory.

Possible, yes. Especially video could allocate some RAM anyway.

>>  and 645 MB free, and "xm list" shows that dom0 is
>> using 3286 MB. Already, things are weird because 645 free + 3286 for the
>> dom0 makes 3931 and not 3990, and I never asked that my dom0 takes
>> anything (at boot-up it should take all the available memory).
>>   
> 
> I would assume that extra 59MB is memory Xen has allocated for its own
> purposes on behalf of the domain.  And Xen won't give all memory to dom0
> at boot; it will always leave some memory free.

Quite not. This already allocated RAM that seems to be gone, isn't
always the same amount. Plus 60MB, I never ever saw Xen taking that much.

>> Then I do:
>>
>> xm mem-set Domain-0 2048
>>
>> and then I get:
>> xm list Domain-0: 2048
>> xm info Free RAM: 1627
>>
>> That makes a total of 3675 MB, which is less than the total of what I
>> had at boot time without using xm mem-set.
> 
> So the apparent overhead has grown from 59MB to 315MB?

No. Apparently, the memory manager of the hypervisor is somehow broken.
These values aren't always the same, this is quite random and weird,
even the first amount of "lost RAM" just right after booting is not
always the same (10s of MB). Once, I even had up to a gig that was gone.

>>  Ever more weirdness: the
>> amount of free RAM after a xm mem-set call does vary randomly each time
>> I boot my laptop (a one year old Lenovo T500).
>>   
> 
> How much variation?  By a few pages?  10s of MB?

100s of MB variation.

> What happens if you just boot with dom0_mem=2048MB at boot time rather
> than shrinking it after boot?

If I do that, I can't even boot, when X starts (KDM), my screen becomes
dark and it seems that the video initialization fails to be done
correctly! I'd be hard to tell if it's a driver timing or a Xen issue.

> This should all be an internal-to-Xen issue, so I wouldn't expect the
> kernel to have any effect.

Agreed with the diagnostic, this is an hypervisor issue.

>> "xm debug-keys m" shows nothing, when exactly should I type it (sorry,
>> I'm not familiar with Xen debugging...).
>>   
> 
> The result will appear on the console, so you may need to follow it with
> "xm dmesg" to see the output.
> 
> I don't know all that much about how Xen does its internal memory
> management, but aside from the ~300MB apparently lost by shrinking dom0
> the amounts of memory you're talking about are small, and well within
> the sort of range I'd expect for allocator overhead.

I've been using Xen for YEARS, and no, this is not a normal behavior
that I'm seeing here. Having 1GB gone is NOT a normal situation, this is
a lot more than just few bytes for the internals of the hypervisor.

Thomas