Re: Re: [PATCH 0 of 3] Patches for PCI passthrough with modified E820 (v3) - resent.

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

> > Linux version 2.6.18-194.el5xen (mockbuild@builder10.centos.org) (gcc version 4.
> > 1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Fri Apr 2 15:34:40 EDT 2010
> > BIOS-provided physical RAM map:
> >  Xen: 0000000000000000 - 00000000bffb0000 (usable)
> >  Xen: 00000000bffb0000 - 00000000bffbe000 (ACPI data)
> >  Xen: 00000000bffbe000 - 00000000bffe0000 (ACPI NVS)
> >  Xen: 00000000bffe0000 - 00000000c0000000 (reserved)
> >  Xen: 00000000fec00000 - 00000000fec01000 (reserved)
> >  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
> >  Xen: 00000000fff00000 - 0000000100000000 (reserved)
> >  Xen: 0000000100000000 - 0000000140850000 (usable)
> > ..
> > Memory: 3026548k/5251392k available (2512k kernel code, 118176k reserved, 1396k 
> > ..
> > when it finished booting, I found that 4GBs are available:
> > [root@g-rhel5-amd64 ~]# cat /proc/meminfo  | grep MemTotal
> > MemTotal:      4186112 kB
> 
> Can you actually allocate and use (nearly) all of that 4G? (modulo
> kernel allocations/overheads etc). The above doesn't really show that

memhog 4G worked great.. but then I noticed it started slowing down and
it was using the swap disk?
> the p2m above 4G is actually sane (I admit I'd be surprised if we got
> away with it enough to boot if it wasn't...).

So your inquisitive questions did find an issue. It looks as while
top and /proc/meminfo both report 4G, they also report over 1G being
'used'.

Mem:   4186112k total,  1451684k used,  2734428k free,    13664k buffers

I think that you are spot on with the P2M between bffb0 and
10000 not being used. And the P2M's after 10000 are.

So then I played with the 'maxmem=4096', 'memory=2048' :

BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000080000000 (usable)
 Xen: 0000000080000000 - 00000000bffb0000 type 5
 Xen: 00000000bffb0000 - 00000000bffbe000 (ACPI data)
 Xen: 00000000bffbe000 - 00000000bffe0000 (ACPI NVS)
 Xen: 00000000bffe0000 - 00000000c0000000 (reserved)
 Xen: 00000000fec00000 - 00000000fec01000 (reserved)
 Xen: 00000000fee00000 - 00000000fee01000 (reserved)
 Xen: 00000000fff00000 - 0000000100000000 (reserved)
 Xen: 0000000100000000 - 0000000180800000 (usable)

and when I ballooned the memory up: 'xl memset  4096' it showed 
up it was using the memory above the 4G:

Mem:   4186112k total,   419964k used,  3766148k free,    13776k buffers

So much much better. And yes, memhog 4G ran with no trouble.


PVOPS is much better equiped for this: 'memory=4096':

[    0.000000] released 261886 pages of unused memory
[    0.000000] 1-1 mapping on bffb0->100000
[    0.000000] Set 262224 page(s) to 1-1 mapping.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 00000000bffb0000 (usable)
[    0.000000]  Xen: 00000000bffb0000 - 00000000bffbe000 (ACPI data)
[    0.000000]  Xen: 00000000bffbe000 - 00000000bffe0000 (ACPI NVS)
[    0.000000]  Xen: 00000000bffe0000 - 00000000c0000000 (reserved)
[    0.000000]  Xen: 00000000fec00000 - 00000000fec01000 (reserved)
[    0.000000]  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  Xen: 00000000fff00000 - 0000000100000000 (reserved)
[    0.000000]  Xen: 0000000100000000 - 000000018074e000 (usable)
..
[    0.000000] Memory: 2535068k/6298936k available (4653k kernel code, 1049344k absent, 2714524k reserved, 4034k data, 716k init)

top shows:
Mem:   2978632k total,   514812k used,  2463820k free,        0k buffers

and dom0:
konrad@phenom: sudo xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2718     6     r-----  633351.2
latest                                      20  3073     1     -b----       4.1


and after ballonning:
Mem:   4027200k total,   518884k used,  3508316k free,        0k buffers

konrad@phenom:~$ sudo xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  2718     6     r-----  633385.5
latest                                      20  4097     1     -b----       7.3

(thought it was curious, the balloon driver thought the target_kb was 4G
and not 3G (which is what 'xl list' showed). Giving a big number (I did 9G, but
probably 5G would have done) to ./devices/system/xen_memory/xen_memory0/target_kb
made it balloon up.

Looks like there is some need for fixes in the balloon accounting code.

Anyhow, seems that if you are using RHEL5, SLES11, you need to be carefull to
use 'memory' and 'maxmem'. With the PVOPS, need to balloon up and you are OK.

Thought I do want to see about writting the code that would automatically balloon
back to the amount of memory that was deflated.