Re: [Xen-users] Re: Xen 3.1 released!

Re: [Xen-users] Re: Xen 3.1 released!

[Tomasz Chmielewski]

Petersson, Mats schrieb:

>>>  - 32-on-64 PV guest support (run PAE PV VMs on a 64-bit Xen!); and
>> I checked - indeed it works. It is possible to create a 32 
>> bit PV guest 
>> on a 64-bit Xen.
>>
>> However, migration of such hosts doesn't seem to work: 
>> whenever I try to 
>> migrate a 32-bit PV guest from a 64-bit Xen to a 32-bit Xen (or the 
>> other way around), it fails, and the guest domain is not 
>> reachable anymore.
> 
> No, I seem to remember that this is a known bug - check the Xen Devel
> mailing list archive for the last few weeks, I'm 99% sure there is a
> mention about this particular problem, and I don't think it's
> (completely) fixed in unstable yet either. 

I didn't find any posts about it - at least, not after the 3.1 release 
date (maybe it was discussed earlier).


Also, there's one more feature which doesn't work for me:

  - Preliminary save/restore/migrate support for HVM (e.g. Windows) VMs;


What does "preliminary" mean here? I tried saving/restoring HVM domains, 
but it just doesn't work.

When I try to do /etc/init.d/xendomains stop, it does something for a 
long time, and then, my SSH session is just disconnected.

When I log in again, xend doesn't work anymore:

# xm list
Error: Unable to connect to xend: Connection refused. Is xend running?


When I start xend again, and then start xendomains, there is my domain 
in a weird state:

# xm list
Name                                      ID   Mem VCPUs      State 
Time(s)
Domain-0                                   0   256     2     r-----    134.9
migrating-w2k3-001                         2  1759     1     ---s--     64.4



Similar happens when I try to save the domain manually:

# xm save w2k3-001 /srv/check-2k

And Xen server restarted.

(...)

I inspected the logs, and it was the OOM-killer. Should it ever happen 
when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB 
total), Xen guest 1.7 GB.

-- 
Tomasz Chmielewski
http://wpkg.org

RE: [Xen-users] Re: Xen 3.1 released!

[Petersson, Mats]

 

> -----Original Message-----
> From: Tomasz Chmielewski [mailto:mangoo@wpkg.org] 
> Sent: 25 May 2007 11:54
> To: Petersson, Mats
> Cc: xen-users@lists.xensource.com; xen-devel@lists.xensource.com
> Subject: Re: [Xen-users] Re: Xen 3.1 released!
> 
> Petersson, Mats schrieb:
> 
> >>>  - 32-on-64 PV guest support (run PAE PV VMs on a 64-bit 
> Xen!); and
> >> I checked - indeed it works. It is possible to create a 32 
> >> bit PV guest 
> >> on a 64-bit Xen.
> >>
> >> However, migration of such hosts doesn't seem to work: 
> >> whenever I try to 
> >> migrate a 32-bit PV guest from a 64-bit Xen to a 32-bit 
> Xen (or the 
> >> other way around), it fails, and the guest domain is not 
> >> reachable anymore.
> > 
> > No, I seem to remember that this is a known bug - check the 
> Xen Devel
> > mailing list archive for the last few weeks, I'm 99% sure there is a
> > mention about this particular problem, and I don't think it's
> > (completely) fixed in unstable yet either. 
> 
> I didn't find any posts about it - at least, not after the 
> 3.1 release 
> date (maybe it was discussed earlier).

Yes, it was before the release that someone pointed this out (someone is
most likely either Gerd Hoffman or Jan Beulich). 

> 
> 
> Also, there's one more feature which doesn't work for me:
> 
>   - Preliminary save/restore/migrate support for HVM (e.g. 
> Windows) VMs;
> 
> 
> What does "preliminary" mean here? I tried saving/restoring 
> HVM domains, 
> but it just doesn't work.

It is the first PUBLIC release of the hvm save/restore. 
> 
> When I try to do /etc/init.d/xendomains stop, it does something for a 
> long time, and then, my SSH session is just disconnected.
> 
> When I log in again, xend doesn't work anymore:
> 
> # xm list
> Error: Unable to connect to xend: Connection refused. Is xend running?
> 
> 
> When I start xend again, and then start xendomains, there is 
> my domain 
> in a weird state:
> 
> # xm list
> Name                                      ID   Mem VCPUs      State 
> Time(s)
> Domain-0                                   0   256     2     
> r-----    134.9
> migrating-w2k3-001                         2  1759     1     
> ---s--     64.4
> 
> 
> 
> Similar happens when I try to save the domain manually:
> 
> # xm save w2k3-001 /srv/check-2k
> 
> And Xen server restarted.
> 
> (...)
> 
> I inspected the logs, and it was the OOM-killer. Should it 
> ever happen 
> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB 
> total), Xen guest 1.7 GB.

Hmm, that's probably something to do with it. Although I'm not sure. 

I've saved 0.5GB domains ona  256MB Dom0 and that works. I'll try to
blow mine up a bit further and see what happens. 

--
Mats
> 
> -- 
> Tomasz Chmielewski
> http://wpkg.org
> 
> 
> 

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Tomasz Chmielewski]

Tomasz Chmielewski schrieb:

> Also, there's one more feature which doesn't work for me:
> 
>  - Preliminary save/restore/migrate support for HVM (e.g. Windows) VMs;
> 
> 
> What does "preliminary" mean here? I tried saving/restoring HVM domains, 
> but it just doesn't work.
> 
> When I try to do /etc/init.d/xendomains stop, it does something for a 
> long time, and then, my SSH session is just disconnected.
> 
> When I log in again, xend doesn't work anymore:
> 
> # xm list
> Error: Unable to connect to xend: Connection refused. Is xend running?
> 
> 
> When I start xend again, and then start xendomains, there is my domain 
> in a weird state:
> 
> # xm list
> Name                                      ID   Mem VCPUs      State Time(s)
> Domain-0                                   0   256     2     r-----    
> 134.9
> migrating-w2k3-001                         2  1759     1     ---s--     
> 64.4
> 
> 
> 
> Similar happens when I try to save the domain manually:
> 
> # xm save w2k3-001 /srv/check-2k
> 
> And Xen server restarted.
> 
> (...)
> 
> I inspected the logs, and it was the OOM-killer. Should it ever happen 
> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB 
> total), Xen guest 1.7 GB.

It appears that saving of HVM somains needs more memory: in this 
particular case, when I increased memory for dom0, about 350 MB RAM was 
used in total during the domain save.

After that, save/restore was fine.

So I just removed dom0_mem entry from grub; I don't think it's needed.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Keir Fraser]

On 25/5/07 12:14, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:

>> I inspected the logs, and it was the OOM-killer. Should it ever happen
>> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB
>> total), Xen guest 1.7 GB.
> 
> It appears that saving of HVM somains needs more memory: in this
> particular case, when I increased memory for dom0, about 350 MB RAM was
> used in total during the domain save.

That's kind of surprising, but I guess totally plausible if you don't
configure a swap partition for dom0?

 -- Keir

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Tomasz Chmielewski]

Keir Fraser schrieb:
> 
> 
> On 25/5/07 12:14, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:
> 
>>> I inspected the logs, and it was the OOM-killer. Should it ever happen
>>> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB
>>> total), Xen guest 1.7 GB.
>> It appears that saving of HVM somains needs more memory: in this
>> particular case, when I increased memory for dom0, about 350 MB RAM was
>> used in total during the domain save.
> 
> That's kind of surprising, but I guess totally plausible if you don't
> configure a swap partition for dom0?

No - dom0 has a 2 GB swap partition.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Keir Fraser]

On 25/5/07 13:57, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:

>>>> I inspected the logs, and it was the OOM-killer. Should it ever happen
>>>> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB
>>>> total), Xen guest 1.7 GB.
>>> It appears that saving of HVM somains needs more memory: in this
>>> particular case, when I increased memory for dom0, about 350 MB RAM was
>>> used in total during the domain save.
>> 
>> That's kind of surprising, but I guess totally plausible if you don't
>> configure a swap partition for dom0?
> 
> No - dom0 has a 2 GB swap partition.

Hmmm... That's not so explicable. Obviously something in save/restore is
causing unpageable allocations to happen. Could be pagetables for mappings
of HVM guest memory, or something like that. Needs investigating further, I
think.

 -- Keir

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Tomasz Chmielewski]

Keir Fraser schrieb:
> 
> 
> On 25/5/07 13:57, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:
> 
>>>>> I inspected the logs, and it was the OOM-killer. Should it ever happen
>>>>> when saving a guest domain? Xen host has 256 MB RAM for dom0 (3 GB
>>>>> total), Xen guest 1.7 GB.
>>>> It appears that saving of HVM somains needs more memory: in this
>>>> particular case, when I increased memory for dom0, about 350 MB RAM was
>>>> used in total during the domain save.
>>> That's kind of surprising, but I guess totally plausible if you don't
>>> configure a swap partition for dom0?
>> No - dom0 has a 2 GB swap partition.
> 
> Hmmm... That's not so explicable. Obviously something in save/restore is
> causing unpageable allocations to happen. Could be pagetables for mappings
> of HVM guest memory, or something like that. Needs investigating further, I
> think.

Could be.

Now that I repeated the save/restore a couple of times, total RAM usage 
for dom0 jumps to ~700 MB sometimes during domain save (dom0 has now 
about 2.3 GB RAM, domU has 1.7 GB RAM; 4 GB RAM installed now in the 
machine).


-- 
Tomasz Chmielewski
http://wpkg.org

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Keir Fraser]

On 25/5/07 14:20, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:

>> Hmmm... That's not so explicable. Obviously something in save/restore is
>> causing unpageable allocations to happen. Could be pagetables for mappings
>> of HVM guest memory, or something like that. Needs investigating further, I
>> think.
> 
> Could be.
> 
> Now that I repeated the save/restore a couple of times, total RAM usage
> for dom0 jumps to ~700 MB sometimes during domain save (dom0 has now
> about 2.3 GB RAM, domU has 1.7 GB RAM; 4 GB RAM installed now in the
> machine).

The only thing I would expect could so dramatically inflate dom0's memory
usage would be caching of disc blocks from the saved image file. But we
added explicit cache flushing to both domain-save and domain-restore before
releasing 3.1. The flushes are only advisory though, so perhaps they are
getting ignored for some reason.

 -- Keir

Re: Re: [Xen-users] Re: Xen 3.1 released!

[Tomasz Chmielewski]

Keir Fraser schrieb:
> On 25/5/07 14:20, "Tomasz Chmielewski" <mangoo@wpkg.org> wrote:
> 
>>> Hmmm... That's not so explicable. Obviously something in save/restore is
>>> causing unpageable allocations to happen. Could be pagetables for mappings
>>> of HVM guest memory, or something like that. Needs investigating further, I
>>> think.
>> Could be.
>>
>> Now that I repeated the save/restore a couple of times, total RAM usage
>> for dom0 jumps to ~700 MB sometimes during domain save (dom0 has now
>> about 2.3 GB RAM, domU has 1.7 GB RAM; 4 GB RAM installed now in the
>> machine).
> 
> The only thing I would expect could so dramatically inflate dom0's memory
> usage would be caching of disc blocks from the saved image file. But we
> added explicit cache flushing to both domain-save and domain-restore before
> releasing 3.1. The flushes are only advisory though, so perhaps they are
> getting ignored for some reason.

For your notice, I just saw 1.3 GB RAM usage when doing a 1.7 GB HVM 
domain save :)


-- 
Tomasz Chmielewski
http://wpkg.org