Re: Live vm migration broken in latest xen-unstable

Re: Live vm migration broken in latest xen-unstable

[John Byrne]

I have live migration working fine on x86-64 with changeset 11433; 
however, the reason I thought it was broken is that it currently seems 
to need a lot more memory. Using SuSE's latest xenpreview bits I could 
migrate a 3GB guest between two machines (6GB and 4GB). With the new 
bits, a 2GB guest could be migrated off the 6GB machine, but the 4GB 
machine could not migrate it back. I currently have the VM set to 1.5GB 
and it ping-pongs fine. Any idea why so much extra memory seems to be 
required?

John Byrne


> sanjay kushwaha wrote:
> Hi Tim,
> Yes. This patch solved my problem. now live VM migration happens but I
> am experiencing another problem. when I reattach to the migrated VM
> console on the destination machine, I see the following traceback in
> the dmesg of the guest VM.
> 
 > ...snipped...

Live vm migration broken in latest xen-unstable

[sanjay kushwaha]

[-- Attachment #1.1: Type: text/plain, Size: 2614 bytes --]

Folks,
I am experiencing that live migration is not working in latest xen-unstable.
I get the following message during migration

[root@pc5 ksanjay]# xm migrate --live 1 199.77.138.23
Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
[root@pc5 ksanjay]#

I traced the problem to a function in xen named set_sh_allocation() in file
xen/arch/x86/mm/shadow/common.c

tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python script
which makes the following hypercall

    if (live) {
        if (xc_shadow_control(xc_handle, dom,
                              XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
                              NULL, 0, NULL, 0, NULL) < 0) {
            ERR("Couldn't enable shadow mode");
            goto out;
        }
        last_iter = 0;
    } else {
-----------

this particular hypercall leads to the call of set_sh_allocation which fails
in the following code

        if ( d->arch.shadow.total_pages < pages )
        {
            /* Need to allocate more memory from domheap */
            pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
            if ( pg == NULL )
            {
                SHADOW_PRINTK("failed to allocate shadow pages.\n");
                return -ENOMEM;
            }

alloc_domheap_pages fails and returns NULL. however I think I have enough
memory available so this function should not fail.

Is there anybody else experiencing the same problem? Could someone please
tell me how to fix it?

Below is the xm info for my machine.

Thanks for your help,
Sanjay

[root@pc5 ~]# xm info
host                   : pc5
release                : 2.6.16.13-xen0
version                : #3 Fri Sep 1 17:13:13 EDT 2006
machine                : i686
nr_cpus                : 4
nr_nodes               : 1
sockets_per_node       : 2
cores_per_socket       : 1
threads_per_core       : 2
cpu_mhz                : 2791
hw_caps                : bfebfbff:00000000:00000000:00000080:00004400
total_memory           : 511
free_memory            : 1
xen_major              : 3
xen_minor              : 0
xen_extra              : -unstable
xen_caps               : xen-3.0-x86_32
xen_pagesize           : 4096
platform_params        : virt_start=0xfc000000
xen_changeset          : Mon Aug 28 08:08:41 2006 +0100 11267:68a1b61ecd28
cc_compiler            : gcc version 4.0.2 20051125 (Red Hat 4.0.2-8)
cc_compile_by          : root
cc_compile_domain      : netlab.cc.gatech.edu
cc_compile_date        : Fri Sep  1 17:12:28 EDT 2006
xend_config_format     : 2
[root@pc5 ~]#

-- 
----------------------
PhD Student, Georgia Tech
http://www.cc.gatech.edu/~ksanjay/

[-- Attachment #1.2: Type: text/html, Size: 5436 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Live vm migration broken in latest xen-unstable

[Ewan Mellor]

On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:

> Folks,
> I am experiencing that live migration is not working in latest
> xen-unstable. I get the following message during migration
> 
> [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
> Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
> [root@pc5 ksanjay]#
> 
> I traced the problem to a function in xen named set_sh_allocation() in
> file xen/arch/x86/mm/shadow/common.c
> 
> tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python
> script which makes the following hypercall
> 
>     if (live) {
>         if (xc_shadow_control(xc_handle, dom,
>                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
>                               NULL, 0, NULL, 0, NULL) < 0) {
>             ERR("Couldn't enable shadow mode");
>             goto out;
>         }
>         last_iter = 0;
>     } else {
> -----------
> 
> this particular hypercall leads to the call of set_sh_allocation which
> fails in the following code
> 
>         if ( d-> arch.shadow.total_pages < pages )
>         {
>             /* Need to allocate more memory from domheap */
>             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
>             if ( pg == NULL )
>             {
>                 SHADOW_PRINTK("failed to allocate shadow pages.\n");
>                 return -ENOMEM;
>             }
> 
> alloc_domheap_pages fails and returns NULL. however I think I have enough
> memory available so this function should not fail.
> 
> Is there anybody else experiencing the same problem? Could someone please
> tell me how to fix it?

I've put some changes into xen-unstable today which might help.  The last fix
is on its way through testing now.  Look out for xen-unstable changeset
11422, and try that, see how you get on.

Cheers,

Ewan.

Re: Live vm migration broken in latest xen-unstable

[sanjay kushwaha]

[-- Attachment #1.1: Type: text/plain, Size: 2363 bytes --]

Hi Ewan,
I did a "hg pull -u" on my tree which also got the changeset 11422. but I am
still facing the same problem. btw this changeset seems to be specific to
hvm domain while I am facing this problem with paravirtualized domain.

Thanks,
Sanjay

On 9/5/06, Ewan Mellor <ewan@xensource.com> wrote:
>
> On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:
>
> > Folks,
> > I am experiencing that live migration is not working in latest
> > xen-unstable. I get the following message during migration
> >
> > [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
> > Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
> > [root@pc5 ksanjay]#
> >
> > I traced the problem to a function in xen named set_sh_allocation() in
> > file xen/arch/x86/mm/shadow/common.c
> >
> > tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python
> > script which makes the following hypercall
> >
> >     if (live) {
> >         if (xc_shadow_control(xc_handle, dom,
> >                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
> >                               NULL, 0, NULL, 0, NULL) < 0) {
> >             ERR("Couldn't enable shadow mode");
> >             goto out;
> >         }
> >         last_iter = 0;
> >     } else {
> > -----------
> >
> > this particular hypercall leads to the call of set_sh_allocation which
> > fails in the following code
> >
> >         if ( d-> arch.shadow.total_pages < pages )
> >         {
> >             /* Need to allocate more memory from domheap */
> >             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
> >             if ( pg == NULL )
> >             {
> >                 SHADOW_PRINTK("failed to allocate shadow pages.\n");
> >                 return -ENOMEM;
> >             }
> >
> > alloc_domheap_pages fails and returns NULL. however I think I have
> enough
> > memory available so this function should not fail.
> >
> > Is there anybody else experiencing the same problem? Could someone
> please
> > tell me how to fix it?
>
> I've put some changes into xen-unstable today which might help.  The last
> fix
> is on its way through testing now.  Look out for xen-unstable changeset
> 11422, and try that, see how you get on.
>
> Cheers,
>
> Ewan.
>



-- 
----------------------
PhD Student, Georgia Tech
http://www.cc.gatech.edu/~ksanjay/ <http://www.cc.gatech.edu/%7Eksanjay/>

[-- Attachment #1.2: Type: text/html, Size: 4203 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Live vm migration broken in latest xen-unstable

[Yoshiaki Tamura]

sanjay kushwaha wrote:
> Hi Ewan,
> I did a "hg pull -u" on my tree which also got the changeset 11422. but 
> I am
> still facing the same problem. btw this changeset seems to be specific to
> hvm domain while I am facing this problem with paravirtualized domain.
> 
> Thanks,
> Sanjay

I've tested live migration with paravirt domain for changeset 11429,
but didn't have such a problem.
Did you also rebuild xen and dom0 kernel after you updated the repository?
If not, I would recommend to do so.

Yoshi Tamura

> 
> On 9/5/06, Ewan Mellor <ewan@xensource.com> wrote:
>>
>> On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:
>>
>> > Folks,
>> > I am experiencing that live migration is not working in latest
>> > xen-unstable. I get the following message during migration
>> >
>> > [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
>> > Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
>> > [root@pc5 ksanjay]#
>> >
>> > I traced the problem to a function in xen named set_sh_allocation() in
>> > file xen/arch/x86/mm/shadow/common.c
>> >
>> > tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python
>> > script which makes the following hypercall
>> >
>> >     if (live) {
>> >         if (xc_shadow_control(xc_handle, dom,
>> >                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
>> >                               NULL, 0, NULL, 0, NULL) < 0) {
>> >             ERR("Couldn't enable shadow mode");
>> >             goto out;
>> >         }
>> >         last_iter = 0;
>> >     } else {
>> > -----------
>> >
>> > this particular hypercall leads to the call of set_sh_allocation which
>> > fails in the following code
>> >
>> >         if ( d-> arch.shadow.total_pages < pages )
>> >         {
>> >             /* Need to allocate more memory from domheap */
>> >             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
>> >             if ( pg == NULL )
>> >             {
>> >                 SHADOW_PRINTK("failed to allocate shadow pages.\n");
>> >                 return -ENOMEM;
>> >             }
>> >
>> > alloc_domheap_pages fails and returns NULL. however I think I have
>> enough
>> > memory available so this function should not fail.
>> >
>> > Is there anybody else experiencing the same problem? Could someone
>> please
>> > tell me how to fix it?
>>
>> I've put some changes into xen-unstable today which might help.  The last
>> fix
>> is on its way through testing now.  Look out for xen-unstable changeset
>> 11422, and try that, see how you get on.
>>
>> Cheers,
>>
>> Ewan.
>>
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


-- 
TAMURA, Yoshiaki

NTT Cyber Space Labs
OSS Computing Project
Kernel Group
E-mail: tamura.yoshiaki@lab.ntt.co.jp
TEL: (046)-859-2771
FAX: (046)-855-1152
Address: 1-1 Hikarinooka, Yokosuka
	 Kanagawa 239-0847 JAPAN

Re: Live vm migration broken in latest xen-unstable

[Tim Deegan]

[-- Attachment #1: Type: text/plain, Size: 2871 bytes --]

Hi Sanjay,

Does the attached patch fix this problem for you?  It tries to make sure
there is enough spare memory to enable shadow pagetables on the domain
before starting the migration.

Cheers,

Tim

At 15:09 -0400 on 05 Sep (1157468999), sanjay kushwaha wrote:
> Hi Ewan,
> I did a "hg pull -u" on my tree which also got the changeset 11422. but I am
> still facing the same problem. btw this changeset seems to be specific to
> hvm domain while I am facing this problem with paravirtualized domain.
> 
> Thanks,
> Sanjay
> 
> On 9/5/06, Ewan Mellor <ewan@xensource.com> wrote:
> >
> >On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:
> >
> >> Folks,
> >> I am experiencing that live migration is not working in latest
> >> xen-unstable. I get the following message during migration
> >>
> >> [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
> >> Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
> >> [root@pc5 ksanjay]#
> >>
> >> I traced the problem to a function in xen named set_sh_allocation() in
> >> file xen/arch/x86/mm/shadow/common.c
> >>
> >> tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python
> >> script which makes the following hypercall
> >>
> >>     if (live) {
> >>         if (xc_shadow_control(xc_handle, dom,
> >>                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
> >>                               NULL, 0, NULL, 0, NULL) < 0) {
> >>             ERR("Couldn't enable shadow mode");
> >>             goto out;
> >>         }
> >>         last_iter = 0;
> >>     } else {
> >> -----------
> >>
> >> this particular hypercall leads to the call of set_sh_allocation which
> >> fails in the following code
> >>
> >>         if ( d-> arch.shadow.total_pages < pages )
> >>         {
> >>             /* Need to allocate more memory from domheap */
> >>             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
> >>             if ( pg == NULL )
> >>             {
> >>                 SHADOW_PRINTK("failed to allocate shadow pages.\n");
> >>                 return -ENOMEM;
> >>             }
> >>
> >> alloc_domheap_pages fails and returns NULL. however I think I have
> >enough
> >> memory available so this function should not fail.
> >>
> >> Is there anybody else experiencing the same problem? Could someone
> >please
> >> tell me how to fix it?
> >
> >I've put some changes into xen-unstable today which might help.  The last
> >fix
> >is on its way through testing now.  Look out for xen-unstable changeset
> >11422, and try that, see how you get on.
> >
> >Cheers,
> >
> >Ewan.
> >
> 
> 
> 
> -- 
> ----------------------
> PhD Student, Georgia Tech
> http://www.cc.gatech.edu/~ksanjay/ <http://www.cc.gatech.edu/%7Eksanjay/>

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


[-- Attachment #2: lm.diff --]
[-- Type: text/plain, Size: 1947 bytes --]

diff -r 1de184deaa9c tools/python/xen/xend/XendDomain.py
--- a/tools/python/xen/xend/XendDomain.py	Wed Sep 06 13:16:02 2006 +0100
+++ b/tools/python/xen/xend/XendDomain.py	Thu Sep 07 18:54:45 2006 +0100
@@ -420,6 +420,10 @@ class XendDomain:
         """ The following call may raise a XendError exception """
         dominfo.testMigrateDevices(True, dst)
 
+        if live:
+            """ Make sure there's memory free for enabling shadow mode """
+            dominfo.checkLiveMigrateMemory()
+
         if port == 0:
             port = xroot.get_xend_relocation_port()
         try:
diff -r 1de184deaa9c tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py	Wed Sep 06 13:16:02 2006 +0100
+++ b/tools/python/xen/xend/XendDomainInfo.py	Thu Sep 07 22:05:10 2006 +0100
@@ -49,6 +49,7 @@ from xen.xend.xenstore.xsutil import Get
 from xen.xend.xenstore.xsutil import GetDomainPath, IntroduceDomain
 from xen.xend.xenstore.xswatch import xswatch
 
+from xen.xend import arch
 
 """Shutdown code for poweroff."""
 DOMAIN_POWEROFF = 0
@@ -1502,6 +1503,19 @@ class XendDomainInfo:
             self.image.createDeviceModel()
 
     ## public:
+
+    def checkLiveMigrateMemory(self):
+        """ Make sure there's enough memory to migrate this domain """
+        overhead_kb = 0
+        if arch.type == "x86":
+            # 1MB per vcpu plus 4Kib/Mib of RAM.  This is higher than 
+            # the minimum that Xen would allocate if no value were given.
+            overhead_kb = self.info['vcpus'] * 1024 + self.info['maxmem'] * 4
+            overhead_kb = ((overhead_kb + 1023) / 1024) * 1024
+            # The domain might already have some shadow memory
+            overhead_kb -= xc.shadow_mem_control(self.domid) * 1024
+        if overhead_kb > 0:
+            balloon.free(overhead_kb)
 
     def testMigrateDevices(self, network, dst):
         """ Notify all device about intention of migration

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Live vm migration broken in latest xen-unstable

[sanjay kushwaha]

Hi Tim,
Yes. This patch solved my problem. now live VM migration happens but I
am experiencing another problem. when I reattach to the migrated VM
console on the destination machine, I see the following traceback in
the dmesg of the guest VM.

------------[ cut here ]------------
kernel BUG at drivers/xen/netfront/netfront.c:717!
invalid opcode: 0000 [#1]
SMP
Modules linked in:
CPU:    0
EIP:    0061:[<c02598e0>]    Not tainted VLI
EFLAGS: 00010082   (2.6.16.13-xenU #15)
EIP is at network_alloc_rx_buffers+0x470/0x4b0
eax: c056fc80   ebx: ccbc0d80   ecx: d1001040   edx: c05d0000
esi: c05d02a0   edi: c05d034c   ebp: c0551f14   esp: c0551eac
ds: 007b   es: 007b   ss: 0069
Process xenwatch (pid: 8, threadinfo=c0550000 task=c057a540)
Stack: <0>00000208 00000000 0000cbc1 00000000 c05d034c c05d31b8
c05d0000 00000000
       00000337 00000337 0000002f 00000208 ccbc1000 000000d1 cdd25838 00000100
       000000d1 00000011 c05d0000 00000001 c02f8013 c05d0000 00000001 c05d02a0
Call Trace:
 [<c01058ed>] show_stack_log_lvl+0xcd/0x120
 [<c0105aeb>] show_registers+0x1ab/0x240
 [<c0105e11>] die+0x111/0x240
 [<c0106178>] do_trap+0x98/0xe0
 [<c0106491>] do_invalid_op+0xa1/0xb0
 [<c01052c7>] error_code+0x2b/0x30
 [<c025a554>] backend_changed+0x1a4/0x250
 [<c025517e>] otherend_changed+0x7e/0x90
 [<c0253361>] xenwatch_handle_callback+0x21/0x60
 [<c025345d>] xenwatch_thread+0xbd/0x160
 [<c0132b0c>] kthread+0xec/0xf0
 [<c0102c45>] kernel_thread_helper+0x5/0x10
Code: 82 04 15 00 00 8d 9e f8 14 00 00 e8 db 78 ea ff 8b 5d d8 39 9a
fc 14 00 00 0f 84 16 fe ff ff e9 51 ff ff ff 8d b4 26 00 00 00 00 <0f>
0b cd 02 2c a0 2f c0 e9 cd fc ff ff 0f 0b d1 02 2c a0 2f c0

[root@localhost ~]#


Does anyone know if this is a known problem?

Thanks for your help.
Sanjay

On 9/7/06, Tim Deegan <Tim.Deegan@xensource.com> wrote:
> Hi Sanjay,
>
> Does the attached patch fix this problem for you?  It tries to make sure
> there is enough spare memory to enable shadow pagetables on the domain
> before starting the migration.
>
> Cheers,
>
> Tim
>
> At 15:09 -0400 on 05 Sep (1157468999), sanjay kushwaha wrote:
> > Hi Ewan,
> > I did a "hg pull -u" on my tree which also got the changeset 11422. but I am
> > still facing the same problem. btw this changeset seems to be specific to
> > hvm domain while I am facing this problem with paravirtualized domain.
> >
> > Thanks,
> > Sanjay
> >
> > On 9/5/06, Ewan Mellor <ewan@xensource.com> wrote:
> > >
> > >On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:
> > >
> > >> Folks,
> > >> I am experiencing that live migration is not working in latest
> > >> xen-unstable. I get the following message during migration
> > >>
> > >> [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
> > >> Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
> > >> [root@pc5 ksanjay]#
> > >>
> > >> I traced the problem to a function in xen named set_sh_allocation() in
> > >> file xen/arch/x86/mm/shadow/common.c
> > >>
> > >> tools/libxc/xc_linux_save.c:xc_linux_save() is called from the python
> > >> script which makes the following hypercall
> > >>
> > >>     if (live) {
> > >>         if (xc_shadow_control(xc_handle, dom,
> > >>                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
> > >>                               NULL, 0, NULL, 0, NULL) < 0) {
> > >>             ERR("Couldn't enable shadow mode");
> > >>             goto out;
> > >>         }
> > >>         last_iter = 0;
> > >>     } else {
> > >> -----------
> > >>
> > >> this particular hypercall leads to the call of set_sh_allocation which
> > >> fails in the following code
> > >>
> > >>         if ( d-> arch.shadow.total_pages < pages )
> > >>         {
> > >>             /* Need to allocate more memory from domheap */
> > >>             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
> > >>             if ( pg == NULL )
> > >>             {
> > >>                 SHADOW_PRINTK("failed to allocate shadow pages.\n");
> > >>                 return -ENOMEM;
> > >>             }
> > >>
> > >> alloc_domheap_pages fails and returns NULL. however I think I have
> > >enough
> > >> memory available so this function should not fail.
> > >>
> > >> Is there anybody else experiencing the same problem? Could someone
> > >please
> > >> tell me how to fix it?
> > >
> > >I've put some changes into xen-unstable today which might help.  The last
> > >fix
> > >is on its way through testing now.  Look out for xen-unstable changeset
> > >11422, and try that, see how you get on.
> > >
> > >Cheers,
> > >
> > >Ewan.
> > >
> >
> >
> >
> > --
> > ----------------------
> > PhD Student, Georgia Tech
> > http://www.cc.gatech.edu/~ksanjay/ <http://www.cc.gatech.edu/%7Eksanjay/>
>
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>
>
>


-- 
----------------------
PhD Student, Georgia Tech
http://www.cc.gatech.edu/~ksanjay/

Re: Live vm migration broken in latest xen-unstable

[sanjay kushwaha]

[-- Attachment #1.1: Type: text/plain, Size: 5774 bytes --]

Hi Folks,
I am facing another problem. after the vm migration, the frontend of guest
VM doesn't properly attaches with the new backend. I observed that after VM
migration, the disk requests sent by blkfront driver are not seen by the
blkif_schedule process in the blkback driver. Do you think it is a known
problem? has anybody else observed this?

Thanks,
Sanjay

On 9/8/06, sanjay kushwaha <sanjay.kushwaha@gmail.com> wrote:
>
> Hi Tim,
> Yes. This patch solved my problem. now live VM migration happens but I
> am experiencing another problem. when I reattach to the migrated VM
> console on the destination machine, I see the following traceback in
> the dmesg of the guest VM.
>
> ------------[ cut here ]------------
> kernel BUG at drivers/xen/netfront/netfront.c:717!
> invalid opcode: 0000 [#1]
> SMP
> Modules linked in:
> CPU:    0
> EIP:    0061:[<c02598e0>]    Not tainted VLI
> EFLAGS: 00010082   (2.6.16.13-xenU #15)
> EIP is at network_alloc_rx_buffers+0x470/0x4b0
> eax: c056fc80   ebx: ccbc0d80   ecx: d1001040   edx: c05d0000
> esi: c05d02a0   edi: c05d034c   ebp: c0551f14   esp: c0551eac
> ds: 007b   es: 007b   ss: 0069
> Process xenwatch (pid: 8, threadinfo=c0550000 task=c057a540)
> Stack: <0>00000208 00000000 0000cbc1 00000000 c05d034c c05d31b8
> c05d0000 00000000
>        00000337 00000337 0000002f 00000208 ccbc1000 000000d1 cdd25838
> 00000100
>        000000d1 00000011 c05d0000 00000001 c02f8013 c05d0000 00000001
> c05d02a0
> Call Trace:
> [<c01058ed>] show_stack_log_lvl+0xcd/0x120
> [<c0105aeb>] show_registers+0x1ab/0x240
> [<c0105e11>] die+0x111/0x240
> [<c0106178>] do_trap+0x98/0xe0
> [<c0106491>] do_invalid_op+0xa1/0xb0
> [<c01052c7>] error_code+0x2b/0x30
> [<c025a554>] backend_changed+0x1a4/0x250
> [<c025517e>] otherend_changed+0x7e/0x90
> [<c0253361>] xenwatch_handle_callback+0x21/0x60
> [<c025345d>] xenwatch_thread+0xbd/0x160
> [<c0132b0c>] kthread+0xec/0xf0
> [<c0102c45>] kernel_thread_helper+0x5/0x10
> Code: 82 04 15 00 00 8d 9e f8 14 00 00 e8 db 78 ea ff 8b 5d d8 39 9a
> fc 14 00 00 0f 84 16 fe ff ff e9 51 ff ff ff 8d b4 26 00 00 00 00 <0f>
> 0b cd 02 2c a0 2f c0 e9 cd fc ff ff 0f 0b d1 02 2c a0 2f c0
>
> [root@localhost ~]#
>
>
> Does anyone know if this is a known problem?
>
> Thanks for your help.
> Sanjay
>
> On 9/7/06, Tim Deegan <Tim.Deegan@xensource.com> wrote:
> > Hi Sanjay,
> >
> > Does the attached patch fix this problem for you?  It tries to make sure
> > there is enough spare memory to enable shadow pagetables on the domain
> > before starting the migration.
> >
> > Cheers,
> >
> > Tim
> >
> > At 15:09 -0400 on 05 Sep (1157468999), sanjay kushwaha wrote:
> > > Hi Ewan,
> > > I did a "hg pull -u" on my tree which also got the changeset 11422.
> but I am
> > > still facing the same problem. btw this changeset seems to be specific
> to
> > > hvm domain while I am facing this problem with paravirtualized domain.
> > >
> > > Thanks,
> > > Sanjay
> > >
> > > On 9/5/06, Ewan Mellor <ewan@xensource.com> wrote:
> > > >
> > > >On Fri, Sep 01, 2006 at 05:36:00PM -0400, sanjay kushwaha wrote:
> > > >
> > > >> Folks,
> > > >> I am experiencing that live migration is not working in latest
> > > >> xen-unstable. I get the following message during migration
> > > >>
> > > >> [root@pc5 ksanjay]# xm migrate --live 1 [1]199.77.138.23
> > > >> Error: /usr/lib/xen/bin/xc_save 18 1 0 0 1 failed
> > > >> [root@pc5 ksanjay]#
> > > >>
> > > >> I traced the problem to a function in xen named set_sh_allocation()
> in
> > > >> file xen/arch/x86/mm/shadow/common.c
> > > >>
> > > >> tools/libxc/xc_linux_save.c:xc_linux_save() is called from the
> python
> > > >> script which makes the following hypercall
> > > >>
> > > >>     if (live) {
> > > >>         if (xc_shadow_control(xc_handle, dom,
> > > >>                               XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY,
> > > >>                               NULL, 0, NULL, 0, NULL) < 0) {
> > > >>             ERR("Couldn't enable shadow mode");
> > > >>             goto out;
> > > >>         }
> > > >>         last_iter = 0;
> > > >>     } else {
> > > >> -----------
> > > >>
> > > >> this particular hypercall leads to the call of set_sh_allocation
> which
> > > >> fails in the following code
> > > >>
> > > >>         if ( d-> arch.shadow.total_pages < pages )
> > > >>         {
> > > >>             /* Need to allocate more memory from domheap */
> > > >>             pg = alloc_domheap_pages(NULL, SHADOW_MAX_ORDER, 0);
> > > >>             if ( pg == NULL )
> > > >>             {
> > > >>                 SHADOW_PRINTK("failed to allocate shadow
> pages.\n");
> > > >>                 return -ENOMEM;
> > > >>             }
> > > >>
> > > >> alloc_domheap_pages fails and returns NULL. however I think I have
> > > >enough
> > > >> memory available so this function should not fail.
> > > >>
> > > >> Is there anybody else experiencing the same problem? Could someone
> > > >please
> > > >> tell me how to fix it?
> > > >
> > > >I've put some changes into xen-unstable today which might help.  The
> last
> > > >fix
> > > >is on its way through testing now.  Look out for xen-unstable
> changeset
> > > >11422, and try that, see how you get on.
> > > >
> > > >Cheers,
> > > >
> > > >Ewan.
> > > >
> > >
> > >
> > >
> > > --
> > > ----------------------
> > > PhD Student, Georgia Tech
> > > http://www.cc.gatech.edu/~ksanjay/ <
> http://www.cc.gatech.edu/%7Eksanjay/>
> >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> >
> >
> >
> >
>
>
> --
> ----------------------
> PhD Student, Georgia Tech
> http://www.cc.gatech.edu/~ksanjay/
>



-- 
----------------------
PhD Student, Georgia Tech
http://www.cc.gatech.edu/~ksanjay/

[-- Attachment #1.2: Type: text/html, Size: 8980 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel