netif_suspend / resume

netif_suspend / resume

[Jacob Gorm Hansen]

hi,

does netif_suspend() have any side effects with regards to the state of 
memory pages that are currently awaiting I/O?

It seems to basically unbind from the event channels for the vifs, does 
that guarantee that Xen or the driver domain will not decide to flip any 
of the client domain's pages after this point? How does the client 
domain reclaim outstanding sbks during/after a suspend?

Thanks,
Jacob

RE: netif_suspend / resume

[Ian Pratt]

 

> does netif_suspend() have any side effects with regards to 
> the state of memory pages that are currently awaiting I/O?
> 
> It seems to basically unbind from the event channels for the 
> vifs, does that guarantee that Xen or the driver domain will 
> not decide to flip any of the client domain's pages after 
> this point? How does the client domain reclaim outstanding 
> sbks during/after a suspend?

The guest keeps its own shadow copy of the state as regards which pfn's
it has queued as {free, rx, tx}bufs. After a resume it 'gathers' those
pages and requeues the free and tx bufs to the new backend. 

We currently junk any outstanding rxbufs, but in reality there almost
never are any as the vif interrupt will be serviced before the suspend
thread gets to run. When we switch over to grant tables we'll be able to
process these on the destination too, as we'll know the memory has been
syncronized. It won't make difference, though.

Ian 

Re: netif_suspend / resume

[Jacob Gorm Hansen]

Ian Pratt wrote:

> The guest keeps its own shadow copy of the state as regards which pfn's
> it has queued as {free, rx, tx}bufs. After a resume it 'gathers' those
> pages and requeues the free and tx bufs to the new backend. 
> 
> We currently junk any outstanding rxbufs, but in reality there almost
> never are any as the vif interrupt will be serviced before the suspend
> thread gets to run. When we switch over to grant tables we'll be able to
> process these on the destination too, as we'll know the memory has been
> syncronized. It won't make difference, though.

With the latest unstable, I am trying to do this:

     __cli();

     netif_suspend();
     time_suspend();
     ctrl_if_suspend();
     irq_suspend();


     irq_resume();
     ctrl_if_resume();
     time_resume();
     netif_resume();


     __sti();

(The domU has no blockdev support compiled in).

But I immediately crash trying to reference 0x050d4286 in 
network_tx_buf_gc(), at the last instruction in the disasm below:

c0222e20 <network_tx_buf_gc>:
c0222e20:   55                      push   %ebp
c0222e21:   57                      push   %edi
c0222e22:   56                      push   %esi
c0222e23:   53                      push   %ebx
c0222e24:   83 ec 04                sub    $0x4,%esp
c0222e27:   8b 44 24 18             mov    0x18(%esp),%eax
c0222e2b:   89 04 24                mov    %eax,(%esp)
c0222e2e:   89 c6                   mov    %eax,%esi
c0222e30:   81 c6 20 02 00 00       add    $0x220,%esi
c0222e36:   83 be 88 00 00 00 02    cmpl   $0x2,0x88(%esi)
c0222e3d:   74 06                   je     c0222e45 <network_tx_buf_gc+0x25>
c0222e3f:   58                      pop    %eax
c0222e40:   5b                      pop    %ebx
c0222e41:   5e                      pop    %esi
c0222e42:   5f                      pop    %edi
c0222e43:   5d                      pop    %ebp
c0222e44:   c3                      ret
c0222e45:   8b 56 74                mov    0x74(%esi),%edx
c0222e48:   90                      nop
c0222e49:   8d b4 26 00 00 00 00    lea    0x0(%esi),%esi
c0222e50:   8b 7a 08                mov    0x8(%edx),%edi
c0222e53:   f0 83 44 24 00 00       lock addl $0x0,0x0(%esp)
c0222e59:   8b 5e 6c                mov    0x6c(%esi),%ebx
c0222e5c:   39 fb                   cmp    %edi,%ebx
c0222e5e:   74 42                   je     c0222ea2 <network_tx_buf_gc+0x82>
c0222e60:   0f b6 c3                movzbl %bl,%eax
c0222e63:   8b 6e 74                mov    0x74(%esi),%ebp
c0222e66:   8d 04 40                lea    (%eax,%eax,2),%eax
c0222e69:   c1 e0 02                shl    $0x2,%eax
c0222e6c:   8b 96 a8 00 00 00       mov    0xa8(%esi),%edx
c0222e72:   01 e8                   add    %ebp,%eax
c0222e74:   0f b7 40 10             movzwl 0x10(%eax),%eax
c0222e78:   8b 8c 86 a8 00 00 00    mov    0xa8(%esi,%eax,4),%ecx
c0222e7f:   89 94 86 a8 00 00 00    mov    %edx,0xa8(%esi,%eax,4)
c0222e86:   89 86 a8 00 00 00       mov    %eax,0xa8(%esi)
c0222e8c:   ff 89 84 00 00 00       decl   0x84(%ecx)
                                            ^^^^^^^^^^
Any hints for debugging this?

thanks,
Jacob

Re: netif_suspend / resume

[Kip Macy]

If you're not averse to GDB you can use the gdbserver to look at the
state. If it is calling domain_crash you can enable coredump.
Evidently some interface has changed that has broken gdbserver for the
moment, but once I finish with -testing this weekend I'll probably get
to submitting a patch.

    -Kip

On 5/20/05, Jacob Gorm Hansen <jacobg@diku.dk> wrote:
> Ian Pratt wrote:
> 
> > The guest keeps its own shadow copy of the state as regards which pfn's
> > it has queued as {free, rx, tx}bufs. After a resume it 'gathers' those
> > pages and requeues the free and tx bufs to the new backend.
> >
> > We currently junk any outstanding rxbufs, but in reality there almost
> > never are any as the vif interrupt will be serviced before the suspend
> > thread gets to run. When we switch over to grant tables we'll be able to
> > process these on the destination too, as we'll know the memory has been
> > syncronized. It won't make difference, though.
> 
> With the latest unstable, I am trying to do this:
> 
>      __cli();
> 
>      netif_suspend();
>      time_suspend();
>      ctrl_if_suspend();
>      irq_suspend();
> 
> 
>      irq_resume();
>      ctrl_if_resume();
>      time_resume();
>      netif_resume();
> 
> 
>      __sti();
> 
> (The domU has no blockdev support compiled in).
> 
> But I immediately crash trying to reference 0x050d4286 in
> network_tx_buf_gc(), at the last instruction in the disasm below:
> 
> c0222e20 <network_tx_buf_gc>:
> c0222e20:   55                      push   %ebp
> c0222e21:   57                      push   %edi
> c0222e22:   56                      push   %esi
> c0222e23:   53                      push   %ebx
> c0222e24:   83 ec 04                sub    $0x4,%esp
> c0222e27:   8b 44 24 18             mov    0x18(%esp),%eax
> c0222e2b:   89 04 24                mov    %eax,(%esp)
> c0222e2e:   89 c6                   mov    %eax,%esi
> c0222e30:   81 c6 20 02 00 00       add    $0x220,%esi
> c0222e36:   83 be 88 00 00 00 02    cmpl   $0x2,0x88(%esi)
> c0222e3d:   74 06                   je     c0222e45 <network_tx_buf_gc+0x25>
> c0222e3f:   58                      pop    %eax
> c0222e40:   5b                      pop    %ebx
> c0222e41:   5e                      pop    %esi
> c0222e42:   5f                      pop    %edi
> c0222e43:   5d                      pop    %ebp
> c0222e44:   c3                      ret
> c0222e45:   8b 56 74                mov    0x74(%esi),%edx
> c0222e48:   90                      nop
> c0222e49:   8d b4 26 00 00 00 00    lea    0x0(%esi),%esi
> c0222e50:   8b 7a 08                mov    0x8(%edx),%edi
> c0222e53:   f0 83 44 24 00 00       lock addl $0x0,0x0(%esp)
> c0222e59:   8b 5e 6c                mov    0x6c(%esi),%ebx
> c0222e5c:   39 fb                   cmp    %edi,%ebx
> c0222e5e:   74 42                   je     c0222ea2 <network_tx_buf_gc+0x82>
> c0222e60:   0f b6 c3                movzbl %bl,%eax
> c0222e63:   8b 6e 74                mov    0x74(%esi),%ebp
> c0222e66:   8d 04 40                lea    (%eax,%eax,2),%eax
> c0222e69:   c1 e0 02                shl    $0x2,%eax
> c0222e6c:   8b 96 a8 00 00 00       mov    0xa8(%esi),%edx
> c0222e72:   01 e8                   add    %ebp,%eax
> c0222e74:   0f b7 40 10             movzwl 0x10(%eax),%eax
> c0222e78:   8b 8c 86 a8 00 00 00    mov    0xa8(%esi,%eax,4),%ecx
> c0222e7f:   89 94 86 a8 00 00 00    mov    %edx,0xa8(%esi,%eax,4)
> c0222e86:   89 86 a8 00 00 00       mov    %eax,0xa8(%esi)
> c0222e8c:   ff 89 84 00 00 00       decl   0x84(%ecx)
>                                             ^^^^^^^^^^
> Any hints for debugging this?
> 
> thanks,
> Jacob
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: netif_suspend / resume

[Jacob Gorm Hansen]

Kip Macy wrote:
> If you're not averse to GDB you can use the gdbserver to look at the
> state. If it is calling domain_crash you can enable coredump.
> Evidently some interface has changed that has broken gdbserver for the
> moment, but once I finish with -testing this weekend I'll probably get
> to submitting a patch.

Thanks, I would like to play with gdb and xen at some point. Actually, I 
should probably rephrase my question:

xenlinux in unstable crashes after executing the following code, and I 
don't think it should. Am I wrong?

     __cli();

     netif_suspend();
     time_suspend();
     ctrl_if_suspend();
     irq_suspend();


     irq_resume();
     ctrl_if_resume();
     time_resume();
     netif_resume();


     __sti();

/Jacob

Re: netif_suspend / resume

[Keir Fraser]

On 23 May 2005, at 23:27, Jacob Gorm Hansen wrote:

> Thanks, I would like to play with gdb and xen at some point. Actually, 
> I should probably rephrase my question:
>
> xenlinux in unstable crashes after executing the following code, and I 
> don't think it should. Am I wrong?

Some of the resume code assumes it is running in a new domain context 
so that, for example, no event channels are bound to anything. Some of 
these assumptions are probably biting you.

  -- Keir