suspend operation in XEND

suspend operation in XEND

[Nauzad Sadry]

Hello folks

During Save & Migration, XFRD sends "xfr.suspend" message to XEND.
This results in XEND suspending the domainU. A part of the suspension
should involve XEND sending DISCONNECT message to the front-end driver
in domainU

I am trying to dig through the XEND python code & I am unable to find
this information. Can someone help ??

Also can someone guide me through the set of functions that are called
by XEND as a part of suspend. What is the use of the the
domain_restart_schedule() ??

Thanks

Nauzad


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: suspend operation in XEND

[Mark Williamson]

> During Save & Migration, XFRD sends "xfr.suspend" message to XEND.
> This results in XEND suspending the domainU. A part of the suspension
> should involve XEND sending DISCONNECT message to the front-end driver
> in domainU

I don't think you'll find it.  Migrations / suspensions are treated like a 
backend driver restart - at resume the frontend driver is notified that it 
needs to reconnect to the backend and is responsible for requeuing any 
pending requests.

Instead of explicitly disconnecting the domain at suspend, the frontend driver 
just sorts everything out at resume.

Cheers,
Mark

> I am trying to dig through the XEND python code & I am unable to find
> this information. Can someone help ??
>
> Also can someone guide me through the set of functions that are called
> by XEND as a part of suspend. What is the use of the the
> domain_restart_schedule() ??
>
> Thanks
>
> Nauzad
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: suspend operation in XEND

[Nauzad Sadry]

So when does the event channel between the backend & front end drivers
break during suspend operation in XEND ??

Also when is the NETIF_INTERFACE_STATUS_DISCONNECTED status sent to
the front-end driver? I see code in the front-end driver that
processes this status type which results in the front-driver going to
a suspend state (like unregistering IRQs etc)

Nauzad


On Thu, 20 Jan 2005 22:54:25 +0000, Mark Williamson <maw48@cl.cam.ac.uk> wrote:
> > During Save & Migration, XFRD sends "xfr.suspend" message to XEND.
> > This results in XEND suspending the domainU. A part of the suspension
> > should involve XEND sending DISCONNECT message to the front-end driver
> > in domainU
> 
> I don't think you'll find it.  Migrations / suspensions are treated like a
> backend driver restart - at resume the frontend driver is notified that it
> needs to reconnect to the backend and is responsible for requeuing any
> pending requests.
> 
> Instead of explicitly disconnecting the domain at suspend, the frontend driver
> just sorts everything out at resume.
> 
> Cheers,
> Mark
> 
> > I am trying to dig through the XEND python code & I am unable to find
> > this information. Can someone help ??
> >
> > Also can someone guide me through the set of functions that are called
> > by XEND as a part of suspend. What is the use of the the
> > domain_restart_schedule() ??
> >
> > Thanks
> >
> > Nauzad
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> > Tool for open source databases. Create drag-&-drop reports. Save time
> > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> > Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/xen-devel
>


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: suspend operation in XEND

[Mark Williamson]

> So when does the event channel between the backend & front end drivers
> break during suspend operation in XEND ??

The event channel itself will get destroyed when the domain is destroyed 
(after the domain's contents are saved).  From the guests PoV, no teardown of 
event channels or shared memory occurs before suspend - it all gets sorted 
out on resume.

> Also when is the NETIF_INTERFACE_STATUS_DISCONNECTED status sent to
> the front-end driver? I see code in the front-end driver that
> processes this status type which results in the front-driver going to
> a suspend state (like unregistering IRQs etc)

Interfaces go through various states.  During normal operation, these look 
like the following:
* DISCONNECTED -> CLOSED -> CONNECTED when an interface is attached to the 
domain
* CONNECTED -> CLOSED -> DISCONNECTED when an interface is removed.  In this 
case, the interface is in the CLOSED state when the DISCONNECTED message 
arrives, so this is the final disconnect for the interface

If the interface is in the CONNECTED state and a DISCONNECTED message is 
received, it means that the connection with the backend has been broken for 
some reason (e.g. the domain migrated to a new host, the backend driver 
domain was rebooted, etc).  When this happens, Xend will send this message to 
the domain.

If the guest sees this then it must go through the DISCONNECTED -> CLOSED -> 
CONNECTED transition again, talking on the control interface to set up shared 
memory and bind event channels with the new backend.  At this point, it also 
frees resources associated with the old device channel that are no longer 
needed (e.g. old IRQs).

HTH,
Mark

> On Thu, 20 Jan 2005 22:54:25 +0000, Mark Williamson <maw48@cl.cam.ac.uk> 
wrote:
> > > During Save & Migration, XFRD sends "xfr.suspend" message to XEND.
> > > This results in XEND suspending the domainU. A part of the suspension
> > > should involve XEND sending DISCONNECT message to the front-end driver
> > > in domainU
> >
> > I don't think you'll find it.  Migrations / suspensions are treated like
> > a backend driver restart - at resume the frontend driver is notified that
> > it needs to reconnect to the backend and is responsible for requeuing any
> > pending requests.
> >
> > Instead of explicitly disconnecting the domain at suspend, the frontend
> > driver just sorts everything out at resume.
> >
> > Cheers,
> > Mark
> >
> > > I am trying to dig through the XEND python code & I am unable to find
> > > this information. Can someone help ??
> > >
> > > Also can someone guide me through the set of functions that are called
> > > by XEND as a part of suspend. What is the use of the the
> > > domain_restart_schedule() ??
> > >
> > > Thanks
> > >
> > > Nauzad
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> > > Tool for open source databases. Create drag-&-drop reports. Save time
> > > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> > > Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: suspend operation in XEND

[Nauzad Sadry]

when the domainU is being suspended I see "shutdown_suspend_t" message
sent to the domain thru the event channel by XEND.

what does the domainU do on receiving this message ??

is the DISCONNECTED message sent by XEND as a part of suspend_domain
or as a part of destroy_domain??

Thanks,

Nauzad

On Fri, 21 Jan 2005 00:28:46 +0000, Mark Williamson <maw48@cl.cam.ac.uk> wrote:
> > So when does the event channel between the backend & front end drivers
> > break during suspend operation in XEND ??
> 
> The event channel itself will get destroyed when the domain is destroyed
> (after the domain's contents are saved).  From the guests PoV, no teardown of
> event channels or shared memory occurs before suspend - it all gets sorted
> out on resume.
> 
> > Also when is the NETIF_INTERFACE_STATUS_DISCONNECTED status sent to
> > the front-end driver? I see code in the front-end driver that
> > processes this status type which results in the front-driver going to
> > a suspend state (like unregistering IRQs etc)
> 
> Interfaces go through various states.  During normal operation, these look
> like the following:
> * DISCONNECTED -> CLOSED -> CONNECTED when an interface is attached to the
> domain
> * CONNECTED -> CLOSED -> DISCONNECTED when an interface is removed.  In this
> case, the interface is in the CLOSED state when the DISCONNECTED message
> arrives, so this is the final disconnect for the interface
> 
> If the interface is in the CONNECTED state and a DISCONNECTED message is
> received, it means that the connection with the backend has been broken for
> some reason (e.g. the domain migrated to a new host, the backend driver
> domain was rebooted, etc).  When this happens, Xend will send this message to
> the domain.
> 
> If the guest sees this then it must go through the DISCONNECTED -> CLOSED ->
> CONNECTED transition again, talking on the control interface to set up shared
> memory and bind event channels with the new backend.  At this point, it also
> frees resources associated with the old device channel that are no longer
> needed (e.g. old IRQs).
> 
> HTH,
> Mark
> 
> > On Thu, 20 Jan 2005 22:54:25 +0000, Mark Williamson <maw48@cl.cam.ac.uk>
> wrote:
> > > > During Save & Migration, XFRD sends "xfr.suspend" message to XEND.
> > > > This results in XEND suspending the domainU. A part of the suspension
> > > > should involve XEND sending DISCONNECT message to the front-end driver
> > > > in domainU
> > >
> > > I don't think you'll find it.  Migrations / suspensions are treated like
> > > a backend driver restart - at resume the frontend driver is notified that
> > > it needs to reconnect to the backend and is responsible for requeuing any
> > > pending requests.
> > >
> > > Instead of explicitly disconnecting the domain at suspend, the frontend
> > > driver just sorts everything out at resume.
> > >
> > > Cheers,
> > > Mark
> > >
> > > > I am trying to dig through the XEND python code & I am unable to find
> > > > this information. Can someone help ??
> > > >
> > > > Also can someone guide me through the set of functions that are called
> > > > by XEND as a part of suspend. What is the use of the the
> > > > domain_restart_schedule() ??
> > > >
> > > > Thanks
> > > >
> > > > Nauzad
> > > >
> > > >
> > > > -------------------------------------------------------
> > > > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> > > > Tool for open source databases. Create drag-&-drop reports. Save time
> > > > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> > > > Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/xen-devel
>


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: suspend operation in XEND

[Mark Williamson]

> when the domainU is being suspended I see "shutdown_suspend_t" message
> sent to the domain thru the event channel by XEND.
>
> what does the domainU do on receiving this message ??

There's various other stuff a domain does do when suspending.  See 
linux-2.6.10-xen-sparse/arch/xen/kernel/reboot.c.  In particular, the 
function __do_suspend() calls various subsystems, creates a "suspend record" 
and then hypercalls into Xen with HYPERVISOR_suspend(), telling it where the 
suspend record actually is.

When the domain sees the hypercall return, it has been resumed, so this 
function runs through and resumes various kernel subsystems.

> is the DISCONNECTED message sent by XEND as a part of suspend_domain
> or as a part of destroy_domain??

OK, looking at the code, this doesn't work quite how I thought it did - sorry.

* A DISCONNECTED message on a CONNECTED interface means "your backend has 
disappeared because of failure  - please reconnect to the new backend".  So 
Xend sends it on resume to notify the domain that it needs to reconnect when 
a driver domain is restarted.  This only occurs on driver domain reboots.
* A DISCONNECTED message on a CLOSED interface means "this interface (which 
was in the process is tear down) has now gone".  This would only occur when 
an interface is torn down and I'm not sure that actually happens in practice 
(?)

I originally thought suspend / resume was now treated exactly like a backend 
failure but apparently that's not quite right:

__do_suspend() (as mentioned above) notifies the drivers when a suspend will 
occur.  They can use this to tear down some stuff (the block driver doesn't 
appear to need to do anything, it seems).  When the domain is restored, the 
second half of __do_suspend() notifies the drivers that a resume has occurred 
and they send messages to the domain controller requesting a reconnect.

After that, they just requeue pending requests, as they would also do in the 
case of a backend restart.

Any DISCONNECTED messages that occur will be during the resume as part of the 
reconnection protocol but it's too late at night to figure out exactly what's 
involved.  They won't happen during the suspend AFAIK.

HTH,
Mark


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl