Re: reboot driver domain, vifX.Y = NO-CARRIER?

[Jason Cooper <xen@lakedaemon.net>]

Hi Marek,

On Sat, May 05, 2018 at 01:03:15AM +0200, Marek Marczykowski-Górecki wrote:
> On Fri, May 04, 2018 at 06:13:25PM -0400, Rich Persaud wrote:
> > > On May 1, 2018, at 08:53, Jason Cooper <xen@lakedaemon.net> wrote:
> > > 
> > > add the link to xen-users thread of me talking to myself.  :-))
> > > 
> > >> On Tue, May 01, 2018 at 12:37:51PM +0000, Jason Cooper wrote:
> > >> When I was first digging into this, I started a thread on xen-users [1],
> > >> I've attached my xl-reboot.sh script here so you can see exactly what
> > >> I'm attempting to do:
> > > 
> > > [1] https://marc.info/?l=xen-users&m=152389443206023&w=2
> > 
> > You may want to look at the code (toolstack and/or frontend-backend
> > drivers) for Qubes and OpenXT, both of which use network driver
> > domains and support wired/wireless networks.  
> > 
> > Operational restart of a measured, non-persistent driver domain
> > (instead of host) is a benefit of Xen disaggregation architectures.
> 
> In Qubes, on backend restart, we do equivalent of xl network-detach &&
> xl network-attach (as you do in xl-reboot.sh). xl itself doesn't provide
> any place to plug such script, but we use libvirt which provide events.
> Also, we have full control over domain config (libvirt XML), so don't
> need to extract vif list from xenstore...
> 
> The problem you describe looks related to
> https://lkml.org/lkml/2018/2/28/289, but fix is included in 4.16...
> There was also related libxl patch:
> https://xen.markmail.org/thread/6qbgmwyjqsshjus7
> (but it applies to the case where you first shutdown backend and only
> then do xl network-detach)
> 
> Do you have xl devd running in your driver domain? Without that xl
> network-attach wont work (AFAIR udev isn't used here anymore).

Yes, I've now modified the init script (xendomains in Gentoo) to create
a key /tool/vmstatus/$domname/status, start the domU, loop until it gets
it's domid, and -chmod the key.  It then does a -watch on that key.  In
the domU, *after* xl devd is started, it writes "online" to that key.

This allows me to automatically bring up the driver domains, and make
sure they're ready for connections before proceeding to booting the next
VM.  This only occurs when the host boots.

After the driver domains are up, the rest of the domains are started in
parallel.

> Also note that backend shutdown/restart/crash was a source of many
> problems in frontend kernel and toolstack in the past. Even simple
> dynamic network-attach/detach sometimes is problematic for the frontend.
> Links:
> https://github.com/QubesOS/qubes-issues/issues/3657 (frontend kernel
> problem)
> https://github.com/QubesOS/qubes-issues/issues/1426 (toolstack problem,
> + libvirt)
> https://github.com/QubesOS/qubes-issues/issues/975 (frontend kernel
> problem)

Mmm, clearly the state machine and it's implementation needs some
review.  I'm building v4.16.7 and we'll see how it goes for my usecase.

Thanks for all the pointers!

thx,

Jason.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel