Re: qemu and xl semantics

[Ian Campbell <Ian.Campbell@eu.citrix.com>]

On Wed, 2010-12-22 at 16:47 +0000, Christoph Egger wrote: 
> On Wednesday 22 December 2010 17:08:38 Ian Campbell wrote:
> > On Wed, 2010-12-22 at 15:42 +0000, Christoph Egger wrote:
> > > On Monday 20 December 2010 11:23:59 Ian Campbell wrote:
> > > > On Fri, 2010-12-17 at 17:21 +0000, Christoph Egger wrote:
> > > > > On Friday 17 December 2010 11:32:41 Ian Campbell wrote:
> > > > > > On Fri, 2010-12-17 at 09:49 +0000, Christoph Egger wrote:
> > > > > > > On Friday 17 December 2010 10:15:14 Ian Campbell wrote:
> > > > > > > > On Fri, 2010-12-17 at 09:00 +0000, Christoph Egger wrote:
> > > >
> > > > The only recent qemu change which I am aware of in this area is the
> > > > backport of the blkback functionality from upstream Qemu. However this
> > > > should only be enabled in cases where blkback or blktap are not
> > > > available and furthermore is tied to libxl/xl so shouldn't have done
> > > > anything on xend (although shouldn't != doesn't).
> > >
> > > Is this supposed to interact with the Dom0 PV disk backend driver ?
> >
> > The qemu disk backend is intended to be used when there is no dom0 PV
> > disk backend driver in the kernel, if there is a driver in the kernel
> > then it is intended that the kernel driver should be used.
> >
> > The background to this is that we have just gotten the basic dom0
> > support into the upstream Linux kernel and are about to start on
> > upstreaming the backend drivers. However we observed that there may not
> > be any need to upstream a block backend since a userspace implementation
> > may well suffice. It's not a done decision but the early signs are good,
> > especially compared with blktap which hits userspace anyway.
> >
> > Even if we do eventually decide to upstream the Linux blkback using the
> > disk backend in qemu tides us over for the time being.
> 
> NetBSD has a kernel backend driver since Xen 1.x days ... upstream.
> The last major change happened to it when it got updated to fit for Xen 2
> and still fits well for Xen 3/4, too.

That's cool, I didn't realise the block protocol had been so consistent
over the releases.

> > > > > xbdback backend/vbd/1/832: can't VOP_OPEN device 0xe13: 16
> > > > > 16 means EBUSY.
> > > >
> > > > But it works other than this message?
> > >
> > > Well, that means that there are now two processes which want to open the
> > > vbd simultaneously. The first one wins. And on guest shutdown the
> > > VOP_CLOSE fails.
> >
> > Right. This suggests that the qemu disk backend is being erroneously
> > activated on NetBSD when what you actually want is to use the xbdback
> > process. I think you likely need to patch libxl to do the right thing
> > for NetBSD, currently the decision is based on the presence or absence
> > of blktap, I guess it needs extending to detect NetBSD's backend too.
> 
> xbdback is actually the kernel backend driver. The conflicting processes
> are qemu-dm and the script that assigns the disk through the loopback device.
> 
> It seems to work in either way depending on which process wins...

Hrm, you shouldn't be getting both in the first place...

> 
> > Alternatively perhaps NetBSD also wants to transition to using the qemu
> > based block backend, I suppose there is benefit to be had from sharing
> > this code between Linux and NetBSD dom0.
> 
> Yes, I think so as long as it doesn't start requiring a kernel driver
> other than xbdback.

If you choose this route then there should be no kernel driver at all,
that's the point. 

> > > I did some more debugging and figured out xenbackendd runs the vif-bridge
> > > script so network with a PV driver works.
> >
> > > The scripts not running is the one associated with 'vbd' and qemu-ifup.
> > > AFAIK  qemu-dm always run qemu-ifup which is no longer the case.
> > >
> > > qemu-ifup adds tap interface to the bridge.
> >
> > This changed (on Linux) quite a while ago so I didn't think of it.
> > qemu-ifup is not used there any more and instead the vif-bridge script
> > is called for both PV and TAP devices. See xen-unstable.hg
> > 21944:0232bc7c9544, I guess either some equivalent change is needed to
> > tools/hotplug/NetBSD or the libxl bit needs to be conditional on NetBSD.
> >
> 
> hmm... so when I start the guest with 'xm' then qemu runs qemu-ifup.
> when I start the guest with 'xl' then qemu does not.
> 
> So when I adjust the script then I fix one and break the other.
> So to have it working on both the best way is to have the libxl bit
> conditional on NetBSD.

Or add the "script=no" stuff to the qemu command line in xend too so
that it behaves like libxl. I think this would be preferred (assuming it
works).

Ian.