Running netback/blkback in non-dom0 domains

Running netback/blkback in non-dom0 domains

[Anton Burtsev]

Hi, 

Is there a way to configure/hack Xen to run backend devices in 
a non-dom0 domain (in domU). Like, for example connect two domU 
domains with a block device.

Thanks, 
Anton

Re: Running netback/blkback in non-dom0 domains

[Samuel Thibault]

Anton Burtsev, le Thu 17 Jul 2008 16:02:32 -0600, a écrit :
> Is there a way to configure/hack Xen to run backend devices in 
> a non-dom0 domain (in domU). Like, for example connect two domU 
> domains with a block device.

Or, possibly more useful, with a network device :)

Samuel

Re: Running netback/blkback in non-dom0 domains

[Mark Williamson]

On Thursday 17 July 2008, Anton Burtsev wrote:
> Hi,
>
> Is there a way to configure/hack Xen to run backend devices in
> a non-dom0 domain (in domU). Like, for example connect two domU
> domains with a block device.

Yes, I think this should Just Work with the right configuration parameters...

Set:
blkif = 'yes'
or
netif = 'yes'

As appropriate in your domain config file for the "backend" domU.  Then in the 
frontend domU's config file you can specify something like:

disks = [ 'phy:hda1,hda1,w,BACK' ]

substituting the domid of the backend domain for BACK (its name might work, 
possibly, I'm not sure).  Specifying file: or tap: disks in the backend domU 
to work in this way might be a bit of a fiddle (if it's possible) but phy: 
should work just fine.

or you can do:

vif 
= "type=TYPE,mac=MAC,bridge=BRIDGE,ip=IPADDR,script=SCRIPT,backend=DOM,vifname=NAME"

again, specifying the domid (or possibly the name) of the backend domain.

If you start the backend and then the frontend from these config files they 
should be able to connect do virtual device IO.

Cheers,
Mark


-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

Re: Running netback/blkback in non-dom0 domains

[Anton Burtsev]

Thanks Mark,

Yes, this generally works (I've tried block devices on xen-3.2-testing).  
Some additional details: 

  1) If you build two different kernels dom0 and domU don't forget to 
     enable XEN_BACKEND, XEN_BLKDEV_BACKEND, XEN_NETDEV_BACKEND in your 
     domU config. 

  2) To connect backend you need a domU with a Linux system with working  
     hotplug scripts (I guess BSD and other systems use something else).  
     Hotplug scripts in turn depend on working xenstore-write, 
     xenstore-read, xenstore-..., etc. tools. 

Some bugs/unclear features:

  1) If you ask to import the same device, but from different domains, 
     like /dev/loop0 below:
     
         disk = ['phy:/dev/loop0,sda1,w', 'phy:/dev/loop0,sda2,w,block-backend-domain']   
     
     Domain is not created with an error:     
       Error: The file "/dev/loop0" is already used

  2) If you export a block device from dom0 through a domU to the second 
     domU, e.g.
     
       /dev/loop0  ---->  /dev/sda2  ---->  /dev/sda2
	 dom0               domU(1)           domU(2)
     
     device is not accessible in domU(2) with an error 
        Buffer I/O error on device sda2, logical block 25840 
     Xen complains: 
        (XEN) grant_table.c:360:d0 Could not pin grant frame 99551

Anton
  
On Fri, Jul 18, 2008 at 12:30:48AM +0100, Mark Williamson wrote:
> On Thursday 17 July 2008, Anton Burtsev wrote:
> > Hi,
> >
> > Is there a way to configure/hack Xen to run backend devices in
> > a non-dom0 domain (in domU). Like, for example connect two domU
> > domains with a block device.
> 
> Yes, I think this should Just Work with the right configuration parameters...
> 
> Set:
> blkif = 'yes'
> or
> netif = 'yes'
> 
> As appropriate in your domain config file for the "backend" domU.  Then in the 
> frontend domU's config file you can specify something like:
> 
> disks = [ 'phy:hda1,hda1,w,BACK' ]
> 
> substituting the domid of the backend domain for BACK (its name might work, 
> possibly, I'm not sure).  Specifying file: or tap: disks in the backend domU 
> to work in this way might be a bit of a fiddle (if it's possible) but phy: 
> should work just fine.
> 
> or you can do:
> 
> vif 
> = "type=TYPE,mac=MAC,bridge=BRIDGE,ip=IPADDR,script=SCRIPT,backend=DOM,vifname=NAME"
> 
> again, specifying the domid (or possibly the name) of the backend domain.
> 
> If you start the backend and then the frontend from these config files they 
> should be able to connect do virtual device IO.
> 
> Cheers,
> Mark
> 
> 
> -- 
> Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

Re: Running netback/blkback in non-dom0 domains

[Samuel Thibault]

Anton Burtsev, le Mon 21 Jul 2008 12:11:37 -0600, a écrit :
>   2) If you export a block device from dom0 through a domU to the second 
>      domU, e.g.
>      
>        /dev/loop0  ---->  /dev/sda2  ---->  /dev/sda2
> 	 dom0               domU(1)           domU(2)
>      
>      device is not accessible in domU(2) with an error 
>         Buffer I/O error on device sda2, logical block 25840 
>      Xen complains: 
>         (XEN) grant_table.c:360:d0 Could not pin grant frame 99551

I guess it is because domU(1) tries to directly expose domU(2)'s data
pages to dom0, which it is not allowed to do.

Samuel