Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Brian Buhrow]

}On 28/11/12 20:40, Toby Karyadi wrote:
}> On 11/28/12 10:29 AM, Thor Lancelot Simon wrote:
}>> On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
}>>> On 28/11/12 14:26, Thor Lancelot Simon wrote:
}>>>> I am assuming there's some reason we want this, rather than handling
}>>>> such requests in-kernel.  But what is it?
}>>> I should have marked this as "experimental", or something like this. I'm
}>>> not sure if switching to Qdisk for all image file backends is needed.
}>>> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
}>>>
}>>> "loop device had problems with flushing dirty pages (specifically, doing
}>>> a large number of writes to an NFS-backed image don't result in the OOM
}>>> killer going berserk)."
}>>>
}>>> I'm not sure if NetBSD is in the same situation, but if I remember
}>>> correctly (haven't tried that in a long time), trying to use a disk file
}>>> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
}>>> issue is: http://gnats.netbsd.org/40726.
}>> It seems highly unlikely to me that a problem with the loop device driver
}>> on Linux is tremendously relevant to NetBSD.
}>>
}>> But, even if there is some problem with vnd backed by NFS, that hardly
}>> seems like it would be a good reason to make a change that reduces I/O
}>> throughput for the *non* NFS-backed case by at least 20%.  Why would one
}>> keep disk images for guests, as a general rule, on NFS, rather than simply
}>> doing the NFS mounts on the guests themselves, or using a more sensible
}>> protocol like iSCSI?  I have to assume most folks using files as disks are
}>> storing them on local filesystems on the dom0, and wrecking performance
}>> for that case to solve a problem with NFS that may or may not be hypothetical
}>> seems like a very, very bad trade-off.
}>>
}>> Thor
}> Yeah, I usually don't pipe up, but I strongly dislike being forced to 
}> use qemu to read raw disk file instead of being able to use vnd on 
}> netbsd dom0.
}
}I'm not a fan of Qemu either, but I don't know an easy way to detect if
}an image is in a NFS share, or if an image is in qcow, qcow2 or raw. I'm
}open to a solution that only launches Qemu if image format is different
}than raw or if the image file is in a NFS share.
	I admit that I don't know a lot about this particular problem and I'm
joining the discussion late, but isn't there a relatively easy set of steps
to follow to determinne if qemu needs to be called?

1. Look at the filesystem type of the filesystem in which the target
resides.  If it's an NFS filesystem, call qemu.  Otherwise, go to step 2.

2.  Examine the file and see if it has a signature for QCOW or QCOW2 format.
(How does qemu figure out which it is?  Is it a simple bit of code that
could be snagged for re-use?)  Alternatively, check to see if it's a raw
image, and, if it is not, call qemu, otherwise, call vnd and use the
existing backend  block driver.  

	I seem to remember that under xen3.3, there is a python script that does 
essentially step 2, except that it's tuned to detecte linux images or X86
partition tables and knows nothing about NetBSD partitions or FreeBSD
slices.

-Brian

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Ian Campbell]

On Thu, 2012-11-29 at 18:18 +0000, Brian Buhrow wrote:

> 2.  Examine the file and see if it has a signature for QCOW or QCOW2 format.
> (How does qemu figure out which it is?  Is it a simple bit of code that
> could be snagged for re-use?)  Alternatively, check to see if it's a raw
> image, and, if it is not, call qemu, otherwise, call vnd and use the
> existing backend  block driver.  

You should never probe for raw vs qcow (or any other container format)
in this way.

Imagine a VM with a "raw" disk. A malicious administrator of this VM
could write a qcow header to it turning it into a snapshot backed by any
path they like in dom0 (say /etc/shadow, ssh host keys, etc). Then they
reboot their guest, the probing determines that it is now a qcow image
with the given backing file, the VM admin simply reads from the disk to
get at the dom0 file.

The format of the image must always be specified by the user.

Ian.

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Aaron J. Grier]

On Thu, Nov 29, 2012 at 10:18:41AM -0800, Brian Buhrow wrote:
> 1. Look at the filesystem type of the filesystem in which the target
> resides.  If it's an NFS filesystem, call qemu.  Otherwise, go to step
> 2.

vnd can handle backing files on NFS.

-- 
  Aaron J. Grier | "Not your ordinary poofy goof." | agrier@poofygoof.com

[PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monne]

This forces libxl to use Qemu when a raw image file is used as a disk
backend. NetBSD currently only supports qemu-traditional, so
device_model_version="qemu-xen-traditional" has to be added to the
config file when using image files.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 tools/libxl/libxl_netbsd.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libxl/libxl_netbsd.c b/tools/libxl/libxl_netbsd.c
index 9587833..76e5f40 100644
--- a/tools/libxl/libxl_netbsd.c
+++ b/tools/libxl/libxl_netbsd.c
@@ -19,10 +19,10 @@
 
 int libxl__try_phy_backend(mode_t st_mode)
 {
-    if (S_ISREG(st_mode) || S_ISBLK(st_mode))
-        return 1;
+    if (!S_ISBLK(st_mode))
+        return 0;
 
-    return 0;
+    return 1;
 }
 
 char *libxl__devid_to_localdev(libxl__gc *gc, int devid)
-- 
1.7.7.5 (Apple Git-26)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Thor Lancelot Simon]

On Wed, Nov 28, 2012 at 02:20:51PM +0100, Roger Pau Monne wrote:
> This forces libxl to use Qemu when a raw image file is used as a disk
> backend. NetBSD currently only supports qemu-traditional, so

Wait -- do I grasp correctly that this patch _forces_ NetBSD as a dom0
to handle file-backed I/O requests in a way that includes additional
context switches to userspace, and which your previous email said
resulted in a 30% reduction in throughput compared to dom0 I/O?

I am assuming there's some reason we want this, rather than handling
such requests in-kernel.  But what is it?

Thor

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monné]

On 28/11/12 14:26, Thor Lancelot Simon wrote:
> On Wed, Nov 28, 2012 at 02:20:51PM +0100, Roger Pau Monne wrote:
>> This forces libxl to use Qemu when a raw image file is used as a disk
>> backend. NetBSD currently only supports qemu-traditional, so
> 
> Wait -- do I grasp correctly that this patch _forces_ NetBSD as a dom0
> to handle file-backed I/O requests in a way that includes additional
> context switches to userspace, and which your previous email said
> resulted in a 30% reduction in throughput compared to dom0 I/O?
> 
> I am assuming there's some reason we want this, rather than handling
> such requests in-kernel.  But what is it?

I should have marked this as "experimental", or something like this. I'm
not sure if switching to Qdisk for all image file backends is needed.
>From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:

"loop device had problems with flushing dirty pages (specifically, doing
a large number of writes to an NFS-backed image don't result in the OOM
killer going berserk)."

I'm not sure if NetBSD is in the same situation, but if I remember
correctly (haven't tried that in a long time), trying to use a disk file
on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
issue is: http://gnats.netbsd.org/40726.

If this is solved, and the vnd device is solid in NetBSD we could only
use Qdisk when the image format is different than raw (qcow, qcow2,
vmdk, vdi...).

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Thor Lancelot Simon]

On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
> On 28/11/12 14:26, Thor Lancelot Simon wrote:
> > 
> > I am assuming there's some reason we want this, rather than handling
> > such requests in-kernel.  But what is it?
> 
> I should have marked this as "experimental", or something like this. I'm
> not sure if switching to Qdisk for all image file backends is needed.
> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
> 
> "loop device had problems with flushing dirty pages (specifically, doing
> a large number of writes to an NFS-backed image don't result in the OOM
> killer going berserk)."
> 
> I'm not sure if NetBSD is in the same situation, but if I remember
> correctly (haven't tried that in a long time), trying to use a disk file
> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
> issue is: http://gnats.netbsd.org/40726.

It seems highly unlikely to me that a problem with the loop device driver
on Linux is tremendously relevant to NetBSD.

But, even if there is some problem with vnd backed by NFS, that hardly
seems like it would be a good reason to make a change that reduces I/O
throughput for the *non* NFS-backed case by at least 20%.  Why would one
keep disk images for guests, as a general rule, on NFS, rather than simply
doing the NFS mounts on the guests themselves, or using a more sensible
protocol like iSCSI?  I have to assume most folks using files as disks are
storing them on local filesystems on the dom0, and wrecking performance
for that case to solve a problem with NFS that may or may not be hypothetical
seems like a very, very bad trade-off.

Thor

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monné]

On 28/11/12 16:29, Thor Lancelot Simon wrote:
> On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
>> On 28/11/12 14:26, Thor Lancelot Simon wrote:
>>>
>>> I am assuming there's some reason we want this, rather than handling
>>> such requests in-kernel.  But what is it?
>>
>> I should have marked this as "experimental", or something like this. I'm
>> not sure if switching to Qdisk for all image file backends is needed.
>> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
>>
>> "loop device had problems with flushing dirty pages (specifically, doing
>> a large number of writes to an NFS-backed image don't result in the OOM
>> killer going berserk)."
>>
>> I'm not sure if NetBSD is in the same situation, but if I remember
>> correctly (haven't tried that in a long time), trying to use a disk file
>> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
>> issue is: http://gnats.netbsd.org/40726.
> 
> It seems highly unlikely to me that a problem with the loop device driver
> on Linux is tremendously relevant to NetBSD.
> 
> But, even if there is some problem with vnd backed by NFS, that hardly
> seems like it would be a good reason to make a change that reduces I/O
> throughput for the *non* NFS-backed case by at least 20%.  Why would one
> keep disk images for guests, as a general rule, on NFS, rather than simply
> doing the NFS mounts on the guests themselves, or using a more sensible
> protocol like iSCSI?  I have to assume most folks using files as disks are
> storing them on local filesystems on the dom0, and wrecking performance
> for that case to solve a problem with NFS that may or may not be hypothetical
> seems like a very, very bad trade-off.

Quite a lot of people use disk images on NFS, because it's probably the
easiest shared storage that allows migration of guests.

Even if disks on NFS was a remote possibility not used by much people I
don't think it's acceptable to have a Dom0 crash when you try to create
a guest with a disk on NFS, that is by far much more worse than
performance degradation in my opinion.

If this is not accepted, I will have to detect image format somewhere
and use qemu when the format is different than raw.

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Jeff Rizzo]

On 11/28/12 7:48 AM, Roger Pau Monné wrote:
> On 28/11/12 16:29, Thor Lancelot Simon wrote:
>
>> It seems highly unlikely to me that a problem with the loop device driver
>> on Linux is tremendously relevant to NetBSD.
>>
>> But, even if there is some problem with vnd backed by NFS, that hardly
>> seems like it would be a good reason to make a change that reduces I/O
>> throughput for the *non* NFS-backed case by at least 20%.  Why would one
>> keep disk images for guests, as a general rule, on NFS, rather than simply
>> doing the NFS mounts on the guests themselves, or using a more sensible
>> protocol like iSCSI?  I have to assume most folks using files as disks are
>> storing them on local filesystems on the dom0, and wrecking performance
>> for that case to solve a problem with NFS that may or may not be hypothetical
>> seems like a very, very bad trade-off.
> Quite a lot of people use disk images on NFS, because it's probably the
> easiest shared storage that allows migration of guests.

I agree that this is a case we should be supporting well.  We don't 
support migration yet, but I'd like to see this change in the nearish 
future.

>
> Even if disks on NFS was a remote possibility not used by much people I
> don't think it's acceptable to have a Dom0 crash when you try to create
> a guest with a disk on NFS, that is by far much more worse than
> performance degradation in my opinion.

And, of course, a crash is suboptimal.  :)

>
> If this is not accepted, I will have to detect image format somewhere
> and use qemu when the format is different than raw.

I would definitely like to see some middle-ground compromise that allows 
us to keep decent performance for local files while still allowing them 
for NFS.

+j

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monné]

On 28/11/12 18:42, Jeff Rizzo wrote:
> On 11/28/12 7:48 AM, Roger Pau Monné wrote:
>> On 28/11/12 16:29, Thor Lancelot Simon wrote:
>>
>>> It seems highly unlikely to me that a problem with the loop device driver
>>> on Linux is tremendously relevant to NetBSD.
>>>
>>> But, even if there is some problem with vnd backed by NFS, that hardly
>>> seems like it would be a good reason to make a change that reduces I/O
>>> throughput for the *non* NFS-backed case by at least 20%.  Why would one
>>> keep disk images for guests, as a general rule, on NFS, rather than simply
>>> doing the NFS mounts on the guests themselves, or using a more sensible
>>> protocol like iSCSI?  I have to assume most folks using files as disks are
>>> storing them on local filesystems on the dom0, and wrecking performance
>>> for that case to solve a problem with NFS that may or may not be hypothetical
>>> seems like a very, very bad trade-off.
>> Quite a lot of people use disk images on NFS, because it's probably the
>> easiest shared storage that allows migration of guests.
> 
> I agree that this is a case we should be supporting well.  We don't 
> support migration yet, but I'd like to see this change in the nearish 
> future.
> 
>>
>> Even if disks on NFS was a remote possibility not used by much people I
>> don't think it's acceptable to have a Dom0 crash when you try to create
>> a guest with a disk on NFS, that is by far much more worse than
>> performance degradation in my opinion.
> 
> And, of course, a crash is suboptimal.  :)
> 
>>
>> If this is not accepted, I will have to detect image format somewhere
>> and use qemu when the format is different than raw.
> 
> I would definitely like to see some middle-ground compromise that allows 
> us to keep decent performance for local files while still allowing them 
> for NFS.

Well, I think the performance is not that bad, given that we are using
qemu-traditional and the Dom0 is not MP. We can archive better
performance by fixing qemu-upstream to work with NetBSD and of course
getting a MP Dom0.

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Manuel Bouyer]

On Wed, Nov 28, 2012 at 09:42:45AM -0800, Jeff Rizzo wrote:
> [...]
> >
> >If this is not accepted, I will have to detect image format somewhere
> >and use qemu when the format is different than raw.
> 
> I would definitely like to see some middle-ground compromise that
> allows us to keep decent performance for local files while still
> allowing them for NFS.

using the in-kernel or qemu backend should be configurable.
We have a domU configuration file, lets just use it !

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Manuel Bouyer]

On Wed, Nov 28, 2012 at 07:18:58PM +0100, Roger Pau Monné wrote:
> Well, I think the performance is not that bad, given that we are using
> qemu-traditional and the Dom0 is not MP. We can archive better
> performance by fixing qemu-upstream to work with NetBSD and of course
> getting a MP Dom0.

But it'll still be slower than a in-kernel backend driver.


-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Thor Lancelot Simon]

On Wed, Nov 28, 2012 at 07:18:58PM +0100, Roger Pau Monn? wrote:
> 
> Well, I think the performance is not that bad, given that we are using
> qemu-traditional and the Dom0 is not MP. We can archive better
> performance by fixing qemu-upstream to work with NetBSD and of course
> getting a MP Dom0.

Why is it a reasonable assumption that adding more CPU resource will
make a significant difference to I/O throughput?

-- 
 Thor Lancelot Simon	                                      tls@panix.com
   But as he knew no bad language, he had called him all the names of common
 objects that he could think of, and had screamed: "You lamp!  You towel!  You
 plate!" and so on.              --Sigmund Freud

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Aaron J. Grier]

On Wed, Nov 28, 2012 at 01:46:40PM -0500, Thor Lancelot Simon wrote:
> On Wed, Nov 28, 2012 at 07:18:58PM +0100, Roger Pau Monn? wrote:
> > Well, I think the performance is not that bad, given that we are
> > using qemu-traditional and the Dom0 is not MP. We can archive better
> > performance by fixing qemu-upstream to work with NetBSD and of
> > course getting a MP Dom0.
> 
> Why is it a reasonable assumption that adding more CPU resource will
> make a significant difference to I/O throughput?

throughput might not be affected, but wouldn't additional code execution
affect latency?

-- 
  Aaron J. Grier | "Not your ordinary poofy goof." | agrier@poofygoof.com

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Toby Karyadi]

On 11/28/12 10:29 AM, Thor Lancelot Simon wrote:
> On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
>> On 28/11/12 14:26, Thor Lancelot Simon wrote:
>>> I am assuming there's some reason we want this, rather than handling
>>> such requests in-kernel.  But what is it?
>> I should have marked this as "experimental", or something like this. I'm
>> not sure if switching to Qdisk for all image file backends is needed.
>> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
>>
>> "loop device had problems with flushing dirty pages (specifically, doing
>> a large number of writes to an NFS-backed image don't result in the OOM
>> killer going berserk)."
>>
>> I'm not sure if NetBSD is in the same situation, but if I remember
>> correctly (haven't tried that in a long time), trying to use a disk file
>> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
>> issue is: http://gnats.netbsd.org/40726.
> It seems highly unlikely to me that a problem with the loop device driver
> on Linux is tremendously relevant to NetBSD.
>
> But, even if there is some problem with vnd backed by NFS, that hardly
> seems like it would be a good reason to make a change that reduces I/O
> throughput for the *non* NFS-backed case by at least 20%.  Why would one
> keep disk images for guests, as a general rule, on NFS, rather than simply
> doing the NFS mounts on the guests themselves, or using a more sensible
> protocol like iSCSI?  I have to assume most folks using files as disks are
> storing them on local filesystems on the dom0, and wrecking performance
> for that case to solve a problem with NFS that may or may not be hypothetical
> seems like a very, very bad trade-off.
>
> Thor
Yeah, I usually don't pipe up, but I strongly dislike being forced to 
use qemu to read raw disk file instead of being able to use vnd on 
netbsd dom0.

There was/is? a bug in xentools 4.1 in libxl that prevents using vnd for 
raw disk file. The problem boils down to libxl not notifying xenbackendd 
(via xenstored) to call /usr/pkg/etc/xen/scripts/block at the right time 
such that vnconfig doesn't get called when it's time to shutdown the 
domU, and hence you end up with a bunch of vnd devices even after the 
corresponding domU's have been shutdown. I created a patch to fix that: 
http://mail-index.netbsd.org/port-xen/2012/05/29/msg007252.html 
Additionally I put in an experimental feature that allows specifying 
other types of backend aside from phy: or file:, and as long as the 
/usr/pkg/etc/xen/scripts/block sh script is modified accordingly, the 
domU will just get the appropriate device in dom0. Then we can use, for 
example, vnd:, or iscsi: and from the POV of the domU, it's just another 
'physical' device that the dom0 manages. I think the way xenbackendd 
works, in conjunction with xenstored, is pretty flexible and decoupled, 
at least with xen 3.3 and 4.1; I don't know what the story is for xen 
4.2. Was there a rumor of retiring xenbackendd?

I'm also concerned about the trend, which to me is more of putting more 
and more policy in libxl, as opposed to functionality. I saw this back 
when I was trying to fix the bug above, hopefully this trend is no 
longer true. For example, I had the impression (I don't remember the 
details) that the code that parses the disk configuration to a certain 
degree decides whether the disk is a block device or a regular file 
based on the prefix (phy: vs file:), which is not true in netbsd dom0, 
because the file path is then used to setup a vnd device, which is from 
libxl's POV is just a block device. Most probably it's born out of the 
fact that libxl is developed in a mostly linux environment. I do 
appreciate Roger's effort in improving xen support in netbsd, but 
hopefully some BSDism would travel back to xen as well.

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Roger Pau Monné]

On 28/11/12 20:40, Toby Karyadi wrote:
> On 11/28/12 10:29 AM, Thor Lancelot Simon wrote:
>> On Wed, Nov 28, 2012 at 03:41:56PM +0100, Roger Pau Monn? wrote:
>>> On 28/11/12 14:26, Thor Lancelot Simon wrote:
>>>> I am assuming there's some reason we want this, rather than handling
>>>> such requests in-kernel.  But what is it?
>>> I should have marked this as "experimental", or something like this. I'm
>>> not sure if switching to Qdisk for all image file backends is needed.
>>> >From the Xen wiki (http://wiki.xen.org/wiki/Blktap) I've found that:
>>>
>>> "loop device had problems with flushing dirty pages (specifically, doing
>>> a large number of writes to an NFS-backed image don't result in the OOM
>>> killer going berserk)."
>>>
>>> I'm not sure if NetBSD is in the same situation, but if I remember
>>> correctly (haven't tried that in a long time), trying to use a disk file
>>> on a NFS share caused the NetBSD Dom0 kernel to crash. The PR for this
>>> issue is: http://gnats.netbsd.org/40726.
>> It seems highly unlikely to me that a problem with the loop device driver
>> on Linux is tremendously relevant to NetBSD.
>>
>> But, even if there is some problem with vnd backed by NFS, that hardly
>> seems like it would be a good reason to make a change that reduces I/O
>> throughput for the *non* NFS-backed case by at least 20%.  Why would one
>> keep disk images for guests, as a general rule, on NFS, rather than simply
>> doing the NFS mounts on the guests themselves, or using a more sensible
>> protocol like iSCSI?  I have to assume most folks using files as disks are
>> storing them on local filesystems on the dom0, and wrecking performance
>> for that case to solve a problem with NFS that may or may not be hypothetical
>> seems like a very, very bad trade-off.
>>
>> Thor
> Yeah, I usually don't pipe up, but I strongly dislike being forced to 
> use qemu to read raw disk file instead of being able to use vnd on 
> netbsd dom0.

I'm not a fan of Qemu either, but I don't know an easy way to detect if
an image is in a NFS share, or if an image is in qcow, qcow2 or raw. I'm
open to a solution that only launches Qemu if image format is different
than raw or if the image file is in a NFS share.

> There was/is? a bug in xentools 4.1 in libxl that prevents using vnd for 
> raw disk file. The problem boils down to libxl not notifying xenbackendd 
> (via xenstored) to call /usr/pkg/etc/xen/scripts/block at the right time 
> such that vnconfig doesn't get called when it's time to shutdown the 
> domU, and hence you end up with a bunch of vnd devices even after the 
> corresponding domU's have been shutdown. I created a patch to fix that: 
> http://mail-index.netbsd.org/port-xen/2012/05/29/msg007252.html 
> Additionally I put in an experimental feature that allows specifying 
> other types of backend aside from phy: or file:, and as long as the 
> /usr/pkg/etc/xen/scripts/block sh script is modified accordingly, the 
> domU will just get the appropriate device in dom0. Then we can use, for 
> example, vnd:, or iscsi: and from the POV of the domU, it's just another 
> 'physical' device that the dom0 manages. I think the way xenbackendd 
> works, in conjunction with xenstored, is pretty flexible and decoupled, 
> at least with xen 3.3 and 4.1; I don't know what the story is for xen 
> 4.2. Was there a rumor of retiring xenbackendd?

Yes, xenbackendd is deprecated and hotplug scripts are called from libxl
directly (that's true for both NetBSD and Linux).

> I'm also concerned about the trend, which to me is more of putting more 
> and more policy in libxl, as opposed to functionality. I saw this back 
> when I was trying to fix the bug above, hopefully this trend is no 
> longer true. For example, I had the impression (I don't remember the 
> details) that the code that parses the disk configuration to a certain 
> degree decides whether the disk is a block device or a regular file 
> based on the prefix (phy: vs file:), which is not true in netbsd dom0, 
> because the file path is then used to setup a vnd device, which is from 
> libxl's POV is just a block device. Most probably it's born out of the 
> fact that libxl is developed in a mostly linux environment. I do 
> appreciate Roger's effort in improving xen support in netbsd, but 
> hopefully some BSDism would travel back to xen as well.

Yes, most libxl developers are Linux users, so most of the functionality
in libxl is designed with Linux in mind. The only way to change that is
to get BSD developers to contribute to libxl and the Xen project in
general, if we are not active players in this game, we should not expect
that BSD will be taken into account.

As an example, libxl is now working in NetBSD without any external
patches, and AFAIK all functionality is working, and Linux developers
are very careful to try not to break this compatibility.

>From my POV, I think your patches to add new functionality to disk
backends should be rebased against xen-unstable and posted to xen-devel,
so that they are added to upstream Xen.

Having a bunch of out-of-tree patches in pkgsrc is not the way to move
forward, because we are not able to test NetBSD with latest Xen. Being
able to test xen-unstable easily will allow us to find bugs and develop
patches before Xen is released, thus keeping this pkgsrc patch queue
minimal.

Re: [PATCH 2/2] libxl: switch NetBSD image file handling to Qemu

[Manuel Bouyer]

On Wed, Nov 28, 2012 at 02:20:51PM +0100, Roger Pau Monne wrote:
> This forces libxl to use Qemu when a raw image file is used as a disk
> backend. NetBSD currently only supports qemu-traditional, so
> device_model_version="qemu-xen-traditional" has to be added to the
> config file when using image files.

Why ? This impose a trip to userland, which is expensive.
xenbackendd should still call the backend script, right ?
Why not let it do its job and map the file to a vnd(4) ?

I don't support this change.

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--