Re: libxl: problem with devices in PV

[Christoph Egger <Christoph.Egger@amd.com>]

On 07/20/11 15:30, Ian Campbell wrote:
> On Wed, 2011-07-20 at 14:10 +0100, Roger Pau Monné wrote:
>> 2011/7/20 Ian Campbell<Ian.Campbell@citrix.com>:
>>> On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
>>>> On Wed, 20 Jul 2011, Roger Pau Monné wrote:
>>>>> Hello,
>>>>>
>>>>> I'm trying to run PV machines using the new libxenlight toolstack on
>>>>> NetBSD. So far, I've been able to connect to the domu using the
>>>>> console (I was unable to do so before). I'm attaching a little patch
>>>>> that removes setting the consback to IOEMU when detecting a qdisk
>>>>> (that was preventing the domu from even booting). With the
>>>>> introduction of libxenlight, Xen doesn't use vbd for disk backend
>>>>> anymore, it uses qdisk, which I assume Qemu automatically attaches to
>>>>> running guests. It works fine with HVM guests, but it seems to fail
>>>>> with PV guests. The config file I'm using is:
>>>>>
>>>>
>>>> Xen uses qdisk only when blktap is not available.
>>>> I suggest you install blktap if you can because it is significantly
>>>> faster than qdisk at the moment.
>>>
>>> This is a NetBSD host so I don't think blktap is an option.
>>
>> NetBSD has the vnd driver, which provides a disk interface to a file
>> (it's basically a loop device):
>>
>> http://netbsd.gw.com/cgi-bin/man-cgi?vnd+4+NetBSD-current
>>
>> It is used with xend and xenbackendd to mount raw disks.
>
> If I understand correctly (which is remotish possibility):
>
> xenbackendd watches the backend/vbd directory and creates disks based on
> it. A "vbd" directory roughly corresponds to a "phy:" type device
> configuration in libxl.

Correct.

> xend, at least on Linux, internally handles "file:" by setting up a loop
> device and translating the config into a "phy:"==vbd refering to
> the /dev/loopN device. I suspect that on NetBSD xend used to just create
> the vbd referring to the file directly and let xenbackendd take care of
> any necessary loop back stuff.

More precisely, xenbackendd invokes the hotplug scripts and those take
care of choosing a spare vnd device.

> libxl on the other hand does not do this loop device magic but instead
> palms "file:" off onto either qdisk or blktap and furthermore libxl
> requires that a "phy:" disk configuration refers to a block device and
> not a file.
 >
> The upshot is that the backend selection logic in libxl is apparently
> not correct for NetBSD which _can_ handle files passed as "phy:" due to
> xenbackendd doing the right thing via the vnd driver.

Right.

> Fortunately Ian Jackson recently cleaned up the backend selection logic
> in libxl and it should be much cleaner and easier to express these sorts
> of system specific things, presumably by patching disk_try_backend to
> allow files as well as block devices for LIBXL_DISK_BACKEND_PHY on
> NetBSD.

Roger: In libxl/libxl_device.c:disk_try_backend() can you try disabling
the second check in the LIBXL_DISK_BACKEND_PHY case, please?

Or surrounding it with #ifdef __Linux__ ... #endif should do it.

Christoph

>>>> Otherwise if you use a physical partition and specify phy: in the config
>>>> file you should get the kernel based blkback that is the fastest option
>>>> available.
>>>
>>> phy: would be worth a go since it should tie into NetBSD's equivalent of
>>> blkback, whereas file: presumably goes to qdisk in the absence of
>>> blktap.
>>
>> Haven't tried phy:/ with unstable, but I supose it works, since it
>> worked in previous versions.
>
> Since you are passing a file name a block device it is possible that
> libxl today might reject it when used with phy:/...
>
> [...]
>> What I don't get is why qdisk work with HVM machines but not with PV
>> machines, shouldn't it be the same?
>
> I suspect that in the HVM case you aren't really getting a qdisk (a PV
> backend) but actually an emulated disk device. Or rather you are
> probably getting both but the guest only tries to use the emulated one
> and hence the PV backend never tries to initialise and so never falls
> over due to lack of gntdev.
>
> Can you confirm if your HVM guest uses emulated or PV frontends?
>
> Ian.


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632