Re: xl PVUSB pass-through

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Sun, Mar 10, 2013 at 11:46:50PM +0100, Marek Marczykowski wrote:
> On 07.03.2013 15:11, Pasi Kärkkäinen wrote:
> > On Thu, Mar 07, 2013 at 10:07:32AM +0000, George Dunlap wrote:
> >> On Wed, Mar 6, 2013 at 3:59 PM, Stefan <sstanisi@cbnco.com> wrote:
> >>> Good Morning:
> >>>
> >>> Our Xen clusters depend heavily on that feature. The lack of it prevents us
> >>> from transitioning to the new toolstack. Currently we use PVUSB to attach a
> >>> USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP Blade
> >>> Server with the Token mounted on an internal USB Port to our domU CA server
> >>> (SLES 11)
> >>>
> >>> I'd like to know how likely it is to make it to the next release. Does
> >>> anyone have a solution that doesn't require PVUSB? I'd like to give a hand
> >>> if that can help making PVUSB part of Xen 4.3.
> >>
> >> For PVUSB to work for any particular VM, you need two things:
> >> * PVUSB kernel support in the guest VM and dom0 (or driver domain)
> >> * The toolstack has to know how to connect the two.
> >>
> >> "Classic xen" kernels like those supported by SuSE have PVUSB support,
> >> and xend has support for setting up the connection.
> >>
> >> So regarding PVUSB there are two tasks:
> >>
> >> 1. Teach xl/libxl how to set up the connection
> >> 2. Port PVUSB to pvops kernels.
> >>
> > 
> > PVUSB drivers have already been ported to pvops kernels,
> > and the PVUSB drivers can be found from Konrad's git repo.
> > 
> > What is missing is upstreaming the drivers.
> 
> Actually the drivers aren't working well :(
> We've tried to use them in Qubes OS, but it is definitely not production ready
> yet.
> 
> Details in this thread (mostly on Qubes-specific tools, but also on kernel
> pvusb drivers):
> https://groups.google.com/group/qubes-devel/browse_thread/thread/e002ae940061d897/b4d63fbbb1b29b48
> 
> Some quotes (from Alexandre Bezroutchko who was testing/debugging the drivers):
> "There might be more than one problem actually. The easiest to reproduce
> is kernel memory management issue which reliably causes oopses in
> kfree() USB Ethernet adapter is attached. This happens when the backend
> tries to free memory block used for USB event handling. I started
> debugging it, activated memory-related debugging options in the kernel
> and it started crashing much earlier, during bootstrap."

I believe the fix for that was posted by Boris:

https://lkml.org/lkml/2013/2/26/420
> 
> "Another problem I saw is bus resets. When I try to pass-through USB
> stick, it gets attached but it takes a minute or so before it starts
> working in domU. During this time there are some bus timeout messages or
> something in dmesg. This could have something to do with missed events
> in frontend-backend communication. I didn't investigate it though, guess
> memory management issue needs to be sorted out first."
> 
> 
> Trying USBIP (as fallback for PVUSB) points out same problem as in PVUSB:
> "The actual problem is USBIP module frees URB transfer buffers
> and USB core does the same afterwards. There is a flag controlling this
> behaviour (introduced in this patch [1] in kernel). Apparently this flag is
> controlled remotely by the frontend, it is mentioned in usbip_protocol.txt
> part of the protocol. This is odd, why it can possibly be up to the
> frontend to decide whether to free the transfer buffer or not. This
> transfer buffer is allocated at stub_rx.c:485, right after URB holding it
> is allocated with urb_alloc_urb() at lines 472/475. Eventually the transfer
> buffer is freed in stub_main.c:239 and stub_tx.c:31, right before URB is
> freed with usb_free_urb(). The latter internally triggers urb_destroy()
> which, if URB_FREE_BUFFER flag is set in URB, calls kfree() and actually
> causes all the troubles. I think proper solution would be to mask out
> URB_FREE_BUFFER from the received transfer flags. There is already some
> filtering for the flags stub_rx.c:437 but URB_FREE_BUFFER is explicitly
> permitted, which is odd.
> 
> By the way, it is possible that pvusb was plagued by the same issue -- at
> the first glance I don't find any attempt to filter value of
> transfer_flags, normally should happen somewhere around usbback.c:526, but
> it does not.
> 
> If I'm not mistaken the above problem only affects the backend, so frontend
> lockups something else probably. Maybe it is a side effect of my patch to
> the lockup problem :). To troubleshoot stalls I figure I can try to run
> debugger against the stalled kernel over a virtual serial link and dump
> stack trace on the stalled CPU, this might clairfy where the problem is."
> 
> Some fix (workaround?) is here:
> https://github.com/grwl/qubes-kernel/commit/4f9a1404eb00b679b507a906a8ea26dea0a4874b
> But this is only one issue of many.
> 
> > 
> >> It sounds like the main thing you need is to have the xl/libxl
> >> support.  This should just be a matter of looking at what xend does in
> >> setting up the PVUSB connection, and duplicating that in xl.  If you'd
> >> be willing to give that a try, that would be great -- we'd love to
> >> help out.
> >>
> >> If you've never submitted patches before,
> >> http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful
> >> advice.
> >>
> >> If for some reason PVUSB doesn't make it into 4.3, another option you
> >> can explore is running your VMs in PVHVM mode; xl will definitely
> >> support USB pass-through for HVM domains in 4.3.
> >>
> > 
> > 
> > -- Pasi
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> > 
> 
> 
> -- 
> Best Regards / Pozdrawiam,
> Marek Marczykowski
> Invisible Things Lab
> 



> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel