From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: xl PVUSB pass-through Date: Mon, 11 Mar 2013 09:46:00 -0400 Message-ID: <20130311134600.GL23018@phenom.dumpdata.com> References: <513662D2.2050109@cbnco.com> <513718D2.5060202@eu.citrix.com> <513767DE.3000605@cbnco.com> <20130307141104.GX8912@reaktio.net> <513D0D5A.40403@invisiblethingslab.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <513D0D5A.40403@invisiblethingslab.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Marek Marczykowski Cc: George Dunlap , Stefan , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Sun, Mar 10, 2013 at 11:46:50PM +0100, Marek Marczykowski wrote: > On 07.03.2013 15:11, Pasi K=E4rkk=E4inen wrote: > > On Thu, Mar 07, 2013 at 10:07:32AM +0000, George Dunlap wrote: > >> On Wed, Mar 6, 2013 at 3:59 PM, Stefan wrote: > >>> Good Morning: > >>> > >>> Our Xen clusters depend heavily on that feature. The lack of it preve= nts us > >>> from transitioning to the new toolstack. Currently we use PVUSB to at= tach a > >>> USB Smartcard reader through our dom0 (SLES 11 SP1) running on an HP = Blade > >>> Server with the Token mounted on an internal USB Port to our domU CA = server > >>> (SLES 11) > >>> > >>> I'd like to know how likely it is to make it to the next release. Does > >>> anyone have a solution that doesn't require PVUSB? I'd like to give a= hand > >>> if that can help making PVUSB part of Xen 4.3. > >> > >> For PVUSB to work for any particular VM, you need two things: > >> * PVUSB kernel support in the guest VM and dom0 (or driver domain) > >> * The toolstack has to know how to connect the two. > >> > >> "Classic xen" kernels like those supported by SuSE have PVUSB support, > >> and xend has support for setting up the connection. > >> > >> So regarding PVUSB there are two tasks: > >> > >> 1. Teach xl/libxl how to set up the connection > >> 2. Port PVUSB to pvops kernels. > >> > > = > > PVUSB drivers have already been ported to pvops kernels, > > and the PVUSB drivers can be found from Konrad's git repo. > > = > > What is missing is upstreaming the drivers. > = > Actually the drivers aren't working well :( > We've tried to use them in Qubes OS, but it is definitely not production = ready > yet. > = > Details in this thread (mostly on Qubes-specific tools, but also on kernel > pvusb drivers): > https://groups.google.com/group/qubes-devel/browse_thread/thread/e002ae94= 0061d897/b4d63fbbb1b29b48 > = > Some quotes (from Alexandre Bezroutchko who was testing/debugging the dri= vers): > "There might be more than one problem actually. The easiest to reproduce > is kernel memory management issue which reliably causes oopses in > kfree() USB Ethernet adapter is attached. This happens when the backend > tries to free memory block used for USB event handling. I started > debugging it, activated memory-related debugging options in the kernel > and it started crashing much earlier, during bootstrap." I believe the fix for that was posted by Boris: https://lkml.org/lkml/2013/2/26/420 > = > "Another problem I saw is bus resets. When I try to pass-through USB > stick, it gets attached but it takes a minute or so before it starts > working in domU. During this time there are some bus timeout messages or > something in dmesg. This could have something to do with missed events > in frontend-backend communication. I didn't investigate it though, guess > memory management issue needs to be sorted out first." > = > = > Trying USBIP (as fallback for PVUSB) points out same problem as in PVUSB: > "The actual problem is USBIP module frees URB transfer buffers > and USB core does the same afterwards. There is a flag controlling this > behaviour (introduced in this patch [1] in kernel). Apparently this flag = is > controlled remotely by the frontend, it is mentioned in usbip_protocol.txt > part of the protocol. This is odd, why it can possibly be up to the > frontend to decide whether to free the transfer buffer or not. This > transfer buffer is allocated at stub_rx.c:485, right after URB holding it > is allocated with urb_alloc_urb() at lines 472/475. Eventually the transf= er > buffer is freed in stub_main.c:239 and stub_tx.c:31, right before URB is > freed with usb_free_urb(). The latter internally triggers urb_destroy() > which, if URB_FREE_BUFFER flag is set in URB, calls kfree() and actually > causes all the troubles. I think proper solution would be to mask out > URB_FREE_BUFFER from the received transfer flags. There is already some > filtering for the flags stub_rx.c:437 but URB_FREE_BUFFER is explicitly > permitted, which is odd. > = > By the way, it is possible that pvusb was plagued by the same issue -- at > the first glance I don't find any attempt to filter value of > transfer_flags, normally should happen somewhere around usbback.c:526, but > it does not. > = > If I'm not mistaken the above problem only affects the backend, so fronte= nd > lockups something else probably. Maybe it is a side effect of my patch to > the lockup problem :). To troubleshoot stalls I figure I can try to run > debugger against the stalled kernel over a virtual serial link and dump > stack trace on the stalled CPU, this might clairfy where the problem is." > = > Some fix (workaround?) is here: > https://github.com/grwl/qubes-kernel/commit/4f9a1404eb00b679b507a906a8ea2= 6dea0a4874b > But this is only one issue of many. > = > > = > >> It sounds like the main thing you need is to have the xl/libxl > >> support. This should just be a matter of looking at what xend does in > >> setting up the PVUSB connection, and duplicating that in xl. If you'd > >> be willing to give that a try, that would be great -- we'd love to > >> help out. > >> > >> If you've never submitted patches before, > >> http://wiki.xen.org/wiki/Submitting_Xen_Patches has some helpful > >> advice. > >> > >> If for some reason PVUSB doesn't make it into 4.3, another option you > >> can explore is running your VMs in PVHVM mode; xl will definitely > >> support USB pass-through for HVM domains in 4.3. > >> > > = > > = > > -- Pasi > > = > > = > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > = > = > = > -- = > Best Regards / Pozdrawiam, > Marek Marczykowski > Invisible Things Lab > = > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel