Re: usbip doesn't work with userspace code that accesses USB devices

[Demi Marie Obenour <demi@invisiblethingslab.com>]

[-- Attachment #1: Type: text/plain, Size: 4806 bytes --]

On Tue, Mar 05, 2024 at 12:52:22AM +0100, Marek Marczykowski-Górecki wrote:
> On Mon, Mar 04, 2024 at 11:46:04PM +0100, Marek Marczykowski-Górecki wrote:
> > On Mon, Mar 04, 2024 at 09:04:00PM +0000, Greg KH wrote:
> > > On Mon, Mar 04, 2024 at 03:01:51PM -0500, Demi Marie Obenour wrote:
> > > > Qubes OS users are reporting that MTP doesn't work with USB passthrough.
> > > > Fastboot (used for flashing a custom OS to an Android device) also
> > > > doesn't work.  Kernel-mode drivers, such as Bluetooth and USB storage,
> > > > seem to usually work as expected.  Since MTP and fastboot are both
> > > > implemented in userspace, it appears that there is some problem with the
> > > > interaction of usbip, our USB proxy (which is based on USBIP), and
> > > > userspace programs that interact with USB devices directly.
> > > > 
> > > > The bug report can be found at [1] and the source code for the USB proxy
> > > > can be found at [2].  The script used on the sending side (the one with
> > > > the physical USB controller) is at [3] and the script used by the
> > > > receiving side (the one the device is attached to) is at [4].  All of
> > > > these links are for the current version as of this email being sent, so
> > > > that anyone looking at this email in the future doesn't get confused.
> > > > 
> > > > Is this a bug in usbip, or is this due to usbip being used incorrectly?
> > > 
> > > I'm amazed that usbip works with usbfs at all, I didn't think that was a
> > > thing.
> > > 
> > > If you have a reproducer, or some error messages somewhere, that might
> > > be the easiest way forward.  In reading the bug report, it looks like
> > > the "bridge" you all made can't handle the device disconnecting itself
> > > properly?  But that's just a guess, could be anything.
> > 
> > Device disconnecting itself indeed is an issue (our proxy doesn't
> > automatically reconnect it, at least not yet). But that's definitely not
> > the only issue, things break also when disconnect is not involved.
> > 
> > Terminology:
> > 1. sys-usb - a VM where USB controller (a PCI device) lives; here
> > usbip-host is attached to the device
> > 2. testvm - target VM where usbip is connected; here vhci-hcd is used
> > 3. qvm-usb - tool that connects the above two (equivalent of
> > userspace part of standard usbip)
> > 
> > Specific steps:
> > 1. Connect android phone - at this point it's only in sys-usb
> > 2. Switch its mode to file transfer - observe reconnect in sys-usb
> > 3. Use qvm-usb to attach it to the testvm
> > 4. Call jmtpfs -d /mnt in testvm
> 
> Or maybe reset or something is involved here too?
> 
> After using qvm-usb to attach _and detach_ the device, it stays bound to
> usbip-host driver (that's intentional). But then, even after re-binding
> back to the "usb" driver, jmtpfs called in sys-usb directly fails the
> same way, including failure to reset.
> 
> In fact, even without usbip involved at all, jmtpfs directly in sys-usb
> works only once. The second attempt (without either physically reconnecting
> the phone, or changing its more to "no data transfer" and back to "file
> transfer") fails the same way. After terminating the first instance, I
> see just this logged:
> 
>     [921332.525210] usb 2-1: reset high-speed USB device number 22 using xhci_hcd

What happens if the device is not bound to _any_ driver in sys-usb?  For
instance, does the problem go away if the only USB-related drivers in
sys-usb are xhci-pci, usbip-host, and their dependencies?  If not, what
about if sys-usb's kernel has no other USB-related drivers included at
all?

Also, if you have not done so already, could you try uninstalling gvfs
from sys-usb's template?  gvfs might be interacting with the device.
Using a -minimal template might help.

> Since both problematic cases involve Android, maybe the actual issue is
> somewhere in Android instead of usbip? But then, fastboot could be a
> completely different issue, likely related to reconnection (this one I
> haven't tried to reproduce, and can't find much details in our issues
> tracker).
> On the other hand, USB tethering works just fine, so at least some
> Android functions do work with usbip (but then, it's a kernel driver,
> not usbfs).
> Could be also an issue with just MTP implementation on either side of
> the connection (I did tried also gvfs instead of jmtpfs with similar
> result, but they likely use the same implementation). Since that's
> Android, the device side is Linux too, but I don't know how MTP is
> implemented there (I don't see MTP gadget function in upstream Linux).

I suspect MTP is implemented in userspace somewhere in AOSP.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]