[USB Isolation] USB virt drivers access between guests instead of host -> guest?

[USB Isolation] USB virt drivers access between guests instead of host -> guest?

[privacymiscoccasion]

Hi everyone,

I'm coming over from reading about Qubes OS, which uses the Xen 
hypervisor. In Qubes, the way that untrusted devices like USBs are 
handled is that they are pass through to a VM, which then (I presume) 
allows other guests to access them using virtual drivers.

I'm looking for a theoretical explanation on how this would be possible 
with KVM. I am not a developer and thus am having difficulty 
understanding how one would let a guest access virtual drivers 
connecting to hardware devices like USB and PCIe from another guest.

Any help/practical examples of this would be greatly appreciated. This 
seems to be a hard topic to find and so far I haven't come across 
anything like this.

Thanks!

Re: [USB Isolation] USB virt drivers access between guests instead of host -> guest?

[Sean Christopherson]

On Mon, Jul 22, 2024, privacymiscoccasion@cock.li wrote:
> Hi everyone,
> 
> I'm coming over from reading about Qubes OS, which uses the Xen hypervisor.
> In Qubes, the way that untrusted devices like USBs are handled is that they
> are pass through to a VM, which then (I presume) allows other guests to
> access them using virtual drivers.
> 
> I'm looking for a theoretical explanation on how this would be possible with
> KVM. I am not a developer and thus am having difficulty understanding how
> one would let a guest access virtual drivers connecting to hardware devices
> like USB and PCIe from another guest.
> 
> Any help/practical examples of this would be greatly appreciated. This seems
> to be a hard topic to find and so far I haven't come across anything like
> this.

In Linux, this would be done via VFIO[1].  VFIO allows assigning devices to host
userspace, and thus to KVM guests.  Very rougly speaking, most assets that get
exposed to KVM guests are proxied through host userspace.  I haven't actually
read the DPDK docs[2], but if you get stuck with VFIO in particular, my guess is
that they're a good starting point (beyond any VFIO+KVM tutorials).

[1] https://docs.kernel.org/driver-api/vfio.html
[2] https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html

Re: [USB Isolation] USB virt drivers access between guests instead of host -> guest?

[privacymiscoccasion]

On 2024-07-22 19:30, Sean Christopherson wrote:
> On Mon, Jul 22, 2024, privacymiscoccasion@cock.li wrote:
>> Hi everyone,
>> 
>> I'm coming over from reading about Qubes OS, which uses the Xen 
>> hypervisor.
>> In Qubes, the way that untrusted devices like USBs are handled is that 
>> they
>> are pass through to a VM, which then (I presume) allows other guests 
>> to
>> access them using virtual drivers.
>> 
>> I'm looking for a theoretical explanation on how this would be 
>> possible with
>> KVM. I am not a developer and thus am having difficulty understanding 
>> how
>> one would let a guest access virtual drivers connecting to hardware 
>> devices
>> like USB and PCIe from another guest.
>> 
>> Any help/practical examples of this would be greatly appreciated. This 
>> seems
>> to be a hard topic to find and so far I haven't come across anything 
>> like
>> this.
> 
> In Linux, this would be done via VFIO[1].  VFIO allows assigning 
> devices to host
> userspace, and thus to KVM guests.  Very rougly speaking, most assets 
> that get
> exposed to KVM guests are proxied through host userspace.  I haven't 
> actually
> read the DPDK docs[2], but if you get stuck with VFIO in particular, my 
> guess is
> that they're a good starting point (beyond any VFIO+KVM tutorials).
> 
> [1] https://docs.kernel.org/driver-api/vfio.html
> [2] https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html

Hello,

Thank you for your response. Indeed, I have been looking at VFIO since 
it's the first step to achieving such a configuration. However, from 
what I understand, VFIO assists in "passing through" the hardware 
controller/device(s) to a VM.

I do not follow how this fulfills the second part of my desired 
configuration, i.e. allowing other guests to access USB 
functionality/attached devices through a secure API with access control 
mechanisms. I want the guest to be able to assign devices to other 
guests, while maintaining the necessary security posture (since this can 
become an attack vector). I might have missed something though, so I'll 
go back and read again.

Thank you for your time.

Re: [USB Isolation] USB virt drivers access between guests instead of host -> guest?

[privacymiscoccasion]

On 2024-07-22 22:39, privacymiscoccasion@cock.li wrote:
> On 2024-07-22 19:30, Sean Christopherson wrote:
>> On Mon, Jul 22, 2024, privacymiscoccasion@cock.li wrote:
>>> Hi everyone,
>>> 
>>> I'm coming over from reading about Qubes OS, which uses the Xen 
>>> hypervisor.
>>> In Qubes, the way that untrusted devices like USBs are handled is 
>>> that they
>>> are pass through to a VM, which then (I presume) allows other guests 
>>> to
>>> access them using virtual drivers.
>>> 
>>> I'm looking for a theoretical explanation on how this would be 
>>> possible with
>>> KVM. I am not a developer and thus am having difficulty understanding 
>>> how
>>> one would let a guest access virtual drivers connecting to hardware 
>>> devices
>>> like USB and PCIe from another guest.
>>> 
>>> Any help/practical examples of this would be greatly appreciated. 
>>> This seems
>>> to be a hard topic to find and so far I haven't come across anything 
>>> like
>>> this.
>> 
>> In Linux, this would be done via VFIO[1].  VFIO allows assigning 
>> devices to host
>> userspace, and thus to KVM guests.  Very rougly speaking, most assets 
>> that get
>> exposed to KVM guests are proxied through host userspace.  I haven't 
>> actually
>> read the DPDK docs[2], but if you get stuck with VFIO in particular, 
>> my guess is
>> that they're a good starting point (beyond any VFIO+KVM tutorials).
>> 
>> [1] https://docs.kernel.org/driver-api/vfio.html
>> [2] https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html
> 
> Hello,
> 
> Thank you for your response. Indeed, I have been looking at VFIO since
> it's the first step to achieving such a configuration. However, from
> what I understand, VFIO assists in "passing through" the hardware
> controller/device(s) to a VM.
> 
> I do not follow how this fulfills the second part of my desired
> configuration, i.e. allowing other guests to access USB
> functionality/attached devices through a secure API with access
> control mechanisms. I want the guest to be able to assign devices to
> other guests, while maintaining the necessary security posture (since
> this can become an attack vector). I might have missed something
> though, so I'll go back and read again.
> 
> Thank you for your time.

Hi,

As I was reading more about the second part, I came across a few 
interesting projects around this space.

Relevant to USB:
- [Linux Kernel supports 
USB/IP](https://www.kernel.org/doc/html/latest/usb/usbip_protocol.html)
- [Arch wiki tutorial on 
USB/IP](https://wiki.archlinux.org/title/USB/IP)

Relevant to PCIe slicing:
- [vhost-user with DPDK](https://wiki.qemu.org/Features/VirtioVhostUser) 
in QEMU.
- [Redhat guide on Vhost User with 
dpdk](https://www.redhat.com/en/blog/hands-vhost-user-warm-welcome-dpdk)

Relevant to PCIe over IP:
- [A virtio-net EP function to share PCIe devices over 
IP](https://lwn.net/Articles/922124/) - this has the list of relevant 
patches for this feature in the Kernel.

My goal is to have 2 guests, one in charge of distributing access to 
USBs, and the other for access to GPU resources. I will have to read 
more about the support each one has and their usability but this is 
certainly wonderful news.

Thanks!