USB Passthrough 1.1 performance problem...

USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Hi,


I am using a tv card in a VM and get jerky video.As I understand it, the 
VM is using USB 1.1. However, when I set the USB controller in the BIOS 
of my server to Fullspeed (12 Mbit/s) which is the USB 1.1 speed I am 
able to get perfect results on the host but still on the guest the video 
is jerky.

Are there some tuning parameters I can use or perhaps even kernel 
configuration paramters on the host to solve this?

Cheers
   Erik

Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
              OS: Opensuse 11.3 64 bit

Guest:   OS: Opensuse 11.3 64 bit

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Erik Brakkee wrote:
> Hi,
>
>
> I am using a tv card in a VM and get jerky video.As I understand it, 
> the VM is using USB 1.1. However, when I set the USB controller in the 
> BIOS of my server to Fullspeed (12 Mbit/s) which is the USB 1.1 speed 
> I am able to get perfect results on the host but still on the guest 
> the video is jerky.
>
> Are there some tuning parameters I can use or perhaps even kernel 
> configuration paramters on the host to solve this?
>
> Cheers
>   Erik
>
> Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
>              OS: Opensuse 11.3 64 bit
>
> Guest:   OS: Opensuse 11.3 64 bit
I can say now that I am giving up on getting this to work. One 
alternative was to use PCI passthrough the USB hardware,  but that 
didn't work for the USB that was on the motherboard. So I bought a USB 
PCI card and tried to use PCI passthrough for that. Unfortunately other 
problems occured there.

For one, the problem with 4K alignment. But I could fix that by using 
the pci=resource_alignment=... kernel parameter. In my grub/menu.lst it 
says:

    kernel /vmlinuz-2.6.34.7-0.5-default root=/dev/hsystem/root quiet
    showopts intel_iommu=on
    pci=resource_alignment=01:04.0;01:04.1;01:04.2 noirqdebug vga=0x31a


The noirqdebug flas was needed to avoid the host from disabling the IRQ 
(it was a shared IRQ).

Using this, I could configure PCI passthrough and start the VM. Also the 
USB device showed up there. Only it did not work at all.

Here is a summary of my journey up until know:

The original approach I wanted to use was to pass my old PCI card (WinTV 
PVR-500) to a VM. This card is a well supported card and has been doing 
fine for me. Because of the PCI passthrough problems with the wintv 
card, I decided to try a USB card instead. This gave me a 'ctrl buffer 
too small' issue that I could solve by taking the source RPM for kvm and 
applying a known patch from red hat (increasing buffer size from 2048 to 
8192). But then I got jerky video, probably due to USB 1.1 issues. To 
bypass these I could use PCI passthrough for USB. But with the PCI 
passthrough of this card I am again running into issues probably related 
to Shared IRQs. So, after all this I am back to square one.

I have now modified my approach so instead of running a separate minimal 
host with my old server as a guest, I am now running the old server 
(same install) on the new hardware, using it as a host. I would 
definitely be interested in trying this out further in the future. I 
even tried Xen for a brief moment, only to realize that my host and 
guest felt slower (slower startup and execution) and much more difficult 
to handle.

 From the experience of the last two days fulltime trying to get things 
working I can only conclude that the following two features would be 
really important to have:

    * Extended PCI passthrough support
          o shared IRQ support
          o supporting cases where memory is not aligned on a 4K boundary
    * USB passthrough
          o support USB 2.0
          o support USB 3.0 (but taking one step at a time, 2.0 would
            also be great).

 From the above, PCI passthrough improvements are most important as they 
will also allow (cheap) workarounds for USB passthrough issues.

I am definitely interested in hearing about new releases from KVM. Is 
there a mailing list I should subscribe to get information about the 
latest releases without intermediate discussions? I would also be 
interested in using latest stable versions of KVM for opensuse. Is there 
any repository I can configure to get those?

Cheers
   Erik

Re: USB Passthrough 1.1 performance problem...

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 4101 bytes --]

Am 12.12.2010 21:59, Erik Brakkee wrote:
> Erik Brakkee wrote:
>> Hi,
>>
>>
>> I am using a tv card in a VM and get jerky video.As I understand it,
>> the VM is using USB 1.1. However, when I set the USB controller in the
>> BIOS of my server to Fullspeed (12 Mbit/s) which is the USB 1.1 speed
>> I am able to get perfect results on the host but still on the guest
>> the video is jerky.

USB-passthrough at device level implies USB host controller emulation.
This comes with some overhead and will probably never reach native
performance. Virtualization also comes with occasion high latency spikes
(due to host activity delaying time-critical guest jobs). Those issues
are inherent. Moreover, you may run into emulation bugs, but they are
fixable in principle.

>>
>> Are there some tuning parameters I can use or perhaps even kernel
>> configuration paramters on the host to solve this?
>>
>> Cheers
>>   Erik
>>
>> Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
>>              OS: Opensuse 11.3 64 bit
>>
>> Guest:   OS: Opensuse 11.3 64 bit
> I can say now that I am giving up on getting this to work. One
> alternative was to use PCI passthrough the USB hardware,  but that
> didn't work for the USB that was on the motherboard. So I bought a USB
> PCI card and tried to use PCI passthrough for that. Unfortunately other
> problems occured there.
> 
> For one, the problem with 4K alignment. But I could fix that by using
> the pci=resource_alignment=... kernel parameter. In my grub/menu.lst it
> says:
> 
>    kernel /vmlinuz-2.6.34.7-0.5-default root=/dev/hsystem/root quiet
>    showopts intel_iommu=on
>    pci=resource_alignment=01:04.0;01:04.1;01:04.2 noirqdebug vga=0x31a
> 
> 
> The noirqdebug flas was needed to avoid the host from disabling the IRQ
> (it was a shared IRQ).
> 
> Using this, I could configure PCI passthrough and start the VM. Also the
> USB device showed up there. Only it did not work at all.
> 
> Here is a summary of my journey up until know:
> 
> The original approach I wanted to use was to pass my old PCI card (WinTV
> PVR-500) to a VM. This card is a well supported card and has been doing
> fine for me. Because of the PCI passthrough problems with the wintv
> card, I decided to try a USB card instead. This gave me a 'ctrl buffer
> too small' issue that I could solve by taking the source RPM for kvm and
> applying a known patch from red hat (increasing buffer size from 2048 to
> 8192). But then I got jerky video, probably due to USB 1.1 issues. To
> bypass these I could use PCI passthrough for USB. But with the PCI
> passthrough of this card I am again running into issues probably related
> to Shared IRQs. So, after all this I am back to square one.
> 
> I have now modified my approach so instead of running a separate minimal
> host with my old server as a guest, I am now running the old server
> (same install) on the new hardware, using it as a host. I would
> definitely be interested in trying this out further in the future. I
> even tried Xen for a brief moment, only to realize that my host and
> guest felt slower (slower startup and execution) and much more difficult
> to handle.
> 
> From the experience of the last two days fulltime trying to get things
> working I can only conclude that the following two features would be
> really important to have:
> 
>    * Extended PCI passthrough support
>          o shared IRQ support

Addressed by the series I sent out today.

>          o supporting cases where memory is not aligned on a 4K boundary

Hmm, I'm seeing warnings here when passing through one of my EHCIs, but
no fatal errors.

>    * USB passthrough
>          o support USB 2.0
>          o support USB 3.0 (but taking one step at a time, 2.0 would
>            also be great).

Note that this will not solve any real-time issue (if that is part of
your problem). E.g.: While my EHCIs work nicely in PCI-passthrough
scenarios, I'm unable to use certain webcams that sooner or later run
out of sync.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Jan Kiszka wrote:
>
>>> Are there some tuning parameters I can use or perhaps even kernel
>>> configuration paramters on the host to solve this?
>>>
>>> Cheers
>>>    Erik
>>>
>>> Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
>>>               OS: Opensuse 11.3 64 bit
>>>
>>> Guest:   OS: Opensuse 11.3 64 bit
>>>        
>> I can say now that I am giving up on getting this to work. One
>> alternative was to use PCI passthrough the USB hardware,  but that
>> didn't work for the USB that was on the motherboard. So I bought a USB
>> PCI card and tried to use PCI passthrough for that. Unfortunately other
>> problems occured there.
>>
>> For one, the problem with 4K alignment. But I could fix that by using
>> the pci=resource_alignment=... kernel parameter. In my grub/menu.lst it
>> says:
>>
>>     kernel /vmlinuz-2.6.34.7-0.5-default root=/dev/hsystem/root quiet
>>     showopts intel_iommu=on
>>     pci=resource_alignment=01:04.0;01:04.1;01:04.2 noirqdebug vga=0x31a
>>
>>
>> The noirqdebug flas was needed to avoid the host from disabling the IRQ
>> (it was a shared IRQ).
>>
>> Using this, I could configure PCI passthrough and start the VM. Also the
>> USB device showed up there. Only it did not work at all.
>>
>> Here is a summary of my journey up until know:
>>
>> The original approach I wanted to use was to pass my old PCI card (WinTV
>> PVR-500) to a VM. This card is a well supported card and has been doing
>> fine for me. Because of the PCI passthrough problems with the wintv
>> card, I decided to try a USB card instead. This gave me a 'ctrl buffer
>> too small' issue that I could solve by taking the source RPM for kvm and
>> applying a known patch from red hat (increasing buffer size from 2048 to
>> 8192). But then I got jerky video, probably due to USB 1.1 issues. To
>> bypass these I could use PCI passthrough for USB. But with the PCI
>> passthrough of this card I am again running into issues probably related
>> to Shared IRQs. So, after all this I am back to square one.
>>
>> I have now modified my approach so instead of running a separate minimal
>> host with my old server as a guest, I am now running the old server
>> (same install) on the new hardware, using it as a host. I would
>> definitely be interested in trying this out further in the future. I
>> even tried Xen for a brief moment, only to realize that my host and
>> guest felt slower (slower startup and execution) and much more difficult
>> to handle.
>>
>>  From the experience of the last two days fulltime trying to get things
>> working I can only conclude that the following two features would be
>> really important to have:
>>
>>     * Extended PCI passthrough support
>>           o shared IRQ support
>>      
> Addressed by the series I sent out today.
>    
Does this mean I have a chance now that PCI passthrough of my WinTV 
PVR-500 might work now?
What version is this and where can I get this for opensuse?

I still have the setup I used for testing with the host OS still 
installed but not running so it would be really easy to try out new 
releases of KVM (it is not a serious production server after all but 
mainly used to run some websites and mailing lists).

>    
>>           o supporting cases where memory is not aligned on a 4K boundary
>>      
> Hmm, I'm seeing warnings here when passing through one of my EHCIs, but
> no fatal errors.
>    
In my case, the domain just didn't start.
Btw. I was using 0.12.5 on opensuse 11.3 but could only find the sources 
for 0.12.3 on download.opensuse.org (perhaps I looked wrong) and I 
patched those for th 4K issue. PCI passthrough also did not work with my 
wintv pci card with KVM 0.12.5.
>    
>>     * USB passthrough
>>           o support USB 2.0
>>           o support USB 3.0 (but taking one step at a time, 2.0 would
>>             also be great).
>>      
> Note that this will not solve any real-time issue (if that is part of
> your problem). E.g.: While my EHCIs work nicely in PCI-passthrough
> scenarios, I'm unable to use certain webcams that sooner or later run
> out of sync.
>
> Jan
>
>    
Is your point in this case that USB in a VM based on PCI passthrough 
will always have problems when it comes to more real-time issues or does 
this only apply to USB passthrough? I can imagine that PCI passthrough 
is better since it uses hardware support. By the way, I have seen issues 
in the past whereby the tv card stopped working because of high load on 
the server running natively so real-time issues also exist apart from 
virtualization.

Apart from this, I am definitely interested in getting updates for KVM 
on opensuse. Is there a channel for that?

Re: USB Passthrough 1.1 performance problem...

[Alexander Graf]

On 12.12.2010, at 23:31, Erik Brakkee wrote:

> Jan Kiszka wrote:
>> 
>>>> Are there some tuning parameters I can use or perhaps even kernel
>>>> configuration paramters on the host to solve this?
>>>> 
>>>> Cheers
>>>>   Erik
>>>> 
>>>> Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
>>>>              OS: Opensuse 11.3 64 bit
>>>> 
>>>> Guest:   OS: Opensuse 11.3 64 bit
>>>>       
>>> I can say now that I am giving up on getting this to work. One
>>> alternative was to use PCI passthrough the USB hardware,  but that
>>> didn't work for the USB that was on the motherboard. So I bought a USB
>>> PCI card and tried to use PCI passthrough for that. Unfortunately other
>>> problems occured there.
>>> 
>>> For one, the problem with 4K alignment. But I could fix that by using
>>> the pci=resource_alignment=... kernel parameter. In my grub/menu.lst it
>>> says:
>>> 
>>>    kernel /vmlinuz-2.6.34.7-0.5-default root=/dev/hsystem/root quiet
>>>    showopts intel_iommu=on
>>>    pci=resource_alignment=01:04.0;01:04.1;01:04.2 noirqdebug vga=0x31a
>>> 
>>> 
>>> The noirqdebug flas was needed to avoid the host from disabling the IRQ
>>> (it was a shared IRQ).
>>> 
>>> Using this, I could configure PCI passthrough and start the VM. Also the
>>> USB device showed up there. Only it did not work at all.
>>> 
>>> Here is a summary of my journey up until know:
>>> 
>>> The original approach I wanted to use was to pass my old PCI card (WinTV
>>> PVR-500) to a VM. This card is a well supported card and has been doing
>>> fine for me. Because of the PCI passthrough problems with the wintv
>>> card, I decided to try a USB card instead. This gave me a 'ctrl buffer
>>> too small' issue that I could solve by taking the source RPM for kvm and
>>> applying a known patch from red hat (increasing buffer size from 2048 to
>>> 8192). But then I got jerky video, probably due to USB 1.1 issues. To
>>> bypass these I could use PCI passthrough for USB. But with the PCI
>>> passthrough of this card I am again running into issues probably related
>>> to Shared IRQs. So, after all this I am back to square one.
>>> 
>>> I have now modified my approach so instead of running a separate minimal
>>> host with my old server as a guest, I am now running the old server
>>> (same install) on the new hardware, using it as a host. I would
>>> definitely be interested in trying this out further in the future. I
>>> even tried Xen for a brief moment, only to realize that my host and
>>> guest felt slower (slower startup and execution) and much more difficult
>>> to handle.
>>> 
>>> From the experience of the last two days fulltime trying to get things
>>> working I can only conclude that the following two features would be
>>> really important to have:
>>> 
>>>    * Extended PCI passthrough support
>>>          o shared IRQ support
>>>     
>> Addressed by the series I sent out today.
>>   
> Does this mean I have a chance now that PCI passthrough of my WinTV PVR-500 might work now?
> What version is this and where can I get this for opensuse?
> 
> I still have the setup I used for testing with the host OS still installed but not running so it would be really easy to try out new releases of KVM (it is not a serious production server after all but mainly used to run some websites and mailing lists).
> 
>>   
>>>          o supporting cases where memory is not aligned on a 4K boundary
>>>     
>> Hmm, I'm seeing warnings here when passing through one of my EHCIs, but
>> no fatal errors.
>>   
> In my case, the domain just didn't start.
> Btw. I was using 0.12.5 on opensuse 11.3 but could only find the sources for 0.12.3 on download.opensuse.org (perhaps I looked wrong) and I patched those for th 4K issue. PCI passthrough also did not work with my wintv pci card with KVM 0.12.5.

The source rpm for the 11.3 update channel is here:

  http://download.opensuse.org/update/11.3/rpm/src/kvm-0.12.5-1.2.1.src.rpm

>>   
>>>    * USB passthrough
>>>          o support USB 2.0
>>>          o support USB 3.0 (but taking one step at a time, 2.0 would
>>>            also be great).
>>>     
>> Note that this will not solve any real-time issue (if that is part of
>> your problem). E.g.: While my EHCIs work nicely in PCI-passthrough
>> scenarios, I'm unable to use certain webcams that sooner or later run
>> out of sync.
>> 
>> Jan
>> 
>>   
> Is your point in this case that USB in a VM based on PCI passthrough will always have problems when it comes to more real-time issues or does this only apply to USB passthrough? I can imagine that PCI passthrough is better since it uses hardware support. By the way, I have seen issues in the past whereby the tv card stopped working because of high load on the server running natively so real-time issues also exist apart from virtualization.

IIRC the reason that PCI passthrough with EHCI performs as badly as it does is that BARs < 4k get passed through using the slow path (trap to qemu, issue MMIO in user space). Unfortunately, EHCI seems to have a 256 byte BAR region usually that is used for some handshaking:

00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI])
        Subsystem: ATI Technologies Inc SB700/SB800 USB EHCI Controller
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 64 bytes
        Interrupt: pin B routed to IRQ 17
        Region 0: Memory at c8014400 (32-bit, non-prefetchable) [size=256]


> Apart from this, I am definitely interested in getting updates for KVM on opensuse. Is there a channel for that?

The most current development version for the user space part is in the openSUSE build service on the Virtualization repo:

  http://download.opensuse.org/repositories/Virtualization/openSUSE_11.3/

Please be aware that that version is less tested than the one delivered with the distribution :).


Alex

Re: USB Passthrough 1.1 performance problem...

[Gerd Hoffmann]

   Hi,

> I am using a tv card in a VM and get jerky video.As I understand it, the
> VM is using USB 1.1. However, when I set the USB controller in the BIOS
> of my server to Fullspeed (12 Mbit/s) which is the USB 1.1 speed I am
> able to get perfect results on the host but still on the guest the video
> is jerky.

There is a patch series from Hans de Goede on qemu-devel which adds 
buffering for isochronous usb transfers to the usb passthrough code. 
Certainly worth a try.

cheers,
   Gerd

Re: USB Passthrough 1.1 performance problem...

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 3436 bytes --]

Am 12.12.2010 23:31, Erik Brakkee wrote:
> Jan Kiszka wrote:
>>
>>>> Are there some tuning parameters I can use or perhaps even kernel
>>>> configuration paramters on the host to solve this?
>>>>
>>>> Cheers
>>>>    Erik
>>>>
>>>> Host:    Motherboard Supermicro X8DTi-F, Intel Xeon L5630, 12MB
>>>>               OS: Opensuse 11.3 64 bit
>>>>
>>>> Guest:   OS: Opensuse 11.3 64 bit
>>>>        
>>> I can say now that I am giving up on getting this to work. One
>>> alternative was to use PCI passthrough the USB hardware,  but that
>>> didn't work for the USB that was on the motherboard. So I bought a USB
>>> PCI card and tried to use PCI passthrough for that. Unfortunately other
>>> problems occured there.
>>>
>>> For one, the problem with 4K alignment. But I could fix that by using
>>> the pci=resource_alignment=... kernel parameter. In my grub/menu.lst it
>>> says:
>>>
>>>     kernel /vmlinuz-2.6.34.7-0.5-default root=/dev/hsystem/root quiet
>>>     showopts intel_iommu=on
>>>     pci=resource_alignment=01:04.0;01:04.1;01:04.2 noirqdebug vga=0x31a
>>>
>>>
>>> The noirqdebug flas was needed to avoid the host from disabling the IRQ
>>> (it was a shared IRQ).
>>>
>>> Using this, I could configure PCI passthrough and start the VM. Also the
>>> USB device showed up there. Only it did not work at all.
>>>
>>> Here is a summary of my journey up until know:
>>>
>>> The original approach I wanted to use was to pass my old PCI card (WinTV
>>> PVR-500) to a VM. This card is a well supported card and has been doing
>>> fine for me. Because of the PCI passthrough problems with the wintv
>>> card, I decided to try a USB card instead. This gave me a 'ctrl buffer
>>> too small' issue that I could solve by taking the source RPM for kvm and
>>> applying a known patch from red hat (increasing buffer size from 2048 to
>>> 8192). But then I got jerky video, probably due to USB 1.1 issues. To
>>> bypass these I could use PCI passthrough for USB. But with the PCI
>>> passthrough of this card I am again running into issues probably related
>>> to Shared IRQs. So, after all this I am back to square one.
>>>
>>> I have now modified my approach so instead of running a separate minimal
>>> host with my old server as a guest, I am now running the old server
>>> (same install) on the new hardware, using it as a host. I would
>>> definitely be interested in trying this out further in the future. I
>>> even tried Xen for a brief moment, only to realize that my host and
>>> guest felt slower (slower startup and execution) and much more difficult
>>> to handle.
>>>
>>>  From the experience of the last two days fulltime trying to get things
>>> working I can only conclude that the following two features would be
>>> really important to have:
>>>
>>>     * Extended PCI passthrough support
>>>           o shared IRQ support
>>>      
>> Addressed by the series I sent out today.
>>    
> Does this mean I have a chance now that PCI passthrough of my WinTV
> PVR-500 might work now?
> What version is this and where can I get this for opensuse?

Currently you have to clone my git trees [1, 2], then build and install
those to have the feature. Will take a while to see it in releases, and
after that also Opensuse packages.

Jan

[1] git://git.kiszka.org/linux-kvm.git queues/dev-assign
[2] git://git.kiszka.org/qemu-kvm.git queues/dev-assign


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: USB Passthrough 1.1 performance problem...

[Kenni Lund]

2010/12/12 Erik Brakkee <erik@brakkee.org>:
> Does this mean I have a chance now that PCI passthrough of my WinTV PVR-500
> might work now?

Passthrough of a PVR-500 has been working for a long time. I've been
running with passthrough of a PVR-500 in my HTPC, since
November/December 2009...so it should work with any recent kernel and
any recent version of qemu-kvm you can find today - No patching
needed. The only issue I had with the PVR-500 card, was when *I*
didn't free up the shared interrupts...once I fixed that, it "just
worked".

On the other hand, I've never had success with passthrough of USB.
I've spend a bunch of time trying to get various USB cards to work
with passthrough, I even purchased 3 USB cards, just to test USB
passthrough with different brands, interfaces and versions (PCI, PCIe,
USB 2.0, USB 3.0, etc). I gave up on that 5 months ago -
http://article.gmane.org/gmane.comp.emulators.kvm.devel/56719

> What version is this and where can I get this for opensuse?

I can't remember if I started out with the PVR-500 card with 0.11 or
0.12 ...I think it was 0.11...but anyway, you'll hopefully not run
with such an old version today, so any version should work.

Best regards
Kenni

Re: USB Passthrough 1.1 performance problem...

[Kenni Lund]

2010/12/14 Erik Brakkee <erik@brakkee.org>:
>> From: Kenni Lund <kenni@kelu.dk>
>>> Does this mean I have a chance now that PCI passthrough of my WinTV
>>> PVR-500
>>> might work now?
>>
>> Passthrough of a PVR-500 has been working for a long time. I've been
>> running with passthrough of a PVR-500 in my HTPC, since
>> November/December 2009...so it should work with any recent kernel and
>> any recent version of qemu-kvm you can find today - No patching
>> needed. The only issue I had with the PVR-500 card, was when *I*
>> didn't free up the shared interrupts...once I fixed that, it "just
>> worked".
>
> How did you free up those shared interrupts then? I tried different slots
> but always get conflicts with the USB irqs.

I did an unbind of the conflicting device (eg. disabled it). I moved
the PVR-500 card around in the different slots and once I got a
conflict with the integrated sound card, I left the PVR-500 card in
that slot (it's a headless machine, so no need for sound) and
configured unbind of the sound card at boot time. On my old system I
think it was conflicting with one of the USB controllers as well, but
it didn't really matter, as I only lost a few of the ports on the back
of the computer for that particular USB controller - I still had
plenty of USB ports left and if I really needed more ports, I could
just plug in an extra USB PCI card.

My /etc/rc.local boot script looks like the following today:
--
#Remove HDA conflicting with ivtv1
echo "0000:00:1b.0" > /sys/bus/pci/drivers/HDA\ Intel/unbind

# ivtv0
echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:04:08.0" > /sys/bus/pci/drivers/ivtv/unbind
echo "0000:04:08.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id

# ivtv1
echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:04:09.0" > /sys/bus/pci/drivers/ivtv/unbind
echo "0000:04:09.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id
--

Best regards
Kenni

Re: USB Passthrough 1.1 performance problem...

[Avi Kivity]

On 12/13/2010 10:25 AM, Alexander Graf wrote:
> >>
> >  Is your point in this case that USB in a VM based on PCI passthrough will always have problems when it comes to more real-time issues or does this only apply to USB passthrough? I can imagine that PCI passthrough is better since it uses hardware support. By the way, I have seen issues in the past whereby the tv card stopped working because of high load on the server running natively so real-time issues also exist apart from virtualization.
>
> IIRC the reason that PCI passthrough with EHCI performs as badly as it does is that BARs<  4k get passed through using the slow path (trap to qemu, issue MMIO in user space). Unfortunately, EHCI seems to have a 256 byte BAR region usually that is used for some handshaking:
>
> 00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI])
>          Subsystem: ATI Technologies Inc SB700/SB800 USB EHCI Controller
>          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
>          Latency: 64, Cache Line Size: 64 bytes
>          Interrupt: pin B routed to IRQ 17
>          Region 0: Memory at c8014400 (32-bit, non-prefetchable) [size=256]
>

That could certainly be optimized.  If the BAR is all along in its page, 
both on guest and host (if not, we can migrate it, at least on the 
host), we can use the same offset within the page on the host as it 
appears on the guest, and assign the entire page.

We should make sure SeaBIOS uses a minimum alignment of 4k for mmio BARs.

-- 
error compiling committee.c: too many arguments to function

Re: USB Passthrough 1.1 performance problem...

[Alexander Graf]

Am 14.12.2010 um 11:02 schrieb Avi Kivity <avi@redhat.com>:

> On 12/13/2010 10:25 AM, Alexander Graf wrote:
>> >>
>> >  Is your point in this case that USB in a VM based on PCI passthrough will always have problems when it comes to more real-time issues or does this only apply to USB passthrough? I can imagine that PCI passthrough is better since it uses hardware support. By the way, I have seen issues in the past whereby the tv card stopped working because of high load on the server running natively so real-time issues also exist apart from virtualization.
>> 
>> IIRC the reason that PCI passthrough with EHCI performs as badly as it does is that BARs<  4k get passed through using the slow path (trap to qemu, issue MMIO in user space). Unfortunately, EHCI seems to have a 256 byte BAR region usually that is used for some handshaking:
>> 
>> 00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller (prog-if 20 [EHCI])
>>         Subsystem: ATI Technologies Inc SB700/SB800 USB EHCI Controller
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
>>         Latency: 64, Cache Line Size: 64 bytes
>>         Interrupt: pin B routed to IRQ 17
>>         Region 0: Memory at c8014400 (32-bit, non-prefetchable) [size=256]
>> 
> 
> That could certainly be optimized.  If the BAR is all along in its page, both on guest and host (if not, we can migrate it, at least on the host), we can use the same offset within the page on the host as it appears on the guest, and assign the entire page.
> 
> We should make sure SeaBIOS uses a minimum alignment of 4k for mmio BARs.

Yep, I agree :). Back when I tried that, it seemed rather hard to change BAR mappings after init from user space. But it's certainly a thing the vfio stuff could easily tackle!

Alex

Re: USB Passthrough 1.1 performance problem...

[Kenni Lund]

2010/12/14 Erik Brakkee <erik@brakkee.org>:
>> From: Kenni Lund <kenni@kelu.dk>
>> 2010/12/14 Erik Brakkee <erik@brakkee.org>:
>>>>
>>>> From: Kenni Lund <kenni@kelu.dk>
>>>>>
>>>>> Does this mean I have a chance now that PCI passthrough of my WinTV
>>>>> PVR-500
>>>>> might work now?
>>>>
>>>> Passthrough of a PVR-500 has been working for a long time. I've been
>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>> November/December 2009...so it should work with any recent kernel and
>>>> any recent version of qemu-kvm you can find today - No patching
>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>> didn't free up the shared interrupts...once I fixed that, it "just
>>>> worked".
>>>
>>> How did you free up those shared interrupts then? I tried different slots
>>> but always get conflicts with the USB irqs.
>>
>> I did an unbind of the conflicting device (eg. disabled it). I moved
>> the PVR-500 card around in the different slots and once I got a
>> conflict with the integrated sound card, I left the PVR-500 card in
>> that slot (it's a headless machine, so no need for sound) and
>> configured unbind of the sound card at boot time. On my old system I
>> think it was conflicting with one of the USB controllers as well, but
>> it didn't really matter, as I only lost a few of the ports on the back
>> of the computer for that particular USB controller - I still had
>> plenty of USB ports left and if I really needed more ports, I could
>> just plug in an extra USB PCI card.
>>
>> My /etc/rc.local boot script looks like the following today:
>> --
>> #Remove HDA conflicting with ivtv1
>> echo "0000:00:1b.0" > /sys/bus/pci/drivers/HDA\ Intel/unbind
>>
>> # ivtv0
>> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
>> echo "0000:04:08.0" > /sys/bus/pci/drivers/ivtv/unbind
>> echo "0000:04:08.0" > /sys/bus/pci/drivers/pci-stub/bind
>> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id
>>
>> # ivtv1
>> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
>> echo "0000:04:09.0" > /sys/bus/pci/drivers/ivtv/unbind
>> echo "0000:04:09.0" > /sys/bus/pci/drivers/pci-stub/bind
>> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id
>
> I did not try unbinding the usb device so I can also try that.
>
> I don'.t understand what is happening with the 4444 0016. I configured the
> pci card in kvm and I believe kvm does the binding to pci-stub in recent
> versions. Where is the 4444 0016%oming from?

Okay, qemu-kvm might do it today, I don't know - I haven't changed
that script for the past year. But are you sure that it's not
libvirt/virsh/virt-manager which does that for you?

Anyway, it's coming from lspci -n. See the wiki page:
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

I can't remember why I run remove_id in the end, it's probably
unneeded, but I can't remember (and it works, so I don't toch it).

Best regards
Kenni

Re: USB Passthrough 1.1 performance problem...

[Daniel P. Berrange]

On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
> 2010/12/14 Erik Brakkee <erik@brakkee.org>:
> >> From: Kenni Lund <kenni@kelu.dk>
> >> 2010/12/14 Erik Brakkee <erik@brakkee.org>:
> >>>>
> >>>> From: Kenni Lund <kenni@kelu.dk>
> >>>>>
> >>>>> Does this mean I have a chance now that PCI passthrough of my WinTV
> >>>>> PVR-500
> >>>>> might work now?
> >>>>
> >>>> Passthrough of a PVR-500 has been working for a long time. I've been
> >>>> running with passthrough of a PVR-500 in my HTPC, since
> >>>> November/December 2009...so it should work with any recent kernel and
> >>>> any recent version of qemu-kvm you can find today - No patching
> >>>> needed. The only issue I had with the PVR-500 card, was when *I*
> >>>> didn't free up the shared interrupts...once I fixed that, it "just
> >>>> worked".
> >>>
> >>> How did you free up those shared interrupts then? I tried different slots
> >>> but always get conflicts with the USB irqs.
> >>
> >> I did an unbind of the conflicting device (eg. disabled it). I moved
> >> the PVR-500 card around in the different slots and once I got a
> >> conflict with the integrated sound card, I left the PVR-500 card in
> >> that slot (it's a headless machine, so no need for sound) and
> >> configured unbind of the sound card at boot time. On my old system I
> >> think it was conflicting with one of the USB controllers as well, but
> >> it didn't really matter, as I only lost a few of the ports on the back
> >> of the computer for that particular USB controller - I still had
> >> plenty of USB ports left and if I really needed more ports, I could
> >> just plug in an extra USB PCI card.
> >>
> >> My /etc/rc.local boot script looks like the following today:
> >> --
> >> #Remove HDA conflicting with ivtv1
> >> echo "0000:00:1b.0" > /sys/bus/pci/drivers/HDA\ Intel/unbind
> >>
> >> # ivtv0
> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
> >> echo "0000:04:08.0" > /sys/bus/pci/drivers/ivtv/unbind
> >> echo "0000:04:08.0" > /sys/bus/pci/drivers/pci-stub/bind
> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id
> >>
> >> # ivtv1
> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id
> >> echo "0000:04:09.0" > /sys/bus/pci/drivers/ivtv/unbind
> >> echo "0000:04:09.0" > /sys/bus/pci/drivers/pci-stub/bind
> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id
> >
> > I did not try unbinding the usb device so I can also try that.
> >
> > I don'.t understand what is happening with the 4444 0016. I configured the
> > pci card in kvm and I believe kvm does the binding to pci-stub in recent
> > versions. Where is the 4444 0016%oming from?
> 
> Okay, qemu-kvm might do it today, I don't know - I haven't changed
> that script for the past year. But are you sure that it's not
> libvirt/virsh/virt-manager which does that for you?

If you use the managed="yes" attribute on the <hostdev> in libvirt
XML, then libvirt will automatically do the pcistub bind/unbind,
followed by a device reset at guest startup & the reverse at shutdown.
If you have conflicting devices on the bus though, libvirt won't
attempt to unbind them, unless you had also explicitly assigned all
those conflicting devices to the same guest.

Daniel

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Daniel P. Berrange wrote:
> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
>    
>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>      
>>>> From: Kenni Lund<kenni@kelu.dk>
>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>          
>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>              
>>>>>>> Does this mean I have a chance now that PCI passthrough of my WinTV
>>>>>>> PVR-500
>>>>>>> might work now?
>>>>>>>                
>>>>>> Passthrough of a PVR-500 has been working for a long time. I've been
>>>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>>>> November/December 2009...so it should work with any recent kernel and
>>>>>> any recent version of qemu-kvm you can find today - No patching
>>>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>>>> didn't free up the shared interrupts...once I fixed that, it "just
>>>>>> worked".
>>>>>>              
>>>>> How did you free up those shared interrupts then? I tried different slots
>>>>> but always get conflicts with the USB irqs.
>>>>>            
>>>> I did an unbind of the conflicting device (eg. disabled it). I moved
>>>> the PVR-500 card around in the different slots and once I got a
>>>> conflict with the integrated sound card, I left the PVR-500 card in
>>>> that slot (it's a headless machine, so no need for sound) and
>>>> configured unbind of the sound card at boot time. On my old system I
>>>> think it was conflicting with one of the USB controllers as well, but
>>>> it didn't really matter, as I only lost a few of the ports on the back
>>>> of the computer for that particular USB controller - I still had
>>>> plenty of USB ports left and if I really needed more ports, I could
>>>> just plug in an extra USB PCI card.
>>>>
>>>> My /etc/rc.local boot script looks like the following today:
>>>> --
>>>> #Remove HDA conflicting with ivtv1
>>>> echo "0000:00:1b.0">  /sys/bus/pci/drivers/HDA\ Intel/unbind
>>>>
>>>> # ivtv0
>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>
>>>> # ivtv1
>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>          
>>> I did not try unbinding the usb device so I can also try that.
>>>
>>> I don'.t understand what is happening with the 4444 0016. I configured the
>>> pci card in kvm and I believe kvm does the binding to pci-stub in recent
>>> versions. Where is the 4444 0016%oming from?
>>>        
>> Okay, qemu-kvm might do it today, I don't know - I haven't changed
>> that script for the past year. But are you sure that it's not
>> libvirt/virsh/virt-manager which does that for you?
>>      
> If you use the managed="yes" attribute on the<hostdev>  in libvirt
> XML, then libvirt will automatically do the pcistub bind/unbind,
> followed by a device reset at guest startup&  the reverse at shutdown.
> If you have conflicting devices on the bus though, libvirt won't
> attempt to unbind them, unless you had also explicitly assigned all
> those conflicting devices to the same guest.
>
> Daniel
>    
I definitely have to try again (right now having some stability problems 
on the server that I am debugging).

The shared IRQs are as follows:

   16:          0          0          0          0          0          
0          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
   18:     252995          0          0          0          0          
0          0          0   IO-APIC-fasteoi   ehci_hcd:usb1, 
uhci_hcd:usb8, ivtv0
   19:      58281          0          0          0          0          
0          0          0   IO-APIC-fasteoi   ata_piix, ata_piix, 
uhci_hcd:usb5, uhci_hcd:usb7, ivtv1
   21:          0          0          0          0          0          
0          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
   23:        713       6906          0      76919          0          
0          0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6

So I have IRQ sharing with usb1, usb8, usb5, usb7. I have also read that 
ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how 
would I now identify the USB PCI devices that I would need to unbind to 
get rid of the sharing with the USB ports? It also doesn't really matter 
in which slot I put the PVR-500 card because both cards share IRQs with 
USB in all cases.

I have also used an add on USB PCI card but still got these conflicts. I 
was considering to get a PCIe USB card instead to try out in the hope 
that that would use different IRQs. Is that a realistic expectation? 
That way, I could disable all on-board USB (in the BIOS even) and use 
the add-on USB only.

Re: USB Passthrough 1.1 performance problem...

[Kenni Lund]

2010/12/14 Erik Brakkee <erik@brakkee.org>:
> Daniel P. Berrange wrote:
>>
>> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
>>
>>>
>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>
>>>>>
>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>
>>>>>>>
>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>>
>>>>>>>>
>>>>>>>> Does this mean I have a chance now that PCI passthrough of my WinTV
>>>>>>>> PVR-500
>>>>>>>> might work now?
>>>>>>>>
>>>>>>>
>>>>>>> Passthrough of a PVR-500 has been working for a long time. I've been
>>>>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>>>>> November/December 2009...so it should work with any recent kernel and
>>>>>>> any recent version of qemu-kvm you can find today - No patching
>>>>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>>>>> didn't free up the shared interrupts...once I fixed that, it "just
>>>>>>> worked".
>>>>>>>
>>>>>>
>>>>>> How did you free up those shared interrupts then? I tried different
>>>>>> slots
>>>>>> but always get conflicts with the USB irqs.
>>>>>>
>>>>>
>>>>> I did an unbind of the conflicting device (eg. disabled it). I moved
>>>>> the PVR-500 card around in the different slots and once I got a
>>>>> conflict with the integrated sound card, I left the PVR-500 card in
>>>>> that slot (it's a headless machine, so no need for sound) and
>>>>> configured unbind of the sound card at boot time. On my old system I
>>>>> think it was conflicting with one of the USB controllers as well, but
>>>>> it didn't really matter, as I only lost a few of the ports on the back
>>>>> of the computer for that particular USB controller - I still had
>>>>> plenty of USB ports left and if I really needed more ports, I could
>>>>> just plug in an extra USB PCI card.
>>>>>
>>>>> My /etc/rc.local boot script looks like the following today:
>>>>> --
>>>>> #Remove HDA conflicting with ivtv1
>>>>> echo "0000:00:1b.0">  /sys/bus/pci/drivers/HDA\ Intel/unbind
>>>>>
>>>>> # ivtv0
>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>
>>>>> # ivtv1
>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>
>>>>
>>>> I did not try unbinding the usb device so I can also try that.
>>>>
>>>> I don'.t understand what is happening with the 4444 0016. I configured
>>>> the
>>>> pci card in kvm and I believe kvm does the binding to pci-stub in recent
>>>> versions. Where is the 4444 0016%oming from?
>>>>
>>>
>>> Okay, qemu-kvm might do it today, I don't know - I haven't changed
>>> that script for the past year. But are you sure that it's not
>>> libvirt/virsh/virt-manager which does that for you?
>>>
>>
>> If you use the managed="yes" attribute on the<hostdev>  in libvirt
>> XML, then libvirt will automatically do the pcistub bind/unbind,
>> followed by a device reset at guest startup&  the reverse at shutdown.
>> If you have conflicting devices on the bus though, libvirt won't
>> attempt to unbind them, unless you had also explicitly assigned all
>> those conflicting devices to the same guest.
>>
>> Daniel
>>
>
> I definitely have to try again (right now having some stability problems on
> the server that I am debugging).
>
> The shared IRQs are as follows:
>
>  16:          0          0          0          0          0          0
>    0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>  18:     252995          0          0          0          0          0
>    0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb8, ivtv0
>  19:      58281          0          0          0          0          0
>    0          0   IO-APIC-fasteoi   ata_piix, ata_piix, uhci_hcd:usb5,
> uhci_hcd:usb7, ivtv1
>  21:          0          0          0          0          0          0
>    0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>  23:        713       6906          0      76919          0          0
>    0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6
>
> So I have IRQ sharing with usb1, usb8, usb5, usb7.

Uff....and your ata HDD controller. I guess i was much luckier than
you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted
with USB.

> I have also read that
> ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how
> would I now identify the USB PCI devices that I would need to unbind to get
> rid of the sharing with the USB ports?

Play around with:
lspci -v
lspci -n
lsusb -v
lsusb -t

You can also just start by unbinding the first one and take note when
you hit the right ones...once you unbind one, it will disappear from
cat /proc/interrupts. When you're down to having only ivtv0 on one
interrupt and only ivtv1 on another interrupt, then you're ready to
bind with pci-stub and boot your guest.

> It also doesn't really matter in
> which slot I put the PVR-500 card because both cards share IRQs with USB in
> all cases.

You also have your conflicting ata controller to take into
consideration. I don't remember if "ata_piix" and "ata_piix" is IDE
only, if it is, you might not even use it. Otherwise it might be
easier for you to run qemu-kvm git with the new patches for shared
interrupts...it really depends on your setup.

> I have also used an add on USB PCI card but still got these conflicts. I was
> considering to get a PCIe USB card instead to try out in the hope that that
> would use different IRQs. Is that a realistic expectation? That way, I could
> disable all on-board USB (in the BIOS even) and use the add-on USB only.

Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got
its own unique IRQs which wasn't shared with anything.

Best regards
Kenni

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

<quote who="Kenni Lund">
> 2010/12/14 Erik Brakkee <erik@brakkee.org>:
>> Daniel P. Berrange wrote:
>>>
>>> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
>>>
>>>>
>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>
>>>>>>
>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>>
>>>>>>>>
>>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Does this mean I have a chance now that PCI passthrough of my
>>>>>>>>> WinTV
>>>>>>>>> PVR-500
>>>>>>>>> might work now?
>>>>>>>>>
>>>>>>>>
>>>>>>>> Passthrough of a PVR-500 has been working for a long time. I've
>>>>>>>> been
>>>>>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>>>>>> November/December 2009...so it should work with any recent kernel
>>>>>>>> and
>>>>>>>> any recent version of qemu-kvm you can find today - No patching
>>>>>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>>>>>> didn't free up the shared interrupts...once I fixed that, it "just
>>>>>>>> worked".
>>>>>>>>
>>>>>>>
>>>>>>> How did you free up those shared interrupts then? I tried different
>>>>>>> slots
>>>>>>> but always get conflicts with the USB irqs.
>>>>>>>
>>>>>>
>>>>>> I did an unbind of the conflicting device (eg. disabled it). I moved
>>>>>> the PVR-500 card around in the different slots and once I got a
>>>>>> conflict with the integrated sound card, I left the PVR-500 card in
>>>>>> that slot (it's a headless machine, so no need for sound) and
>>>>>> configured unbind of the sound card at boot time. On my old system I
>>>>>> think it was conflicting with one of the USB controllers as well,
>>>>>> but
>>>>>> it didn't really matter, as I only lost a few of the ports on the
>>>>>> back
>>>>>> of the computer for that particular USB controller - I still had
>>>>>> plenty of USB ports left and if I really needed more ports, I could
>>>>>> just plug in an extra USB PCI card.
>>>>>>
>>>>>> My /etc/rc.local boot script looks like the following today:
>>>>>> --
>>>>>> #Remove HDA conflicting with ivtv1
>>>>>> echo "0000:00:1b.0">  /sys/bus/pci/drivers/HDA\ Intel/unbind
>>>>>>
>>>>>> # ivtv0
>>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>>>> echo "0000:04:08.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>
>>>>>> # ivtv1
>>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/new_id
>>>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/ivtv/unbind
>>>>>> echo "0000:04:09.0">  /sys/bus/pci/drivers/pci-stub/bind
>>>>>> echo "4444 0016">  /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>
>>>>>
>>>>> I did not try unbinding the usb device so I can also try that.
>>>>>
>>>>> I don'.t understand what is happening with the 4444 0016. I
>>>>> configured
>>>>> the
>>>>> pci card in kvm and I believe kvm does the binding to pci-stub in
>>>>> recent
>>>>> versions. Where is the 4444 0016%oming from?
>>>>>
>>>>
>>>> Okay, qemu-kvm might do it today, I don't know - I haven't changed
>>>> that script for the past year. But are you sure that it's not
>>>> libvirt/virsh/virt-manager which does that for you?
>>>>
>>>
>>> If you use the managed="yes" attribute on the<hostdev>  in libvirt
>>> XML, then libvirt will automatically do the pcistub bind/unbind,
>>> followed by a device reset at guest startup&  the reverse at shutdown.
>>> If you have conflicting devices on the bus though, libvirt won't
>>> attempt to unbind them, unless you had also explicitly assigned all
>>> those conflicting devices to the same guest.
>>>
>>> Daniel
>>>
>>
>> I definitely have to try again (right now having some stability problems
>> on
>> the server that I am debugging).
>>
>> The shared IRQs are as follows:
>>
>>  16:          0          0          0          0          0          0
>>    0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>>  18:     252995          0          0          0          0          0
>>    0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb8, ivtv0
>>  19:      58281          0          0          0          0          0
>>    0          0   IO-APIC-fasteoi   ata_piix, ata_piix, uhci_hcd:usb5,
>> uhci_hcd:usb7, ivtv1
>>  21:          0          0          0          0          0          0
>>    0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>>  23:        713       6906          0      76919          0          0
>>    0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6
>>
>> So I have IRQ sharing with usb1, usb8, usb5, usb7.
>
> Uff....and your ata HDD controller. I guess i was much luckier than
> you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted
> with USB.
>
>> I have also read that
>> ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how
>> would I now identify the USB PCI devices that I would need to unbind to
>> get
>> rid of the sharing with the USB ports?
>
> Play around with:
> lspci -v
> lspci -n
> lsusb -v
> lsusb -t
>
> You can also just start by unbinding the first one and take note when
> you hit the right ones...once you unbind one, it will disappear from
> cat /proc/interrupts. When you're down to having only ivtv0 on one
> interrupt and only ivtv1 on another interrupt, then you're ready to
> bind with pci-stub and boot your guest.
>
>> It also doesn't really matter in
>> which slot I put the PVR-500 card because both cards share IRQs with USB
>> in
>> all cases.
>
> You also have your conflicting ata controller to take into
> consideration. I don't remember if "ata_piix" and "ata_piix" is IDE
> only, if it is, you might not even use it. Otherwise it might be
> easier for you to run qemu-kvm git with the new patches for shared
> interrupts...it really depends on your setup.
>
>> I have also used an add on USB PCI card but still got these conflicts. I
>> was
>> considering to get a PCIe USB card instead to try out in the hope that
>> that
>> would use different IRQs. Is that a realistic expectation? That way, I
>> could
>> disable all on-board USB (in the BIOS even) and use the add-on USB only.
>
> Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got
> its own unique IRQs which wasn't shared with anything.
>

I think I am a bit out of luck here. The two IRQs that are interesting are
18 and 19. One of them has the SMBUS and the other one has SATA drives. I
think the best for me is to wait until shared IRQ support in KVM is
available. My only option is to try it out every once in a while.

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Erik Brakkee wrote:
> <quote who="Kenni Lund">
>    
>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>      
>>> Daniel P. Berrange wrote:
>>>        
>>>> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
>>>>
>>>>          
>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>
>>>>>            
>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>>>
>>>>>>>                
>>>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>>>> Does this mean I have a chance now that PCI passthrough of my
>>>>>>>>>> WinTV
>>>>>>>>>> PVR-500
>>>>>>>>>> might work now?
>>>>>>>>>>
>>>>>>>>>>                      
>>>>>>>>> Passthrough of a PVR-500 has been working for a long time. I've
>>>>>>>>> been
>>>>>>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>>>>>>> November/December 2009...so it should work with any recent kernel
>>>>>>>>> and
>>>>>>>>> any recent version of qemu-kvm you can find today - No patching
>>>>>>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>>>>>>> didn't free up the shared interrupts...once I fixed that, it "just
>>>>>>>>> worked".
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>> How did you free up those shared interrupts then? I tried different
>>>>>>>> slots
>>>>>>>> but always get conflicts with the USB irqs.
>>>>>>>>
>>>>>>>>                  
>>>>>>> I did an unbind of the conflicting device (eg. disabled it). I moved
>>>>>>> the PVR-500 card around in the different slots and once I got a
>>>>>>> conflict with the integrated sound card, I left the PVR-500 card in
>>>>>>> that slot (it's a headless machine, so no need for sound) and
>>>>>>> configured unbind of the sound card at boot time. On my old system I
>>>>>>> think it was conflicting with one of the USB controllers as well,
>>>>>>> but
>>>>>>> it didn't really matter, as I only lost a few of the ports on the
>>>>>>> back
>>>>>>> of the computer for that particular USB controller - I still had
>>>>>>> plenty of USB ports left and if I really needed more ports, I could
>>>>>>> just plug in an extra USB PCI card.
>>>>>>>
>>>>>>> My /etc/rc.local boot script looks like the following today:
>>>>>>> --
>>>>>>> #Remove HDA conflicting with ivtv1
>>>>>>> echo "0000:00:1b.0">   /sys/bus/pci/drivers/HDA\ Intel/unbind
>>>>>>>
>>>>>>> # ivtv0
>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
>>>>>>> echo "0000:04:08.0">   /sys/bus/pci/drivers/ivtv/unbind
>>>>>>> echo "0000:04:08.0">   /sys/bus/pci/drivers/pci-stub/bind
>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>>
>>>>>>> # ivtv1
>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
>>>>>>> echo "0000:04:09.0">   /sys/bus/pci/drivers/ivtv/unbind
>>>>>>> echo "0000:04:09.0">   /sys/bus/pci/drivers/pci-stub/bind
>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>>
>>>>>>>                
>>>>>> I did not try unbinding the usb device so I can also try that.
>>>>>>
>>>>>> I don'.t understand what is happening with the 4444 0016. I
>>>>>> configured
>>>>>> the
>>>>>> pci card in kvm and I believe kvm does the binding to pci-stub in
>>>>>> recent
>>>>>> versions. Where is the 4444 0016%oming from?
>>>>>>
>>>>>>              
>>>>> Okay, qemu-kvm might do it today, I don't know - I haven't changed
>>>>> that script for the past year. But are you sure that it's not
>>>>> libvirt/virsh/virt-manager which does that for you?
>>>>>
>>>>>            
>>>> If you use the managed="yes" attribute on the<hostdev>   in libvirt
>>>> XML, then libvirt will automatically do the pcistub bind/unbind,
>>>> followed by a device reset at guest startup&   the reverse at shutdown.
>>>> If you have conflicting devices on the bus though, libvirt won't
>>>> attempt to unbind them, unless you had also explicitly assigned all
>>>> those conflicting devices to the same guest.
>>>>
>>>> Daniel
>>>>
>>>>          
>>> I definitely have to try again (right now having some stability problems
>>> on
>>> the server that I am debugging).
>>>
>>> The shared IRQs are as follows:
>>>
>>>   16:          0          0          0          0          0          0
>>>     0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>>>   18:     252995          0          0          0          0          0
>>>     0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb8, ivtv0
>>>   19:      58281          0          0          0          0          0
>>>     0          0   IO-APIC-fasteoi   ata_piix, ata_piix, uhci_hcd:usb5,
>>> uhci_hcd:usb7, ivtv1
>>>   21:          0          0          0          0          0          0
>>>     0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>>>   23:        713       6906          0      76919          0          0
>>>     0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6
>>>
>>> So I have IRQ sharing with usb1, usb8, usb5, usb7.
>>>        
>> Uff....and your ata HDD controller. I guess i was much luckier than
>> you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted
>> with USB.
>>
>>      
>>> I have also read that
>>> ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, how
>>> would I now identify the USB PCI devices that I would need to unbind to
>>> get
>>> rid of the sharing with the USB ports?
>>>        
>> Play around with:
>> lspci -v
>> lspci -n
>> lsusb -v
>> lsusb -t
>>
>> You can also just start by unbinding the first one and take note when
>> you hit the right ones...once you unbind one, it will disappear from
>> cat /proc/interrupts. When you're down to having only ivtv0 on one
>> interrupt and only ivtv1 on another interrupt, then you're ready to
>> bind with pci-stub and boot your guest.
>>
>>      
>>> It also doesn't really matter in
>>> which slot I put the PVR-500 card because both cards share IRQs with USB
>>> in
>>> all cases.
>>>        
>> You also have your conflicting ata controller to take into
>> consideration. I don't remember if "ata_piix" and "ata_piix" is IDE
>> only, if it is, you might not even use it. Otherwise it might be
>> easier for you to run qemu-kvm git with the new patches for shared
>> interrupts...it really depends on your setup.
>>
>>      
>>> I have also used an add on USB PCI card but still got these conflicts. I
>>> was
>>> considering to get a PCIe USB card instead to try out in the hope that
>>> that
>>> would use different IRQs. Is that a realistic expectation? That way, I
>>> could
>>> disable all on-board USB (in the BIOS even) and use the add-on USB only.
>>>        
>> Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got
>> its own unique IRQs which wasn't shared with anything.
>>
>>      
> I think I am a bit out of luck here. The two IRQs that are interesting are
> 18 and 19. One of them has the SMBUS and the other one has SATA drives. I
> think the best for me is to wait until shared IRQ support in KVM is
> available. My only option is to try it out every once in a while.
>
>
>
>    
Meanwhile, I haven't given up and experimented with determining the 
effect of unbinding the various USB PCI devices. I haven't tried 
virtualization yet though. The problem is I managed to free up one tuner 
(ivtv0) from shared IRQs, all USB ports are still operational, but then 
how would I determine which USB ports still support USB 2.0?

I have now at least found a way to make ivtv0 available without any 
shared IRQ by unbinding two USB PCI devices: one UHCI and one EHCI device.

The list of USB PCI devices is:

00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 
EHCI Controller #2
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 
EHCI Controller #1

I unbound  00:1a.7 and 00:1d.2. To my great surprise, all USB ports on 
the server are still operational. I tried this by inserting a USB memory 
stick and trying to mount it. The motherboard has a total of 8 USB 
connections with 6 USB in use, I don't really understand this. Was I 
simply in luck?

The output of lsucb after doing the unbinding is:

Bus 005 Device 003: ID 046b:ff10 American Megatrends, Inc.
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 002: ID 045e:001e Microsoft Corp. IntelliMouse Explorer
Bus 006 Device 003: ID 051d:0002 American Power Conversion 
Uninterruptible Power Supply
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

Alternatively, I am also considering to by a new PCIe card. In 
particular, the ASUS U3S6 looks interesting because it has 2 PCIe ports 
and 2 USB 3.0 ports. This means that I could disable 2 more USB devices 
(00:1a.2 and 00:1d.1) and get some extra USB devices. The ata_piix 
drivers are being used for an internal disk and CDROM. The RAID array is 
managed through arcmsr (Areca) so I don't have a problem there. Of 
course, if I managed to get completely free of shared IRQs with this 
add-on card, then the issue of determining which of the remaining ports 
is still USB 2.0 is not that important anymore.

So, enough ideas to try out.

Re: USB Passthrough 1.1 performance problem...

[Erik Brakkee]

Erik Brakkee wrote:
> Erik Brakkee wrote:
>> <quote who="Kenni Lund">
>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>> Daniel P. Berrange wrote:
>>>>> On Tue, Dec 14, 2010 at 12:55:04PM +0100, Kenni Lund wrote:
>>>>>
>>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>>
>>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>>> 2010/12/14 Erik Brakkee<erik@brakkee.org>:
>>>>>>>>
>>>>>>>>>> From: Kenni Lund<kenni@kelu.dk>
>>>>>>>>>>
>>>>>>>>>>> Does this mean I have a chance now that PCI passthrough of my
>>>>>>>>>>> WinTV
>>>>>>>>>>> PVR-500
>>>>>>>>>>> might work now?
>>>>>>>>>>>
>>>>>>>>>> Passthrough of a PVR-500 has been working for a long time. I've
>>>>>>>>>> been
>>>>>>>>>> running with passthrough of a PVR-500 in my HTPC, since
>>>>>>>>>> November/December 2009...so it should work with any recent 
>>>>>>>>>> kernel
>>>>>>>>>> and
>>>>>>>>>> any recent version of qemu-kvm you can find today - No patching
>>>>>>>>>> needed. The only issue I had with the PVR-500 card, was when *I*
>>>>>>>>>> didn't free up the shared interrupts...once I fixed that, it 
>>>>>>>>>> "just
>>>>>>>>>> worked".
>>>>>>>>>>
>>>>>>>>> How did you free up those shared interrupts then? I tried 
>>>>>>>>> different
>>>>>>>>> slots
>>>>>>>>> but always get conflicts with the USB irqs.
>>>>>>>>>
>>>>>>>> I did an unbind of the conflicting device (eg. disabled it). I 
>>>>>>>> moved
>>>>>>>> the PVR-500 card around in the different slots and once I got a
>>>>>>>> conflict with the integrated sound card, I left the PVR-500 
>>>>>>>> card in
>>>>>>>> that slot (it's a headless machine, so no need for sound) and
>>>>>>>> configured unbind of the sound card at boot time. On my old 
>>>>>>>> system I
>>>>>>>> think it was conflicting with one of the USB controllers as well,
>>>>>>>> but
>>>>>>>> it didn't really matter, as I only lost a few of the ports on the
>>>>>>>> back
>>>>>>>> of the computer for that particular USB controller - I still had
>>>>>>>> plenty of USB ports left and if I really needed more ports, I 
>>>>>>>> could
>>>>>>>> just plug in an extra USB PCI card.
>>>>>>>>
>>>>>>>> My /etc/rc.local boot script looks like the following today:
>>>>>>>> -- 
>>>>>>>> #Remove HDA conflicting with ivtv1
>>>>>>>> echo "0000:00:1b.0">   /sys/bus/pci/drivers/HDA\ Intel/unbind
>>>>>>>>
>>>>>>>> # ivtv0
>>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
>>>>>>>> echo "0000:04:08.0">   /sys/bus/pci/drivers/ivtv/unbind
>>>>>>>> echo "0000:04:08.0">   /sys/bus/pci/drivers/pci-stub/bind
>>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>>>
>>>>>>>> # ivtv1
>>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/new_id
>>>>>>>> echo "0000:04:09.0">   /sys/bus/pci/drivers/ivtv/unbind
>>>>>>>> echo "0000:04:09.0">   /sys/bus/pci/drivers/pci-stub/bind
>>>>>>>> echo "4444 0016">   /sys/bus/pci/drivers/pci-stub/remove_id
>>>>>>>>
>>>>>>> I did not try unbinding the usb device so I can also try that.
>>>>>>>
>>>>>>> I don'.t understand what is happening with the 4444 0016. I
>>>>>>> configured
>>>>>>> the
>>>>>>> pci card in kvm and I believe kvm does the binding to pci-stub in
>>>>>>> recent
>>>>>>> versions. Where is the 4444 0016%oming from?
>>>>>>>
>>>>>> Okay, qemu-kvm might do it today, I don't know - I haven't changed
>>>>>> that script for the past year. But are you sure that it's not
>>>>>> libvirt/virsh/virt-manager which does that for you?
>>>>>>
>>>>> If you use the managed="yes" attribute on the<hostdev>   in libvirt
>>>>> XML, then libvirt will automatically do the pcistub bind/unbind,
>>>>> followed by a device reset at guest startup&   the reverse at 
>>>>> shutdown.
>>>>> If you have conflicting devices on the bus though, libvirt won't
>>>>> attempt to unbind them, unless you had also explicitly assigned all
>>>>> those conflicting devices to the same guest.
>>>>>
>>>>> Daniel
>>>>>
>>>> I definitely have to try again (right now having some stability 
>>>> problems
>>>> on
>>>> the server that I am debugging).
>>>>
>>>> The shared IRQs are as follows:
>>>>
>>>>   16:          0          0          0          0          
>>>> 0          0
>>>>     0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>>>>   18:     252995          0          0          0          
>>>> 0          0
>>>>     0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb8, 
>>>> ivtv0
>>>>   19:      58281          0          0          0          
>>>> 0          0
>>>>     0          0   IO-APIC-fasteoi   ata_piix, ata_piix, 
>>>> uhci_hcd:usb5,
>>>> uhci_hcd:usb7, ivtv1
>>>>   21:          0          0          0          0          
>>>> 0          0
>>>>     0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>>>>   23:        713       6906          0      76919          
>>>> 0          0
>>>>     0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb6
>>>>
>>>> So I have IRQ sharing with usb1, usb8, usb5, usb7.
>>> Uff....and your ata HDD controller. I guess i was much luckier than
>>> you are, my ivtv0 didn't conflict at all and ivtv1 only conflicted
>>> with USB.
>>>
>>>> I have also read that
>>>> ehci refers to USB 2.0 and uhci to USB 1.1 is that correct? Anyway, 
>>>> how
>>>> would I now identify the USB PCI devices that I would need to 
>>>> unbind to
>>>> get
>>>> rid of the sharing with the USB ports?
>>> Play around with:
>>> lspci -v
>>> lspci -n
>>> lsusb -v
>>> lsusb -t
>>>
>>> You can also just start by unbinding the first one and take note when
>>> you hit the right ones...once you unbind one, it will disappear from
>>> cat /proc/interrupts. When you're down to having only ivtv0 on one
>>> interrupt and only ivtv1 on another interrupt, then you're ready to
>>> bind with pci-stub and boot your guest.
>>>
>>>> It also doesn't really matter in
>>>> which slot I put the PVR-500 card because both cards share IRQs 
>>>> with USB
>>>> in
>>>> all cases.
>>> You also have your conflicting ata controller to take into
>>> consideration. I don't remember if "ata_piix" and "ata_piix" is IDE
>>> only, if it is, you might not even use it. Otherwise it might be
>>> easier for you to run qemu-kvm git with the new patches for shared
>>> interrupts...it really depends on your setup.
>>>
>>>> I have also used an add on USB PCI card but still got these 
>>>> conflicts. I
>>>> was
>>>> considering to get a PCIe USB card instead to try out in the hope that
>>>> that
>>>> would use different IRQs. Is that a realistic expectation? That way, I
>>>> could
>>>> disable all on-board USB (in the BIOS even) and use the add-on USB 
>>>> only.
>>> Most likely, the PCIe USB 3.0 card I bought supported MSI-X, so it got
>>> its own unique IRQs which wasn't shared with anything.
>>>
>> I think I am a bit out of luck here. The two IRQs that are 
>> interesting are
>> 18 and 19. One of them has the SMBUS and the other one has SATA 
>> drives. I
>> think the best for me is to wait until shared IRQ support in KVM is
>> available. My only option is to try it out every once in a while.
>>
>>
>>
> Meanwhile, I haven't given up and experimented with determining the 
> effect of unbinding the various USB PCI devices. I haven't tried 
> virtualization yet though. The problem is I managed to free up one 
> tuner (ivtv0) from shared IRQs, all USB ports are still operational, 
> but then how would I determine which USB ports still support USB 2.0?
>
> I have now at least found a way to make ivtv0 available without any 
> shared IRQ by unbinding two USB PCI devices: one UHCI and one EHCI 
> device.
>
> The list of USB PCI devices is:
>
> 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #4
> 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #5
> 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #6
> 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 
> EHCI Controller #2
> 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #1
> 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #2
> 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB 
> UHCI Controller #3
> 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 
> EHCI Controller #1
>
> I unbound  00:1a.7 and 00:1d.2. To my great surprise, all USB ports on 
> the server are still operational. I tried this by inserting a USB 
> memory stick and trying to mount it. The motherboard has a total of 8 
> USB connections with 6 USB in use, I don't really understand this. Was 
> I simply in luck?
>
> The output of lsucb after doing the unbinding is:
>
> Bus 005 Device 003: ID 046b:ff10 American Megatrends, Inc.
> Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
> Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
> Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
> Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
> Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
> Bus 006 Device 002: ID 045e:001e Microsoft Corp. IntelliMouse Explorer
> Bus 006 Device 003: ID 051d:0002 American Power Conversion 
> Uninterruptible Power Supply
> Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
>
> Alternatively, I am also considering to by a new PCIe card. In 
> particular, the ASUS U3S6 looks interesting because it has 2 PCIe 
> ports and 2 USB 3.0 ports. This means that I could disable 2 more USB 
> devices (00:1a.2 and 00:1d.1) and get some extra USB devices. The 
> ata_piix drivers are being used for an internal disk and CDROM. The 
> RAID array is managed through arcmsr (Areca) so I don't have a problem 
> there. Of course, if I managed to get completely free of shared IRQs 
> with this add-on card, then the issue of determining which of the 
> remaining ports is still USB 2.0 is not that important anymore.
>
> So, enough ideas to try out.
>
In the mean while I figured it out completely. I disabled the 4 USB PCI 
devices as mentioned above which leads to one USB port becoming unusable 
(acceptable). Then I could configure the SATA 'handling' in the BIOS. 
The default was plain SATA, but I could also select Intel RAID, Adaptec 
RAID, and AHCI. In any case Intel RAID and AHCI solved the problem with 
the ata_piix shared IRQ (did not check), so I used the most modern one 
(Intel RAID). (motherboard Supermicro X8DTi-F).

Meanwhile I did a successful PCI passthrough test recording two channels 
simultanesously and it worked!

PS. I used to do backups to a backup SATA disk over USB but since I 
turned on the Intel RAID, the SATA ports on the motherboard became hot 
swappable so I can now backup over (e)SATA which gives me a 5 times 
speedup in bandwidth and frees up one USB port, compensating for the one 
I lost.

So all is well and I can continue my virtualized setup according to my 
original plans.

Cheers
   Erik

Re: USB Passthrough 1.1 performance problem...

[Kevin O'Connor]

On Tue, Dec 14, 2010 at 12:02:03PM +0200, Avi Kivity wrote:
> That could certainly be optimized.  If the BAR is all along in its
> page, both on guest and host (if not, we can migrate it, at least on
> the host), we can use the same offset within the page on the host as
> it appears on the guest, and assign the entire page.
> 
> We should make sure SeaBIOS uses a minimum alignment of 4k for mmio BARs.

Shouldn't be too hard to do that.

-Kevin


--- a/src/pciinit.c
+++ b/src/pciinit.c
@@ -10,6 +10,7 @@
 #include "biosvar.h" // GET_EBDA
 #include "pci_ids.h" // PCI_VENDOR_ID_INTEL
 #include "pci_regs.h" // PCI_COMMAND
+#include "memmap.h" // PAGE_SIZE
 #include "dev-i440fx.h"
 
 #define PCI_ROM_SLOT 6
@@ -90,6 +91,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
             type = "prefmem";
             msg = "decrease BUILD_PCIMEM_SIZE and recompile. size %x";
         } else {
+            if (size < PAGE_SIZE)
+                size = PAGE_SIZE;
             r = &pci_bios_mem_region;
             type = "mem";
             msg = "increase BUILD_PCIMEM_SIZE and recompile.";