Fritz!Card, MSIs and virtual machines

Fritz!Card, MSIs and virtual machines

[Binarus]

Dear all,

we are running a fairly complex fax system in a KVM guest.

The KVM host is Ubuntu 11.04 (64 bit), the guest is Windows 2003 R2 server.

The fax system needs ISDN hardware, so we have tried to passthrough a PCI ISDN card (AVM Fritz!card PCI) to the guest. We were partly successful:

The guest has recognized the hardware which we have passed through and runs like a charm, including the fax system.

But for passing through the PCI ISDN card, we had to deactivate two USB devices on the mainboard, i.e. we had to deactivate their normal linux drivers and handle the devices over to the PCI stub driver. This is a thing we surely don't want, so we are searching for a solution which makes this unnecessary.

Background:

The ISDN card is sharing it's interrupt with these two USB devices. As the KVM website states: PCI passthrough is only supported for devices which support MSIs IF these devices share an interrupt with other devices; this restriction does not apply for PCI-E devices. Needless to say that the Fritz!card is a PCI device and obviously does not support Message Signaled Interrupts.

So there are a few questions (if any of them is answered yes, our problem is solved):

- Does anybody know of a PCI-E ISDN card (hoping that our fax software will support it)?
- Does anybody know of a PCI ISDN card which supports MSIs (hoping that it's compatible to our fax system)?
- The KVM host (Ubuntu 11.04) currently runs a (customized) kernel 2.6.37. In the meantime, there are newer kernels. Is there a kernel which the issue mentioned above (PCI passthrough of devices with shared interrupts only if devices support MSIs) does not apply to? Please forgive that we're asking instead of testing, but this is a production server which we really are not allowed to play around with, and we currently don't get hands on a box with identical hardware.
- Is it possible to run a guest with a mixture of I/O virtualization technologies, i.e. run the most part of the guest by using KVM with VT-d and VT-x, but "emulate" the PCI device without using these technologies? I think that will not be possible, but it's worth asking since performance will not be the problem: some time ago, we have run the same guest under XEN 3.x (which had no problem with the PCI-ISDN passthrough) without VT-d or such with sufficient performance (although the hardware was slow compared to the current server).

Thank you very much for any ideas,

Peter

Re: Fritz!Card, MSIs and virtual machines

[Andreas Färber]

Am 07.01.2012 18:52, schrieb Binarus:
> - Is it possible to run a guest with a mixture of I/O virtualization technologies, i.e. run the most part of the guest by using KVM with VT-d and VT-x, but "emulate" the PCI device without using these technologies?

Yes, graphics card, network card, etc. are emulated by default.

I'm just not aware of anyone having written such an emulation for QEMU.
Meaning, you would need to find a card with sufficient documentation
and/or Open Source drivers for implementing and to figure out how to
pass ISDN through to your host from your implementation.

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: Fritz!Card, MSIs and virtual machines

[Jan Kiszka]

On 2012-01-07 18:52, Binarus wrote:
> Dear all,
> 
> we are running a fairly complex fax system in a KVM guest.
> 
> The KVM host is Ubuntu 11.04 (64 bit), the guest is Windows 2003 R2 server.
> 
> The fax system needs ISDN hardware, so we have tried to passthrough a PCI ISDN card (AVM Fritz!card PCI) to the guest. We were partly successful:
> 
> The guest has recognized the hardware which we have passed through and runs like a charm, including the fax system.
> 
> But for passing through the PCI ISDN card, we had to deactivate two USB devices on the mainboard, i.e. we had to deactivate their normal linux drivers and handle the devices over to the PCI stub driver. This is a thing we surely don't want, so we are searching for a solution which makes this unnecessary.
> 
> Background:
> 
> The ISDN card is sharing it's interrupt with these two USB devices. As the KVM website states: PCI passthrough is only supported for devices which support MSIs IF these devices share an interrupt with other devices; this restriction does not apply for PCI-E devices. Needless to say that the Fritz!card is a PCI device and obviously does not support Message Signaled Interrupts.

There are patches in the queue to enable legacy interrupt sharing for
PCI 2.3 compliant devices at least. But I bet the Fritz hardware
predates even this (just like my Fritz!Card DSL v2).

> 
> So there are a few questions (if any of them is answered yes, our problem is solved):
> 
> - Does anybody know of a PCI-E ISDN card (hoping that our fax software will support it)?
> - Does anybody know of a PCI ISDN card which supports MSIs (hoping that it's compatible to our fax system)?
> - The KVM host (Ubuntu 11.04) currently runs a (customized) kernel 2.6.37. In the meantime, there are newer kernels. Is there a kernel which the issue mentioned above (PCI passthrough of devices with shared interrupts only if devices support MSIs) does not apply to? Please forgive that we're asking instead of testing, but this is a production server which we really are not allowed to play around with, and we currently don't get hands on a box with identical hardware.
> - Is it possible to run a guest with a mixture of I/O virtualization technologies, i.e. run the most part of the guest by using KVM with VT-d and VT-x, but "emulate" the PCI device without using these technologies? I think that will not be possible, but it's worth asking since performance will not be the problem: some time ago, we have run the same guest under XEN 3.x (which had no problem with the PCI-ISDN passthrough) without VT-d or such with sufficient performance (although the hardware was slow compared to the current server).

Emulating an ISDN adapter could be fairly complicated, specifically if
the interface the binary Windows driver expects is not well documented.
Even for Linux-on-Linux, i.e. when you can debug both host and guest
properly, this will be quite some work (just check the driver code of
some popular ISDN adapters Linux supports).

And it may turn out to be timing sensitive, e.g. for fax services.
Emulation will happen in user space, i.e. in still fairly
undeterministic QEMU hands, while device path through mostly happens in
kernel space under KVM control.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: Fritz!Card, MSIs and virtual machines

[Binarus]

> There are patches in the queue to enable legacy interrupt sharing for
> PCI 2.3 compliant devices at least. But I bet the Fritz hardware
> predates even this (just like my Fritz!Card DSL v2).

Thank you very much for this hint. Do you know when these will be out?

Does anybody know about a PCI ISDN card which is PCI 2.3 compliant?

> Emulating an ISDN adapter could be fairly complicated, specifically if
> the interface the binary Windows driver expects is not well documented.
> Even for Linux-on-Linux, i.e. when you can debug both host and guest
> properly, this will be quite some work (just check the driver code of
> some popular ISDN adapters Linux supports).
> 
> And it may turn out to be timing sensitive, e.g. for fax services.
> Emulation will happen in user space, i.e. in still fairly
> undeterministic QEMU hands, while device path through mostly happens in
> kernel space under KVM control.

I also think that it's not a good idea to solve the problem that way.

Perhaps there is such a thing like a USB ISDN adapter. We then could pass the USB port which this adapter is attached to to the virtual machine. But even if I am able to find such a device, I doubt that our fax system supports it. Nevertheless, there are chances that it works.

Regards,

Peter

P.S. Thanks for bothering with our exotic problem!

Re: Fritz!Card, MSIs and virtual machines

[Binarus]

On 08.01.2012 17:51, Andreas Färber wrote:
> Am 07.01.2012 18:52, schrieb Binarus:
>> - Is it possible to run a guest with a mixture of I/O virtualization 
>> technologies, i.e. run the most part of the guest by using KVM with VT-d 
>> and VT-x, but "emulate" the PCI device without using these technologies?
> 
> Yes, graphics card, network card, etc. are emulated by default.
> 
> I'm just not aware of anyone having written such an emulation for QEMU.
> Meaning, you would need to find a card with sufficient documentation
> and/or Open Source drivers for implementing and to figure out how to
> pass ISDN through to your host from your implementation.

Many thanks for answering.

As Jan has pointed out in his reply, ISDN is very timing sensitive, so an emulation (which is in user space) might not work at all. Furthermore, we don't have the resources to write such an emulation, and I doubt that somebody else will do it.

Fax via VOIP still does not work reliably, so there must be many people who need ISDN in virtualized guests. I am wondering how other people are dealing with that problem.

Regards,

Peter

Re: Fritz!Card, MSIs and virtual machines

[rm]

Binarus <lists <at> binarus.de> writes:
> Perhaps there is such a thing like a USB ISDN adapter. We then could pass the
USB port which this adapter is
> attached to to the virtual machine. But even if I am able to find such a
device, I doubt that our fax system
> supports it. Nevertheless, there are chances that it works.
> 
> Regards,
> 
> Peter
> 
> P.S. Thanks for bothering with our exotic problem!

Hello, even this is a few Months old.

I got a similar issue but i didnt even tried to use a passtrough PCI device.
My instinct told me that this might be a long frustating night and it looks like
i was right :)

Anyway we got a ISDN USB Modem also a Fritz which can be passtrough to the Guest
but guess what - the guest instantly turn off - complete segfault not even a BSOD

Its also a win 2003 and has already the Fritz drivers (its a p2v)

So i came to 2 possible solutions both iam not very shure about.

1. buy a ISDN Serial modem and passtrough the serial port - which should be fine.
2. get somewho the usb box running on the host - map it to an tty and run it
from there.

My concernc specially for method 2 is that isdn riginig in problem on ttys 
but i think its worth a try


just to find out if its posisble to get the usb box running on the host :)



anyway serial ISDN modems shoul dbe fine also for your application
serial is serial - so it shoul dbe supported by any fax solution in that
universe and even in some others

in germany thers only one manufacturer i know of producing that stuff - its
devolo having such a piece but heavy pricy - cheapest is 130€ up to 200€ 
compared to pci cards for 20 bucks a bit oveprices but hey if it works :)


anyway - well usb and pci passtrough support in KVM is not so much supported as
vmware supports it - i know it sucks a bit but actually it does make sense.

you put the server in a 19" rack to protect it form heat, cleaning personell and
even lightning, you buy a usv for several hundreds to protect it against
powersprice and failure and then you put in a usb or pci modem - one lightning
into your PABS (phonesystem) and your well protected server isnt worth the
metall its made off

from my point of view - networkcards and raidcontroller are the only pci(e)
devices in a server - usb only a keyboard, a mouse and in emergencys a usb drive
managed by the amdin himself - for eveything else theres a network device
(phone, fax, coffeemachine, whatever) 


of course doenst help us by our problem but at least there are some real
arguments against pheriphere devices in a server