Xen 4.2.0-rc4 issues with DVB tuner

Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

[-- Attachment #1.1: Type: text/plain, Size: 1794 bytes --]

On Thu, Aug 30, 2012 at 6:33 PM, Konrad Rzeszutek Wilk <konrad@kernel.org>wrote:

Hi,


> > I've just upgraded a server of mine from a Core i3 2100T to an i7 3770,
> in order
> > to do full virtualization with VTd.
> >
> > I'm using kernel 3.5.2 and Xen from git://xenbits.xen.org/xen.git @
> commit
> > 37d7ccdc2f50d659f1eb8ec11ee4bf8a8376926d (Fri Aug 24).
> >
> > Since there are various issues I'm gonna comment on them all. I'd
> appreciate
> > if you help me deciding which bug reports to file, and where to file
> them.
>
> Its easier if there are seperate emails and then we can track them
> step-by-step.
> >
> > Upon booting under the xen virtualizer everything works fine but I cannot
> > suspend the machine and I have reception problems on the DVB-T tuners
>
> Right. The suspend (well, the resume part) is not yet working.
> > installed on the system.
>
> That sounds familiar - but without more details its a bit unclear.
>

I replaced the problematic (due to BIOS bugs) GigaByte motherboard with
an ASRock Z77M-Pro4 and the difference is awesome. All the issues I had
with the IOMMU, ACPI and ASPM are gone.

I still have to pass the intel_iommu=igfx_off kernel parameter or I continue
to see lots of DMAR errors for the first couple of seconds after booting
and I can´t use the DVB tuners either.

This last point is a showstopper for me and I don´t know where the
problem might come from. The cx23885 module (which my tuners use)
loads fine, as do the devices under /dev/dvb. Upon tuning a channel,
however, there is not data received.

Does anyone have the slightest idea what might be causing this?

I wonder whether it would work within a DomU with the PCIe tuner
passed through.


-- 
Javier Marcet <jmarcet@gmail.com>

[-- Attachment #1.2: Type: text/html, Size: 2547 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Konrad Rzeszutek Wilk]

> I replaced the problematic (due to BIOS bugs) GigaByte motherboard with
> an ASRock Z77M-Pro4 and the difference is awesome. All the issues I had
> with the IOMMU, ACPI and ASPM are gone.
>
> I still have to pass the intel_iommu=igfx_off kernel parameter or I continue
> to see lots of DMAR errors for the first couple of seconds after booting
> and I can´t use the DVB tuners either.
>
> This last point is a showstopper for me and I don´t know where the
> problem might come from. The cx23885 module (which my tuners use)
> loads fine, as do the devices under /dev/dvb. Upon tuning a channel,
> however, there is not data received.
>
> Does anyone have the slightest idea what might be causing this?

Are there any warnings or such being printed? Are the interrupts increasing?
Does lspci show anything 'disabled' in the guest?
>
> I wonder whether it would work within a DomU with the PCIe tuner
> passed through.

I've a 4 channel security card passed in and it works nicely. The
trick is that you need 'iommu=soft' on the domU command line.
>
>
> --
> Javier Marcet <jmarcet@gmail.com>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Fri, Sep 14, 2012 at 10:47 PM, Konrad Rzeszutek Wilk
<konrad@kernel.org> wrote:

>> This last point is a showstopper for me and I don´t know where the
>> problem might come from. The cx23885 module (which my tuners use)
>> loads fine, as do the devices under /dev/dvb. Upon tuning a channel,
>> however, there is not data received.
>>
>> Does anyone have the slightest idea what might be causing this?
>
> Are there any warnings or such being printed? Are the interrupts increasing?
> Does lspci show anything 'disabled' in the guest?

You have misunderstood me. I was trying to access the dvb tuners from the dom0.

Using kvm I have sucessfully passthrough a sata controller, but have some issues
passin through the dvb tuner.

>> I wonder whether it would work within a DomU with the PCIe tuner
>> passed through.
>
> I've a 4 channel security card passed in and it works nicely. The
> trick is that you need 'iommu=soft' on the domU command line.

All right. But is it normal that it does not work under the dom0?
The same kernel running on bare metal has no problems.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Konrad Rzeszutek Wilk]

On Sun, Sep 16, 2012 at 04:57:16PM +0200, Javier Marcet wrote:
> On Fri, Sep 14, 2012 at 10:47 PM, Konrad Rzeszutek Wilk
> <konrad@kernel.org> wrote:
> 
> >> This last point is a showstopper for me and I don´t know where the
> >> problem might come from. The cx23885 module (which my tuners use)
> >> loads fine, as do the devices under /dev/dvb. Upon tuning a channel,
> >> however, there is not data received.
> >>
> >> Does anyone have the slightest idea what might be causing this?
> >
> > Are there any warnings or such being printed? Are the interrupts increasing?
> > Does lspci show anything 'disabled' in the guest?
> 
> You have misunderstood me. I was trying to access the dvb tuners from the dom0.

Ah.
> 
> Using kvm I have sucessfully passthrough a sata controller, but have some issues
> passin through the dvb tuner.
> 
> >> I wonder whether it would work within a DomU with the PCIe tuner
> >> passed through.
> >
> > I've a 4 channel security card passed in and it works nicely. The
> > trick is that you need 'iommu=soft' on the domU command line.
> 
> All right. But is it normal that it does not work under the dom0?

No. PV domU and PV dom0 are pretty much the same in the way of handling
interrupts, ports, etc.

If you crank up the debug level of the kernel (debug loglevel=8) and
of the driver do you get anything obvious? What about the questions
I've asked?

> The same kernel running on bare metal has no problems.
> 
> 
> -- 
> Javier Marcet <jmarcet@gmail.com>
> 

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Mon, Sep 17, 2012 at 4:28 PM, Konrad Rzeszutek Wilk
<konrad@kernel.org> wrote:

>> >> This last point is a showstopper for me and I don´t know where the
>> >> problem might come from. The cx23885 module (which my tuners use)
>> >> loads fine, as do the devices under /dev/dvb. Upon tuning a channel,
>> >> however, there is not data received.
>> >>
>> >> Does anyone have the slightest idea what might be causing this?
>> >
>> > Are there any warnings or such being printed? Are the interrupts increasing?
>> > Does lspci show anything 'disabled' in the guest?
>>
>> You have misunderstood me. I was trying to access the dvb tuners from the dom0.
>
> Ah.
>>
>> Using kvm I have sucessfully passthrough a sata controller, but have some issues
>> passin through the dvb tuner.
>>
>> >> I wonder whether it would work within a DomU with the PCIe tuner
>> >> passed through.
>> >
>> > I've a 4 channel security card passed in and it works nicely. The
>> > trick is that you need 'iommu=soft' on the domU command line.
>>
>> All right. But is it normal that it does not work under the dom0?
>
> No. PV domU and PV dom0 are pretty much the same in the way of handling
> interrupts, ports, etc.

Just as I guessed.

> If you crank up the debug level of the kernel (debug loglevel=8) and
> of the driver do you get anything obvious? What about the questions
> I've asked?

So far I haven't found anything worth mentioning.

All pci devices are enabled, interrupts look normal and there isn't any
kind of warning or error message. The only thing you notice is that
the tuner produces nearly no data. I say nearly because you get some
bytes, albeit nothing like the several megabits per second it should output.

I'll try increasing the debug level and report back.

By the way, that problem I had about the cpu capabilities is not Xen's fault,
it is a bug within libvirt.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Mon, Sep 17, 2012 at 4:28 PM, Konrad Rzeszutek Wilk
<konrad@kernel.org> wrote:

>> >> I wonder whether it would work within a DomU with the PCIe tuner
>> >> passed through.
>> >
>> > I've a 4 channel security card passed in and it works nicely. The
>> > trick is that you need 'iommu=soft' on the domU command line.
>>
>> All right. But is it normal that it does not work under the dom0?
>
> No. PV domU and PV dom0 are pretty much the same in the way of handling
> interrupts, ports, etc.
>
> If you crank up the debug level of the kernel (debug loglevel=8) and
> of the driver do you get anything obvious? What about the questions
> I've asked?

Booting with 'debug loglevel=8' as kernel parameters I still get no errors or
warnings anywhere.

I'm out of ideas and I don't have any other PCIe tuner card to try.

It is looking more and more as a cx23885 (the module the card uses) issue.
It is really odd since I you get some data, but just like if there was very
poor reception, more noise than signal.

If you can think of some other way to debug this, I'll try it.


-- 
Javier Marcet <jmarcet@gmail.com>

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Mon, Sep 17, 2012 at 10:09 PM, Javier Marcet <jmarcet@gmail.com> wrote:

>>> >> I wonder whether it would work within a DomU with the PCIe tuner
>>> >> passed through.
>>> >
>>> > I've a 4 channel security card passed in and it works nicely. The
>>> > trick is that you need 'iommu=soft' on the domU command line.
>>>
>>> All right. But is it normal that it does not work under the dom0?
>>
>> No. PV domU and PV dom0 are pretty much the same in the way of handling
>> interrupts, ports, etc.
>>
>> If you crank up the debug level of the kernel (debug loglevel=8) and
>> of the driver do you get anything obvious? What about the questions
>> I've asked?
>
> Booting with 'debug loglevel=8' as kernel parameters I still get no errors or
> warnings anywhere.
>
> I'm out of ideas and I don't have any other PCIe tuner card to try.
>
> It is looking more and more as a cx23885 (the module the card uses) issue.
> It is really odd since I you get some data, but just like if there was very
> poor reception, more noise than signal.
>
> If you can think of some other way to debug this, I'll try it.

I've just realized that I have quite a lot of debug options in the dvb
driver. Right
now I have one tuner in use, but when it finishes (or tomorrow) I'll enable some
debug output in the dvb subsystem and the cx23885 module in particular and
see whether I can get some more info.

Even so, if you deem worth checking something else, I'm open to try anything.


-- 
Javier Marcet <jmarcet@gmail.com>

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Mon, Sep 17, 2012 at 10:56 PM, Javier Marcet <jmarcet@gmail.com> wrote:

Still nothing. Not even enabling the debug options within the DVB
subsystem I get
anything which doesn't look perfectly normal, exactly like when it's
working fine.

I have just posted a message on the linux media mailing list looking
for some help
from the dvb maintainers.


-- 
Javier Marcet <jmarcet@gmail.com>

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 2:08 AM, Javier Marcet <jmarcet@gmail.com> wrote:

Hi,

> Still nothing. Not even enabling the debug options within the DVB
> subsystem I get
> anything which doesn't look perfectly normal, exactly like when it's
> working fine.
>
> I have just posted a message on the linux media mailing list looking
> for some help
> from the dvb maintainers.

I have some news from the linux-media mailing list. There, Devin Heitmueller,
said this:

"""
> Initially I thought Xen would be the cause of the problem, but after
> having written on
> the Xen development mailing list and talked about it with a couple
> developers, it isn't
> very clear where the problem is. So far I haven't been able to get the
> smallest warning
> or error.

This is a very common problem when attempting to use any PCI/PCIe
tuner under a hypervisor.  Essentially the issue is all of the
virtualization solutions provide very poor interrupt latency, which
results in the data being lost.

Devices delivering a high bitrate stream of data in realtime are much
more likely for this problem to be visible since such devices have
very little buffering (it's not like a hard drive controller where it
can just deliver the data slower).  The problem is not specific to the
cx23885 - pretty much all of the PCI/PCIe bridges used in tuner cards
work this way, and they cannot really be blamed for expecting to run
in an environment with really crappy interrupt latency.

I won't go as far as to say, "abandon all hope", but you're not really
likely to find any help in this forum.
"""

What I don´t understand is how graphics pass through even works
and a tuner card has problems due to the introduced latencies in the
IRQs.

Do you have any more info about this?


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Keir Fraser]

On 18/09/2012 09:41, "Javier Marcet" <jmarcet@gmail.com> wrote:

> What I don´t understand is how graphics pass through even works
and a tuner
> card has problems due to the introduced latencies in the
IRQs.

Do you have
> any more info about this?

In this respect sound is harder than graphics. Try pinning your dom0 to a
CPU core, and set affinities on other domains so that dom0 does not compete
for the core with any other domains.

 -- Keir

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 10:57 AM, Keir Fraser <keir.xen@gmail.com> wrote:

>> What I don´t understand is how graphics pass through even works
> and a tuner
>> card has problems due to the introduced latencies in the
> IRQs.
>
> Do you have
>> any more info about this?
>
> In this respect sound is harder than graphics. Try pinning your dom0 to a
> CPU core, and set affinities on other domains so that dom0 does not compete
> for the core with any other domains.

All right, I can try that. However, the problem happens without any
domU running, just
the dom0. Do you still think that pinning to a cpu core or setting
affinities can help?


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Sander Eikelenboom]

Tuesday, September 18, 2012, 12:52:01 PM, you wrote:

> On Tue, Sep 18, 2012 at 10:57 AM, Keir Fraser <keir.xen@gmail.com> wrote:

>>> What I don´t understand is how graphics pass through even works
>> and a tuner
>>> card has problems due to the introduced latencies in the
>> IRQs.
>>
>> Do you have
>>> any more info about this?
>>
>> In this respect sound is harder than graphics. Try pinning your dom0 to a
>> CPU core, and set affinities on other domains so that dom0 does not compete
>> for the core with any other domains.

> All right, I can try that. However, the problem happens without any
> domU running, just
> the dom0. Do you still think that pinning to a cpu core or setting
> affinities can help?

I haven't followed the thread closely, but are you sure the device works OK on baremetal (linux without xen) ?
If you have problems in dom0 it seems it hasn't got much to do with passthrough ?

--
Sander


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 4:33 PM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:

>>>> What I don´t understand is how graphics pass through even works
>>> and a tuner
>>>> card has problems due to the introduced latencies in the
>>> IRQs.
>>>
>>> Do you have
>>>> any more info about this?
>>>
>>> In this respect sound is harder than graphics. Try pinning your dom0 to a
>>> CPU core, and set affinities on other domains so that dom0 does not compete
>>> for the core with any other domains.
>
>> All right, I can try that. However, the problem happens without any
>> domU running, just
>> the dom0. Do you still think that pinning to a cpu core or setting
>> affinities can help?
>
> I haven't followed the thread closely, but are you sure the device works OK on baremetal (linux without xen) ?
> If you have problems in dom0 it seems it hasn't got much to do with passthrough ?

You are right, I haven´t reached the point of trying to passthrough it
to a domU.

I´m sure the device works fine on bare metal. I have it working on an
HTPC which makes 2-3
recordings per day, without issues.

On the linux media mailing list I was said this problem was common
with all tuner cards,
more generally with any card which generates lots of interruptions and
that I should better
forget about it.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Sander Eikelenboom]

Tuesday, September 18, 2012, 5:12:28 PM, you wrote:

> On Tue, Sep 18, 2012 at 4:33 PM, Sander Eikelenboom
> <linux@eikelenboom.it> wrote:

>>>>> What I don´t understand is how graphics pass through even works
>>>> and a tuner
>>>>> card has problems due to the introduced latencies in the
>>>> IRQs.
>>>>
>>>> Do you have
>>>>> any more info about this?
>>>>
>>>> In this respect sound is harder than graphics. Try pinning your dom0 to a
>>>> CPU core, and set affinities on other domains so that dom0 does not compete
>>>> for the core with any other domains.
>>
>>> All right, I can try that. However, the problem happens without any
>>> domU running, just
>>> the dom0. Do you still think that pinning to a cpu core or setting
>>> affinities can help?
>>
>> I haven't followed the thread closely, but are you sure the device works OK on baremetal (linux without xen) ?
>> If you have problems in dom0 it seems it hasn't got much to do with passthrough ?

> You are right, I haven´t reached the point of trying to passthrough it
> to a domU.

> I´m sure the device works fine on bare metal. I have it working on an
> HTPC which makes 2-3
> recordings per day, without issues.

But that is a different machine, but with the same card ?
If that's right it would also try baremetal first on the same machine and card you are trying to get it to work with xen.

> On the linux media mailing list I was said this problem was common
> with all tuner cards,
> more generally with any card which generates lots of interruptions and
> that I should better
> forget about it.

Hmm that's a quite standard answer for not standard cases :-)

I have and have had multiple devices passed through, and it's working quite good, sound card, a analog tv tuner and a 8 channel dvr card.

Since it's a DVB card, i'm also wondering on what resolution you are trying to grab video, perhaps first try it on a quite low resolution (requiring less throughput and processing power), and if you are trying to transcode stuff directly.

--
Sander



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 6:08 PM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:

>>>>>> What I don´t understand is how graphics pass through even works
>>>>> and a tuner
>>>>>> card has problems due to the introduced latencies in the
>>>>> IRQs.
>>>>>
>>>>> Do you have
>>>>>> any more info about this?
>>>>>
>>>>> In this respect sound is harder than graphics. Try pinning your dom0 to a
>>>>> CPU core, and set affinities on other domains so that dom0 does not compete
>>>>> for the core with any other domains.
>>>
>>>> All right, I can try that. However, the problem happens without any
>>>> domU running, just
>>>> the dom0. Do you still think that pinning to a cpu core or setting
>>>> affinities can help?
>>>
>>> I haven't followed the thread closely, but are you sure the device works OK on baremetal (linux without xen) ?
>>> If you have problems in dom0 it seems it hasn't got much to do with passthrough ?
>
>> You are right, I haven´t reached the point of trying to passthrough it
>> to a domU.
>
>> I´m sure the device works fine on bare metal. I have it working on an
>> HTPC which makes 2-3
>> recordings per day, without issues.
>
> But that is a different machine, but with the same card ?
> If that's right it would also try baremetal first on the same machine and card you are trying to get it to work with xen.

No, it is the same machine. It is working as an HTPC, NAS, web and ftp
server right now.
I recently upgraded the CPU to be able to add virtualization as well.

>> On the linux media mailing list I was said this problem was common
>> with all tuner cards,
>> more generally with any card which generates lots of interruptions and
>> that I should better
>> forget about it.
>
> Hmm that's a quite standard answer for not standard cases :-)

Yes, I guess so. What was clear is that the linux media devs are not
going to help too much with this.

> I have and have had multiple devices passed through, and it's working quite good, sound card, a analog tv tuner and a 8 channel dvr card.

So far I have successfully passed through an ASMedia SATA controller
which seemed to work all right. That was with KVM and a Windows 7
guest, though. Trying to pass through the same DVB card (which has two
tuners) didn't exactly work. It shows up as two PCIe devices. On one
of them I can install the driver and gives no errors. On the other one
I can't even install the driver.

Anyhow, I could not tune any channel although I'm not sure the
software provided by Terratec was the best to try.

> Since it's a DVB card, i'm also wondering on what resolution you are trying to grab video, perhaps first try it on a quite low resolution (requiring less throughput and processing power), and if you are trying to transcode stuff directly.

Hmm, all the tests I have done, once it was clear I could not use vdr
since it didn't receive data, has been with 'mplayer -dumpstream' on a
SD channel. I-m pretty certain that it doesn't go over the 2mbps. I
get some data but as mpeg stream it is completely broken.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Konrad Rzeszutek Wilk]

On Tue, Sep 18, 2012 at 10:41:37AM +0200, Javier Marcet wrote:
> On Tue, Sep 18, 2012 at 2:08 AM, Javier Marcet <jmarcet@gmail.com> wrote:
> 
> Hi,
> 
> > Still nothing. Not even enabling the debug options within the DVB
> > subsystem I get
> > anything which doesn't look perfectly normal, exactly like when it's
> > working fine.
> >
> > I have just posted a message on the linux media mailing list looking
> > for some help
> > from the dvb maintainers.
> 
> I have some news from the linux-media mailing list. There, Devin Heitmueller,
> said this:
> 
> """
> > Initially I thought Xen would be the cause of the problem, but after
> > having written on
> > the Xen development mailing list and talked about it with a couple
> > developers, it isn't
> > very clear where the problem is. So far I haven't been able to get the
> > smallest warning
> > or error.
> 
> This is a very common problem when attempting to use any PCI/PCIe
> tuner under a hypervisor.  Essentially the issue is all of the
> virtualization solutions provide very poor interrupt latency, which
> results in the data being lost.
> 
> Devices delivering a high bitrate stream of data in realtime are much
> more likely for this problem to be visible since such devices have
> very little buffering (it's not like a hard drive controller where it
> can just deliver the data slower).  The problem is not specific to the
> cx23885 - pretty much all of the PCI/PCIe bridges used in tuner cards
> work this way, and they cannot really be blamed for expecting to run
> in an environment with really crappy interrupt latency.
> 
> I won't go as far as to say, "abandon all hope", but you're not really
> likely to find any help in this forum.
> """

Not sure about the interrupt latency. If you run this little program
http://darnok.org/xen/read_interrupts.c
when capturing in both dom0 and in baremetal are the rate about the
same?
> 
> What I don´t understand is how graphics pass through even works
> and a tuner card has problems due to the introduced latencies in the
> IRQs.

The latency is very very miniscule. One could actually verify that this
is a problem by inserting said latency in the driver so that under
baremetal the latency would show up.

But the other issue is that if the data is being lost you would at least
get some. So the buffers ought to have "something" in them? Maybe that
depends on what compression you use on the coder? If you jus do YUV420
or the RGB variants and just do a simple 'cat /dev/vide0 > /tmp/file'
does the output contain anything? Or is just a blob of empty data?

> 
> Do you have any more info about this?

The other issue could be related to the vmalloc_32 being in usage
and not using the DMA API.

I recall posting a patch for this .. Ah:
http://lists.xen.org/archives/html/xen-devel/2012-01/msg01927.html

Perhaps this is the issue?

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 7:17 PM, Konrad Rzeszutek Wilk
<konrad@kernel.org> wrote:

>> I have some news from the linux-media mailing list. There, Devin Heitmueller,
>> said this:
>>
>> """
>> > Initially I thought Xen would be the cause of the problem, but after
>> > having written on
>> > the Xen development mailing list and talked about it with a couple
>> > developers, it isn't
>> > very clear where the problem is. So far I haven't been able to get the
>> > smallest warning
>> > or error.
>>
>> This is a very common problem when attempting to use any PCI/PCIe
>> tuner under a hypervisor.  Essentially the issue is all of the
>> virtualization solutions provide very poor interrupt latency, which
>> results in the data being lost.
>>
>> Devices delivering a high bitrate stream of data in realtime are much
>> more likely for this problem to be visible since such devices have
>> very little buffering (it's not like a hard drive controller where it
>> can just deliver the data slower).  The problem is not specific to the
>> cx23885 - pretty much all of the PCI/PCIe bridges used in tuner cards
>> work this way, and they cannot really be blamed for expecting to run
>> in an environment with really crappy interrupt latency.
>>
>> I won't go as far as to say, "abandon all hope", but you're not really
>> likely to find any help in this forum.
>> """
>
> Not sure about the interrupt latency. If you run this little program
> http://darnok.org/xen/read_interrupts.c
> when capturing in both dom0 and in baremetal are the rate about the
> same?
>>
>> What I don´t understand is how graphics pass through even works
>> and a tuner card has problems due to the introduced latencies in the
>> IRQs.
>
> The latency is very very miniscule. One could actually verify that this
> is a problem by inserting said latency in the driver so that under
> baremetal the latency would show up.
>
> But the other issue is that if the data is being lost you would at least
> get some. So the buffers ought to have "something" in them? Maybe that
> depends on what compression you use on the coder? If you jus do YUV420
> or the RGB variants and just do a simple 'cat /dev/vide0 > /tmp/file'
> does the output contain anything? Or is just a blob of empty data?

The output contains data, part of the mpeg stream without any continuity.

>> Do you have any more info about this?
>
> The other issue could be related to the vmalloc_32 being in usage
> and not using the DMA API.
>
> I recall posting a patch for this .. Ah:
> http://lists.xen.org/archives/html/xen-devel/2012-01/msg01927.html
>
> Perhaps this is the issue?

Bingo Konrad! That was it :) At least I have just applied the patch,
rebooted under Xen and this time I got a good dump.

Right now I have my usual system running flawlessly under Xen. Time to
take a look again at the suspend/resume issue which I nearly had
working a few days ago.

Thanks a lot for all your help :)

P.S. I'm going to post this info on the linux media mailing list.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Konrad Rzeszutek Wilk]

On Tue, Sep 18, 2012 at 08:26:33PM +0200, Javier Marcet wrote:
> On Tue, Sep 18, 2012 at 7:17 PM, Konrad Rzeszutek Wilk
> <konrad@kernel.org> wrote:
> 
> >> I have some news from the linux-media mailing list. There, Devin Heitmueller,
> >> said this:
> >>
> >> """
> >> > Initially I thought Xen would be the cause of the problem, but after
> >> > having written on
> >> > the Xen development mailing list and talked about it with a couple
> >> > developers, it isn't
> >> > very clear where the problem is. So far I haven't been able to get the
> >> > smallest warning
> >> > or error.
> >>
> >> This is a very common problem when attempting to use any PCI/PCIe
> >> tuner under a hypervisor.  Essentially the issue is all of the
> >> virtualization solutions provide very poor interrupt latency, which
> >> results in the data being lost.
> >>
> >> Devices delivering a high bitrate stream of data in realtime are much
> >> more likely for this problem to be visible since such devices have
> >> very little buffering (it's not like a hard drive controller where it
> >> can just deliver the data slower).  The problem is not specific to the
> >> cx23885 - pretty much all of the PCI/PCIe bridges used in tuner cards
> >> work this way, and they cannot really be blamed for expecting to run
> >> in an environment with really crappy interrupt latency.
> >>
> >> I won't go as far as to say, "abandon all hope", but you're not really
> >> likely to find any help in this forum.
> >> """
> >
> > Not sure about the interrupt latency. If you run this little program
> > http://darnok.org/xen/read_interrupts.c
> > when capturing in both dom0 and in baremetal are the rate about the
> > same?
> >>
> >> What I don´t understand is how graphics pass through even works
> >> and a tuner card has problems due to the introduced latencies in the
> >> IRQs.
> >
> > The latency is very very miniscule. One could actually verify that this
> > is a problem by inserting said latency in the driver so that under
> > baremetal the latency would show up.
> >
> > But the other issue is that if the data is being lost you would at least
> > get some. So the buffers ought to have "something" in them? Maybe that
> > depends on what compression you use on the coder? If you jus do YUV420
> > or the RGB variants and just do a simple 'cat /dev/vide0 > /tmp/file'
> > does the output contain anything? Or is just a blob of empty data?
> 
> The output contains data, part of the mpeg stream without any continuity.
> 
I see.
> >> Do you have any more info about this?
> >
> > The other issue could be related to the vmalloc_32 being in usage
> > and not using the DMA API.
> >
> > I recall posting a patch for this .. Ah:
> > http://lists.xen.org/archives/html/xen-devel/2012-01/msg01927.html
> >
> > Perhaps this is the issue?
> 
> Bingo Konrad! That was it :) At least I have just applied the patch,
> rebooted under Xen and this time I got a good dump.

Woohoo!
> 
> Right now I have my usual system running flawlessly under Xen. Time to
> take a look again at the suspend/resume issue which I nearly had
> working a few days ago.

Keep in mind that for the suspend/resume you need out of mainline
patches for right now. I hadn't yet upstreamed all the parts. They are
in  git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
devel/acpi-s3.vXX (I believe 11 would apply cleanly to your tree?)

Keep in mind that there were some bugs in Xen 4.2 in regards to resume.
Look on xen-devel for emails between Jan and Ben Guthro.
> 
> Thanks a lot for all your help :)
> 
> P.S. I'm going to post this info on the linux media mailing list.

If you could CC me on it I can provide the technical explanation
for the 'DMA-API-ing the vmalloc_32' call.

Re: Xen 4.2.0-rc4 issues with DVB tuner

[Javier Marcet]

On Tue, Sep 18, 2012 at 9:34 PM, Konrad Rzeszutek Wilk
<konrad@kernel.org> wrote:

>> >> This is a very common problem when attempting to use any PCI/PCIe
>> >> tuner under a hypervisor.  Essentially the issue is all of the
>> >> virtualization solutions provide very poor interrupt latency, which
>> >> results in the data being lost.
>> >>
>> >> Devices delivering a high bitrate stream of data in realtime are much
>> >> more likely for this problem to be visible since such devices have
>> >> very little buffering (it's not like a hard drive controller where it
>> >> can just deliver the data slower).  The problem is not specific to the
>> >> cx23885 - pretty much all of the PCI/PCIe bridges used in tuner cards
>> >> work this way, and they cannot really be blamed for expecting to run
>> >> in an environment with really crappy interrupt latency.
>> >>
>> >> I won't go as far as to say, "abandon all hope", but you're not really
>> >> likely to find any help in this forum.
>> >> """
>> >
>> > Not sure about the interrupt latency. If you run this little program
>> > http://darnok.org/xen/read_interrupts.c
>> > when capturing in both dom0 and in baremetal are the rate about the
>> > same?
>> >>
>> >> What I don´t understand is how graphics pass through even works
>> >> and a tuner card has problems due to the introduced latencies in the
>> >> IRQs.
>> >
>> > The latency is very very miniscule. One could actually verify that this
>> > is a problem by inserting said latency in the driver so that under
>> > baremetal the latency would show up.
>> >
>> > But the other issue is that if the data is being lost you would at least
>> > get some. So the buffers ought to have "something" in them? Maybe that
>> > depends on what compression you use on the coder? If you jus do YUV420
>> > or the RGB variants and just do a simple 'cat /dev/vide0 > /tmp/file'
>> > does the output contain anything? Or is just a blob of empty data?
>>
>> The output contains data, part of the mpeg stream without any continuity.
>>
> I see.
>> >> Do you have any more info about this?
>> >
>> > The other issue could be related to the vmalloc_32 being in usage
>> > and not using the DMA API.
>> >
>> > I recall posting a patch for this .. Ah:
>> > http://lists.xen.org/archives/html/xen-devel/2012-01/msg01927.html
>> >
>> > Perhaps this is the issue?
>>
>> Bingo Konrad! That was it :) At least I have just applied the patch,
>> rebooted under Xen and this time I got a good dump.
>
> Woohoo!

:)

>> Right now I have my usual system running flawlessly under Xen. Time to
>> take a look again at the suspend/resume issue which I nearly had
>> working a few days ago.
>
> Keep in mind that for the suspend/resume you need out of mainline
> patches for right now. I hadn't yet upstreamed all the parts. They are
> in  git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
> devel/acpi-s3.vXX (I believe 11 would apply cleanly to your tree?)
>
> Keep in mind that there were some bugs in Xen 4.2 in regards to resume.
> Look on xen-devel for emails between Jan and Ben Guthro.

I have the needed patches in place and I have followed that thread, at
least until a few days ago. I have opened a new one about the only
problem I still have with S3: http://goo.gl/vYK42

>> Thanks a lot for all your help :)
>>
>> P.S. I'm going to post this info on the linux media mailing list.
>
> If you could CC me on it I can provide the technical explanation
> for the 'DMA-API-ing the vmalloc_32' call.

I have CCed you in my last reply to the thread. I'm pretty certain
Devin understood well the problem and what the solution your patch
gives.


-- 
Javier Marcet <jmarcet@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

The neverending load increase....

[Carsten Schiers]

[-- Attachment #1.1: Type: text/plain, Size: 1198 bytes --]

Hi Konrad,

 
Yes me again, the guy with the double bounce buffering problem with the DVB card when using more than 

4GB of RAM and PVOPS.

 
If you are still interested: gues what do you think what happened between 3rd and fourth on riker? Yes,

I switched to PVOPS ;o).

 

 
So, I am currently on Xen 4.1 with 3.6.7 PVOPS Kernel for Dom0 and riker. Before the increase, I was using

an old-style kernel, no other changes in Domain.

 
Compared to last time we checked on it, I switched mainboard, CPU (AMD->Intel), and now have 16GB. 

 
So just in case, we would like to check on it further, please feel free to advise or patch a kernel and I 

am more than happy to try out.

 
I use the PVOPS kernel now because of some glitches when recording and I want to test whether they persist

on PVOPS. I am going to switch to Xen 4.2 soon, because I want to modify the scheduler with the new parameters,

as I guess the Domain doesn’t get enough attention, there are 3 PCI cards for it…

 
If this is not interesting, please simply let me know, it’s not really an issue, it’s more for curiosity.

 
BR,

Carsten.


[-- Attachment #1.2.1: Type: text/html, Size: 5993 bytes --]

[-- Attachment #1.2.2: Type: image/png, Size: 28738 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: The neverending load increase....

[Konrad Rzeszutek Wilk]

On Wed, Dec 05, 2012 at 11:06:02PM +0100, Carsten Schiers wrote:
> Hi Konrad,

Heya!
> 
>  
> Yes me again, the guy with the double bounce buffering problem with the DVB card when using more than 
> 
> 4GB of RAM and PVOPS.
> 
>  
> If you are still interested: gues what do you think what happened between 3rd and fourth on riker? Yes,
> 
> I switched to PVOPS ;o).
> 
>  
> 
>  
> So, I am currently on Xen 4.1 with 3.6.7 PVOPS Kernel for Dom0 and riker. Before the increase, I was using
> 
> an old-style kernel, no other changes in Domain.
> 
>  
> Compared to last time we checked on it, I switched mainboard, CPU (AMD->Intel), and now have 16GB. 
> 
>  
> So just in case, we would like to check on it further, please feel free to advise or patch a kernel and I 
> 
> am more than happy to try out.

Did you try doing it in an PVHVM guest?

> 
>  
> I use the PVOPS kernel now because of some glitches when recording and I want to test whether they persist
> 
> on PVOPS. I am going to switch to Xen 4.2 soon, because I want to modify the scheduler with the new parameters,

credit2 scheduler?
> 
> as I guess the Domain doesn’t get enough attention, there are 3 PCI cards for it…

Was it you who had IOMMU issues? Meaning the the AMD-VI would throw out errors when
the cards were doing IO?
> 
>  
> If this is not interesting, please simply let me know, it’s not really an issue, it’s more for curiosity.

It is always interesting! Please do keep me abreast!
> 
>  
> BR,
> 
> Carsten.
> 

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel