pci adapter not working in dom0. works under plain Linux.

pci adapter not working in dom0. works under plain Linux.

[James Harper]

I bought an unsupported DVB card (by mistake - I bought the v2 version instead of the supported v1 version), and I've just finished getting it working under Linux, but spent the last day or so trying to figure out why it keeps dropping buffers even though the reception is perfect. As soon as I noticed that the lost data is always in PAGE_SIZE multiples, I figured I'd better boot in plain Linux instead of Xen, and when I did that it worked perfectly.

The card is based on the cx23885 chipset, which is already supported (my work is just porting a different tuner/demod), so I suspect it's actually a bug in the existing cx23885 driver. I'm hoping there is just a flag missing or something when the adapter allocates buffers for DMA. Can anyone offer any suggestions as to what I'm looking for before I dive in?

Thanks

James

Re: pci adapter not working in dom0. works under plain Linux.

[Konrad Rzeszutek Wilk]

On Sun, Jun 08, 2014 at 11:01:25AM +0000, James Harper wrote:
> I bought an unsupported DVB card (by mistake - I bought the v2 version instead of the supported v1 version), and I've just finished getting it working under Linux, but spent the last day or so trying to figure out why it keeps dropping buffers even though the reception is perfect. As soon as I noticed that the lost data is always in PAGE_SIZE multiples, I figured I'd better boot in plain Linux instead of Xen, and when I did that it worked perfectly.

What happens if you boot dom0 with 'dom0_mem=max:1G' and limit the amount of memory
Xen sees by using 'mem=4G' on the Xen command line?


> 
> The card is based on the cx23885 chipset, which is already supported (my work is just porting a different tuner/demod), so I suspect it's actually a bug in the existing cx23885 driver. I'm hoping there is just a flag missing or something when the adapter allocates buffers for DMA. Can anyone offer any suggestions as to what I'm looking for before I dive in?
> 
> Thanks
> 
> James
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: pci adapter not working in dom0. works under plain Linux.

[James Harper]

> 
> On Sun, Jun 08, 2014 at 11:01:25AM +0000, James Harper wrote:
> > I bought an unsupported DVB card (by mistake - I bought the v2 version
> instead of the supported v1 version), and I've just finished getting it working
> under Linux, but spent the last day or so trying to figure out why it keeps
> dropping buffers even though the reception is perfect. As soon as I noticed
> that the lost data is always in PAGE_SIZE multiples, I figured I'd better boot in
> plain Linux instead of Xen, and when I did that it worked perfectly.
> 
> What happens if you boot dom0 with 'dom0_mem=max:1G' and limit the
> amount of memory
> Xen sees by using 'mem=4G' on the Xen command line?
> 

That seems to make the problem go away. The machine itself has 5GB of memory in it, so (assuming an even spread) only 1/5 of allocations would come from >4GB memory right? This matches the missing pages of data in the resulting DMA transfers.

Where should I start looking for the fix? The card is PCIe so I would have thought it would have supported DMA to memory > 4GB, is that the problem? I think one of my USB devices has the same problem too.

Btw, what is dom0_mem=max:1G supposed to do? Dom0 still has ~3.7GB memory allocated to it on boot.

Thanks!

James

Re: pci adapter not working in dom0. works under plain Linux.

[Konrad Rzeszutek Wilk]

On Tue, Jun 10, 2014 at 11:19:18AM +0000, James Harper wrote:
> > 
> > On Sun, Jun 08, 2014 at 11:01:25AM +0000, James Harper wrote:
> > > I bought an unsupported DVB card (by mistake - I bought the v2 version
> > instead of the supported v1 version), and I've just finished getting it working
> > under Linux, but spent the last day or so trying to figure out why it keeps
> > dropping buffers even though the reception is perfect. As soon as I noticed
> > that the lost data is always in PAGE_SIZE multiples, I figured I'd better boot in
> > plain Linux instead of Xen, and when I did that it worked perfectly.
> > 
> > What happens if you boot dom0 with 'dom0_mem=max:1G' and limit the
> > amount of memory
> > Xen sees by using 'mem=4G' on the Xen command line?
> > 
> 
> That seems to make the problem go away. The machine itself has 5GB of memory in it, so (assuming an even spread) only 1/5 of allocations would come from >4GB memory right? This matches the missing pages of data in the resulting DMA transfers.

That was my theory.
> 
> Where should I start looking for the fix? The card is PCIe so I would have thought it would have supported DMA to memory > 4GB, is that the problem? I think one of my USB devices has the same problem too.

Ugh. They all should do the right job - that is using the DMA API.
> 
> Btw, what is dom0_mem=max:1G supposed to do? Dom0 still has ~3.7GB memory allocated to it on boot.

Oh, I think I mentioned that it should have been on the 'dom0' line but
I meant Xen line.

If you try removing the 'mem=4G' and just use 'dom0_mem=max:1G' please?
That should limit the amount of memory the dom0 uses (it sees the
rest of the E820 - but does not use it) - and that means that the
drivers should _also_ use said below memory.

I had some preliminary patches for the V4L API to use a better
DMA ops, but I fear that they are bitrotten (and I can't even
find them!). But lets try first determine if the workarounds
work for you.
> 
> Thanks!
> 
> James

Re: pci adapter not working in dom0. works under plain Linux.

[James Harper]

> > > What happens if you boot dom0 with 'dom0_mem=max:1G' and limit the
> > > amount of memory
> > > Xen sees by using 'mem=4G' on the Xen command line?
> > >
> >
> > That seems to make the problem go away. The machine itself has 5GB of
> memory in it, so (assuming an even spread) only 1/5 of allocations would
> come from >4GB memory right? This matches the missing pages of data in
> the resulting DMA transfers.
> 
> That was my theory.
> >
> > Where should I start looking for the fix? The card is PCIe so I would have
> thought it would have supported DMA to memory > 4GB, is that the
> problem? I think one of my USB devices has the same problem too.
> 
> Ugh. They all should do the right job - that is using the DMA API.
> >
> > Btw, what is dom0_mem=max:1G supposed to do? Dom0 still has ~3.7GB
> memory allocated to it on boot.
> 
> Oh, I think I mentioned that it should have been on the 'dom0' line but
> I meant Xen line.
> 
> If you try removing the 'mem=4G' and just use 'dom0_mem=max:1G'
> please?
> That should limit the amount of memory the dom0 uses (it sees the
> rest of the E820 - but does not use it) - and that means that the
> drivers should _also_ use said below memory.
> 
> I had some preliminary patches for the V4L API to use a better
> DMA ops, but I fear that they are bitrotten (and I can't even
> find them!). But lets try first determine if the workarounds
> work for you.
> >

That also produced no errors, so mem=4G fixes the problem, as does dom0_mem=max:1G

James

Re: pci adapter not working in dom0. works under plain Linux.

[James Harper]

> 
> That also produced no errors, so mem=4G fixes the problem, as does
> dom0_mem=max:1G
> 

Actually now I'm not so sure. When I booted with dom0_mem=max:1G I didn't get any errors from a vlc command I was running, but then I tried a quick record using mythtv and got nothing. No errors or anything, but the recording just bailed immediately. Maybe it didn't have enough memory to run... not sure.

Out of time for further testing tonight.

James

Re: pci adapter not working in dom0. works under plain Linux.

[Jan Beulich]

>>> On 10.06.14 at 16:00, <james@ejbdigital.com.au> wrote:
>> 
>> That also produced no errors, so mem=4G fixes the problem, as does
>> dom0_mem=max:1G
>> 
> 
> Actually now I'm not so sure. When I booted with dom0_mem=max:1G I didn't 
> get any errors from a vlc command I was running, but then I tried a quick 
> record using mythtv and got nothing. No errors or anything, but the recording 
> just bailed immediately. Maybe it didn't have enough memory to run... not 
> sure.

In fact I'd be rather surprised if dom0_mem= had any effect here.

Jan

Re: pci adapter not working in dom0. works under plain Linux.

[Sander Eikelenboom]

Tuesday, June 10, 2014, 3:08:32 PM, you wrote:

> On Tue, Jun 10, 2014 at 11:19:18AM +0000, James Harper wrote:
>> > 
>> > On Sun, Jun 08, 2014 at 11:01:25AM +0000, James Harper wrote:
>> > > I bought an unsupported DVB card (by mistake - I bought the v2 version
>> > instead of the supported v1 version), and I've just finished getting it working
>> > under Linux, but spent the last day or so trying to figure out why it keeps
>> > dropping buffers even though the reception is perfect. As soon as I noticed
>> > that the lost data is always in PAGE_SIZE multiples, I figured I'd better boot in
>> > plain Linux instead of Xen, and when I did that it worked perfectly.
>> > 
>> > What happens if you boot dom0 with 'dom0_mem=max:1G' and limit the
>> > amount of memory
>> > Xen sees by using 'mem=4G' on the Xen command line?
>> > 
>> 
>> That seems to make the problem go away. The machine itself has 5GB of memory in it, so (assuming an even spread) only 1/5 of allocations would come from >4GB memory right? This matches the missing pages of data in the resulting DMA transfers.

> That was my theory.
>> 
>> Where should I start looking for the fix? The card is PCIe so I would have thought it would have supported DMA to memory > 4GB, is that the problem? I think one of my USB devices has the same problem too.

> Ugh. They all should do the right job - that is using the DMA API.
>> 
>> Btw, what is dom0_mem=max:1G supposed to do? Dom0 still has ~3.7GB memory allocated to it on boot.

> Oh, I think I mentioned that it should have been on the 'dom0' line but
> I meant Xen line.

> If you try removing the 'mem=4G' and just use 'dom0_mem=max:1G' please?
> That should limit the amount of memory the dom0 uses (it sees the
> rest of the E820 - but does not use it) - and that means that the
> drivers should _also_ use said below memory.

> I had some preliminary patches for the V4L API to use a better
> DMA ops, but I fear that they are bitrotten (and I can't even
> find them!). But lets try first determine if the workarounds
> work for you.

Were you by any means referring to this thread / patch: 
http://lists.xen.org/archives/html/xen-devel/2012-01/msg01982.html ?

--
Sander

>> 
>> Thanks!
>> 
>> James

Re: pci adapter not working in dom0. works under plain Linux.

[James Harper]

> 
> > I had some preliminary patches for the V4L API to use a better
> > DMA ops, but I fear that they are bitrotten (and I can't even
> > find them!). But lets try first determine if the workarounds
> > work for you.
> 
> Were you by any means referring to this thread / patch:
> http://lists.xen.org/archives/html/xen-devel/2012-01/msg01982.html ?
> 

That patch applies and compiles once you take into account some moved files. I haven't tested yet though - my mythtv box is busy recording :)

I notice there is a 'videobuf2-dma-sg.c' file in the current tree, which I assume is a 'second generation' v4l implementation that drivers will be progressively moved over to. I can't tell if the dma implementation in there is correct, but it certainly doesn't return any hits in a grep for 32...

A grep for the export 'vb2_dma_sg_memops' only shows up 2 drivers using it though. What's the procedure around rebasing and submitting a patch that someone else wrote?

I think Konrad's fix is the right way to go. Someone else suggested fixing vmalloc32, which is probably still a good idea, but when most of the devices these days are probably quite happy with 64 bit addresses so sticking with the artificial 4G limit is probably a bit silly.

I'll follow up once I've tested later tonight or tomorrow.

Thanks!

James

Re: pci adapter not working in dom0. works under plain Linux.

[Sander Eikelenboom]

Wednesday, June 11, 2014, 1:53:48 PM, you wrote:

>> 
>> > I had some preliminary patches for the V4L API to use a better
>> > DMA ops, but I fear that they are bitrotten (and I can't even
>> > find them!). But lets try first determine if the workarounds
>> > work for you.
>> 
>> Were you by any means referring to this thread / patch:
>> http://lists.xen.org/archives/html/xen-devel/2012-01/msg01982.html ?
>> 

> That patch applies and compiles once you take into account some moved files. I haven't tested yet though - my mythtv box is busy recording :)
> I notice there is a 'videobuf2-dma-sg.c' file in the current tree, which I assume is a 'second generation' v4l implementation that drivers will be progressively moved over to. I can't tell if the dma implementation in there is correct, but it certainly doesn't return any hits in a grep for 32...
> A grep for the export 'vb2_dma_sg_memops' only shows up 2 drivers using it though. What's the procedure around rebasing and submitting a patch that someone else wrote?

Yeah the conversion is going slowly .. i tried to start it for the cx25821 .. 
but the problem is it's "all or nothing" you can't do it in little steps.

> I think Konrad's fix is the right way to go. Someone else suggested fixing vmalloc32, which is probably still a good idea, but when most of the devices these days are probably quite happy with 64 bit addresses so sticking with the artificial 4G limit is probably a bit silly.
> I'll follow up once I've tested later tonight or tomorrow.

> Thanks!

> James

Re: pci adapter not working in dom0. works under plain Linux.

[Konrad Rzeszutek Wilk]

On Wed, Jun 11, 2014 at 02:12:13PM +0200, Sander Eikelenboom wrote:
> 
> Wednesday, June 11, 2014, 1:53:48 PM, you wrote:
> 
> >> 
> >> > I had some preliminary patches for the V4L API to use a better
> >> > DMA ops, but I fear that they are bitrotten (and I can't even
> >> > find them!). But lets try first determine if the workarounds
> >> > work for you.
> >> 
> >> Were you by any means referring to this thread / patch:
> >> http://lists.xen.org/archives/html/xen-devel/2012-01/msg01982.html ?
> >> 
> 
> > That patch applies and compiles once you take into account some moved files. I haven't tested yet though - my mythtv box is busy recording :)
> > I notice there is a 'videobuf2-dma-sg.c' file in the current tree, which I assume is a 'second generation' v4l implementation that drivers will be progressively moved over to. I can't tell if the dma implementation in there is correct, but it certainly doesn't return any hits in a grep for 32...
> > A grep for the export 'vb2_dma_sg_memops' only shows up 2 drivers using it though. What's the procedure around rebasing and submitting a patch that someone else wrote?
> 
> Yeah the conversion is going slowly .. i tried to start it for the cx25821 .. 
> but the problem is it's "all or nothing" you can't do it in little steps.

You mean the patch I wrote that I totally forgot about until
Sander awesomely found it? You can go ahead and have it come
from you (aka, the author can be you). The only thing I would
ask you to keep is my SOB (so it is known that it came through
my hands) in the patch and maybe mention: "Rework of Konrad's initial
patch" (optional).

You should not treat this as a general rule though - but the yardstick 
I've done is - contact the author of the patch - ask them if they are OK
with the patch and me reworking it. If they don't respond or are unreachable
I take the ownership of the patch and start reworking. At that point
I just make sure that I keep their original Copyright in the file or mention
it as an SoB.


> 
> > I think Konrad's fix is the right way to go. Someone else suggested fixing vmalloc32, which is probably still a good idea, but when most of the devices these days are probably quite happy with 64 bit addresses so sticking with the artificial 4G limit is probably a bit silly.
> > I'll follow up once I've tested later tonight or tomorrow.
> 
> > Thanks!
> 
> > James
> 
> 
> 

Re: pci adapter not working in dom0. works under plain Linux.

[Sander Eikelenboom]

Wednesday, June 11, 2014, 3:47:10 PM, you wrote:

> On Wed, Jun 11, 2014 at 02:12:13PM +0200, Sander Eikelenboom wrote:
>> 
>> Wednesday, June 11, 2014, 1:53:48 PM, you wrote:
>> 
>> >> 
>> >> > I had some preliminary patches for the V4L API to use a better
>> >> > DMA ops, but I fear that they are bitrotten (and I can't even
>> >> > find them!). But lets try first determine if the workarounds
>> >> > work for you.
>> >> 
>> >> Were you by any means referring to this thread / patch:
>> >> http://lists.xen.org/archives/html/xen-devel/2012-01/msg01982.html ?
>> >> 
>> 
>> > That patch applies and compiles once you take into account some moved files. I haven't tested yet though - my mythtv box is busy recording :)
>> > I notice there is a 'videobuf2-dma-sg.c' file in the current tree, which I assume is a 'second generation' v4l implementation that drivers will be progressively moved over to. I can't tell if the dma implementation in there is correct, but it certainly doesn't return any hits in a grep for 32...
>> > A grep for the export 'vb2_dma_sg_memops' only shows up 2 drivers using it though. What's the procedure around rebasing and submitting a patch that someone else wrote?
>> 
>> Yeah the conversion is going slowly .. i tried to start it for the cx25821 .. 
>> but the problem is it's "all or nothing" you can't do it in little steps.

> You mean the patch I wrote that I totally forgot about until
> Sander awesomely found it? You can go ahead and have it come
> from you (aka, the author can be you). The only thing I would
> ask you to keep is my SOB (so it is known that it came through
> my hands) in the patch and maybe mention: "Rework of Konrad's initial
> patch" (optional).

> You should not treat this as a general rule though - but the yardstick 
> I've done is - contact the author of the patch - ask them if they are OK
> with the patch and me reworking it. If they don't respond or are unreachable
> I take the ownership of the patch and start reworking. At that point
> I just make sure that I keep their original Copyright in the file or mention
> it as an SoB.

Wasn't that hard .. i had a vague memory of it .. and you supplied the keywords 
for searching your self: "konrad" and "v4l" :-)

>> 
>> > I think Konrad's fix is the right way to go. Someone else suggested fixing vmalloc32, which is probably still a good idea, but when most of the devices these days are probably quite happy with 64 bit addresses so sticking with the artificial 4G limit is probably a bit silly.
>> > I'll follow up once I've tested later tonight or tomorrow.
>> 
>> > Thanks!
>> 
>> > James
>> 
>> 
>>