Kernel Oops, dvb_dmxdev_filter_reset, bisected

Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Thomas Voegtle]

Hello,

yesterday I moved from 2.6.31.y to 2.6.32 and found a reproducable kernel 
oops.
Bug is in Linus' tree, too.

I use a Hauppauge Nova-T Stick with dvb_usb_dib0700

I start mplayer (no problems so far) and then alevt. Then there comes
the Oops:

Oops: 0000 [#1] SMP
last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Modules linked in: i915 drm_kms_helper video backlight output microcode 
loop mt2060 dvb_usb_dib0700 dib7000p dib7000m dib0070 dvb_usb dib8000 
dvb_core dib3000mc dibx000_common uhci_hcd ehci_hcd usbcore

Pid: 3429, comm: alevt Not tainted 2.6.33-rc6 #17 MS-7267/MS-7267
EIP: 0060:[<f86bb11b>] EFLAGS: 00210246 CPU: 1
EIP is at dvb_dmxdev_filter_reset+0x1a/0x80 [dvb_core]
EAX: 00000000 EBX: f886b204 ECX: fffffff8 EDX: e0cb3000
ESI: f886b204 EDI: f886b208 EBP: f27ff440 ESP: e0cb3f48
  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process alevt (pid: 3429, ti=e0cb3000 task=e9dcdb20 task.ti=e0cb3000)
Stack:
  00000008 f886b204 f75bc88c f86bb1b9 f27ff440 f886b27c f75bc8d0 00000008
<0> f7111744 f6e5fc54 f27ff440 c1074013 f7111744 f741dcc0 f6e5fc54 
00000000
<0> f27ff440 f74673c0 e0cb3000 c1071a92 f74673c0 00000003 080674e8 
c1071af2
Call Trace:
  [<f86bb1b9>] ? dvb_demux_release+0x38/0x107 [dvb_core]
  [<c1074013>] ? __fput+0xd5/0x169
  [<c1071a92>] ? filp_close+0x45/0x4b
  [<c1071af2>] ? sys_close+0x5a/0x8d
  [<c1002710>] ? sysenter_do_call+0x12/0x26
  [<c12b0000>] ? pci_read_bridge_bases+0x173/0x2fe
Code: 75 dd 8d 46 54 e8 c2 86 00 00 31 c0 5b 5e 5f 5d c3 57 56 53 89 c3 83 
78 4c 01 76 6f 83 78 48 02 75 49 8b 40 04 8d 7b 04 8d 48 f8 <8b> 41 08 8d 
70 f8 eb 28 8b 41 08 8b 51 0c 89 50 04 89 02 c7 41
EIP: [<f86bb11b>] dvb_dmxdev_filter_reset+0x1a/0x80 [dvb_core] SS:ESP 
0068:e0cb3f48
CR2: 0000000000000000
---[ end trace 629e2045091796f7 ]---



I bisected this down to:

root@scratchy:/data/kernel/linux-2.6# git bisect bad
1cb662a3144992259edfd3cf9f54a6b25a913a0f is first bad commit
commit 1cb662a3144992259edfd3cf9f54a6b25a913a0f
Author: Andreas Oberritter <obi@linuxtv.org>
Date:   Tue Jul 14 20:28:50 2009 -0300

     V4L/DVB (12275): Add two new ioctls: DMX_ADD_PID and DMX_REMOVE_PID
...


Reverting the patch on top of 2.6.33-rc6, I can start mplayer 
and alevt with no problems.

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Chicken Shack]

Am Montag, den 01.02.2010, 19:06 +0100 schrieb Thomas Voegtle:
> Hello,
> 
> yesterday I moved from 2.6.31.y to 2.6.32 and found a reproducable kernel 
> oops.
> Bug is in Linus' tree, too.
> 
> I use a Hauppauge Nova-T Stick with dvb_usb_dib0700
> 
> I start mplayer (no problems so far) and then alevt. Then there comes
> the Oops:
> 
> Oops: 0000 [#1] SMP
> last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> Modules linked in: i915 drm_kms_helper video backlight output microcode 
> loop mt2060 dvb_usb_dib0700 dib7000p dib7000m dib0070 dvb_usb dib8000 
> dvb_core dib3000mc dibx000_common uhci_hcd ehci_hcd usbcore
> 
> Pid: 3429, comm: alevt Not tainted 2.6.33-rc6 #17 MS-7267/MS-7267
> EIP: 0060:[<f86bb11b>] EFLAGS: 00210246 CPU: 1
> EIP is at dvb_dmxdev_filter_reset+0x1a/0x80 [dvb_core]
> EAX: 00000000 EBX: f886b204 ECX: fffffff8 EDX: e0cb3000
> ESI: f886b204 EDI: f886b208 EBP: f27ff440 ESP: e0cb3f48
>   DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
> Process alevt (pid: 3429, ti=e0cb3000 task=e9dcdb20 task.ti=e0cb3000)
> Stack:
>   00000008 f886b204 f75bc88c f86bb1b9 f27ff440 f886b27c f75bc8d0 00000008
> <0> f7111744 f6e5fc54 f27ff440 c1074013 f7111744 f741dcc0 f6e5fc54 
> 00000000
> <0> f27ff440 f74673c0 e0cb3000 c1071a92 f74673c0 00000003 080674e8 
> c1071af2
> Call Trace:
>   [<f86bb1b9>] ? dvb_demux_release+0x38/0x107 [dvb_core]
>   [<c1074013>] ? __fput+0xd5/0x169
>   [<c1071a92>] ? filp_close+0x45/0x4b
>   [<c1071af2>] ? sys_close+0x5a/0x8d
>   [<c1002710>] ? sysenter_do_call+0x12/0x26
>   [<c12b0000>] ? pci_read_bridge_bases+0x173/0x2fe
> Code: 75 dd 8d 46 54 e8 c2 86 00 00 31 c0 5b 5e 5f 5d c3 57 56 53 89 c3 83 
> 78 4c 01 76 6f 83 78 48 02 75 49 8b 40 04 8d 7b 04 8d 48 f8 <8b> 41 08 8d 
> 70 f8 eb 28 8b 41 08 8b 51 0c 89 50 04 89 02 c7 41
> EIP: [<f86bb11b>] dvb_dmxdev_filter_reset+0x1a/0x80 [dvb_core] SS:ESP 
> 0068:e0cb3f48
> CR2: 0000000000000000
> ---[ end trace 629e2045091796f7 ]---
> 
> 
> 
> I bisected this down to:
> 
> root@scratchy:/data/kernel/linux-2.6# git bisect bad
> 1cb662a3144992259edfd3cf9f54a6b25a913a0f is first bad commit
> commit 1cb662a3144992259edfd3cf9f54a6b25a913a0f
> Author: Andreas Oberritter <obi@linuxtv.org>
> Date:   Tue Jul 14 20:28:50 2009 -0300
> 
>      V4L/DVB (12275): Add two new ioctls: DMX_ADD_PID and DMX_REMOVE_PID
> ...
> 
> 
> Reverting the patch on top of 2.6.33-rc6, I can start mplayer 
> and alevt with no problems.

Hi Thomas,

thanks for reproducing that kernel oops.

Question:

Can you also confirm / reproduce that alevt does not follow the new TV
or radio channel if the new channel, tuned by dvbstream / mplayer for
example, is part of another transponder?

Normal, i. e. expected behaviour can be desribed in the following
example:

a. You start mplayer://ZDF, then you start alevt, and ZDF teletext
should be visible.

b. You change the channel to mplayer://Das Erste.
Now alevt should follow the new tuning and tune one channel of the
transponder containing the ARD bouquet.

But instead of that alevt hangs and cannot be finished by an ordinary
quit. You need _violence_ a la "killall -9 alevt" or, on the command
line: STRG-C as shortcut.

Can you reproduce / confirm that, Thomas?

Thanks

CS
 




> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Thomas Voegtle]

On Mon, 1 Feb 2010, Chicken Shack wrote:

> Hi Thomas,
>
> thanks for reproducing that kernel oops.
>
> Question:
>
> Can you also confirm / reproduce that alevt does not follow the new TV
> or radio channel if the new channel, tuned by dvbstream / mplayer for
> example, is part of another transponder?
>
> Normal, i. e. expected behaviour can be desribed in the following
> example:
>
> a. You start mplayer://ZDF, then you start alevt, and ZDF teletext
> should be visible.
>
> b. You change the channel to mplayer://Das Erste.
> Now alevt should follow the new tuning and tune one channel of the
> transponder containing the ARD bouquet.
>
> But instead of that alevt hangs and cannot be finished by an ordinary
> quit. You need _violence_ a la "killall -9 alevt" or, on the command
> line: STRG-C as shortcut.
>
> Can you reproduce / confirm that, Thomas?


Yes, I can confirm that. And yes, it is annoying.


thanks,

Thoams

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Chicken Shack]

Am Montag, den 01.02.2010, 21:50 +0100 schrieb Thomas Voegtle:
> On Mon, 1 Feb 2010, Chicken Shack wrote:
> 
> > Hi Thomas,
> >
> > thanks for reproducing that kernel oops.
> >
> > Question:
> >
> > Can you also confirm / reproduce that alevt does not follow the new TV
> > or radio channel if the new channel, tuned by dvbstream / mplayer for
> > example, is part of another transponder?
> >
> > Normal, i. e. expected behaviour can be desribed in the following
> > example:
> >
> > a. You start mplayer://ZDF, then you start alevt, and ZDF teletext
> > should be visible.
> >
> > b. You change the channel to mplayer://Das Erste.
> > Now alevt should follow the new tuning and tune one channel of the
> > transponder containing the ARD bouquet.
> >
> > But instead of that alevt hangs and cannot be finished by an ordinary
> > quit. You need _violence_ a la "killall -9 alevt" or, on the command
> > line: STRG-C as shortcut.
> >
> > Can you reproduce / confirm that, Thomas?
> 
> 
> Yes, I can confirm that. And yes, it is annoying.


Thank you, Thomas.

I think that the tasks to work on are clear now. All my hopes rest on
Andy Walls now......
To be honest I would be much happier if more people would volunteer and
perform a task splitting due to lack of time......


The thing is:
Looking at the code in vbi.c (using grep -e .....) I in fact saw a vbi
reset function call.
But this vbi reset function call does not touch the DVB demux device
(which would mean f. ex. to set the teletext pid to zero and stuff like
that.....).

Proof (which you can easily find out if you have an analogue bttv card).

The hangup does not happen if you use alevt-dvb in analogue mode.
It only happens because the DVB implementation needs a little bit of
care by a highly experienced and competent person.

The vbi reset function or even some similar system call needs to be
extended or added to fulfil DVB needs.

The DVB implementation is reduced to demux filter release (start) and
demux filter stop. Reset does not seem to exist, and that's why the
proggy does not follow the new channel as part of a different
transponder if a new channel is being tuned by some external application
like kaffeine or mplayer.....


> thanks,
> 
> Thoams


My turn so say Thank you

CS


> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Mauro Carvalho Chehab]

Thomas Voegtle wrote:
> 
> Hello,
> 
> yesterday I moved from 2.6.31.y to 2.6.32 and found a reproducable
> kernel oops.
> Bug is in Linus' tree, too.

Please try the two patches I sent to the ML today that fixes two potential
situations where the OOPS bug may hit.

I suspect that alevt-dvb is doing something wrong with the memory of the machine.

The more likely case happens when there's no more available memory for vmalloc(). 
In this case, this code fails:

        dvbdemux->feed = vmalloc(dvbdemux->feednum * sizeof(struct dvb_demux_feed));
        if (!dvbdemux->feed) {
                vfree(dvbdemux->filter);
                return -ENOMEM;

And the driver frees the memory. However, before the patch, it used to forget
to reset dvbdemux->filter to NULL. Later, dvb_dmx_release() is called, and it
tries to free an already freed memory, causing the OOPS.

After applying those two patches, I can't see any other potential cause
for the problem. Someone with aletv and a DVB-T signal with Teletext
(it is not my case, I am at an ISDB-T area) should now take a look on 
what's the application is doing and why aletv-dvb is exhausting the computer 
memory used by vmalloc.

> I bisected this down to:
> 
> root@scratchy:/data/kernel/linux-2.6# git bisect bad
> 1cb662a3144992259edfd3cf9f54a6b25a913a0f is first bad commit
> commit 1cb662a3144992259edfd3cf9f54a6b25a913a0f
> Author: Andreas Oberritter <obi@linuxtv.org>
> Date:   Tue Jul 14 20:28:50 2009 -0300
> 
>     V4L/DVB (12275): Add two new ioctls: DMX_ADD_PID and DMX_REMOVE_PID
> ...
> 
> 
> Reverting the patch on top of 2.6.33-rc6, I can start mplayer and alevt
> with no problems.

If reverting this patch solves the issue, then I can see 2 possible reasons
for the breakage:
	1) the behavior of the old ioctls changed a little bit for Teletext;
	2) your mplayer version has support to the new ioctls, and is doing
something different. In this case, as aletv-dvb is not prepared for the 
changes, it hits some internal bug, and it is working badly.

In any case, someone needs to dig at aletv and check what's happening there, 
identifying the root cause. So, the better is to contact aletv-dvb maintainer. 

It the error is due to (2), the fix is a patch to aletv-dvb. If it is (1),
then a regression has occurred, and a patch to the kernel is needed. For someone
to write such patch, he needs to know exactly where's the bug: e. g. what's the
difference between the previous driver response and the new broken response.

Cheers,
Mauro

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Thomas Voegtle]

On Mon, 1 Feb 2010, Mauro Carvalho Chehab wrote:

> Thomas Voegtle wrote:
>>
>> Hello,
>>
>> yesterday I moved from 2.6.31.y to 2.6.32 and found a reproducable
>> kernel oops.
>> Bug is in Linus' tree, too.
>
> Please try the two patches I sent to the ML today that fixes two potential
> situations where the OOPS bug may hit.
>
> I suspect that alevt-dvb is doing something wrong with the memory of the machine.
>
> The more likely case happens when there's no more available memory for vmalloc().
> In this case, this code fails:
>
>        dvbdemux->feed = vmalloc(dvbdemux->feednum * sizeof(struct dvb_demux_feed));
>        if (!dvbdemux->feed) {
>                vfree(dvbdemux->filter);
>                return -ENOMEM;
>
> And the driver frees the memory. However, before the patch, it used to forget
> to reset dvbdemux->filter to NULL. Later, dvb_dmx_release() is called, and it
> tries to free an already freed memory, causing the OOPS.
>
> After applying those two patches, I can't see any other potential cause
> for the problem. Someone with aletv and a DVB-T signal with Teletext
> (it is not my case, I am at an ISDB-T area) should now take a look on
> what's the application is doing and why aletv-dvb is exhausting the computer
> memory used by vmalloc.


I applied these two patches again 2.6.33-rc6 but it wasn't different.
mplayer+alevt => oops.


>> I bisected this down to:
>>
>> root@scratchy:/data/kernel/linux-2.6# git bisect bad
>> 1cb662a3144992259edfd3cf9f54a6b25a913a0f is first bad commit
>> commit 1cb662a3144992259edfd3cf9f54a6b25a913a0f
>> Author: Andreas Oberritter <obi@linuxtv.org>
>> Date:   Tue Jul 14 20:28:50 2009 -0300
>>
>>     V4L/DVB (12275): Add two new ioctls: DMX_ADD_PID and DMX_REMOVE_PID
>> ...
>>
>>
>> Reverting the patch on top of 2.6.33-rc6, I can start mplayer and alevt
>> with no problems.
>
> If reverting this patch solves the issue, then I can see 2 possible reasons
> for the breakage:
> 	1) the behavior of the old ioctls changed a little bit for Teletext;
> 	2) your mplayer version has support to the new ioctls, and is doing
> something different. In this case, as aletv-dvb is not prepared for the
> changes, it hits some internal bug, and it is working badly.
>
> In any case, someone needs to dig at aletv and check what's happening there,
> identifying the root cause. So, the better is to contact aletv-dvb maintainer.
>
> It the error is due to (2), the fix is a patch to aletv-dvb. If it is (1),
> then a regression has occurred, and a patch to the kernel is needed. For someone
> to write such patch, he needs to know exactly where's the bug: e. g. what's the
> difference between the previous driver response and the new broken response.


mplayer is brand new and as I am using suse 11.1, alevt might be a little 
bit old.


thanks,

Thomas

Re: Kernel Oops, dvb_dmxdev_filter_reset, bisected

[Chicken Shack]

Am Montag, den 01.02.2010, 17:46 -0200 schrieb Mauro Carvalho Chehab:
> Thomas Voegtle wrote:
> > 
> > Hello,
> > 
> > yesterday I moved from 2.6.31.y to 2.6.32 and found a reproducable
> > kernel oops.
> > Bug is in Linus' tree, too.


Mauro,
To begin with: Thanks for your investigation and patch authoring of
course.

Please let me state you where you are definitely wrong (unfortunately):


> Please try the two patches I sent to the ML today that fixes two potential
> situations where the OOPS bug may hit.


I tried them. As I wrote already, patch 1 bewares the system from
hanging up itself completely if alevt-dvb is being started for the
second time after the kernel oops.

But what concerns the real problem of the patch in question and
alevt-dvb as application, none of the two patches even touches even one
of the 2 described root problems, unfortunately.

So everything is just the same, nothing essential has changed.

> I suspect that alevt-dvb is doing something wrong with the memory of the machine.


I guess that this assumption is wrong.


> The more likely case happens when there's no more available memory for vmalloc(). 
> In this case, this code fails:
> 
>         dvbdemux->feed = vmalloc(dvbdemux->feednum * sizeof(struct dvb_demux_feed));
>         if (!dvbdemux->feed) {
>                 vfree(dvbdemux->filter);
>                 return -ENOMEM;
> 
> And the driver frees the memory. However, before the patch, it used to forget
> to reset dvbdemux->filter to NULL. Later, dvb_dmx_release() is called, and it
> tries to free an already freed memory, causing the OOPS.


This is interesting to be read, but it is not the root problem in our
special case.


> After applying those two patches, I can't see any other potential cause
> for the problem. Someone with aletv and a DVB-T signal with Teletext
> (it is not my case, I am at an ISDB-T area) should now take a look on 
> what's the application is doing and why aletv-dvb is exhausting the computer 
> memory used by vmalloc.

Even if it is not propagated officially the DVB-patched versions of
alevt-dvb also run with DVB-S equipment. All you need to do is to feed
them with the correct ttpid (teletext pid) on the command line.

Depends on your distro, as a variety of versions is being spread....
If you wish I can send you my corrected and enhanced version that you
can compile in 30 seconds.....


> > I bisected this down to:
> > 
> > root@scratchy:/data/kernel/linux-2.6# git bisect bad
> > 1cb662a3144992259edfd3cf9f54a6b25a913a0f is first bad commit
> > commit 1cb662a3144992259edfd3cf9f54a6b25a913a0f
> > Author: Andreas Oberritter <obi@linuxtv.org>
> > Date:   Tue Jul 14 20:28:50 2009 -0300
> > 
> >     V4L/DVB (12275): Add two new ioctls: DMX_ADD_PID and DMX_REMOVE_PID
> > ...
> > 
> > 
> > Reverting the patch on top of 2.6.33-rc6, I can start mplayer and alevt
> > with no problems.
> 
> If reverting this patch solves the issue, then I can see 2 possible reasons
> for the breakage:
> 	1) the behavior of the old ioctls changed a little bit for Teletext;
> 	2) your mplayer version has support to the new ioctls, and is doing
> something different. In this case, as aletv-dvb is not prepared for the 
> changes, it hits some internal bug, and it is working badly.

That sounds very very good and realistic.
That's why I appended my version of alevt-dvb as attachment of my mail
to Andy Walls.


> In any case, someone needs to dig at aletv and check what's happening there, 
> identifying the root cause. So, the better is to contact aletv-dvb maintainer. 

This is erratic in so far as there is no official maintainer of
alevt-dvb.

The program was written about 10 years ago by a guy called Edgar
Toernig, arrivable under <froese@gmx.de>.

Edgar never felt responsible for the DVB patches of alevt which were
written by a guy from Switzerland called Thomas Sailer two years later
(2001).

There have been lots of contributions for this beautiful piece of
software: a russian and a greek codepage implementation, the rather
crude DVB-patch by Thomas Sailer, lots of fonts and bugfixes all over
the years.

Edgar Toernig, the original author, never felt responsible for the DVB
part of his software - if he ever "maintained" something, then it was
and still is the analogue part of the program.

But the DVB implementation exists since 9 years and it did its work up
until kernel 2.6.31 final.

With Obi's patch, signed by you and Brandon, the problems started,
introducing kernel 2.6.32-rc1.
There hasn't been any issue for about 9 years concerning the DVB part of
the program.......


So call the DVB part of this beautiful piece of software a stepchild and
you're bloody right....


> It the error is due to (2), the fix is a patch to aletv-dvb. If it is (1),
> then a regression has occurred, and a patch to the kernel is needed. For someone
> to write such patch, he needs to know exactly where's the bug: e. g. what's the
> difference between the previous driver response and the new broken response.


We need both:
1. A volunteer who dives into alevt-dvb: The DVB implementation is
rather primitive and limited to 2 files: vbi.c and vbi.h.

2. A volunteer who dives into the pitfalls of the new demux device and
writes a kernel patch....

In clear words: The new DVB demux device can serve a multiplicity of
filters for a multiplicity of parallel recordings for instance.

A teletext application is something rather primitive in comparison to
what the actual Demux device can manage:

Its needs are utmost spartan: The needed filter type DMX_PES_OTHER and
that's it! Nothing more!


SO IF the new DVB demux device can control a multiplicity of filters
(AUDIO, VIDEO, OTHER, etc.) since 2.6.32-rc1 then why the hell is it
incapable to control something absolutely primitive like a dumb teletext
application????

So this is my basic thesis (in question form) why the issue can never be
a simple application issue for itself (i. e. the fault is situated only
in the architecture of the application). 


> Cheers,
> Mauro

It would be rather crazy to throw his beautiful piece of software into
the trashcan out of time issues or newer API issues.

So please help! And if you need further info then please ask me!

Regards

CS


> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html