Re: kernel Oops: 0003 on usbhid_submit_report

Re: kernel Oops: 0003 on usbhid_submit_report

[Greg KH]

<adding linux-input@ where the hid developers live...>

On Mon, Aug 12, 2013 at 01:56:40PM +0200, Andreas Lillebø Holm wrote:
> When communicating with AT90USB1287, at random intervals (1/25 boots)
> the linux hid_output_field Oopses and kills the communicating thread.
> The AT90USB1287 microcontroller uses LUFA library for usb/hid
> communication. It is trigged by a ioctl call from userspace and fails
> in a kernel paging request.  The system is after the oops in a state
> where no hid commands is sent anymore and only a boot can fix the
> system.
> 
> Keywords: usbhid hid
> 
> Kernel version: Linux version 3.8.13-03081301-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201305311535 SMP Fri May 31 19:44:30 UTC 2013
> 
> Oopses:
> http://paste.debian.net/24305
> http://paste.debian.net/24306
> http://paste.debian.net/24307
> 
> Code:
> The error is triggered by:
> ioctl(fd,HIDIOCSUSAGES, &ref_multi_u);
> ioctl(fd,HIDIOCSREPORT, &rep_info_u);
> 
> Notes:
> It is very hard to reproduce so seems like race condition…
> 
> Any tips to resolve/workaround this issue is appreciated and please
> let me know if my information is incomplete (This is my first kernel
> bug report)

Any chance you can try a supported kernel, like 3.10.6 or 3.11-rc5 to
see if that also causes problems?  We can't do anything with
distro-specific kernel releases like your 3.8.13 release from Ubuntu,
sorry.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Andreas Lillebø Holm]

Hi,

On Tuesday, August 13, 2013 at 8:40 PM, Greg KH wrote:
> > When communicating with AT90USB1287, at random intervals (1/25 boots)
> > the linux hid_output_field Oopses and kills the communicating thread.
> > The AT90USB1287 microcontroller uses LUFA library for usb/hid
> > communication. It is trigged by a ioctl call from userspace and fails
> > in a kernel paging request. The system is after the oops in a state
> > where no hid commands is sent anymore and only a boot can fix the
> > system.
> >  
> > Keywords: usbhid hid
> >  
> > Kernel version: Linux version 3.8.13-03081301-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201305311535 SMP Fri May 31 19:44:30 UTC 2013
> >  
> > Oopses:
> > http://paste.debian.net/24305
> > http://paste.debian.net/24306
> > http://paste.debian.net/24307
> >  
> > Code:
> > The error is triggered by:
> > ioctl(fd,HIDIOCSUSAGES, &ref_multi_u);
> > ioctl(fd,HIDIOCSREPORT, &rep_info_u);
> >  
> > Notes:
> > It is very hard to reproduce so seems like race condition…
> >  
> > Any tips to resolve/workaround this issue is appreciated and please
> > let me know if my information is incomplete (This is my first kernel
> > bug report)
>  
>  
>  
>  
> Any chance you can try a supported kernel, like 3.10.6 or 3.11-rc5 to
> see if that also causes problems? We can't do anything with
> distro-specific kernel releases like your 3.8.13 release from Ubuntu,
> sorry.


I've now tried with kernel 3.10.6 (http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.10.6-saucy/) and I can trigger the same issue on this kernel.

Here is a paste from last Oops on this kernel:
http://paste.debian.net/24993/  

I am also using usbmon to monitor the usb bus traffic, but cannot see anything that should cause the driver to Oops.

Is there any way to find out what can trigger this issue?

Thanks
Andreas L. Holm


--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Greg KH]

On Wed, Aug 14, 2013 at 03:38:04PM +0200, Andreas Lillebø Holm wrote:
> Hi,
> 
> On Tuesday, August 13, 2013 at 8:40 PM, Greg KH wrote:
> > > When communicating with AT90USB1287, at random intervals (1/25 boots)
> > > the linux hid_output_field Oopses and kills the communicating thread.
> > > The AT90USB1287 microcontroller uses LUFA library for usb/hid
> > > communication. It is trigged by a ioctl call from userspace and fails
> > > in a kernel paging request. The system is after the oops in a state
> > > where no hid commands is sent anymore and only a boot can fix the
> > > system.
> > >  
> > > Keywords: usbhid hid
> > >  
> > > Kernel version: Linux version 3.8.13-03081301-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201305311535 SMP Fri May 31 19:44:30 UTC 2013
> > >  
> > > Oopses:
> > > http://paste.debian.net/24305
> > > http://paste.debian.net/24306
> > > http://paste.debian.net/24307
> > >  
> > > Code:
> > > The error is triggered by:
> > > ioctl(fd,HIDIOCSUSAGES, &ref_multi_u);
> > > ioctl(fd,HIDIOCSREPORT, &rep_info_u);
> > >  
> > > Notes:
> > > It is very hard to reproduce so seems like race condition…
> > >  
> > > Any tips to resolve/workaround this issue is appreciated and please
> > > let me know if my information is incomplete (This is my first kernel
> > > bug report)
> >  
> >  
> >  
> >  
> > Any chance you can try a supported kernel, like 3.10.6 or 3.11-rc5 to
> > see if that also causes problems? We can't do anything with
> > distro-specific kernel releases like your 3.8.13 release from Ubuntu,
> > sorry.
> 
> 
> I've now tried with kernel 3.10.6 (http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.10.6-saucy/) and I can trigger the same issue on this kernel.
> 
> Here is a paste from last Oops on this kernel:
> http://paste.debian.net/24993/  
> 
> I am also using usbmon to monitor the usb bus traffic, but cannot see anything that should cause the driver to Oops.
> 
> Is there any way to find out what can trigger this issue?

Jiri should be the one to know this, Jiri, any ideas?

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Andreas Lillebø Holm]

Hi,

I just wanna followup this issue a bit more.

We have now tested on several hardware platform, and it is only on one platform it fails. The platform it fails on is Atom Intel e6xx CPU with EG20T PCH platform controller. We do not observe it on the Intel i7 3517UE CPU with Intel 6/7 series system controller.

We see this error around 4 times over 100 boots / 2 hours runtime.

After the Oops our process is in uninterruptable sleep state (D state). The process is not killable  when in this state. The process also uses around 70% CPU, maybe because it is waiting for the ioctl system call to return?

We haven't found another way to recover from besides rebooting the system.

How can we proceed to fix this issue?

Andreas


On Thursday, August 15, 2013 at 8:37 AM, Greg KH wrote:

> > > > When communicating with AT90USB1287, at random intervals (1/25 boots)
> > > > the linux hid_output_field Oopses and kills the communicating thread.
> > > > The AT90USB1287 microcontroller uses LUFA library for usb/hid
> > > > communication. It is trigged by a ioctl call from userspace and fails
> > > > in a kernel paging request. The system is after the oops in a state
> > > > where no hid commands is sent anymore and only a boot can fix the
> > > > system.
> > > >  
> > > > Keywords: usbhid hid
> > > >  
> > > > Kernel version: Linux version 3.8.13-03081301-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201305311535 SMP Fri May 31 19:44:30 UTC 2013
> > > >  
> > > > Oopses:
> > > > http://paste.debian.net/24305
> > > > http://paste.debian.net/24306
> > > > http://paste.debian.net/24307
> > > >  
> > > > Code:
> > > > The error is triggered by:
> > > > ioctl(fd,HIDIOCSUSAGES, &ref_multi_u);
> > > > ioctl(fd,HIDIOCSREPORT, &rep_info_u);
> > > >  
> > > > Notes:
> > > > It is very hard to reproduce so seems like race condition…
> > > >  
> > > > Any tips to resolve/workaround this issue is appreciated and please
> > > > let me know if my information is incomplete (This is my first kernel
> > > > bug report)
> > >  
> > >  
> > > Any chance you can try a supported kernel, like 3.10.6 or 3.11-rc5 to
> > > see if that also causes problems? We can't do anything with
> > > distro-specific kernel releases like your 3.8.13 release from Ubuntu,
> > > sorry.
> >  
> >  
> > I've now tried with kernel 3.10.6 (http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.10.6-saucy/) and I can trigger the same issue on this kernel.
> >  
> > Here is a paste from last Oops on this kernel:
> > http://paste.debian.net/24993/  
> >  
> > I am also using usbmon to monitor the usb bus traffic, but cannot see anything that should cause the driver to Oops.
> >  
> > Is there any way to find out what can trigger this issue?

  


--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Benjamin Tissoires]

On Fri, Aug 23, 2013 at 12:52 PM, Andreas Lillebø Holm
<andreas-nr7O5T6ryCTQT0dZR+AlfA@public.gmane.org> wrote:
> Hi,
>
> I just wanna followup this issue a bit more.
>
> We have now tested on several hardware platform, and it is only on one platform it fails. The platform it fails on is Atom Intel e6xx CPU with EG20T PCH platform controller. We do not observe it on the Intel i7 3517UE CPU with Intel 6/7 series system controller.
>
> We see this error around 4 times over 100 boots / 2 hours runtime.
>
> After the Oops our process is in uninterruptable sleep state (D state). The process is not killable  when in this state. The process also uses around 70% CPU, maybe because it is waiting for the ioctl system call to return?
>
> We haven't found another way to recover from besides rebooting the system.
>
> How can we proceed to fix this issue?

IIRC, Jiri fixed some paging requests with the two following patches:
http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca
http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=bc197eedef1ae082ec662c64c3f4aa302821fb7a

At least, these two patches are related to hid_output_report(). I
think they are scheduled for 3.12, so I can not find them in Linus'
tree.

It would worth trying them :)

Cheers,
Benjamin

>
>
> On Thursday, August 15, 2013 at 8:37 AM, Greg KH wrote:
>
>> > > > When communicating with AT90USB1287, at random intervals (1/25 boots)
>> > > > the linux hid_output_field Oopses and kills the communicating thread.
>> > > > The AT90USB1287 microcontroller uses LUFA library for usb/hid
>> > > > communication. It is trigged by a ioctl call from userspace and fails
>> > > > in a kernel paging request. The system is after the oops in a state
>> > > > where no hid commands is sent anymore and only a boot can fix the
>> > > > system.
>> > > >
>> > > > Keywords: usbhid hid
>> > > >
>> > > > Kernel version: Linux version 3.8.13-03081301-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201305311535 SMP Fri May 31 19:44:30 UTC 2013
>> > > >
>> > > > Oopses:
>> > > > http://paste.debian.net/24305
>> > > > http://paste.debian.net/24306
>> > > > http://paste.debian.net/24307
>> > > >
>> > > > Code:
>> > > > The error is triggered by:
>> > > > ioctl(fd,HIDIOCSUSAGES, &ref_multi_u);
>> > > > ioctl(fd,HIDIOCSREPORT, &rep_info_u);
>> > > >
>> > > > Notes:
>> > > > It is very hard to reproduce so seems like race condition…
>> > > >
>> > > > Any tips to resolve/workaround this issue is appreciated and please
>> > > > let me know if my information is incomplete (This is my first kernel
>> > > > bug report)
>> > >
>> > >
>> > > Any chance you can try a supported kernel, like 3.10.6 or 3.11-rc5 to
>> > > see if that also causes problems? We can't do anything with
>> > > distro-specific kernel releases like your 3.8.13 release from Ubuntu,
>> > > sorry.
>> >
>> >
>> > I've now tried with kernel 3.10.6 (http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.10.6-saucy/) and I can trigger the same issue on this kernel.
>> >
>> > Here is a paste from last Oops on this kernel:
>> > http://paste.debian.net/24993/
>> >
>> > I am also using usbmon to monitor the usb bus traffic, but cannot see anything that should cause the driver to Oops.
>> >
>> > Is there any way to find out what can trigger this issue?
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-input" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Jiri Kosina]

On Fri, 23 Aug 2013, Benjamin Tissoires wrote:

> IIRC, Jiri fixed some paging requests with the two following patches:
> http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca
> http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=bc197eedef1ae082ec662c64c3f4aa302821fb7a
> 
> At least, these two patches are related to hid_output_report(). I
> think they are scheduled for 3.12, so I can not find them in Linus'
> tree.
> 
> It would worth trying them :)

All three reported oopses happened on page boundary exactly, so this 
pretty much looks like my implement() fix is the one.

Andreas, could you please apply

	http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca

and report back whether it makes your problem go away?

Thanks,

-- 
Jiri Kosina
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Andreas Lillebø Holm]

>  
> All three reported oopses happened on page boundary exactly, so this  
> pretty much looks like my implement() fix is the one.
>  
> Andreas, could you please apply
>  
> http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca
>  
> and report back whether it makes your problem go away?

I will build and test this patch and hopefully it will resolve my issues :)

--  
Andreas Lillebø Holm


--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Andreas Lillebø Holm]

On Friday, August 23, 2013 at 2:49 PM, Jiri Kosina wrote:
> On Fri, 23 Aug 2013, Benjamin Tissoires wrote:
>  
> > IIRC, Jiri fixed some paging requests with the two following patches:
> > http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca
> > http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=bc197eedef1ae082ec662c64c3f4aa302821fb7a
> >  
> > At least, these two patches are related to hid_output_report(). I
> > think they are scheduled for 3.12, so I can not find them in Linus'
> > tree.
> >  
> > It would worth trying them :)
>  
> All three reported oopses happened on page boundary exactly, so this  
> pretty much looks like my implement() fix is the one.
>  
> Andreas, could you please apply
>  
> http://git.kernel.org/cgit/linux/kernel/git/jikos/hid.git/commit/?h=for-next&id=27ce405039bfe6d3f4143415c638f56a3df77dca
>  
> and report back whether it makes your problem go away?
Hi,

You are my hero Jiri :-) These two patches fixes the issues I was seeing and I can no longer reproduce them.

I have only tried them on one hardware platform yet; the one that I had failing: Atom Intel e6xx CPU with EG20T PCH platform controller. I have had approximately 300 boots in our test rigg now and no fails yet. Before was the failrate 1/25 boots.

I made the kernel based on 3.10.9 stable tag and applied them directly on there. I do recommend that these patches makes it way into the stable tree :-)

Thanks again :)

Best regards
Andreas Lillebø Holm

--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel Oops: 0003 on usbhid_submit_report

[Jiri Kosina]

On Mon, 26 Aug 2013, Andreas Lillebø Holm wrote:

> You are my hero Jiri :-) These two patches fixes the issues I was seeing 
> and I can no longer reproduce them.

Thanks a lot for confirmation.

This has been a very long-standing bug, but for some reasons I have 
started to see devices actually triggering it only now.

My plan is to push it to Linus in the upcoming merge window, and once it 
proves to be stable, I'll do a -stable backport.

Thanks again,

-- 
Jiri Kosina
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html