Re: kernel locks due to USB I/O

[Greg Kroah-Hartman <gregkh@linuxfoundation.org>]

On Tue, Nov 10, 2020 at 06:42:17PM -0500, Alberto Sentieri wrote:
> 1) The current Ubuntu Kernel is 5.4.0-53. Do you want me to upgrade it to
> 5.9, from kernel.org? Or is there a Ubuntu 5.9 package that I can use? It
> would be easy to do it If there is a Ubuntu package with 5.9, which I would
> install and, after the tests, uninstall.

There might be a Ubuntu package, we don't know.

The issue is that the Ubuntu kernel you are using is an old one, and we
(the community) have no idea what is in there.  We can help you if you
use a kernel.org release, otherwise you really should get support from
Ubuntu as you are relying on them for this type of thing (and probably
paying them for support as you are using it in a corporate setting and
relying on it, right?)

> 2) Why do you believe that 5.9 would solve the problem? I am asking that
> because I cannot change the production machine for a test if I cannot go
> back to the original state. There is always a risk involved.

You should always be able to choose a different kernel version at boot
time, no need to ever get rid of your older one in case something goes
wrong.

There has been lots of changes in the past year of kernel development,
as always, from 5.4 to 5.9, so something that we did then might have
resolved this.  Maybe not, but we don't know if you can't try as to try
to remember what we did a year ago is hard, as I am sure you can imagine
:)

> 3) It is one single thread dealing with all 36 devices. Each device has its
> own co-routine (not preemptive), but all co-routines are executed by a
> unique thread.

Are you sure that these threads are not just getting stalled somehow
when a response doesn't come back properly and are not timing out, which
your userspace co-routines should be doing if the bus stalls, right?

One thing to note, USB hubs are almost never tested with that many
devices at once, so you might have to upgrade to one that says it really
will work with that many devices, and provide enough power to them.  We
have seen more low-power issues on USB hubs and connections over the
years to know that this is a real issue, and one that the OS can't do
much about, but can affect it.

> 4) By network console, do you mean ssh? It dies as well when it locks. The
> screen is the regular GNOME3 screen and nothing can be seen there. Every
> time it locks they send a picture, and I cannot see anything meaningful
> there. I am thinking about disabling GNOME3, but I need their blessing for
> that.

No, netconsole is something else, see the kernel documentation for the
full details.

thanks,

greg k-h