Re: USB disconnect -> kernel panic

Re: USB disconnect -> kernel panic

[Greg KH]

On Mon, Jan 02, 2012 at 10:22:55AM -0800, Sarah Sharp wrote:
> On Sat, Dec 24, 2011 at 03:32:26PM -0700, Bdale Garbee wrote:
> > Hi Sarah!  Merry Christmas!
> > 
> > I've been working on another production run of Altus Metrum products
> > today, and in the process of flashing and calibrating a pile of our
> > TeleDongle units, I've been seeing sporadic kernel panics when
> > unplugging them from my notebook USB.  Has now happened three times this
> > morning. 
> > 
> > I'm running 3.1.6 now, but I saw this under 3.1.5 too, at least once.
> > The notebook is an HP 2540p.
> > 
> > Keith suggested I send you the attached photo of my screen showing the
> > panic info. 
> 
> It's somewhere in the tty layer, and doesn't look USB related.  I think
> Greg KH is the tty maintainer right now, so you might want to ask him.

We've had some cdc_acm driver problems in the past, but I thought they
were all resolved.  3.1.3 should have fixed those problems.  We have
some more cleanups queued up for 3.3, but nothing that should be fixing
something like this.

Oliver, any thoughts about this (see the original post on linux-usb for
the crash dump.)

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB disconnect -> kernel panic

[Bdale Garbee]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]

On Tue, 3 Jan 2012 15:58:47 -0800, Greg KH <greg-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> wrote:
> On Mon, Jan 02, 2012 at 10:22:55AM -0800, Sarah Sharp wrote:
> > On Sat, Dec 24, 2011 at 03:32:26PM -0700, Bdale Garbee wrote:
> > > Hi Sarah!  Merry Christmas!
> > > 
> > > I've been working on another production run of Altus Metrum products
> > > today, and in the process of flashing and calibrating a pile of our
> > > TeleDongle units, I've been seeing sporadic kernel panics when
> > > unplugging them from my notebook USB.  Has now happened three times this
> > > morning. 
> > > 
> > > I'm running 3.1.6 now, but I saw this under 3.1.5 too, at least once.
> > > The notebook is an HP 2540p.
> > > 
> > > Keith suggested I send you the attached photo of my screen showing the
> > > panic info. 
> > 
> > It's somewhere in the tty layer, and doesn't look USB related.  I think
> > Greg KH is the tty maintainer right now, so you might want to ask him.
> 
> We've had some cdc_acm driver problems in the past, but I thought they
> were all resolved.  3.1.3 should have fixed those problems.  We have
> some more cleanups queued up for 3.3, but nothing that should be fixing
> something like this.
> 
> Oliver, any thoughts about this (see the original post on linux-usb for
> the crash dump.)

For what it's worth, it happened to me again today running 3.2.0-rc7.  I
didn't bother photographing the screen again, though I'm happy to do
that if it'll help.

Bdale

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: USB disconnect -> kernel panic

[Greg KH]

On Tue, Jan 03, 2012 at 08:09:03PM -0700, Bdale Garbee wrote:
> On Tue, 3 Jan 2012 15:58:47 -0800, Greg KH <greg-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> wrote:
> > On Mon, Jan 02, 2012 at 10:22:55AM -0800, Sarah Sharp wrote:
> > > On Sat, Dec 24, 2011 at 03:32:26PM -0700, Bdale Garbee wrote:
> > > > Hi Sarah!  Merry Christmas!
> > > > 
> > > > I've been working on another production run of Altus Metrum products
> > > > today, and in the process of flashing and calibrating a pile of our
> > > > TeleDongle units, I've been seeing sporadic kernel panics when
> > > > unplugging them from my notebook USB.  Has now happened three times this
> > > > morning. 
> > > > 
> > > > I'm running 3.1.6 now, but I saw this under 3.1.5 too, at least once.
> > > > The notebook is an HP 2540p.
> > > > 
> > > > Keith suggested I send you the attached photo of my screen showing the
> > > > panic info. 
> > > 
> > > It's somewhere in the tty layer, and doesn't look USB related.  I think
> > > Greg KH is the tty maintainer right now, so you might want to ask him.
> > 
> > We've had some cdc_acm driver problems in the past, but I thought they
> > were all resolved.  3.1.3 should have fixed those problems.  We have
> > some more cleanups queued up for 3.3, but nothing that should be fixing
> > something like this.
> > 
> > Oliver, any thoughts about this (see the original post on linux-usb for
> > the crash dump.)
> 
> For what it's worth, it happened to me again today running 3.2.0-rc7.  I
> didn't bother photographing the screen again, though I'm happy to do
> that if it'll help.

Ok, can you try the patch below and let me know if it solves the problem
or not?

thanks,

greg k-h

-------------------
From: Thilo-Alexander Ginkel <thilo-6Cr7QIIRUuHQT0dZR+AlfA@public.gmane.org>
Subject: usb: cdc-acm: Fix acm_tty_hangup() vs.  acm_tty_close() race

There is a race condition involving acm_tty_hangup() and acm_tty_close() where
hangup() would attempt to access tty->driver_data without proper locking and
NULL checking after close() has potentially already set it to NULL.
One possibility to (sporadically) trigger this behavior is to perform a
suspend/resume cycle with a running WWAN data connection.

This patch addresses the issue by introducing a NULL check for
tty->driver_data in acm_tty_hangup() protected by open_mutex and exiting
gracefully when hangup() is invoked on a device that has already been closed.

Signed-off-by: Thilo-Alexander Ginkel <thilo-6Cr7QIIRUuHQT0dZR+AlfA@public.gmane.org>
---
 drivers/usb/class/cdc-acm.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/class/cdc-acm.c b/drivers/usb/class/cdc-acm.c
index a8078d0..97f2e58 100644
--- a/drivers/usb/class/cdc-acm.c
+++ b/drivers/usb/class/cdc-acm.c
@@ -554,10 +554,18 @@ static void acm_port_down(struct acm *acm)
 
 static void acm_tty_hangup(struct tty_struct *tty)
 {
-	struct acm *acm = tty->driver_data;
-	tty_port_hangup(&acm->port);
+	struct acm *acm;
+
 	mutex_lock(&open_mutex);
+	acm = tty->driver_data;
+
+	if (!acm)
+		goto out;
+
+	tty_port_hangup(&acm->port);
 	acm_port_down(acm);
+
+out:
 	mutex_unlock(&open_mutex);
 }
 
-- 
1.7.5.4

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB disconnect -> kernel panic

[Alan Cox]

> Ok, can you try the patch below and let me know if it solves the problem
> or not?

If it does the right fix is to stop clearing tty->driver_data. If
tty->driver_data can go NULL outside of the final release then its
probably an exploitable hole.

Re: USB disconnect -> kernel panic

[Greg KH]

On Wed, Jan 04, 2012 at 11:26:59PM +0000, Alan Cox wrote:
> > Ok, can you try the patch below and let me know if it solves the problem
> > or not?
> 
> If it does the right fix is to stop clearing tty->driver_data. If
> tty->driver_data can go NULL outside of the final release then its
> probably an exploitable hole.

So in acm_tty_close() we shouldn't set driver_data to NULL after we
unregister the device?

Ah, that should happen in tty_operations->cleanup()?  It seems that
cdc_acm.c doesn't have that function, which is causing this problem now,
right?

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB disconnect -> kernel panic

[Bdale Garbee]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

On Wed, 4 Jan 2012 15:01:49 -0800, Greg KH <greg@kroah.com> wrote:
> Ok, can you try the patch below and let me know if it solves the problem
> or not?

It's hard to prove a negative, but after applying the patch to 3.2.0-rc7
and running with it for a while doing frequent power cycles on our
devices attached via USB, abruptly disconnecting USB during active
serial connections, etc .. I haven't seen the panic again.

Will report if it happens again with the patch in place, but so far so good.

Bdale

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: USB disconnect -> kernel panic

[Greg KH]

On Wed, Jan 04, 2012 at 10:36:48PM -0700, Bdale Garbee wrote:
> On Wed, 4 Jan 2012 15:01:49 -0800, Greg KH <greg-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> wrote:
> > Ok, can you try the patch below and let me know if it solves the problem
> > or not?
> 
> It's hard to prove a negative, but after applying the patch to 3.2.0-rc7
> and running with it for a while doing frequent power cycles on our
> devices attached via USB, abruptly disconnecting USB during active
> serial connections, etc .. I haven't seen the panic again.
> 
> Will report if it happens again with the patch in place, but so far so good.

Thanks, I'll queue this up then, and work to solve this in the "correct"
way in a week or so.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB disconnect -> kernel panic

[Oliver Neukum]

Am Mittwoch, 4. Januar 2012, 00:58:47 schrieb Greg KH:
> On Mon, Jan 02, 2012 at 10:22:55AM -0800, Sarah Sharp wrote:
> > On Sat, Dec 24, 2011 at 03:32:26PM -0700, Bdale Garbee wrote:
> > > Hi Sarah!  Merry Christmas!
> > > 
> > > I've been working on another production run of Altus Metrum products
> > > today, and in the process of flashing and calibrating a pile of our
> > > TeleDongle units, I've been seeing sporadic kernel panics when
> > > unplugging them from my notebook USB.  Has now happened three times this
> > > morning. 
> > > 
> > > I'm running 3.1.6 now, but I saw this under 3.1.5 too, at least once.
> > > The notebook is an HP 2540p.
> > > 
> > > Keith suggested I send you the attached photo of my screen showing the
> > > panic info. 
> > 
> > It's somewhere in the tty layer, and doesn't look USB related.  I think
> > Greg KH is the tty maintainer right now, so you might want to ask him.
> 
> We've had some cdc_acm driver problems in the past, but I thought they
> were all resolved.  3.1.3 should have fixed those problems.  We have
> some more cleanups queued up for 3.3, but nothing that should be fixing
> something like this.
> 
> Oliver, any thoughts about this (see the original post on linux-usb for
> the crash dump.)

Do you have a pointer? I assumed this to be this issue:
http://www.spinics.net/lists/linux-usb/msg55779.html
And I am waiting for the announced patch.

	Regards
		Oliver
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: USB disconnect -> kernel panic

[Keith Packard]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

On Wed, 4 Jan 2012 10:01:57 +0100, Oliver Neukum <oneukum@suse.de> wrote:

> Do you have a pointer? I assumed this to be this issue:
> http://www.spinics.net/lists/linux-usb/msg55779.html
> And I am waiting for the announced patch.

The patch was posted to the list about half an hour after the Thilo's
first message:

From:	Thilo-Alexander Ginkel <thilo@ginkel.com>
Subject: [PATCH 1/1] usb: cdc-acm: Fix acm_tty_hangup() vs. acm_tty_close() race
Date:	Sat, 17 Dec 2011 10:55:10 +0100
Message-Id: <1324115710-14756-1-git-send-email-thilo@ginkel.com>

-- 
keith.packard@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]