* USB hangs @ 2004-01-11 0:07 Alan Cox 2004-01-11 0:23 ` Matthew Dharm 2004-01-11 18:46 ` Lukas Postupa 0 siblings, 2 replies; 25+ messages in thread From: Alan Cox @ 2004-01-11 0:07 UTC (permalink / raw) To: Marcelo Tosatti, Linux Kernel Mailing List [-- Attachment #1: Type: text/plain, Size: 623 bytes --] With the various fixes people had been posting USB storage writing was still hanging repeatedly when doing a 20Gb rsync to usb-storage disks with a low memory system. Doing things like while(true) sync() made it hang even more often. After a bit of digging the following seems to fix it Not sure if 2.6 needs this as well. The failure path seems to be ->scsi_done in the USB storage thread issues a new command causes USB to kmalloc GFP_KERNEL causes a page out queues a page out to the USB storage thread Deadlock. Setting PF_MEMALLOC should stop the storage thread ever causing pageout itself so deadlocking. [-- Attachment #2: a1 --] [-- Type: text/plain, Size: 335 bytes --] --- drivers/usb/storage/usb.c~ 2004-01-09 02:06:35.000000000 +0000 +++ drivers/usb/storage/usb.c 2004-01-09 02:06:35.000000000 +0000 @@ -332,6 +332,8 @@ /* set our name for identification purposes */ sprintf(current->comm, "usb-storage-%d", us->host_number); + + current->flags |= PF_MEMALLOC; unlock_kernel(); ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: USB hangs 2004-01-11 0:07 USB hangs Alan Cox @ 2004-01-11 0:23 ` Matthew Dharm 2004-01-11 0:49 ` [linux-usb-devel] " Oliver Neukum 2004-01-11 2:33 ` Alan Cox 2004-01-11 18:46 ` Lukas Postupa 1 sibling, 2 replies; 25+ messages in thread From: Matthew Dharm @ 2004-01-11 0:23 UTC (permalink / raw) To: Alan Cox Cc: Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH [-- Attachment #1: Type: text/plain, Size: 1459 bytes --] Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those down and eliminated them. Matt On Sun, Jan 11, 2004 at 12:07:17AM +0000, Alan Cox wrote: > With the various fixes people had been posting USB storage > writing was still hanging repeatedly when doing a 20Gb rsync > to usb-storage disks with a low memory system. Doing things > like while(true) sync() made it hang even more often. > > After a bit of digging the following seems to fix it > > Not sure if 2.6 needs this as well. > > The failure path seems to be > > ->scsi_done in the USB storage thread > issues a new command > causes USB to kmalloc GFP_KERNEL > causes a page out > queues a page out to the USB storage thread > Deadlock. > > Setting PF_MEMALLOC should stop the storage thread ever causing pageout > itself so deadlocking. > > --- drivers/usb/storage/usb.c~ 2004-01-09 02:06:35.000000000 +0000 > +++ drivers/usb/storage/usb.c 2004-01-09 02:06:35.000000000 +0000 > @@ -332,6 +332,8 @@ > > /* set our name for identification purposes */ > sprintf(current->comm, "usb-storage-%d", us->host_number); > + > + current->flags |= PF_MEMALLOC; > > unlock_kernel(); > -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver Your lips are twitching. You're playing Quake aren't you. -- Stef to Greg User Friendly, 8/11/1998 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 0:23 ` Matthew Dharm @ 2004-01-11 0:49 ` Oliver Neukum 2004-01-11 1:01 ` Matthew Dharm 2004-01-11 1:40 ` David Brownell 2004-01-11 2:33 ` Alan Cox 1 sibling, 2 replies; 25+ messages in thread From: Oliver Neukum @ 2004-01-11 0:49 UTC (permalink / raw) To: Matthew Dharm, Alan Cox Cc: Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Am Sonntag, 11. Januar 2004 01:23 schrieb Matthew Dharm: > Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > down and eliminated them. > static int ohci_mem_init (struct ohci *ohci) { ohci->td_cache = pci_pool_create ("ohci_td", ohci->ohci_dev, sizeof (struct td), 32 /* byte alignment */, 0 /* no page-crossing issues */, GFP_KERNEL | OHCI_MEM_FLAGS); if (!ohci->td_cache) return -ENOMEM; ohci->dev_cache = pci_pool_create ("ohci_dev", ohci->ohci_dev, sizeof (struct ohci_device), 16 /* byte alignment */, 0 /* no page-crossing issues */, GFP_KERNEL | OHCI_MEM_FLAGS); if (!ohci->dev_cache) return -ENOMEM; return 0; } This one here looks dangerous. Regards Oliver ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 0:49 ` [linux-usb-devel] " Oliver Neukum @ 2004-01-11 1:01 ` Matthew Dharm 2004-01-11 1:06 ` Oliver Neukum 2004-01-11 1:40 ` David Brownell 1 sibling, 1 reply; 25+ messages in thread From: Matthew Dharm @ 2004-01-11 1:01 UTC (permalink / raw) To: Oliver Neukum Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH [-- Attachment #1: Type: text/plain, Size: 1238 bytes --] On Sun, Jan 11, 2004 at 01:49:34AM +0100, Oliver Neukum wrote: > Am Sonntag, 11. Januar 2004 01:23 schrieb Matthew Dharm: > > Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > > down and eliminated them. > > > > static int ohci_mem_init (struct ohci *ohci) > { > ohci->td_cache = pci_pool_create ("ohci_td", ohci->ohci_dev, > sizeof (struct td), > 32 /* byte alignment */, > 0 /* no page-crossing issues */, > GFP_KERNEL | OHCI_MEM_FLAGS); > if (!ohci->td_cache) > return -ENOMEM; > ohci->dev_cache = pci_pool_create ("ohci_dev", ohci->ohci_dev, > sizeof (struct ohci_device), > 16 /* byte alignment */, > 0 /* no page-crossing issues */, > GFP_KERNEL | OHCI_MEM_FLAGS); > if (!ohci->dev_cache) > return -ENOMEM; > return 0; > } > > This one here looks dangerous. I'll agree that it looks dangerous, tho pci_pool_create() is something I know little about. Is this 2.4 or 2.6 code here? Matt -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver Okay, this isn't funny anymore! Let me down! I'll tell Bill on you!! -- Microsoft Salesman User Friendly, 4/1/1998 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 1:01 ` Matthew Dharm @ 2004-01-11 1:06 ` Oliver Neukum 0 siblings, 0 replies; 25+ messages in thread From: Oliver Neukum @ 2004-01-11 1:06 UTC (permalink / raw) To: Matthew Dharm Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH > I'll agree that it looks dangerous, tho pci_pool_create() is something I > know little about. > > Is this 2.4 or 2.6 code here? 2.4 / usb-ohci.h Regards Oliver ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 0:49 ` [linux-usb-devel] " Oliver Neukum 2004-01-11 1:01 ` Matthew Dharm @ 2004-01-11 1:40 ` David Brownell 1 sibling, 0 replies; 25+ messages in thread From: David Brownell @ 2004-01-11 1:40 UTC (permalink / raw) To: Oliver Neukum Cc: Matthew Dharm, Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Oliver Neukum wrote: > Am Sonntag, 11. Januar 2004 01:23 schrieb Matthew Dharm: > >>Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those >>down and eliminated them. >> > > > static int ohci_mem_init (struct ohci *ohci) > { > ... > } > > This one here looks dangerous. Since that happens as part of controller initialization, long before usb-storage could possibly come into play ... it doesn't seem to me like even a remote concern! ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: USB hangs 2004-01-11 0:23 ` Matthew Dharm 2004-01-11 0:49 ` [linux-usb-devel] " Oliver Neukum @ 2004-01-11 2:33 ` Alan Cox 2004-01-11 8:02 ` [linux-usb-devel] " Oliver Neukum 2004-01-11 23:25 ` David Brownell 1 sibling, 2 replies; 25+ messages in thread From: Alan Cox @ 2004-01-11 2:33 UTC (permalink / raw) To: Matthew Dharm Cc: Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Sul, 2004-01-11 at 00:23, Matthew Dharm wrote: > Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > down and eliminated them. Not sure. I just worked from tracebacks. I needed it to work rather than having the time to go hunting for specific faults. Plus I'd argue PF_MEMALLOC is a better solution anyway. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 2:33 ` Alan Cox @ 2004-01-11 8:02 ` Oliver Neukum 2004-01-11 22:39 ` Alan Cox 2004-01-11 23:25 ` David Brownell 1 sibling, 1 reply; 25+ messages in thread From: Oliver Neukum @ 2004-01-11 8:02 UTC (permalink / raw) To: Alan Cox, Matthew Dharm Cc: Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Am Sonntag, 11. Januar 2004 03:33 schrieb Alan Cox: > On Sul, 2004-01-11 at 00:23, Matthew Dharm wrote: > > Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > > down and eliminated them. > > Not sure. I just worked from tracebacks. I needed it to work rather > than having the time to go hunting for specific faults. Plus I'd > argue PF_MEMALLOC is a better solution anyway. Until recently this line from usb-ohci.h read GFP_KERNEL instead of GFP_NOIO #define ALLOC_FLAGS (in_interrupt () || current->state != TASK_RUNNING ? GFP_ATOMIC : GFP_NOIO) Was it an earlier kernel without that change? Regards Oliver ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 8:02 ` [linux-usb-devel] " Oliver Neukum @ 2004-01-11 22:39 ` Alan Cox 2004-01-11 23:29 ` Oliver Neukum 0 siblings, 1 reply; 25+ messages in thread From: Alan Cox @ 2004-01-11 22:39 UTC (permalink / raw) To: Oliver Neukum Cc: Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Sul, 2004-01-11 at 08:02, Oliver Neukum wrote: > Until recently this line from usb-ohci.h read GFP_KERNEL instead of GFP_NOIO > > #define ALLOC_FLAGS (in_interrupt () || current->state != TASK_RUNNING ? GFP_ATOMIC : GFP_NOIO) > > Was it an earlier kernel without that change? Its an UHCI controller. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 22:39 ` Alan Cox @ 2004-01-11 23:29 ` Oliver Neukum 2004-01-12 15:53 ` Alan Stern 0 siblings, 1 reply; 25+ messages in thread From: Oliver Neukum @ 2004-01-11 23:29 UTC (permalink / raw) To: Alan Cox Cc: Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Am Sonntag, 11. Januar 2004 23:39 schrieb Alan Cox: > On Sul, 2004-01-11 at 08:02, Oliver Neukum wrote: > > Until recently this line from usb-ohci.h read GFP_KERNEL instead of GFP_NOIO > > > > #define ALLOC_FLAGS (in_interrupt () || current->state != TASK_RUNNING ? GFP_ATOMIC : GFP_NOIO) > > > > Was it an earlier kernel without that change? > > Its an UHCI controller. OK. I got some more. Greg please apply. Regards Oliver You can import this changeset into BK by piping this whole message to: '| bk receive [path to repository]' or apply the patch as usual. =================================================================== ChangeSet@1.1218, 2004-01-12 00:26:13+01:00, oliver@vermuden.neukum.org - avoid GFP_KERNEL in block IO path usb.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff -Nru a/drivers/usb/usb.c b/drivers/usb/usb.c --- a/drivers/usb/usb.c Mon Jan 12 00:27:37 2004 +++ b/drivers/usb/usb.c Mon Jan 12 00:27:37 2004 @@ -1198,7 +1198,7 @@ int usb_control_msg(struct usb_device *dev, unsigned int pipe, __u8 request, __u8 requesttype, __u16 value, __u16 index, void *data, __u16 size, int timeout) { - struct usb_ctrlrequest *dr = kmalloc(sizeof(struct usb_ctrlrequest), GFP_KERNEL); + struct usb_ctrlrequest *dr = kmalloc(sizeof(struct usb_ctrlrequest), GFP_NOIO); int ret; if (!dr) @@ -1958,7 +1958,7 @@ if (result < 0) return result; - buffer = kmalloc(sizeof(status), GFP_KERNEL); + buffer = kmalloc(sizeof(status), GFP_NOIO); if (!buffer) { err("unable to allocate memory for configuration descriptors"); return -ENOMEM; =================================================================== This BitKeeper patch contains the following changesets: 1.1218 ## Wrapped with gzip_uu ## begin 664 bkpatch26304 M'XL(`.G;`4```[V4;V_3,!#&7\>?XJ2]V1A-[FPG:8*""EL9U::U*NSUE#IN M4Z5I1OX,@?+A<1M4F!H0#(G(D2/?^7>/SX]R`G>5+D.KV*P?=<E.X'U1U:%E MOO,FT5M[JYNLR>VB7)G8O"A,S$F+7#O=!N=CJ77EK$J]XI*9E%E<JQ1,I`HM MLL5AI?[RH$-K/KZZNWDS9RR*X"*-MRO]0=<016R1C9)&;^RL+.)T5ZT]A%N. M2.@B<2$]Y"WZ2&[K^4GL2X&!Z\J$-+%.SZA'^%.41"(DCWO";VGHRX!=`MG$ M:0@H'22'.""&W`M)G".%B/!K-)P3#)"]A7_7?\$4#"!^+-8)7+V;W5^/Y[?C M&UAO8;$I5`:3*3S$=<JNP?`XL=F/!K+!7SZ,88SL=8_JI-P=M7*::K%[;?63 M>I>,>BFD'[3#P(_=6+E2J2%*J7_3HG[D[A8,DGN$K4`,Y-X21ZG]UGBF2+9S MZ:C#J"+OQP@<"D*/^^098<*7>WM(<60._`-S<!CP_VR.KIE3&)2?]\-<]NRX MK\]PS"5Q)"`V^3Y;55TVJ@;#NU=UN2GUIT97-;Q(2H@@R^.-479:K;_J8GG: MGWOV<G^8V^ED>O;*%`B\KD`W6XMFN=2]L+ANJJ>;#_\9E6J554T>Q<E22"67 *[!M1:89=X@0````` ` end ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 23:29 ` Oliver Neukum @ 2004-01-12 15:53 ` Alan Stern 0 siblings, 0 replies; 25+ messages in thread From: Alan Stern @ 2004-01-12 15:53 UTC (permalink / raw) To: Oliver Neukum Cc: Alan Cox, Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Mon, 12 Jan 2004, Oliver Neukum wrote: > diff -Nru a/drivers/usb/usb.c b/drivers/usb/usb.c > --- a/drivers/usb/usb.c Mon Jan 12 00:27:37 2004 > +++ b/drivers/usb/usb.c Mon Jan 12 00:27:37 2004 > @@ -1198,7 +1198,7 @@ > int usb_control_msg(struct usb_device *dev, unsigned int pipe, __u8 request, __u8 requesttype, > __u16 value, __u16 index, void *data, __u16 size, int timeout) > { > - struct usb_ctrlrequest *dr = kmalloc(sizeof(struct usb_ctrlrequest), GFP_KERNEL); > + struct usb_ctrlrequest *dr = kmalloc(sizeof(struct usb_ctrlrequest), GFP_NOIO); > int ret; > > if (!dr) > @@ -1958,7 +1958,7 @@ > if (result < 0) > return result; > > - buffer = kmalloc(sizeof(status), GFP_KERNEL); > + buffer = kmalloc(sizeof(status), GFP_NOIO); > if (!buffer) { > err("unable to allocate memory for configuration descriptors"); > return -ENOMEM; Note that these changes have essentially already been incorporated into 2.6, so only 2.4 needs updating. Alan Stern ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 2:33 ` Alan Cox 2004-01-11 8:02 ` [linux-usb-devel] " Oliver Neukum @ 2004-01-11 23:25 ` David Brownell 2004-01-11 23:31 ` Matthew Dharm 2004-01-11 23:33 ` Oliver Neukum 1 sibling, 2 replies; 25+ messages in thread From: David Brownell @ 2004-01-11 23:25 UTC (permalink / raw) To: Alan Cox Cc: Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Alan Cox wrote: > On Sul, 2004-01-11 at 00:23, Matthew Dharm wrote: > >>Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those >>down and eliminated them. > > > Not sure. I just worked from tracebacks. I needed it to work rather > than having the time to go hunting for specific faults. Plus I'd > argue PF_MEMALLOC is a better solution anyway. It certainly seems like a more comprehensive fix for that particular class of problems! :) ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 23:25 ` David Brownell @ 2004-01-11 23:31 ` Matthew Dharm 2004-01-12 4:11 ` David Brownell 2004-01-11 23:33 ` Oliver Neukum 1 sibling, 1 reply; 25+ messages in thread From: Matthew Dharm @ 2004-01-11 23:31 UTC (permalink / raw) To: David Brownell Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH [-- Attachment #1: Type: text/plain, Size: 1012 bytes --] On Sun, Jan 11, 2004 at 03:25:06PM -0800, David Brownell wrote: > Alan Cox wrote: > >On Sul, 2004-01-11 at 00:23, Matthew Dharm wrote: > > > >>Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > >>down and eliminated them. > > > > > >Not sure. I just worked from tracebacks. I needed it to work rather > >than having the time to go hunting for specific faults. Plus I'd > >argue PF_MEMALLOC is a better solution anyway. > > It certainly seems like a more comprehensive fix for that > particular class of problems! :) Is it really more comprehensive? As I see it, it will only affect code executed in the context of the usb-storage thread. But, what about code which is invoked in tasklets or other contexts? Matt -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver I'm just trying to think of a way to say "up yours" without getting fired. -- Stef User Friendly, 10/8/1998 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 23:31 ` Matthew Dharm @ 2004-01-12 4:11 ` David Brownell 2004-01-12 7:39 ` Matthew Dharm 0 siblings, 1 reply; 25+ messages in thread From: David Brownell @ 2004-01-12 4:11 UTC (permalink / raw) To: Matthew Dharm Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH >>> Plus I'd >>>argue PF_MEMALLOC is a better solution anyway. >> >>It certainly seems like a more comprehensive fix for that >>particular class of problems! :) > > > Is it really more comprehensive? As I see it, it will only affect code > executed in the context of the usb-storage thread. But, what about code > which is invoked in tasklets or other contexts? Isn't it true that only that thread is allowed to submit usb-storage i/o requests? - Dave ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 4:11 ` David Brownell @ 2004-01-12 7:39 ` Matthew Dharm 2004-01-12 8:37 ` Oliver Neukum 0 siblings, 1 reply; 25+ messages in thread From: Matthew Dharm @ 2004-01-12 7:39 UTC (permalink / raw) To: David Brownell Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH [-- Attachment #1: Type: text/plain, Size: 946 bytes --] On Sun, Jan 11, 2004 at 08:11:58PM -0800, David Brownell wrote: > > >>> Plus I'd > >>>argue PF_MEMALLOC is a better solution anyway. > >> > >>It certainly seems like a more comprehensive fix for that > >>particular class of problems! :) > > > > > >Is it really more comprehensive? As I see it, it will only affect code > >executed in the context of the usb-storage thread. But, what about code > >which is invoked in tasklets or other contexts? > > Isn't it true that only that thread is allowed to > submit usb-storage i/o requests? That's very true. What I'm concerned about is the downstream effects of a usb_submit_urb() or the corresponding scatter-gather equivalents. Matt -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver You should try to see the techs say "three piece suit". -- The Chief User Friendly, 11/23/1997 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 7:39 ` Matthew Dharm @ 2004-01-12 8:37 ` Oliver Neukum 2004-01-12 16:27 ` Alan Stern 0 siblings, 1 reply; 25+ messages in thread From: Oliver Neukum @ 2004-01-12 8:37 UTC (permalink / raw) To: Matthew Dharm, David Brownell Cc: Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Am Montag, 12. Januar 2004 08:39 schrieb Matthew Dharm: > On Sun, Jan 11, 2004 at 08:11:58PM -0800, David Brownell wrote: > > > > >>> Plus I'd > > >>>argue PF_MEMALLOC is a better solution anyway. > > >> > > >>It certainly seems like a more comprehensive fix for that > > >>particular class of problems! :) > > > > > > > > >Is it really more comprehensive? As I see it, it will only affect code > > >executed in the context of the usb-storage thread. But, what about code > > >which is invoked in tasklets or other contexts? > > > > Isn't it true that only that thread is allowed to > > submit usb-storage i/o requests? > > That's very true. > > What I'm concerned about is the downstream effects of a usb_submit_urb() or > the corresponding scatter-gather equivalents. In 2.4 they all run in interrupt or thread context IIRC. Problematic is the SCSI error handling thread. It can call usb_reset_device() which calls down and does allocations. Does that thread also do the PF_MEMALLOC trick? Regards Oliver ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 8:37 ` Oliver Neukum @ 2004-01-12 16:27 ` Alan Stern 2004-01-12 20:56 ` Alan Cox 2004-01-16 13:14 ` Pavel Machek 0 siblings, 2 replies; 25+ messages in thread From: Alan Stern @ 2004-01-12 16:27 UTC (permalink / raw) To: Oliver Neukum Cc: Matthew Dharm, David Brownell, Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Mon, 12 Jan 2004, Oliver Neukum wrote: > In 2.4 they all run in interrupt or thread context IIRC. > Problematic is the SCSI error handling thread. It can call usb_reset_device() > which calls down and does allocations. > Does that thread also do the PF_MEMALLOC trick? In 2.4 it doesn't, which is rather surpising considering how many storage devices run over SCSI transports. In 2.6 it sets PF_IOTHREAD. I don't know if that subsumes the function of PF_MEMALLOC or not. The state of kerneldoc for much of the Linux core functionality is shocking. Alan Stern ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 16:27 ` Alan Stern @ 2004-01-12 20:56 ` Alan Cox 2004-01-16 13:14 ` Pavel Machek 1 sibling, 0 replies; 25+ messages in thread From: Alan Cox @ 2004-01-12 20:56 UTC (permalink / raw) To: Alan Stern Cc: Oliver Neukum, Matthew Dharm, David Brownell, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Llu, 2004-01-12 at 16:27, Alan Stern wrote: > On Mon, 12 Jan 2004, Oliver Neukum wrote: > > > In 2.4 they all run in interrupt or thread context IIRC. > > Problematic is the SCSI error handling thread. It can call usb_reset_device() > > which calls down and does allocations. > > Does that thread also do the PF_MEMALLOC trick? > > In 2.4 it doesn't, which is rather surpising considering how many storage > devices run over SCSI transports. > > In 2.6 it sets PF_IOTHREAD. I don't know if that subsumes the function of > PF_MEMALLOC or not. The state of kerneldoc for much of the Linux core > functionality is shocking. Core scsi assumes that scsi drivers will use scsi_malloc/free so really ignores the issue as someone elses problem. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 16:27 ` Alan Stern 2004-01-12 20:56 ` Alan Cox @ 2004-01-16 13:14 ` Pavel Machek 1 sibling, 0 replies; 25+ messages in thread From: Pavel Machek @ 2004-01-16 13:14 UTC (permalink / raw) To: Alan Stern Cc: Oliver Neukum, Matthew Dharm, David Brownell, Alan Cox, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Hi! > > In 2.4 they all run in interrupt or thread context IIRC. > > Problematic is the SCSI error handling thread. It can call usb_reset_device() > > which calls down and does allocations. > > Does that thread also do the PF_MEMALLOC trick? > > In 2.4 it doesn't, which is rather surpising considering how many storage > devices run over SCSI transports. > > In 2.6 it sets PF_IOTHREAD. I don't know if that subsumes the function of > PF_MEMALLOC or not. The state of kerneldoc for much of the Linux core > functionality is shocking. PF_IOTHREAD is there for suspend/resume. It does not affect anything else. Pavel -- When do you have a heart between your knees? [Johanka's followup: and *two* hearts?] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 23:25 ` David Brownell 2004-01-11 23:31 ` Matthew Dharm @ 2004-01-11 23:33 ` Oliver Neukum 2004-01-12 0:09 ` Alan Cox 1 sibling, 1 reply; 25+ messages in thread From: Oliver Neukum @ 2004-01-11 23:33 UTC (permalink / raw) To: David Brownell, Alan Cox Cc: Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH Am Montag, 12. Januar 2004 00:25 schrieb David Brownell: > Alan Cox wrote: > > On Sul, 2004-01-11 at 00:23, Matthew Dharm wrote: > > > >>Where is USB kmalloc'ing with GFP_KERNEL? I thought we tracked all those > >>down and eliminated them. > > > > > > Not sure. I just worked from tracebacks. I needed it to work rather > > than having the time to go hunting for specific faults. Plus I'd > > argue PF_MEMALLOC is a better solution anyway. > > It certainly seems like a more comprehensive fix for that > particular class of problems! :) For users of a kernel thread it helps. But what affects storage also make affect anything else that has a filesystem running over it. Plus it forces us to keep the storage thread model, which might be a solution that needs to be revisited. Regards Oliver ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-11 23:33 ` Oliver Neukum @ 2004-01-12 0:09 ` Alan Cox 2004-01-12 0:25 ` Matthew Dharm 0 siblings, 1 reply; 25+ messages in thread From: Alan Cox @ 2004-01-12 0:09 UTC (permalink / raw) To: Oliver Neukum Cc: David Brownell, Matthew Dharm, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH On Sul, 2004-01-11 at 23:33, Oliver Neukum wrote: > For users of a kernel thread it helps. But what affects storage > also make affect anything else that has a filesystem running > over it. Plus it forces us to keep the storage thread model, which > might be a solution that needs to be revisited. Its a larger hammer, for 2.6 I agree that moving the right code to GFP_NOIO is far better a solution. For 2.4 I just want it working with minimal risk of screwups. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs 2004-01-12 0:09 ` Alan Cox @ 2004-01-12 0:25 ` Matthew Dharm 0 siblings, 0 replies; 25+ messages in thread From: Matthew Dharm @ 2004-01-12 0:25 UTC (permalink / raw) To: Alan Cox Cc: Oliver Neukum, David Brownell, Marcelo Tosatti, Linux Kernel Mailing List, USB Developers, Greg KH [-- Attachment #1: Type: text/plain, Size: 1087 bytes --] On Mon, Jan 12, 2004 at 12:09:41AM +0000, Alan Cox wrote: > On Sul, 2004-01-11 at 23:33, Oliver Neukum wrote: > > For users of a kernel thread it helps. But what affects storage > > also make affect anything else that has a filesystem running > > over it. Plus it forces us to keep the storage thread model, which > > might be a solution that needs to be revisited. > > Its a larger hammer, for 2.6 I agree that moving the right code to > GFP_NOIO is far better a solution. For 2.4 I just want it working with > minimal risk of screwups. Well, I have no objection to adding that to 2.4 -- either push to Marcelo yourself or send it to Greg K-H for inclusion in his 2.4 tree and eventual push upstream. But we do need to do some sort of 2.6 audit for this sort of thing. Matt -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver G: Baaap booop BAHHHP. Mir: 9600 Baud? Mik: No, no! 9600 goes baap booop, not booop bahhhp! -- Greg, Miranda and Mike User Friendly, 12/31/1998 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: USB hangs 2004-01-11 0:07 USB hangs Alan Cox 2004-01-11 0:23 ` Matthew Dharm @ 2004-01-11 18:46 ` Lukas Postupa 2004-01-11 20:04 ` Matthew Dharm 1 sibling, 1 reply; 25+ messages in thread From: Lukas Postupa @ 2004-01-11 18:46 UTC (permalink / raw) To: Alan Cox; +Cc: linux-kernel [-- Attachment #1: Type: text/plain, Size: 2537 bytes --] Alan Cox wrote: > With the various fixes people had been posting USB storage > writing was still hanging repeatedly when doing a 20Gb rsync > to usb-storage disks with a low memory system. Doing things > like while(true) sync() made it hang even more often. > > After a bit of digging the following seems to fix it > > Not sure if 2.6 needs this as well. I have similiar problems with kernel 2.6.0 on Intel architecture with 512 MB Ram. My Abit IT7-MAX2 2.0 mainboard has two USB - EHCI controllers: 00:1d.7 USB Controller: Intel Corp. 82801DB USB EHCI Controller (rev 02) 02:06.2 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 51) I'm using rsync to transfer my data to usb storage. Connecting usb storage to one of the 4 ports of VIA EHCI controller and then transferring data to it works good. But connecting usb storage to one of the 6 ports of INTEL EHCI controller and then transferring data to it, will hang up usb storage: Buffer I/O error on device sda1, logical block 121479 lost page write due to I/O error on sda1 Buffer I/O error on device sda1, logical block 121480 lost page write due to I/O error on sda1 ehci_hcd 0000:00:1d.7: GetStatus port 2 status 001002 POWER sig=se0 CSC hub 6-0:1.0: port 2, status 100, change 1, 12 Mb/s usb 6-2: USB disconnect, address 4 usb 6-2: usb_disable_device nuking all URBs usb 6-2: unregistering interface 6-2:1.0 usb-storage: storage_disconnect() called usb-storage: usb_stor_stop_transport called usb-storage: -- dissociate_dev usb-storage: -- sending exit command to thread usb-storage: *** thread awakened. usb-storage: -- exit command received usb-storage: -- usb_stor_release_resources finished usb 6-2: unregistering device VIA EHCI controller uses interrupt 21. INTEL EHCI controller uses interrupt 23. cat /proc/interrupts: CPU0 0: 16675721 IO-APIC-edge timer 1: 6876 IO-APIC-edge i8042 2: 0 XT-PIC cascade 9: 5 IO-APIC-level acpi 12: 191706 IO-APIC-edge i8042 14: 251218 IO-APIC-edge ide0 15: 1 IO-APIC-edge ide1 16: 1446615 IO-APIC-level uhci_hcd, nvidia 18: 0 IO-APIC-level uhci_hcd, uhci_hcd 19: 278 IO-APIC-level uhci_hcd, uhci_hcd, EMU10K1 21: 257509 IO-APIC-level ehci_hcd 22: 7640 IO-APIC-level eth0 23: 322432 IO-APIC-level ehci_hcd NMI: 0 LOC: 16675149 ERR: 0 MIS: 0 Any ideas? Lukas [-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: USB hangs 2004-01-11 18:46 ` Lukas Postupa @ 2004-01-11 20:04 ` Matthew Dharm 0 siblings, 0 replies; 25+ messages in thread From: Matthew Dharm @ 2004-01-11 20:04 UTC (permalink / raw) To: Lukas Postupa; +Cc: Alan Cox, linux-kernel [-- Attachment #1: Type: text/plain, Size: 3231 bytes --] Did the patch Alan posted make any difference? I've been getting several reports recently of problems with usb-storage and EHCI controllers. If you unload the ehci driver, your device will run at 1.1 speeds -- can you try that and see if it works? Matt On Sun, Jan 11, 2004 at 07:46:29PM +0100, Lukas Postupa wrote: > Alan Cox wrote: > > With the various fixes people had been posting USB storage > > writing was still hanging repeatedly when doing a 20Gb rsync > > to usb-storage disks with a low memory system. Doing things > > like while(true) sync() made it hang even more often. > > > > After a bit of digging the following seems to fix it > > > > Not sure if 2.6 needs this as well. > > I have similiar problems with kernel 2.6.0 on Intel architecture with > 512 MB Ram. > > My Abit IT7-MAX2 2.0 mainboard has two USB - EHCI controllers: > > 00:1d.7 USB Controller: Intel Corp. 82801DB USB EHCI Controller (rev 02) > > 02:06.2 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 51) > > I'm using rsync to transfer my data to usb storage. > Connecting usb storage to one of the 4 ports of VIA EHCI controller and > then transferring data to it works good. > > But connecting usb storage to one of the 6 ports of INTEL EHCI > controller and then transferring data to it, will hang up usb storage: > > Buffer I/O error on device sda1, logical block 121479 > lost page write due to I/O error on sda1 > Buffer I/O error on device sda1, logical block 121480 > lost page write due to I/O error on sda1 > ehci_hcd 0000:00:1d.7: GetStatus port 2 status 001002 POWER sig=se0 CSC > hub 6-0:1.0: port 2, status 100, change 1, 12 Mb/s > usb 6-2: USB disconnect, address 4 > usb 6-2: usb_disable_device nuking all URBs > usb 6-2: unregistering interface 6-2:1.0 > usb-storage: storage_disconnect() called > usb-storage: usb_stor_stop_transport called > usb-storage: -- dissociate_dev > usb-storage: -- sending exit command to thread > usb-storage: *** thread awakened. > usb-storage: -- exit command received > usb-storage: -- usb_stor_release_resources finished > usb 6-2: unregistering device > > VIA EHCI controller uses interrupt 21. > INTEL EHCI controller uses interrupt 23. > > cat /proc/interrupts: > > CPU0 > 0: 16675721 IO-APIC-edge timer > 1: 6876 IO-APIC-edge i8042 > 2: 0 XT-PIC cascade > 9: 5 IO-APIC-level acpi > 12: 191706 IO-APIC-edge i8042 > 14: 251218 IO-APIC-edge ide0 > 15: 1 IO-APIC-edge ide1 > 16: 1446615 IO-APIC-level uhci_hcd, nvidia > 18: 0 IO-APIC-level uhci_hcd, uhci_hcd > 19: 278 IO-APIC-level uhci_hcd, uhci_hcd, EMU10K1 > 21: 257509 IO-APIC-level ehci_hcd > 22: 7640 IO-APIC-level eth0 > 23: 322432 IO-APIC-level ehci_hcd > NMI: 0 > LOC: 16675149 > ERR: 0 > MIS: 0 > > Any ideas? > > Lukas -- Matthew Dharm Home: mdharm-usb@one-eyed-alien.net Maintainer, Linux USB Mass Storage Driver Stef, you just got beaten by a ball of DIRT. -- Greg User Friendly, 12/7/1997 [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [linux-usb-devel] Re: USB hangs @ 2004-01-28 16:50 Martin Bogomolni 0 siblings, 0 replies; 25+ messages in thread From: Martin Bogomolni @ 2004-01-28 16:50 UTC (permalink / raw) To: linux-kernel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon 12 Jan, Oliver Neukum wrote: [...] ChangeSet@1.1218, 2004-01-12 00:26:13+01:00, oliver@vermuden.neukum.org - avoid GFP_KERNEL in block IO path [...] Oliver, Alan, et. al. I applied Oliver & Alan Cox's patches to the 2.4.24 kernel, and am still seeing hangs when copying large amounts of data over USB on IBM systems containing two EHCI controllers (Intel 82801EB). I see similar errors to the ones Lucas Postupa had in the kernel logs : Buffer I/O error on device sda1, logical block 134522 lost page write due to I/O error on sda1 Unloading the EHCI driver, and reverting to 1.1 speeds, does not solve the issue. - -Martin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQFAF+7vgZQNxll+EIcRAtkKAJ476IhZ+fLcyj73mDjNptU3rWmQ/ACeIIpz M5S8ast+WlfEHhKrbAtuPg0= =NH9l -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2004-01-28 17:18 UTC | newest] Thread overview: 25+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2004-01-11 0:07 USB hangs Alan Cox 2004-01-11 0:23 ` Matthew Dharm 2004-01-11 0:49 ` [linux-usb-devel] " Oliver Neukum 2004-01-11 1:01 ` Matthew Dharm 2004-01-11 1:06 ` Oliver Neukum 2004-01-11 1:40 ` David Brownell 2004-01-11 2:33 ` Alan Cox 2004-01-11 8:02 ` [linux-usb-devel] " Oliver Neukum 2004-01-11 22:39 ` Alan Cox 2004-01-11 23:29 ` Oliver Neukum 2004-01-12 15:53 ` Alan Stern 2004-01-11 23:25 ` David Brownell 2004-01-11 23:31 ` Matthew Dharm 2004-01-12 4:11 ` David Brownell 2004-01-12 7:39 ` Matthew Dharm 2004-01-12 8:37 ` Oliver Neukum 2004-01-12 16:27 ` Alan Stern 2004-01-12 20:56 ` Alan Cox 2004-01-16 13:14 ` Pavel Machek 2004-01-11 23:33 ` Oliver Neukum 2004-01-12 0:09 ` Alan Cox 2004-01-12 0:25 ` Matthew Dharm 2004-01-11 18:46 ` Lukas Postupa 2004-01-11 20:04 ` Matthew Dharm -- strict thread matches above, loose matches on Subject: below -- 2004-01-28 16:50 [linux-usb-devel] " Martin Bogomolni
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox