* [PATCH resend] fs/proc: Move kfree outside pde_unload_lock [not found] <515D9F8A.2060505@sgi.com> @ 2013-04-04 15:53 ` Nathan Zimmer 2013-04-04 16:11 ` Al Viro 0 siblings, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-04 15:53 UTC (permalink / raw) To: viro, akpm Cc: linux-kernel, linux-fsdevel, Nathan Zimmer, Eric W. Biederman, David Woodhouse, stable This moves a kfree outside a spinlock to help scaling on larger (512 core) systems. This should be some relief until we can move the section to use the rcu. I ran a simple test which just reads from /proc/cpuinfo. Lower is better, as you can see the worst case scenario is improved. baseline moved kfree tasks read-sec read-sec 1 0.0141 0.0141 2 0.0140 0.0140 4 0.0140 0.0141 8 0.0145 0.0145 16 0.0553 0.0548 32 0.1688 0.1622 64 0.5017 0.3856 128 1.7005 0.9710 256 5.2513 2.6519 512 8.0529 6.2976 Cc: "Eric W. Biederman" <ebiederm@xmission.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Alexander Viro <viro@zeniv.linux.org.uk> Cc: David Woodhouse <dwmw2@infradead.org> Cc: <stable@vger.kernel.org> Acked-by: Alexey Dobriyan <adobriyan@gmail.com> Signed-off-by: Nathan Zimmer <nzimmer@sgi.com> --- fs/proc/inode.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/proc/inode.c b/fs/proc/inode.c index 439ae688..863608b 100644 --- a/fs/proc/inode.c +++ b/fs/proc/inode.c @@ -403,11 +403,10 @@ static int proc_reg_release(struct inode *inode, struct file *file) } pde->pde_users++; release = pde->proc_fops->release; - if (pdeo) { + if (pdeo) list_del(&pdeo->lh); - kfree(pdeo); - } spin_unlock(&pde->pde_unload_lock); + kfree(pdeo); if (release) rv = release(inode, file); -- 1.8.1.2 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-04 15:53 ` [PATCH resend] fs/proc: Move kfree outside pde_unload_lock Nathan Zimmer @ 2013-04-04 16:11 ` Al Viro 2013-04-04 17:12 ` Nathan Zimmer 0 siblings, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-04 16:11 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Thu, Apr 04, 2013 at 10:53:39AM -0500, Nathan Zimmer wrote: > This moves a kfree outside a spinlock to help scaling on larger (512 core) > systems. This should be some relief until we can move the section to use > the rcu. Umm... That'll get wrecked as soon as fixes from #experimental go in; FWIW, I'd probably make close_pdeo() return pdeo or NULL, depending on whether we want it freed. With kfree() itself taken to callers. But there's much bigger fish to fry there - turn use_pde() into return atomic_inc_unless_negative(&pde->pde_users), unuse_pde() into if (atomic_dec_return(&pde->pde_users) == BIAS) complete(pde->....) and make sure entry_rundown() sets completion *before* adding BIAS to pde_users and waits for it only if the sum was equal to BIAS. The spinlock is still needed, but only on the "now taking care of any pdeo that might still be around" side of things - it protects pdeo list. Again, see the last two commits of vfs.git#experimental. I'd certainly appreciate any extra eyes on that sucker... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-04 16:11 ` Al Viro @ 2013-04-04 17:12 ` Nathan Zimmer 2013-04-04 20:44 ` Al Viro 0 siblings, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-04 17:12 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/04/2013 11:11 AM, Al Viro wrote: > On Thu, Apr 04, 2013 at 10:53:39AM -0500, Nathan Zimmer wrote: >> This moves a kfree outside a spinlock to help scaling on larger (512 core) >> systems. This should be some relief until we can move the section to use >> the rcu. > Umm... That'll get wrecked as soon as fixes from #experimental go in; > FWIW, I'd probably make close_pdeo() return pdeo or NULL, depending on > whether we want it freed. With kfree() itself taken to callers. > But there's much bigger fish to fry there - turn use_pde() into > return atomic_inc_unless_negative(&pde->pde_users), unuse_pde() into > if (atomic_dec_return(&pde->pde_users) == BIAS) complete(pde->....) > and make sure entry_rundown() sets completion *before* adding BIAS > to pde_users and waits for it only if the sum was equal to BIAS. > The spinlock is still needed, but only on the "now taking care of > any pdeo that might still be around" side of things - it protects > pdeo list. > > Again, see the last two commits of vfs.git#experimental. I'd certainly > appreciate any extra eyes on that sucker... Ok I am cloning the tree now. It does look like the patches would conflict. I'll run some tests and take a deeper look. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-04 17:12 ` Nathan Zimmer @ 2013-04-04 20:44 ` Al Viro 2013-04-05 17:05 ` Nathan Zimmer 0 siblings, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-04 20:44 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Thu, Apr 04, 2013 at 12:12:05PM -0500, Nathan Zimmer wrote: > Ok I am cloning the tree now. > It does look like the patches would conflict. > I'll run some tests and take a deeper look. FWIW, I've just pushed there a tentative patch that switches to hopefully saner locking (head should be at cb673c115c1f99d3480471ca5d8cb3f89a1e3bee). Is that more or less what you want wrt spinlock contention? One note: for any given pde_opener, close_pdeo() can be called at most by two threads - final fput() and remove_proc_entry() resp. I think the use of completion + flag is safe there; pde->pde_unload_lock should serialize the critical areas. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-04 20:44 ` Al Viro @ 2013-04-05 17:05 ` Nathan Zimmer 2013-04-05 17:36 ` Al Viro 0 siblings, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-05 17:05 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/04/2013 03:44 PM, Al Viro wrote: > On Thu, Apr 04, 2013 at 12:12:05PM -0500, Nathan Zimmer wrote: > >> Ok I am cloning the tree now. >> It does look like the patches would conflict. >> I'll run some tests and take a deeper look. > FWIW, I've just pushed there a tentative patch that switches to hopefully > saner locking (head should be at cb673c115c1f99d3480471ca5d8cb3f89a1e3bee). > Is that more or less what you want wrt spinlock contention? > > One note: for any given pde_opener, close_pdeo() can be called at most > by two threads - final fput() and remove_proc_entry() resp. I think > the use of completion + flag is safe there; pde->pde_unload_lock > should serialize the critical areas. Something isn't quite right. I keep getting hung during boot. dracut: Mounted root filesystem /dev/sda8 dracut: Switching root I'll try to get some more info on a smaller box. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-05 17:05 ` Nathan Zimmer @ 2013-04-05 17:36 ` Al Viro 2013-04-05 20:56 ` Nathan Zimmer 0 siblings, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-05 17:36 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Fri, Apr 05, 2013 at 12:05:26PM -0500, Nathan Zimmer wrote: > On 04/04/2013 03:44 PM, Al Viro wrote: > >On Thu, Apr 04, 2013 at 12:12:05PM -0500, Nathan Zimmer wrote: > > > >>Ok I am cloning the tree now. > >>It does look like the patches would conflict. > >>I'll run some tests and take a deeper look. > >FWIW, I've just pushed there a tentative patch that switches to hopefully > >saner locking (head should be at cb673c115c1f99d3480471ca5d8cb3f89a1e3bee). > >Is that more or less what you want wrt spinlock contention? > > > >One note: for any given pde_opener, close_pdeo() can be called at most > >by two threads - final fput() and remove_proc_entry() resp. I think > >the use of completion + flag is safe there; pde->pde_unload_lock > >should serialize the critical areas. > > Something isn't quite right. I keep getting hung during boot. > dracut: Mounted root filesystem /dev/sda8 > dracut: Switching root > > I'll try to get some more info on a smaller box. Umm... Try to add WARN_ON(1) in entry_rundown(), just to see what's getting hit (don't bother with entry name, stack trace will be enough). ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-05 17:36 ` Al Viro @ 2013-04-05 20:56 ` Nathan Zimmer 2013-04-05 21:00 ` Al Viro 0 siblings, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-05 20:56 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/05/2013 12:36 PM, Al Viro wrote: > On Fri, Apr 05, 2013 at 12:05:26PM -0500, Nathan Zimmer wrote: >> On 04/04/2013 03:44 PM, Al Viro wrote: >>> On Thu, Apr 04, 2013 at 12:12:05PM -0500, Nathan Zimmer wrote: >>> >>>> Ok I am cloning the tree now. >>>> It does look like the patches would conflict. >>>> I'll run some tests and take a deeper look. >>> FWIW, I've just pushed there a tentative patch that switches to hopefully >>> saner locking (head should be at cb673c115c1f99d3480471ca5d8cb3f89a1e3bee). >>> Is that more or less what you want wrt spinlock contention? >>> >>> One note: for any given pde_opener, close_pdeo() can be called at most >>> by two threads - final fput() and remove_proc_entry() resp. I think >>> the use of completion + flag is safe there; pde->pde_unload_lock >>> should serialize the critical areas. >> Something isn't quite right. I keep getting hung during boot. >> dracut: Mounted root filesystem /dev/sda8 >> dracut: Switching root >> >> I'll try to get some more info on a smaller box. > Umm... Try to add WARN_ON(1) in entry_rundown(), just to see what's > getting hit (don't bother with entry name, stack trace will be enough). That didn't produce anything. I'll run some bisections over the weekend and see what I can sort out. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-05 20:56 ` Nathan Zimmer @ 2013-04-05 21:00 ` Al Viro 2013-04-08 15:34 ` Nathan Zimmer 0 siblings, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-05 21:00 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Fri, Apr 05, 2013 at 03:56:17PM -0500, Nathan Zimmer wrote: > That didn't produce anything. I'll run some bisections over the > weekend and see what I can sort out. *Ugh* I'd try to build with DEBUG_KMEMLEAK and slapped printks on the entry and exit from close_pdeo(). If that doesn't show anything interesting, it's probably unrelated to procfs... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-05 21:00 ` Al Viro @ 2013-04-08 15:34 ` Nathan Zimmer 2013-04-08 15:58 ` Al Viro 0 siblings, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-08 15:34 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/05/2013 04:00 PM, Al Viro wrote: > On Fri, Apr 05, 2013 at 03:56:17PM -0500, Nathan Zimmer wrote: > >> That didn't produce anything. I'll run some bisections over the >> weekend and see what I can sort out. > *Ugh* > > I'd try to build with DEBUG_KMEMLEAK and slapped printks on the entry > and exit from close_pdeo(). If that doesn't show anything interesting, > it's probably unrelated to procfs... My bisection pointed me to this commit: e41efbf13c15 At this point I am assuming my issue is unrelated to procfs. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-08 15:34 ` Nathan Zimmer @ 2013-04-08 15:58 ` Al Viro 2013-04-08 16:42 ` Nathan Zimmer 2013-04-08 20:52 ` Nathan Zimmer 0 siblings, 2 replies; 23+ messages in thread From: Al Viro @ 2013-04-08 15:58 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Mon, Apr 08, 2013 at 10:34:07AM -0500, Nathan Zimmer wrote: > On 04/05/2013 04:00 PM, Al Viro wrote: > >On Fri, Apr 05, 2013 at 03:56:17PM -0500, Nathan Zimmer wrote: > > > >>That didn't produce anything. I'll run some bisections over the > >>weekend and see what I can sort out. > >*Ugh* > > > >I'd try to build with DEBUG_KMEMLEAK and slapped printks on the entry > >and exit from close_pdeo(). If that doesn't show anything interesting, > >it's probably unrelated to procfs... > My bisection pointed me to this commit: e41efbf13c15 > At this point I am assuming my issue is unrelated to procfs. Huh? That commit simply moves three functions and one struct from one file to another... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-08 15:58 ` Al Viro @ 2013-04-08 16:42 ` Nathan Zimmer 2013-04-08 20:52 ` Nathan Zimmer 1 sibling, 0 replies; 23+ messages in thread From: Nathan Zimmer @ 2013-04-08 16:42 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/08/2013 10:58 AM, Al Viro wrote: > On Mon, Apr 08, 2013 at 10:34:07AM -0500, Nathan Zimmer wrote: >> On 04/05/2013 04:00 PM, Al Viro wrote: >>> On Fri, Apr 05, 2013 at 03:56:17PM -0500, Nathan Zimmer wrote: >>> >>>> That didn't produce anything. I'll run some bisections over the >>>> weekend and see what I can sort out. >>> *Ugh* >>> >>> I'd try to build with DEBUG_KMEMLEAK and slapped printks on the entry >>> and exit from close_pdeo(). If that doesn't show anything interesting, >>> it's probably unrelated to procfs... >> My bisection pointed me to this commit: e41efbf13c15 >> At this point I am assuming my issue is unrelated to procfs. > Huh? That commit simply moves three functions and one struct from one file to > another... Yea, I was hoping it made more sense to you then it did do me. I have some lab time later. I'll verify that is the problem commit. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-08 15:58 ` Al Viro 2013-04-08 16:42 ` Nathan Zimmer @ 2013-04-08 20:52 ` Nathan Zimmer 2013-04-08 21:23 ` Al Viro 1 sibling, 1 reply; 23+ messages in thread From: Nathan Zimmer @ 2013-04-08 20:52 UTC (permalink / raw) To: Al Viro Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On 04/08/2013 10:58 AM, Al Viro wrote: > On Mon, Apr 08, 2013 at 10:34:07AM -0500, Nathan Zimmer wrote: >> On 04/05/2013 04:00 PM, Al Viro wrote: >>> On Fri, Apr 05, 2013 at 03:56:17PM -0500, Nathan Zimmer wrote: >>> >>>> That didn't produce anything. I'll run some bisections over the >>>> weekend and see what I can sort out. >>> *Ugh* >>> >>> I'd try to build with DEBUG_KMEMLEAK and slapped printks on the entry >>> and exit from close_pdeo(). If that doesn't show anything interesting, >>> it's probably unrelated to procfs... >> My bisection pointed me to this commit: e41efbf13c15 >> At this point I am assuming my issue is unrelated to procfs. > Huh? That commit simply moves three functions and one struct from one file to > another... Further digging seems to indicate 9984d7394618df9, the one right after the commit I previously identified. Not sure what I did wrong with my bisect to put it off by one. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-08 20:52 ` Nathan Zimmer @ 2013-04-08 21:23 ` Al Viro 2013-04-08 21:48 ` hangs on boot in 9984d7394618df9 Al Viro 2013-04-08 21:56 ` [PATCH resend] fs/proc: Move kfree outside pde_unload_lock Nathan Zimmer 0 siblings, 2 replies; 23+ messages in thread From: Al Viro @ 2013-04-08 21:23 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable On Mon, Apr 08, 2013 at 03:52:08PM -0500, Nathan Zimmer wrote: > Further digging seems to indicate 9984d7394618df9, the one right > after the commit I previously identified. > Not sure what I did wrong with my bisect to put it off by one. Ugh... Can't reproduce here ;-/ Could you give more details on your setup? ^ permalink raw reply [flat|nested] 23+ messages in thread
* hangs on boot in 9984d7394618df9 2013-04-08 21:23 ` Al Viro @ 2013-04-08 21:48 ` Al Viro 2013-04-08 22:17 ` Stephen Warren 2013-04-08 21:56 ` [PATCH resend] fs/proc: Move kfree outside pde_unload_lock Nathan Zimmer 1 sibling, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-08 21:48 UTC (permalink / raw) To: Nathan Zimmer Cc: akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, Stephen Warren On Mon, Apr 08, 2013 at 10:23:27PM +0100, Al Viro wrote: > On Mon, Apr 08, 2013 at 03:52:08PM -0500, Nathan Zimmer wrote: > > > Further digging seems to indicate 9984d7394618df9, the one right > > after the commit I previously identified. > > Not sure what I did wrong with my bisect to put it off by one. > > Ugh... Can't reproduce here ;-/ Could you give more details on your > setup? Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) into vfs.git#pipe-splitup; could you check which part triggers that hang? Should propagate in a few... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 21:48 ` hangs on boot in 9984d7394618df9 Al Viro @ 2013-04-08 22:17 ` Stephen Warren 2013-04-08 22:45 ` Doug Anderson 2013-04-08 22:46 ` Al Viro 0 siblings, 2 replies; 23+ messages in thread From: Stephen Warren @ 2013-04-08 22:17 UTC (permalink / raw) To: Al Viro Cc: Nathan Zimmer, akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On 04/08/2013 03:48 PM, Al Viro wrote: > On Mon, Apr 08, 2013 at 10:23:27PM +0100, Al Viro wrote: >> On Mon, Apr 08, 2013 at 03:52:08PM -0500, Nathan Zimmer wrote: >> >>> Further digging seems to indicate 9984d7394618df9, the one right >>> after the commit I previously identified. >>> Not sure what I did wrong with my bisect to put it off by one. >> >> Ugh... Can't reproduce here ;-/ Could you give more details on your >> setup? > > Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) > into vfs.git#pipe-splitup; could you check which part triggers that > hang? Should propagate in a few... It looks like "pipe: unify ->release() and ->open()" introduces the problem. Note that I had to add a prototype for fifo_open() before the structs that reference it for that commit to compile. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 22:17 ` Stephen Warren @ 2013-04-08 22:45 ` Doug Anderson 2013-04-08 23:06 ` Al Viro 2013-04-08 22:46 ` Al Viro 1 sibling, 1 reply; 23+ messages in thread From: Doug Anderson @ 2013-04-08 22:45 UTC (permalink / raw) To: Stephen Warren Cc: Al Viro, Nathan Zimmer, Andrew Morton, linux-kernel@vger.kernel.org, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org Hi, On Mon, Apr 8, 2013 at 3:17 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: >> Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) >> into vfs.git#pipe-splitup; could you check which part triggers that >> hang? Should propagate in a few... > > It looks like "pipe: unify ->release() and ->open()" introduces the > problem. Note that I had to add a prototype for fifo_open() before the > structs that reference it for that commit to compile. It sounds like Stephen has provided you the info you needed so not doing any extra testing now, but I figured I'd chime in that I hit problems this morning with linux-next and it appears to be the same thing. I did a revert of 9984d7394618df9 and (plus a revert of a handful of patches to the same file) and problems are resolved. The failure case is really weird in that everything works well booting to a simple /bin/bash but fails when you do more complex tasks. Anyway: If you need some extra testing feel free to CC me. -Doug ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 22:45 ` Doug Anderson @ 2013-04-08 23:06 ` Al Viro 2013-04-08 23:20 ` Stephen Warren 2013-04-08 23:46 ` Doug Anderson 0 siblings, 2 replies; 23+ messages in thread From: Al Viro @ 2013-04-08 23:06 UTC (permalink / raw) To: Doug Anderson Cc: Stephen Warren, Nathan Zimmer, Andrew Morton, linux-kernel@vger.kernel.org, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On Mon, Apr 08, 2013 at 03:45:31PM -0700, Doug Anderson wrote: > Hi, > > On Mon, Apr 8, 2013 at 3:17 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: > >> Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) > >> into vfs.git#pipe-splitup; could you check which part triggers that > >> hang? Should propagate in a few... > > > > It looks like "pipe: unify ->release() and ->open()" introduces the > > problem. Note that I had to add a prototype for fifo_open() before the > > structs that reference it for that commit to compile. > > It sounds like Stephen has provided you the info you needed so not > doing any extra testing now, but I figured I'd chime in that I hit > problems this morning with linux-next and it appears to be the same > thing. I did a revert of 9984d7394618df9 and (plus a revert of a > handful of patches to the same file) and problems are resolved. The > failure case is really weird in that everything works well booting to > a simple /bin/bash but fails when you do more complex tasks. > > Anyway: If you need some extra testing feel free to CC me. Folks, see if vfs.git#experimental works for you; the PITA had apparently been caused by change of open() semantics for /proc/<pid>/fd/<some_pipe> - it started to behave like a FIFO, i.e. wait for peer to show up. Normally that's not a problem, but if you have closed e.g. the write end of a pipe and try to open /proc/<pid>/fd/<read_end_of_pipe>, you'll get open() waiting for writers to appear. Which isn't what we used to do here (open succeeded immediately) and apparently that was enough to trip drakut. Branch head should be at 574179469f7370aadb9cbac1ceca7c3723c17bee. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 23:06 ` Al Viro @ 2013-04-08 23:20 ` Stephen Warren 2013-04-08 23:46 ` Doug Anderson 1 sibling, 0 replies; 23+ messages in thread From: Stephen Warren @ 2013-04-08 23:20 UTC (permalink / raw) To: Al Viro Cc: Doug Anderson, Nathan Zimmer, Andrew Morton, linux-kernel@vger.kernel.org, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On 04/08/2013 05:06 PM, Al Viro wrote: > On Mon, Apr 08, 2013 at 03:45:31PM -0700, Doug Anderson wrote: >> Hi, >> >> On Mon, Apr 8, 2013 at 3:17 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: >>>> Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) >>>> into vfs.git#pipe-splitup; could you check which part triggers that >>>> hang? Should propagate in a few... >>> >>> It looks like "pipe: unify ->release() and ->open()" introduces the >>> problem. Note that I had to add a prototype for fifo_open() before the >>> structs that reference it for that commit to compile. >> >> It sounds like Stephen has provided you the info you needed so not >> doing any extra testing now, but I figured I'd chime in that I hit >> problems this morning with linux-next and it appears to be the same >> thing. I did a revert of 9984d7394618df9 and (plus a revert of a >> handful of patches to the same file) and problems are resolved. The >> failure case is really weird in that everything works well booting to >> a simple /bin/bash but fails when you do more complex tasks. >> >> Anyway: If you need some extra testing feel free to CC me. > > Folks, see if vfs.git#experimental works for you; the PITA had apparently > been caused by change of open() semantics for /proc/<pid>/fd/<some_pipe> - > it started to behave like a FIFO, i.e. wait for peer to show up. Normally > that's not a problem, but if you have closed e.g. the write end of a pipe > and try to open /proc/<pid>/fd/<read_end_of_pipe>, you'll get open() waiting > for writers to appear. Which isn't what we used to do here (open succeeded > immediately) and apparently that was enough to trip drakut. > > Branch head should be at 574179469f7370aadb9cbac1ceca7c3723c17bee. That branch works. Thanks! ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 23:06 ` Al Viro 2013-04-08 23:20 ` Stephen Warren @ 2013-04-08 23:46 ` Doug Anderson 2013-04-09 17:12 ` Nathan Zimmer 1 sibling, 1 reply; 23+ messages in thread From: Doug Anderson @ 2013-04-08 23:46 UTC (permalink / raw) To: Al Viro Cc: Stephen Warren, Nathan Zimmer, Andrew Morton, linux-kernel@vger.kernel.org, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org Al, On Mon, Apr 8, 2013 at 4:06 PM, Al Viro <viro@zeniv.linux.org.uk> wrote: > Folks, see if vfs.git#experimental works for you; the PITA had apparently > been caused by change of open() semantics for /proc/<pid>/fd/<some_pipe> - > it started to behave like a FIFO, i.e. wait for peer to show up. Normally > that's not a problem, but if you have closed e.g. the write end of a pipe > and try to open /proc/<pid>/fd/<read_end_of_pipe>, you'll get open() waiting > for writers to appear. Which isn't what we used to do here (open succeeded > immediately) and apparently that was enough to trip drakut. > > Branch head should be at 574179469f7370aadb9cbac1ceca7c3723c17bee. That branch booted fine for me and didn't show any problems. I wasn't easily able to merge onto linux-next and test there though. I tried applying these the 4 top commits of your branch to "next-20130408" and it didn't solve my problems. A full merge of your branch to linux-next showed conflicts and I didn't dig. -Doug ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 23:46 ` Doug Anderson @ 2013-04-09 17:12 ` Nathan Zimmer 0 siblings, 0 replies; 23+ messages in thread From: Nathan Zimmer @ 2013-04-09 17:12 UTC (permalink / raw) To: Doug Anderson Cc: Al Viro, Stephen Warren, Andrew Morton, linux-kernel@vger.kernel.org, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On 04/08/2013 06:46 PM, Doug Anderson wrote: > Al, > > On Mon, Apr 8, 2013 at 4:06 PM, Al Viro <viro@zeniv.linux.org.uk> wrote: >> Folks, see if vfs.git#experimental works for you; the PITA had apparently >> been caused by change of open() semantics for /proc/<pid>/fd/<some_pipe> - >> it started to behave like a FIFO, i.e. wait for peer to show up. Normally >> that's not a problem, but if you have closed e.g. the write end of a pipe >> and try to open /proc/<pid>/fd/<read_end_of_pipe>, you'll get open() waiting >> for writers to appear. Which isn't what we used to do here (open succeeded >> immediately) and apparently that was enough to trip drakut. >> >> Branch head should be at 574179469f7370aadb9cbac1ceca7c3723c17bee. > That branch booted fine for me and didn't show any problems. > > I wasn't easily able to merge onto linux-next and test there though. > I tried applying these the 4 top commits of your branch to > "next-20130408" and it didn't solve my problems. A full merge of your > branch to linux-next showed conflicts and I didn't dig. > > -Doug I booted great for me too. Also the numbers from the scaling test are improved also, at least on my 128p box. I'll verify on a 512 or larger box when I have the chance. However I think I still should to provide some relief for the older kernels. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 22:17 ` Stephen Warren 2013-04-08 22:45 ` Doug Anderson @ 2013-04-08 22:46 ` Al Viro 2013-04-08 22:57 ` Al Viro 1 sibling, 1 reply; 23+ messages in thread From: Al Viro @ 2013-04-08 22:46 UTC (permalink / raw) To: Stephen Warren Cc: Nathan Zimmer, akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On Mon, Apr 08, 2013 at 04:17:17PM -0600, Stephen Warren wrote: > On 04/08/2013 03:48 PM, Al Viro wrote: > > On Mon, Apr 08, 2013 at 10:23:27PM +0100, Al Viro wrote: > >> On Mon, Apr 08, 2013 at 03:52:08PM -0500, Nathan Zimmer wrote: > >> > >>> Further digging seems to indicate 9984d7394618df9, the one right > >>> after the commit I previously identified. > >>> Not sure what I did wrong with my bisect to put it off by one. > >> > >> Ugh... Can't reproduce here ;-/ Could you give more details on your > >> setup? > > > > Anyway, I've just pushed a splitup of that commit (carved in 3 pieces) > > into vfs.git#pipe-splitup; could you check which part triggers that > > hang? Should propagate in a few... > > It looks like "pipe: unify ->release() and ->open()" introduces the > problem. Note that I had to add a prototype for fifo_open() before the > structs that reference it for that commit to compile. Very interesting... Just in case, could you try this on top of that branch and see if it triggers? diff --git a/fs/pipe.c b/fs/pipe.c index 8ce279b..b6cd51b 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -1022,6 +1022,11 @@ static int fifo_open(struct inode *inode, struct file *filp) /* We can only do regular read/write on fifos */ filp->f_mode &= (FMODE_READ | FMODE_WRITE); + if (inode->i_sb->s_magic == PIPEFS_MAGIC) { + WARN_ON(filp->f_flags & O_NONBLOCK); + filp->f_flags &= ~O_NONBLOCK; + } + switch (filp->f_mode) { case FMODE_READ: /* ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: hangs on boot in 9984d7394618df9 2013-04-08 22:46 ` Al Viro @ 2013-04-08 22:57 ` Al Viro 0 siblings, 0 replies; 23+ messages in thread From: Al Viro @ 2013-04-08 22:57 UTC (permalink / raw) To: Stephen Warren Cc: Nathan Zimmer, akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable, linux-next@vger.kernel.org On Mon, Apr 08, 2013 at 11:46:37PM +0100, Al Viro wrote: > Very interesting... Just in case, could you try this on top of that > branch and see if it triggers? > > diff --git a/fs/pipe.c b/fs/pipe.c > index 8ce279b..b6cd51b 100644 > --- a/fs/pipe.c > +++ b/fs/pipe.c > @@ -1022,6 +1022,11 @@ static int fifo_open(struct inode *inode, struct file *filp) > /* We can only do regular read/write on fifos */ > filp->f_mode &= (FMODE_READ | FMODE_WRITE); > > + if (inode->i_sb->s_magic == PIPEFS_MAGIC) { > + WARN_ON(filp->f_flags & O_NONBLOCK); > + filp->f_flags &= ~O_NONBLOCK; > + } *d'oh* OK, I'm an idiot. I see what's going on now... Give me a few and I'll push a fix. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH resend] fs/proc: Move kfree outside pde_unload_lock 2013-04-08 21:23 ` Al Viro 2013-04-08 21:48 ` hangs on boot in 9984d7394618df9 Al Viro @ 2013-04-08 21:56 ` Nathan Zimmer 1 sibling, 0 replies; 23+ messages in thread From: Nathan Zimmer @ 2013-04-08 21:56 UTC (permalink / raw) To: Al Viro Cc: Nathan Zimmer, akpm, linux-kernel, linux-fsdevel, Eric W. Biederman, David Woodhouse, stable [-- Attachment #1: Type: text/plain, Size: 540 bytes --] On Mon, Apr 08, 2013 at 10:23:27PM +0100, Al Viro wrote: > On Mon, Apr 08, 2013 at 03:52:08PM -0500, Nathan Zimmer wrote: > > > Further digging seems to indicate 9984d7394618df9, the one right > > after the commit I previously identified. > > Not sure what I did wrong with my bisect to put it off by one. > > Ugh... Can't reproduce here ;-/ Could you give more details on your > setup? It is a fairly small system, 12 processors. It is based on rhel 6.4. Here is my config if that helps. Other then that I am not sure what you need. [-- Attachment #2: .config --] [-- Type: application/x-config, Size: 129352 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2013-04-09 17:12 UTC | newest] Thread overview: 23+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <515D9F8A.2060505@sgi.com> 2013-04-04 15:53 ` [PATCH resend] fs/proc: Move kfree outside pde_unload_lock Nathan Zimmer 2013-04-04 16:11 ` Al Viro 2013-04-04 17:12 ` Nathan Zimmer 2013-04-04 20:44 ` Al Viro 2013-04-05 17:05 ` Nathan Zimmer 2013-04-05 17:36 ` Al Viro 2013-04-05 20:56 ` Nathan Zimmer 2013-04-05 21:00 ` Al Viro 2013-04-08 15:34 ` Nathan Zimmer 2013-04-08 15:58 ` Al Viro 2013-04-08 16:42 ` Nathan Zimmer 2013-04-08 20:52 ` Nathan Zimmer 2013-04-08 21:23 ` Al Viro 2013-04-08 21:48 ` hangs on boot in 9984d7394618df9 Al Viro 2013-04-08 22:17 ` Stephen Warren 2013-04-08 22:45 ` Doug Anderson 2013-04-08 23:06 ` Al Viro 2013-04-08 23:20 ` Stephen Warren 2013-04-08 23:46 ` Doug Anderson 2013-04-09 17:12 ` Nathan Zimmer 2013-04-08 22:46 ` Al Viro 2013-04-08 22:57 ` Al Viro 2013-04-08 21:56 ` [PATCH resend] fs/proc: Move kfree outside pde_unload_lock Nathan Zimmer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).