[AIO] aio-2.6.13-rc6-B1

[AIO] aio-2.6.13-rc6-B1

[Benjamin LaHaise]

The bugfix followup to the last aio rollup is now available at:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff

with the split up in:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/

This fixes the bugs noticed in the -B0 variant.  Major changes in this 
patchset are:

	- added aio semaphore ops
	- aio thread based fallbacks
	- vectored aio file_operations
	- aio sendmsg/recvmsg via thread fallbacks
	- retry based aio pipe operations

Comments?

		-ben

 arch/i386/Kconfig              |    4 
 arch/i386/kernel/semaphore.c   |  185 +-----------
 arch/um/Kconfig_i386           |    4 
 arch/um/Kconfig_x86_64         |    4 
 arch/x86_64/Kconfig            |    4 
 arch/x86_64/kernel/Makefile    |    2 
 arch/x86_64/kernel/semaphore.c |  180 ------------
 arch/x86_64/lib/thunk.S        |    1 
 description                    |    9 
 drivers/usb/gadget/inode.c     |    7 
 fs/aio.c                       |  610 +++++++++++++++++++++++++++++++++++------
 fs/bad_inode.c                 |    2 
 fs/block_dev.c                 |    9 
 fs/buffer.c                    |    2 
 fs/ext2/file.c                 |    2 
 fs/ext3/file.c                 |   16 -
 fs/inode.c                     |    2 
 fs/jfs/file.c                  |    2 
 fs/ntfs/file.c                 |    2 
 fs/pipe.c                      |  194 ++++++++++---
 fs/read_write.c                |  182 ++++++++----
 fs/reiserfs/file.c             |   10 
 include/asm-i386/semaphore.h   |   42 ++
 include/asm-x86_64/semaphore.h |   44 ++
 include/linux/aio.h            |   38 ++
 include/linux/aio_abi.h        |   13 
 include/linux/fs.h             |    9 
 include/linux/net.h            |    4 
 include/linux/pagemap.h        |   29 +
 include/linux/sched.h          |   13 
 include/linux/wait.h           |   42 ++
 include/linux/writeback.h      |    2 
 kernel/exit.c                  |    2 
 kernel/fork.c                  |    7 
 kernel/sched.c                 |   14 
 kernel/wait.c                  |   40 +-
 lib/Makefile                   |    1 
 lib/semaphore-sleepers.c       |  253 +++++++++++++++++
 mm/filemap.c                   |  164 ++++++++---
 net/socket.c                   |   97 +++++-
 40 files changed, 1593 insertions(+), 654 deletions(-)

Re: [AIO] aio-2.6.13-rc6-B1

[Suparna Bhattacharya]

On Wed, Aug 17, 2005 at 02:44:07PM -0400, Benjamin LaHaise wrote:
> The bugfix followup to the last aio rollup is now available at:
> 
> 	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff
> 
> with the split up in:
> 
> 	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/
> 
> This fixes the bugs noticed in the -B0 variant.  Major changes in this 
> patchset are:
> 
> 	- added aio semaphore ops
> 	- aio thread based fallbacks
> 	- vectored aio file_operations
> 	- aio sendmsg/recvmsg via thread fallbacks
> 	- retry based aio pipe operations
> 
> Comments?

Tried http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-all.diff  (since
there is no aio-2.6.13-rc6-B1-all.diff). It hangs during bootup :(

...
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Starting balanced_irq
Using IPI Shortcut mode
VFS: Cannot open root device "sda6" or unknown-block(8,6)
Please append a correct "root=" boot option
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)


Regards
Suparna

> 
> 		-ben
> 
>  arch/i386/Kconfig              |    4 
>  arch/i386/kernel/semaphore.c   |  185 +-----------
>  arch/um/Kconfig_i386           |    4 
>  arch/um/Kconfig_x86_64         |    4 
>  arch/x86_64/Kconfig            |    4 
>  arch/x86_64/kernel/Makefile    |    2 
>  arch/x86_64/kernel/semaphore.c |  180 ------------
>  arch/x86_64/lib/thunk.S        |    1 
>  description                    |    9 
>  drivers/usb/gadget/inode.c     |    7 
>  fs/aio.c                       |  610 +++++++++++++++++++++++++++++++++++------
>  fs/bad_inode.c                 |    2 
>  fs/block_dev.c                 |    9 
>  fs/buffer.c                    |    2 
>  fs/ext2/file.c                 |    2 
>  fs/ext3/file.c                 |   16 -
>  fs/inode.c                     |    2 
>  fs/jfs/file.c                  |    2 
>  fs/ntfs/file.c                 |    2 
>  fs/pipe.c                      |  194 ++++++++++---
>  fs/read_write.c                |  182 ++++++++----
>  fs/reiserfs/file.c             |   10 
>  include/asm-i386/semaphore.h   |   42 ++
>  include/asm-x86_64/semaphore.h |   44 ++
>  include/linux/aio.h            |   38 ++
>  include/linux/aio_abi.h        |   13 
>  include/linux/fs.h             |    9 
>  include/linux/net.h            |    4 
>  include/linux/pagemap.h        |   29 +
>  include/linux/sched.h          |   13 
>  include/linux/wait.h           |   42 ++
>  include/linux/writeback.h      |    2 
>  kernel/exit.c                  |    2 
>  kernel/fork.c                  |    7 
>  kernel/sched.c                 |   14 
>  kernel/wait.c                  |   40 +-
>  lib/Makefile                   |    1 
>  lib/semaphore-sleepers.c       |  253 +++++++++++++++++
>  mm/filemap.c                   |  164 ++++++++---
>  net/socket.c                   |   97 +++++-
>  40 files changed, 1593 insertions(+), 654 deletions(-)
> --
> To unsubscribe, send a message with 'unsubscribe linux-aio' in
> the body to majordomo@kvack.org.  For more info on Linux AIO,
> see: http://www.kvack.org/aio/
> Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>

-- 
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India

Re: [AIO] aio-2.6.13-rc6-B1

[Benjamin LaHaise]

On Thu, Aug 18, 2005 at 03:32:59PM +0530, Suparna Bhattacharya wrote:
> Using IPI Shortcut mode
> VFS: Cannot open root device "sda6" or unknown-block(8,6)
> Please append a correct "root=" boot option
> Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)

Are you sure the scsi driver is configured in?  This doesn't look related 
to any of the changes in the patch.  I'm going to be away from email for 
the next week, so you'll have to figure this out.

		-ben

Re: [AIO] aio-2.6.13-rc6-B1

[Suparna Bhattacharya]

On Wed, Aug 17, 2005 at 07:16:49PM -0400, Benjamin LaHaise wrote:
> On Thu, Aug 18, 2005 at 03:32:59PM +0530, Suparna Bhattacharya wrote:
> > Using IPI Shortcut mode
> > VFS: Cannot open root device "sda6" or unknown-block(8,6)
> > Please append a correct "root=" boot option
> > Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)
> 
> Are you sure the scsi driver is configured in?  This doesn't look related 
> to any of the changes in the patch.  I'm going to be away from email for 
> the next week, so you'll have to figure this out.

You are right, it does look like a different problem - happens even without
the patchset, just with vanilla 2.6.13-rc6. I'll refresh my build and try
again to see if it is a real problem..

Regards
Suparna

> 
> 		-ben
> --
> To unsubscribe, send a message with 'unsubscribe linux-aio' in
> the body to majordomo@kvack.org.  For more info on Linux AIO,
> see: http://www.kvack.org/aio/
> Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>

-- 
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India

Re: [AIO] aio-2.6.13-rc6-B1

[Sébastien Dugué]

On Wed, 2005-08-17 at 14:44 -0400, Benjamin LaHaise wrote:
> The bugfix followup to the last aio rollup is now available at:
> 
> 	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff
> 
> with the split up in:
> 
> 	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/
> 
> This fixes the bugs noticed in the -B0 variant.  Major changes in this 
> patchset are:
> 
> 	- added aio semaphore ops
> 	- aio thread based fallbacks
> 	- vectored aio file_operations
> 	- aio sendmsg/recvmsg via thread fallbacks
> 	- retry based aio pipe operations
> 
> Comments?
> 
> 		-ben
> 

  Hi Ben,

  what's the point of calling wake_up_locked(&sem->wait) in 
aio_down_wait? We're already in a wakeup path and end up
calling __wake_up_common recursively.

  I think it may be one of the cause of my kernel hanging at the
very beginning.

  When I remove this call things go further but at some point a
semaphore wait queue gets thrashed and __wake_up_common tries to
call an invalid callback function.

  Any input appreciated.

  Sébastien.

-- 
------------------------------------------------------

  Sébastien Dugué                BULL/FREC:B1-247
  phone: (+33) 476 29 77 70      Bullcom: 229-7770

  mailto:sebastien.dugue@bull.net

  Linux POSIX AIO: http://www.bullopensource.org/posix
  
------------------------------------------------------

Re: [AIO] aio-2.6.13-rc6-B1

[Benjamin LaHaise]

On Tue, Sep 20, 2005 at 12:23:10PM +0200, Sébastien Dugué wrote:
>   what's the point of calling wake_up_locked(&sem->wait) in 
> aio_down_wait? We're already in a wakeup path and end up
> calling __wake_up_common recursively.

That's necessary to kick the next semaphore op in the list.  The 
list_del_init() right above that makes sure that we don't recurse 
and run the routine again.

>   I think it may be one of the cause of my kernel hanging at the
> very beginning.
> 
>   When I remove this call things go further but at some point a
> semaphore wait queue gets thrashed and __wake_up_common tries to
> call an invalid callback function.

This patch from Zach might make a difference.  Let me know if it changes 
the symptoms at all.  Sorry if it doesn't apply cleanly, as it is against 
a base kernel.  Basically, we could sleep while holding ctx_lock, which 
does Bad Things(tm) on SMP systems.

		-ben


Index: 2.6.13-git12-lock-kiocb/fs/aio.c
===================================================================
--- 2.6.13-git12-lock-kiocb.orig/fs/aio.c
+++ 2.6.13-git12-lock-kiocb/fs/aio.c
@@ -398,7 +398,7 @@ static struct kiocb fastcall *__aio_get_
 	if (unlikely(!req))
 		return NULL;
 
-	req->ki_flags = 1 << KIF_LOCKED;
+	req->ki_flags = 0;
 	req->ki_users = 2;
 	req->ki_key = 0;
 	req->ki_ctx = ctx;
@@ -717,6 +717,8 @@ static ssize_t aio_run_iocb(struct kiocb
 	iocb->ki_run_list.next = iocb->ki_run_list.prev = NULL;
 	spin_unlock_irq(&ctx->ctx_lock);
 
+	lock_kiocb(iocb);
+
 	/* Quit retrying if the i/o has been cancelled */
 	if (kiocbIsCancelled(iocb)) {
 		ret = -EINTR;
@@ -781,6 +783,7 @@ out:
 			aio_queue_work(ctx);
 		}
 	}
+	unlock_kiocb(iocb);
 	return ret;
 }
 
@@ -805,9 +808,7 @@ static int __aio_run_iocbs(struct kioctx
 		 * Hold an extra reference while retrying i/o.
 		 */
 		iocb->ki_users++;       /* grab extra reference */
-		lock_kiocb(iocb);
 		aio_run_iocb(iocb);
-		unlock_kiocb(iocb);
 		if (__aio_put_req(ctx, iocb))  /* drop extra ref */
 			put_ioctx(ctx);
  	}
@@ -1549,7 +1550,6 @@ int fastcall io_submit_one(struct kioctx
 
 	spin_lock_irq(&ctx->ctx_lock);
 	aio_run_iocb(req);
-	unlock_kiocb(req);
 	if (!list_empty(&ctx->run_list)) {
 		/* drain the run list */
 		while (__aio_run_iocbs(ctx))

Re: [AIO] aio-2.6.13-rc6-B1

[Sébastien Dugué]

On Tue, 2005-09-20 at 15:13 -0400, Benjamin LaHaise wrote:
> On Tue, Sep 20, 2005 at 12:23:10PM +0200, Sébastien Dugué wrote:
> >   what's the point of calling wake_up_locked(&sem->wait) in 
> > aio_down_wait? We're already in a wakeup path and end up
> > calling __wake_up_common recursively.
> 
> That's necessary to kick the next semaphore op in the list.  The 
> list_del_init() right above that makes sure that we don't recurse 
> and run the routine again.

  OK, understood.

> 
> >   I think it may be one of the cause of my kernel hanging at the
> > very beginning.
> > 
> >   When I remove this call things go further but at some point a
> > semaphore wait queue gets thrashed and __wake_up_common tries to
> > call an invalid callback function.
> 
> This patch from Zach might make a difference.  Let me know if it changes 
> the symptoms at all.  Sorry if it doesn't apply cleanly, as it is against 
> a base kernel.  Basically, we could sleep while holding ctx_lock, which 
> does Bad Things(tm) on SMP systems.
> 

  Well, it does not change a thing (I was not expecting it to). I think 
the problem rather lies in the async semaphores (aio_down/aio_up)
mechanism and not in the fs aio.

  What leads me to this is that the crash occurs only when there is
contention on an inode semaphore, which seems to happen frequently
with pipes and not so frequently with regular file IO.

  And as the pipes are the only users (aside from xxx_aio_writev) of 
this mecanism it shows right after the kernel is booted. Without the
90_pipe_aio.diff patch, the kernel boots normally but Oopses during
shutdown involving again a semaphore operation.

  Any ideas?

  Sébastien.

-- 
------------------------------------------------------

  Sébastien Dugué                BULL/FREC:B1-247
  phone: (+33) 476 29 77 70      Bullcom: 229-7770

  mailto:sebastien.dugue@bull.net

  Linux POSIX AIO: http://www.bullopensource.org/posix
  
------------------------------------------------------