* [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
@ 2010-02-01 22:08 Christoph Hellwig
2010-02-03 10:55 ` Christoph Hellwig
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2010-02-01 22:08 UTC (permalink / raw)
To: xfs
Make sure all log buffers have made it to disk before we free them.
Otherwise we might reference freed buffers or a NULL mp->m_log from
the I/O completion handlers.
This fixes kernel.org bz #15150.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reported-by: Ed Cashin <ecashin@coraid.com>
Reported-by: ghani <abdelghani@ezono.com>
Tested-by: ghani <abdelghani@ezono.com>
Index: linux-2.6/fs/xfs/xfs_log.c
===================================================================
--- linux-2.6.orig/fs/xfs/xfs_log.c 2009-11-09 22:09:08.858026060 +0100
+++ linux-2.6/fs/xfs/xfs_log.c 2009-11-09 22:13:13.958255857 +0100
@@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log)
xlog_in_core_t *iclog, *next_iclog;
int i;
+ xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1);
+
iclog = log->l_iclog;
for (i=0; i<log->l_iclog_bufs; i++) {
sv_destroy(&iclog->ic_force_wait);
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
2010-02-01 22:08 [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Christoph Hellwig
@ 2010-02-03 10:55 ` Christoph Hellwig
2010-02-04 8:14 ` Dave Chinner
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2010-02-03 10:55 UTC (permalink / raw)
To: xfs
Actually I'll take this one back - the log buffers aren't delwri
so xfs_flush_buftarg probably only helped by timing or the
xfs_buf_runall_queues on xfslogd_workqueue. Need to think about
this a bit more.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
2010-02-03 10:55 ` Christoph Hellwig
@ 2010-02-04 8:14 ` Dave Chinner
2010-02-04 16:10 ` Christoph Hellwig
0 siblings, 1 reply; 6+ messages in thread
From: Dave Chinner @ 2010-02-04 8:14 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote:
> Actually I'll take this one back - the log buffers aren't delwri
> so xfs_flush_buftarg probably only helped by timing or the
> xfs_buf_runall_queues on xfslogd_workqueue. Need to think about
> this a bit more.
Can you point me at the bug report? IIRC I've seen this in the
past where we freed the log before we've done all the correct
shutdown processing and they got fixed by correcting the
order of shutdown to ensure the log is idle before freeing
it...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
2010-02-04 8:14 ` Dave Chinner
@ 2010-02-04 16:10 ` Christoph Hellwig
2010-02-05 2:58 ` Dave Chinner
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2010-02-04 16:10 UTC (permalink / raw)
To: Dave Chinner; +Cc: Christoph Hellwig, xfs
On Thu, Feb 04, 2010 at 07:14:22PM +1100, Dave Chinner wrote:
> On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote:
> > Actually I'll take this one back - the log buffers aren't delwri
> > so xfs_flush_buftarg probably only helped by timing or the
> > xfs_buf_runall_queues on xfslogd_workqueue. Need to think about
> > this a bit more.
>
> Can you point me at the bug report? IIRC I've seen this in the
> past where we freed the log before we've done all the correct
> shutdown processing and they got fixed by correcting the
> order of shutdown to ensure the log is idle before freeing
> it...
Yes, that I was going to look into next. The recent report
is http://bugzilla.kernel.org/show_bug.cgi?id=15150 and there's
also a very similar one from Ed Cashin <ecashin@coraid.com> in
mail archives from November.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
2010-02-04 16:10 ` Christoph Hellwig
@ 2010-02-05 2:58 ` Dave Chinner
2010-02-05 11:35 ` Christoph Hellwig
0 siblings, 1 reply; 6+ messages in thread
From: Dave Chinner @ 2010-02-05 2:58 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
On Thu, Feb 04, 2010 at 11:10:50AM -0500, Christoph Hellwig wrote:
> On Thu, Feb 04, 2010 at 07:14:22PM +1100, Dave Chinner wrote:
> > On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote:
> > > Actually I'll take this one back - the log buffers aren't delwri
> > > so xfs_flush_buftarg probably only helped by timing or the
> > > xfs_buf_runall_queues on xfslogd_workqueue. Need to think about
> > > this a bit more.
> >
> > Can you point me at the bug report? IIRC I've seen this in the
> > past where we freed the log before we've done all the correct
> > shutdown processing and they got fixed by correcting the
> > order of shutdown to ensure the log is idle before freeing
> > it...
>
> Yes, that I was going to look into next. The recent report
> is http://bugzilla.kernel.org/show_bug.cgi?id=15150
That was from 2.6.27. I didn't think that we finished the
mount/unmount path cleanup that fixed all those problems until
2.6.28 or .29...
> and there's
> also a very similar one from Ed Cashin <ecashin@coraid.com> in
> mail archives from November.
Yeah, that's clearly IO completion after the call to
xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call
which waits for all hashed buffers on the buftarg except for
those that are XBF_FS_MANAGED - the superblock.
Looking at xfs_fs_put_super(), the superblock buffer is freed after
xfs_unmountfs() frees the log, so if the superblock was logged
and is undergoing IO then xfs_unmountfs_wait() would not have
blocked on it and it could leak past the "no more IO should
bein progress" barrier that xfs_unmountfs_wait() is supposed to
provide.
Does this look like a plausible cause of the problem?
Is so, given that the superblock is the only buffer that is marked
XBF_FS_MANAGED now and it seems to me that xfs_wait_buftarg()
should be waiting on it, maybe the fix needed is to
xfs_wait_buftarg()....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log
2010-02-05 2:58 ` Dave Chinner
@ 2010-02-05 11:35 ` Christoph Hellwig
0 siblings, 0 replies; 6+ messages in thread
From: Christoph Hellwig @ 2010-02-05 11:35 UTC (permalink / raw)
To: Dave Chinner; +Cc: Christoph Hellwig, xfs
On Fri, Feb 05, 2010 at 01:58:11PM +1100, Dave Chinner wrote:
> > and there's
> > also a very similar one from Ed Cashin <ecashin@coraid.com> in
> > mail archives from November.
>
> Yeah, that's clearly IO completion after the call to
> xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call
> which waits for all hashed buffers on the buftarg except for
> those that are XBF_FS_MANAGED - the superblock.
>
> Looking at xfs_fs_put_super(), the superblock buffer is freed after
> xfs_unmountfs() frees the log, so if the superblock was logged
> and is undergoing IO then xfs_unmountfs_wait() would not have
> blocked on it and it could leak past the "no more IO should
> bein progress" barrier that xfs_unmountfs_wait() is supposed to
> provide.
>
> Does this look like a plausible cause of the problem?
We just did a xfs_unmountfs_writesb before the call to
xfs_unmountfs_wait, so the superblock buffer should be a sync buffer at
that point, so something more fishy is going on here. syncd is also
stopped before that call, so that gets rid of another possible case
of an async buffer pending.
Independent of that I think xfs_wait_buftarg waiting for the SB buffer
seems correct to me, but I fear something else is going on here.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-02-05 11:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-02-01 22:08 [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Christoph Hellwig
2010-02-03 10:55 ` Christoph Hellwig
2010-02-04 8:14 ` Dave Chinner
2010-02-04 16:10 ` Christoph Hellwig
2010-02-05 2:58 ` Dave Chinner
2010-02-05 11:35 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox