* Are two transactions running in parallel OK? @ 2012-05-17 20:08 Jan Kara 2012-05-18 10:28 ` Dave Chinner 0 siblings, 1 reply; 3+ messages in thread From: Jan Kara @ 2012-05-17 20:08 UTC (permalink / raw) To: xfs; +Cc: sandeen, dchinner Hello, I've been into a source of lockdep warning I got with XFS when testing my filesystem freezing patches. The culprit seems to be that when doing direct IO, XFS starts a transaction in xfs_setfilesize_trans_alloc() and attaches that transaction to endio structure. Then it goes on and starts another transaction in xfs_iomap_write_direct() which creates a possible deadlock with filesystem freezing (if the second transaction start happens after we start blocking new transactions, it gets blocked, but the first transaction isn't ever completed). So first I wanted to ask whether my analysis is correct. If yes, I was also wondering whether this cannot cause a deadlock (at least in theory) if the second transaction would block waiting for log space but we couldn't possibly free enough of it due to the first transaction being held open? If freezing deadlock is the only problem with this code, then I guess we could avoid waiting for filesystem freezing when starting the second transaction (although it might end up being rather ugly). Or if anyone else has other idea how to solve this, I'm listening ;). Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Are two transactions running in parallel OK? 2012-05-17 20:08 Are two transactions running in parallel OK? Jan Kara @ 2012-05-18 10:28 ` Dave Chinner 2012-05-18 13:21 ` Jan Kara 0 siblings, 1 reply; 3+ messages in thread From: Dave Chinner @ 2012-05-18 10:28 UTC (permalink / raw) To: Jan Kara; +Cc: sandeen, dchinner, xfs On Thu, May 17, 2012 at 10:08:31PM +0200, Jan Kara wrote: > Hello, > > I've been into a source of lockdep warning I got with XFS when testing my > filesystem freezing patches. The culprit seems to be that when doing direct > IO, XFS starts a transaction in xfs_setfilesize_trans_alloc() and attaches > that transaction to endio structure. Then it goes on and starts another > transaction in xfs_iomap_write_direct() which creates a possible deadlock > with filesystem freezing (if the second transaction start happens after we > start blocking new transactions, it gets blocked, but the first transaction > isn't ever completed). Both occur within the context of an active write IO - why would new transactions be blocked while there are still active write operations that require allocation transactions occurring? > So first I wanted to ask whether my analysis is correct. If yes, I was also > wondering whether this cannot cause a deadlock (at least in theory) if the > second transaction would block waiting for log space but we couldn't > possibly free enough of it due to the first transaction being held open? Don't think so. The first transaction reservation is for the inode size update, but it doesn't hold anything locked so it will not hold up log tail pushing so the second transaction reservation will not get blocked by it. The onyl way that could happen is if the combination of the two transactions is greater than 25% of the log, and given that the size update transaction reservation is only about 600 bytes, that can't occur.... > If freezing deadlock is the only problem with this code, then I guess we > could avoid waiting for filesystem freezing when starting the second > transaction (although it might end up being rather ugly). Or if anyone else > has other idea how to solve this, I'm listening ;). I'm confused about why the active sb_start_write() of the direct IO wouldn't hold off the freeze until the IO has completed. That should completely protect the write against freeze until the IO completes, which AFAICT means the lockdep report is a false positive.... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Are two transactions running in parallel OK? 2012-05-18 10:28 ` Dave Chinner @ 2012-05-18 13:21 ` Jan Kara 0 siblings, 0 replies; 3+ messages in thread From: Jan Kara @ 2012-05-18 13:21 UTC (permalink / raw) To: Dave Chinner; +Cc: sandeen, dchinner, Jan Kara, xfs On Fri 18-05-12 20:28:29, Dave Chinner wrote: > On Thu, May 17, 2012 at 10:08:31PM +0200, Jan Kara wrote: > > Hello, > > > > I've been into a source of lockdep warning I got with XFS when testing my > > filesystem freezing patches. The culprit seems to be that when doing direct > > IO, XFS starts a transaction in xfs_setfilesize_trans_alloc() and attaches > > that transaction to endio structure. Then it goes on and starts another > > transaction in xfs_iomap_write_direct() which creates a possible deadlock > > with filesystem freezing (if the second transaction start happens after we > > start blocking new transactions, it gets blocked, but the first transaction > > isn't ever completed). > > Both occur within the context of an active write IO - why would > new transactions be blocked while there are still active write > operations that require allocation transactions occurring? Yeah, you are right that sb_start_write() will actually protect the transaction start so the deadlock with freezing cannot really occur. > > So first I wanted to ask whether my analysis is correct. If yes, I was also > > wondering whether this cannot cause a deadlock (at least in theory) if the > > second transaction would block waiting for log space but we couldn't > > possibly free enough of it due to the first transaction being held open? > > Don't think so. The first transaction reservation is for the inode > size update, but it doesn't hold anything locked so it will not hold > up log tail pushing so the second transaction reservation will not > get blocked by it. The onyl way that could happen is if the > combination of the two transactions is greater than 25% of the log, > and given that the size update transaction reservation is only about > 600 bytes, that can't occur.... OK, I see. I was thinking that it's likely OK but I was wondering about the details and couldn't quite figure it out from reading the transaction code. > > If freezing deadlock is the only problem with this code, then I guess we > > could avoid waiting for filesystem freezing when starting the second > > transaction (although it might end up being rather ugly). Or if anyone else > > has other idea how to solve this, I'm listening ;). > > I'm confused about why the active sb_start_write() of the direct IO > wouldn't hold off the freeze until the IO has completed. That should > completely protect the write against freeze until the IO completes, > which AFAICT means the lockdep report is a false positive.... Yes, it will, you are right. I was too tired to realize this yesterday. So I just have to figure out how to properly instrument lockdep to avoid these warnings. Thanks for having a look! Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2012-05-18 13:21 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-05-17 20:08 Are two transactions running in parallel OK? Jan Kara 2012-05-18 10:28 ` Dave Chinner 2012-05-18 13:21 ` Jan Kara
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox