[PATCH] jfs: drain lazy commit queue during unmount to prevent use-after-free

[PATCH] jfs: drain lazy commit queue during unmount to prevent use-after-free

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

The jfsCommit kernel thread processes committed transactions from
TxAnchor.unlock_queue via jfs_lazycommit().  During filesystem
unmount, jfs_umount() calls jfs_flush_journal(log, 2) which waits
for the log commit queue (log->cqueue) to drain.  However, after
log I/O completes, lazy transactions are moved to
TxAnchor.unlock_queue for asynchronous processing by jfsCommit.

If jfs_umount() proceeds to free the jfs_log (via lmLogClose) or
jfs_sb_info (via kfree in jfs_put_super) while entries referencing
this superblock remain on unlock_queue, the jfsCommit thread will
access freed memory when it later processes these entries:

- jfs_lazycommit reads sbi->commit_state (UAF of jfs_sb_info)
- txLazyCommit accesses JFS_SBI(tblk->sb)->log and takes
  log->gclock (UAF of jfs_log)

Add txLazyDrain() which waits for all entries in
TxAnchor.unlock_queue belonging to the unmounting superblock to be
processed, and also waits for any in-flight txLazyCommit
(IN_LAZYCOMMIT) for this superblock to complete.  Call it from both
jfs_umount() and jfs_umount_rw() after jfs_flush_journal().

Reported-by: syzbot+c244f4a09ca85dd2ebc1@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=c244f4a09ca85dd2ebc1
Tested-by: syzbot+c244f4a09ca85dd2ebc1@syzkaller.appspotmail.com
Reported-by: syzbot+885a4f3281b8d99c48d8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=885a4f3281b8d99c48d8
Tested-by: syzbot+885a4f3281b8d99c48d8@syzkaller.appspotmail.com
Fixes: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 ("Linux-2.6.12-rc2")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
 fs/jfs/jfs_txnmgr.c | 35 +++++++++++++++++++++++++++++++++++
 fs/jfs/jfs_txnmgr.h |  1 +
 fs/jfs/jfs_umount.c |  8 ++++++++
 3 files changed, 44 insertions(+)

diff --git a/fs/jfs/jfs_txnmgr.c b/fs/jfs/jfs_txnmgr.c
index 083dbbb0c3268..67a9908b5a4d9 100644
--- a/fs/jfs/jfs_txnmgr.c
+++ b/fs/jfs/jfs_txnmgr.c
@@ -2791,6 +2791,41 @@ void txLazyUnlock(struct tblock * tblk)
 	LAZY_UNLOCK(flags);
 }
 
+
+/*
+ * txLazyDrain
+ *
+ * Wait for all pending lazy commit entries for this superblock
+ * to be processed by the jfsCommit thread.  Must be called
+ * before freeing per-filesystem structures during unmount.
+ */
+void txLazyDrain(struct super_block *sb)
+{
+	struct jfs_sb_info *sbi = JFS_SBI(sb);
+	struct tblock *tblk;
+	unsigned long flags;
+	bool found;
+
+	do {
+		found = false;
+		LAZY_LOCK(flags);
+		list_for_each_entry(tblk, &TxAnchor.unlock_queue, cqueue) {
+			if (tblk->sb == sb) {
+				found = true;
+				break;
+			}
+		}
+		if (!found && (sbi->commit_state & IN_LAZYCOMMIT))
+			found = true;
+		LAZY_UNLOCK(flags);
+
+		if (found) {
+			wake_up(&jfs_commit_thread_wait);
+			schedule_timeout_uninterruptible(1);
+		}
+	} while (found);
+}
+
 static void LogSyncRelease(struct metapage * mp)
 {
 	struct jfs_log *log = mp->log;
diff --git a/fs/jfs/jfs_txnmgr.h b/fs/jfs/jfs_txnmgr.h
index ba71eb5ced567..fbbaed26c52bd 100644
--- a/fs/jfs/jfs_txnmgr.h
+++ b/fs/jfs/jfs_txnmgr.h
@@ -291,6 +291,7 @@ extern void txFreelock(struct inode *);
 extern int lmLog(struct jfs_log *, struct tblock *, struct lrd *,
 		 struct tlock *);
 extern void txQuiesce(struct super_block *);
+extern void txLazyDrain(struct super_block *sb);
 extern void txResume(struct super_block *);
 extern void txLazyUnlock(struct tblock *);
 extern int jfs_lazycommit(void *);
diff --git a/fs/jfs/jfs_umount.c b/fs/jfs/jfs_umount.c
index 18569f1eaabdb..657707361be2a 100644
--- a/fs/jfs/jfs_umount.c
+++ b/fs/jfs/jfs_umount.c
@@ -58,6 +58,13 @@ int jfs_umount(struct super_block *sb)
 		 */
 		jfs_flush_journal(log, 2);
 
+	/*
+	 * Drain any pending lazy commit entries for this filesystem so
+	 * the jfsCommit thread does not access freed structures.
+	 */
+	if (log)
+		txLazyDrain(sb);
+
 	/*
 	 * Hold log lock so write_special_inodes (lmLogSync) cannot see
 	 * this sbi with a NULL inode pointer while iterating log->sb_list.
@@ -142,6 +149,7 @@ int jfs_umount_rw(struct super_block *sb)
 	 * remove file system from log active file system list.
 	 */
 	jfs_flush_journal(log, 2);
+	txLazyDrain(sb);
 
 	/*
 	 * Make sure all metadata makes it to disk
-- 
2.47.3

[PATCH v2 0/2] jfs: fix use-after-free races during unmount

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

Two related UAF races exist in JFS unmount:

1. jfs_lazycommit accesses freed jfs_sb_info/jfs_log after unmount
   proceeds past jfs_flush_journal but before jfsCommit drains
   TxAnchor.unlock_queue.

2. lbmIODone (BIO completion in softirq) accesses freed lbuf fields
   after lbmLogShutdown frees all lbufs from the freelist.

V1 fixed only race #1.  Syzbot testing showed that race #2 can
still trigger independently.  This V2 adds a second patch for the
BIO completion race.

Changes since v1:
 - Split into two patches (one per race)
 - NEW: patch 2/2 adds atomic io_count to drain in-flight BIO
   completions before lbmLogShutdown frees lbufs

Tristan Madani (2):
  jfs: drain lazy commit queue during unmount to prevent use-after-free
  jfs: wait for in-flight log I/O before freeing lbufs in lbmLogShutdown

 fs/jfs/jfs_logmgr.c | 10 ++++++++++
 fs/jfs/jfs_logmgr.h |  2 ++
 fs/jfs/jfs_txnmgr.c | 35 +++++++++++++++++++++++++++++++++++
 fs/jfs/jfs_txnmgr.h |  1 +
 fs/jfs/jfs_umount.c  |  8 ++++++++
 5 files changed, 56 insertions(+)

-- 
2.47.3

[PATCH v2 1/2] jfs: drain lazy commit queue during unmount to prevent use-after-free

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

The jfsCommit kernel thread processes committed transactions from
TxAnchor.unlock_queue via jfs_lazycommit().  During filesystem
unmount, jfs_umount() calls jfs_flush_journal(log, 2) which waits
for the log commit queue (log->cqueue) to drain.  However, after
log I/O completes, lazy transactions are moved to
TxAnchor.unlock_queue for asynchronous processing by jfsCommit.

If jfs_umount() proceeds to free the jfs_log (via lmLogClose) or
jfs_sb_info (via kfree in jfs_put_super) while entries referencing
this superblock remain on unlock_queue, the jfsCommit thread will
access freed memory when it later processes these entries:

- jfs_lazycommit reads sbi->commit_state (UAF of jfs_sb_info)
- txLazyCommit accesses JFS_SBI(tblk->sb)->log and takes
  log->gclock (UAF of jfs_log)

Add txLazyDrain() which waits for all entries in
TxAnchor.unlock_queue belonging to the unmounting superblock to be
processed, and also waits for any in-flight txLazyCommit
(IN_LAZYCOMMIT) for this superblock to complete.  Call it from both
jfs_umount() and jfs_umount_rw() after jfs_flush_journal().

Reported-by: syzbot+c244f4a09ca85dd2ebc1@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=c244f4a09ca85dd2ebc1
Tested-by: syzbot+c244f4a09ca85dd2ebc1@syzkaller.appspotmail.com
Reported-by: syzbot+885a4f3281b8d99c48d8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=885a4f3281b8d99c48d8
Tested-by: syzbot+885a4f3281b8d99c48d8@syzkaller.appspotmail.com
Fixes: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 ("Linux-2.6.12-rc2")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
 fs/jfs/jfs_txnmgr.c | 35 +++++++++++++++++++++++++++++++++++
 fs/jfs/jfs_txnmgr.h |  1 +
 fs/jfs/jfs_umount.c |  8 ++++++++
 3 files changed, 44 insertions(+)

diff --git a/fs/jfs/jfs_txnmgr.c b/fs/jfs/jfs_txnmgr.c
index 083dbbb0c3268..67a9908b5a4d9 100644
--- a/fs/jfs/jfs_txnmgr.c
+++ b/fs/jfs/jfs_txnmgr.c
@@ -2791,6 +2791,41 @@ void txLazyUnlock(struct tblock * tblk)
 	LAZY_UNLOCK(flags);
 }
 
+
+/*
+ * txLazyDrain
+ *
+ * Wait for all pending lazy commit entries for this superblock
+ * to be processed by the jfsCommit thread.  Must be called
+ * before freeing per-filesystem structures during unmount.
+ */
+void txLazyDrain(struct super_block *sb)
+{
+	struct jfs_sb_info *sbi = JFS_SBI(sb);
+	struct tblock *tblk;
+	unsigned long flags;
+	bool found;
+
+	do {
+		found = false;
+		LAZY_LOCK(flags);
+		list_for_each_entry(tblk, &TxAnchor.unlock_queue, cqueue) {
+			if (tblk->sb == sb) {
+				found = true;
+				break;
+			}
+		}
+		if (!found && (sbi->commit_state & IN_LAZYCOMMIT))
+			found = true;
+		LAZY_UNLOCK(flags);
+
+		if (found) {
+			wake_up(&jfs_commit_thread_wait);
+			schedule_timeout_uninterruptible(1);
+		}
+	} while (found);
+}
+
 static void LogSyncRelease(struct metapage * mp)
 {
 	struct jfs_log *log = mp->log;
diff --git a/fs/jfs/jfs_txnmgr.h b/fs/jfs/jfs_txnmgr.h
index ba71eb5ced567..80ce468eadde0 100644
--- a/fs/jfs/jfs_txnmgr.h
+++ b/fs/jfs/jfs_txnmgr.h
@@ -291,6 +291,7 @@ extern void txFreelock(struct inode *);
 extern int lmLog(struct jfs_log *, struct tblock *, struct lrd *,
 		 struct tlock *);
 extern void txQuiesce(struct super_block *);
+extern void txLazyDrain(struct super_block *sb);
 extern void txResume(struct super_block *);
 extern void txLazyUnlock(struct tblock *);
 extern int jfs_lazycommit(void *);
diff --git a/fs/jfs/jfs_umount.c b/fs/jfs/jfs_umount.c
index 18569f1eaabdb..657707361be2a 100644
--- a/fs/jfs/jfs_umount.c
+++ b/fs/jfs/jfs_umount.c
@@ -58,6 +58,13 @@ int jfs_umount(struct super_block *sb)
 		 */
 		jfs_flush_journal(log, 2);
 
+	/*
+	 * Drain any pending lazy commit entries for this filesystem so
+	 * the jfsCommit thread does not access freed structures.
+	 */
+	if (log)
+		txLazyDrain(sb);
+
 	/*
 	 * Hold log lock so write_special_inodes (lmLogSync) cannot see
 	 * this sbi with a NULL inode pointer while iterating log->sb_list.
@@ -142,6 +149,7 @@ int jfs_umount_rw(struct super_block *sb)
 	 * remove file system from log active file system list.
 	 */
 	jfs_flush_journal(log, 2);
+	txLazyDrain(sb);
 
 	/*
 	 * Make sure all metadata makes it to disk
-- 
2.47.3

[PATCH v2 2/2] jfs: wait for in-flight log I/O before freeing lbufs in lbmLogShutdown

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

lbmLogShutdown() frees log buffer (lbuf) pages and structures from
log->lbuf_free.  However, BIO completions (lbmIODone) may still be
executing in softirq context when lbmLogShutdown() runs, because
lbmIODone accesses lbuf fields (l_flag, l_log, l_freelist) before
returning the buffer to the freelist.

If lbmLogShutdown() runs concurrently with lbmIODone in-flight,
it can free an lbuf that lbmIODone is still accessing -- resulting
in a use-after-free.

Fix this by adding an atomic io_count to struct jfs_log that tracks
in-flight BIO operations.  lbmStartIO increments it before submit_bio
(or before calling lbmIODone directly for no_integrity mode), and
lbmIODone decrements it after all lbuf accesses are complete.
lbmLogShutdown waits for io_count to reach zero before freeing any
lbufs.

Reported-by: syzbot+c244f4a09ca85dd2ebc1@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=c244f4a09ca85dd2ebc1
Reported-by: syzbot+885a4f3281b8d99c48d8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=885a4f3281b8d99c48d8
Fixes: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 ("Linux-2.6.12-rc2")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
 fs/jfs/jfs_logmgr.c | 12 ++++++++++++
 fs/jfs/jfs_logmgr.h |  2 ++
 2 files changed, 14 insertions(+)

diff --git a/fs/jfs/jfs_logmgr.c b/fs/jfs/jfs_logmgr.c
index 306165e61438c..95e95f71ec0fa 100644
--- a/fs/jfs/jfs_logmgr.c
+++ b/fs/jfs/jfs_logmgr.c
@@ -1804,6 +1804,8 @@ static int lbmLogInit(struct jfs_log * log)
 	 * avoid deadlock here.
 	 */
 	init_waitqueue_head(&log->free_wait);
+	atomic_set(&log->io_count, 0);
+	init_waitqueue_head(&log->io_done_wait);
 
 	log->lbuf_free = NULL;
 
@@ -1855,6 +1857,9 @@ static void lbmLogShutdown(struct jfs_log * log)
 
 	jfs_info("lbmLogShutdown: log:0x%p", log);
 
+	/* Wait for all in-flight log I/O to complete */
+	wait_event(log->io_done_wait, !atomic_read(&log->io_count));
+
 	lbuf = log->lbuf_free;
 	while (lbuf) {
 		struct lbuf *next = lbuf->l_freelist;
@@ -1976,6 +1981,8 @@ static int lbmRead(struct jfs_log * log, int pn, struct lbuf ** bpp)
 
 	bio->bi_end_io = lbmIODone;
 	bio->bi_private = bp;
+
+	atomic_inc(&log->io_count);
 	/*check if journaling to disk has been disabled*/
 	if (log->no_integrity) {
 		bio->bi_iter.bi_size = 0;
@@ -2123,6 +2130,8 @@ static void lbmStartIO(struct lbuf * bp)
 	bio->bi_end_io = lbmIODone;
 	bio->bi_private = bp;
 
+	atomic_inc(&log->io_count);
+
 	/* check if journaling to disk has been disabled */
 	if (log->no_integrity) {
 		bio->bi_iter.bi_size = 0;
@@ -2299,6 +2308,9 @@ static void lbmIODone(struct bio *bio)
 out:
 	bp->l_flag |= lbmDONE;
 	LCACHE_UNLOCK(flags);
+
+	if (atomic_dec_and_test(&bp->l_log->io_count))
+		wake_up(&bp->l_log->io_done_wait);
 }
 
 int jfsIOWait(void *arg)
diff --git a/fs/jfs/jfs_logmgr.h b/fs/jfs/jfs_logmgr.h
index 09e0ef6aeccef..cbf38ed27c950 100644
--- a/fs/jfs/jfs_logmgr.h
+++ b/fs/jfs/jfs_logmgr.h
@@ -367,6 +367,8 @@ struct jfs_log {
 
 	struct lbuf *lbuf_free;	/* 4: free lbufs */
 	wait_queue_head_t free_wait;	/* 4: */
+	atomic_t io_count;		/* in-flight log I/O count */
+	wait_queue_head_t io_done_wait;	/* wait for io_count == 0 */
 
 	/* log write */
 	int logtid;		/* 4: log tid */
-- 
2.47.3