[PATCH v1 29/36] ext4: snapshot race conditions - concurrent COW bitmap operations

[amir73il@users.sourceforge.net]

From: Amir Goldstein <amir73il@users.sf.net>

Wait for pending COW bitmap creations to complete.
When concurrent tasks try to COW buffers from the same block group
for the first time, the first task to reset the COW bitmap cache
is elected to create the new COW bitmap block.  The rest of the tasks
wait (in msleep(1) loop), until the COW bitmap cache is uptodate.
The COWing task copies the bitmap block into the new COW bitmap block
and updates the COW bitmap cache with the new block number.


Signed-off-by: Amir Goldstein <amir73il@users.sf.net>
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/snapshot.c |   45 ++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/fs/ext4/snapshot.c b/fs/ext4/snapshot.c
index 2724381..000e655 100644
--- a/fs/ext4/snapshot.c
+++ b/fs/ext4/snapshot.c
@@ -248,9 +248,48 @@ ext4_snapshot_read_cow_bitmap(handle_t *handle, struct inode *snapshot,
 
 	bitmap_blk = ext4_block_bitmap(sb, desc);
 
-	ext4_lock_group(sb, block_group);
-	cow_bitmap_blk = grp->bg_cow_bitmap;
-	ext4_unlock_group(sb, block_group);
+	/*
+	 * Handle concurrent COW bitmap operations.
+	 * bg_cow_bitmap has 3 states:
+	 * = 0 - uninitialized (after mount and after snapshot take).
+	 * = bg_block_bitmap - marks pending COW of block bitmap.
+	 * other - location of initialized COW bitmap block.
+	 *
+	 * The first task to access block group after mount or snapshot take,
+	 * will read the uninitialized state, mark pending COW state, initialize
+	 * the COW bitmap block and update COW bitmap cache.  Other tasks will
+	 * busy wait until the COW bitmap cache is in initialized state, before
+	 * reading the COW bitmap block.
+	 */
+	do {
+		ext4_lock_group(sb, block_group);
+		cow_bitmap_blk = grp->bg_cow_bitmap;
+		if (cow_bitmap_blk == 0)
+			/* mark pending COW of bitmap block */
+			grp->bg_cow_bitmap = bitmap_blk;
+		ext4_unlock_group(sb, block_group);
+
+		if (cow_bitmap_blk == 0) {
+			snapshot_debug(3, "initializing COW bitmap #%u "
+					"of snapshot (%u)...\n",
+					block_group, snapshot->i_generation);
+			/* sleep 1 tunable delay unit */
+			snapshot_test_delay(SNAPTEST_BITMAP);
+			break;
+		}
+		if (cow_bitmap_blk == bitmap_blk) {
+			/* wait for another task to COW bitmap block */
+			snapshot_debug_once(2, "waiting for pending COW "
+					    "bitmap #%d...\n", block_group);
+			/*
+			 * This is an unlikely event that can happen only once
+			 * per block_group/snapshot, so msleep(1) is sufficient
+			 * and there is no need for a wait queue.
+			 */
+			msleep(1);
+		}
+		/* XXX: Should we fail after N retries? */
+	} while (cow_bitmap_blk == 0 || cow_bitmap_blk == bitmap_blk);
 	if (cow_bitmap_blk)
 		return sb_bread(sb, cow_bitmap_blk);
 
-- 
1.7.4.1