[PATCH] ext4: Fix bh->b_state corruption

[PATCH] ext4: Fix bh->b_state corruption

[Jan Kara]

From: Jan Kara <jack@suse.com>

ext4 can update bh->b_state non-atomically in _ext4_get_block() and
ext4_da_get_block_prep(). Usually this is fine since bh is just a
temporary storage for mapping information on stack but in some cases it
can be fully living bh attached to a page. In such case non-atomic
update of bh->b_state can race with an atomic update which then gets
lost. Usually when we are mapping bh and thus updating bh->b_state
non-atomically, nobody else touches the bh and so things work out fine
but there is one case to especially worry about: ext4_finish_bio() uses
BH_Uptodate_Lock on the first bh in the page to synchronize handling of
PageWriteback state. So when blocksize < pagesize, we can be atomically
modifying bh->b_state of a buffer that actually isn't under IO and thus
can race e.g. with delalloc trying to map that buffer. The result is
that we can mistakenly set / clear BH_Uptodate_Lock bit resulting in the
corruption of PageWriteback state or missed unlock of BH_Uptodate_Lock.

Fix the problem by always updating bh->b_state bits atomically.

CC: stable@vger.kernel.org
Reported-by: Nikolay Borisov <kernel@kyup.com>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/inode.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index ea433a7f4bca..06bda0361e7c 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -657,6 +657,34 @@ has_zeroout:
 	return retval;
 }
 
+/*
+ * Update EXT4_MAP_FLAGS in bh->b_state. For buffer heads attached to pages
+ * we have to be careful as someone else may be manipulating b_state as well.
+ */
+static void ext4_update_bh_state(struct buffer_head *bh, unsigned long flags)
+{
+	unsigned long old_state;
+	unsigned long new_state;
+
+	flags &= EXT4_MAP_FLAGS;
+
+	/* Dummy buffer_head? Set non-atomically. */
+	if (!bh->b_page) {
+		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | flags;
+		return;
+	}
+	/*
+	 * Someone else may be modifying b_state. Be careful! This is ugly but
+	 * once we get rid of using bh as a container for mapping information
+	 * to pass to / from get_block functions, this can go away.
+	 */
+	do {
+		old_state = READ_ONCE(bh->b_state);
+		new_state = (old_state & ~EXT4_MAP_FLAGS) | flags;
+	} while (unlikely(
+		 cmpxchg(&bh->b_state, old_state, new_state) != old_state));
+}
+
 /* Maximum number of blocks we map for direct IO at once. */
 #define DIO_MAX_BLOCKS 4096
 
@@ -693,7 +721,7 @@ static int _ext4_get_block(struct inode *inode, sector_t iblock,
 		ext4_io_end_t *io_end = ext4_inode_aio(inode);
 
 		map_bh(bh, inode->i_sb, map.m_pblk);
-		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
+		ext4_update_bh_state(bh, map.m_flags);
 		if (IS_DAX(inode) && buffer_unwritten(bh)) {
 			/*
 			 * dgc: I suspect unwritten conversion on ext4+DAX is
@@ -1669,7 +1697,7 @@ int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,
 		return ret;
 
 	map_bh(bh, inode->i_sb, map.m_pblk);
-	bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
+	ext4_update_bh_state(bh, map.m_flags);
 
 	if (buffer_unwritten(bh)) {
 		/* A delayed write to unwritten bh should be marked
-- 
2.6.2

Re: [PATCH] ext4: Fix bh->b_state corruption

[Nikolay Borisov]

Ping on that one, it seems it's going to miss the 4.5 merge window?

On 01/07/2016 06:55 PM, Jan Kara wrote:
> From: Jan Kara <jack@suse.com>
> 
> ext4 can update bh->b_state non-atomically in _ext4_get_block() and
> ext4_da_get_block_prep(). Usually this is fine since bh is just a
> temporary storage for mapping information on stack but in some cases it
> can be fully living bh attached to a page. In such case non-atomic
> update of bh->b_state can race with an atomic update which then gets
> lost. Usually when we are mapping bh and thus updating bh->b_state
> non-atomically, nobody else touches the bh and so things work out fine
> but there is one case to especially worry about: ext4_finish_bio() uses
> BH_Uptodate_Lock on the first bh in the page to synchronize handling of
> PageWriteback state. So when blocksize < pagesize, we can be atomically
> modifying bh->b_state of a buffer that actually isn't under IO and thus
> can race e.g. with delalloc trying to map that buffer. The result is
> that we can mistakenly set / clear BH_Uptodate_Lock bit resulting in the
> corruption of PageWriteback state or missed unlock of BH_Uptodate_Lock.
> 
> Fix the problem by always updating bh->b_state bits atomically.
> 
> CC: stable@vger.kernel.org
> Reported-by: Nikolay Borisov <kernel@kyup.com>
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
>  fs/ext4/inode.c | 32 ++++++++++++++++++++++++++++++--
>  1 file changed, 30 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index ea433a7f4bca..06bda0361e7c 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -657,6 +657,34 @@ has_zeroout:
>  	return retval;
>  }
>  
> +/*
> + * Update EXT4_MAP_FLAGS in bh->b_state. For buffer heads attached to pages
> + * we have to be careful as someone else may be manipulating b_state as well.
> + */
> +static void ext4_update_bh_state(struct buffer_head *bh, unsigned long flags)
> +{
> +	unsigned long old_state;
> +	unsigned long new_state;
> +
> +	flags &= EXT4_MAP_FLAGS;
> +
> +	/* Dummy buffer_head? Set non-atomically. */
> +	if (!bh->b_page) {
> +		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | flags;
> +		return;
> +	}
> +	/*
> +	 * Someone else may be modifying b_state. Be careful! This is ugly but
> +	 * once we get rid of using bh as a container for mapping information
> +	 * to pass to / from get_block functions, this can go away.
> +	 */
> +	do {
> +		old_state = READ_ONCE(bh->b_state);
> +		new_state = (old_state & ~EXT4_MAP_FLAGS) | flags;
> +	} while (unlikely(
> +		 cmpxchg(&bh->b_state, old_state, new_state) != old_state));
> +}
> +
>  /* Maximum number of blocks we map for direct IO at once. */
>  #define DIO_MAX_BLOCKS 4096
>  
> @@ -693,7 +721,7 @@ static int _ext4_get_block(struct inode *inode, sector_t iblock,
>  		ext4_io_end_t *io_end = ext4_inode_aio(inode);
>  
>  		map_bh(bh, inode->i_sb, map.m_pblk);
> -		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
> +		ext4_update_bh_state(bh, map.m_flags);
>  		if (IS_DAX(inode) && buffer_unwritten(bh)) {
>  			/*
>  			 * dgc: I suspect unwritten conversion on ext4+DAX is
> @@ -1669,7 +1697,7 @@ int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,
>  		return ret;
>  
>  	map_bh(bh, inode->i_sb, map.m_pblk);
> -	bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
> +	ext4_update_bh_state(bh, map.m_flags);
>  
>  	if (buffer_unwritten(bh)) {
>  		/* A delayed write to unwritten bh should be marked
> 

Re: [PATCH] ext4: Fix bh->b_state corruption

[Jan Kara]

On Thu 07-01-16 17:55:21, Jan Kara wrote:
> From: Jan Kara <jack@suse.com>
> 
> ext4 can update bh->b_state non-atomically in _ext4_get_block() and
> ext4_da_get_block_prep(). Usually this is fine since bh is just a
> temporary storage for mapping information on stack but in some cases it
> can be fully living bh attached to a page. In such case non-atomic
> update of bh->b_state can race with an atomic update which then gets
> lost. Usually when we are mapping bh and thus updating bh->b_state
> non-atomically, nobody else touches the bh and so things work out fine
> but there is one case to especially worry about: ext4_finish_bio() uses
> BH_Uptodate_Lock on the first bh in the page to synchronize handling of
> PageWriteback state. So when blocksize < pagesize, we can be atomically
> modifying bh->b_state of a buffer that actually isn't under IO and thus
> can race e.g. with delalloc trying to map that buffer. The result is
> that we can mistakenly set / clear BH_Uptodate_Lock bit resulting in the
> corruption of PageWriteback state or missed unlock of BH_Uptodate_Lock.
> 
> Fix the problem by always updating bh->b_state bits atomically.
> 
> CC: stable@vger.kernel.org
> Reported-by: Nikolay Borisov <kernel@kyup.com>
> Signed-off-by: Jan Kara <jack@suse.cz>

Ping Ted? This seems to have fallen through the cracks?

								Honza

> ---
>  fs/ext4/inode.c | 32 ++++++++++++++++++++++++++++++--
>  1 file changed, 30 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index ea433a7f4bca..06bda0361e7c 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -657,6 +657,34 @@ has_zeroout:
>  	return retval;
>  }
>  
> +/*
> + * Update EXT4_MAP_FLAGS in bh->b_state. For buffer heads attached to pages
> + * we have to be careful as someone else may be manipulating b_state as well.
> + */
> +static void ext4_update_bh_state(struct buffer_head *bh, unsigned long flags)
> +{
> +	unsigned long old_state;
> +	unsigned long new_state;
> +
> +	flags &= EXT4_MAP_FLAGS;
> +
> +	/* Dummy buffer_head? Set non-atomically. */
> +	if (!bh->b_page) {
> +		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | flags;
> +		return;
> +	}
> +	/*
> +	 * Someone else may be modifying b_state. Be careful! This is ugly but
> +	 * once we get rid of using bh as a container for mapping information
> +	 * to pass to / from get_block functions, this can go away.
> +	 */
> +	do {
> +		old_state = READ_ONCE(bh->b_state);
> +		new_state = (old_state & ~EXT4_MAP_FLAGS) | flags;
> +	} while (unlikely(
> +		 cmpxchg(&bh->b_state, old_state, new_state) != old_state));
> +}
> +
>  /* Maximum number of blocks we map for direct IO at once. */
>  #define DIO_MAX_BLOCKS 4096
>  
> @@ -693,7 +721,7 @@ static int _ext4_get_block(struct inode *inode, sector_t iblock,
>  		ext4_io_end_t *io_end = ext4_inode_aio(inode);
>  
>  		map_bh(bh, inode->i_sb, map.m_pblk);
> -		bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
> +		ext4_update_bh_state(bh, map.m_flags);
>  		if (IS_DAX(inode) && buffer_unwritten(bh)) {
>  			/*
>  			 * dgc: I suspect unwritten conversion on ext4+DAX is
> @@ -1669,7 +1697,7 @@ int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,
>  		return ret;
>  
>  	map_bh(bh, inode->i_sb, map.m_pblk);
> -	bh->b_state = (bh->b_state & ~EXT4_MAP_FLAGS) | map.m_flags;
> +	ext4_update_bh_state(bh, map.m_flags);
>  
>  	if (buffer_unwritten(bh)) {
>  		/* A delayed write to unwritten bh should be marked
> -- 
> 2.6.2
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH] ext4: Fix bh->b_state corruption

[Theodore Ts'o]

On Thu, Feb 18, 2016 at 05:09:48PM +0100, Jan Kara wrote:
> On Thu 07-01-16 17:55:21, Jan Kara wrote:
> > From: Jan Kara <jack@suse.com>
> > 
> > ext4 can update bh->b_state non-atomically in _ext4_get_block() and
> > ext4_da_get_block_prep(). Usually this is fine since bh is just a
> > temporary storage for mapping information on stack but in some cases it
> > can be fully living bh attached to a page. In such case non-atomic
> > update of bh->b_state can race with an atomic update which then gets
> > lost. Usually when we are mapping bh and thus updating bh->b_state
> > non-atomically, nobody else touches the bh and so things work out fine
> > but there is one case to especially worry about: ext4_finish_bio() uses
> > BH_Uptodate_Lock on the first bh in the page to synchronize handling of
> > PageWriteback state. So when blocksize < pagesize, we can be atomically
> > modifying bh->b_state of a buffer that actually isn't under IO and thus
> > can race e.g. with delalloc trying to map that buffer. The result is
> > that we can mistakenly set / clear BH_Uptodate_Lock bit resulting in the
> > corruption of PageWriteback state or missed unlock of BH_Uptodate_Lock.
> > 
> > Fix the problem by always updating bh->b_state bits atomically.
> > 
> > CC: stable@vger.kernel.org
> > Reported-by: Nikolay Borisov <kernel@kyup.com>
> > Signed-off-by: Jan Kara <jack@suse.cz>
> 
> Ping Ted? This seems to have fallen through the cracks?

Oops, sorry.  Thanks, applied.

						- Ted