Open bugs found by fuzzing as of 2016-07-30

Open bugs found by fuzzing as of 2016-07-30

[Vegard Nossum]

Hi,

It's been two weeks since I posted the first list of bugs found using
AFL: https://www.spinics.net/lists/linux-ext4/msg53022.html

With a bunch of ext4 patches going into 4.8 we're down from 15 to 6
with current linus/master:

1. general protection fault: 0000 [#1] KASAN
http://139.162.151.198/f/ext4/57be666646a37e9821d52bc64846a3b3b785ee7a

2. kernel BUG at fs/buffer.c:3061!
http://139.162.151.198/f/ext4/7df880da89c82579c15ca8bc786a3467ca9c47f7

3. kernel BUG at fs/ext4/inode.c:3738!
http://139.162.151.198/f/ext4/5bdefda69f39b2f2c56d9b67d5b7d9e2cc8dfd5f
(discussion: https://www.spinics.net/lists/linux-ext4/msg53032.html)

4. kernel BUG at fs/ext4/mballoc.c:3191!
http://139.162.151.198/f/ext4/34284738d67f0405325b2c43211c56020b9d0211

5. kernel BUG at fs/jbd2/commit.c:825!
http://139.162.151.198/f/ext4/3143febf7925bd1ea398bd1a775551133bd69ffd

6. WARNING: CPU: 0 PID: 58 at fs/ext4/ext4.h:2748 
ext4_block_bitmap_csum_set+0x358/0x600
http://139.162.151.198/f/ext4/9628c19aff0bbaaae4149a03486305c7f6cd7523


Vegard

Re: Open bugs found by fuzzing as of 2016-07-30

[nborisov]

On 30.07.2016 16:04, Vegard Nossum wrote:
> Hi,

Hi Vegard,

> 
> It's been two weeks since I posted the first list of bugs found using
> AFL: https://www.spinics.net/lists/linux-ext4/msg53022.html
> 
> With a bunch of ext4 patches going into 4.8 we're down from 15 to 6
> with current linus/master:

Are the patches going into 4.8 tagged for stable or are they going to go
just in to 4.8?

> 
> 1. general protection fault: 0000 [#1] KASAN
> http://139.162.151.198/f/ext4/57be666646a37e9821d52bc64846a3b3b785ee7a
> 
> 2. kernel BUG at fs/buffer.c:3061!
> http://139.162.151.198/f/ext4/7df880da89c82579c15ca8bc786a3467ca9c47f7
> 
> 3. kernel BUG at fs/ext4/inode.c:3738!
> http://139.162.151.198/f/ext4/5bdefda69f39b2f2c56d9b67d5b7d9e2cc8dfd5f
> (discussion: https://www.spinics.net/lists/linux-ext4/msg53032.html)
> 
> 4. kernel BUG at fs/ext4/mballoc.c:3191!
> http://139.162.151.198/f/ext4/34284738d67f0405325b2c43211c56020b9d0211
> 
> 5. kernel BUG at fs/jbd2/commit.c:825!
> http://139.162.151.198/f/ext4/3143febf7925bd1ea398bd1a775551133bd69ffd
> 
> 6. WARNING: CPU: 0 PID: 58 at fs/ext4/ext4.h:2748
> ext4_block_bitmap_csum_set+0x358/0x600
> http://139.162.151.198/f/ext4/9628c19aff0bbaaae4149a03486305c7f6cd7523
> 
> 
> Vegard
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Open bugs found by fuzzing as of 2016-07-30

[Vegard Nossum]

On 07/30/2016 08:39 PM, nborisov wrote:
> On 30.07.2016 16:04, Vegard Nossum wrote:
>> It's been two weeks since I posted the first list of bugs found using
>> AFL: https://www.spinics.net/lists/linux-ext4/msg53022.html
>>
>> With a bunch of ext4 patches going into 4.8 we're down from 15 to 6
>> with current linus/master:
>
> Are the patches going into 4.8 tagged for stable or are they going to go
> just in to 4.8?

It seems that most of them are indeed tagged for stable; for the patches
I submitted or reported for, it looks like the only one without a stable
tag is a warning about large memory allocations -- commit
7bc9491645118c9461bd21099c31755ff6783593.

Keep in mind that these bugs were found by fuzzing/intentional
corruption and so you are unlikely to run into them by chance.


Vegard

Re: Open bugs found by fuzzing as of 2016-07-30

[Theodore Ts'o]

On Sat, Jul 30, 2016 at 09:25:15PM +0200, Vegard Nossum wrote:
> 
> It seems that most of them are indeed tagged for stable; for the patches
> I submitted or reported for, it looks like the only one without a stable
> tag is a warning about large memory allocations -- commit
> 7bc9491645118c9461bd21099c31755ff6783593.

... and that's an oversight.  Commit 7bc949164511 ("ext4: verify
extent header depth") should have been marked for stable.  I've added
stable@vger.kernel.org to the cc.  Could you please it to the stable
kernels?  Many thanks!!

					- Ted

Re: Open bugs found by fuzzing as of 2016-07-30

[Greg KH]

On Sun, Jul 31, 2016 at 12:37:27AM -0400, Theodore Ts'o wrote:
> On Sat, Jul 30, 2016 at 09:25:15PM +0200, Vegard Nossum wrote:
> > 
> > It seems that most of them are indeed tagged for stable; for the patches
> > I submitted or reported for, it looks like the only one without a stable
> > tag is a warning about large memory allocations -- commit
> > 7bc9491645118c9461bd21099c31755ff6783593.
> 
> ... and that's an oversight.  Commit 7bc949164511 ("ext4: verify
> extent header depth") should have been marked for stable.  I've added
> stable@vger.kernel.org to the cc.  Could you please it to the stable
> kernels?  Many thanks!!

Now queued up, thanks.

greg k-h

Re: Open bugs found by fuzzing as of 2016-07-30

[Theodore Ts'o]

On Wed, Aug 03, 2016 at 07:43:59AM +0200, Greg KH wrote:
> 
> Now queued up, thanks.
>

Thank you, greg!

					- Ted

Re: Open bugs found by fuzzing as of 2016-07-30

[Theodore Ts'o]

On Sat, Jul 30, 2016 at 03:04:43PM +0200, Vegard Nossum wrote:
> Hi,
> 
> It's been two weeks since I posted the first list of bugs found using
> AFL: https://www.spinics.net/lists/linux-ext4/msg53022.html
> 
> With a bunch of ext4 patches going into 4.8 we're down from 15 to 6
> with current linus/master...

Does this patch bring things down further?  I expect it should at the
very list address

> 6. WARNING: CPU: 0 PID: 58 at fs/ext4/ext4.h:2748
> ext4_block_bitmap_csum_set+0x358/0x600
> http://139.162.151.198/f/ext4/9628c19aff0bbaaae4149a03486305c7f6cd7523

... and possibly others.

If there are any remaining of these bugs where the superblock is
sufficiently corrupt that dumpe2fs refuses to print anything, could
you print a hex dump of the superblock (located at offset 1024) so we
could see what is going on?

					- Ted

commit 0a8bffdacb178a43a1be61270f22517de76ee8f8
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Mon Aug 1 00:51:02 2016 -0400

    ext4: validate that metadata blocks do not overlap superblock
    
    A number of fuzzing failures seem to be caused by allocation bitmaps
    or other metadata blocks being pointed at the superblock.
    
    This can cause kernel BUG or WARNings once the superblock is
    overwritten, so validate the group descriptor blocks to make sure this
    doesn't happen.
    
    Signed-off-by: Theodore Ts'o <tytso@mit.edu>

diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index e2622ba..2942fda 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -2211,6 +2211,7 @@ void ext4_group_desc_csum_set(struct super_block *sb, __u32 block_group,
 
 /* Called at mount-time, super-block is locked */
 static int ext4_check_descriptors(struct super_block *sb,
+				  ext4_fsblk_t sb_block,
 				  ext4_group_t *first_not_zeroed)
 {
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
@@ -2241,6 +2242,11 @@ static int ext4_check_descriptors(struct super_block *sb,
 			grp = i;
 
 		block_bitmap = ext4_block_bitmap(sb, gdp);
+		if (block_bitmap == sb_block) {
+			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
+				 "Block bitmap for group %u overlaps "
+				 "superblock", i);
+		}
 		if (block_bitmap < first_block || block_bitmap > last_block) {
 			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
 			       "Block bitmap for group %u not in group "
@@ -2248,6 +2254,11 @@ static int ext4_check_descriptors(struct super_block *sb,
 			return 0;
 		}
 		inode_bitmap = ext4_inode_bitmap(sb, gdp);
+		if (inode_bitmap == sb_block) {
+			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
+				 "Inode bitmap for group %u overlaps "
+				 "superblock", i);
+		}
 		if (inode_bitmap < first_block || inode_bitmap > last_block) {
 			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
 			       "Inode bitmap for group %u not in group "
@@ -2255,6 +2266,11 @@ static int ext4_check_descriptors(struct super_block *sb,
 			return 0;
 		}
 		inode_table = ext4_inode_table(sb, gdp);
+		if (inode_table == sb_block) {
+			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
+				 "Inode table for group %u overlaps "
+				 "superblock", i);
+		}
 		if (inode_table < first_block ||
 		    inode_table + sbi->s_itb_per_group - 1 > last_block) {
 			ext4_msg(sb, KERN_ERR, "ext4_check_descriptors: "
@@ -3757,7 +3773,7 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 			goto failed_mount2;
 		}
 	}
-	if (!ext4_check_descriptors(sb, &first_not_zeroed)) {
+	if (!ext4_check_descriptors(sb, logical_sb_block, &first_not_zeroed)) {
 		ext4_msg(sb, KERN_ERR, "group descriptors corrupted!");
 		ret = -EFSCORRUPTED;
 		goto failed_mount2;

Re: Open bugs found by fuzzing as of 2016-07-30

[Vegard Nossum]

On 08/01/2016 06:55 AM, Theodore Ts'o wrote:
> On Sat, Jul 30, 2016 at 03:04:43PM +0200, Vegard Nossum wrote:
>> Hi,
>>
>> It's been two weeks since I posted the first list of bugs found using
>> AFL: https://www.spinics.net/lists/linux-ext4/msg53022.html
>>
>> With a bunch of ext4 patches going into 4.8 we're down from 15 to 6
>> with current linus/master...
>
> Does this patch bring things down further?  I expect it should at the
> very list address
>
>> 6. WARNING: CPU: 0 PID: 58 at fs/ext4/ext4.h:2748
>> ext4_block_bitmap_csum_set+0x358/0x600
>> http://139.162.151.198/f/ext4/9628c19aff0bbaaae4149a03486305c7f6cd7523
>
> ... and possibly others.

I applied the patch, but I didn't see any of the bugs go away,
unfortunately.

IIRC there were still bugs in ext4_init_block_bitmap() where the
ext4_set_bit() calls for the block bitmap + inode bitmap + inode table
were writing beyond the end of bh->b_data. I think tmp < start or
something and then the ext4_set_bit() calls actually end up writing
into the superblock itself, causing either ext4_inode_table() or
sbi->s_itb_per_group to start returning bogus values and further corrupt
things. I'll have another look, maybe add some printks.

> If there are any remaining of these bugs where the superblock is
> sufficiently corrupt that dumpe2fs refuses to print anything, could
> you print a hex dump of the superblock (located at offset 1024) so we
> could see what is going on?

I've added the hex dumps and updated the pages.


Vegard