found an error that e2fsck does not fix

found an error that e2fsck does not fix

[Bo Branten]

Hello,

I am debugging an Windows driver for ext4 and when creating an zero length 
file the error below shows up on Linux, while this ofcause is an bug in 
the Windows driver (looks like it forgets to flush some metadata) I wanted 
to report here that every time I run fsck on this test volume it will mark 
it as clean again but without fixing the error, so if I mount and unmount 
the filesystem again it is again in need of fsck wich in turn will mark it 
clean witout fixing the problem.

Test case, testa and testb is created from Windows and only testb gets one 
character written to with notepad:

bo@ubuntu:~$ cd /mnt2/src
bo@ubuntu:/mnt2/src$ ls -l
ls: cannot access testa: Input/output error
total 4
drwxr-xr-x 15 bo bo 1024 Mar 13  1994 linux-1.0
drwxr-xr-x 13 bo bo 1024 Mar  7  1995 linux-1.2.0
drwxr-xr-x 15 bo bo 1024 Jun  9  1996 linux-2.0
-?????????  ? ?  ?     ?            ? testa
-rw-r--r--  1 bo bo    1 Nov 27 18:21 testb
bo@ubuntu:/mnt2/src$ dmesg|tail
[ 1657.016023] ieee80211 phy0: rt2x00queue_flush_queue: Warning - Queue 2 
failed to flush
[ 1773.388021] ieee80211 phy0: rt2x00queue_flush_queue: Warning - Queue 2 
failed to flush
[ 1774.908022] ieee80211 phy0: rt2x00queue_flush_queue: Warning - Queue 2 
failed to flush
[ 2254.276023] ieee80211 phy0: rt2x00queue_flush_queue: Warning - Queue 2 
failed to flush
[ 3064.029061] EXT4-fs (sdb5): mounted filesystem with ordered data mode. 
Opts: (null)
[ 3081.376794] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: 
inode #3641: comm ls: bad header/extent: invalid magic - magic 0, entries 
0, max 0(0), depth 0(0)
[ 4097.233826] EXT4-fs (sdb5): mounted filesystem with ordered data mode. 
Opts: (null)
[ 4108.835169] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: 
inode #3641: comm ls: bad header/extent: invalid magic - magic 0, entries 
0, max 0(0), depth 0(0)
[ 6333.410987] EXT4-fs (sdb5): mounted filesystem with ordered data mode. 
Opts: (null)
[ 6338.997450] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: 
inode #3641: comm ls: bad header/extent: invalid magic - magic 0, entries 
0, max 0(0), depth 0(0)
bo@ubuntu:/mnt2/src$

Re: found an error that e2fsck does not fix

[Theodore Ts'o]

On Fri, Nov 27, 2015 at 08:11:34PM +0100, Bo Branten wrote:
> 
> I am debugging an Windows driver for ext4 and when creating an zero length
> file the error below shows up on Linux, while this ofcause is an bug in the
> Windows driver (looks like it forgets to flush some metadata) I wanted to
> report here that every time I run fsck on this test volume it will mark it
> as clean again but without fixing the error, so if I mount and unmount the
> filesystem again it is again in need of fsck wich in turn will mark it clean
> witout fixing the problem.
> 
> Test case, testa and testb is created from Windows and only testb gets one
> character written to with notepad:

This test case doesn't really help since I don't have access to your
potentially buggy Windows driver for ext4.

What would be helpful:

(a) the output of "e2fsck -fy /dev/sdXX" run twice.  If e2fsck isn't
fixing a problem, then a forced check should show it trying to fix the
problem again.  Also the exact output of what e2fsck prints would be
heplful.  It's possible that mounting and unmounting the file system
using the Windows driver is causing the problem, so running "e2fsck
-fy" twice in succeession will determine whether or not this is the
case.  In general, "e2fsck -fy" should fix all corruptions in a single
run.  So a second run of "e2fsck -fy", if it shows errors, is an
indication of either (a) a hardware problem (e.g., so an update of the
file system image is getting lost or the disk image is geting
changed/corrupted between runs of e2fsck), or (b) a bug in e2fsck.

(b) See the REPORTING BUGS section of the e2fsck man page; in
particular, note its suggestion to use e2image.  The e2image man page
will have more details, but the short version is to send me the output
of "e2image -r /dev/sdXX - | bzip2 > sdXX.e2i.bz2".

If you can reproduce the problem on an arbitrary file system, create a
small test file system "mke2fs -b 4096 /dev/sdXX 1M" and then either
use the e2image directions described above to send me a minimal test
case.

Thanks,

					- Ted

Re: found an error that e2fsck does not fix

[Bo Branten]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3351 bytes --]


Hello,

first I want to say that we have found the bug: ext4_ext_tree_init was 
only called when first extending the file and not at create time.

However e2fsck 1.42.13 (17-May-2015) does not fix the error so I have 
created the test images you asked for:

/* create test fs */
sudo dd bs=1024 count=1024 if=/dev/zero of=/dev/sdb5
sudo mke2fs -b 4096 /dev/sdb5 1M
sudo tune2fs -O extents /dev/sdb5

/* reboot to Windows and create file 'testa' and 'testb' */

/* create the images attached */
sudo dd bs=1024 count=1024 if=/dev/sdb5 of=ext4.img
sudo e2image -r /dev/sdb5 - | bzip2 > sdb5.e2i.bz2

/* then try the fs from Linux */
bo@ubuntu:~$ sudo mount -t ext4 /dev/sdb5 /mnt
bo@ubuntu:~$ ls -l /mnt
ls: cannot access /mnt/testa: Input/output error
total 20
drwx------ 2 root root 16384 Nov 28 18:09 lost+found
-????????? ? ?    ?        ?            ? testa
-rw-r--r-- 1 root root     1 Nov 28 18:15 testb
bo@ubuntu:~$ cat /mnt/testa
cat: /mnt/testa: Input/output error
bo@ubuntu:~$ cat /mnt/testb
bo@ubuntu:~$ sudo umount /mnt
bo@ubuntu:~$ sudo ~bo/src/e2fsprogs-1.42.13/e2fsck/e2fsck -fy /dev/sdb5
e2fsck 1.42.13 (17-May-2015)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb5: 13/128 files (0.0% non-contiguous), 15/256 blocks
bo@ubuntu:~$ sudo ~bo/src/e2fsprogs-1.42.13/e2fsck/e2fsck -fy /dev/sdb5
e2fsck 1.42.13 (17-May-2015)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb5: 13/128 files (0.0% non-contiguous), 15/256 blocks
bo@ubuntu:~$ sudo mount -t ext4 /dev/sdb5 /mnt
bo@ubuntu:~$ ls -l /mnt
ls: cannot access /mnt/testa: Input/output error
total 20
drwx------ 2 root root 16384 Nov 28 18:09 lost+found
-????????? ? ?    ?        ?            ? testa
-rw-r--r-- 1 root root     1 Nov 28 18:15 testb
bo@ubuntu:~$ cat /mnt/testa
cat: /mnt/testa: Input/output error
bo@ubuntu:~$ cat /mnt/testb
bbo@ubuntu:~$ 
bbo@ubuntu:~$ dmesg|tail
[ 1376.496018] wlan0: associate with ec:43:f6:37:14:14 (try 1/3)
[ 1376.501718] wlan0: RX AssocResp from ec:43:f6:37:14:14 (capab=0xc11 status=0 aid=3)
[ 1376.501967] wlan0: associated
[ 1407.949284] EXT4-fs (sdb5): mounted filesystem without journal. Opts: (null)
[ 1420.748311] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm ls: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
[ 1429.169785] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm bash: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
[ 1430.328150] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm cat: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
[ 1457.061338] EXT4-fs (sdb5): mounted filesystem without journal. Opts: (null)
[ 1469.698684] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm ls: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
[ 1474.838003] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm cat: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
bo@ubuntu:~$

[-- Attachment #2: Type: APPLICATION/octet-stream, Size: 453 bytes --]

[-- Attachment #3: Type: APPLICATION/octet-stream, Size: 449 bytes --]

Re: found an error that e2fsck does not fix

[Darrick J. Wong]

On Sat, Nov 28, 2015 at 07:08:49PM +0100, Bo Branten wrote:
> 
> Hello,
> 
> first I want to say that we have found the bug: ext4_ext_tree_init
> was only called when first extending the file and not at create
> time.

Looks like i_block[] is zeroed, so when e2fsck opens the extent tree in
check_blocks, ext2fs_extent_open2() adds the extent tree header, which causes
the header check to pass.  Nothing ever writes out the inode, so the new header
isn't written to the filesystem.

Since the kernel expects to find an extent header if the inode flag is set,
I'll fix up e2fsck to zap an inode if the flag is set and no header is found.

Thank you for the tiny image file, that makes it very easy to create a
regression test!

--D

> 
> However e2fsck 1.42.13 (17-May-2015) does not fix the error so I
> have created the test images you asked for:
> 
> /* create test fs */
> sudo dd bs=1024 count=1024 if=/dev/zero of=/dev/sdb5
> sudo mke2fs -b 4096 /dev/sdb5 1M
> sudo tune2fs -O extents /dev/sdb5
> 
> /* reboot to Windows and create file 'testa' and 'testb' */
> 
> /* create the images attached */
> sudo dd bs=1024 count=1024 if=/dev/sdb5 of=ext4.img
> sudo e2image -r /dev/sdb5 - | bzip2 > sdb5.e2i.bz2
> 
> /* then try the fs from Linux */
> bo@ubuntu:~$ sudo mount -t ext4 /dev/sdb5 /mnt
> bo@ubuntu:~$ ls -l /mnt
> ls: cannot access /mnt/testa: Input/output error
> total 20
> drwx------ 2 root root 16384 Nov 28 18:09 lost+found
> -????????? ? ?    ?        ?            ? testa
> -rw-r--r-- 1 root root     1 Nov 28 18:15 testb
> bo@ubuntu:~$ cat /mnt/testa
> cat: /mnt/testa: Input/output error
> bo@ubuntu:~$ cat /mnt/testb
> bo@ubuntu:~$ sudo umount /mnt
> bo@ubuntu:~$ sudo ~bo/src/e2fsprogs-1.42.13/e2fsck/e2fsck -fy /dev/sdb5
> e2fsck 1.42.13 (17-May-2015)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/sdb5: 13/128 files (0.0% non-contiguous), 15/256 blocks
> bo@ubuntu:~$ sudo ~bo/src/e2fsprogs-1.42.13/e2fsck/e2fsck -fy /dev/sdb5
> e2fsck 1.42.13 (17-May-2015)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/sdb5: 13/128 files (0.0% non-contiguous), 15/256 blocks
> bo@ubuntu:~$ sudo mount -t ext4 /dev/sdb5 /mnt
> bo@ubuntu:~$ ls -l /mnt
> ls: cannot access /mnt/testa: Input/output error
> total 20
> drwx------ 2 root root 16384 Nov 28 18:09 lost+found
> -????????? ? ?    ?        ?            ? testa
> -rw-r--r-- 1 root root     1 Nov 28 18:15 testb
> bo@ubuntu:~$ cat /mnt/testa
> cat: /mnt/testa: Input/output error
> bo@ubuntu:~$ cat /mnt/testb
> bbo@ubuntu:~$ bbo@ubuntu:~$ dmesg|tail
> [ 1376.496018] wlan0: associate with ec:43:f6:37:14:14 (try 1/3)
> [ 1376.501718] wlan0: RX AssocResp from ec:43:f6:37:14:14 (capab=0xc11 status=0 aid=3)
> [ 1376.501967] wlan0: associated
> [ 1407.949284] EXT4-fs (sdb5): mounted filesystem without journal. Opts: (null)
> [ 1420.748311] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm ls: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
> [ 1429.169785] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm bash: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
> [ 1430.328150] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm cat: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
> [ 1457.061338] EXT4-fs (sdb5): mounted filesystem without journal. Opts: (null)
> [ 1469.698684] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm ls: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
> [ 1474.838003] EXT4-fs error (device sdb5): ext4_ext_check_inode:464: inode #12: comm cat: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)
> bo@ubuntu:~$