* Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS [not found] ` <CAOWv6b1m8A2S3sjDsJxOrRVztch6iq4zH9eFq5DKFMqtWWWpEw@mail.gmail.com> @ 2013-09-25 2:25 ` InvTraySts 2013-09-25 16:12 ` Jan Kara 0 siblings, 1 reply; 8+ messages in thread From: InvTraySts @ 2013-09-25 2:25 UTC (permalink / raw) To: linux-ext4, linux-fsdevel So long story short, I had a server running that had a processor fail while powered on, causing the file systems to become corrupt. I replaced the motherboard, processor and power supply just to be on the safe side. However, I am at a bit of a loss as to what to do now. I was working sandeen in the OFTC IRC channel, but, on his recommendation he suggested me to post something to the mailing list. Lets start off with one drive at a time (I have 4 that are corrupt). The specific logical drive in question was in RAID1 on a Dell PERC 5/i card. If I try to mount this using: mount -t ext4 /dev/sda1 /media/tmp It complains in dmesg with the following output: 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: comm mount: bad extra_isize (18013 != 256) [685621.845213] EXT4-fs (sda1): no journal found However, if I run dumpe2fs -f /dev/sda1 I get the following output: root@server:~# dumpe2fs -f /dev/sda1 dumpe2fs 1.42.5 (29-Jul-2012) Filesystem volume name: root Last mounted on: /media/ubuntu/root Filesystem UUID: f959e195-[removed] Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: not clean with errors Errors behavior: Continue Filesystem OS type: Linux Inode count: 4849664 Block count: 19398144 Reserved block count: 969907 Free blocks: 17034219 Free inodes: 4592929 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1019 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Sat May 25 14:59:50 2013 Last mount time: Sat Aug 24 11:04:25 2013 Last write time: Tue Sep 24 13:55:36 2013 Mount count: 0 Maximum mount count: -1 Last checked: Sat Aug 24 16:56:09 2013 Check interval: 0 (<none>) Lifetime writes: 107 GB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f Journal backup: inode blocks FS Error count: 8 First error time: Sat Aug 24 13:44:55 2013 First error function: ext4_iget First error line #: 3889 First error inode #: 8 First error block #: 0 Last error time: Tue Sep 24 13:55:36 2013 Last error function: ext4_iget Last error line #: 3888 Last error inode #: 8 Last error block #: 0 dumpe2fs: Corrupt extent header while reading journal super block So I attempted to clone the drive to a 2TB backup drive that is empty, and currently I am having more problems with the cloned drive than I am with the original. sandeen said something about using tune2fs to tell it to remove the has_journal flag, but I might need some assistance with that. I would appreciate any help that you could give me, as I know my chances of recovering data are slim, but I would definitely like to try and recover as much data as I can. Thanks Andrew ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-25 2:25 ` Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS InvTraySts @ 2013-09-25 16:12 ` Jan Kara 2013-09-25 19:24 ` InvTraySts 0 siblings, 1 reply; 8+ messages in thread From: Jan Kara @ 2013-09-25 16:12 UTC (permalink / raw) To: InvTraySts; +Cc: linux-ext4, linux-fsdevel On Tue 24-09-13 22:25:49, InvTraySts wrote: > So long story short, I had a server running that had a processor fail > while powered on, causing the file systems to become corrupt. I > replaced the motherboard, processor and power supply just to be on the > safe side. However, I am at a bit of a loss as to what to do now. I > was working sandeen in the OFTC IRC channel, but, on his > recommendation he suggested me to post something to the mailing list. > > Lets start off with one drive at a time (I have 4 that are corrupt). > The specific logical drive in question was in RAID1 on a Dell PERC 5/i > card. > If I try to mount this using: > mount -t ext4 /dev/sda1 /media/tmp > > It complains in dmesg with the following output: > 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > comm mount: bad extra_isize (18013 != 256) > [685621.845213] EXT4-fs (sda1): no journal found > > > However, if I run dumpe2fs -f /dev/sda1 I get the following output: > root@server:~# dumpe2fs -f /dev/sda1 > dumpe2fs 1.42.5 (29-Jul-2012) > Filesystem volume name: root > Last mounted on: /media/ubuntu/root > Filesystem UUID: f959e195-[removed] > Filesystem magic number: 0xEF53 > Filesystem revision #: 1 (dynamic) > Filesystem features: has_journal ext_attr resize_inode dir_index > filetype extent flex_bg sparse_super large_file huge_file uninit_bg > dir_nlink extra_isize > Filesystem flags: signed_directory_hash > Default mount options: user_xattr acl > Filesystem state: not clean with errors > Errors behavior: Continue > Filesystem OS type: Linux > Inode count: 4849664 > Block count: 19398144 > Reserved block count: 969907 > Free blocks: 17034219 > Free inodes: 4592929 > First block: 0 > Block size: 4096 > Fragment size: 4096 > Reserved GDT blocks: 1019 > Blocks per group: 32768 > Fragments per group: 32768 > Inodes per group: 8192 > Inode blocks per group: 512 > Flex block group size: 16 > Filesystem created: Sat May 25 14:59:50 2013 > Last mount time: Sat Aug 24 11:04:25 2013 > Last write time: Tue Sep 24 13:55:36 2013 > Mount count: 0 > Maximum mount count: -1 > Last checked: Sat Aug 24 16:56:09 2013 > Check interval: 0 (<none>) > Lifetime writes: 107 GB > Reserved blocks uid: 0 (user root) > Reserved blocks gid: 0 (group root) > First inode: 11 > Inode size: 256 > Required extra isize: 28 > Desired extra isize: 28 > Journal inode: 8 > Default directory hash: half_md4 > Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > Journal backup: inode blocks > FS Error count: 8 > First error time: Sat Aug 24 13:44:55 2013 > First error function: ext4_iget > First error line #: 3889 > First error inode #: 8 > First error block #: 0 > Last error time: Tue Sep 24 13:55:36 2013 > Last error function: ext4_iget > Last error line #: 3888 > Last error inode #: 8 > Last error block #: 0 > dumpe2fs: Corrupt extent header while reading journal super block OK, so really journal inode (inode #8) looks toast but superblock looks OK. > So I attempted to clone the drive to a 2TB backup drive that is empty, > and currently I am having more problems with the cloned drive than I > am with the original. > > sandeen said something about using tune2fs to tell it to remove the > has_journal flag, but I might need some assistance with that. Yes, you can do that with: tune2fs -f -O ^has_journal /dev/sda1 Let's see what mount will say after that. Another option is to run debugfs /dev/sda1 Then you can use ls, cd, and other debugfs commands to move within the filesystem and investigate things. If that will work, you have a reasonable chance of getting at least some data back. Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-25 16:12 ` Jan Kara @ 2013-09-25 19:24 ` InvTraySts 2013-09-25 21:28 ` Jan Kara 0 siblings, 1 reply; 8+ messages in thread From: InvTraySts @ 2013-09-25 19:24 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4, linux-fsdevel And am cloning the drive without the sync parameter this time. root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror After it finished, I attempted to run dumpe2fs and it still responds with: root@server:~# dumpe2fs /dev/sdf1 dumpe2fs 1.42.5 (29-Jul-2012) dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 Couldn't find valid filesystem superblock. So I went ahead and tried to run the tune2fs command: root@server:~# tune2fs -f -O ^has_journal /dev/sda1 tune2fs 1.42.5 (29-Jul-2012) tune2fs: Bad magic number in super-block while trying to open /dev/sda1 Couldn't find valid filesystem superblock. Which also fails, yet dumpe2fs on /dev/sda1 works fine. On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> So long story short, I had a server running that had a processor fail >> while powered on, causing the file systems to become corrupt. I >> replaced the motherboard, processor and power supply just to be on the >> safe side. However, I am at a bit of a loss as to what to do now. I >> was working sandeen in the OFTC IRC channel, but, on his >> recommendation he suggested me to post something to the mailing list. >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> card. >> If I try to mount this using: >> mount -t ext4 /dev/sda1 /media/tmp >> >> It complains in dmesg with the following output: >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> comm mount: bad extra_isize (18013 != 256) >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> root@server:~# dumpe2fs -f /dev/sda1 >> dumpe2fs 1.42.5 (29-Jul-2012) >> Filesystem volume name: root >> Last mounted on: /media/ubuntu/root >> Filesystem UUID: f959e195-[removed] >> Filesystem magic number: 0xEF53 >> Filesystem revision #: 1 (dynamic) >> Filesystem features: has_journal ext_attr resize_inode dir_index >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> dir_nlink extra_isize >> Filesystem flags: signed_directory_hash >> Default mount options: user_xattr acl >> Filesystem state: not clean with errors >> Errors behavior: Continue >> Filesystem OS type: Linux >> Inode count: 4849664 >> Block count: 19398144 >> Reserved block count: 969907 >> Free blocks: 17034219 >> Free inodes: 4592929 >> First block: 0 >> Block size: 4096 >> Fragment size: 4096 >> Reserved GDT blocks: 1019 >> Blocks per group: 32768 >> Fragments per group: 32768 >> Inodes per group: 8192 >> Inode blocks per group: 512 >> Flex block group size: 16 >> Filesystem created: Sat May 25 14:59:50 2013 >> Last mount time: Sat Aug 24 11:04:25 2013 >> Last write time: Tue Sep 24 13:55:36 2013 >> Mount count: 0 >> Maximum mount count: -1 >> Last checked: Sat Aug 24 16:56:09 2013 >> Check interval: 0 (<none>) >> Lifetime writes: 107 GB >> Reserved blocks uid: 0 (user root) >> Reserved blocks gid: 0 (group root) >> First inode: 11 >> Inode size: 256 >> Required extra isize: 28 >> Desired extra isize: 28 >> Journal inode: 8 >> Default directory hash: half_md4 >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> Journal backup: inode blocks >> FS Error count: 8 >> First error time: Sat Aug 24 13:44:55 2013 >> First error function: ext4_iget >> First error line #: 3889 >> First error inode #: 8 >> First error block #: 0 >> Last error time: Tue Sep 24 13:55:36 2013 >> Last error function: ext4_iget >> Last error line #: 3888 >> Last error inode #: 8 >> Last error block #: 0 >> dumpe2fs: Corrupt extent header while reading journal super block > OK, so really journal inode (inode #8) looks toast but superblock looks > OK. > >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> and currently I am having more problems with the cloned drive than I >> am with the original. >> >> sandeen said something about using tune2fs to tell it to remove the >> has_journal flag, but I might need some assistance with that. > Yes, you can do that with: > tune2fs -f -O ^has_journal /dev/sda1 > > Let's see what mount will say after that. > > Another option is to run > debugfs /dev/sda1 > > Then you can use ls, cd, and other debugfs commands to move within the > filesystem and investigate things. If that will work, you have a reasonable > chance of getting at least some data back. > > Honza > -- > Jan Kara <jack@suse.cz> > SUSE Labs, CR On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> So long story short, I had a server running that had a processor fail >> while powered on, causing the file systems to become corrupt. I >> replaced the motherboard, processor and power supply just to be on the >> safe side. However, I am at a bit of a loss as to what to do now. I >> was working sandeen in the OFTC IRC channel, but, on his >> recommendation he suggested me to post something to the mailing list. >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> card. >> If I try to mount this using: >> mount -t ext4 /dev/sda1 /media/tmp >> >> It complains in dmesg with the following output: >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> comm mount: bad extra_isize (18013 != 256) >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> root@server:~# dumpe2fs -f /dev/sda1 >> dumpe2fs 1.42.5 (29-Jul-2012) >> Filesystem volume name: root >> Last mounted on: /media/ubuntu/root >> Filesystem UUID: f959e195-[removed] >> Filesystem magic number: 0xEF53 >> Filesystem revision #: 1 (dynamic) >> Filesystem features: has_journal ext_attr resize_inode dir_index >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> dir_nlink extra_isize >> Filesystem flags: signed_directory_hash >> Default mount options: user_xattr acl >> Filesystem state: not clean with errors >> Errors behavior: Continue >> Filesystem OS type: Linux >> Inode count: 4849664 >> Block count: 19398144 >> Reserved block count: 969907 >> Free blocks: 17034219 >> Free inodes: 4592929 >> First block: 0 >> Block size: 4096 >> Fragment size: 4096 >> Reserved GDT blocks: 1019 >> Blocks per group: 32768 >> Fragments per group: 32768 >> Inodes per group: 8192 >> Inode blocks per group: 512 >> Flex block group size: 16 >> Filesystem created: Sat May 25 14:59:50 2013 >> Last mount time: Sat Aug 24 11:04:25 2013 >> Last write time: Tue Sep 24 13:55:36 2013 >> Mount count: 0 >> Maximum mount count: -1 >> Last checked: Sat Aug 24 16:56:09 2013 >> Check interval: 0 (<none>) >> Lifetime writes: 107 GB >> Reserved blocks uid: 0 (user root) >> Reserved blocks gid: 0 (group root) >> First inode: 11 >> Inode size: 256 >> Required extra isize: 28 >> Desired extra isize: 28 >> Journal inode: 8 >> Default directory hash: half_md4 >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> Journal backup: inode blocks >> FS Error count: 8 >> First error time: Sat Aug 24 13:44:55 2013 >> First error function: ext4_iget >> First error line #: 3889 >> First error inode #: 8 >> First error block #: 0 >> Last error time: Tue Sep 24 13:55:36 2013 >> Last error function: ext4_iget >> Last error line #: 3888 >> Last error inode #: 8 >> Last error block #: 0 >> dumpe2fs: Corrupt extent header while reading journal super block > OK, so really journal inode (inode #8) looks toast but superblock looks > OK. > >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> and currently I am having more problems with the cloned drive than I >> am with the original. >> >> sandeen said something about using tune2fs to tell it to remove the >> has_journal flag, but I might need some assistance with that. > Yes, you can do that with: > tune2fs -f -O ^has_journal /dev/sda1 > > Let's see what mount will say after that. > > Another option is to run > debugfs /dev/sda1 > > Then you can use ls, cd, and other debugfs commands to move within the > filesystem and investigate things. If that will work, you have a reasonable > chance of getting at least some data back. > > Honza > -- > Jan Kara <jack@suse.cz> > SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-25 19:24 ` InvTraySts @ 2013-09-25 21:28 ` Jan Kara 2013-09-26 0:08 ` InvTraySts 0 siblings, 1 reply; 8+ messages in thread From: Jan Kara @ 2013-09-25 21:28 UTC (permalink / raw) To: InvTraySts; +Cc: Jan Kara, linux-ext4, linux-fsdevel On Wed 25-09-13 15:24:34, InvTraySts wrote: > And am cloning the drive without the sync parameter this time. > root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror > After it finished, I attempted to run dumpe2fs and it still responds with: > root@server:~# dumpe2fs /dev/sdf1 > dumpe2fs 1.42.5 (29-Jul-2012) > dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 > Couldn't find valid filesystem superblock. Well, that's likely because the partition table on /dev/sdf didn't get reread. You can run 'partprobe /dev/sdf' to tell the kernel about the new partition table. Honza > So I went ahead and tried to run the tune2fs command: > root@server:~# tune2fs -f -O ^has_journal /dev/sda1 > tune2fs 1.42.5 (29-Jul-2012) > tune2fs: Bad magic number in super-block while trying to open /dev/sda1 > Couldn't find valid filesystem superblock. > > Which also fails, yet dumpe2fs on /dev/sda1 works fine. > > > On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> So long story short, I had a server running that had a processor fail > >> while powered on, causing the file systems to become corrupt. I > >> replaced the motherboard, processor and power supply just to be on the > >> safe side. However, I am at a bit of a loss as to what to do now. I > >> was working sandeen in the OFTC IRC channel, but, on his > >> recommendation he suggested me to post something to the mailing list. > >> > >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> card. > >> If I try to mount this using: > >> mount -t ext4 /dev/sda1 /media/tmp > >> > >> It complains in dmesg with the following output: > >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> comm mount: bad extra_isize (18013 != 256) > >> [685621.845213] EXT4-fs (sda1): no journal found > >> > >> > >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> root@server:~# dumpe2fs -f /dev/sda1 > >> dumpe2fs 1.42.5 (29-Jul-2012) > >> Filesystem volume name: root > >> Last mounted on: /media/ubuntu/root > >> Filesystem UUID: f959e195-[removed] > >> Filesystem magic number: 0xEF53 > >> Filesystem revision #: 1 (dynamic) > >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> dir_nlink extra_isize > >> Filesystem flags: signed_directory_hash > >> Default mount options: user_xattr acl > >> Filesystem state: not clean with errors > >> Errors behavior: Continue > >> Filesystem OS type: Linux > >> Inode count: 4849664 > >> Block count: 19398144 > >> Reserved block count: 969907 > >> Free blocks: 17034219 > >> Free inodes: 4592929 > >> First block: 0 > >> Block size: 4096 > >> Fragment size: 4096 > >> Reserved GDT blocks: 1019 > >> Blocks per group: 32768 > >> Fragments per group: 32768 > >> Inodes per group: 8192 > >> Inode blocks per group: 512 > >> Flex block group size: 16 > >> Filesystem created: Sat May 25 14:59:50 2013 > >> Last mount time: Sat Aug 24 11:04:25 2013 > >> Last write time: Tue Sep 24 13:55:36 2013 > >> Mount count: 0 > >> Maximum mount count: -1 > >> Last checked: Sat Aug 24 16:56:09 2013 > >> Check interval: 0 (<none>) > >> Lifetime writes: 107 GB > >> Reserved blocks uid: 0 (user root) > >> Reserved blocks gid: 0 (group root) > >> First inode: 11 > >> Inode size: 256 > >> Required extra isize: 28 > >> Desired extra isize: 28 > >> Journal inode: 8 > >> Default directory hash: half_md4 > >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> Journal backup: inode blocks > >> FS Error count: 8 > >> First error time: Sat Aug 24 13:44:55 2013 > >> First error function: ext4_iget > >> First error line #: 3889 > >> First error inode #: 8 > >> First error block #: 0 > >> Last error time: Tue Sep 24 13:55:36 2013 > >> Last error function: ext4_iget > >> Last error line #: 3888 > >> Last error inode #: 8 > >> Last error block #: 0 > >> dumpe2fs: Corrupt extent header while reading journal super block > > OK, so really journal inode (inode #8) looks toast but superblock looks > > OK. > > > >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> and currently I am having more problems with the cloned drive than I > >> am with the original. > >> > >> sandeen said something about using tune2fs to tell it to remove the > >> has_journal flag, but I might need some assistance with that. > > Yes, you can do that with: > > tune2fs -f -O ^has_journal /dev/sda1 > > > > Let's see what mount will say after that. > > > > Another option is to run > > debugfs /dev/sda1 > > > > Then you can use ls, cd, and other debugfs commands to move within the > > filesystem and investigate things. If that will work, you have a reasonable > > chance of getting at least some data back. > > > > Honza > > -- > > Jan Kara <jack@suse.cz> > > SUSE Labs, CR > > > On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> So long story short, I had a server running that had a processor fail > >> while powered on, causing the file systems to become corrupt. I > >> replaced the motherboard, processor and power supply just to be on the > >> safe side. However, I am at a bit of a loss as to what to do now. I > >> was working sandeen in the OFTC IRC channel, but, on his > >> recommendation he suggested me to post something to the mailing list. > >> > >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> card. > >> If I try to mount this using: > >> mount -t ext4 /dev/sda1 /media/tmp > >> > >> It complains in dmesg with the following output: > >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> comm mount: bad extra_isize (18013 != 256) > >> [685621.845213] EXT4-fs (sda1): no journal found > >> > >> > >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> root@server:~# dumpe2fs -f /dev/sda1 > >> dumpe2fs 1.42.5 (29-Jul-2012) > >> Filesystem volume name: root > >> Last mounted on: /media/ubuntu/root > >> Filesystem UUID: f959e195-[removed] > >> Filesystem magic number: 0xEF53 > >> Filesystem revision #: 1 (dynamic) > >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> dir_nlink extra_isize > >> Filesystem flags: signed_directory_hash > >> Default mount options: user_xattr acl > >> Filesystem state: not clean with errors > >> Errors behavior: Continue > >> Filesystem OS type: Linux > >> Inode count: 4849664 > >> Block count: 19398144 > >> Reserved block count: 969907 > >> Free blocks: 17034219 > >> Free inodes: 4592929 > >> First block: 0 > >> Block size: 4096 > >> Fragment size: 4096 > >> Reserved GDT blocks: 1019 > >> Blocks per group: 32768 > >> Fragments per group: 32768 > >> Inodes per group: 8192 > >> Inode blocks per group: 512 > >> Flex block group size: 16 > >> Filesystem created: Sat May 25 14:59:50 2013 > >> Last mount time: Sat Aug 24 11:04:25 2013 > >> Last write time: Tue Sep 24 13:55:36 2013 > >> Mount count: 0 > >> Maximum mount count: -1 > >> Last checked: Sat Aug 24 16:56:09 2013 > >> Check interval: 0 (<none>) > >> Lifetime writes: 107 GB > >> Reserved blocks uid: 0 (user root) > >> Reserved blocks gid: 0 (group root) > >> First inode: 11 > >> Inode size: 256 > >> Required extra isize: 28 > >> Desired extra isize: 28 > >> Journal inode: 8 > >> Default directory hash: half_md4 > >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> Journal backup: inode blocks > >> FS Error count: 8 > >> First error time: Sat Aug 24 13:44:55 2013 > >> First error function: ext4_iget > >> First error line #: 3889 > >> First error inode #: 8 > >> First error block #: 0 > >> Last error time: Tue Sep 24 13:55:36 2013 > >> Last error function: ext4_iget > >> Last error line #: 3888 > >> Last error inode #: 8 > >> Last error block #: 0 > >> dumpe2fs: Corrupt extent header while reading journal super block > > OK, so really journal inode (inode #8) looks toast but superblock looks > > OK. > > > >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> and currently I am having more problems with the cloned drive than I > >> am with the original. > >> > >> sandeen said something about using tune2fs to tell it to remove the > >> has_journal flag, but I might need some assistance with that. > > Yes, you can do that with: > > tune2fs -f -O ^has_journal /dev/sda1 > > > > Let's see what mount will say after that. > > > > Another option is to run > > debugfs /dev/sda1 > > > > Then you can use ls, cd, and other debugfs commands to move within the > > filesystem and investigate things. If that will work, you have a reasonable > > chance of getting at least some data back. > > > > Honza > > -- > > Jan Kara <jack@suse.cz> > > SUSE Labs, CR -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-25 21:28 ` Jan Kara @ 2013-09-26 0:08 ` InvTraySts 2013-09-26 8:53 ` Jan Kara 0 siblings, 1 reply; 8+ messages in thread From: InvTraySts @ 2013-09-26 0:08 UTC (permalink / raw) To: Jan Kara, Eric Sandeen; +Cc: linux-ext4, linux-fsdevel (Going to merge these two back, because both of you are actually helping me, but with the two conversations being segregated like this it makes it hard to correlate with you both) The partprobe did work with getting the partition table reread. After that, the tune2fs sorta worked. root@server:~# tune2fs -f -O ^has_journal /dev/sdf1 tune2fs 1.42.5 (29-Jul-2012) root@server:~# mount /dev/sdf1 /media/tmp root@server:~# ls -l /media/tmp/ total 0 When I try and use the debugfs /dev/sdf1 root@server:~# debugfs /dev/sdf1 debugfs 1.42.5 (29-Jul-2012) debugfs: ls EXT2 directory corrupted debugfs: ls / /: EXT2 directory corrupted The drive is supposed to be ext4. root@server:~# dumpe2fs /dev/sdf1 dumpe2fs 1.42.5 (29-Jul-2012) Filesystem volume name: root Last mounted on: /media/ubuntu/root Filesystem UUID: removed Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: not clean with errors Errors behavior: Continue Filesystem OS type: Linux Inode count: 4849664 Block count: 19398144 Reserved block count: 969907 Free blocks: 17066988 Free inodes: 4592929 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1019 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Sat May 25 14:59:50 2013 Last mount time: Wed Sep 25 19:59:07 2013 Last write time: Wed Sep 25 19:59:51 2013 Mount count: 1 Maximum mount count: -1 Last checked: Sat Aug 24 16:56:09 2013 Check interval: 0 (<none>) Lifetime writes: 107 GB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Default directory hash: half_md4 Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f Journal backup: inode blocks FS Error count: 9 First error time: Sat Aug 24 13:44:55 2013 First error function: ext4_iget First error line #: 3889 First error inode #: 8 First error block #: 0 Last error time: Wed Sep 25 19:59:12 2013 Last error function: htree_dirblock_to_tree Last error line #: 892 Last error inode #: 2 Last error block #: 20037 [...] [output scrolls very fast, and is very large] Group 10: (Blocks 327680-360447) [ITABLE_ZEROED] Checksum 0xecaa, unused inodes 0 Block bitmap at 1035 (bg #0 + 1035), Inode bitmap at 1051 (bg #0 + 1051) Inode table at 6177-6688 (bg #0 + 6177) 0 free blocks, 16 free inodes, 0 directories Free blocks: 327680, 327683-327685, 327687, 327689-327690, 327692-327693, 327695-327697, 327700-327701, 327703, 327705, 327707-327709, 327711-327715, 327718-327727, 327730-327808, 327810-327823, 327825-327842, 327846-327855, 327857-327875, 327878, 327881 (the total amount of pages that the dumpe2fs comes out with is about 240 pages. something that can't be pasted. If attachments would be required, I can attach a file with it). On Wed, Sep 25, 2013 at 5:28 PM, Jan Kara <jack@suse.cz> wrote: > On Wed 25-09-13 15:24:34, InvTraySts wrote: >> And am cloning the drive without the sync parameter this time. >> root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror >> After it finished, I attempted to run dumpe2fs and it still responds with: >> root@server:~# dumpe2fs /dev/sdf1 >> dumpe2fs 1.42.5 (29-Jul-2012) >> dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 >> Couldn't find valid filesystem superblock. > Well, that's likely because the partition table on /dev/sdf didn't get > reread. You can run 'partprobe /dev/sdf' to tell the kernel about the new > partition table. > > Honza > >> So I went ahead and tried to run the tune2fs command: >> root@server:~# tune2fs -f -O ^has_journal /dev/sda1 >> tune2fs 1.42.5 (29-Jul-2012) >> tune2fs: Bad magic number in super-block while trying to open /dev/sda1 >> Couldn't find valid filesystem superblock. >> >> Which also fails, yet dumpe2fs on /dev/sda1 works fine. >> >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> >> So long story short, I had a server running that had a processor fail >> >> while powered on, causing the file systems to become corrupt. I >> >> replaced the motherboard, processor and power supply just to be on the >> >> safe side. However, I am at a bit of a loss as to what to do now. I >> >> was working sandeen in the OFTC IRC channel, but, on his >> >> recommendation he suggested me to post something to the mailing list. >> >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> >> card. >> >> If I try to mount this using: >> >> mount -t ext4 /dev/sda1 /media/tmp >> >> >> >> It complains in dmesg with the following output: >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> >> comm mount: bad extra_isize (18013 != 256) >> >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> >> root@server:~# dumpe2fs -f /dev/sda1 >> >> dumpe2fs 1.42.5 (29-Jul-2012) >> >> Filesystem volume name: root >> >> Last mounted on: /media/ubuntu/root >> >> Filesystem UUID: f959e195-[removed] >> >> Filesystem magic number: 0xEF53 >> >> Filesystem revision #: 1 (dynamic) >> >> Filesystem features: has_journal ext_attr resize_inode dir_index >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> >> dir_nlink extra_isize >> >> Filesystem flags: signed_directory_hash >> >> Default mount options: user_xattr acl >> >> Filesystem state: not clean with errors >> >> Errors behavior: Continue >> >> Filesystem OS type: Linux >> >> Inode count: 4849664 >> >> Block count: 19398144 >> >> Reserved block count: 969907 >> >> Free blocks: 17034219 >> >> Free inodes: 4592929 >> >> First block: 0 >> >> Block size: 4096 >> >> Fragment size: 4096 >> >> Reserved GDT blocks: 1019 >> >> Blocks per group: 32768 >> >> Fragments per group: 32768 >> >> Inodes per group: 8192 >> >> Inode blocks per group: 512 >> >> Flex block group size: 16 >> >> Filesystem created: Sat May 25 14:59:50 2013 >> >> Last mount time: Sat Aug 24 11:04:25 2013 >> >> Last write time: Tue Sep 24 13:55:36 2013 >> >> Mount count: 0 >> >> Maximum mount count: -1 >> >> Last checked: Sat Aug 24 16:56:09 2013 >> >> Check interval: 0 (<none>) >> >> Lifetime writes: 107 GB >> >> Reserved blocks uid: 0 (user root) >> >> Reserved blocks gid: 0 (group root) >> >> First inode: 11 >> >> Inode size: 256 >> >> Required extra isize: 28 >> >> Desired extra isize: 28 >> >> Journal inode: 8 >> >> Default directory hash: half_md4 >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> >> Journal backup: inode blocks >> >> FS Error count: 8 >> >> First error time: Sat Aug 24 13:44:55 2013 >> >> First error function: ext4_iget >> >> First error line #: 3889 >> >> First error inode #: 8 >> >> First error block #: 0 >> >> Last error time: Tue Sep 24 13:55:36 2013 >> >> Last error function: ext4_iget >> >> Last error line #: 3888 >> >> Last error inode #: 8 >> >> Last error block #: 0 >> >> dumpe2fs: Corrupt extent header while reading journal super block >> > OK, so really journal inode (inode #8) looks toast but superblock looks >> > OK. >> > >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> >> and currently I am having more problems with the cloned drive than I >> >> am with the original. >> >> >> >> sandeen said something about using tune2fs to tell it to remove the >> >> has_journal flag, but I might need some assistance with that. >> > Yes, you can do that with: >> > tune2fs -f -O ^has_journal /dev/sda1 >> > >> > Let's see what mount will say after that. >> > >> > Another option is to run >> > debugfs /dev/sda1 >> > >> > Then you can use ls, cd, and other debugfs commands to move within the >> > filesystem and investigate things. If that will work, you have a reasonable >> > chance of getting at least some data back. >> > >> > Honza >> > -- >> > Jan Kara <jack@suse.cz> >> > SUSE Labs, CR >> >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> >> So long story short, I had a server running that had a processor fail >> >> while powered on, causing the file systems to become corrupt. I >> >> replaced the motherboard, processor and power supply just to be on the >> >> safe side. However, I am at a bit of a loss as to what to do now. I >> >> was working sandeen in the OFTC IRC channel, but, on his >> >> recommendation he suggested me to post something to the mailing list. >> >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> >> card. >> >> If I try to mount this using: >> >> mount -t ext4 /dev/sda1 /media/tmp >> >> >> >> It complains in dmesg with the following output: >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> >> comm mount: bad extra_isize (18013 != 256) >> >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> >> root@server:~# dumpe2fs -f /dev/sda1 >> >> dumpe2fs 1.42.5 (29-Jul-2012) >> >> Filesystem volume name: root >> >> Last mounted on: /media/ubuntu/root >> >> Filesystem UUID: f959e195-[removed] >> >> Filesystem magic number: 0xEF53 >> >> Filesystem revision #: 1 (dynamic) >> >> Filesystem features: has_journal ext_attr resize_inode dir_index >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> >> dir_nlink extra_isize >> >> Filesystem flags: signed_directory_hash >> >> Default mount options: user_xattr acl >> >> Filesystem state: not clean with errors >> >> Errors behavior: Continue >> >> Filesystem OS type: Linux >> >> Inode count: 4849664 >> >> Block count: 19398144 >> >> Reserved block count: 969907 >> >> Free blocks: 17034219 >> >> Free inodes: 4592929 >> >> First block: 0 >> >> Block size: 4096 >> >> Fragment size: 4096 >> >> Reserved GDT blocks: 1019 >> >> Blocks per group: 32768 >> >> Fragments per group: 32768 >> >> Inodes per group: 8192 >> >> Inode blocks per group: 512 >> >> Flex block group size: 16 >> >> Filesystem created: Sat May 25 14:59:50 2013 >> >> Last mount time: Sat Aug 24 11:04:25 2013 >> >> Last write time: Tue Sep 24 13:55:36 2013 >> >> Mount count: 0 >> >> Maximum mount count: -1 >> >> Last checked: Sat Aug 24 16:56:09 2013 >> >> Check interval: 0 (<none>) >> >> Lifetime writes: 107 GB >> >> Reserved blocks uid: 0 (user root) >> >> Reserved blocks gid: 0 (group root) >> >> First inode: 11 >> >> Inode size: 256 >> >> Required extra isize: 28 >> >> Desired extra isize: 28 >> >> Journal inode: 8 >> >> Default directory hash: half_md4 >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> >> Journal backup: inode blocks >> >> FS Error count: 8 >> >> First error time: Sat Aug 24 13:44:55 2013 >> >> First error function: ext4_iget >> >> First error line #: 3889 >> >> First error inode #: 8 >> >> First error block #: 0 >> >> Last error time: Tue Sep 24 13:55:36 2013 >> >> Last error function: ext4_iget >> >> Last error line #: 3888 >> >> Last error inode #: 8 >> >> Last error block #: 0 >> >> dumpe2fs: Corrupt extent header while reading journal super block >> > OK, so really journal inode (inode #8) looks toast but superblock looks >> > OK. >> > >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> >> and currently I am having more problems with the cloned drive than I >> >> am with the original. >> >> >> >> sandeen said something about using tune2fs to tell it to remove the >> >> has_journal flag, but I might need some assistance with that. >> > Yes, you can do that with: >> > tune2fs -f -O ^has_journal /dev/sda1 >> > >> > Let's see what mount will say after that. >> > >> > Another option is to run >> > debugfs /dev/sda1 >> > >> > Then you can use ls, cd, and other debugfs commands to move within the >> > filesystem and investigate things. If that will work, you have a reasonable >> > chance of getting at least some data back. >> > >> > Honza >> > -- >> > Jan Kara <jack@suse.cz> >> > SUSE Labs, CR > -- > Jan Kara <jack@suse.cz> > SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-26 0:08 ` InvTraySts @ 2013-09-26 8:53 ` Jan Kara 2013-09-26 17:45 ` InvTraySts 0 siblings, 1 reply; 8+ messages in thread From: Jan Kara @ 2013-09-26 8:53 UTC (permalink / raw) To: InvTraySts; +Cc: Jan Kara, Eric Sandeen, linux-ext4, linux-fsdevel On Wed 25-09-13 20:08:38, InvTraySts wrote: > (Going to merge these two back, because both of you are actually > helping me, but with the two conversations being segregated like this > it makes it hard to correlate with you both) > The partprobe did work with getting the partition table reread. > After that, the tune2fs sorta worked. > root@server:~# tune2fs -f -O ^has_journal /dev/sdf1 > tune2fs 1.42.5 (29-Jul-2012) > root@server:~# mount /dev/sdf1 /media/tmp > root@server:~# ls -l /media/tmp/ > total 0 > > When I try and use the debugfs /dev/sdf1 > > root@server:~# debugfs /dev/sdf1 > debugfs 1.42.5 (29-Jul-2012) > debugfs: ls > EXT2 directory corrupted > debugfs: ls / > /: EXT2 directory corrupted > > The drive is supposed to be ext4. 'EXT2' in the message doesn't mean anything. It is always there. But we see that even root directory is corrupted. That's bad. You can try what 'stat /' says in debugfs? Possibly also run 'dump / <somefile>' to dump raw directory data to <somefile> and attach the file here. But also given the output of e2fsck I'm afraid there won't be much to save on the drive... Honza > root@server:~# dumpe2fs /dev/sdf1 > dumpe2fs 1.42.5 (29-Jul-2012) > Filesystem volume name: root > Last mounted on: /media/ubuntu/root > Filesystem UUID: removed > Filesystem magic number: 0xEF53 > Filesystem revision #: 1 (dynamic) > Filesystem features: ext_attr resize_inode dir_index filetype > extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink > extra_isize > Filesystem flags: signed_directory_hash > Default mount options: user_xattr acl > Filesystem state: not clean with errors > Errors behavior: Continue > Filesystem OS type: Linux > Inode count: 4849664 > Block count: 19398144 > Reserved block count: 969907 > Free blocks: 17066988 > Free inodes: 4592929 > First block: 0 > Block size: 4096 > Fragment size: 4096 > Reserved GDT blocks: 1019 > Blocks per group: 32768 > Fragments per group: 32768 > Inodes per group: 8192 > Inode blocks per group: 512 > Flex block group size: 16 > Filesystem created: Sat May 25 14:59:50 2013 > Last mount time: Wed Sep 25 19:59:07 2013 > Last write time: Wed Sep 25 19:59:51 2013 > Mount count: 1 > Maximum mount count: -1 > Last checked: Sat Aug 24 16:56:09 2013 > Check interval: 0 (<none>) > Lifetime writes: 107 GB > Reserved blocks uid: 0 (user root) > Reserved blocks gid: 0 (group root) > First inode: 11 > Inode size: 256 > Required extra isize: 28 > Desired extra isize: 28 > Default directory hash: half_md4 > Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > Journal backup: inode blocks > FS Error count: 9 > First error time: Sat Aug 24 13:44:55 2013 > First error function: ext4_iget > First error line #: 3889 > First error inode #: 8 > First error block #: 0 > Last error time: Wed Sep 25 19:59:12 2013 > Last error function: htree_dirblock_to_tree > Last error line #: 892 > Last error inode #: 2 > Last error block #: 20037 > > [...] > [output scrolls very fast, and is very large] > Group 10: (Blocks 327680-360447) [ITABLE_ZEROED] > Checksum 0xecaa, unused inodes 0 > Block bitmap at 1035 (bg #0 + 1035), Inode bitmap at 1051 (bg #0 + 1051) > Inode table at 6177-6688 (bg #0 + 6177) > 0 free blocks, 16 free inodes, 0 directories > Free blocks: 327680, 327683-327685, 327687, 327689-327690, > 327692-327693, 327695-327697, 327700-327701, 327703, 327705, > 327707-327709, 327711-327715, 327718-327727, 327730-327808, > 327810-327823, 327825-327842, 327846-327855, 327857-327875, 327878, > 327881 > > (the total amount of pages that the dumpe2fs comes out with is about > 240 pages. something that can't be pasted. If attachments would be > required, I can attach a file with it). > > On Wed, Sep 25, 2013 at 5:28 PM, Jan Kara <jack@suse.cz> wrote: > > On Wed 25-09-13 15:24:34, InvTraySts wrote: > >> And am cloning the drive without the sync parameter this time. > >> root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror > >> After it finished, I attempted to run dumpe2fs and it still responds with: > >> root@server:~# dumpe2fs /dev/sdf1 > >> dumpe2fs 1.42.5 (29-Jul-2012) > >> dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 > >> Couldn't find valid filesystem superblock. > > Well, that's likely because the partition table on /dev/sdf didn't get > > reread. You can run 'partprobe /dev/sdf' to tell the kernel about the new > > partition table. > > > > Honza > > > >> So I went ahead and tried to run the tune2fs command: > >> root@server:~# tune2fs -f -O ^has_journal /dev/sda1 > >> tune2fs 1.42.5 (29-Jul-2012) > >> tune2fs: Bad magic number in super-block while trying to open /dev/sda1 > >> Couldn't find valid filesystem superblock. > >> > >> Which also fails, yet dumpe2fs on /dev/sda1 works fine. > >> > >> > >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> >> So long story short, I had a server running that had a processor fail > >> >> while powered on, causing the file systems to become corrupt. I > >> >> replaced the motherboard, processor and power supply just to be on the > >> >> safe side. However, I am at a bit of a loss as to what to do now. I > >> >> was working sandeen in the OFTC IRC channel, but, on his > >> >> recommendation he suggested me to post something to the mailing list. > >> >> > >> >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> >> card. > >> >> If I try to mount this using: > >> >> mount -t ext4 /dev/sda1 /media/tmp > >> >> > >> >> It complains in dmesg with the following output: > >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> >> comm mount: bad extra_isize (18013 != 256) > >> >> [685621.845213] EXT4-fs (sda1): no journal found > >> >> > >> >> > >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> >> root@server:~# dumpe2fs -f /dev/sda1 > >> >> dumpe2fs 1.42.5 (29-Jul-2012) > >> >> Filesystem volume name: root > >> >> Last mounted on: /media/ubuntu/root > >> >> Filesystem UUID: f959e195-[removed] > >> >> Filesystem magic number: 0xEF53 > >> >> Filesystem revision #: 1 (dynamic) > >> >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> >> dir_nlink extra_isize > >> >> Filesystem flags: signed_directory_hash > >> >> Default mount options: user_xattr acl > >> >> Filesystem state: not clean with errors > >> >> Errors behavior: Continue > >> >> Filesystem OS type: Linux > >> >> Inode count: 4849664 > >> >> Block count: 19398144 > >> >> Reserved block count: 969907 > >> >> Free blocks: 17034219 > >> >> Free inodes: 4592929 > >> >> First block: 0 > >> >> Block size: 4096 > >> >> Fragment size: 4096 > >> >> Reserved GDT blocks: 1019 > >> >> Blocks per group: 32768 > >> >> Fragments per group: 32768 > >> >> Inodes per group: 8192 > >> >> Inode blocks per group: 512 > >> >> Flex block group size: 16 > >> >> Filesystem created: Sat May 25 14:59:50 2013 > >> >> Last mount time: Sat Aug 24 11:04:25 2013 > >> >> Last write time: Tue Sep 24 13:55:36 2013 > >> >> Mount count: 0 > >> >> Maximum mount count: -1 > >> >> Last checked: Sat Aug 24 16:56:09 2013 > >> >> Check interval: 0 (<none>) > >> >> Lifetime writes: 107 GB > >> >> Reserved blocks uid: 0 (user root) > >> >> Reserved blocks gid: 0 (group root) > >> >> First inode: 11 > >> >> Inode size: 256 > >> >> Required extra isize: 28 > >> >> Desired extra isize: 28 > >> >> Journal inode: 8 > >> >> Default directory hash: half_md4 > >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> >> Journal backup: inode blocks > >> >> FS Error count: 8 > >> >> First error time: Sat Aug 24 13:44:55 2013 > >> >> First error function: ext4_iget > >> >> First error line #: 3889 > >> >> First error inode #: 8 > >> >> First error block #: 0 > >> >> Last error time: Tue Sep 24 13:55:36 2013 > >> >> Last error function: ext4_iget > >> >> Last error line #: 3888 > >> >> Last error inode #: 8 > >> >> Last error block #: 0 > >> >> dumpe2fs: Corrupt extent header while reading journal super block > >> > OK, so really journal inode (inode #8) looks toast but superblock looks > >> > OK. > >> > > >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> >> and currently I am having more problems with the cloned drive than I > >> >> am with the original. > >> >> > >> >> sandeen said something about using tune2fs to tell it to remove the > >> >> has_journal flag, but I might need some assistance with that. > >> > Yes, you can do that with: > >> > tune2fs -f -O ^has_journal /dev/sda1 > >> > > >> > Let's see what mount will say after that. > >> > > >> > Another option is to run > >> > debugfs /dev/sda1 > >> > > >> > Then you can use ls, cd, and other debugfs commands to move within the > >> > filesystem and investigate things. If that will work, you have a reasonable > >> > chance of getting at least some data back. > >> > > >> > Honza > >> > -- > >> > Jan Kara <jack@suse.cz> > >> > SUSE Labs, CR > >> > >> > >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> >> So long story short, I had a server running that had a processor fail > >> >> while powered on, causing the file systems to become corrupt. I > >> >> replaced the motherboard, processor and power supply just to be on the > >> >> safe side. However, I am at a bit of a loss as to what to do now. I > >> >> was working sandeen in the OFTC IRC channel, but, on his > >> >> recommendation he suggested me to post something to the mailing list. > >> >> > >> >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> >> card. > >> >> If I try to mount this using: > >> >> mount -t ext4 /dev/sda1 /media/tmp > >> >> > >> >> It complains in dmesg with the following output: > >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> >> comm mount: bad extra_isize (18013 != 256) > >> >> [685621.845213] EXT4-fs (sda1): no journal found > >> >> > >> >> > >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> >> root@server:~# dumpe2fs -f /dev/sda1 > >> >> dumpe2fs 1.42.5 (29-Jul-2012) > >> >> Filesystem volume name: root > >> >> Last mounted on: /media/ubuntu/root > >> >> Filesystem UUID: f959e195-[removed] > >> >> Filesystem magic number: 0xEF53 > >> >> Filesystem revision #: 1 (dynamic) > >> >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> >> dir_nlink extra_isize > >> >> Filesystem flags: signed_directory_hash > >> >> Default mount options: user_xattr acl > >> >> Filesystem state: not clean with errors > >> >> Errors behavior: Continue > >> >> Filesystem OS type: Linux > >> >> Inode count: 4849664 > >> >> Block count: 19398144 > >> >> Reserved block count: 969907 > >> >> Free blocks: 17034219 > >> >> Free inodes: 4592929 > >> >> First block: 0 > >> >> Block size: 4096 > >> >> Fragment size: 4096 > >> >> Reserved GDT blocks: 1019 > >> >> Blocks per group: 32768 > >> >> Fragments per group: 32768 > >> >> Inodes per group: 8192 > >> >> Inode blocks per group: 512 > >> >> Flex block group size: 16 > >> >> Filesystem created: Sat May 25 14:59:50 2013 > >> >> Last mount time: Sat Aug 24 11:04:25 2013 > >> >> Last write time: Tue Sep 24 13:55:36 2013 > >> >> Mount count: 0 > >> >> Maximum mount count: -1 > >> >> Last checked: Sat Aug 24 16:56:09 2013 > >> >> Check interval: 0 (<none>) > >> >> Lifetime writes: 107 GB > >> >> Reserved blocks uid: 0 (user root) > >> >> Reserved blocks gid: 0 (group root) > >> >> First inode: 11 > >> >> Inode size: 256 > >> >> Required extra isize: 28 > >> >> Desired extra isize: 28 > >> >> Journal inode: 8 > >> >> Default directory hash: half_md4 > >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> >> Journal backup: inode blocks > >> >> FS Error count: 8 > >> >> First error time: Sat Aug 24 13:44:55 2013 > >> >> First error function: ext4_iget > >> >> First error line #: 3889 > >> >> First error inode #: 8 > >> >> First error block #: 0 > >> >> Last error time: Tue Sep 24 13:55:36 2013 > >> >> Last error function: ext4_iget > >> >> Last error line #: 3888 > >> >> Last error inode #: 8 > >> >> Last error block #: 0 > >> >> dumpe2fs: Corrupt extent header while reading journal super block > >> > OK, so really journal inode (inode #8) looks toast but superblock looks > >> > OK. > >> > > >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> >> and currently I am having more problems with the cloned drive than I > >> >> am with the original. > >> >> > >> >> sandeen said something about using tune2fs to tell it to remove the > >> >> has_journal flag, but I might need some assistance with that. > >> > Yes, you can do that with: > >> > tune2fs -f -O ^has_journal /dev/sda1 > >> > > >> > Let's see what mount will say after that. > >> > > >> > Another option is to run > >> > debugfs /dev/sda1 > >> > > >> > Then you can use ls, cd, and other debugfs commands to move within the > >> > filesystem and investigate things. If that will work, you have a reasonable > >> > chance of getting at least some data back. > >> > > >> > Honza > >> > -- > >> > Jan Kara <jack@suse.cz> > >> > SUSE Labs, CR > > -- > > Jan Kara <jack@suse.cz> > > SUSE Labs, CR -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-26 8:53 ` Jan Kara @ 2013-09-26 17:45 ` InvTraySts 2013-09-26 20:18 ` Jan Kara 0 siblings, 1 reply; 8+ messages in thread From: InvTraySts @ 2013-09-26 17:45 UTC (permalink / raw) To: Jan Kara; +Cc: Eric Sandeen, linux-ext4, linux-fsdevel root@server:~# debugfs /dev/sdf1 debugfs 1.42.5 (29-Jul-2012) debugfs: stat / stat: A block group is missing an inode table while reading inode 2 debugfs: dump /etc/ dump: Usage: dump_inode [-p] <file> <output_file> debugfs: dump /etc/passwd /root/passwd.recovery /etc/passwd: A block group is missing an inode table debugfs: ls /etc/ /etc/: A block group is missing an inode table So now I am getting a different error message, even though literally nothing has changed other than the time I ran the command. I figured the passwd file would be the easiest thing to recover, since I don't remember the exact paths for other application configurations without doing some digging for the paths and such. At one point, before the mailing list and me joining the #ext channel, I was able to see that there was a lost and found folder, I just couldn't navigate it. So, I am not sure why that has changed either. I assume logical corruption to the extent it has could cause almost anything. On Thu, Sep 26, 2013 at 4:53 AM, Jan Kara <jack@suse.cz> wrote: > On Wed 25-09-13 20:08:38, InvTraySts wrote: >> (Going to merge these two back, because both of you are actually >> helping me, but with the two conversations being segregated like this >> it makes it hard to correlate with you both) >> The partprobe did work with getting the partition table reread. >> After that, the tune2fs sorta worked. >> root@server:~# tune2fs -f -O ^has_journal /dev/sdf1 >> tune2fs 1.42.5 (29-Jul-2012) >> root@server:~# mount /dev/sdf1 /media/tmp >> root@server:~# ls -l /media/tmp/ >> total 0 >> >> When I try and use the debugfs /dev/sdf1 >> >> root@server:~# debugfs /dev/sdf1 >> debugfs 1.42.5 (29-Jul-2012) >> debugfs: ls >> EXT2 directory corrupted >> debugfs: ls / >> /: EXT2 directory corrupted >> >> The drive is supposed to be ext4. > 'EXT2' in the message doesn't mean anything. It is always there. But we > see that even root directory is corrupted. That's bad. You can try what > 'stat /' says in debugfs? Possibly also run 'dump / <somefile>' to dump raw > directory data to <somefile> and attach the file here. But also given the > output of e2fsck I'm afraid there won't be much to save on the drive... > > Honza > >> root@server:~# dumpe2fs /dev/sdf1 >> dumpe2fs 1.42.5 (29-Jul-2012) >> Filesystem volume name: root >> Last mounted on: /media/ubuntu/root >> Filesystem UUID: removed >> Filesystem magic number: 0xEF53 >> Filesystem revision #: 1 (dynamic) >> Filesystem features: ext_attr resize_inode dir_index filetype >> extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink >> extra_isize >> Filesystem flags: signed_directory_hash >> Default mount options: user_xattr acl >> Filesystem state: not clean with errors >> Errors behavior: Continue >> Filesystem OS type: Linux >> Inode count: 4849664 >> Block count: 19398144 >> Reserved block count: 969907 >> Free blocks: 17066988 >> Free inodes: 4592929 >> First block: 0 >> Block size: 4096 >> Fragment size: 4096 >> Reserved GDT blocks: 1019 >> Blocks per group: 32768 >> Fragments per group: 32768 >> Inodes per group: 8192 >> Inode blocks per group: 512 >> Flex block group size: 16 >> Filesystem created: Sat May 25 14:59:50 2013 >> Last mount time: Wed Sep 25 19:59:07 2013 >> Last write time: Wed Sep 25 19:59:51 2013 >> Mount count: 1 >> Maximum mount count: -1 >> Last checked: Sat Aug 24 16:56:09 2013 >> Check interval: 0 (<none>) >> Lifetime writes: 107 GB >> Reserved blocks uid: 0 (user root) >> Reserved blocks gid: 0 (group root) >> First inode: 11 >> Inode size: 256 >> Required extra isize: 28 >> Desired extra isize: 28 >> Default directory hash: half_md4 >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> Journal backup: inode blocks >> FS Error count: 9 >> First error time: Sat Aug 24 13:44:55 2013 >> First error function: ext4_iget >> First error line #: 3889 >> First error inode #: 8 >> First error block #: 0 >> Last error time: Wed Sep 25 19:59:12 2013 >> Last error function: htree_dirblock_to_tree >> Last error line #: 892 >> Last error inode #: 2 >> Last error block #: 20037 >> >> [...] >> [output scrolls very fast, and is very large] >> Group 10: (Blocks 327680-360447) [ITABLE_ZEROED] >> Checksum 0xecaa, unused inodes 0 >> Block bitmap at 1035 (bg #0 + 1035), Inode bitmap at 1051 (bg #0 + 1051) >> Inode table at 6177-6688 (bg #0 + 6177) >> 0 free blocks, 16 free inodes, 0 directories >> Free blocks: 327680, 327683-327685, 327687, 327689-327690, >> 327692-327693, 327695-327697, 327700-327701, 327703, 327705, >> 327707-327709, 327711-327715, 327718-327727, 327730-327808, >> 327810-327823, 327825-327842, 327846-327855, 327857-327875, 327878, >> 327881 >> >> (the total amount of pages that the dumpe2fs comes out with is about >> 240 pages. something that can't be pasted. If attachments would be >> required, I can attach a file with it). >> >> On Wed, Sep 25, 2013 at 5:28 PM, Jan Kara <jack@suse.cz> wrote: >> > On Wed 25-09-13 15:24:34, InvTraySts wrote: >> >> And am cloning the drive without the sync parameter this time. >> >> root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror >> >> After it finished, I attempted to run dumpe2fs and it still responds with: >> >> root@server:~# dumpe2fs /dev/sdf1 >> >> dumpe2fs 1.42.5 (29-Jul-2012) >> >> dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 >> >> Couldn't find valid filesystem superblock. >> > Well, that's likely because the partition table on /dev/sdf didn't get >> > reread. You can run 'partprobe /dev/sdf' to tell the kernel about the new >> > partition table. >> > >> > Honza >> > >> >> So I went ahead and tried to run the tune2fs command: >> >> root@server:~# tune2fs -f -O ^has_journal /dev/sda1 >> >> tune2fs 1.42.5 (29-Jul-2012) >> >> tune2fs: Bad magic number in super-block while trying to open /dev/sda1 >> >> Couldn't find valid filesystem superblock. >> >> >> >> Which also fails, yet dumpe2fs on /dev/sda1 works fine. >> >> >> >> >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: >> >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> >> >> So long story short, I had a server running that had a processor fail >> >> >> while powered on, causing the file systems to become corrupt. I >> >> >> replaced the motherboard, processor and power supply just to be on the >> >> >> safe side. However, I am at a bit of a loss as to what to do now. I >> >> >> was working sandeen in the OFTC IRC channel, but, on his >> >> >> recommendation he suggested me to post something to the mailing list. >> >> >> >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> >> >> card. >> >> >> If I try to mount this using: >> >> >> mount -t ext4 /dev/sda1 /media/tmp >> >> >> >> >> >> It complains in dmesg with the following output: >> >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> >> >> comm mount: bad extra_isize (18013 != 256) >> >> >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> >> >> >> >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> >> >> root@server:~# dumpe2fs -f /dev/sda1 >> >> >> dumpe2fs 1.42.5 (29-Jul-2012) >> >> >> Filesystem volume name: root >> >> >> Last mounted on: /media/ubuntu/root >> >> >> Filesystem UUID: f959e195-[removed] >> >> >> Filesystem magic number: 0xEF53 >> >> >> Filesystem revision #: 1 (dynamic) >> >> >> Filesystem features: has_journal ext_attr resize_inode dir_index >> >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> >> >> dir_nlink extra_isize >> >> >> Filesystem flags: signed_directory_hash >> >> >> Default mount options: user_xattr acl >> >> >> Filesystem state: not clean with errors >> >> >> Errors behavior: Continue >> >> >> Filesystem OS type: Linux >> >> >> Inode count: 4849664 >> >> >> Block count: 19398144 >> >> >> Reserved block count: 969907 >> >> >> Free blocks: 17034219 >> >> >> Free inodes: 4592929 >> >> >> First block: 0 >> >> >> Block size: 4096 >> >> >> Fragment size: 4096 >> >> >> Reserved GDT blocks: 1019 >> >> >> Blocks per group: 32768 >> >> >> Fragments per group: 32768 >> >> >> Inodes per group: 8192 >> >> >> Inode blocks per group: 512 >> >> >> Flex block group size: 16 >> >> >> Filesystem created: Sat May 25 14:59:50 2013 >> >> >> Last mount time: Sat Aug 24 11:04:25 2013 >> >> >> Last write time: Tue Sep 24 13:55:36 2013 >> >> >> Mount count: 0 >> >> >> Maximum mount count: -1 >> >> >> Last checked: Sat Aug 24 16:56:09 2013 >> >> >> Check interval: 0 (<none>) >> >> >> Lifetime writes: 107 GB >> >> >> Reserved blocks uid: 0 (user root) >> >> >> Reserved blocks gid: 0 (group root) >> >> >> First inode: 11 >> >> >> Inode size: 256 >> >> >> Required extra isize: 28 >> >> >> Desired extra isize: 28 >> >> >> Journal inode: 8 >> >> >> Default directory hash: half_md4 >> >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> >> >> Journal backup: inode blocks >> >> >> FS Error count: 8 >> >> >> First error time: Sat Aug 24 13:44:55 2013 >> >> >> First error function: ext4_iget >> >> >> First error line #: 3889 >> >> >> First error inode #: 8 >> >> >> First error block #: 0 >> >> >> Last error time: Tue Sep 24 13:55:36 2013 >> >> >> Last error function: ext4_iget >> >> >> Last error line #: 3888 >> >> >> Last error inode #: 8 >> >> >> Last error block #: 0 >> >> >> dumpe2fs: Corrupt extent header while reading journal super block >> >> > OK, so really journal inode (inode #8) looks toast but superblock looks >> >> > OK. >> >> > >> >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> >> >> and currently I am having more problems with the cloned drive than I >> >> >> am with the original. >> >> >> >> >> >> sandeen said something about using tune2fs to tell it to remove the >> >> >> has_journal flag, but I might need some assistance with that. >> >> > Yes, you can do that with: >> >> > tune2fs -f -O ^has_journal /dev/sda1 >> >> > >> >> > Let's see what mount will say after that. >> >> > >> >> > Another option is to run >> >> > debugfs /dev/sda1 >> >> > >> >> > Then you can use ls, cd, and other debugfs commands to move within the >> >> > filesystem and investigate things. If that will work, you have a reasonable >> >> > chance of getting at least some data back. >> >> > >> >> > Honza >> >> > -- >> >> > Jan Kara <jack@suse.cz> >> >> > SUSE Labs, CR >> >> >> >> >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: >> >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: >> >> >> So long story short, I had a server running that had a processor fail >> >> >> while powered on, causing the file systems to become corrupt. I >> >> >> replaced the motherboard, processor and power supply just to be on the >> >> >> safe side. However, I am at a bit of a loss as to what to do now. I >> >> >> was working sandeen in the OFTC IRC channel, but, on his >> >> >> recommendation he suggested me to post something to the mailing list. >> >> >> >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). >> >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i >> >> >> card. >> >> >> If I try to mount this using: >> >> >> mount -t ext4 /dev/sda1 /media/tmp >> >> >> >> >> >> It complains in dmesg with the following output: >> >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: >> >> >> comm mount: bad extra_isize (18013 != 256) >> >> >> [685621.845213] EXT4-fs (sda1): no journal found >> >> >> >> >> >> >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: >> >> >> root@server:~# dumpe2fs -f /dev/sda1 >> >> >> dumpe2fs 1.42.5 (29-Jul-2012) >> >> >> Filesystem volume name: root >> >> >> Last mounted on: /media/ubuntu/root >> >> >> Filesystem UUID: f959e195-[removed] >> >> >> Filesystem magic number: 0xEF53 >> >> >> Filesystem revision #: 1 (dynamic) >> >> >> Filesystem features: has_journal ext_attr resize_inode dir_index >> >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg >> >> >> dir_nlink extra_isize >> >> >> Filesystem flags: signed_directory_hash >> >> >> Default mount options: user_xattr acl >> >> >> Filesystem state: not clean with errors >> >> >> Errors behavior: Continue >> >> >> Filesystem OS type: Linux >> >> >> Inode count: 4849664 >> >> >> Block count: 19398144 >> >> >> Reserved block count: 969907 >> >> >> Free blocks: 17034219 >> >> >> Free inodes: 4592929 >> >> >> First block: 0 >> >> >> Block size: 4096 >> >> >> Fragment size: 4096 >> >> >> Reserved GDT blocks: 1019 >> >> >> Blocks per group: 32768 >> >> >> Fragments per group: 32768 >> >> >> Inodes per group: 8192 >> >> >> Inode blocks per group: 512 >> >> >> Flex block group size: 16 >> >> >> Filesystem created: Sat May 25 14:59:50 2013 >> >> >> Last mount time: Sat Aug 24 11:04:25 2013 >> >> >> Last write time: Tue Sep 24 13:55:36 2013 >> >> >> Mount count: 0 >> >> >> Maximum mount count: -1 >> >> >> Last checked: Sat Aug 24 16:56:09 2013 >> >> >> Check interval: 0 (<none>) >> >> >> Lifetime writes: 107 GB >> >> >> Reserved blocks uid: 0 (user root) >> >> >> Reserved blocks gid: 0 (group root) >> >> >> First inode: 11 >> >> >> Inode size: 256 >> >> >> Required extra isize: 28 >> >> >> Desired extra isize: 28 >> >> >> Journal inode: 8 >> >> >> Default directory hash: half_md4 >> >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f >> >> >> Journal backup: inode blocks >> >> >> FS Error count: 8 >> >> >> First error time: Sat Aug 24 13:44:55 2013 >> >> >> First error function: ext4_iget >> >> >> First error line #: 3889 >> >> >> First error inode #: 8 >> >> >> First error block #: 0 >> >> >> Last error time: Tue Sep 24 13:55:36 2013 >> >> >> Last error function: ext4_iget >> >> >> Last error line #: 3888 >> >> >> Last error inode #: 8 >> >> >> Last error block #: 0 >> >> >> dumpe2fs: Corrupt extent header while reading journal super block >> >> > OK, so really journal inode (inode #8) looks toast but superblock looks >> >> > OK. >> >> > >> >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, >> >> >> and currently I am having more problems with the cloned drive than I >> >> >> am with the original. >> >> >> >> >> >> sandeen said something about using tune2fs to tell it to remove the >> >> >> has_journal flag, but I might need some assistance with that. >> >> > Yes, you can do that with: >> >> > tune2fs -f -O ^has_journal /dev/sda1 >> >> > >> >> > Let's see what mount will say after that. >> >> > >> >> > Another option is to run >> >> > debugfs /dev/sda1 >> >> > >> >> > Then you can use ls, cd, and other debugfs commands to move within the >> >> > filesystem and investigate things. If that will work, you have a reasonable >> >> > chance of getting at least some data back. >> >> > >> >> > Honza >> >> > -- >> >> > Jan Kara <jack@suse.cz> >> >> > SUSE Labs, CR >> > -- >> > Jan Kara <jack@suse.cz> >> > SUSE Labs, CR > -- > Jan Kara <jack@suse.cz> > SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS 2013-09-26 17:45 ` InvTraySts @ 2013-09-26 20:18 ` Jan Kara 0 siblings, 0 replies; 8+ messages in thread From: Jan Kara @ 2013-09-26 20:18 UTC (permalink / raw) To: InvTraySts; +Cc: Jan Kara, Eric Sandeen, linux-ext4, linux-fsdevel On Thu 26-09-13 13:45:34, InvTraySts wrote: > root@server:~# debugfs /dev/sdf1 > debugfs 1.42.5 (29-Jul-2012) > debugfs: stat / > stat: A block group is missing an inode table while reading inode 2 > debugfs: dump /etc/ > dump: Usage: dump_inode [-p] <file> <output_file> > debugfs: dump /etc/passwd /root/passwd.recovery > /etc/passwd: A block group is missing an inode table > debugfs: ls /etc/ > /etc/: A block group is missing an inode table > > So now I am getting a different error message, even though literally > nothing has changed other than the time I ran the command. I figured > the passwd file would be the easiest thing to recover, since I don't > remember the exact paths for other application configurations without > doing some digging for the paths and such. > > At one point, before the mailing list and me joining the #ext channel, > I was able to see that there was a lost and found folder, I just > couldn't navigate it. So, I am not sure why that has changed either. I > assume logical corruption to the extent it has could cause almost > anything. Isn't the fs already modified from fsck? Anyway, the fs really looks hopelessly damaged so I'm afraid we won't be able to get any data from it. Honza > On Thu, Sep 26, 2013 at 4:53 AM, Jan Kara <jack@suse.cz> wrote: > > On Wed 25-09-13 20:08:38, InvTraySts wrote: > >> (Going to merge these two back, because both of you are actually > >> helping me, but with the two conversations being segregated like this > >> it makes it hard to correlate with you both) > >> The partprobe did work with getting the partition table reread. > >> After that, the tune2fs sorta worked. > >> root@server:~# tune2fs -f -O ^has_journal /dev/sdf1 > >> tune2fs 1.42.5 (29-Jul-2012) > >> root@server:~# mount /dev/sdf1 /media/tmp > >> root@server:~# ls -l /media/tmp/ > >> total 0 > >> > >> When I try and use the debugfs /dev/sdf1 > >> > >> root@server:~# debugfs /dev/sdf1 > >> debugfs 1.42.5 (29-Jul-2012) > >> debugfs: ls > >> EXT2 directory corrupted > >> debugfs: ls / > >> /: EXT2 directory corrupted > >> > >> The drive is supposed to be ext4. > > 'EXT2' in the message doesn't mean anything. It is always there. But we > > see that even root directory is corrupted. That's bad. You can try what > > 'stat /' says in debugfs? Possibly also run 'dump / <somefile>' to dump raw > > directory data to <somefile> and attach the file here. But also given the > > output of e2fsck I'm afraid there won't be much to save on the drive... > > > > Honza > > > >> root@server:~# dumpe2fs /dev/sdf1 > >> dumpe2fs 1.42.5 (29-Jul-2012) > >> Filesystem volume name: root > >> Last mounted on: /media/ubuntu/root > >> Filesystem UUID: removed > >> Filesystem magic number: 0xEF53 > >> Filesystem revision #: 1 (dynamic) > >> Filesystem features: ext_attr resize_inode dir_index filetype > >> extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink > >> extra_isize > >> Filesystem flags: signed_directory_hash > >> Default mount options: user_xattr acl > >> Filesystem state: not clean with errors > >> Errors behavior: Continue > >> Filesystem OS type: Linux > >> Inode count: 4849664 > >> Block count: 19398144 > >> Reserved block count: 969907 > >> Free blocks: 17066988 > >> Free inodes: 4592929 > >> First block: 0 > >> Block size: 4096 > >> Fragment size: 4096 > >> Reserved GDT blocks: 1019 > >> Blocks per group: 32768 > >> Fragments per group: 32768 > >> Inodes per group: 8192 > >> Inode blocks per group: 512 > >> Flex block group size: 16 > >> Filesystem created: Sat May 25 14:59:50 2013 > >> Last mount time: Wed Sep 25 19:59:07 2013 > >> Last write time: Wed Sep 25 19:59:51 2013 > >> Mount count: 1 > >> Maximum mount count: -1 > >> Last checked: Sat Aug 24 16:56:09 2013 > >> Check interval: 0 (<none>) > >> Lifetime writes: 107 GB > >> Reserved blocks uid: 0 (user root) > >> Reserved blocks gid: 0 (group root) > >> First inode: 11 > >> Inode size: 256 > >> Required extra isize: 28 > >> Desired extra isize: 28 > >> Default directory hash: half_md4 > >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> Journal backup: inode blocks > >> FS Error count: 9 > >> First error time: Sat Aug 24 13:44:55 2013 > >> First error function: ext4_iget > >> First error line #: 3889 > >> First error inode #: 8 > >> First error block #: 0 > >> Last error time: Wed Sep 25 19:59:12 2013 > >> Last error function: htree_dirblock_to_tree > >> Last error line #: 892 > >> Last error inode #: 2 > >> Last error block #: 20037 > >> > >> [...] > >> [output scrolls very fast, and is very large] > >> Group 10: (Blocks 327680-360447) [ITABLE_ZEROED] > >> Checksum 0xecaa, unused inodes 0 > >> Block bitmap at 1035 (bg #0 + 1035), Inode bitmap at 1051 (bg #0 + 1051) > >> Inode table at 6177-6688 (bg #0 + 6177) > >> 0 free blocks, 16 free inodes, 0 directories > >> Free blocks: 327680, 327683-327685, 327687, 327689-327690, > >> 327692-327693, 327695-327697, 327700-327701, 327703, 327705, > >> 327707-327709, 327711-327715, 327718-327727, 327730-327808, > >> 327810-327823, 327825-327842, 327846-327855, 327857-327875, 327878, > >> 327881 > >> > >> (the total amount of pages that the dumpe2fs comes out with is about > >> 240 pages. something that can't be pasted. If attachments would be > >> required, I can attach a file with it). > >> > >> On Wed, Sep 25, 2013 at 5:28 PM, Jan Kara <jack@suse.cz> wrote: > >> > On Wed 25-09-13 15:24:34, InvTraySts wrote: > >> >> And am cloning the drive without the sync parameter this time. > >> >> root@server:~# dd if=/dev/sda of=/dev/sdf bs=4096 conv=notrunc,noerror > >> >> After it finished, I attempted to run dumpe2fs and it still responds with: > >> >> root@server:~# dumpe2fs /dev/sdf1 > >> >> dumpe2fs 1.42.5 (29-Jul-2012) > >> >> dumpe2fs: Bad magic number in super-block while trying to open /dev/sdf1 > >> >> Couldn't find valid filesystem superblock. > >> > Well, that's likely because the partition table on /dev/sdf didn't get > >> > reread. You can run 'partprobe /dev/sdf' to tell the kernel about the new > >> > partition table. > >> > > >> > Honza > >> > > >> >> So I went ahead and tried to run the tune2fs command: > >> >> root@server:~# tune2fs -f -O ^has_journal /dev/sda1 > >> >> tune2fs 1.42.5 (29-Jul-2012) > >> >> tune2fs: Bad magic number in super-block while trying to open /dev/sda1 > >> >> Couldn't find valid filesystem superblock. > >> >> > >> >> Which also fails, yet dumpe2fs on /dev/sda1 works fine. > >> >> > >> >> > >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > >> >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> >> >> So long story short, I had a server running that had a processor fail > >> >> >> while powered on, causing the file systems to become corrupt. I > >> >> >> replaced the motherboard, processor and power supply just to be on the > >> >> >> safe side. However, I am at a bit of a loss as to what to do now. I > >> >> >> was working sandeen in the OFTC IRC channel, but, on his > >> >> >> recommendation he suggested me to post something to the mailing list. > >> >> >> > >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> >> >> card. > >> >> >> If I try to mount this using: > >> >> >> mount -t ext4 /dev/sda1 /media/tmp > >> >> >> > >> >> >> It complains in dmesg with the following output: > >> >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> >> >> comm mount: bad extra_isize (18013 != 256) > >> >> >> [685621.845213] EXT4-fs (sda1): no journal found > >> >> >> > >> >> >> > >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> >> >> root@server:~# dumpe2fs -f /dev/sda1 > >> >> >> dumpe2fs 1.42.5 (29-Jul-2012) > >> >> >> Filesystem volume name: root > >> >> >> Last mounted on: /media/ubuntu/root > >> >> >> Filesystem UUID: f959e195-[removed] > >> >> >> Filesystem magic number: 0xEF53 > >> >> >> Filesystem revision #: 1 (dynamic) > >> >> >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> >> >> dir_nlink extra_isize > >> >> >> Filesystem flags: signed_directory_hash > >> >> >> Default mount options: user_xattr acl > >> >> >> Filesystem state: not clean with errors > >> >> >> Errors behavior: Continue > >> >> >> Filesystem OS type: Linux > >> >> >> Inode count: 4849664 > >> >> >> Block count: 19398144 > >> >> >> Reserved block count: 969907 > >> >> >> Free blocks: 17034219 > >> >> >> Free inodes: 4592929 > >> >> >> First block: 0 > >> >> >> Block size: 4096 > >> >> >> Fragment size: 4096 > >> >> >> Reserved GDT blocks: 1019 > >> >> >> Blocks per group: 32768 > >> >> >> Fragments per group: 32768 > >> >> >> Inodes per group: 8192 > >> >> >> Inode blocks per group: 512 > >> >> >> Flex block group size: 16 > >> >> >> Filesystem created: Sat May 25 14:59:50 2013 > >> >> >> Last mount time: Sat Aug 24 11:04:25 2013 > >> >> >> Last write time: Tue Sep 24 13:55:36 2013 > >> >> >> Mount count: 0 > >> >> >> Maximum mount count: -1 > >> >> >> Last checked: Sat Aug 24 16:56:09 2013 > >> >> >> Check interval: 0 (<none>) > >> >> >> Lifetime writes: 107 GB > >> >> >> Reserved blocks uid: 0 (user root) > >> >> >> Reserved blocks gid: 0 (group root) > >> >> >> First inode: 11 > >> >> >> Inode size: 256 > >> >> >> Required extra isize: 28 > >> >> >> Desired extra isize: 28 > >> >> >> Journal inode: 8 > >> >> >> Default directory hash: half_md4 > >> >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> >> >> Journal backup: inode blocks > >> >> >> FS Error count: 8 > >> >> >> First error time: Sat Aug 24 13:44:55 2013 > >> >> >> First error function: ext4_iget > >> >> >> First error line #: 3889 > >> >> >> First error inode #: 8 > >> >> >> First error block #: 0 > >> >> >> Last error time: Tue Sep 24 13:55:36 2013 > >> >> >> Last error function: ext4_iget > >> >> >> Last error line #: 3888 > >> >> >> Last error inode #: 8 > >> >> >> Last error block #: 0 > >> >> >> dumpe2fs: Corrupt extent header while reading journal super block > >> >> > OK, so really journal inode (inode #8) looks toast but superblock looks > >> >> > OK. > >> >> > > >> >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> >> >> and currently I am having more problems with the cloned drive than I > >> >> >> am with the original. > >> >> >> > >> >> >> sandeen said something about using tune2fs to tell it to remove the > >> >> >> has_journal flag, but I might need some assistance with that. > >> >> > Yes, you can do that with: > >> >> > tune2fs -f -O ^has_journal /dev/sda1 > >> >> > > >> >> > Let's see what mount will say after that. > >> >> > > >> >> > Another option is to run > >> >> > debugfs /dev/sda1 > >> >> > > >> >> > Then you can use ls, cd, and other debugfs commands to move within the > >> >> > filesystem and investigate things. If that will work, you have a reasonable > >> >> > chance of getting at least some data back. > >> >> > > >> >> > Honza > >> >> > -- > >> >> > Jan Kara <jack@suse.cz> > >> >> > SUSE Labs, CR > >> >> > >> >> > >> >> On Wed, Sep 25, 2013 at 12:12 PM, Jan Kara <jack@suse.cz> wrote: > >> >> > On Tue 24-09-13 22:25:49, InvTraySts wrote: > >> >> >> So long story short, I had a server running that had a processor fail > >> >> >> while powered on, causing the file systems to become corrupt. I > >> >> >> replaced the motherboard, processor and power supply just to be on the > >> >> >> safe side. However, I am at a bit of a loss as to what to do now. I > >> >> >> was working sandeen in the OFTC IRC channel, but, on his > >> >> >> recommendation he suggested me to post something to the mailing list. > >> >> >> > >> >> >> Lets start off with one drive at a time (I have 4 that are corrupt). > >> >> >> The specific logical drive in question was in RAID1 on a Dell PERC 5/i > >> >> >> card. > >> >> >> If I try to mount this using: > >> >> >> mount -t ext4 /dev/sda1 /media/tmp > >> >> >> > >> >> >> It complains in dmesg with the following output: > >> >> >> 685621.845207] EXT4-fs error (device sda1): ext4_iget:3888: inode #8: > >> >> >> comm mount: bad extra_isize (18013 != 256) > >> >> >> [685621.845213] EXT4-fs (sda1): no journal found > >> >> >> > >> >> >> > >> >> >> However, if I run dumpe2fs -f /dev/sda1 I get the following output: > >> >> >> root@server:~# dumpe2fs -f /dev/sda1 > >> >> >> dumpe2fs 1.42.5 (29-Jul-2012) > >> >> >> Filesystem volume name: root > >> >> >> Last mounted on: /media/ubuntu/root > >> >> >> Filesystem UUID: f959e195-[removed] > >> >> >> Filesystem magic number: 0xEF53 > >> >> >> Filesystem revision #: 1 (dynamic) > >> >> >> Filesystem features: has_journal ext_attr resize_inode dir_index > >> >> >> filetype extent flex_bg sparse_super large_file huge_file uninit_bg > >> >> >> dir_nlink extra_isize > >> >> >> Filesystem flags: signed_directory_hash > >> >> >> Default mount options: user_xattr acl > >> >> >> Filesystem state: not clean with errors > >> >> >> Errors behavior: Continue > >> >> >> Filesystem OS type: Linux > >> >> >> Inode count: 4849664 > >> >> >> Block count: 19398144 > >> >> >> Reserved block count: 969907 > >> >> >> Free blocks: 17034219 > >> >> >> Free inodes: 4592929 > >> >> >> First block: 0 > >> >> >> Block size: 4096 > >> >> >> Fragment size: 4096 > >> >> >> Reserved GDT blocks: 1019 > >> >> >> Blocks per group: 32768 > >> >> >> Fragments per group: 32768 > >> >> >> Inodes per group: 8192 > >> >> >> Inode blocks per group: 512 > >> >> >> Flex block group size: 16 > >> >> >> Filesystem created: Sat May 25 14:59:50 2013 > >> >> >> Last mount time: Sat Aug 24 11:04:25 2013 > >> >> >> Last write time: Tue Sep 24 13:55:36 2013 > >> >> >> Mount count: 0 > >> >> >> Maximum mount count: -1 > >> >> >> Last checked: Sat Aug 24 16:56:09 2013 > >> >> >> Check interval: 0 (<none>) > >> >> >> Lifetime writes: 107 GB > >> >> >> Reserved blocks uid: 0 (user root) > >> >> >> Reserved blocks gid: 0 (group root) > >> >> >> First inode: 11 > >> >> >> Inode size: 256 > >> >> >> Required extra isize: 28 > >> >> >> Desired extra isize: 28 > >> >> >> Journal inode: 8 > >> >> >> Default directory hash: half_md4 > >> >> >> Directory Hash Seed: 01a8f605-b2bc-41ee-b7b5-11d843ab622f > >> >> >> Journal backup: inode blocks > >> >> >> FS Error count: 8 > >> >> >> First error time: Sat Aug 24 13:44:55 2013 > >> >> >> First error function: ext4_iget > >> >> >> First error line #: 3889 > >> >> >> First error inode #: 8 > >> >> >> First error block #: 0 > >> >> >> Last error time: Tue Sep 24 13:55:36 2013 > >> >> >> Last error function: ext4_iget > >> >> >> Last error line #: 3888 > >> >> >> Last error inode #: 8 > >> >> >> Last error block #: 0 > >> >> >> dumpe2fs: Corrupt extent header while reading journal super block > >> >> > OK, so really journal inode (inode #8) looks toast but superblock looks > >> >> > OK. > >> >> > > >> >> >> So I attempted to clone the drive to a 2TB backup drive that is empty, > >> >> >> and currently I am having more problems with the cloned drive than I > >> >> >> am with the original. > >> >> >> > >> >> >> sandeen said something about using tune2fs to tell it to remove the > >> >> >> has_journal flag, but I might need some assistance with that. > >> >> > Yes, you can do that with: > >> >> > tune2fs -f -O ^has_journal /dev/sda1 > >> >> > > >> >> > Let's see what mount will say after that. > >> >> > > >> >> > Another option is to run > >> >> > debugfs /dev/sda1 > >> >> > > >> >> > Then you can use ls, cd, and other debugfs commands to move within the > >> >> > filesystem and investigate things. If that will work, you have a reasonable > >> >> > chance of getting at least some data back. > >> >> > > >> >> > Honza > >> >> > -- > >> >> > Jan Kara <jack@suse.cz> > >> >> > SUSE Labs, CR > >> > -- > >> > Jan Kara <jack@suse.cz> > >> > SUSE Labs, CR > > -- > > Jan Kara <jack@suse.cz> > > SUSE Labs, CR -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-09-26 20:18 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAOWv6b1Jq0MAtELB_brhg_G3S2svEW8EaW9A8ciddTUdqDWFdQ@mail.gmail.com>
[not found] ` <CAOWv6b1m8A2S3sjDsJxOrRVztch6iq4zH9eFq5DKFMqtWWWpEw@mail.gmail.com>
2013-09-25 2:25 ` Fwd: Need help with Data Recovery on Ext4 partitions that became corrupted on running OS InvTraySts
2013-09-25 16:12 ` Jan Kara
2013-09-25 19:24 ` InvTraySts
2013-09-25 21:28 ` Jan Kara
2013-09-26 0:08 ` InvTraySts
2013-09-26 8:53 ` Jan Kara
2013-09-26 17:45 ` InvTraySts
2013-09-26 20:18 ` Jan Kara
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).