From: Jens Pranaitis <pranaitis@phil.hhu.de>
To: linux-ext4@vger.kernel.org
Subject: filesystem unrepairable after resizing from 16TB to 20TB
Date: Wed, 19 Mar 2014 11:34:04 +0100 [thread overview]
Message-ID: <5329729C.8020503@phil.hhu.de> (raw)
[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]
Dear all,
after resizing a 64bit ext4 filesystem from 16TB to 20TB, which
completed without errors, I started a rsync of ~3TB, about 30min into
the copy the following kernel messages started to pop up:
Mar 17 12:03:31 s_local@backup1/backup1 kernel: [2081501.742613] EXT4-fs
(dm-2): mounted filesystem with ordered data mode. Opts: acl,user_xattr
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.489419] EXT4-fs
error (device dm-2): ext4_mb_generate_buddy:755: group 1, 30820 clusters
in bitmap, 30207 in gd
[...]
I attached the full kernel log from the day of the resize.
There are a lot of broken files and when trying to fix them with e2fsck
the program exits:
Inode 2195807 has an invalid extent node (blk 35192760, lblk 0)
Clear? yes
e2fsck: e2fsck_read_bitmaps: illegal bitmap block(s) for /dev/vg-data/backup
/dev/vg-data/backup: ***** FILE SYSTEM WAS MODIFIED *****
e2fsck: aborted
You can find the output of e2fsck -n /dev/vg-data/backup at
http://user.phil.hhu.de/~pranaitis/e2fsck.log.xz
the uncompressed log size is 357MB so I thought I'd rather not send it
via this list.
The kernel version is 3.11-0.bpo.2-amd64 from Debian backports. The
e2fsprogs version is vanilla 1.42.9.
The machine itself is a VM created with VMWare ESXi 5.5.0 using Thick
Provisioned disks.
I don't have any hope of salvaging the filesystem, but is there anything
I can do to help in preventing such issues in the future?
Kind regards,
Jens Pranaitis
--
IKM-Serviceteam der Philosophischen Fakultät
HHU Düsseldorf
Gebäude 23.21, Ebene 04, Raum 88
Tel.: 0211/81-13077
[-- Attachment #2: kernel-ext4-mar17.log --]
[-- Type: text/x-log, Size: 6013 bytes --]
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.701826] scsi 0:0:3:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.701859] scsi target0:0:3: Beginning Domain Validation
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702784] scsi target0:0:3: Domain Validation skipping write tests
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702787] scsi target0:0:3: Ending Domain Validation
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702851] scsi target0:0:3: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703410] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703468] sd 0:0:3:0: [sdf] 8589934592 512-byte logical blocks: (4.39 TB/4.00 TiB)
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703516] sd 0:0:3:0: [sdf] Write Protect is off
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703522] sd 0:0:3:0: [sdf] Mode Sense: 61 00 00 00
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703655] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703660] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.704728] sd 0:0:3:0: Attached scsi generic sg6 type 0
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.704841] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.705008] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.705014] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.707892] sdf: unknown partition table
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708146] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708281] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708286] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708478] sd 0:0:3:0: [sdf] Attached SCSI disk
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071695] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071849] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071855] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.075132] sdf: sdf1
Mar 17 12:03:31 s_local@backup1/backup1 kernel: [2081501.742613] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: acl,user_xattr
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.489419] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 1, 30820 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490021] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 2, 30812 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490369] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 3, 30628 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490675] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 4, 30820 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491144] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 5, 30636 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491536] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 6, 30628 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491882] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 7, 30444 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492239] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 8, 30820 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492523] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 9, 30620 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492698] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 10, 30628 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492870] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 11, 30428 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493043] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 12, 30636 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493215] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 13, 30436 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493397] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 14, 30444 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.494014] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 15, 30244 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.517545] JBD2: Spotted dirty metadata buffer (dev = dm-2, blocknr = 0). There's a risk of filesystem corruption in case of system crash.
Mar 17 12:37:36 s_local@backup1/backup1 kernel: [2083549.233904] EXT4-fs error (device dm-2): mb_free_blocks:1426: group 1607, block 52661289:freeing already freed block (bit 3113)
Mar 17 12:37:36 s_local@backup1/backup1 kernel: [2083549.234045] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 1607, 32768 clusters in bitmap, 32769 in gd
reply other threads:[~2014-03-19 10:41 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5329729C.8020503@phil.hhu.de \
--to=pranaitis@phil.hhu.de \
--cc=linux-ext4@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.