From: Jens Pranaitis <pranaitis@phil.hhu.de>
To: linux-ext4@vger.kernel.org
Subject: filesystem unrepairable after resizing from 16TB to 20TB
Date: Wed, 19 Mar 2014 11:34:04 +0100 [thread overview]
Message-ID: <5329729C.8020503@phil.hhu.de> (raw)
[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]
Dear all,
after resizing a 64bit ext4 filesystem from 16TB to 20TB, which
completed without errors, I started a rsync of ~3TB, about 30min into
the copy the following kernel messages started to pop up:
Mar 17 12:03:31 s_local@backup1/backup1 kernel: [2081501.742613] EXT4-fs
(dm-2): mounted filesystem with ordered data mode. Opts: acl,user_xattr
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.489419] EXT4-fs
error (device dm-2): ext4_mb_generate_buddy:755: group 1, 30820 clusters
in bitmap, 30207 in gd
[...]
I attached the full kernel log from the day of the resize.
There are a lot of broken files and when trying to fix them with e2fsck
the program exits:
Inode 2195807 has an invalid extent node (blk 35192760, lblk 0)
Clear? yes
e2fsck: e2fsck_read_bitmaps: illegal bitmap block(s) for /dev/vg-data/backup
/dev/vg-data/backup: ***** FILE SYSTEM WAS MODIFIED *****
e2fsck: aborted
You can find the output of e2fsck -n /dev/vg-data/backup at
http://user.phil.hhu.de/~pranaitis/e2fsck.log.xz
the uncompressed log size is 357MB so I thought I'd rather not send it
via this list.
The kernel version is 3.11-0.bpo.2-amd64 from Debian backports. The
e2fsprogs version is vanilla 1.42.9.
The machine itself is a VM created with VMWare ESXi 5.5.0 using Thick
Provisioned disks.
I don't have any hope of salvaging the filesystem, but is there anything
I can do to help in preventing such issues in the future?
Kind regards,
Jens Pranaitis
--
IKM-Serviceteam der Philosophischen Fakultät
HHU Düsseldorf
Gebäude 23.21, Ebene 04, Raum 88
Tel.: 0211/81-13077
[-- Attachment #2: kernel-ext4-mar17.log --]
[-- Type: text/x-log, Size: 6013 bytes --]
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.701826] scsi 0:0:3:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.701859] scsi target0:0:3: Beginning Domain Validation
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702784] scsi target0:0:3: Domain Validation skipping write tests
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702787] scsi target0:0:3: Ending Domain Validation
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.702851] scsi target0:0:3: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703410] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703468] sd 0:0:3:0: [sdf] 8589934592 512-byte logical blocks: (4.39 TB/4.00 TiB)
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703516] sd 0:0:3:0: [sdf] Write Protect is off
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703522] sd 0:0:3:0: [sdf] Mode Sense: 61 00 00 00
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703655] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.703660] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.704728] sd 0:0:3:0: Attached scsi generic sg6 type 0
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.704841] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.705008] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.705014] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.707892] sdf: unknown partition table
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708146] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708281] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708286] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:28 s_local@backup1/backup1 kernel: [2078195.708478] sd 0:0:3:0: [sdf] Attached SCSI disk
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071695] sd 0:0:3:0: [sdf] Very big device. Trying to use READ CAPACITY(16).
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071849] sd 0:0:3:0: [sdf] Cache data unavailable
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.071855] sd 0:0:3:0: [sdf] Assuming drive cache: write through
Mar 17 11:08:53 s_local@backup1/backup1 kernel: [2078221.075132] sdf: sdf1
Mar 17 12:03:31 s_local@backup1/backup1 kernel: [2081501.742613] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: acl,user_xattr
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.489419] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 1, 30820 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490021] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 2, 30812 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490369] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 3, 30628 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.490675] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 4, 30820 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491144] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 5, 30636 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491536] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 6, 30628 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.491882] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 7, 30444 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492239] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 8, 30820 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492523] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 9, 30620 clusters in bitmap, 30207 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492698] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 10, 30628 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.492870] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 11, 30428 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493043] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 12, 30636 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493215] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 13, 30436 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.493397] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 14, 30444 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.494014] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 15, 30244 clusters in bitmap, 32768 in gd
Mar 17 12:37:10 s_local@backup1/backup1 kernel: [2083523.517545] JBD2: Spotted dirty metadata buffer (dev = dm-2, blocknr = 0). There's a risk of filesystem corruption in case of system crash.
Mar 17 12:37:36 s_local@backup1/backup1 kernel: [2083549.233904] EXT4-fs error (device dm-2): mb_free_blocks:1426: group 1607, block 52661289:freeing already freed block (bit 3113)
Mar 17 12:37:36 s_local@backup1/backup1 kernel: [2083549.234045] EXT4-fs error (device dm-2): ext4_mb_generate_buddy:755: group 1607, 32768 clusters in bitmap, 32769 in gd
reply other threads:[~2014-03-19 10:41 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5329729C.8020503@phil.hhu.de \
--to=pranaitis@phil.hhu.de \
--cc=linux-ext4@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).