From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gregory L Shomo Subject: Re: parent transid troubles Date: Wed, 20 Apr 2011 09:20:20 -0400 Message-ID: References: <1303304693-sup-9469@think> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: linux-btrfs@vger.kernel.org To: Chris Mason Return-path: In-Reply-To: <1303304693-sup-9469@think> (message from Chris Mason on Wed, 20 Apr 2011 09:06:21 -0400) List-ID: Chris Mason writes: > Excerpts from Gregory L Shomo's message of 2011-04-20 08:56:02 -0400: >> Chris Mason writes: >> >> > Excerpts from Gregory L Shomo's message of 2011-04-19 15:08:13 -0400: >> >> Hello list- >> >> >> >> Under heavy load (i/o), one of our fileservers lost two drives >> >> in a raid6 configuration. After the drives were synchronized, >> >> we can no longer mount the multiple-device btrfs filesystem >> >> due to (at least) parent transid verification. >> >> >> >> btrfsck built from git commit 1b444cd2e6ab8dcafdd47dbaeaae369dd1517c17 >> >> runs for a while and then aborts on 'failed to find block number'. >> >> Sample output includes : >> > >> > Looks like the rebuild gave you older copies of some of the blocks. >> > btrfsck will exit out pretty early when it sees problems, but I'd say >> > most of your FS is there. >> > >> > Can you please do a btrfs-debug-tree /dev/xxx > out, I'd like to see how >> > far we get. >> > >> > What errors do you get when trying to mount the FS? >> > >> > -chris >> >> I'm not sure how far we will get, but btrfs-debug-tree >> has been running for over 12h now and the screenlog is >> at 80Gb. This may not be surprising, as the filesystem >> is large (60T) and has millions of files. >> >> From the logs at boottime, we have >> >> btrfs: failed to read the system array on sdd1 >> btrfs: open_ctree failed >> >> Should we wait for the btrfs-debug-tree to finish >> before executing an other mount command ? > > For btrfs-debug-tree to run this long, big parts of your FS must be > valid. Also, btrfs-debug-tree must have been able to read the sys > array (which mount was complaining about). > > How easily can you try a newer kernel? We need to make sure and do > readonly operations (mount -o ro), but we may be able to pull out a > bunch of files. > > -chris Sure, we're up for that. Should we rebuild the kernel, or just the btrfs module ? If the kernel, is linux-2.6.38.3 a good choice, or should we build 2.6.39-rc4 ? If we only need to rebuild the btrfs module, should we use Monday's commit to btrfs-unstable ? - greg