From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vc0-f174.google.com ([209.85.220.174]:56227 "EHLO mail-vc0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751187Ab2FDE1H (ORCPT ); Mon, 4 Jun 2012 00:27:07 -0400 Received: by vcbf11 with SMTP id f11so2243468vcb.19 for ; Sun, 03 Jun 2012 21:27:06 -0700 (PDT) Message-ID: <4FCC3919.9000406@gmail.com> Date: Mon, 04 Jun 2012 00:27:05 -0400 From: Maxim Mikheev MIME-Version: 1.0 To: Liu Bo CC: linux-btrfs@vger.kernel.org, Jan Schmidt , Arne Jansen Subject: Re: Help with data recovering References: <4FC54A5D.8000600@gmail.com> <4FC55043.4050004@gmail.com> <4FC55AB7.3000903@gmail.com> <4FC6D128.60308@gmail.com> <4FCA189F.9050000@gmail.com> <4FCC0DD7.9050807@cn.fujitsu.com> <4FCC12BA.9070701@gmail.com> <4FCC1A6F.8050805@cn.fujitsu.com> <4FCC1AE1.8050802@gmail.com> <4FCC2496.6070805@cn.fujitsu.com> <4FCC27ED.9040503@gmail.com> In-Reply-To: <4FCC27ED.9040503@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Sender: linux-btrfs-owner@vger.kernel.org List-ID: Hi Everyone, Can I do anything else? Max On 06/03/2012 11:13 PM, Maxim Mikheev wrote: > I tried: > > max@s0:~$ sudo btrfs-restore /dev/sdb ~/restored > parent transid verify failed on 5468060241920 wanted 9096 found 7621 > parent transid verify failed on 5468060241920 wanted 9096 found 7621 > parent transid verify failed on 5468060241920 wanted 9096 found 7621 > parent transid verify failed on 5468060241920 wanted 9096 found 7621 > Ignoring transid failure > leaf parent key incorrect 5468060241920 > parent transid verify failed on 5333392302080 wanted 9096 found 4585 > parent transid verify failed on 5333392302080 wanted 9096 found 4585 > Root objectid is 5 > ret is -3 > > max@s0:~$ ls -lahs restored/Irina/ > total 12K > 4.0K drwxr-xr-x 3 root root 4.0K Jun 3 23:12 . > 4.0K drwxrwxr-x 3 max max 4.0K Jun 3 23:12 .. > 4.0K drwxr-xr-x 2 root root 4.0K Jun 3 23:12 .idmapdir2 > max@s0:~$ ls -lahs restored/Irina/.idmapdir2/ > total 8.0K > 4.0K drwxr-xr-x 2 root root 4.0K Jun 3 23:12 . > 4.0K drwxr-xr-x 3 root root 4.0K Jun 3 23:12 .. > 0 -rw-r--r-- 1 root root 0 Jun 3 23:12 4.bucket.lock > 0 -rw-r--r-- 1 root root 0 Jun 3 23:12 7.bucket > max@s0:~$ > > > dmesg: > [ 4764.795798] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid > 2 transid 9096 /dev/sdb > [ 4764.796901] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid > 3 transid 9095 /dev/sdd > [ 4764.797888] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid > 1 transid 9096 /dev/sda > [ 4764.799309] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid > 4 transid 9096 /dev/sde > [ 4764.801220] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid > 5 transid 9096 /dev/sdf > > Command: > sudo btrfs-find-root /dev/sdb > gave me many lines like these: > Well block 4046409486336 seems great, but generation doesn't match, > have=9087, want=9096 > Well block 4046414626816 seems great, but generation doesn't match, > have=9088, want=9096 > Well block 4148447113216 seems great, but generation doesn't match, > have=7618, want=9096 > Well block 4148522024960 seems great, but generation doesn't match, > have=9089, want=9096 > Well block 4148539457536 seems great, but generation doesn't match, > have=9090, want=9096 > Well block 4455562448896 seems great, but generation doesn't match, > have=9092, want=9096 > Well block 4455568302080 seems great, but generation doesn't match, > have=9091, want=9096 > Well block 4848395739136 seems great, but generation doesn't match, > have=9093, want=9096 > Well block 4923796594688 seems great, but generation doesn't match, > have=9094, want=9096 > Well block 4923798065152 seems great, but generation doesn't match, > have=9095, want=9096 > > max@s0:~$ sudo btrfs-restore -t 4923798065152 /dev/sdb ~/restored > parent transid verify failed on 4923798065152 wanted 9096 found 9095 > parent transid verify failed on 4923798065152 wanted 9096 found 9095 > parent transid verify failed on 4923798065152 wanted 9096 found 9095 > parent transid verify failed on 4923798065152 wanted 9096 found 9095 > Ignoring transid failure > Root objectid is 5 > ret is -3 > > > On 06/03/2012 10:59 PM, Liu Bo wrote: >> On 06/04/2012 10:18 AM, Maxim Mikheev wrote: >> >>> Hi Liu, >>> >>> 1) all of them not working (see dmesg at the end) >>> 2) >>> max@s0:~$ sudo btrfs scrub start /dev/sdb >>> ERROR: getting dev info for scrub failed: Inappropriate ioctl for >>> device >>> max@s0:~$ sudo btrfs scrub start /dev/sda >>> ERROR: getting dev info for scrub failed: Inappropriate ioctl for >>> device >>> max@s0:~$ sudo btrfs scrub start /dev/sdd >>> ERROR: getting dev info for scrub failed: Inappropriate ioctl for >>> device >>> max@s0:~$ sudo btrfs scrub start /dev/sde >>> ERROR: getting dev info for scrub failed: Inappropriate ioctl for >>> device >>> max@s0:~$ sudo btrfs scrub start /dev/sdf >>> ERROR: getting dev info for scrub failed: Inappropriate ioctl for >>> device >>> >> >> (add Jan and Arne to cc, they are authors of scrub) >> >> I'm not an expert on scrub, and I'm not clear how to scrub a device >> directly :( >> >> btw, have you tried restore (for attempting to recover data from an >> unmountable filesystem): >> >> https://btrfs.wiki.kernel.org/index.php/Restore >> >> thanks, >> liubo >> >> >>> dmesg after all operations: >>> [ 2183.864056] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid 2 >>> transid 9096 /dev/sdb >>> [ 2183.916128] btrfs: disk space caching is enabled >>> [ 2191.863409] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2191.872937] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2191.873666] btrfs read error corrected: ino 1 off 5468060241920 (dev >>> /dev/sdb sector 2143292648) >>> [ 2191.873678] Failed to read block groups: -5 >>> [ 2191.884636] btrfs: open_ctree failed >>> [ 2222.910225] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid 3 >>> transid 9095 /dev/sdd >>> [ 2222.959128] btrfs: disk space caching is enabled >>> [ 2231.264285] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2231.274306] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2231.275194] btrfs read error corrected: ino 1 off 5468060241920 (dev >>> /dev/sdd sector 2143292648) >>> [ 2231.275207] Failed to read block groups: -5 >>> [ 2231.288795] btrfs: open_ctree failed >>> [ 2240.624691] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid 4 >>> transid 9096 /dev/sde >>> [ 2240.671344] btrfs: disk space caching is enabled >>> [ 2248.916772] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2248.928106] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2248.929091] btrfs read error corrected: ino 1 off 5468060241920 (dev >>> /dev/sdd sector 2143292648) >>> [ 2248.929105] Failed to read block groups: -5 >>> [ 2248.939081] btrfs: open_ctree failed >>> [ 2253.829071] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid 5 >>> transid 9096 /dev/sdf >>> [ 2253.879940] btrfs: disk space caching is enabled >>> [ 2261.754357] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2261.767118] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2261.767929] btrfs read error corrected: ino 1 off 5468060241920 (dev >>> /dev/sdb sector 2143292648) >>> [ 2261.767942] Failed to read block groups: -5 >>> [ 2261.778219] btrfs: open_ctree failed >>> [ 2309.831415] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 devid 1 >>> transid 9096 /dev/sda >>> [ 2309.904520] btrfs: disk space caching is enabled >>> [ 2318.286463] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2318.302991] parent transid verify failed on 5468060241920 wanted >>> 9096 >>> found 7621 >>> [ 2318.304000] btrfs read error corrected: ino 1 off 5468060241920 (dev >>> /dev/sdd sector 2143292648) >>> [ 2318.304013] Failed to read block groups: -5 >>> [ 2318.314587] btrfs: open_ctree failed >>> >>> On 06/03/2012 10:16 PM, Liu Bo wrote: >>>> On 06/04/2012 09:43 AM, Maxim Mikheev wrote: >>>> >>>>> Hi Liu, >>>>> >>>>> thanks for advice. I tried it before btrfsck. results are here: >>>>> max@s0:~$ sudo mount /tank -o recovery >>>>> [sudo] password for max: >>>>> mount: wrong fs type, bad option, bad superblock on /dev/sdf, >>>>> missing codepage or helper program, or other error >>>>> In some cases useful info is found in syslog - try >>>>> dmesg | tail or so >>>>> >>>>> max@s0:~$ sudo mount -o recovery /tank >>>>> mount: wrong fs type, bad option, bad superblock on /dev/sdf, >>>>> missing codepage or helper program, or other error >>>>> In some cases useful info is found in syslog - try >>>>> dmesg | tail or so >>>>> >>>> Two possible ways: >>>> >>>> 1) >>>> I noticed that your btrfs had 5 partitions in all: >>>> mkfs.btrfs /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf >>>> >>>> Can you try to mount other disk partitions instead by hand, like: >>>> mount /dev/sdb /tank >>>> mount /dev/sdc /tank >>>> mount /dev/sdd /tank >>>> mount /dev/sde /tank >>>> mount /dev/sdf /tank >>>> >>>> 2) >>>> use btrfs's scrub to resort to metadata backups created by RAID1. >>>> >>>> thanks, >>>> liubo >>>> >>>>> dmesg after boot before mount -o recovery: >>>>> [ 51.829352] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 51.841153] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 51.841603] btrfs read error corrected: ino 1 off 5468060241920 >>>>> (dev >>>>> /dev/sdb sector 2143292648) >>>>> [ 51.841610] Failed to read block groups: -5 >>>>> [ 51.848057] btrfs: open_ctree failed >>>>> .............................. >>>>> dmesg after both mounts: >>>>> >>>>> [ 123.687773] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 >>>>> devid 5 >>>>> transid 9096 /dev/sdf >>>>> [ 123.733678] btrfs: use lzo compression >>>>> [ 123.733683] btrfs: enabling auto recovery >>>>> [ 123.733686] btrfs: disk space caching is enabled >>>>> [ 131.699910] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 131.714018] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 131.715059] btrfs read error corrected: ino 1 off 5468060241920 >>>>> (dev >>>>> /dev/sdb sector 2143292648) >>>>> [ 131.715072] Failed to read block groups: -5 >>>>> [ 131.727176] btrfs: open_ctree failed >>>>> [ 161.697873] device fsid c9776e19-37eb-4f9c-bd6b-04e8dde97682 >>>>> devid 5 >>>>> transid 9096 /dev/sdf >>>>> [ 161.746345] btrfs: use lzo compression >>>>> [ 161.746354] btrfs: enabling auto recovery >>>>> [ 161.746358] btrfs: disk space caching is enabled >>>>> [ 169.720823] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 169.732048] parent transid verify failed on 5468060241920 >>>>> wanted 9096 >>>>> found 7621 >>>>> [ 169.732611] btrfs read error corrected: ino 1 off 5468060241920 >>>>> (dev >>>>> /dev/sdb sector 2143292648) >>>>> [ 169.732623] Failed to read block groups: -5 >>>>> [ 169.743437] btrfs: open_ctree failed >>>>> >>>>> So It does not work. I have seen in some posts command: >>>>> >>>>> sudo mount -s 2 -o recovery /tank >>>>> Should I try it? >>>>> >>>>> Please help me, I need to get this data ASAP. >>>>> >>>>> Regards, >>>>> Max >>>>> >>>>> On 06/03/2012 09:22 PM, Liu Bo wrote: >>>>>> On 06/02/2012 09:43 PM, Maxim Mikheev wrote: >>>>>> >>>>>>> Repair was not helpful. >>>>>>> Is any other ways to get access to data? >>>>>>> >>>>>>> Please help.... >>>>>>> >>>>>> Hi Maxim, >>>>>> >>>>>> Besides btrfsck --repair, we also have a recovery mount option to >>>>>> deal >>>>>> with your situation, >>>>>> maybe you can try mount xxx -o recovery and see if it helps? >>>>>> >>>>>> >>>>>> thanks, >>>>>> liubo >>>>>> >>>>>>> On 05/30/2012 11:15 PM, Michael K wrote: >>>>>>>> Let it run to completion. There is little you can do other than >>>>>>>> hope >>>>>>>> and wait. >>>>>>>> >>>>>>>> On May 30, 2012 9:02 PM, "Maxim Mikheev">>>>>>> > wrote: >>>>>>>> >>>>>>>> btrfsck --repair running already for 26 hours. >>>>>>>> >>>>>>>> Is it have sense to wait more? >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> On 05/29/2012 07:36 PM, cwillu wrote: >>>>>>>> >>>>>>>> On Tue, May 29, 2012 at 5:24 PM, Maxim >>>>>>>> Mikheev> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Thank you for your answer. >>>>>>>> >>>>>>>> >>>>>>>> The system kernel was and now: >>>>>>>> >>>>>>>> Linux s0 3.4.0-030400-generic #201205210521 SMP Mon >>>>>>>> May 21 >>>>>>>> 09:22:02 UTC 2012 >>>>>>>> x86_64 x86_64 x86_64 GNU/Linux >>>>>>>> >>>>>>>> the raid was created by: >>>>>>>> mkfs.btrfs /dev/sdb /dev/sdc /dev/sdd /dev/sde >>>>>>>> /dev/sdf >>>>>>>> >>>>>>>> Disk are connected through RocketRaid 2670. >>>>>>>> >>>>>>>> for mounting I used line in fstab: >>>>>>>> UUID=c9776e19-37eb-4f9c-bd6b-04e8dde97682 /tank >>>>>>>> btrfs >>>>>>>> defaults,compress=lzo 0 1 >>>>>>>> >>>>>>>> On machine was running several Virtual machines. >>>>>>>> Only one >>>>>>>> was actively using >>>>>>>> disks. >>>>>>>> >>>>>>>> VM has active several threads: >>>>>>>> 1. 2 threads reading big files (50GB each) >>>>>>>> 2. reading from 50 files and writing one big file >>>>>>>> 3. The kernel panic happens when I run another >>>>>>>> program >>>>>>>> with 30 threads of >>>>>>>> reading/writing of small files. >>>>>>>> >>>>>>>> Virtual Machine accessed to underline btrfs >>>>>>>> through 9-p >>>>>>>> file system which >>>>>>>> actively used xattr. >>>>>>>> >>>>>>>> After reboot system was in this stage. >>>>>>>> >>>>>>>> I hope that btrfsck --repair will not make it >>>>>>>> worse, >>>>>>>> It is >>>>>>>> now running. >>>>>>>> >>>>>>>> **twitch** >>>>>>>> >>>>>>>> Well, I also hope it won't make it worse. Do not >>>>>>>> cancel it >>>>>>>> now, let >>>>>>>> it finish (aborting it will make things worse), but I >>>>>>>> suggest >>>>>>>> waiting >>>>>>>> until a few more people have weighed in before >>>>>>>> attempting >>>>>>>> anything >>>>>>>> beyond that. >>>>>>>> >>>>>>>> -- >>>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>> linux-btrfs" in >>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>> >>>>>>>> More majordomo info at >>>>>>>> http://vger.kernel.org/majordomo-info.html >>>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>> linux-btrfs" in >>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe >>>>> linux-btrfs" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe >>> linux-btrfs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>