* Recover file after truncate
@ 2017-07-26 0:48 Andy Bennett
2017-07-26 11:15 ` Brian Foster
0 siblings, 1 reply; 7+ messages in thread
From: Andy Bennett @ 2017-07-26 0:48 UTC (permalink / raw)
To: linux-xfs
Hi,
I have a file (~6GiB) that I was doing some analysis on. In the course of
this I managed to do 'xxd something-else my-file' and overwrote it with <
4.0K of data.
Running xfs_bmap on it gives me:
-----
my-file:
0: [0..7]: 236935320..236935327
-----
Running xfs_bmap on another, similar big, file that was created around the
same time gives me:
-----
other-file:
0: [0..4194175]: 335806496..340000671
1: [4194176..13828055]: 342870632..352504511
-----
A quick experiment with xxd suggests that the inode of the file remains the
same if it exists before xxd writes to it. So my new file still has the
same inode as the one I want to recover.
There hasn't been any write activity on this file system since I made this
mistake.
Is there any hope of recovering any part of the original data?
I have an xxd dump of the first 624 bytes of the original file and there
are some recurring features in it.
Grepping through the partition for that signature gives me this:
-----
$ sudo grep -obUaP "\x66\x66\xa2" /dev/nvme0n1p8 |tee TRACE
grep: exceeded PCRE's line length limit
519554640:ff¢
4377654787:ff¢
7961215381:ff¢
10165641473:ff¢
10849981825:ff¢
17851384491:ff¢
23231901998:ff¢
33898050969:ff¢
41781142596:ff¢
51651699392:ff¢
56040569029:ff¢
56277711167:ff¢
56814897544:ff¢
61037797435:ff¢
61269592210:ff¢
73946170693:ff¢
75199462354:ff¢
76071192135:ff¢
-----
The partition is on an NVMe SSD.
I'm not sure how to use xfs_logprint but converting the inode number to hex
and grepping through doesn't seem to give me any matches.
Is there a way to find out how this file was allocated and restore it or
fish the data out some how?
My mount options are
-----
/dev/nvme0n1p8 on /var/spool type xfs (rw,relatime,attr2,inode64,noquota)
-----
...and I'm running under Debian Jessie.
I've not (knowingly) got any manual TRIM or DISCARD jobs that will run.
What's the best way of working out what current files the offsets from grep
correspond to? I guess the ones that don't correspond to a current file
might be my data?
Thanks for anything you can do to help.
Regards,
@ndy
--
andyjpb@ashurst.eu.org
http://www.ashurst.eu.org/
0x7EBA75FF
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: Recover file after truncate 2017-07-26 0:48 Recover file after truncate Andy Bennett @ 2017-07-26 11:15 ` Brian Foster 2017-07-26 22:12 ` Andy Bennett 0 siblings, 1 reply; 7+ messages in thread From: Brian Foster @ 2017-07-26 11:15 UTC (permalink / raw) To: Andy Bennett; +Cc: linux-xfs On Wed, Jul 26, 2017 at 01:48:59AM +0100, Andy Bennett wrote: > Hi, > > I have a file (~6GiB) that I was doing some analysis on. In the course of > this I managed to do 'xxd something-else my-file' and overwrote it with < > 4.0K of data. > This basically truncated the file and allocated new blocks for the xxd output. > > Running xfs_bmap on it gives me: > > ----- > my-file: > 0: [0..7]: 236935320..236935327 > ----- > > Running xfs_bmap on another, similar big, file that was created around the > same time gives me: > > ----- > other-file: > 0: [0..4194175]: 335806496..340000671 > 1: [4194176..13828055]: 342870632..352504511 A ~2GB extent followed by a ~4.5GB extent. > ----- > > > A quick experiment with xxd suggests that the inode of the file remains the > same if it exists before xxd writes to it. So my new file still has the same > inode as the one I want to recover. > > There hasn't been any write activity on this file system since I made this > mistake. > > > Is there any hope of recovering any part of the original data? > I suspect that most of your file data currently sits on free blocks in the filesystem. If you had a bmap from before this incident, you could probably point to exactly where they are. Since I assume that is unlikely, you'd have to run something like you have below to try and identify where that data is in the block device and try to piece together the starting block and length of the associated extents. I suppose that if you can distinguish the file data in any particular 4k block from other random data then it might be possible to identify the exact start/length values of the extents directly from the block device. If you are luckier still and the original file was allocated with only one or two extents similar to the above example, then you may be able to piece together the file by putting the extents in the appropriate order. FWIW, there are tools such as photorec that supposedly are able to do this kind of thing with known file formats. Note that I don't think there's any way to reassign the blocks to the inode without doing some form of manual surgery on the filesystem, which I would not recommend. If you actually were able to locate the file data on the raw block device, the best course of action is to copy the data from the block device to a new file on a separate filesystem and then copy that file back into the original fs. Also note that once you start writing anything to the original fs, the original data is at risk of being overwritten. You may want to decommission the original storage or make a copy to preserve the original state. > I have an xxd dump of the first 624 bytes of the original file and there are > some recurring features in it. > > Grepping through the partition for that signature gives me this: > > ----- > $ sudo grep -obUaP "\x66\x66\xa2" /dev/nvme0n1p8 |tee TRACE > grep: exceeded PCRE's line length limit > 519554640:ff¢ > 4377654787:ff¢ > 7961215381:ff¢ > 10165641473:ff¢ > 10849981825:ff¢ > 17851384491:ff¢ > 23231901998:ff¢ > 33898050969:ff¢ > 41781142596:ff¢ > 51651699392:ff¢ > 56040569029:ff¢ > 56277711167:ff¢ > 56814897544:ff¢ > 61037797435:ff¢ > 61269592210:ff¢ > 73946170693:ff¢ > 75199462354:ff¢ > 76071192135:ff¢ > ----- > > > The partition is on an NVMe SSD. > > > I'm not sure how to use xfs_logprint but converting the inode number to hex > and grepping through doesn't seem to give me any matches. > What does the xfs_logprint output look like? Also, what is the inode number of the original file? It is possible to get a hint on the startblock and length of the original extent(s) if the log still has EFI/EFD items present that describe the extent free operations, and that information can be corroborated against either the inode information and/or with the regions where file data is found in the blockdev scan. Do note that the simple act of mounting the filesystem runs the risk of overwriting previous log data. An 'xfs_metadump -go' might be a good way to preserve current log content. Brian > > > Is there a way to find out how this file was allocated and restore it or > fish the data out some how? > > My mount options are > > ----- > /dev/nvme0n1p8 on /var/spool type xfs (rw,relatime,attr2,inode64,noquota) > ----- > > ...and I'm running under Debian Jessie. > > I've not (knowingly) got any manual TRIM or DISCARD jobs that will run. > > > > What's the best way of working out what current files the offsets from grep > correspond to? I guess the ones that don't correspond to a current file > might be my data? > > > > Thanks for anything you can do to help. > > > > > Regards, > @ndy > > -- > andyjpb@ashurst.eu.org > http://www.ashurst.eu.org/ > 0x7EBA75FF > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Recover file after truncate 2017-07-26 11:15 ` Brian Foster @ 2017-07-26 22:12 ` Andy Bennett 2017-07-27 12:43 ` Brian Foster 0 siblings, 1 reply; 7+ messages in thread From: Andy Bennett @ 2017-07-26 22:12 UTC (permalink / raw) To: Brian Foster; +Cc: linux-xfs Hi, >> A quick experiment with xxd suggests that the inode of the >> file remains the >> same if it exists before xxd writes to it. So my new file >> still has the same >> inode as the one I want to recover. >> >> There hasn't been any write activity on this file system since >> I made this ... > > I suspect that most of your file data currently sits on free blocks in > the filesystem. If you had a bmap from before this incident, you could > probably point to exactly where they are. Since I assume that is > unlikely, Yeah... I captured the file and subsequently managed to delete it before I had a chance to run a backup! :-( > you'd have to run something like you have below to try and > identify where that data is in the block device and try to piece > together the starting block and length of the associated extents. > > I suppose that if you can distinguish the file data in any particular 4k > block from other random data then it might be possible to identify the > exact start/length values of the extents directly from the block device. The actual data is IQ samples that I've captured from a device I hired but have now sent back. ...so it's essentially random binary data and I wouldn't like to try to write a classifier for it but I can easily pipe it through my toolchain without header or framing woes and see if it produces output that is pleasing. So, to that extent, if I can get in the general vicinity of the file I can just chop the edges around until I'm happy. Hey, my 6 minute sample was a little over 6GiB so I'm sure I could just listen to the whole block device in a few hours. :-) > If you are luckier still and the original file was allocated with only > one or two extents similar to the above example, then you may be able to > piece together the file by putting the extents in the appropriate order. > FWIW, there are tools such as photorec that supposedly are able to do > this kind of thing with known file formats. I'm hoping it's in a couple of extents. How do I identify extents on disk? > Note that I don't think there's any way to reassign the blocks to the > inode without doing some form of manual surgery on the filesystem, which > I would not recommend. If you actually were able to locate the file data > on the raw block device, the best course of action is to copy the data > from the block device to a new file on a separate filesystem and then > copy that file back into the original fs. That sounds good, thanks. > Also note that once you start > writing anything to the original fs, the original data is at risk of > being overwritten. You may want to decommission the original storage or > make a copy to preserve the original state. > What does the xfs_logprint output look like? Piping `xfs_logprint /dev/nvme0n1p8 ` thru' tail gives me this: ----- Oper (13): tid: 4d4486ed len: 16 clientid: TRANS flags: none EXTENTS inode data ---------------------------------------------------------------------------- Oper (14): tid: 4d4486ed len: 0 clientid: TRANS flags: COMMIT ============================================================================ cycle: 1 version: 2 lsn: 1,211073 tail_lsn: 1,211069 length of Log Record: 512 prev offset: 211069 num ops: 5 uuid: 7274488e-64ab-4d2a-8f4d-cedec4db7e6f format: little endian linux h_size: 32768 ---------------------------------------------------------------------------- Oper (0): tid: 94e15a8b len: 0 clientid: TRANS flags: START ---------------------------------------------------------------------------- Oper (1): tid: 94e15a8b len: 16 clientid: TRANS flags: none TRAN: type: SWAPEXT tid: 94e15a8b num_items: 2 ---------------------------------------------------------------------------- Oper (2): tid: 94e15a8b len: 24 clientid: TRANS flags: none BUF: #regs: 2 start blkno: 0 (0x0) len: 1 bmap size: 1 flags: 0x9000 Oper (3): tid: 94e15a8b len: 384 clientid: TRANS flags: none SUPER BLOCK Buffer: icount: 6360863066640355328 ifree: 94813521 fdblks: 0 frext: 0 ---------------------------------------------------------------------------- Oper (4): tid: 94e15a8b len: 0 clientid: TRANS flags: COMMIT ============================================================================ cycle: 1 version: 2 lsn: 1,211075 tail_lsn: 1,211073 length of Log Record: 512 prev offset: 211073 num ops: 5 uuid: 7274488e-64ab-4d2a-8f4d-cedec4db7e6f format: little endian linux h_size: 32768 ---------------------------------------------------------------------------- Oper (0): tid: 9f4540d0 len: 0 clientid: TRANS flags: START ---------------------------------------------------------------------------- Oper (1): tid: 9f4540d0 len: 16 clientid: TRANS flags: none TRAN: type: SWAPEXT tid: 9f4540d0 num_items: 2 ---------------------------------------------------------------------------- Oper (2): tid: 9f4540d0 len: 24 clientid: TRANS flags: none BUF: #regs: 2 start blkno: 0 (0x0) len: 1 bmap size: 1 flags: 0x9000 Oper (3): tid: 9f4540d0 len: 384 clientid: TRANS flags: none SUPER BLOCK Buffer: icount: 6360863066640355328 ifree: 94813521 fdblks: 0 frext: 0 ---------------------------------------------------------------------------- Oper (4): tid: 9f4540d0 len: 0 clientid: TRANS flags: COMMIT ============================================================================ xfs_logprint: skipped 159283 zeroed blocks in range: 211077 - 370359 xfs_logprint: physical end of log ============================================================================ xfs_logprint: logical end of log ============================================================================ ----- The whole log is currently 4,044,259 lines long. > Also, what is the inode > number of the original file? 314798163 > It is possible to get a hint on the > startblock and length of the original extent(s) if the log still has > EFI/EFD items present that describe the extent free operations, and that > information can be corroborated against either the inode information > and/or with the regions where file data is found in the blockdev scan. Piping `xfs_logprint /dev/nvme0n1p8 ` thru' less, seeking to the end and searching backwards for 'EFI' gives me this: ----- ============================================================================ cycle: 1 version: 2 lsn: 1,211069 tail_lsn: 1,211067 length of Log Record: 1536 prev offset: 211067 num ops: 15 uuid: 7274488e-64ab-4d2a-8f4d-cedec4db7e6f format: little endian linux h_size: 32768 ---------------------------------------------------------------------------- Oper (0): tid: 4d4486ed len: 0 clientid: TRANS flags: START ---------------------------------------------------------------------------- Oper (1): tid: 4d4486ed len: 16 clientid: TRANS flags: none TRAN: type: SWAPEXT tid: 4d4486ed num_items: 12 ---------------------------------------------------------------------------- Oper (2): tid: 4d4486ed len: 32 clientid: TRANS flags: none EFI: #regs: 1 num_extents: 1 id: 0xffff8bff07e21e58 (s: 0x114112b, l: 1363) ---------------------------------------------------------------------------- Oper (3): tid: 4d4486ed len: 32 clientid: TRANS flags: none EFD: #regs: 1 num_extents: 1 id: 0xffff8bff07e21e58 ---------------------------------------------------------------------------- Oper (4): tid: 4d4486ed len: 24 clientid: TRANS flags: none BUF: #regs: 2 start blkno: 1 (0x1) len: 1 bmap size: 1 flags: 0x2800 Oper (5): tid: 4d4486ed len: 128 clientid: TRANS flags: none AGF Buffer: XAGF Out of space ---------------------------------------------------------------------------- Oper (6): tid: 4d4486ed len: 24 clientid: TRANS flags: none BUF: #regs: 2 start blkno: 8 (0x8) len: 8 bmap size: 1 flags: 0x2000 Oper (7): tid: 4d4486ed len: 128 clientid: TRANS flags: none BUF DATA ---------------------------------------------------------------------------- Oper (8): tid: 4d4486ed len: 24 clientid: TRANS flags: none BUF: #regs: 3 start blkno: 16 (0x10) len: 8 bmap size: 1 flags: 0x2000 Oper (9): tid: 4d4486ed len: 128 clientid: TRANS flags: none BUF DATA Oper (10): tid: 4d4486ed len: 128 clientid: TRANS flags: none BUF DATA ---------------------------------------------------------------------------- Oper (11): tid: 4d4486ed len: 56 clientid: TRANS flags: none INODE: #regs: 3 ino: 0x89c95dd flags: 0x5 dsize: 16 blkno: 144479680 len: 32 boff: 14848 Oper (12): tid: 4d4486ed len: 176 clientid: TRANS flags: none INODE CORE magic 0x494e mode 0100600 version 3 format 2 nlink 1 uid 501 gid 100 atime 0x59790f5b mtime 0x59790f6d ctime 0x59790f6d size 0x3a9c161 nblocks 0x3a9d extsize 0x0 nextents 0x1 naextents 0x0 forkoff 0 dmevmask 0x0 dmstate 0x0 flags 0x0 gen 0x51eeced9 Oper (13): tid: 4d4486ed len: 16 clientid: TRANS flags: none EXTENTS inode data ---------------------------------------------------------------------------- Oper (14): tid: 4d4486ed len: 0 clientid: TRANS flags: COMMIT ----- > Do note that the simple act of mounting the filesystem runs the risk of > overwriting previous log data. An 'xfs_metadump -go' might be a good way > to preserve current log content. I've been hibernating the machine and haven't tried remounting it read-only: I'm not expecting anything to do any serious writes on that partition and I don't think there will be anything much at all that tries to write there. > Brian Thanks for your help! Regards, @ndy -- andyjpb@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Recover file after truncate 2017-07-26 22:12 ` Andy Bennett @ 2017-07-27 12:43 ` Brian Foster 2017-07-27 15:04 ` Brian Foster 2017-07-30 14:41 ` Andy Bennett 0 siblings, 2 replies; 7+ messages in thread From: Brian Foster @ 2017-07-27 12:43 UTC (permalink / raw) To: Andy Bennett; +Cc: linux-xfs On Wed, Jul 26, 2017 at 11:12:19PM +0100, Andy Bennett wrote: > Hi, > > ... > > > > If you are luckier still and the original file was allocated with only > > one or two extents similar to the above example, then you may be able to > > piece together the file by putting the extents in the appropriate order. > > FWIW, there are tools such as photorec that supposedly are able to do > > this kind of thing with known file formats. > > I'm hoping it's in a couple of extents. How do I identify extents on disk? > File extents can be listed via userspace tools like xfs_bmap, as you've shown previously. Once blocks are freed from a file, they are returned to per allocation group btree data structures on disk. Note that the extents in the btree may no longer represent the exact extents allocated to your file because other, contiguous free space could have been merged to form larger free extents. The free space btree can be traversed offline via the xfs_db utility. Something like the following could give you the free extent information for each AG in the fs. for i in $(seq 0 <maxag>); do echo agno $i xfs_db -c "agf $i" -c "addr bnoroot" -c "btdump" <dev> done You can obtain the AG count of the fs via 'xfs_info <mnt>' or: xfs_db -c "sb 0" -c "p agcount" <dev> If I were you, I would probably take the following high level approach to attempt a recovery: - grep/search through the block device for blocks that appear to be associated with your file. - For each identified offset in the raw blockdev, convert the 4k block address to an AG number and AG relative block: xfs_db -c "convert daddr <block> agno" -c "convert daddr <block> agbno" <dev> - Attempt to locate a free extent that covers that block in the free extent output for the associated AG number. Note that all of the btump output collected above is in the form of AG relative block offsets, so each AG has a block 0, 1, etc. - If found, convert the starting agblock of the free extent back to a disk address: xfs_db -c "convert agno N agbno B daddr" <dev> - Based on the size of the free extent, dd that number of blocks off the raw block device to a file on a separate device: dd if=<dev> bs=4k skip=<freeblockaddr> count=<count> of=extentfile.N Verify the file contains the data you expect based on the initial search. - Repeat until you've copied each candidate extent to its own file on the separate fs. Then try to piece the individual extent files together into a new file that resembles the original file, combining them however necessary and/or trimming the extents if the boundaries seem to include unrelated data, etc. > ... > > What does the xfs_logprint output look like? > ... > Piping `xfs_logprint /dev/nvme0n1p8 ` thru' less, seeking to the end and > searching backwards for 'EFI' gives me this: > > ----- > ============================================================================ > cycle: 1 version: 2 lsn: 1,211069 tail_lsn: 1,211067 > length of Log Record: 1536 prev offset: 211067 num ops: 15 > uuid: 7274488e-64ab-4d2a-8f4d-cedec4db7e6f format: little endian linux > h_size: 32768 > ---------------------------------------------------------------------------- ... > ---------------------------------------------------------------------------- > Oper (2): tid: 4d4486ed len: 32 clientid: TRANS flags: none > EFI: #regs: 1 num_extents: 1 id: 0xffff8bff07e21e58 > (s: 0x114112b, l: 1363) A 1363 block extent == ~5MB, so probably not a candidate unless your file was more fragmented than anticipated. Are you filtering the xfs_logprint output here? If so, please attach the full output from xfs_logprint. Brian > ---------------------------------------------------------------------------- > Oper (3): tid: 4d4486ed len: 32 clientid: TRANS flags: none > EFD: #regs: 1 num_extents: 1 id: 0xffff8bff07e21e58 > ---------------------------------------------------------------------------- > Oper (4): tid: 4d4486ed len: 24 clientid: TRANS flags: none > BUF: #regs: 2 start blkno: 1 (0x1) len: 1 bmap size: 1 flags: 0x2800 > Oper (5): tid: 4d4486ed len: 128 clientid: TRANS flags: none > AGF Buffer: XAGF Out of space > ---------------------------------------------------------------------------- > Oper (6): tid: 4d4486ed len: 24 clientid: TRANS flags: none > BUF: #regs: 2 start blkno: 8 (0x8) len: 8 bmap size: 1 flags: 0x2000 > Oper (7): tid: 4d4486ed len: 128 clientid: TRANS flags: none > BUF DATA > ---------------------------------------------------------------------------- > Oper (8): tid: 4d4486ed len: 24 clientid: TRANS flags: none > BUF: #regs: 3 start blkno: 16 (0x10) len: 8 bmap size: 1 flags: 0x2000 > Oper (9): tid: 4d4486ed len: 128 clientid: TRANS flags: none > BUF DATA > Oper (10): tid: 4d4486ed len: 128 clientid: TRANS flags: none > BUF DATA > ---------------------------------------------------------------------------- > Oper (11): tid: 4d4486ed len: 56 clientid: TRANS flags: none > INODE: #regs: 3 ino: 0x89c95dd flags: 0x5 dsize: 16 > blkno: 144479680 len: 32 boff: 14848 > Oper (12): tid: 4d4486ed len: 176 clientid: TRANS flags: none > INODE CORE > magic 0x494e mode 0100600 version 3 format 2 > nlink 1 uid 501 gid 100 > atime 0x59790f5b mtime 0x59790f6d ctime 0x59790f6d > size 0x3a9c161 nblocks 0x3a9d extsize 0x0 nextents 0x1 > naextents 0x0 forkoff 0 dmevmask 0x0 dmstate 0x0 > flags 0x0 gen 0x51eeced9 > Oper (13): tid: 4d4486ed len: 16 clientid: TRANS flags: none > EXTENTS inode data > ---------------------------------------------------------------------------- > Oper (14): tid: 4d4486ed len: 0 clientid: TRANS flags: COMMIT ----- > > > > Do note that the simple act of mounting the filesystem runs the risk of > > overwriting previous log data. An 'xfs_metadump -go' might be a good way > > to preserve current log content. > > I've been hibernating the machine and haven't tried remounting it read-only: > I'm not expecting anything to do any serious writes on that partition and I > don't think there will be anything much at all that tries to write there. > > > > Brian > > Thanks for your help! > > > > > Regards, > @ndy > > -- > andyjpb@ashurst.eu.org > http://www.ashurst.eu.org/ > 0x7EBA75FF > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Recover file after truncate 2017-07-27 12:43 ` Brian Foster @ 2017-07-27 15:04 ` Brian Foster 2017-07-30 14:36 ` Andy Bennett 2017-07-30 14:41 ` Andy Bennett 1 sibling, 1 reply; 7+ messages in thread From: Brian Foster @ 2017-07-27 15:04 UTC (permalink / raw) To: Andy Bennett; +Cc: linux-xfs On Thu, Jul 27, 2017 at 08:43:50AM -0400, Brian Foster wrote: > On Wed, Jul 26, 2017 at 11:12:19PM +0100, Andy Bennett wrote: > > Hi, > > > > ... > > Are you filtering the xfs_logprint output here? If so, please attach the > full output from xfs_logprint. > Andy sent the logprint output privately due to the size of the attachment. Andy, The last reference I see of your inode in the log before the size goes to zero is this: ---------------------------------------------------------------------------- Oper (2): tid: c6720129 len: 56 clientid: TRANS flags: none INODE: #regs: 2 ino: 0x12c37053 flags: 0x1 dsize: 0 blkno: 235989736 len: 32 boff: 9728 Oper (3): tid: c6720129 len: 176 clientid: TRANS flags: none INODE CORE magic 0x494e mode 0100644 version 3 format 2 nlink 1 uid 501 gid 100 atime 0x5977c96e mtime 0x5973ac16 ctime 0x5973ac16 size 0x1a81f2218 nblocks 0x1a81f3 extsize 0x0 nextents 0x2 naextents 0x0 forkoff 0 dmevmask 0x0 dmstate 0x0 flags 0x0 gen 0xb8791e20 ---------------------------------------------------------------------------- This suggests the file size was 0x1a81f2218 (7115579928) and it had 0x1a81f3 (1737203) blocks across two extents (roughly 6.6GB). Note that the file size is not exactly aligned to the (blockcount * 4096). Further along, the following EFI exists: ---------------------------------------------------------------------------- Oper (19): tid: 6a85c7fc len: 48 clientid: TRANS flags: none EFI: #regs: 1 num_extents: 2 id: 0xffff8bfcb4f38990 (s: 0x12a9233, l: 688643) (s: 0x34ae0d5, l: 1048560) ---------------------------------------------------------------------------- ... which frees two extents that add up to 688643 + 1048560 = 1737203 (0x1a81f3) blocks. The starting filesystem blocks for these extents are 0x12a9233 and 0x34ae0d5, respectively. If you can convert those starting fsb values to agno/agbno combinations as shown in my previous mail, you could confirm whether they reside in the free space btrees for the associated allocation group of each extent. If so, you could convert each start fsb to a raw daddr (xfs_db -c "convert fsb 0x12a9233 daddr" <dev>) and use that to try and read each extent off the raw device as previously discussed. I have no idea whether those extents contain the file data you're after (note that I'm not sure we've actually confirmed your file data would have made it to disk), but if the file size values look sane that might be worth a try before you go fishing the device for raw data. Brian > Brian > > > ---------------------------------------------------------------------------- > > Oper (3): tid: 4d4486ed len: 32 clientid: TRANS flags: none > > EFD: #regs: 1 num_extents: 1 id: 0xffff8bff07e21e58 > > ---------------------------------------------------------------------------- > > Oper (4): tid: 4d4486ed len: 24 clientid: TRANS flags: none > > BUF: #regs: 2 start blkno: 1 (0x1) len: 1 bmap size: 1 flags: 0x2800 > > Oper (5): tid: 4d4486ed len: 128 clientid: TRANS flags: none > > AGF Buffer: XAGF Out of space > > ---------------------------------------------------------------------------- > > Oper (6): tid: 4d4486ed len: 24 clientid: TRANS flags: none > > BUF: #regs: 2 start blkno: 8 (0x8) len: 8 bmap size: 1 flags: 0x2000 > > Oper (7): tid: 4d4486ed len: 128 clientid: TRANS flags: none > > BUF DATA > > ---------------------------------------------------------------------------- > > Oper (8): tid: 4d4486ed len: 24 clientid: TRANS flags: none > > BUF: #regs: 3 start blkno: 16 (0x10) len: 8 bmap size: 1 flags: 0x2000 > > Oper (9): tid: 4d4486ed len: 128 clientid: TRANS flags: none > > BUF DATA > > Oper (10): tid: 4d4486ed len: 128 clientid: TRANS flags: none > > BUF DATA > > ---------------------------------------------------------------------------- > > Oper (11): tid: 4d4486ed len: 56 clientid: TRANS flags: none > > INODE: #regs: 3 ino: 0x89c95dd flags: 0x5 dsize: 16 > > blkno: 144479680 len: 32 boff: 14848 > > Oper (12): tid: 4d4486ed len: 176 clientid: TRANS flags: none > > INODE CORE > > magic 0x494e mode 0100600 version 3 format 2 > > nlink 1 uid 501 gid 100 > > atime 0x59790f5b mtime 0x59790f6d ctime 0x59790f6d > > size 0x3a9c161 nblocks 0x3a9d extsize 0x0 nextents 0x1 > > naextents 0x0 forkoff 0 dmevmask 0x0 dmstate 0x0 > > flags 0x0 gen 0x51eeced9 > > Oper (13): tid: 4d4486ed len: 16 clientid: TRANS flags: none > > EXTENTS inode data > > ---------------------------------------------------------------------------- > > Oper (14): tid: 4d4486ed len: 0 clientid: TRANS flags: COMMIT ----- > > > > > > > Do note that the simple act of mounting the filesystem runs the risk of > > > overwriting previous log data. An 'xfs_metadump -go' might be a good way > > > to preserve current log content. > > > > I've been hibernating the machine and haven't tried remounting it read-only: > > I'm not expecting anything to do any serious writes on that partition and I > > don't think there will be anything much at all that tries to write there. > > > > > > > Brian > > > > Thanks for your help! > > > > > > > > > > Regards, > > @ndy > > > > -- > > andyjpb@ashurst.eu.org > > http://www.ashurst.eu.org/ > > 0x7EBA75FF > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Recover file after truncate 2017-07-27 15:04 ` Brian Foster @ 2017-07-30 14:36 ` Andy Bennett 0 siblings, 0 replies; 7+ messages in thread From: Andy Bennett @ 2017-07-30 14:36 UTC (permalink / raw) To: Brian Foster; +Cc: linux-xfs Hi, > The last reference I see of your inode in the log before the size goes > to zero is this: > > ---------------------------------------------------------------------------- > > Oper (2): tid: c6720129 len: 56 clientid: TRANS flags: none > > INODE: #regs: 2 ino: 0x12c37053 flags: 0x1 dsize: 0 > blkno: 235989736 len: 32 boff: 9728 > Oper (3): tid: c6720129 len: 176 clientid: TRANS flags: none > > INODE CORE > magic 0x494e mode 0100644 version 3 format 2 > nlink 1 uid 501 gid 100 > atime 0x5977c96e mtime 0x5973ac16 ctime 0x5973ac16 > size 0x1a81f2218 nblocks 0x1a81f3 extsize 0x0 nextents 0x2 > naextents 0x0 forkoff 0 dmevmask 0x0 dmstate 0x0 > flags 0x0 gen 0xb8791e20 > ---------------------------------------------------------------------------- > > This suggests the file size was 0x1a81f2218 (7115579928) and it had > 0x1a81f3 (1737203) blocks across two extents (roughly 6.6GB). Note that > the file size is not exactly aligned to the (blockcount * 4096). > > Further along, the following EFI exists: > > ---------------------------------------------------------------------------- > Oper (19): tid: 6a85c7fc len: 48 clientid: TRANS flags: none > EFI: #regs: 1 num_extents: 2 id: 0xffff8bfcb4f38990 > (s: 0x12a9233, l: 688643) (s: 0x34ae0d5, l: 1048560) > ---------------------------------------------------------------------------- > > ... which frees two extents that add up to 688643 + 1048560 = 1737203 > (0x1a81f3) blocks. The starting filesystem blocks for these extents are > 0x12a9233 and 0x34ae0d5, respectively. > > If you can convert those starting fsb values to agno/agbno combinations > as shown in my previous mail, you could confirm whether they reside in > the free space btrees for the associated allocation group of each > extent. Here are the translations for those extents: fsb 0x12a9233 agno 0 agbno 0x12a9233 (19567155) daddr 0x9549198 (156537240) fsb 0x34ae0d5 agno 1 agno 0x14ae0d5 (21684437) daddr 0x15a48150 (363102544) xfs_db's 'freesp -d' output contains: ----- ... 0 19566515 639 0 19567155 1396803 0 20963959 408468 ... 1 20621811 106501 1 21684437 1104130 1 22788568 421464 ... ----- Both chunks of free space appear to have more blocks that the original extents. > If so, you could convert each start fsb to a raw daddr (xfs_db > -c "convert fsb 0x12a9233 daddr" <dev>) and use that to try and read > each extent off the raw device as previously discussed. 156537240 is in 512 byte blocks which is 19567155 in 4096 byte blocks. dd if=/dev/nvme0n1p8 bs=4096 skip=19567155 count=688643 of=extentfile.0 363102544 is in 512 byte blocks which is 45387818 in 4096 byte blocks. dd if=/dev/nvme0n1p8 bs=4096 skip=45387818 count=1048560 of=extentfile.1 These files both look good and 'cat extentfile.0 >> extentfile.1' gives me back the original order of the file. ----- $ wc -c extentfile.1 7115583488 extentfile.1 $ du -hs extentfile.1 6.7G extentfile.1 ----- ...which seems slightly larger than the original file as reported in the 'INODE CORE' message that you found in the log. The recovered file ends up being some 53 extents long; probably because the filesystem I recovered it too was rather full. However, when I run it through my tool chain, I seem to get exactly what I expect. > I have no idea whether those extents contain the file data you're after > (note that I'm not sure we've actually confirmed your file data would > have made it to disk), but if the file size values look sane that might > be worth a try before you go fishing the device for raw data. It seems that it did and we have been able to retrieve it! Thanks so much for your help. I really appreiciate it, especially the time you took to poor through my logs. Let me know if you're ever near London and I'll shout you dinner and drinks. Regards, @ndy -- andyjpb@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Recover file after truncate 2017-07-27 12:43 ` Brian Foster 2017-07-27 15:04 ` Brian Foster @ 2017-07-30 14:41 ` Andy Bennett 1 sibling, 0 replies; 7+ messages in thread From: Andy Bennett @ 2017-07-30 14:41 UTC (permalink / raw) To: Brian Foster; +Cc: linux-xfs Hi, > The free space btree can be traversed offline via the xfs_db utility. > Something like the following could give you the free extent information > for each AG in the fs. > > for i in $(seq 0 <maxag>); do > echo agno $i > xfs_db -c "agf $i" -c "addr bnoroot" -c "btdump" <dev> > done My xfs_db didn't have 'btdump' but I discovered 'freesp -d' which summarised the free regions for all AGs at once. I also needed to use 'xfs_db -r ...' to attach it to the filesystem as it was still mounted read-write. (I wasn't sure if remounting it or unmounting it and mounting it again would destroy the logs.) Thanks again for your help. Regards, @ndy -- andyjpb@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-07-30 14:41 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-07-26 0:48 Recover file after truncate Andy Bennett 2017-07-26 11:15 ` Brian Foster 2017-07-26 22:12 ` Andy Bennett 2017-07-27 12:43 ` Brian Foster 2017-07-27 15:04 ` Brian Foster 2017-07-30 14:36 ` Andy Bennett 2017-07-30 14:41 ` Andy Bennett
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox