Map a disk LBA to filename?

Map a disk LBA to filename?

[Carsten Aulbert]

Hi all,

I have not found it in the archive, thus my question to all:

Along the lines of this doc
http://smartmontools.sourceforge.net/BadBlockHowTo.txt

I would like to find out if that's also possible for XFS, i.e. I have a
disk which tells me, that there is a problem with LBA x and I identify
that this block is within the range of a largish XFS file system.

Now, I would like to identify the file which is possibly broken and
repair it if I can without taking the machine down for hours running the
manufacturers DOS tools. With simple ext[23] this is possible, but I'm
not getting anywhere with reading xfs_db and xfs_ncheck man pages to get
there.

Anyone here with a brief recipe I can get along with?

Cheers

Carsten

-- 
Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics
Callinstrasse 38, 30167 Hannover, Germany
Phone/Fax: +49 511 762-17185 / -17193
http://www.top500.org/system/9234 | http://www.top500.org/connfam/6/list/31

Re: Map a disk LBA to filename?

[Dave Chinner]

On Mon, Oct 27, 2008 at 12:20:27PM +0100, Carsten Aulbert wrote:
> Hi all,
> 
> I have not found it in the archive, thus my question to all:
> 
> Along the lines of this doc
> http://smartmontools.sourceforge.net/BadBlockHowTo.txt
> 
> I would like to find out if that's also possible for XFS, i.e. I have a
> disk which tells me, that there is a problem with LBA x and I identify
> that this block is within the range of a largish XFS file system.
> 
> Now, I would like to identify the file which is possibly broken and
> repair it if I can without taking the machine down for hours running the
> manufacturers DOS tools. With simple ext[23] this is possible, but I'm
> not getting anywhere with reading xfs_db and xfs_ncheck man pages to get
> there.
> 
> Anyone here with a brief recipe I can get along with?

Use xfs_bmap to find the location on disk of the extents in each
file. Recurse over the filesystem until you find the file that owns
the block that went bad.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Map a disk LBA to filename?

[Carsten Aulbert]

Hi Dave (and Peter)

Dave Chinner wrote:
> Use xfs_bmap to find the location on disk of the extents in each
> file. Recurse over the filesystem until you find the file that owns
> the block that went bad.

Sounds like a tedious but doable route to take.

Thanks for the quick reply!

Carsten

Re: Map a disk LBA to filename?

[Michal Soltys]

Carsten Aulbert wrote:
> 
> Dave Chinner wrote:
>> Use xfs_bmap to find the location on disk of the extents in each
>> file. Recurse over the filesystem until you find the file that owns
>> the block that went bad.
> 
> Sounds like a tedious but doable route to take.
> 

Wouldn't something like (under xfs_db) :

getblock -b #block -n
ncheck -i #inode

where required #inode is reported by getblock

do the thing ?

Re: Map a disk LBA to filename?

[Carsten Aulbert]

Hi Michal

Michal Soltys wrote:
> 
> Wouldn't something like (under xfs_db) :
> 
> getblock -b #block -n
> ncheck -i #inode
> 
> where required #inode is reported by getblock
> 
> do the thing ?
> 

Sounds nice, but my (ancient? 2.8.11) version of xfs_db does not know
getblock (only blockget) since that also matches the command line and
the usage looks about to be right, I'm currently trying that one, e.g.

smartctl reports a bad LBA at 36922326. fdisk tells me the partition
starts at 31069773, hence the block under question is 5852553.

xfs_info tells be a bsize of 4096 which I take as the block size, thus
the xfs block to look at should be 731569, right?

xfs_db> blockget -b 731569 -n
setting block 0/731569 to free1
setting block 0/731569 to free2
xfs_db>

hmm, no inode number. Does that mean this block is not used by any file
currently - which might be perfectly fine since this partition is only
31% full.

Everything right so far?

Cheers

Carsten

Re: Map a disk LBA to filename?

[Michal Soltys]

Carsten Aulbert wrote:
> Michal Soltys wrote:
>> 
>> Wouldn't something like (under xfs_db) :
>> 
>> getblock -b #block -n
>> ncheck -i #inode
>> 
>> where required #inode is reported by getblock
>> 
>> do the thing ?
>> 
> 
> Sounds nice, but my (ancient? 2.8.11) version of xfs_db does not know
> getblock (only blockget) since that also matches the command line and
> the usage looks about to be right, I'm currently trying that one, e.g.
> 

Sorry, I meant blockget.

> smartctl reports a bad LBA at 36922326. fdisk tells me the partition
> starts at 31069773, hence the block under question is 5852553.
> 
> xfs_info tells be a bsize of 4096 which I take as the block size, thus
> the xfs block to look at should be 731569, right?
> 
> xfs_db> blockget -b 731569 -n
> setting block 0/731569 to free1
> setting block 0/731569 to free2
> xfs_db>
> 
> hmm, no inode number. Does that mean this block is not used by any file
> currently - which might be perfectly fine since this partition is only
> 31% full.
> 
> Everything right so far?
> 

Yes, calculations look fine. I'm not sure what exactly is the difference 
between free1 and free2 - someone more knowledgable would have to 
comment (and overally comment about this method).

In my case when I check block allocated by some file, I get results such as:

xfs_db> blockget -b 4584722 -n
setting block 0/4584722 to data
setting inode to 36231069 for block 0/4584722
inode 36231069 block 4584722 at offset 810
sb_fdblocks 1330583, aggregate AGF count 7983498
xfs_db> ncheck -i 36231069
    36231069 stuff/data/arst.dat

and further:

xfs_db> inode 36231069
xfs_db> bmap
data offset 0 startblock 4583912 (0/4583912) count 832 flag 0

Here the file lies in the first extent, taking 832*4096 blocks.

Re: Map a disk LBA to filename?

[Dave Chinner]

On Mon, Oct 27, 2008 at 02:03:15PM +0100, Michal Soltys wrote:
> Carsten Aulbert wrote:
>>
>> Dave Chinner wrote:
>>> Use xfs_bmap to find the location on disk of the extents in each
>>> file. Recurse over the filesystem until you find the file that owns
>>> the block that went bad.
>>
>> Sounds like a tedious but doable route to take.
>>
>
> Wouldn't something like (under xfs_db) :
>
> getblock -b #block -n
> ncheck -i #inode
>
> where required #inode is reported by getblock
>
> do the thing ?

Blockget - yes it will. It just does the traversal internally to
build the mapping. With large filesystems xfs_db can run out of
memory building the mapping, which is why I've used the
explicit traverse+xfs_bmap method in the past....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Map a disk LBA to filename?

[Carsten Aulbert]

Hi Dave et al

Dave Chinner wrote:

> Blockget - yes it will. It just does the traversal internally to
> build the mapping. With large filesystems xfs_db can run out of
> memory building the mapping, which is why I've used the
> explicit traverse+xfs_bmap method in the past....

How expensive is this operation (rule of thumb)? We will mostly use this
on compute nodes where the partition is 500-750 GB large and the node
has 8 GB of memory. For the simple test where I apparently hit thin air
I have not seen anything bad in terms of memory consumption.

Cheers

Carsten

Re: Map a disk LBA to filename?

[Dave Chinner]

On Tue, Oct 28, 2008 at 08:11:46AM +0100, Carsten Aulbert wrote:
> Hi Dave et al
> 
> Dave Chinner wrote:
> 
> > Blockget - yes it will. It just does the traversal internally to
> > build the mapping. With large filesystems xfs_db can run out of
> > memory building the mapping, which is why I've used the
> > explicit traverse+xfs_bmap method in the past....
> 
> How expensive is this operation (rule of thumb)? We will mostly use this
> on compute nodes where the partition is 500-750 GB large and the node
> has 8 GB of memory. For the simple test where I apparently hit thin air
> I have not seen anything bad in terms of memory consumption.

Shouldn't be a problem with a filesystem that size. It's when you're
dealing with tens of terabytes in a single filesystem that it can
be a problem...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Map a disk LBA to filename?

[Carsten Aulbert]

Dave Chinner wrote:

> Shouldn't be a problem with a filesystem that size. It's when you're
> dealing with tens of terabytes in a single filesystem that it can
> be a problem...

It used a bit less than 900MB, so no problem at all.

But I think I'm a bit stuck now, because I know not enough about the
conventions used:

# xfs_info /dev/sda6
meta-data=/dev/sda6              isize=256    agcount=16, agsize=7388267
blks
         =                       sectsz=512   attr=1
data     =                       bsize=4096   blocks=118212272, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal               bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks
realtime =none                   extsz=65536  blocks=0, rtextents=0

>From this I gather that this file system as about 118 million blocks for
data, each holding 4096 bytes, this is consistent with the file system
size as reported by df and the like.

However, when I look at a file on this FS, e.g.
ls -l /local/MCP/root/root.tar.bz2
---xr-xr-- 1 root root 134 Sep 26 15:20 /local/MCP/root/root.tar.bz2

where /local is in /dev/sda6 (nevermind the weird permissions).

Running xfs_bmap on this file  gives:

# xfs_bmap -l /local/MCP/root/root.tar.bz2
/local/MCP/root/root.tar.bz2:
        0: [0..7]: 847868600..847868607 8 blocks

But somehow this does not work out since 847868600>118212272.

Any idea where my logical error is?

Thanks

puzzled Carsten

Re: Map a disk LBA to filename?

[KELEMEN Peter]

* Carsten Aulbert (carsten.aulbert@aei.mpg.de) [20081028 08:38]:

> # xfs_bmap -l /local/MCP/root/root.tar.bz2
> /local/MCP/root/root.tar.bz2:
>         0: [0..7]: 847868600..847868607 8 blocks

> But somehow this does not work out since 847868600>118212272.

xfs_bmap(8) talks in basic blocks (512 bytes).
847868600*512 < 118212272*4096.

HTH,
Peter

-- 
    .+'''+.         .+'''+.         .+'''+.         .+'''+.         .+''
 Kelemen Péter     /       \       /       \     Peter.Kelemen@cern.ch
.+'         `+...+'         `+...+'         `+...+'         `+...+'

Re: Map a disk LBA to filename?

[Carsten Aulbert]

Carsten Aulbert wrote:

> 
> Any idea where my logical error is?
> 

Related, more logical errors on my side probably:

Can someone give me a hint what I'm doing wrong here?

xfs_bmap -l  /local/boinc/time_stats_log
/local/boinc/time_stats_log:
        0: [0..127]: 280..407 128 blocks
        1: [128..159]: 728..759 32 blocks
        2: [160..167]: 1016..1023 8 blocks
        3: [168..183]: 54120..54135 16 blocks
        4: [184..207]: 656..679 24 blocks
        5: [208..215]: 3451664..3451671 8 blocks
        6: [216..255]: 3451624..3451663 40 blocks
        7: [256..287]: 3451672..3451703 32 blocks

>From that I assume that this file lives in several block strips, when I
subsequently try to get this back via xfs_db I fail miserably:

xfs_db -i /dev/sda6
xfs_db> blockget -b 660 -n
setting block 0/660 to data
setting inode to 2787 for block 0/660
inode 2787 block 660 at offset 346
xfs_db> ncheck -i 2787
       2787 CT_oneyear_01/S_noise_wu_84/Fstats_105.25_Snoise__WU84_84_out
xfs_db> quit

Is it possible that blocks != bsized_blocks?

Sorry for these questions, but I'm quite lost and I have not been able
to find the answer in the man pages.

Cheers

Carsten

-- 
Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics
Callinstrasse 38, 30167 Hannover, Germany
Phone/Fax: +49 511 762-17185 / -17193
http://www.top500.org/system/9234 | http://www.top500.org/connfam/6/list/31

Re: Map a disk LBA to filename?

[KELEMEN Peter]

* Carsten Aulbert (carsten.aulbert@aei.mpg.de) [20081028 10:14]:

> Is it possible that blocks != bsized_blocks?

xfs_bmap(8) talks in basic blocks (512 bytes).
660 div 8 = 82, so try:

	blockget -b 82 -n

> Sorry for these questions, but I'm quite lost and I have not
> been able to find the answer in the man pages.

The second paragraph of man xfs_bmap says:

“[...] All the file offsets and disk blocks are in units of
512-byte blocks, no matter what the filesystem’s block size
is.”

HTH,
Peter

-- 
    .+'''+.         .+'''+.         .+'''+.         .+'''+.         .+''
 Kelemen Péter     /       \       /       \     Peter.Kelemen@cern.ch
.+'         `+...+'         `+...+'         `+...+'         `+...+'

Re: Map a disk LBA to filename?

[Justin Piszcz]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 900 bytes --]



On Tue, 28 Oct 2008, KELEMEN Peter wrote:

> * Carsten Aulbert (carsten.aulbert@aei.mpg.de) [20081028 10:14]:
>
>> Is it possible that blocks != bsized_blocks?
>
> xfs_bmap(8) talks in basic blocks (512 bytes).
> 660 div 8 = 82, so try:
>
> 	blockget -b 82 -n
>
>> Sorry for these questions, but I'm quite lost and I have not
>> been able to find the answer in the man pages.
>
> The second paragraph of man xfs_bmap says:
>
> ?[...] All the file offsets and disk blocks are in units of
> 512-byte blocks, no matter what the filesystem?s block size
> is.?
>
> HTH,
> Peter
>
> --
>    .+'''+.         .+'''+.         .+'''+.         .+'''+.         .+''
> Kelemen Péter     /       \       /       \     Peter.Kelemen@cern.ch
> .+'         `+...+'         `+...+'         `+...+'         `+...+'
>
>

If you figure it out a little how-to would be nice :)

Justin.

Re: Map a disk LBA to filename?

[Dave Chinner]

On Tue, Oct 28, 2008 at 10:38:27AM +0100, KELEMEN Peter wrote:
> * Carsten Aulbert (carsten.aulbert@aei.mpg.de) [20081028 10:14]:
> 
> > Is it possible that blocks != bsized_blocks?
> 
> xfs_bmap(8) talks in basic blocks (512 bytes).
> 660 div 8 = 82, so try:
> 
> 	blockget -b 82 -n

I think all of you need to know about the "convert" command ;)

e.g. to do the above conversion correctly:

xfs_db> convert daddr 660 fsb
0x52 (82)

The xfs_db man page tells you all the different formats that can
be converted between. The convert command automatically takes into
account the geometry of the filesystem where appropriate (e.g.
filesystem block size).

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Map a disk LBA to filename?

[Michal Soltys]

Carsten Aulbert wrote:
> 
> Carsten Aulbert wrote:
> 
>> Any idea where my logical error is?
>>
> 
> Related, more logical errors on my side probably:
> 
> Can someone give me a hint what I'm doing wrong here?
> 
> xfs_bmap -l  /local/boinc/time_stats_log
> /local/boinc/time_stats_log:
>         0: [0..127]: 280..407 128 blocks
>         1: [128..159]: 728..759 32 blocks
>         2: [160..167]: 1016..1023 8 blocks
>         3: [168..183]: 54120..54135 16 blocks
>         4: [184..207]: 656..679 24 blocks
>         5: [208..215]: 3451664..3451671 8 blocks
>         6: [216..255]: 3451624..3451663 40 blocks
>         7: [256..287]: 3451672..3451703 32 blocks
> 
>>From that I assume that this file lives in several block strips, when I
> subsequently try to get this back via xfs_db I fail miserably:
> 
> xfs_db -i /dev/sda6
> xfs_db> blockget -b 660 -n
> setting block 0/660 to data
> setting inode to 2787 for block 0/660
> inode 2787 block 660 at offset 346
> xfs_db> ncheck -i 2787
>        2787 CT_oneyear_01/S_noise_wu_84/Fstats_105.25_Snoise__WU84_84_out
> xfs_db> quit
> 
> Is it possible that blocks != bsized_blocks?
> 
> Sorry for these questions, but I'm quite lost and I have not been able
> to find the answer in the man pages.
> 

660 is interpreted (by blockget) using filesystem's block size - so it'd
correspond to 5280 in 512 byte units. That file isn't that long.

Another example:

10:37 > ls -al /music/The\ Flashbulb\ -\ 11\ -\ Remember\ Tomorrow.flac

-rw-r--r-- 1 admin users 21921889 2008-09-08 09:07 /music/The Flashbulb
- 11 - Remember Tomorrow.flac

xfs_bmap -l /music/The\ Flashbulb\ -\ 11\ -\ Remember\ Tomorrow.flac

/music/The Flashbulb - 11 - Remember Tomorrow.flac:
         0: [0..35039]: 12140432..12175471 35040 blocks
         1: [35040..42823]: 12200864..12208647 7784 blocks

Let's check file's sector 36000 - from the perspective of filesystem -
sector 12200864+(36000-35040) = 12201824. Using 4096 block size -
1525228. Now let's check using xfs_db:

10:38 > xfs_db -r /dev/mapper/vg1-music
xfs_db> blockget -b 1525228 -n
setting block 0/1525228 to data
setting inode to 126 for block 0/1525228
inode 126 block 1525228 at offset 4500
sb_fdblocks 1330583, aggregate AGF count 2661166
xfs_db> ncheck -i 126
         126 The Flashbulb - 11 - Remember Tomorrow.flac