On-disk data for XFS_DINNODE_FMT_EXTENTS

On-disk data for XFS_DINNODE_FMT_EXTENTS

[hoper]

Hi,

I'm trying to understand how xfs is storing informations,
but I guess that my documentation "XFS Filesystem Structure
2nd Edition Revision 2" is quite outdated :(

I made an XFS filesystem, then made 60 files in /.

# xfs_db -f /root/small 
xfs_db> inode 128
xfs_db> addr
current
	byte offset 32768, length 256
	buffer block 64 (fsbno 8), 8 bbs
	inode 128, dir inode 128, type inode
xfs_db> p
core.magic = 0x494e
core.mode = 040755
core.version = 2
core.format = 2 (extents)
[...]
next_unlinked = null
u.bmx[0] = [startoff,startblock,blockcount,extentflag] 0:[0,12,1,0]

I made a small script in python which retrieve all theases information from disk,
except the "12" (location of the X2DB block = 12*4096). From the manuel, and my 
understanding, this information, 12, should be found just next the di_unlinked.
(page 43 on the manuel). But from an hex editor, from addr 32768, I got this :

49 4E 41 ED 02 02 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 01
56 37 5D 71 2D 26 3D 0B 56 37 5D 6C 02 78 F1 EC 56 37 5D 6C 02 78 F1 EC 00 00 00 00 00 00 10 00
00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00
FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 01 80 00 01 00 00 00 00 00 00 00 00 00 00 00 00

First, 49 4E (IN) that's the begining of the inode core (96 bytes). All informations are here,
and match the output of xfs_db. But after next_unlinked (FF FF FF FF here), and until next IN, 
I can't find any 12 :( Where is it ?

Lot's of thanks to anyone that could give me help on this subject.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: On-disk data for XFS_DINNODE_FMT_EXTENTS

[Brian Foster]

On Tue, Nov 03, 2015 at 10:48:53AM +0100, hoper@free.fr wrote:
> Hi,
> 
> I'm trying to understand how xfs is storing informations,
> but I guess that my documentation "XFS Filesystem Structure
> 2nd Edition Revision 2" is quite outdated :(
> 
> I made an XFS filesystem, then made 60 files in /.
> 
> # xfs_db -f /root/small 
> xfs_db> inode 128
> xfs_db> addr
> current
> 	byte offset 32768, length 256
> 	buffer block 64 (fsbno 8), 8 bbs
> 	inode 128, dir inode 128, type inode
> xfs_db> p
> core.magic = 0x494e
> core.mode = 040755
> core.version = 2
> core.format = 2 (extents)
> [...]
> next_unlinked = null
> u.bmx[0] = [startoff,startblock,blockcount,extentflag] 0:[0,12,1,0]
> 
> I made a small script in python which retrieve all theases information from disk,
> except the "12" (location of the X2DB block = 12*4096). From the manuel, and my 
> understanding, this information, 12, should be found just next the di_unlinked.
> (page 43 on the manuel). But from an hex editor, from addr 32768, I got this :
> 
> 49 4E 41 ED 02 02 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 01
> 56 37 5D 71 2D 26 3D 0B 56 37 5D 6C 02 78 F1 EC 56 37 5D 6C 02 78 F1 EC 00 00 00 00 00 00 10 00
> 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00
> FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 01 80 00 01 00 00 00 00 00 00 00 00 00 00 00 00
> 
> First, 49 4E (IN) that's the begining of the inode core (96 bytes). All informations are here,
> and match the output of xfs_db. But after next_unlinked (FF FF FF FF here), and until next IN, 
> I can't find any 12 :( Where is it ?
> 

The extent information is encoded in a format that's not easily
interpreted from a raw hex dump. See xfs_iformat_extents() to start:

                dp = (xfs_bmbt_rec_t *) XFS_DFORK_PTR(dip, whichfork);
                xfs_validate_extents(ifp, nex, XFS_EXTFMT_INODE(ip));
                for (i = 0; i < nex; i++, dp++) { 
                        xfs_bmbt_rec_host_t *ep = xfs_iext_get_ext(ifp, i);
                        ep->l0 = get_unaligned_be64(&dp->l0);
                        ep->l1 = get_unaligned_be64(&dp->l1);
                }

So we start after di_next_unlinked and read in two 64-bit, big endian
fields. That data in your dump above is:

l0: 0x0000000000000000
l1: 0x0000000001800001

Then, these 64-bit fields have a special encoding themselves to map to
actual extent data. From xfs_format.h:

/*
 * Bmap btree record and extent descriptor.
 *  l0:63 is an extent flag (value 1 indicates non-normal).
 *  l0:9-62 are startoff.
 *  l0:0-8 and l1:21-63 are startblock.
 *  l1:0-20 are blockcount.
 */

We see that l0 is completely zeroed, so we can infer that this is a
normal extent and startoff is zero. The block count is l1:0-20, which is
00001b or 1. The start block is (((l0 & 0x1FF) << 43) | (l1 >> 21)),
which is:

	((0x0 & 0x1FF) << 43) | (0x1800001 >> 21)
	 0x0		      |  0xC
		= 0xC == 12

See __xfs_bmbt_get_all() for the code that implements this decoding.

Brian

> Lot's of thanks to anyone that could give me help on this subject.
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: On-disk data for XFS_DINNODE_FMT_EXTENTS

[Brian Foster]

cc xfs

On Tue, Nov 03, 2015 at 03:58:32PM +0100, hoper wrote:
> Le 2015-11-03 14:50, Brian Foster a écrit :
> >On Tue, Nov 03, 2015 at 10:48:53AM +0100, hoper@free.fr wrote:
> >>Hi,
> >>
> >>I'm trying to understand how xfs is storing informations,
> >>but I guess that my documentation "XFS Filesystem Structure
> >>2nd Edition Revision 2" is quite outdated :(
> >>
> >>I made an XFS filesystem, then made 60 files in /.
> >>
> >># xfs_db -f /root/small
> >>xfs_db> inode 128
> >>xfs_db> addr
> >>current
> >>	byte offset 32768, length 256
> >>	buffer block 64 (fsbno 8), 8 bbs
> >>	inode 128, dir inode 128, type inode
> >>xfs_db> p
> >>core.magic = 0x494e
> >>core.mode = 040755
> >>core.version = 2
> >>core.format = 2 (extents)
> >>[...]
> >>next_unlinked = null
> >>u.bmx[0] = [startoff,startblock,blockcount,extentflag] 0:[0,12,1,0]
> >>
> >>I made a small script in python which retrieve all theses information
> >>from disk,
> >>except the "12" (location of the X2DB block = 12*4096). From the manual,
> >>and my
> >>understanding, this information, 12, should be found just next the
> >>di_unlinked.
> >>(page 43 on the manual). But from an hex editor, from addr 32768, I got
> >>this :
> >>
> >>49 4E 41 ED 02 02 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00
> >>00 00 00 00 00 00 00 01
> >>56 37 5D 71 2D 26 3D 0B 56 37 5D 6C 02 78 F1 EC 56 37 5D 6C 02 78 F1 EC
> >>00 00 00 00 00 00 10 00
> >>00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00 00
> >>00 00 00 00 00 00 00 00
> >>FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 01 80 00 01 00 00 00 00
> >>00 00 00 00 00 00 00 00
> >>
> >>First, 49 4E (IN) that's the beginning of the inode core (96 bytes). All
> >>informations are here,
> >>and match the output of xfs_db. But after next_unlinked (FF FF FF FF
> >>here), and until next IN,
> >>I can't find any 12 :( Where is it ?
> >>
> >
> >The extent information is encoded in a format that's not easily
> >interpreted from a raw hex dump. See xfs_iformat_extents() to start:
> >
> >                dp = (xfs_bmbt_rec_t *) XFS_DFORK_PTR(dip, whichfork);
> >                xfs_validate_extents(ifp, nex, XFS_EXTFMT_INODE(ip));
> >                for (i = 0; i < nex; i++, dp++) {
> >                        xfs_bmbt_rec_host_t *ep = xfs_iext_get_ext(ifp,
> >i);
> >                        ep->l0 = get_unaligned_be64(&dp->l0);
> >                        ep->l1 = get_unaligned_be64(&dp->l1);
> >                }
> >
> >So we start after di_next_unlinked and read in two 64-bit, big endian
> >fields. That data in your dump above is:
> >
> >l0: 0x0000000000000000
> >l1: 0x0000000001800001
> >
> >Then, these 64-bit fields have a special encoding themselves to map to
> >actual extent data. From xfs_format.h:
> >
> >/*
> > * Bmap btree record and extent descriptor.
> > *  l0:63 is an extent flag (value 1 indicates non-normal).
> > *  l0:9-62 are startoff.
> > *  l0:0-8 and l1:21-63 are startblock.
> > *  l1:0-20 are blockcount.
> > */
> >
> >We see that l0 is completely zeroed, so we can infer that this is a
> >normal extent and startoff is zero. The block count is l1:0-20, which is
> >00001b or 1. The start block is (((l0 & 0x1FF) << 43) | (l1 >> 21)),
> >which is:
> >
> >	((0x0 & 0x1FF) << 43) | (0x1800001 >> 21)
> >	 0x0		      |  0xC
> >		= 0xC == 12
> >
> >See __xfs_bmbt_get_all() for the code that implements this decoding.
> >
> >Brian
> 
> 
> A perfect answer. Thank you very much.
> 
> By the way, is there, by any chance, a new version of the manual I was
> looking at ?
> Because I didn't saw theses informations in it... And a good doc is far more
> readable
> for me than source code :)
> 
> 

No, the manual is somewhat out of date. That said, many of the
fundamental bits are unchanged. The following from the existing guide
covers the extent list inode format and actually does describe the
extent record encoding as well (see the first diagram):

http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/Data_Extents.html

Brian

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs