RE: FIEMAP ioctl gets "wrong" address for the extent

["Rebraca Dejan (BSOT/PJ-ES1-Bg)" <Dejan.Rebraca@rs.bosch.com>]

Hi Qu,

I'm using this structure to get the address of file extent:

struct fiemap_extent {
	__u64	fe_logical;  /* logical offset in bytes for the start of
			      * the extent */
	__u64	fe_physical; /* physical offset in bytes for the start
			      * of the extent */
	__u64	fe_length;   /* length in bytes for the extent */
	__u64	fe_reserved64[2];
	__u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
	__u32	fe_reserved[3];
};

And using fe_physical field I verified that it really reflects the offset in filesystem image - I can see that file content begins at this offset.
The problem is that I run into some specific case where file content doesn't begin at fe_physical, I rather have something else at this offset.

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <quwenruo.btrfs@gmx.com> 
Sent: četvrtak, 02. jul 2020. 12:19
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <Dejan.Rebraca@rs.bosch.com>; linux-btrfs@vger.kernel.org
Subject: Re: FIEMAP ioctl gets "wrong" address for the extent



On 2020/7/2 下午5:11, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi all,
> 
> I'm collecting file extents for our application from BtrFs filesystem image.
> I've noticed that for some files a get the "wrong" physical offset for start of the extent.

First thing first, btrfs fiemap ioctl only returns btrfs logical address.
That's an address space in [0, U64_MAX), thus it's not a physical offset you can find in the device.

For example, for a btrfs on a 10G disk, btrfs fiemap can return address at 128G, which you can never find on that disk.

This is not that strange, as btrfs can be a multi-device fs, thus we must have an internal address space, and then map part of the logical address into physical disk space.

> I verified it using hexdump of the filesystem image: when dump the content starting from the address returned from FIEMAP ioctl, I see that the content is absolutely different from the content of the file itself. Also, the FIEMAP ioctl reports regular extent, it is not inline.

If you're using the logical address returned from disk directly, then you won't get the correct data obviously.

What you need is to map the btrfs logical address to physical device offset, that is done by referring to chunk tree.
And even after the conversion, it's not always the case for all profiles.
For SINGLE/DUP/RAID0/RAID1/RAID10/RAID1C*, you can find the data directly, but for RAID5/6, you need to bother the P/Q stripe.

And furthermore, there are compressed data extents, which on-disk data is compressed, which also diffs from the uncompressed data.


For the chunk mapping, you can verify the mapping of <logical address> to <physical address> using btrfs inspect dump-tree -t chunk <device>.

The details of the btrfs_chunk on-disk format can be found here:
https://btrfs.wiki.kernel.org/index.php/On-disk_Format#CHUNK_ITEM_.28e4.29

Thanks,
Qu

> 
> Environment:
> - 4.15.0-96-generic #97~16.04.1-Ubuntu SMP Wed Apr 1 03:03:31 UTC 2020 
> x86_64 x86_64 x86_64 GNU/Linux
> - btrfs-progs v4.4
> 
> Does anyone has any idea? I would appreciate your help on this one.
> Tnx.
> 
> Best regards,
> Dejan
>