Re: [PATCH 1/4] vfs: vfs-level fiemap interface

Anton Altaparmakov
2008-09-13T21:59:43Z

Hi,

On 13 Sep 2008, at 22:29, Theodore Tso wrote:
> On Sat, Sep 13, 2008 at 10:17:07PM +0100, Anton Altaparmakov wrote:
>> Just a quick question (not a criticism of the patches in any way):
>>
>> Is there a specific reason that the flag FIEMAP_EXTENT_DATA_ENCRYPTED
>> exists? I am only asking because there isn't a
>> FIEMAP_EXTENT_DATA_COMPRESSED flag yet the effect on the data is the
>> same. I suppose one could argue that compression is a form of
>> encryption (as one cannot read the data on disk without decompressing
>> it) and thus for compressed files I can just set
>> FIEMAP_EXTENT_DATA_ENCRYPTED, too?
>
> You could, but actually, the interface isn't really set up well to
> deal with compressed files. In order to deal with compressed data you
> really would need two fields in struct fiemap_extent:
> fe_length_physical, fe_length_logical, like this:
>
> struct fiemap_extent {
> __u64 fe_logical; /* logical offset in bytes for the start of
> * the extent from the beginning of the file */
> __u64 fe_physical; /* physical offset in bytes for the start
> * of the extent from the beginning of the disk */
> __u64 fe_length; /* length in bytes for this extent */
> __u64 fe_length_physical; /* length in bytes for this extent as
> * encoded/compressed on disk */
> __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */
> __u32 fe_reserved;
> };

Yes, that could make sense for compressed files although for NTFS at
least it does not matter because the logical regions which do not
exist on disk (due to compression having made the corresponding
physical regions smaller by at least one block) are present in the
extent list as sparse extents, so the extents for a compressed file
could look like this for example (lengths are in blocks):

logical ofs physical ofs length
0 X 14
14 SPARSE 2
16 Y 4
20 SPARSE 12
32 Z 9
41 SPARSE 7
... ... ...

As you may have noticed from the above example in NTFS each block of
16 blocks is compressed independently, hence the sparse extents
interspersed with the compressed extents to bring the logical offset
in line with the physical offset.

> Part of the problem/frustration as we go around and round fs-devel is
> that some people would request new features to support new
> filesystems, and other people have been rejecting new features because
> it makes the interface more complicated.
>
> In fact, the previous version of the patch had a flag for compression,
> but Cristoph Hellwig objected because he claimed there were no users
> for it.

He is plain wrong then! NTFS supports compressed files and always has
done! We only support them read-only at present but that does not
matter for the purposes of FIEMAP. And in any case one day we will
have write support for compressed files, too...

> I don't care how this interface ends up, at least as far as this point
> is concerned, but it's really annoying when some people want to add
> features, and other people are objecting and trying to remove
> features, and months and months and months go by while the patch
> languishes while people play tug of war on the d*mned thing.
>
> Probably the best thing to do is to add a much larger number of
> reserved fields, of size __u32 and __u64, appropriately aligned, and
> then when someone adds a compressed filesystem, we can add the
> structure definition without changing the size of the structure.
> Otherwise, the thou-shalt-not-anticipate-the-needs-of-any-future-
> users-of-the-interface nazi's will strike again, and more months will
> go by with absolutely no progress....

We already have two file systems in the mainline kernel tree that
support transparent compression/decompression: ntfs and zisofs so
there is no "future" anywhere in sight... (-;

Best regards,

Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/