[PATCH 00/02] ext2: Reserve feature flags

[PATCH 00/02] ext2: Reserve feature flags

[Andreas Dilger]

The following patches reserve feature flags for e2fsprogs to avoid conflicts in the future.

1 - reserve a new INCOMPAT_INLINEDATA flag for storing small files in
    the inode itself
2 - reserve a new INCOMPAT_LARGEDIR flag for allowing directories over
    2GB in size and with an htree depth larger than 2 levels

Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.

[PATCH 01/02] ext2: reserve INCOMPAT_INLINEDATA feature flag

[Andreas Dilger]

Reserve the EXT4_FEATURE_INCOMPAT_INLINEDATA feature flag for use
with storing small file data in the i_blocks field (up to 60 bytes)
and/or the large xattr space (after i_extra_isize, up to the large
inode size).  On typical filesystems this can save 1-3% of the space,
and more importantly it also avoids seeking to read the data block.

This will become increasingly important with bigalloc filesystems,
since they consume a full cluster of blocks for each file, including
small directories that could easily fit into the inode itself.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Originally-by: Tao Ma <tm@tao.ma>
---
 lib/ext2fs/ext2_fs.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index 0f8cde8..6cf47f9 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -719,6 +719,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_INCOMPAT_FLEX_BG		0x0200
 #define EXT4_FEATURE_INCOMPAT_EA_INODE		0x0400
 #define EXT4_FEATURE_INCOMPAT_DIRDATA		0x1000
+#define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */
 
 #define EXT2_FEATURE_COMPAT_SUPP	0
 #define EXT2_FEATURE_INCOMPAT_SUPP    (EXT2_FEATURE_INCOMPAT_FILETYPE| \
-- 
1.7.2

[PATCH 02/02] ext2: reserve INCOMPAT_LARGEDIR feature flag

[Andreas Dilger]

Reserve the EXT4_FEATURE_INCOMPAT_LARGEDIR feature flag for use by
directories larger than 2GB in size, or with more than 2-level htree.
This allows directories to exceed the following size limits:

             best case       typical case     worst case
blocksize    8-byte name     64-byte name     255-byte name
=========    ============    ============     =============
 1024        768k entries    168k entries     36k entries
 4096        50M entries     10.9M entries    2.9M entries
16384        3.2B entries    713M entries     195M entries

The 2GB size limit could be handled by an RO_COMPAT feature (as was
done with LARGE_FILE) since it only affects the use of i_size_high,
but the 2-level htree limit is hard coded into the kernel code by
the use of frame->at[2] allowing only 2 levels of index.

Since it is impossible that a directory larger than 2GB would be
created without a 3-level htree for blocksize 4096, a single
INCOMPAT flag is used for both of these cases.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
---
 lib/ext2fs/ext2_fs.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index 6cf47f9..154356a 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -720,6 +720,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_INCOMPAT_EA_INODE		0x0400
 #define EXT4_FEATURE_INCOMPAT_DIRDATA		0x1000
 #define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */
+#define EXT4_FEATURE_INCOMPAT_LARGEDIR		0x4000 /* >2GB or 3-lvl htree */
 
 #define EXT2_FEATURE_COMPAT_SUPP	0
 #define EXT2_FEATURE_INCOMPAT_SUPP    (EXT2_FEATURE_INCOMPAT_FILETYPE| \
-- 
1.7.2

Re: [PATCH 01/02] ext2: reserve INCOMPAT_INLINEDATA feature flag

[Darrick J. Wong]

On Tue, Dec 06, 2011 at 05:13:51PM -0700, Andreas Dilger wrote:
> Reserve the EXT4_FEATURE_INCOMPAT_INLINEDATA feature flag for use
> with storing small file data in the i_blocks field (up to 60 bytes)
> and/or the large xattr space (after i_extra_isize, up to the large
> inode size).  On typical filesystems this can save 1-3% of the space,
> and more importantly it also avoids seeking to read the data block.
> 
> This will become increasingly important with bigalloc filesystems,
> since they consume a full cluster of blocks for each file, including
> small directories that could easily fit into the inode itself.
> 
> Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
> Originally-by: Tao Ma <tm@tao.ma>
> ---
>  lib/ext2fs/ext2_fs.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
> index 0f8cde8..6cf47f9 100644
> --- a/lib/ext2fs/ext2_fs.h
> +++ b/lib/ext2fs/ext2_fs.h
> @@ -719,6 +719,7 @@ struct ext2_super_block {
>  #define EXT4_FEATURE_INCOMPAT_FLEX_BG		0x0200
>  #define EXT4_FEATURE_INCOMPAT_EA_INODE		0x0400
>  #define EXT4_FEATURE_INCOMPAT_DIRDATA		0x1000
> +#define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */

I was using 0x2000 for EXT4_FEATURE_INCOMPAT_BG_USE_META_CSUM, to change the
block group checksum to use crc32c.

--D
> 
>  #define EXT2_FEATURE_COMPAT_SUPP	0
>  #define EXT2_FEATURE_INCOMPAT_SUPP    (EXT2_FEATURE_INCOMPAT_FILETYPE| \
> -- 
> 1.7.2
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 01/02] ext2: reserve INCOMPAT_INLINEDATA feature flag

[Tao Ma]

On 12/07/2011 08:13 AM, Andreas Dilger wrote:
> Reserve the EXT4_FEATURE_INCOMPAT_INLINEDATA feature flag for use
> with storing small file data in the i_blocks field (up to 60 bytes)
> and/or the large xattr space (after i_extra_isize, up to the large
> inode size).  On typical filesystems this can save 1-3% of the space,
> and more importantly it also avoids seeking to read the data block.
> 
> This will become increasingly important with bigalloc filesystems,
> since they consume a full cluster of blocks for each file, including
> small directories that could easily fit into the inode itself.
> 
> Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
> Originally-by: Tao Ma <tm@tao.ma>
I am fine with it. Thanks.

Tao
> ---
>  lib/ext2fs/ext2_fs.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
> index 0f8cde8..6cf47f9 100644
> --- a/lib/ext2fs/ext2_fs.h
> +++ b/lib/ext2fs/ext2_fs.h
> @@ -719,6 +719,7 @@ struct ext2_super_block {
>  #define EXT4_FEATURE_INCOMPAT_FLEX_BG		0x0200
>  #define EXT4_FEATURE_INCOMPAT_EA_INODE		0x0400
>  #define EXT4_FEATURE_INCOMPAT_DIRDATA		0x1000
> +#define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */
>  
>  #define EXT2_FEATURE_COMPAT_SUPP	0
>  #define EXT2_FEATURE_INCOMPAT_SUPP    (EXT2_FEATURE_INCOMPAT_FILETYPE| \

Re: [PATCH 01/02] ext2: reserve INCOMPAT_INLINEDATA feature flag

[Andreas Dilger]

On 2011-12-06, at 5:39 PM, Darrick J. Wong wrote:
> On Tue, Dec 06, 2011 at 05:13:51PM -0700, Andreas Dilger wrote:
>> +#define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */
> 
> I was using 0x2000 for EXT4_FEATURE_INCOMPAT_BG_USE_META_CSUM, to change
> the block group checksum to use crc32c.

I didn't see that...  I was looking at your patch and only noticed
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM.  I suspect your patch is closer
to landing it is better to use the 0x2000 value for BG_USE_META_CSUM,
and change the value for INLINEDATA to 0x8000 instead.

Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.

Re: [PATCH 01/02] ext2: reserve INCOMPAT_INLINEDATA feature flag

[Tao Ma]

On 12/07/2011 05:45 PM, Andreas Dilger wrote:
> On 2011-12-06, at 5:39 PM, Darrick J. Wong wrote:
>> On Tue, Dec 06, 2011 at 05:13:51PM -0700, Andreas Dilger wrote:
>>> +#define EXT4_FEATURE_INCOMPAT_INLINEDATA	0x2000 /* data in inode */
>>
>> I was using 0x2000 for EXT4_FEATURE_INCOMPAT_BG_USE_META_CSUM, to change
>> the block group checksum to use crc32c.
> 
> I didn't see that...  I was looking at your patch and only noticed
> EXT4_FEATURE_RO_COMPAT_METADATA_CSUM.  I suspect your patch is closer
> to landing it is better to use the 0x2000 value for BG_USE_META_CSUM,
> and change the value for INLINEDATA to 0x8000 instead.
OK, I will use this number for inline data for the new version if there
is no objection.

Thanks
Tao