[PATCH -v2]ext4: add the spec for ext4 subdirectory limit(was: Problems with the max value for create directory)

[PATCH -v2]ext4: add the spec for ext4 subdirectory limit(was: Problems with the max value for create directory)

[Zhang Xiliang]

When using hash index, too many long-subdirectory-names and 1024 blocksize, the
subdirectory limit may be less than 32000.

The problem is that indexed diretories have a limit that only allows the trees
to be two levels deep. The fanout is normally big enough that this is effectively
not a problem, but if you use too many long-names, and a 1024 blocksize, you will
run into htree limit much more quickly.

The users may be puzzled when they can't create more than 32000 subdirecties with
using hash index, too many long-names and 1024 blocksize. It should be remind of
in the ext4 document.

Signed-off-by: Zhang Xiliang <zhangxiliang@cn.fujitsu.com>
---
 Documentation/filesystems/ext4.txt |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/Documentation/filesystems/ext4.txt b/Documentation/filesystems/ext4.txt
index cec829b..0e1ec9c 100644
--- a/Documentation/filesystems/ext4.txt
+++ b/Documentation/filesystems/ext4.txt
@@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be
 * extent format more robust in face of on-disk corruption due to magics,
 * internal redundancy in tree
 * improved file allocation (multi-block alloc)
-* fix 32000 subdirectory limit
+* fix 32000 subdirectory limit (*1)
 * nsec timestamps for mtime, atime, ctime, create time
 * inode version field on disk (NFSv4, Lustre)
 * reduced e2fsck time via uninit_bg feature
@@ -100,6 +100,12 @@ Note: More extensive information for getting started with ext4 can be
 * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force
   the ordering)
 
+(*1) Note
+Basically, the limit depends on blocksize and length of subdirectory names in
+the directory. If blocksize is too small (like 1024) and you use too many
+long-names, you may reach a smaller limit. When dir index is off, the limit is
+65000.
+
 2.2 Candidate features for future inclusion
 
 * Online defrag (patches available but not well tested)

Re: [PATCH -v2]ext4: add the spec for ext4 subdirectory limit(was: Problems with the max value for create directory)

[Theodore Tso]

I really don't think this tweak to Documentation/filesystems/ext4.txt
is necessary, but if we going to do this, I want to tighten up the
explanation.

						- Ted

ext4: Add fine print for the 32000 subdirectory limit

Some poeple are reading the ext4 feature list too literally and create
dubious test cases involving very long filenames and 1k blocksize and
then complain when they run into an htree-imposed limit.  So add fine
print to the "fix 32000 subdirectory limit" ext4 feature.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
diff --git a/Documentation/filesystems/ext4.txt b/Documentation/filesystems/ext4.txt
index cec829b..5c484ae 100644
--- a/Documentation/filesystems/ext4.txt
+++ b/Documentation/filesystems/ext4.txt
@@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be
 * extent format more robust in face of on-disk corruption due to magics,
 * internal redundancy in tree
 * improved file allocation (multi-block alloc)
-* fix 32000 subdirectory limit
+* lift 32000 subdirectory limit imposed by i_links_count[1]
 * nsec timestamps for mtime, atime, ctime, create time
 * inode version field on disk (NFSv4, Lustre)
 * reduced e2fsck time via uninit_bg feature
@@ -100,6 +100,9 @@ Note: More extensive information for getting started with ext4 can be
 * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force
   the ordering)
 
+[1] Filesystems with a block size of 1k may see a limit imposed by the
+directory hash tree having a maximum depth of two.
+
 2.2 Candidate features for future inclusion
 
 * Online defrag (patches available but not well tested)

Re: [PATCH -v2]ext4: add the spec for ext4 subdirectory limit(was:Problems with the max value for create directory)

[Zhang Xiliang]

Theodore Tso wrote:
> I really don't think this tweak to Documentation/filesystems/ext4.txt
> is necessary, but if we going to do this, I want to tighten up the
> explanation.
> 
> 						- Ted
> 
I agree.
Thanks.

> ext4: Add fine print for the 32000 subdirectory limit
> 
> Some poeple are reading the ext4 feature list too literally and create
> dubious test cases involving very long filenames and 1k blocksize and
> then complain when they run into an htree-imposed limit.  So add fine
> print to the "fix 32000 subdirectory limit" ext4 feature.
> 
> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
> diff --git a/Documentation/filesystems/ext4.txt b/Documentation/filesystems/ext4.txt
> index cec829b..5c484ae 100644
> --- a/Documentation/filesystems/ext4.txt
> +++ b/Documentation/filesystems/ext4.txt
> @@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be
>  * extent format more robust in face of on-disk corruption due to magics,
>  * internal redundancy in tree
>  * improved file allocation (multi-block alloc)
> -* fix 32000 subdirectory limit
> +* lift 32000 subdirectory limit imposed by i_links_count[1]
>  * nsec timestamps for mtime, atime, ctime, create time
>  * inode version field on disk (NFSv4, Lustre)
>  * reduced e2fsck time via uninit_bg feature
> @@ -100,6 +100,9 @@ Note: More extensive information for getting started with ext4 can be
>  * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force
>    the ordering)
>  
> +[1] Filesystems with a block size of 1k may see a limit imposed by the
> +directory hash tree having a maximum depth of two.
> +
>  2.2 Candidate features for future inclusion
>  
>  * Online defrag (patches available but not well tested)
> 
> 
>