[PATCH/RFC] clone: suggest index version 4 when the index is bigger than 512 KiB

[PATCH/RFC] clone: suggest index version 4 when the index is bigger than 512 KiB

[Nguyễn Thái Ngọc Duy]

I just realized that many of my big repos are still on index v2 while
v4 should reduce its size significantly (3.8M -> 2.9M for linux-2.6
and 25M -> 14M for webkit, for example). I wanted to propose index v4
as the new default version, because I guess that not many people
outside git@vger are aware of it, but perhaps this approach is less
drastic. It only suggest index v4 when the index size after clone hits
512K limit.

I have 170 git repositories (most of them Gnome projects) and only big
ones exceed the limit: webkit, linux-2.6, libreoffice-core, wine
(530K), gentoo-x86. Gimp and banshee are close (510K and 321K). The
rest barely hits 256K. So this hint won't show up often. On second
thought, maybe we should lower it down to 300K to show more often and
raise awareness on index v4 :)

Something I should have done in this patch too, is state that index v4
is only supported since vXXX, that git clients older than that will
not work. But maybe just put that in update-index man page and refer
there.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/clone.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/builtin/clone.c b/builtin/clone.c
index e0aaf13..7cd1b60 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -543,6 +543,7 @@ static int checkout(void)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0, fd;
+	struct stat st;
 
 	if (option_no_checkout)
 		return 0;
@@ -591,6 +592,15 @@ static int checkout(void)
 	if (!err && option_recursive)
 		err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
 
+	if (!err &&
+	    !stat(git_path("index"), &st) &&
+	    st.st_size > 512 * 1024)
+		advise(_("Your index is quite large (%d KiB).\n"
+			 "You may want to update to index version 4 to reduce its size,\n"
+			"as large index files may affect performance, using the command:\n"
+			 "  git update-index --index-version 4"),
+		       st.st_size / 1024);
+
 	return err;
 }
 
-- 
1.8.1.2.536.gf441e6d

Re: [PATCH/RFC] clone: suggest index version 4 when the index is bigger than 512 KiB

[Thomas Ackermann]

Nguyễn Thái Ngọc Duy <pclouds <at> gmail.com> writes:

> 
> I just realized that many of my big repos are still on index v2 while
> v4 should reduce its size significantly (3.8M -> 2.9M for linux-2.6
> and 25M -> 14M for webkit, for example). I wanted to propose index v4
> as the new default version, because I guess that not many people
> outside git <at> vger are aware of it, but perhaps this approach is less
> drastic. It only suggest index v4 when the index size after clone hits
> 512K limit.
> 

Is V4 really recommended for general use?
Default seems to be V3 and all I found in the docs is
the coresponding release not describing V4 as "experimental".
(I have repos with index files > 40MiB (!) which will shrink to
approx. 20MiB with V4 so using V4 would be an interessting option for me.)

---
Thomas

Re: [PATCH/RFC] clone: suggest index version 4 when the index is bigger than 512 KiB

[Duy Nguyen]

On Tue, Feb 19, 2013 at 5:45 PM, Thomas Ackermann <th.acker@arcor.de> wrote:
> Nguyễn Thái Ngọc Duy <pclouds <at> gmail.com> writes:
>
>>
>> I just realized that many of my big repos are still on index v2 while
>> v4 should reduce its size significantly (3.8M -> 2.9M for linux-2.6
>> and 25M -> 14M for webkit, for example). I wanted to propose index v4
>> as the new default version, because I guess that not many people
>> outside git <at> vger are aware of it, but perhaps this approach is less
>> drastic. It only suggest index v4 when the index size after clone hits
>> 512K limit.
>>
>
> Is V4 really recommended for general use?
> Default seems to be V3 and all I found in the docs is
> the coresponding release not describing V4 as "experimental".
> (I have repos with index files > 40MiB (!) which will shrink to
> approx. 20MiB with V4 so using V4 would be an interessting option for me.)

The default could be either v2 or v3, depending on whether you use
some extra features. v4 is basically v3 with pathname compression.
Yes, I think it's ok for general use.
-- 
Duy

Re: [PATCH/RFC] clone: suggest index version 4 when the index is bigger than 512 KiB

[Junio C Hamano]

Thomas Ackermann <th.acker@arcor.de> writes:

> Is V4 really recommended for general use?

If you stick to C-git, I do not think there is any reason to avoid
it.  It is already a mature technology, the difference between 2 and
4 are so trivial that it is very unlikely for a latent bug to be
hiding in corner cases.

It is a different matter if various reimplementations already
support the format.  You can try it out, and if you find that other
tools based on reimplementations of Git you use do not understand
the format, you can safely go back with

	git update-index --index-version 2

after you upgraded to version 4.