Garbage in .git directories???

Garbage in .git directories???

[Dmitry Torokhov]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1000 bytes --]

Hi,

I just did:

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

[dtor@anvil work]$ ls -la .git/
total 40
drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
-rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?í®Ÿ?E ???#V?C??Å¿l??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The similar garbage(?) shows when I clone git's repository.

Is this expected?

-- 
Dmitry

Re: Garbage in .git directories???

[Linus Torvalds]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=utf-8, Size: 668 bytes --]



On Sat, 17 Sep 2005, Dmitry Torokhov wrote:
> 
> git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
> 
> and it seems there is some garbage in .git directory:
>
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?í®Ÿ?E ???#V?C??Å¿l??E#V?C??;H
>                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
> The similar garbage(?) shows when I clone git's repository.
> 
> Is this expected?

Absolutely not. And I don't see it when I try. What filesystem are you 
running this on? Looks like something did a "mkdir()" with an 
uninitialized pointer, but I don't see how that would happen.

		Linus

Re: Garbage in .git directories???

[Dmitry Torokhov]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 808 bytes --]

On Sunday 18 September 2005 12:10, Linus Torvalds wrote:
> 
> On Sat, 17 Sep 2005, Dmitry Torokhov wrote:
> > 
> > git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
> > 
> > and it seems there is some garbage in .git directory:
> >
> > drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?í®Ÿ?E ???#V?C??Å¿l??E#V?C??;H
> >                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
> > The similar garbage(?) shows when I clone git's repository.
> > 
> > Is this expected?
> 
> Absolutely not. And I don't see it when I try. What filesystem are you 
> running this on? Looks like something did a "mkdir()" with an 
> uninitialized pointer, but I don't see how that would happen.
> 
> 		Linus
> 

It is ext3 on 2.6.12-rc6.

-- 
Dmitry

Re: Garbage in .git directories???

[Dmitry Torokhov]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1421 bytes --]

On Sunday 18 September 2005 12:54, Dmitry Torokhov wrote:
> On Sunday 18 September 2005 12:10, Linus Torvalds wrote:
> > 
> > On Sat, 17 Sep 2005, Dmitry Torokhov wrote:
> > > 
> > > git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
> > > 
> > > and it seems there is some garbage in .git directory:
> > >
> > > drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?í®Ÿ?E ???#V?C??Å¿l??E#V?C??;H
> > >                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
> > > The similar garbage(?) shows when I clone git's repository.
> > > 
> > > Is this expected?
> > 
> > Absolutely not. And I don't see it when I try. What filesystem are you 
> > running this on? Looks like something did a "mkdir()" with an 
> > uninitialized pointer, but I don't see how that would happen.
> > 
> > 		Linus
> > 
> 
> It is ext3 on 2.6.12-rc6.
> 

Couple more points - just using rsync to get the data (without using git)
does not not produce that wierd directory, only got clone does.

Btw, after doing a pull from git repository git cone now produces the
following:

sent 403 bytes  received 2164 bytes  570.44 bytes/sec
total size is 533  speedup is 0.21
rsync: link_stat "/scm/git/git.git/objects/info/alternates" (in pub) failed: No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(1173)


-- 
Dmitry

Re: Garbage in .git directories???

[Junio C Hamano]

Dmitry Torokhov <dtor_core@ameritech.net> writes:

> Btw, after doing a pull from git repository git cone now produces the
> following:
>
> sent 403 bytes  received 2164 bytes  570.44 bytes/sec
> total size is 533  speedup is 0.21
> rsync: link_stat "/scm/git/git.git/objects/info/alternates" (in pub) failed: No such file or directory (2)
> rsync error: some files could not be transferred (code 23) at main.c(1173)

The garbage directory does ont reproduce for me, but I know what
this is.  It is an harmless error message -- I'll hide it in the
next update.  It is checking to see if the remote repository
uses alternates just in case -- since git.git does not use it,
it just does not find one (and needlessly complains).

Re: Garbage in .git directories???

[Linus Torvalds]

On Sun, 18 Sep 2005, Dmitry Torokhov wrote:
> 
> Couple more points - just using rsync to get the data (without using git)
> does not not produce that wierd directory, only got clone does.

Does a plain "git-init-db" in a newly created empty directory do the same?  
There was a bug (a long time ago)  that created garbage instead of the
template files, I think.

Also, is there something inside of the garbage subdirectory?

> Btw, after doing a pull from git repository git cone now produces the
> following:

Ignore that one, looks like a silly error. Unrelated.

		Linus

Re: Garbage in .git directories???

[Dmitry Torokhov]

On 9/19/05, Linus Torvalds <torvalds@osdl.org> wrote:
> 
> 
> On Sun, 18 Sep 2005, Dmitry Torokhov wrote:
> >
> > Couple more points - just using rsync to get the data (without using git)
> > does not not produce that wierd directory, only got clone does.
> 
> Does a plain "git-init-db" in a newly created empty directory do the same?
> There was a bug (a long time ago)  that created garbage instead of the
> template files, I think.
> 

Will test once get home...

> Also, is there something inside of the garbage subdirectory?
> 

No, they are always empty.

-- 
Dmitry

Re: Garbage in .git directories???

[H. Peter Anvin]

Dmitry Torokhov wrote:
> 
> git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work
> 
> and it seems there is some garbage in .git directory:
> 
> [dtor@anvil work]$ ls -la .git/
> total 40
> drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
> drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
> -rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
> lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
> drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
> drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
> drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?�?E ???#V?C??ſl??E#V?C??;H
>                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> The similar garbage(?) shows when I clone git's repository.
> 

FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
finding a directory in .git which consists of a single DEL character (\177).

	-hpa

Re: Garbage in .git directories???

[Petr Baudis]

Dear diary, on Mon, Sep 19, 2005 at 09:10:19PM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> Dmitry Torokhov wrote:
> >
> >git clone 
> >rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git 
> >work
> >
> >and it seems there is some garbage in .git directory:
> >
> >[dtor@anvil work]$ ls -la .git/
> >total 40
> >drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
> >drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
> >drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
> >-rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
> >lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
> >drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
> >drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
> >drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
> >drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
> >drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
> >drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?????E 
> >???#V?C????l??E#V?C??;H
> >                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> >The similar garbage(?) shows when I clone git's repository.
> >
> 
> FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
> finding a directory in .git which consists of a single DEL character (\177).

Does cg-init produce it too? It really seems that this has to be a bug
in git-init-db. If that is the case, could you try temporarily renaming
the templates directory?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
If you want the holes in your knowledge showing up try teaching
someone.  -- Alan Cox

Re: Garbage in .git directories???

[Horst von Brand]

H. Peter Anvin <hpa@zytor.com> wrote:

[...]

> FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep
> finding a directory in .git which consists of a single DEL character
> (\177).

Using latest cogito (0.15 63ef5488c38b6660c72f042d4aa3015d72a3bd28) +
latest git (0.99.7), cloning locally I don't see that. My older
repositories (cloned over the net at various times or made locally) don't
show it either.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Garbage in .git directories???

[H. Peter Anvin]

Petr Baudis wrote:
>>
>>FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
>>finding a directory in .git which consists of a single DEL character (\177).
> 
> Does cg-init produce it too? It really seems that this has to be a bug
> in git-init-db. If that is the case, could you try temporarily renaming
> the templates directory?

cg-init does produce it, too.  I'm not sure what you mean with 
temporarily renaming the templates directory?

	-hpa

Re: Garbage in .git directories???

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 01:16:14AM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> Petr Baudis wrote:
> >>
> >>FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
> >>finding a directory in .git which consists of a single DEL character 
> >>(\177).
> >
> >Does cg-init produce it too? It really seems that this has to be a bug
> >in git-init-db. If that is the case, could you try temporarily renaming
> >the templates directory?
> 
> cg-init does produce it, too.  I'm not sure what you mean with 
> temporarily renaming the templates directory?

Preventing git-init-db to copy its stuff into your .git/ - I think the
bug is likely somewhere there. Actually, this can be prevented by
passing an argument to git-init-db:

	git-init-db --template=somethingnonexisting

Another idea - try this patch:

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c
@@ -154,6 +154,7 @@ static void copy_templates(const char *g
 	}
 
 	memcpy(path, git_dir, len);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Garbage in .git directories???

[H. Peter Anvin]

Petr Baudis wrote:
> Dear diary, on Tue, Sep 20, 2005 at 01:16:14AM CEST, I got a letter
> 
> Another idea - try this patch:
> 
> diff --git a/init-db.c b/init-db.c
> --- a/init-db.c
> +++ b/init-db.c
> @@ -154,6 +154,7 @@ static void copy_templates(const char *g
>  	}
>  
>  	memcpy(path, git_dir, len);
> +	path[len] = 0;
>  	copy_templates_1(path, len,
>  			 template_path, template_len,
>  			 dir);
> 

This seems to have worked.

	-hpa

[PATCH] Fix git-init-db creating crap directories

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 01:58:52AM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> Petr Baudis wrote:
> >Dear diary, on Tue, Sep 20, 2005 at 01:16:14AM CEST, I got a letter
> >
> >Another idea - try this patch:
> >
> >diff --git a/init-db.c b/init-db.c
> >--- a/init-db.c
> >+++ b/init-db.c
> >@@ -154,6 +154,7 @@ static void copy_templates(const char *g
> > 	}
> > 
> > 	memcpy(path, git_dir, len);
> >+	path[len] = 0;
> > 	copy_templates_1(path, len,
> > 			 template_path, template_len,
> > 			 dir);
> >
> 
> This seems to have worked.

Great. Resending a marginally better version with a cool log message and
everything.

--

The base target directory for the templates copying was initialized
to git_dir, but git_dir[len] is not zero but / at the time we do the
initialization. This is not what we want for our target directory string
since we pass it to mkdir(), so make it zero-terminated manually.

Signed-off-by: Petr Baudis <pasky@suse.cz>

---
commit dbe1fe2f0d6de58bc0c9f98e6cf9573874612890
tree 30398d409fa27e95e3e592c803f6cd5a93652234
parent 79d2a11ec497fe940ca9a1fd8e0b9f295d39c290
author Petr Baudis <pasky@suse.cz> Tue, 20 Sep 2005 01:55:53 +0200
committer Petr Baudis <xpasky@machine.(none)> Tue, 20 Sep 2005 01:55:53 +0200

 init-db.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c
@@ -153,7 +153,8 @@ static void copy_templates(const char *g
 		return;
 	}
 
-	memcpy(path, git_dir, len);
+	memcpy(path, git_dir, len-1);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Garbage in .git directories???

[Dmitry Torokhov]

On Monday 19 September 2005 09:51, Linus Torvalds wrote:
> 
> On Sun, 18 Sep 2005, Dmitry Torokhov wrote:
> > 
> > Couple more points - just using rsync to get the data (without using git)
> > does not not produce that wierd directory, only got clone does.
> 
> Does a plain "git-init-db" in a newly created empty directory do the same?  
> There was a bug (a long time ago)  that created garbage instead of the
> template files, I think.
>

Yes, plain git-init-db produces the garbage.

[dtor@anvil tst]$ git-init-db
defaulting to local storage area
[dtor@anvil tst]$ ls -l .git/
total 32
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 branches
-rw-rw-r--    1 dtor dtor   58 Sep 19 19:08 description
lrwxrwxrwx    1 dtor dtor   17 Sep 19 19:08 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 info
drwxrwxr-x  260 dtor dtor 4096 Sep 19 19:08 objects
drwxrwxr-x    4 dtor dtor 4096 Sep 19 19:08 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 V?Cl???E ???#V?C??տl??E#V?C??;H
[dtor@anvil tst]$ git --version
git version 0.99.7

(Actually this is last night's pull).


-- 
Dmitry

Re: [PATCH] Fix git-init-db creating crap directories

[H. Peter Anvin]

Petr Baudis wrote:
> diff --git a/init-db.c b/init-db.c
> --- a/init-db.c
> +++ b/init-db.c
> @@ -153,7 +153,8 @@ static void copy_templates(const char *g
>  		return;
>  	}
>  
> -	memcpy(path, git_dir, len);
> +	memcpy(path, git_dir, len-1);
> +	path[len] = 0;
>  	copy_templates_1(path, len,

Wrong!  You're not initializing path[len-1]!

	-hpa

[PATCH] Fix git-init-db creating crap directories (zeroth try)

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 02:09:41AM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> Petr Baudis wrote:
> >diff --git a/init-db.c b/init-db.c
> >--- a/init-db.c
> >+++ b/init-db.c
> >@@ -153,7 +153,8 @@ static void copy_templates(const char *g
> > 		return;
> > 	}
> > 
> >-	memcpy(path, git_dir, len);
> >+	memcpy(path, git_dir, len-1);
> >+	path[len] = 0;
> > 	copy_templates_1(path, len,
> 
> Wrong!  You're not initializing path[len-1]!

Oops, sorry. That's what you get when you want to make things marginally
better. ;-) We indeed want to pass copy_templates_1() the trailing slash
as well.

Let's just settle with the original patch then.

--

The base target directory for the templates copying was initialized
to git_dir, but git_dir[len] is not zero but / at the time we do the
initialization. This is not what we want for our target directory string
since we pass it to mkdir(), so make it zero-terminated manually.

Signed-off-by: Petr Baudis <pasky@suse.cz>

---
diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c
@@ -154,6 +154,7 @@ static void copy_templates(const char *g
 	}
 
 	memcpy(path, git_dir, len);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);


-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] Fix git-init-db creating crap directories (zeroth try)

[H. Peter Anvin]

Petr Baudis wrote:
>>>
>>>-	memcpy(path, git_dir, len);
>>>+	memcpy(path, git_dir, len-1);
>>>+	path[len] = 0;
>>>	copy_templates_1(path, len,
>>
>>Wrong!  You're not initializing path[len-1]!
> 
> 
> Oops, sorry. That's what you get when you want to make things marginally
> better. ;-) We indeed want to pass copy_templates_1() the trailing slash
> as well.
> 
> Let's just settle with the original patch then.
> 

But if len is the index of the '/', then you're not.

Think about it: the memcpy(path, git_dir, len) copies bytes 0..len-1. 
Thus you need to use path[len] = 0 to terminate.

If you want to copy len-1 characters, then you need to use path[len-1] = 0.

	-hpa

Re: Garbage in .git directories???

[Linus Torvalds]

On Mon, 19 Sep 2005, Dmitry Torokhov wrote:
>
> Yes, plain git-init-db produces the garbage.

Hmm. It doesn't do it for me, but I suspect it will be obvious where it
happens if you send us the output from "strace git-init-db"

Thanks,

		Linus

Re: [PATCH] Fix git-init-db creating crap directories (zeroth try)

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 02:42:47AM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> Petr Baudis wrote:
> >>>
> >>>-	memcpy(path, git_dir, len);
> >>>+	memcpy(path, git_dir, len-1);
> >>>+	path[len] = 0;
> >>>	copy_templates_1(path, len,
> >>
> >>Wrong!  You're not initializing path[len-1]!
> >
> >
> >Oops, sorry. That's what you get when you want to make things marginally
> >better. ;-) We indeed want to pass copy_templates_1() the trailing slash
> >as well.
> >
> >Let's just settle with the original patch then.
> >
> 
> But if len is the index of the '/', then you're not.

But fortunately len is not the index of the '/', len-1 is. len is the length
of the leading path fragment including the trailing slash.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: [PATCH] Fix git-init-db creating crap directories (zeroth try)

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

>> >Oops, sorry. That's what you get when you want to make things marginally
>> >better. ;-) We indeed want to pass copy_templates_1() the trailing slash
>> >as well.

Original patch from you is the right one.  Thanks.

Re: [PATCH] Fix git-init-db creating crap directories (zeroth try)

[H. Peter Anvin]

Petr Baudis wrote:
>>
>>But if len is the index of the '/', then you're not.
> 
> But fortunately len is not the index of the '/', len-1 is. len is the length
> of the leading path fragment including the trailing slash.
> 

Then you'd need len-1 both inside the memcpy() and the assignment, of 
course.

	-hpa

Re: Garbage in .git directories???

[Dmitry Torokhov]

On Monday 19 September 2005 19:45, Linus Torvalds wrote:
> 
> On Mon, 19 Sep 2005, Dmitry Torokhov wrote:
> >
> > Yes, plain git-init-db produces the garbage.
> 
> Hmm. It doesn't do it for me, but I suspect it will be obvious where it
> happens if you send us the output from "strace git-init-db"
> 

Peter's fix for copy_templates() fixes the problem for me.

-- 
Dmitry