Error: unable to unlink ... when using "git gc"

Error: unable to unlink ... when using "git gc"

[Johnny Lee]

Hello Everyone,

I'm new in git, and when I was practicing the tutorial:
http://www.kernel.org/pub/software/scm/git/docs/everyday.html

I've had a problem when using "git gc"

Here is the log:
git@tomato:~/golf$ git gc
Generating pack...
Done counting 121847 objects.
Deltifying 121847 objects...
 100% (121847/121847) done
Writing 121847 objects...
 100% (121847/121847) done
Total 121847 (delta 62587), reused 121184 (delta 62253)
Pack pack-44320b247756067835f5e4a789c7eb5bcc5266c3 created.
error: unable to unlink .git/objects/16/b14f5da9e2fcd6f3f38cc9e584cef2f3c90ebe
error: unable to unlink .git/objects/a1/f6672f6e5bd90acf16a43fea774ca655a3ab1e
error: unable to unlink .git/objects/a8/5cca5b91cf77e8df731827849308ef6dae4536
error: unable to unlink .git/objects/ae/64c9d0ea558e403acf139f690c804212d490be
error: unable to unlink .git/objects/b3/a4c72ea70a3bb43b1819a0a74268870849a21d
error: unable to unlink .git/objects/db/5ec507a2ed86169e4a31bba1d95067c4f3fac4
error: unable to unlink .git/objects/ef/65ed2454b8dcbf8837595d8881ca42898a491c
error: unable to unlink .git/objects/fb/92c37ccc8bdbf3c70bc3afaf8261cedbc92d71
Removing unused objects 100%...
Done.

While I'm looking at these "unable to unlink" files, it seems they are
read only:
git@tomato:~/golf$ ls -l .git/objects/16/
total 4
-r--r--r-- 1 johnny johnny 26 2009-01-05 09:25
b14f5da9e2fcd6f3f38cc9e584cef2f3c90ebe

Is there anything wrong here?

BTW. I've used git fsck with/without --full, it works fine and no
error is reported.

Thanks for your consideration,
Johnny

-- 
we all have our crosses to bear

Re: Error: unable to unlink ... when using "git gc"

[Jeff King]

On Tue, Jan 06, 2009 at 03:00:52PM +0800, Johnny Lee wrote:

> While I'm looking at these "unable to unlink" files, it seems they are
> read only:
> git@tomato:~/golf$ ls -l .git/objects/16/
> total 4
> -r--r--r-- 1 johnny johnny 26 2009-01-05 09:25
> b14f5da9e2fcd6f3f38cc9e584cef2f3c90ebe
> 
> Is there anything wrong here?

It is normal for objects to be read-only. However, permission to unlink
a file comes from its directory permission. Is .git/objects/16 writable
by you?

-Peff

Re: Error: unable to unlink ... when using "git gc"

[Jeff King]

[re-adding git@vger; please keep discussion on-list so everyone can
benefit from the result]

On Tue, Jan 06, 2009 at 03:52:12PM +0800, Johnny Lee wrote:

> Thanks Peff, I've checked the permission of .git/objects/16, it's
> created by another user and thus I have no permission to remove it.
> 
> In fact, this is coming from a previous bad practice on setting up a
> collaboration repository on a SSH server, here is what I've done so
> far:
> [...]
> 7. Then the user "git" has changed mode for all the files under .git
> to writable.
> 
> 8. This time, user "johnny" can push successfully.

If you are going to have multiple users sharing a repository, generally
they should be in the same group and the core.sharedrepository config
option should be set (see "git help config", or the "shared" option to
git-init).

I've never used that personally, though. I have always just used POSIX
ACLs, with a default ACL on each directory giving access to everyone.
E.g. (off the top of my head):

  for user in user1 user2 user3; do
    setfacl -R -m u:$user:rwX -m d:u:$user:rwX /path/to/repo
  done

-Peff

Re: Error: unable to unlink ... when using "git gc"

[Johnny Lee]

Ooops, forgot to cc to the mailing list.

Thanks for your sharing Peff, I'm going to check these stuffs out and
get back with any findings.

Regards,
Johnny

On Tue, Jan 6, 2009 at 4:03 PM, Jeff King <peff@peff.net> wrote:
> [re-adding git@vger; please keep discussion on-list so everyone can
> benefit from the result]
>
> On Tue, Jan 06, 2009 at 03:52:12PM +0800, Johnny Lee wrote:
>
>> Thanks Peff, I've checked the permission of .git/objects/16, it's
>> created by another user and thus I have no permission to remove it.
>>
>> In fact, this is coming from a previous bad practice on setting up a
>> collaboration repository on a SSH server, here is what I've done so
>> far:
>> [...]
>> 7. Then the user "git" has changed mode for all the files under .git
>> to writable.
>>
>> 8. This time, user "johnny" can push successfully.
>
> If you are going to have multiple users sharing a repository, generally
> they should be in the same group and the core.sharedrepository config
> option should be set (see "git help config", or the "shared" option to
> git-init).
>
> I've never used that personally, though. I have always just used POSIX
> ACLs, with a default ACL on each directory giving access to everyone.
> E.g. (off the top of my head):
>
>  for user in user1 user2 user3; do
>    setfacl -R -m u:$user:rwX -m d:u:$user:rwX /path/to/repo
>  done
>
> -Peff
>



-- 
we all have our crosses to bear

Re: Error: unable to unlink ... when using "git gc"

[Sitaram Chamarty]

On 2009-01-06, Jeff King <peff@peff.net> wrote:
> If you are going to have multiple users sharing a repository, generally
> they should be in the same group and the core.sharedrepository config
> option should be set (see "git help config", or the "shared" option to
> git-init).
>
> I've never used that personally, though. I have always just used POSIX
> ACLs, with a default ACL on each directory giving access to everyone.
> E.g. (off the top of my head):
>
>   for user in user1 user2 user3; do
>     setfacl -R -m u:$user:rwX -m d:u:$user:rwX /path/to/repo
>   done

If you're not worried about the finer-grained access control
that acl(5) gives you, just do what "git init
--shared=group" does:

    git config core.sharedrepository 1 # as mentioned above
    chmod g+ws .git

Now set the group to something (I use "gitpushers" ;-)

    chgrp -R gitpushers .git

amd make sure all your users are part of that group.

Works fine for small teams...

Re: Error: unable to unlink ... when using "git gc"

[Johnny Lee]

Copy that.

Thanks Sitaram.

We also plan to do it in this way, just a small wondering that it
looks a kind of workaround instead of a more graceful solution.

Regards,
Johnny

On Tue, Jan 6, 2009 at 7:57 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:
> On 2009-01-06, Jeff King <peff@peff.net> wrote:
>> If you are going to have multiple users sharing a repository, generally
>> they should be in the same group and the core.sharedrepository config
>> option should be set (see "git help config", or the "shared" option to
>> git-init).
>>
>> I've never used that personally, though. I have always just used POSIX
>> ACLs, with a default ACL on each directory giving access to everyone.
>> E.g. (off the top of my head):
>>
>>   for user in user1 user2 user3; do
>>     setfacl -R -m u:$user:rwX -m d:u:$user:rwX /path/to/repo
>>   done
>
> If you're not worried about the finer-grained access control
> that acl(5) gives you, just do what "git init
> --shared=group" does:
>
>    git config core.sharedrepository 1 # as mentioned above
>    chmod g+ws .git
>
> Now set the group to something (I use "gitpushers" ;-)
>
>    chgrp -R gitpushers .git
>
> amd make sure all your users are part of that group.
>
> Works fine for small teams...
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
we all have our crosses to bear

Re: Error: unable to unlink ... when using "git gc"

[Sitaram Chamarty]

On 2009-01-06, Johnny Lee <johnnylee194@gmail.com> wrote:

> On Tue, Jan 6, 2009 at 7:57 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:

>> If you're not worried about the finer-grained access control
>> that acl(5) gives you, just do what "git init
>> --shared=group" does:
>>
>>    git config core.sharedrepository 1 # as mentioned above
>>    chmod g+ws .git
>>
>> Now set the group to something (I use "gitpushers" ;-)
>>
>>    chgrp -R gitpushers .git
>>
>> amd make sure all your users are part of that group.

> We also plan to do it in this way, just a small wondering that it
> looks a kind of workaround instead of a more graceful solution.

I wouldn't consider it a workaround.  It uses normal Unix
permissions the way they were designed to, including setgid
for directories.

Actually, I am yet to come up with a situation where I
actually needed ACLs, though they are more generalised, and
fine-grained.

And the maint is all eminently scriptable.

Re: Error: unable to unlink ... when using "git gc"

[Jeff King]

On Tue, Jan 06, 2009 at 03:33:57PM +0000, Sitaram Chamarty wrote:

> > We also plan to do it in this way, just a small wondering that it
> > looks a kind of workaround instead of a more graceful solution.
> 
> I wouldn't consider it a workaround.  It uses normal Unix
> permissions the way they were designed to, including setgid
> for directories.

Yes, I think core.sharedrepository is the "official" way to do this, so
it is definitely not a workaround.

> Actually, I am yet to come up with a situation where I
> actually needed ACLs, though they are more generalised, and
> fine-grained.

I like ACLs mainly because you don't have to bug root to change
permissions (like you do to get them to create or modify a group).

-Peff

Re: Error: unable to unlink ... when using "git gc"

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

On Tuesday 06 January 2009, Sitaram Chamarty <sitaramc@gmail.com> wrote 
about 'Re: Error: unable to unlink ... when using "git gc"':
>On 2009-01-06, Jeff King <peff@peff.net> wrote:
>> If you are going to have multiple users sharing a repository, generally
>> they should be in the same group and the core.sharedrepository config
>> option should be set (see "git help config", or the "shared" option to
>> git-init).
>If you're not worried about the finer-grained access control
>that acl(5) gives you, just do what "git init
>--shared=group" does:
>
>    git config core.sharedrepository 1 # as mentioned above
>    chmod g+ws .git
>
>Now set the group to something (I use "gitpushers" ;-)
>
>    chgrp -R gitpushers .git
>
>amd make sure all your users are part of that group.
>
>Works fine for small teams...

ISTR this breaking here when someone on the team had a umask like 077 and 
was using file:// or ssh:// to push.  I tended up "fixing" things with a 
cronjob, (which is a bit of a hack) IIRC.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss@iguanasuicide.net                     ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.net/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Error: unable to unlink ... when using "git gc"

[Sitaram Chamarty]

On 2009-01-07, Boyd Stephen Smith Jr. <bss@iguanasuicide.net> wrote:
> On Tuesday 06 January 2009, Sitaram Chamarty <sitaramc@gmail.com> wrote=20

>>    chmod g+ws .git
>>
>>Now set the group to something (I use "gitpushers" ;-)
>>
>>    chgrp -R gitpushers .git

> ISTR this breaking here when someone on the team had a umask like 077 and=20
> was using file:// or ssh:// to push.  I tended up "fixing" things with a=20
> cronjob, (which is a bit of a hack) IIRC.

That doesn't sound right.  "git help init" says:
 - 0xxx: 0xxx is an octal number and each file will have mode 0xxx
 - 0xxx will override users umask(2) value, and thus, users
   with a safe umask (0077) can use this option
 - 0660 is equivalent to group.

So when you say "group", you're saying "0660", and when you
say "0660", you're overriding users umask value.

I sorta-kinda tested it (some output of the "find|xargs ls"
is snipped for brevity):

    $ mkdir umt;cd umt
    $ umask         
    0022                                 
    $ git init --shared=group
    Initialized empty shared Git repository in /home/sitaram/t/umt/.git/
    (reverse-i-search)`find': sfind f|ack .git|map -x du -sk |sort -n
    $ find . -type d|xargs ls -ald     
    drwxr-xr-x 3 sitaram sitaram 4096 2009-01-07 15:59 .             
    drwxrwsr-x 7 sitaram sitaram 4096 2009-01-07 15:59 ./.git        

    $ dummy_commit                   
    Created initial commit afb2645: file-30824 at Wed Jan  7 15:59:23 IST 2009
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 100644 file-30824
    $ find . -type d|xargs ls -ald
    drwxr-xr-x 3 sitaram sitaram 4096 2009-01-07 15:59 .
    drwxrwsr-x 9 sitaram sitaram 4096 2009-01-07 15:59 ./.git
    drwxrwsr-x 2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/a7
    drwxrwsr-x 2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/af
    drwxrwsr-x 2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/ca

    $ umask 077
    $ jeeves dummy_commit
    Created commit 232f157: file-3025 at Wed Jan  7 15:59:36 IST 2009
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 100644 file-3025
    $ find . -type d|xargs ls -ald
    drwxr-xr-x  3 sitaram sitaram 4096 2009-01-07 15:59 .
    drwxrwsr-x  9 sitaram sitaram 4096 2009-01-07 15:59 ./.git
    drwxrws---  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/23
    drwxrwsr-x  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/a7
    drwxrws---  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/a8
    drwxrwsr-x  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/af
    drwxrws---  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/c7
    drwxrwsr-x  2 sitaram sitaram 4096 2009-01-07 15:59 ./.git/objects/ca

Re: Error: unable to unlink ... when using "git gc"

[Sitaram Chamarty]

On 2009-01-07, Sitaram Chamarty <sitaramc@gmail.com> wrote:
>     $ dummy_commit                   
>     $ jeeves dummy_commit

err, helper functions.  Should have mentioned.

I do a lot of playing around with git :-)

Re: Error: unable to unlink ... when using "git gc"

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1589 bytes --]

On Wednesday 2009 January 07 04:55:56 you wrote:
> On 2009-01-07, Boyd Stephen Smith Jr. <bss@iguanasuicide.net> wrote:
> > On Tuesday 06 January 2009, Sitaram Chamarty <sitaramc@gmail.com>
> > wrote=20
> >
> >>    chmod g+ws .git
> >>
> >>Now set the group to something (I use "gitpushers" ;-)
> >>
> >>    chgrp -R gitpushers .git
> >
> > ISTR this breaking here when someone on the team had a umask like 077
> > and=20 was using file:// or ssh:// to push.  I tended up "fixing" things
> > with a=20 cronjob, (which is a bit of a hack) IIRC.
>
> That doesn't sound right.  "git help init" says:
>  - 0xxx: 0xxx is an octal number and each file will have mode 0xxx
>  - 0xxx will override users umask(2) value, and thus, users
>    with a safe umask (0077) can use this option
>  - 0660 is equivalent to group.
>
> So when you say "group", you're saying "0660", and when you
> say "0660", you're overriding users umask value.

Very good then.  It is from when I was significantly less experienced in using 
git and managing repos, so I may have forgotten to use the correct options OR 
it could just have been the version of git I was using (1.4.4.4, IIRC) -- 
still using that in at least one place, as it is the current version in 
Debian Etch.

It's good to know it works properly now, if all the right switches are set.
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss@iguanasuicide.net                     ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.net/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Error: unable to unlink ... when using "git gc"

[Sitaram Chamarty]

On 2009-01-07, Boyd Stephen Smith Jr. <bss@iguanasuicide.net> wrote:

> On Wednesday 2009 January 07 04:55:56 you wrote:
>> So when you say "group", you're saying "0660", and when you
>> say "0660", you're overriding users umask value.

> it could just have been the version of git I was using (1.4.4.4, IIRC) --=20
> still using that in at least one place, as it is the current version in=20
> Debian Etch.

1.4.4.4 is 2 years and 2 days old today!  [I've heard
stories about Debian, but never thought it was this
conservative!]

and I think this was fixed in 06cbe85, last April.

Re: Error: unable to unlink ... when using "git gc"

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

On Wednesday 2009 January 07 12:00:22 Sitaram Chamarty wrote:
> On 2009-01-07, Boyd Stephen Smith Jr. <bss@iguanasuicide.net> wrote:
> > On Wednesday 2009 January 07 04:55:56 you wrote:
> >> So when you say "group", you're saying "0660", and when you
> >> say "0660", you're overriding users umask value.
> >
> > it could just have been the version of git I was using (1.4.4.4, IIRC)
> > -- still using that in at least one place, as it is the current
> > version in Debian Etch.
>
> 1.4.4.4 is 2 years and 2 days old today!  [I've heard
> stories about Debian, but never thought it was this
> conservative!]

Once a stable is released, no new versions of packages come in, only 
backported bug and security fixes.  The pre-release freeze also limits new 
versions from being considered.  Lenny should be out RSN, so I can move up to 
1.5.6.5. :)

$ apt-cache policy git-core
git-core:
  Installed: 1:1.4.4.4-4
  Candidate: 1:1.4.4.4-4
  Version table:
     1:1.6.0.6-1 0
        300 http://localhost experimental/main Packages
     1:1.5.6.5-2 0
        700 http://localhost testing/main Packages
        500 http://localhost unstable/main Packages
     1:1.5.6.5-1~bpo40+1 0
        800 http://localhost etch-backports/main Packages
 *** 1:1.4.4.4-4 0
        900 http://localhost stable/main Packages
        100 /var/lib/dpkg/status
     1:1.4.4.4-2.1+etch1 0
        900 http://localhost stable/updates/main Packages
-- 
Boyd Stephen Smith Jr.                     ,= ,-_-. =. 
bss@iguanasuicide.net                     ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy           `-'(. .)`-' 
http://iguanasuicide.net/                      \_/     

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Error: unable to unlink ... when using "git gc"

[Junio C Hamano]

"Johnny Lee" <johnnylee194@gmail.com> writes:

> While I'm looking at these "unable to unlink" files, it seems they are
> read only:
> git@tomato:~/golf$ ls -l .git/objects/16/
> total 4
> -r--r--r-- 1 johnny johnny 26 2009-01-05 09:25
> b14f5da9e2fcd6f3f38cc9e584cef2f3c90ebe


Check "ls -ld .git/objects/16" and see if it is owned by you and writable
by you.

In sane POSIX filesystem semantics, it should not pose with problem with
respect to removal that a file is unwritable.

What counts is the writability of the parent directory.

Re: Error: unable to unlink ... when using "git gc"

[Johnny Lee]

Thanks for your consideration Junio,

Peff has already helped me to figure out the root cause of this error
is a possible bad practice on collaboration work.

Here I attached the previous mail.

>>>>>>
Thanks Peff, I've checked the permission of .git/objects/16, it's
created by another user and thus I have no permission to remove it.

In fact, this is coming from a previous bad practice on setting up a
collaboration repository on a SSH server, here is what I've done so
far:
1. I have a server tomato, which has two users: "git" and "johnny"

2. User "git" has cloned a repository from elsewhere as ~git/golf/.git

3. User "johnny" has cloned this repository from git on another
machine, using command:
git clone johnny@tomato:~git/golf/.git localgit

4. It works fine, and user "johnny" has made some local commits and
wants to push the changes to git's repository, so he uses:
git push

5. But it's reported some errors about permissions, like:
error: Unable to append to logs/refs/heads/cupcake: Permission denied
error: Unable to append to logs/refs/remotes/origin/cupcake: Permission denied
error: Unable to append to logs/refs/remotes/origin/cupcake: Permission denied
ng refs/heads/cupcake failed to write
ng refs/remotes/origin/HEAD failed to write
ng refs/remotes/origin/cupcake failed to write
error: failed to push to 'johnny@tomato:~git/golf/.git'

6. That's normal, since the ~git/golf/.git is created by user "git",
and user "johnny" can't write it.

7. Then the user "git" has changed mode for all the files under .git
to writable.

8. This time, user "johnny" can push successfully.

9. But as you can see here, some files are created by user "johnny",
and thus user "git" can't write them.

It seems not a good practice for collaboration work, would you please
share your experiences with us?

Thanks very much and regards,
Johnny

On Tue, Jan 6, 2009 at 4:59 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Johnny Lee" <johnnylee194@gmail.com> writes:
>
>> While I'm looking at these "unable to unlink" files, it seems they are
>> read only:
>> git@tomato:~/golf$ ls -l .git/objects/16/
>> total 4
>> -r--r--r-- 1 johnny johnny 26 2009-01-05 09:25
>> b14f5da9e2fcd6f3f38cc9e584cef2f3c90ebe
>
>
> Check "ls -ld .git/objects/16" and see if it is owned by you and writable
> by you.
>
> In sane POSIX filesystem semantics, it should not pose with problem with
> respect to removal that a file is unwritable.
>
> What counts is the writability of the parent directory.
>



-- 
we all have our crosses to bear

Re: Error: unable to unlink ... when using "git gc"

[Matthieu Moy]

"Johnny Lee" <johnnylee194@gmail.com> writes:

> Thanks for your consideration Junio,
>
> Peff has already helped me to figure out the root cause of this error
> is a possible bad practice on collaboration work.
>
> Here I attached the previous mail.

See core.sharedRepository in man git-config or
http://www.kernel.org/pub/software/scm/git/docs/git-config.html

-- 
Matthieu