From: "Torsten Bögershausen" <tboegi@web.de>
To: Jochen <zwanzig12@googlemail.com>,
msysgit@googlegroups.com, "Torsten Bögershausen" <tboegi@web.de>,
"Git Mailing List" <git@vger.kernel.org>,
stefanbeller@googlemail.com
Subject: Re: Consecutive git gc fails on Windows network share
Date: Mon, 20 Jan 2014 18:16:16 +0100 [thread overview]
Message-ID: <52DD59E0.4090301@web.de> (raw)
In-Reply-To: <11936e14-7442-4601-8e97-b2062894975b@googlegroups.com>
(No top-posting, please see my comments below)
On 2014-01-20 15.02, Jochen wrote:
>On 01/16/2014 10:55 AM, Jochen Haag wrote:
> Hello,
>
> we want to report the following issue, because it seems to be not an
> intended behaviour. Maybe someone can have a look at it at some point.
> It is not urgent for us.
>
> After upgrading from Git version 1.8.1.msysgit.1 to 1.8.5.2.msysgit.0 we
> observed that a consecutive git gc fails, in case the Git repo is
> located on a Windows network share. Operating system used is Windows 7
> 64 bit SP1.
>
> In case git gc fails temporary packs and index remain in .git/objects/pack/.
>
> "Consecutive" means, that git gc is called on an already packed
> repository (e.g. git gc is called more than once).
>
> This is the output given in case of error:
>
> U:\GitEnv>git gc
> Counting objects: 139, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (71/71), done.
> Writing objects: 100% (139/139), done.
> Total 139 (delta 65), reused 139 (delta 65)
> fatal: renaming
> '.git/objects/pack/.tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack'
> failed: Permission denied
> error: failed to run repack
>
> And here the output with GIT_TRACE = 1:
>
> U:\GitEnv>git gc
> trace: built-in: git 'gc'
> trace: run_command: 'pack-refs' '--all' '--prune'
> trace: built-in: git 'pack-refs' '--all' '--prune'
> trace: run_command: 'reflog' 'expire' '--all'
> trace: built-in: git 'reflog' 'expire' '--all'
> trace: run_command: 'repack' '-d' '-l' '-A'
> '--unpack-unreachable=2.weeks.ago'
> trace: built-in: git 'repack' '-d' '-l' '-A'
> '--unpack-unreachable=2.weeks.ago'
> trace: run_command: 'pack-objects' '--keep-true-parents'
> '--honor-pack-keep' '--non-empty' '--all' '--reflog'
> '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
> '.git/objects/pack/.tmp-7612-pack'
> trace: built-in: git 'pack-objects' '--keep-true-parents'
> '--honor-pack-keep' '--non-empty' '--all' '--reflog'
> '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
> '.git/objects/pack/.tmp-7612-pack'
> Counting objects: 139, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (71/71), done.
> Writing objects: 100% (139/139), done.
> Total 139 (delta 65), reused 139 (delta 65)
> fatal: renaming
> '.git/objects/pack/.tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack'
> failed: Permission denied
> error: failed to run repack
>
> After running git gc twice, this is what is left in .git/objects/pack/:
>
> U:\GitEnv\.git\objects\pack>ls -al
> total 53676
> drwxr-xr-x 1 hgj2fe Administ 0 Jan 16 10:43 .
> drwxr-xr-x 1 hgj2fe Administ 0 Jan 14 12:52 ..
> -r--r--r-- 1 hgj2fe Administ 4964 Jan 16 10:43
> .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx
> -r--r--r-- 1 hgj2fe Administ 18315618 Jan 16 10:43
> .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack
> -r--r--r-- 1 hgj2fe Administ 4964 Jan 16 10:40
> .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx
> -r--r--r-- 1 hgj2fe Administ 18315618 Jan 16 10:40
> .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack
> -r--r--r-- 1 hgj2fe Administ 4964 Jan 14 12:52
> pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx
> -r--r--r-- 1 hgj2fe Administ 18315618 Jan 14 12:52
> pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack
>
> In case we remove the read-only flag of the pack and index manually
> before running git gc again, no problem occurs:
>
> U:\GitEnv\.git\objects\pack>git gc
> trace: built-in: git 'gc'
> trace: run_command: 'pack-refs' '--all' '--prune'
> trace: built-in: git 'pack-refs' '--all' '--prune'
> trace: run_command: 'reflog' 'expire' '--all'
> trace: built-in: git 'reflog' 'expire' '--all'
> trace: run_command: 'repack' '-d' '-l' '-A'
> '--unpack-unreachable=2.weeks.ago'
> trace: built-in: git 'repack' '-d' '-l' '-A'
> '--unpack-unreachable=2.weeks.ago'
> trace: run_command: 'pack-objects' '--keep-true-parents'
> '--honor-pack-keep' '--non-empty' '--all' '--reflog'
> '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
> 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack'
> trace: built-in: git 'pack-objects' '--keep-true-parents'
> '--honor-pack-keep' '--non-empty' '--all' '--reflog'
> '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
> 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack'
> Counting objects: 139, done.
> Delta compression using up to 8 threads.
> Compressing objects: 100% (71/71), done.
> Writing objects: 100% (139/139), done.
> Total 139 (delta 65), reused 139 (delta 65)
> trace: run_command: 'prune-packed'
> trace: built-in: git 'prune-packed'
> trace: run_command: 'update-server-info'
> trace: built-in: git 'update-server-info'
> trace: run_command: 'prune' '--expire' '2.weeks.ago'
> trace: built-in: git 'prune' '--expire' '2.weeks.ago'
> trace: run_command: 'rerere' 'gc'
> trace: built-in: git 'rerere' 'gc'
>
> The problem might be related to this commit:
> https://github.com/msysgit/git/commit/a1bbc6c0176f1fa2d4aa571cc0183a1f0ff9b285
>
>
> Best regards,
>
> Jochen
------
>
>
> Am Freitag, 17. Januar 2014 19:02:07 UTC+1 schrieb Torsten Bögershausen:
>
>
> So, please, what happens if you revert that commit?
> It could be good if you can test it and share the results with the list
> /Torsten
>
>
> Instead of reverting we did some more analysis.
>
> In repack.c we found the following code:
>
> /* Now the ones with the same name are out of the way... */
> for_each_string_list_item(item, &names) {
> for (ext = 0; ext < 2; ext++) {
> char *fname, *fname_old;
> struct stat statbuffer;
> fname = mkpathdup("%s/pack-%s%s",
> packdir, item->string, exts[ext]);
> fname_old = mkpathdup("%s-%s%s",
> packtmp, item->string, exts[ext]);
> if (!stat(fname_old, &statbuffer)) {
> statbuffer.st_mode &= ~(S_IWUSR | S_IWGRP | S_IWOTH);
> chmod(fname_old, statbuffer.st_mode);
> }
> if (rename(fname_old, fname))
> die_errno(_("renaming '%s' failed"), fname_old);
> free(fname);
> free(fname_old);
> }
> }
>
> The rename command replaces a mv -f command of the original shell script. Obviously the -f option can also override a read-only file but rename can not on a network share.
>
> We changed the code as followed:
>
> /* Now the ones with the same name are out of the way... */
> for_each_string_list_item(item, &names) {
> for (ext = 0; ext < 2; ext++) {
> char *fname, *fname_old;
> struct stat statbuffer;
> fname = mkpathdup("%s/pack-%s%s",
> packdir, item->string, exts[ext]);
> fname_old = mkpathdup("%s-%s%s",
> packtmp, item->string, exts[ext]);
> if (!stat(fname_old, &statbuffer)) {
> statbuffer.st_mode &= ~(S_IWUSR | S_IWGRP | S_IWOTH);
> chmod(fname_old, statbuffer.st_mode);
> }
>+++ if (!stat(fname, &statbuffer)) {
>+++ statbuffer.st_mode |= (S_IWUSR | S_IWGRP | S_IWOTH);
>+++ chmod(fname, statbuffer.st_mode);
>+++ }
> if (rename(fname_old, fname))
> die_errno(_("renaming '%s' failed"), fname_old);
> free(fname);
> free(fname_old);
> }
> }
>
> Before rename is called the destination file is made writeable. This worked for us. We are not sure if this is a good implementation.
Jochen,
thanks for sharing the code changes with us.
I allowed me to
a) reconstruct the original mail,
b) add "+++" at the places where you added the stat() and chmod(),
c) and to send the question "is this a good implementation ?" to upstream git.
I think your implementation makes sense.
/Torsten
--
--
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.
You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en
---
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
next parent reply other threads:[~2014-01-20 17:16 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <d10603d6-7740-44f8-909f-7ba1ea77d3a2@googlegroups.com>
[not found] ` <52D9701F.5070009@web.de>
[not found] ` <11936e14-7442-4601-8e97-b2062894975b@googlegroups.com>
2014-01-20 17:16 ` Torsten Bögershausen [this message]
2014-01-20 21:52 ` Consecutive git gc fails on Windows network share Stefan Beller
2014-01-20 23:25 ` [msysGit] " Johannes Schindelin
2014-01-20 23:53 ` Erik Faye-Lund
2014-01-21 16:57 ` Johannes Schindelin
2014-01-21 17:03 ` Erik Faye-Lund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52DD59E0.4090301@web.de \
--to=tboegi@web.de \
--cc=git@vger.kernel.org \
--cc=msysgit@googlegroups.com \
--cc=stefanbeller@googlemail.com \
--cc=zwanzig12@googlemail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.