* Re: [PATCH v3 18/19] setup.c: document get_pathspec()
From: Junio C Hamano @ 2012-12-28 20:36 UTC (permalink / raw)
To: Adam Spiers; +Cc: git list
In-Reply-To: <1356575558-2674-19-git-send-email-git@adamspiers.org>
Adam Spiers <git@adamspiers.org> writes:
> Since we have just created a new pathspec-handling library, now is a
> good time to add some comments explaining get_pathspec().
>
> Signed-off-by: Adam Spiers <git@adamspiers.org>
> ---
Yes, but we would rather not to see new users of this function added
to our codebase in its current form, as explained in the nearby
comment. We would want to migrate everybody to "struct pathspec"
based interface to support magic pathspecs in the longer term.
> setup.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/setup.c b/setup.c
> index 7663a4c..03d6d5c 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -249,6 +249,21 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
> return prefix_path(prefix, prefixlen, copyfrom);
> }
>
> +/*
> + * prefix - a path relative to the root of the working tree
> + * pathspec - a list of paths underneath the prefix path
> + *
> + * Iterates over pathspec, prepending each path with prefix,
> + * and return the resulting list.
> + *
> + * If pathspec is empty, return a singleton list containing prefix.
> + *
> + * If pathspec and prefix are both empty, return an empty list.
> + *
> + * This is typically used by built-in commands such as add.c, in order
> + * to normalize argv arguments provided to the built-in into a list of
> + * paths to process, all relative to the root of the working tree.
> + */
> const char **get_pathspec(const char *prefix, const char **pathspec)
> {
> const char *entry = *pathspec;
^ permalink raw reply
* Re: git diff --ignore-space-at-eol issue
From: Antoine Pelisse @ 2012-12-28 20:33 UTC (permalink / raw)
To: John Moon; +Cc: git
In-Reply-To: <BLU163-W51DAB90003C969CDF837A2CF3F0@phx.gbl>
Then you should give the output of diff --stat, and he will be able to
ignore files with no changes.
The change was originally made for permission changes. diff --stat
needs to show files have changed even though, indeed, there is no diff
output.
You could also use --numstat and filter out files with no changes
(starting by 0 0).
On Fri, Dec 28, 2012 at 8:59 PM, John Moon <johnmoon77@hotmail.com> wrote:
>
>
>> $ git diff --ignore-space-at-eol test.txt
>> $ git diff --ignore-space-at-eol --stat test.txt
>> test.txt | 0
>> 1 file changed, 0 insertions(+), 0 deletions(-)
>> $ git diff --ignore-space-at-eol --name-status test.txt
>> M test.txt
>>
>> The idea is that even though diff doesn't show any differences, stat,
>> shortstat, numstat and name-status reports the file as being changed.
>> This is available since v1.8.1-rc0.
>
> Thanks for the info. Unfortunately it's not what i would expect.
> If i told git diff specifically to ignore line endings, why is --name-status showing me a file as being modified when the only modification is the very thing i told it to ignore.
> The same thing for --stat, why is it showing me a file with zero changes? Just my opinion though.
>
> I'll tell you why this is a problem for me, basically what i am doing is running the "git diff --ignore-space-at-eol --name-status " on my root directory to give to someone else who is not using git to give them the files that i have modified. I don't want to give them a file where only the line ending has changed.
>
> Cheers.
^ permalink raw reply
* Re: [PATCH v3 16/19] pathspec.c: move reusable code from builtin/add.c
From: Junio C Hamano @ 2012-12-28 20:32 UTC (permalink / raw)
To: Adam Spiers; +Cc: git list
In-Reply-To: <1356575558-2674-17-git-send-email-git@adamspiers.org>
Adam Spiers <git@adamspiers.org> writes:
> diff --git a/pathspec.h b/pathspec.h
> new file mode 100644
> index 0000000..8bb670b
> --- /dev/null
> +++ b/pathspec.h
> @@ -0,0 +1,5 @@
> +extern char *find_used_pathspec(const char **pathspec);
> +extern void fill_pathspec_matches(const char **pathspec, char *seen, int specs);
> +extern const char *treat_gitlink(const char *path);
> +extern void treat_gitlinks(const char **pathspec);
> +extern const char **validate_pathspec(const char **argv, const char *prefix);
Protect this against multiple inclusion with "#ifndef PATHSPEC_H".
^ permalink raw reply
* Re: [PATCH v3 00/19] new git check-ignore sub-command
From: Antoine Pelisse @ 2012-12-28 20:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Adam Spiers, git list
In-Reply-To: <CAOkDyE8gfW9TvyP=iE7gVEXOqCpOqMRjpr=Vnyd_pnummy4Qsg@mail.gmail.com>
I think they will interact, but I need to have a deeper look to Adam's series.
If it does, do you want me to base my work on the top of his branch ?
On Fri, Dec 28, 2012 at 8:39 PM, Adam Spiers <git@adamspiers.org> wrote:
> On Fri, Dec 28, 2012 at 6:50 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Adam Spiers <git@adamspiers.org> writes:
>>
>>> This v3 re-roll of my check-ignore series is a reasonably substantial
>>> revamp over v2, and applies on top of Junio's current
>>> nd/attr-match-optim-more branch (82dce998c202).
>>
>> Thanks.
>>
>> Does this (and should this, if it doesn't) interact with the more
>> recent discussion around "git status --untracked/--ignored" [*1*],
>> which also wants to touch the recursive directory traversal logic in
>> "dir.c"?
>
> I cannot think of a reason why they would or should interact. If I'm
> wrong, I expect that either set of unit tests would show me up :-)
^ permalink raw reply
* RE: git diff --ignore-space-at-eol issue
From: John Moon @ 2012-12-28 19:59 UTC (permalink / raw)
To: apelisse; +Cc: git
In-Reply-To: <CALWbr2y3BdqcD-62jhPSQsK3U=8-Dc=R-jxg8H0yqpgVfdHJXw@mail.gmail.com>
> $ git diff --ignore-space-at-eol test.txt
> $ git diff --ignore-space-at-eol --stat test.txt
> test.txt | 0
> 1 file changed, 0 insertions(+), 0 deletions(-)
> $ git diff --ignore-space-at-eol --name-status test.txt
> M test.txt
>
> The idea is that even though diff doesn't show any differences, stat,
> shortstat, numstat and name-status reports the file as being changed.
> This is available since v1.8.1-rc0.
Thanks for the info. Unfortunately it's not what i would expect.
If i told git diff specifically to ignore line endings, why is --name-status showing me a file as being modified when the only modification is the very thing i told it to ignore.
The same thing for --stat, why is it showing me a file with zero changes? Just my opinion though.
I'll tell you why this is a problem for me, basically what i am doing is running the "git diff --ignore-space-at-eol --name-status " on my root directory to give to someone else who is not using git to give them the files that i have modified. I don't want to give them a file where only the line ending has changed.
Cheers.
^ permalink raw reply
* Re: [PATCH v3 00/19] new git check-ignore sub-command
From: Adam Spiers @ 2012-12-28 19:39 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git list, Antoine Pelisse
In-Reply-To: <7v38yqj9ix.fsf@alter.siamese.dyndns.org>
On Fri, Dec 28, 2012 at 6:50 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Adam Spiers <git@adamspiers.org> writes:
>
>> This v3 re-roll of my check-ignore series is a reasonably substantial
>> revamp over v2, and applies on top of Junio's current
>> nd/attr-match-optim-more branch (82dce998c202).
>
> Thanks.
>
> Does this (and should this, if it doesn't) interact with the more
> recent discussion around "git status --untracked/--ignored" [*1*],
> which also wants to touch the recursive directory traversal logic in
> "dir.c"?
I cannot think of a reason why they would or should interact. If I'm
wrong, I expect that either set of unit tests would show me up :-)
^ permalink raw reply
* Re: (unknown)
From: Junio C Hamano @ 2012-12-28 19:33 UTC (permalink / raw)
To: Eric S. Raymond; +Cc: git
In-Reply-To: <20121228164322.B102B4413A@snark.thyrsus.com>
esr@thyrsus.com (Eric S. Raymond) writes:
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Date: Fri, 28 Dec 2012 11:40:59 -0500
> Subject: [PATCH] Add checks to Python scripts for version dependencies.
>
> ---
> contrib/ciabot/ciabot.py | 8 +++++++-
> ...
> diff --git a/contrib/fast-import/import-zips.py b/contrib/fast-import/import-zips.py
> index 82f5ed3..b989941 100755
> --- a/contrib/fast-import/import-zips.py
> +++ b/contrib/fast-import/import-zips.py
> @@ -9,10 +9,15 @@
> ## git log --stat import-zips
>
> from os import popen, path
> -from sys import argv, exit
> +from sys import argv, exit, hexversion
> from time import mktime
> from zipfile import ZipFile
>
> +if hexversion < 0x01060000:
I am assuming that you are carefully limiting what you import from
"sys" by adding only hexversion to the import above, but then can we
refer to sys.stderr below?
> + # The limiter is the zipfile module
> + sys.stderr.write("import-zips.py: requires Python 1.6.0 or later.\n")
> + sys.exit(1)
> +
^ permalink raw reply
* Re: [PATCH] Remove the suggestion to use parsecvs, which is currently broken.
From: Junio C Hamano @ 2012-12-28 19:28 UTC (permalink / raw)
To: Eric S. Raymond; +Cc: git
In-Reply-To: <20121228162025.8565E4413A@snark.thyrsus.com>
esr@thyrsus.com (Eric S. Raymond) writes:
> The parsecvs code has been neglected for a long time, and the only
> public version does not even build correctly. I have been handed
> control of the project and intend to fix this, but until I do it
> cannot be recommended.
>
> Also, the project URL given for Subversion needed to be updated
> to follow their site move.
> ---
> Documentation/git-cvsimport.txt | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
> ...
> --
> <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
>
> A ``decay in the social contract'' is detectable; there is a growing
> feeling, particularly among middle-income taxpayers, that they are not
> getting back, from society and government, their money's worth for
> taxes paid. The tendency is for taxpayers to try to take more control
> of their finances... -- IRS Strategic Plan, (May 1984)
It is funny that you keep forgetting to sign-off your patches and
even send a message with no subject, but still manage to add a
pointer to your homepage and stuff at the end.
Perhaps you would need to form a habit to do "commit -s"?
^ permalink raw reply
* Re: [PATCH v3 00/19] new git check-ignore sub-command
From: Junio C Hamano @ 2012-12-28 18:50 UTC (permalink / raw)
To: Adam Spiers; +Cc: git list, Antoine Pelisse
In-Reply-To: <1356575558-2674-1-git-send-email-git@adamspiers.org>
Adam Spiers <git@adamspiers.org> writes:
> This v3 re-roll of my check-ignore series is a reasonably substantial
> revamp over v2, and applies on top of Junio's current
> nd/attr-match-optim-more branch (82dce998c202).
Thanks.
Does this (and should this, if it doesn't) interact with the more
recent discussion around "git status --untracked/--ignored" [*1*],
which also wants to touch the recursive directory traversal logic in
"dir.c"?
[Reference]
*1* http://thread.gmane.org/gmane.comp.version-control.git/212127/focus=212136
^ permalink raw reply
* Re: [PATCH v2] log: grep author/committer using mailmap
From: Junio C Hamano @ 2012-12-28 18:43 UTC (permalink / raw)
To: Antoine Pelisse; +Cc: git
In-Reply-To: <CALWbr2y63L0-ykdUNGqUOb0LhG=vpXGRcb1KkvssEZmKFJEGeQ@mail.gmail.com>
Antoine Pelisse <apelisse@gmail.com> writes:
> Actually, gprof seems to be unhappy about the number of call to
> strbuf_grow() in map_user() (25% of the time spent in map_user() is
> spent in strbuf_grow()).
>
> That probably comes from the repeated call to strbuf_addch() when
> lowering the email address.
This is about your rewritten implementation that hasn't escaped to
the general public but sitting in 'next', right?
Two things that immediately come to mind are:
- initialization of lowermail can use strbuf_init() instead;
- downcasing can be done in place, i.e. "lowermail.buf[i] = ...".
^ permalink raw reply
* Re: [PATCH v2] log: grep author/committer using mailmap
From: Antoine Pelisse @ 2012-12-28 18:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwqw3l49z.fsf@alter.siamese.dyndns.org>
Actually, gprof seems to be unhappy about the number of call to
strbuf_grow() in map_user() (25% of the time spent in map_user() is
spent in strbuf_grow()).
That probably comes from the repeated call to strbuf_addch() when
lowering the email address.
At this point, we are also copying the '\0' for every char we add,
doubling the copy.
This may not be much of a difference, but it seems to be called 15
millions times when running:
$ git log --author='Junio C Hamano' --use-mailmap
Maybe we should come up with another way to lower this email address afterall.
^ permalink raw reply
* Re: Lockless Refs?
From: Junio C Hamano @ 2012-12-28 17:15 UTC (permalink / raw)
To: Martin Fick; +Cc: Michael Haggerty, Jeff King, git, Shawn Pearce
In-Reply-To: <201212280750.14695.mfick@codeaurora.org>
Martin Fick <mfick@codeaurora.org> writes:
> Hmm, actually I believe that with a small modification to the
> semantics described here it would be possible to make multi
> repo/branch commits work....
>
> Shawn talked about adding multi repo/branch transaction
> semantics to jgit, this might be something that git wants to
> support also at some point?
Shawn may have talked about it and you may have listened to it, but
others wouldn't have any idea what kind of "multi repo/branch
transaction" you are talking about. Is it about "I want to push
this ref to that repo and push this other ref to that other repo",
in what situation will it be used/useful, what are the failure
modes, what are failure tolerances by the expected use cases, ...?
Care to explain?
^ permalink raw reply
* Re: Lockless Refs?
From: Junio C Hamano @ 2012-12-28 16:58 UTC (permalink / raw)
To: Martin Fick; +Cc: Michael Haggerty, Jeff King, git
In-Reply-To: <201212271611.52203.mfick@codeaurora.org>
Martin Fick <mfick@codeaurora.org> writes:
> 3) To create a ref, it must be renamed from the null file (sha
> 0000...) to the new value just as if it were being updated
> from any other value, but there is one extra condition:
> before renaming the null file, a full directory scan must be
> done to ensure that the null file is the only file in the
> directory...
While you are scanning this directory to make sure it is empty, I am
contemplating to create the same ref with a different value. You
finished checking but haven't created the null. I have also scanned,
created the null and renamed it to my value. Now you try to create
the null, succeed, and then rename. We won't know which of the two
non-null values are valid, but worse yet, I think one of them should
have failed in the first place.
Sounds like we would need some form of locking around here. Is your
goal "no locks", or "less locks"?
> I don't know how this new scheme could be made to work with
> the current scheme,...
It is much more important to know if/why yours is better than the
current scheme in the first place. Without an analysis on how the
new scheme interacts with the packed refs and gives better
behaviour, that is kinda difficult.
I think transition plans can wait until that is done. If it is not
even marginally better, we do not have to worry about transitioning
at all. If it is only marginally better, the transition has to be
designed to be no impact to the existing repositories. If it is
vastly better, we might be able to afford a flag day.
^ permalink raw reply
* Re: git diff --ignore-space-at-eol issue
From: Antoine Pelisse @ 2012-12-28 16:46 UTC (permalink / raw)
To: John Moon; +Cc: git
In-Reply-To: <BLU163-W40634B340214076467C88ECF360@phx.gbl>
> The --ignore-space-at-eol option is ignored when used in conjunction
> with --name-status.
> It works fine otherwise.
Indeed the behavior of diff --stat, and etc has been corrected very
recently to make it more consistent across all options.
I don't know if the new behavior is exactly what you expected:
$ git diff --ignore-space-at-eol test.txt
$ git diff --ignore-space-at-eol --stat test.txt
test.txt | 0
1 file changed, 0 insertions(+), 0 deletions(-)
$ git diff --ignore-space-at-eol --name-status test.txt
M test.txt
The idea is that even though diff doesn't show any differences, stat,
shortstat, numstat and name-status reports the file as being changed.
This is available since v1.8.1-rc0.
^ permalink raw reply
* (unknown)
From: Eric S. Raymond @ 2012-12-28 16:43 UTC (permalink / raw)
To: git
From: "Eric S. Raymond" <esr@thyrsus.com>
Date: Fri, 28 Dec 2012 11:40:59 -0500
Subject: [PATCH] Add checks to Python scripts for version dependencies.
---
contrib/ciabot/ciabot.py | 8 +++++++-
contrib/fast-import/import-zips.py | 7 ++++++-
contrib/hg-to-git/hg-to-git.py | 5 +++++
contrib/p4import/git-p4import.py | 5 +++++
contrib/svn-fe/svnrdump_sim.py | 4 ++++
git-p4.py | 8 +++++++-
git-remote-testgit.py | 5 +++++
git_remote_helpers/git/__init__.py | 5 +++++
8 files changed, 44 insertions(+), 3 deletions(-)
diff --git a/contrib/ciabot/ciabot.py b/contrib/ciabot/ciabot.py
index bd24395..81c3ebd 100755
--- a/contrib/ciabot/ciabot.py
+++ b/contrib/ciabot/ciabot.py
@@ -47,7 +47,13 @@
# we default to that.
#
-import os, sys, commands, socket, urllib
+import sys
+if sys.hexversion < 0x02000000:
+ # The limiter is the xml.sax module
+ sys.stderr.write("ciabot.py: requires Python 2.0.0 or later.\n")
+ sys.exit(1)
+
+import os, commands, socket, urllib
from xml.sax.saxutils import escape
# Changeset URL prefix for your repo: when the commit ID is appended
diff --git a/contrib/fast-import/import-zips.py b/contrib/fast-import/import-zips.py
index 82f5ed3..b989941 100755
--- a/contrib/fast-import/import-zips.py
+++ b/contrib/fast-import/import-zips.py
@@ -9,10 +9,15 @@
## git log --stat import-zips
from os import popen, path
-from sys import argv, exit
+from sys import argv, exit, hexversion
from time import mktime
from zipfile import ZipFile
+if hexversion < 0x01060000:
+ # The limiter is the zipfile module
+ sys.stderr.write("import-zips.py: requires Python 1.6.0 or later.\n")
+ sys.exit(1)
+
if len(argv) < 2:
print 'Usage:', argv[0], '<zipfile>...'
exit(1)
diff --git a/contrib/hg-to-git/hg-to-git.py b/contrib/hg-to-git/hg-to-git.py
index 046cb2b..232625a 100755
--- a/contrib/hg-to-git/hg-to-git.py
+++ b/contrib/hg-to-git/hg-to-git.py
@@ -23,6 +23,11 @@ import os, os.path, sys
import tempfile, pickle, getopt
import re
+if sys.hexversion < 0x02030000:
+ # The behavior of the pickle module changed significantly in 2.3
+ sys.stderr.write("hg-to-git.py: requires Python 2.3 or later.\n")
+ sys.exit(1)
+
# Maps hg version -> git version
hgvers = {}
# List of children for each hg revision
diff --git a/contrib/p4import/git-p4import.py b/contrib/p4import/git-p4import.py
index b6e534b..593d6a0 100644
--- a/contrib/p4import/git-p4import.py
+++ b/contrib/p4import/git-p4import.py
@@ -14,6 +14,11 @@ import sys
import time
import getopt
+if sys.hexversion < 0x02020000:
+ # The behavior of the marshal module changed significantly in 2.2
+ sys.stderr.write("git-p4import.py: requires Python 2.2 or later.\n")
+ sys.exit(1)
+
from signal import signal, \
SIGPIPE, SIGINT, SIG_DFL, \
default_int_handler
diff --git a/contrib/svn-fe/svnrdump_sim.py b/contrib/svn-fe/svnrdump_sim.py
index 1cfac4a..95a80ae 100755
--- a/contrib/svn-fe/svnrdump_sim.py
+++ b/contrib/svn-fe/svnrdump_sim.py
@@ -7,6 +7,10 @@ to the highest revision that should be available.
"""
import sys, os
+if sys.hexversion < 0x02040000:
+ # The limiter is the ValueError() calls. This may be too conservative
+ sys.stderr.write("svnrdump-sim.py: requires Python 2.4 or later.\n")
+ sys.exit(1)
def getrevlimit():
var = 'SVNRMAX'
diff --git a/git-p4.py b/git-p4.py
index 551aec9..69f1452 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -8,7 +8,13 @@
# License: MIT <http://www.opensource.org/licenses/mit-license.php>
#
-import optparse, sys, os, marshal, subprocess, shelve
+import sys
+if sys.hexversion < 0x02040000:
+ # The limiter is the subprocess module
+ sys.stderr.write("git-p4: requires Python 2.4 or later.\n")
+ sys.exit(1)
+
+import optparse, os, marshal, subprocess, shelve
import tempfile, getopt, os.path, time, platform
import re, shutil
diff --git a/git-remote-testgit.py b/git-remote-testgit.py
index 5f3ebd2..91faabd 100644
--- a/git-remote-testgit.py
+++ b/git-remote-testgit.py
@@ -31,6 +31,11 @@ from git_remote_helpers.git.exporter import GitExporter
from git_remote_helpers.git.importer import GitImporter
from git_remote_helpers.git.non_local import NonLocalGit
+if sys.hexversion < 0x01050200:
+ # os.makedirs() is the limiter
+ sys.stderr.write("git-remote-testgit: requires Python 1.5.2 or later.\n")
+ sys.exit(1)
+
def get_repo(alias, url):
"""Returns a git repository object initialized for usage.
"""
diff --git a/git_remote_helpers/git/__init__.py b/git_remote_helpers/git/__init__.py
index e69de29..1dbb1b0 100644
--- a/git_remote_helpers/git/__init__.py
+++ b/git_remote_helpers/git/__init__.py
@@ -0,0 +1,5 @@
+import sys
+if sys.hexversion < 0x02040000:
+ # The limiter is the subprocess module
+ sys.stderr.write("git_remote_helpers: requires Python 2.4 or later.\n")
+ sys.exit(1)
--
1.8.1.rc2
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
A ``decay in the social contract'' is detectable; there is a growing
feeling, particularly among middle-income taxpayers, that they are not
getting back, from society and government, their money's worth for
taxes paid. The tendency is for taxpayers to try to take more control
of their finances... -- IRS Strategic Plan, (May 1984)
^ permalink raw reply related
* [PATCH] Remove the suggestion to use parsecvs, which is currently broken.
From: Eric S. Raymond @ 2012-12-28 16:20 UTC (permalink / raw)
To: git
The parsecvs code has been neglected for a long time, and the only
public version does not even build correctly. I have been handed
control of the project and intend to fix this, but until I do it
cannot be recommended.
Also, the project URL given for Subversion needed to be updated
to follow their site move.
---
Documentation/git-cvsimport.txt | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-cvsimport.txt b/Documentation/git-cvsimport.txt
index 98d9881..9d5353e 100644
--- a/Documentation/git-cvsimport.txt
+++ b/Documentation/git-cvsimport.txt
@@ -213,11 +213,9 @@ Problems related to tags:
* Multiple tags on the same revision are not imported.
If you suspect that any of these issues may apply to the repository you
-want to import consider using these alternative tools which proved to be
-more stable in practice:
+want to imort, consider using cvs2git:
-* cvs2git (part of cvs2svn), `http://cvs2svn.tigris.org`
-* parsecvs, `http://cgit.freedesktop.org/~keithp/parsecvs`
+* cvs2git (part of cvs2svn), `http://subversion.apache.org/`
GIT
---
--
1.8.1.rc2
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
A ``decay in the social contract'' is detectable; there is a growing
feeling, particularly among middle-income taxpayers, that they are not
getting back, from society and government, their money's worth for
taxes paid. The tendency is for taxpayers to try to take more control
of their finances... -- IRS Strategic Plan, (May 1984)
^ permalink raw reply related
* Re: Lockless Refs? (Was [PATCH] refs: do not use cached refs in repack_without_ref)
From: Martin Fick @ 2012-12-28 14:50 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Jeff King, git, Junio C Hamano, Shawn Pearce
In-Reply-To: <201212271611.52203.mfick@codeaurora.org>
On Thursday, December 27, 2012 04:11:51 pm Martin Fick wrote:
> On Wednesday, December 26, 2012 01:24:39 am Michael
> Haggerty
>
> wrote:
> > ... lots of discussion about ref locking...
>
> It concerns me that git uses any locking at all, even for
> refs since it has the potential to leave around stale
> locks.
>
> For a single user repo this is not a big deal, the lock
> can always be cleaned up manually (and it is a rare
> occurrence). However, in a multi user server environment,
> possibly even from multiple hosts over a shared
> filesystem such as NFS, stale locks could lead to serious
> downtime and risky recovery (since it is currently hard
> to figure out if a lock really is stale). Even though
> stale locks are probably rare even today in the larger
> shared repo case, as git scales to even larger shared
> repositories, this will eventually become more of a
> problem *1. Naturally, this has me thinking that git
> should possibly consider moving towards a lockless design
> for refs in the long term.
>
> I realize this is hard and that git needs to support many
> different filesystems with different semantics. I had an
> idea I think may be close to a functional lockless design
> for loose refs (one piece at a time) that I thought I
> should propose, just to get the ball rolling, even if it
> is just going to be found to be flawed (I realize that
> history suggests that such schemes usually are). I hope
> that it does not make use of any semantics which are not
> currently expected from git of filesystems. I think it
> relies only on the ability to rename a file atomically,
> and the ability to scan the contents of a directory
> reliably to detect the "ordered" existence of files.
>
> My idea is based on using filenames to store sha1s instead
> of file contents. To do this, the sha1 one of a ref
> would be stored in a file in a directory named after the
> loose ref. I believe this would then make it possible to
> have lockless atomic ref updates by renaming the file.
>
> To more fully illustrate the idea, imagine that any file
> (except for the null file) in the directory will represent
> the value of the ref with its name, then the following
> transitions can represent atomic state changes to a refs
> value and existence:
>
> 1) To update the value from a known value to a new value
> atomically, simply rename the file to the new value. This
> operation should only succeed if the file exists and is
> still named old value before the rename. This should
> even be faster than today's approach, especially on
> remote filesystems since it would require only 1 round
> trip in the success case instead of 3!
>
> 2) To delete the ref, simply delete the filename
> representing the current value of the ref. This ensures
> that you are deleting the ref from a specific value. I
> am not sure if git needs to be able to delete refs
> without knowing their values? If so, this would require
> reading the value and looping until the delete succeeds,
> this may be a bit slow for a constantly updated ref, but
> likely a rare situation (and not likely worse than trying
> to acquire the ref-lock today). Overall, this again
> would likely be faster than today's approach.
>
> 3) To create a ref, it must be renamed from the null file
> (sha 0000...) to the new value just as if it were being
> updated from any other value, but there is one extra
> condition: before renaming the null file, a full
> directory scan must be done to ensure that the null file
> is the only file in the directory (this condition exists
> because creating the directory and null file cannot be
> atomic unless the filesystem supports atomic directory
> renames, an expectation git does not currently make). I
> am not sure how this compares to today's approach, but
> including the setup costs (described below), I suspect it
> is slower.
>
> While this outlines the state changes, some additional
> operations may be needed to setup some starting conditions
> and to clean things up. But these operations could/should
> be performed by any process/thread and would not cause
> any state changes to the ref existence or value. For
> example, when creating a ref, the ref directory would
> need to be created and the null file needs to be created.
> Whenever a null file is detected in the directory at the
> same time as another file, it should be deleted.
> Whenever the directory is empty, it may be deleted
> (perhaps after a grace period to reduce retries during
> ref creation unless the process just deleted the ref).
>
> I don't know how this new scheme could be made to work
> with the current scheme, it seems like perhaps new git
> releases could be made to understand both the old and the
> new, and a config option could be used to tell it which
> method to write new refs with. Since in this new scheme
> ref directory names would conflict with old ref
> filenames, this would likely prevent both schemes from
> erroneously being used
> simultaneously (so they shouldn't corrupt each other),
> except for the fact that refs can be nested in
> directories which confuses things a bit. I am not sure
> what a good solution to this is?
>
> What did I miss, where are my flaws? Does anyone else
> share my concern for stale locks? How could we similarly
> eliminate locks for the packed-refs file?
>
> -Martin
>
>
> *1 We have been concerned with stale locks in the Gerrit
> community when trying to design atomic cross repository
> updates. Of course, while a lockless solution eliminates
> stale locks, it might make it impossible to do atomic
> cross repository updates since all of our solutions so
> far need locks. :(
Hmm, actually I believe that with a small modification to the
semantics described here it would be possible to make multi
repo/branch commits work. Simply allow the ref filename to
be locked by a transaction by appending the transaction ID to
the filename. So if transaction 123 wants to lock master
which points currently to abcde, then it will move
master/abcde to master/abcde_123. If transaction 123 is
designed so that any process can commit/complete/abort it
without requiring any locks which can go stale, then this ref
lock will never go stale either (easy as long as it writes
all its proposed updates somewhere upfront and has atomic
semantics for starting, committing and aborting). On commit,
the ref lock gets updated to its new value: master/newsha and
on abort it gets unlocked: master/abcde.
Shawn talked about adding multi repo/branch transaction
semantics to jgit, this might be something that git wants to
support also at some point?
-Martin
^ permalink raw reply
* Re: [PATCH v2] wt-status: Show ignored files in untracked dirs
From: Antoine Pelisse @ 2012-12-28 14:05 UTC (permalink / raw)
To: Jeff King; +Cc: Junio C Hamano, git
In-Reply-To: <CALWbr2xmtvchR4G37-FzzgScKe4p4RjLc7=Pg8d4K6SWO7tGAQ@mail.gmail.com>
Hey Peff,
I actually have an issue with the behavior we discussed (referenced as 1.A.)
Using the example from Michael's mail, I end up having this:
$ git status --porcelain --ignored
?? .gitignore
?? x
?? y/
!! x.ignore-me
!! y/
y/ is referred as untracked, because it contains untracked files, and
then as ignored because it
contains ignored files.
Showing it twice doesn't feel right though, so I guess we should only
show "?? y/" with untracked=normal,
and "!! y/foo.ignore-me" when using untracked=all
What do you think ?
On Thu, Dec 27, 2012 at 6:35 PM, Antoine Pelisse <apelisse@gmail.com> wrote:
>> By "committed", I assume you meat that you have "dirA/unco" as an
>> untracked file, and "dirA/committed" as a file in the index?
>
> Of course,
>
>> Thanks for putting this together. I agree with the expected output in
>> each case, and I think this covers the cases we have seen (case 1 is
>> Michael's original report, case 2 is what I wrote in my mail, and case 3
>> is the one you just came up with). I can't think offhand of any others.
>
> Great, so I can build some tests reflecting those behaviors while
> waiting more inputs
^ permalink raw reply
* Re: Installation Plan
From: Enrico Weigelt @ 2012-12-28 13:54 UTC (permalink / raw)
To: Dennis Putnam; +Cc: git
In-Reply-To: <50D475A9.9020407@bellsouth.net>
> 7) Clone new repository for development and testing on Windows. (Do I
> need the shared drive any more?)
Not necessarily, depending on how to connect your local repo to the
remote one (your central repo). I'd suggest using ssh protocol: in this
case your windows box will connect to the linux box via ssh and do all
operations via ssh - no network filesystem required.
> 8) When a new version is ready for release, push commit to remote
> repository after which builds will use new code (I'm assuming the
> file copies happen automagically).
Yes, see post-update hook (on the central repo side).
It is executed right after objects have been transfered and refs updated.
(IOW: when your changes made finally it into the cental repo).
Note that the 'git push' operation will wait until that hook is finished.
So, if the build takes a while, you most likely want to do it asychronously.
A nice way is letting the hook just add the new refs to some queue
(you can even use git refs for that) and have another process (in a loop
or via cron) polling for new queue entries and run the build.
cu
--
Mit freundlichen Grüßen / Kind regards
Enrico Weigelt
VNC - Virtual Network Consult GmbH
Head Of Development
Pariser Platz 4a, D-10117 Berlin
Tel.: +49 (30) 3464615-20
Mobile: +49 (151) 27565287
Fax: +49 (30) 3464615-59
enrico.weigelt@vnc.biz; www.vnc.de
^ permalink raw reply
* Re: [PATCH 4/8] wildmatch: support "no FNM_PATHNAME" mode
From: Nguyen Thai Ngoc Duy @ 2012-12-28 7:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwqw2k833.fsf@alter.siamese.dyndns.org>
On Fri, Dec 28, 2012 at 1:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> if (*++p == '*') {
>> const uchar *prev_p = p - 2;
>> while (*++p == '*') {}
>> - if ((prev_p == text || *prev_p == '/') ||
>> + if (!(flags & WM_PATHNAME))
>> + /* without WM_PATHNAME, '*' == '**' */
>> + special = 1;
>> + else if ((prev_p == text || *prev_p == '/') ||
>
> Not a new issue in this patch,
No, it's an issue from nd/wildmatch, 40bbee0 (wildmatch: adjust "**"
behavior - 2012-10-15).
> but here, "prev_p" points into the
> pattern string, two bytes before p, which is the byte before the
> "**" that we are looking at (which might be before the beginning of
> the pattern). "text" is the string we are trying to match that
> pattern against. How can these two pointers be compared to yield a
> meaningful value?
They can't. I wanted to check whether "**" is at the start of the
pattern (so no preceding '/' needed) and used a wrong pointer to
compare to. Funny there is a test about this and it does not catch it
because prev_p access something before the pattern. Will fix.
>
>> (*p == '\0' || *p == '/' ||
>> (p[0] == '\\' && p[1] == '/'))) {
>
> OK. "**/", "**" (end of pattern), and "**\/" are handled here.
>
> Do we have to worry about "**[/]" the same way, or a class never
> matches the directory separator, even if it is a singleton class
> that consists of '/' (which is fine by me)? If so, is "\/" more or
> less like "[/]"?
This is a special case of "**" with FNM_PATHNAME on. With
FNM_PATHNAME, '[]' and '?' cannot match '/' so any patterns with '[/]'
match nothing. I think we don't need to worry about this case.
--
Duy
^ permalink raw reply
* Re: [PATCH 8/8] wildmatch: advance faster in <asterisk> + <literal> patterns
From: Nguyen Thai Ngoc Duy @ 2012-12-28 6:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vpq1uk82q.fsf@alter.siamese.dyndns.org>
On Fri, Dec 28, 2012 at 1:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> + while ((t_ch = *text) != '\0' &&
>> + (!(flags & WM_PATHNAME) || t_ch != '/')) {
>
> Why do we look at (flags & WM_PATHMAME) and not "special" here?
Because I was careless. Thanks for spotting it. I'll fix it and add
some more tests about **<literal> with WM_PATHNAME.
--
Duy
^ permalink raw reply
* Re: Find the starting point of a local branch
From: Martin von Zweigbergk @ 2012-12-28 6:38 UTC (permalink / raw)
To: Woody Wu; +Cc: Seth Robertson, git
In-Reply-To: <20121228051514.GA4028@zuhnb712.ap.bm.net>
On Thu, Dec 27, 2012 at 9:15 PM, Woody Wu <narkewoody@gmail.com> wrote:
> On Mon, Dec 24, 2012 at 09:24:39AM -0800, Martin von Zweigbergk wrote:
>> On Sun, Dec 23, 2012 at 11:31 PM, Woody Wu <narkewoody@gmail.com> wrote:
>> >
>> > This is not working to me since I have more than one local branch that
>> > diverged from the master, and in fact, the branch I have in question was
>> > diverged from another local branch.
>>
>> As Jeff mentions in a later message, "git pull --rebase" would
>> probably do what you want. It works with local branches too.
>>
>
> I think what 'git pull --rebase' would do is to fetch from the origin
> and do a 'git rebase'.
Not if the configured upstream is a local branch (see the
"branch.<name>.*" configuration variables). In that case it will just
rebase the local branch onto the new position of its upstream. If the
upstream is not configured, I believe you can still do "git pull
--rebase . <upstream branch>".
> On one hand, I don't understand 'git rebase' so
> much from the manual, ont the other hand, I did not get the point why
> 'git rebase' has something to do with the thing I want to do (what I
> want is just query some kind of history information).
I may have misunderstood or assumed things incorrectly that you wanted
to rebase the commits on your branch. So why do you want to know?
(Please ignore me if this was answered elsewhere in the thread that I
might have missed.)
Anyway, to answer your question, you could use a method similar to
what "git pull --rebase" uses internally to figure out the branch
point:
git merge-base $(git rev-parse <branch>) $(git rev-list -g <upstream branch>)
Hope that helps
^ permalink raw reply
* Re: [PATCH 8/8] wildmatch: advance faster in <asterisk> + <literal> patterns
From: Junio C Hamano @ 2012-12-28 6:24 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git
In-Reply-To: <1356163028-29967-9-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> compat, '*/*/*' on linux-2.6.git file list 2000 times, before:
> wildmatch 7s 985049us
> fnmatch 2s 735541us or 34.26% faster
>
> and after:
> wildmatch 4s 492549us
> fnmatch 0s 888263us or 19.77% slower
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> wildmatch.c | 21 +++++++++++++++++++++
> 1 file changed, 21 insertions(+)
>
> diff --git a/wildmatch.c b/wildmatch.c
> index 3794c4d..68b02e4 100644
> --- a/wildmatch.c
> +++ b/wildmatch.c
> @@ -132,6 +132,27 @@ static int dowild(const uchar *p, const uchar *text, unsigned int flags)
> while (1) {
> if (t_ch == '\0')
> break;
> + /*
> + * Try to advance faster when an asterisk is
> + * followed by a literal. We know in this case
> + * that the the string before the literal
> + * must belong to "*".
> + */
> + if (!is_glob_special(*p)) {
So far, we have looked at "*x" or "**x" in the pattern, p points at
'x' (not an asterisk), and we have "text" to match. For "text" to
match this pattern, the earlier part of it that is consumed to match
the asterisk must be followed by "x". "special" tells us if we are
allowed to treat '/' as matching the asterisk.
> + p_ch = *p;
> + if ((flags & WM_CASEFOLD) && ISUPPER(p_ch))
> + p_ch = tolower(p_ch);
That "x" in the example is picked up here and stored in "p_ch".
Let's skip over "text" and find that "x" in there.
> + while ((t_ch = *text) != '\0' &&
> + (!(flags & WM_PATHNAME) || t_ch != '/')) {
Why do we look at (flags & WM_PATHMAME) and not "special" here?
> + if ((flags & WM_CASEFOLD) && ISUPPER(t_ch))
> + t_ch = tolower(t_ch);
> + if (t_ch == p_ch)
> + break;
Found it.
> + text++;
> + }
> + if (t_ch != p_ch)
> + return WM_NOMATCH;
If we did not find that "x", then "**x" or "*x" can never match.
OK. And at this point "text" points at that "x" we found, and "p"
points at "x" after the asterisk in the pattern.
Looks good so far. Thanks.
> + }
> if ((matched = dowild(p, text, flags)) != WM_NOMATCH) {
> if (!special || matched != WM_ABORT_TO_STARSTAR)
> return matched;
^ permalink raw reply
* Re: [PATCH 4/8] wildmatch: support "no FNM_PATHNAME" mode
From: Junio C Hamano @ 2012-12-28 6:24 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git
In-Reply-To: <1356163028-29967-5-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> diff --git a/wildmatch.c b/wildmatch.c
> index a79f97e..4fe1d65 100644
> --- a/wildmatch.c
> +++ b/wildmatch.c
> @@ -77,14 +77,17 @@ static int dowild(const uchar *p, const uchar *text, unsigned int flags)
> continue;
> case '?':
> /* Match anything but '/'. */
> - if (t_ch == '/')
> + if ((flags & WM_PATHNAME) && t_ch == '/')
> return WM_NOMATCH;
> continue;
> case '*':
> if (*++p == '*') {
> const uchar *prev_p = p - 2;
> while (*++p == '*') {}
> - if ((prev_p == text || *prev_p == '/') ||
> + if (!(flags & WM_PATHNAME))
> + /* without WM_PATHNAME, '*' == '**' */
> + special = 1;
> + else if ((prev_p == text || *prev_p == '/') ||
Not a new issue in this patch, but here, "prev_p" points into the
pattern string, two bytes before p, which is the byte before the
"**" that we are looking at (which might be before the beginning of
the pattern). "text" is the string we are trying to match that
pattern against. How can these two pointers be compared to yield a
meaningful value?
> (*p == '\0' || *p == '/' ||
> (p[0] == '\\' && p[1] == '/'))) {
OK. "**/", "**" (end of pattern), and "**\/" are handled here.
Do we have to worry about "**[/]" the same way, or a class never
matches the directory separator, even if it is a singleton class
that consists of '/' (which is fine by me)? If so, is "\/" more or
less like "[/]"?
^ permalink raw reply
* Re: Find the starting point of a local branch
From: Woody Wu @ 2012-12-28 5:15 UTC (permalink / raw)
To: Martin von Zweigbergk; +Cc: Seth Robertson, git
In-Reply-To: <CANiSa6iSYvLbp1s8h9pwi=P1m0QdZPqf06hAm+4muChgJUuj=g@mail.gmail.com>
On Mon, Dec 24, 2012 at 09:24:39AM -0800, Martin von Zweigbergk wrote:
> On Sun, Dec 23, 2012 at 11:31 PM, Woody Wu <narkewoody@gmail.com> wrote:
> > On Sun, Dec 23, 2012 at 11:09:58PM -0500, Seth Robertson wrote:
> >>
> >> In message <20121224035825.GA17203@zuhnb712>, Woody Wu writes:
> >>
> >> How can I find out what's the staring reference point (a commit number
> >> or tag name) of a locally created branch? I can use gitk to find out it
> >> but this method is slow, I think there might be a command line to do it
> >> quickly.
> >>
> >> The answer is more complex than you probably suspected.
> >>
> >> Technically, `git log --oneline mybranch | tail -n 1` will tell you
> >> the starting point of any branch. But...I'm sure that isn't what you
> >> want to know.
> >>
> >> You want to know "what commit was I at when I typed `git branch
> >> mybranch`"?
> >
> > Yes, this is exactly I want to know.
> >
> >>The problem is git doesn't record this information and
> >> doesn't have the slightest clue.
> >>
> >> But, you say, I can use `gitk` and see it. See? Right there. That
> >> isn't (necessarily) the "starting point" of the branch, it is the
> >> place where your branch diverged from some other branch. Git is
> >> actually quite able to tell you when the last time your branch
> >> diverged from some other branch. `git merge-base mybranch master`
> >> will tell you this, and is probably the answer you were looking for.
> >
> > This is not working to me since I have more than one local branch that
> > diverged from the master, and in fact, the branch I have in question was
> > diverged from another local branch.
>
> As Jeff mentions in a later message, "git pull --rebase" would
> probably do what you want. It works with local branches too.
>
I think what 'git pull --rebase' would do is to fetch from the origin
and do a 'git rebase'. On one hand, I don't understand 'git rebase' so
much from the manual, ont the other hand, I did not get the point why
'git rebase' has something to do with the thing I want to do (what I
want is just query some kind of history information).
I know, my knowledge about git is still so limit. I will keep study from
the man pages.
> I once tried to add the same cleverness that "git pull --rebase"
> directly in "git rebase" [1], but there were several issues with those
> patches, one of was regarding the performance ("git pull --rebase" can
> be equally slow, but since it often involves network, users probably
> rarely notice). I think it would be nice to at least add it as an
> option to "git rebase" some day. Until then, "git pull --rebase" works
> fine.
>
> [1] http://thread.gmane.org/gmane.comp.version-control.git/166710
--
woody
I can't go back to yesterday - because I was a different person then.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox