* Re: [PATCH 3/3] Remove outdated info in difftool manpage
From: Denton Liu @ 2017-03-03 21:28 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, davvid
In-Reply-To: <alpine.DEB.2.20.1703031645470.3767@virtualbox>
On Fri, Mar 03, 2017 at 04:46:36PM +0100, Johannes Schindelin wrote:
> Hi Denton (or should I address you as Liu?),
Denton is fine, thanks.
>
> On Fri, 3 Mar 2017, Denton Liu wrote:
>
> > When difftool was rewritten in C, it removed the capability to read
> > fallback configs from mergetool. This changes the documentation to
> > reflect this.
>
> Thanks for pointing that out. But that is probably an oversight on my
> part, not an intentional change...
Do you expect to be submitting a patch for this soon? Or, if not, would
it be fine if I went ahead and did it?
>
> Ciao,
> Johannes
^ permalink raw reply
* Re: git status --> Out of memory, realloc failed
From: Carsten Fuchs @ 2017-03-03 21:03 UTC (permalink / raw)
To: René Scharfe; +Cc: git
In-Reply-To: <6ba86c1a-87a9-f886-273b-7f94c30c4a95@web.de>
Hi René,
Am 02.03.2017 um 21:04 schrieb René Scharfe:
> When I use ulimit -v with lower and lower numbers I can provoke mmap failures on
> bigger pack files, but not the realloc failures that you're seeing. And your
> packs should be only up to 20MB anyway (you can check that with "ls -l
> .git/objects/pack/*.pack").
Yes, there are 9 of them, all about 20 MB in size.
> So a shot in the dark: Do you have a lot of untracked files? You could check by
> cloning your repository locally (which copies only tracked contents). Does "git
> status" work on the clone?
I had about 40 modified or untracked files with a combined file size of
about 5 MB.
Before I got your latest mail, I tried something else: Instead of
connecting to the system with the repository via SSH as usual, I
accessed the repository by mounting its file system via gvfs-mount onto
my local desktop machine. Then I used my local, normally working Git to
commit all modified and some untracked files (and deleted those that
were left). Running `git status` and other commands took a very long
time over a network connection of only 6 MBit/s, but eventually I got
everything committed and pushed.
Quite in the spirit of your above words, getting rid of untracked files
helped: Logging normally via SSH into the remote machine again, `git
status` ran normally.
Only then did I see your mail and got another clone, where `git status`
worked equally well – but then `git diff` failed in a similar manner,
continuing the problems of `git status`:
(uiserver):p7715773:~/__TEST__$ git clone
https://CarstenF@bitbucket.org/CarstenF/website-cafu.git
Cloning into 'website-cafu'...
Password for 'https://CarstenF@bitbucket.org':
remote: Counting objects: 44359, done.
remote: Compressing objects: 100% (28777/28777), done.
remote: Total 44359 (delta 19201), reused 38661 (delta 14143)
Receiving objects: 100% (44359/44359), 168.28 MiB | 20.95 MiB/s, done.
Resolving deltas: 100% (19201/19201), done.
Checking connectivity... done.
Checking out files: 100% (18524/18524), done.
(uiserver):p7715773:~/__TEST__$ cd website-cafu/
(uiserver):p7715773:~/__TEST__/website-cafu$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
(uiserver):p7715773:~/__TEST__/website-cafu$ git diff
fatal: unable to create threaded lstat
(uiserver):p7715773:~/__TEST__/website-cafu$ git diff
fatal: Out of memory, realloc failed
fatal: Out of memory, realloc failed
fatal: Out of memory, realloc failed
fatal: recursion detected in die handler
> Another one, darker yet: Does "git config core.preloadIndex 0" help?
That's it! :-)
Once that was set, both `git status` and `git diff` worked normally again!
I cannot test `git status` right now in the same situation as before, as
I no longer have a working directory that is "as dirty" as before, but
the above setting seems to fix the problem promptly and thoroughly!
Many thanks for your help! :-)
Best regards,
Carsten
^ permalink raw reply
* Re: [PATCH v3 0/9] Fix the early config
From: Junio C Hamano @ 2017-03-03 21:35 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, Jeff King, Duy Nguyen
In-Reply-To: <cover.1488562287.git.johannes.schindelin@gmx.de>
Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> Notable notes:
>
> - In contrast to earlier versions, I no longer special-case init and
> clone. Peff pointed out that this adds technical debt, and that we can
> actually argue (for consistency's sake) that early config reads the
> current repository config (if any) even for init and clone.
>
> - The read_early_config() function does not cache Git directory
> discovery nor read values. If needed, this can be implemented later,
> in a separate patch series.
>
> - The alias handling in git.c could possibly benefit from this work, but
> again, this is a separate topic from the current patch series.
As Peff said in his review, I too find the result of this series a
more pleasant read than than original.
2/9 and corresponding 4/9 triggers "ERROR: trailing statements
should be on next line" from ../linux/scripts/checkpatch.pl because
of a line inherited from the original; I'll queue them with an
obvious style fix to work it around.
Thanks.
^ permalink raw reply
* Re: SHA1 collisions found
From: Stefan Beller @ 2017-03-03 21:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Jeff King, Junio C Hamano, Ian Jackson, Joey Hess,
Git Mailing List
In-Reply-To: <CA+55aFwXaSAMF41Dz3u3nS+2S24umdUFv0+k+s18UyPoj+v31g@mail.gmail.com>
On Thu, Mar 2, 2017 at 11:55 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> So: What do you think about the concept?
>
> Linus
One of the things I like about working on Git is its pretty
high standard of testing. So we would need to come up with
good methods of testing this, e.g. when
GIT_TEST_WITH_DEGENERATE_HASH is set, we'd use
an intentionally weak hashing function and have tests for
the collisions. These tests would need cover most of the
workflows that are currently performed with Git
(local creation, fetching, pushing). Writing all these
additional tests (which consists of creating colliding
objects/commits and then performing all these tests),
sounds about as much work as actually converting to
a new hash function. (First locally and then at a later
point in time all the networking related things).
I would not want to go that way.
Stefan
^ permalink raw reply
* Re: SHA1 collisions found
From: Jeff King @ 2017-03-03 22:18 UTC (permalink / raw)
To: Ian Jackson
Cc: Brandon Williams, Jason Cooper, Junio C Hamano, Linus Torvalds,
Joey Hess, Git Mailing List
In-Reply-To: <22713.33728.502854.338516@chiark.greenend.org.uk>
On Fri, Mar 03, 2017 at 02:54:56PM +0000, Ian Jackson wrote:
> > What _does_ get rewritten is what's in your ref files, your pack .idx,
> > etc. Those are all sha256 (or whatever), and work as sha1's do now.
>
> This all sounds very similar to my proposal.
Yeah, sorry I haven't reviewed that more carefully yet.
> > Looking up a sha1 reference from an old object just goes through the
> > extra level of indirection.
>
> I don't understand why this is a level of indirection, rather than
> simply a retention of the existing SHA-1 object database (in parallel,
> but deprecated).
I just meant that we will need to know both the sha1 and the sha-256 of
every object (because the same blob, for example, may be referred to by
either name depending on whether it is a new or a historical tree). So
one way to do that is to have a table mapping sha1 to sha-256, and then
a lookup of sha-1 goes through that before looking up the object content
on disk via sha-256.
But it may also be fine to just keep an index mapping sha1 directly to
object contents. That makes a sha1 lookup slightly faster, but it's more
expensive to do a sha-256 verification (you have to reverse-index the
object location in the sha-256 list).
(And my usual disclaimer: I am using sha-256 as a placeholder; I don't
have a strong opinion on the actual hash choice).
> Perhaps I have misunderstood what you mean by "graft". I assume you
> don't mean info/grafts, because that is not conveyed by transfer
> protocols.
No, I just mean there will be a spot in the commit graph (or many spots,
potentially) where a "v2" commit using sha-256 references (by sha-256) a
"v1" commit that is full of sha-1s. I say "graft" only to differentiate
it from the idea of rewriting the content of those commit objects.
> Specifically, the parent reference in the first post-transition commit
> has to refer to something. What does it refer to ? The possibilities
> seem to be:
>
> 1a. It names the SHA1 hash of an old commit object
> 1b. It names the BLAKE[1] hash of an old commit object, which
> object of course refers to its parents by SHA1.
>
> 2. It names the BLAKE hash of a translated version of the
> old commit object.
>
> 3. It doesn't name the parent, and the old history is not
> automatically transferred by clone and not readily accessible.
>
> (1a) and (1b) are different variants of something like my mixed hash
> proposal. Old and new hashes live side by side.
Thanks for laying out those options. My proposals have definitely been
in the (1b) camp.
I think (1a) is not so bad; it just bumps the transition point one
commit higher. But I think the "rules" for which hash to expect are
easier if they depend on which object the _pointer_ is using, rather
than the _pointee_.
> (2) involves rewriting all of the old history, to recursively generate
> new objects (with BLAKE names, and which in turn refer to other
> rewritten old objects by BLAKE names). The first time a particular
> tree needs to look up an object by a BLAKE name, it needs to run a
> conversion its own entire existing history.
>
> For (2) there would have to be some kind of mapping table in every
> tree, which allowed object names to be maped in both directions. The
> object content translation would have to be reversible, so that the
> actual pre-translation objects would not need to be stored; rather
> they would be reconstructed from the post-translation objects, when
> someone asks for a pre-translation object. In principle it would be
> possible to convert future BLAKE commits to SHA-1 ones, again by
> recursive rewriting.
Hmm. I had initially rejected this as being pretty nasty for accessing
the old format on the fly. But as long as you keep the bidirectional
mapping from the initial expensive conversion, in most cases you only
need to convert at most a single object. E.g., the two cases that are
really interesting are:
- I have an old commit sha1 and I want to run "git show" on it. We
convert the sha1 to the BLAKE name, and then just show the BLAKE
contents as usual.
- I want to verify a commit or tag signature. We need the original
bytes for this. So we convert the sha1 to the BLAKE name to get the
BLAKE'd contents. Then we rewrite only the _single_ object,
converting any of its internal BLAKE references back to sha1s via
the mapping.
That's more appealing than I had originally given it credit for, because
most of the other code just happily uses the BLAKE name internally. Even
a diff across the conversion boundary works at full speed, because it's
using the same names consistently.
The big downside is that the mapping is more expensive to generate than
a 1b-style mapping. In 1b, you just compute both hashes over all
incoming objects and store the sha1 in a side lookup table. But here you
actually need to _rewrite_ each object to come up with its
alternate-universe sha1. And you need to do it in reverse-graph order,
not the arbitrary order that something like index-pack uses.
I had assumed that local repos would generate these tables themselves,
so to not have to trust any other repos. And the alternate-universe
thing is more to ask those local repos to do. But it would also be
workable to distribute the mapping out-of-band (e.g., via a signed tag).
You don't even really need to trust the mapping that much, if it's just
used for historical name lookups.
> I don't think anyone is seriously suggesting (3).
Yeah, agreed.
> Conversely, (2) requires many object names stored elsewhere than in
> git to be updated. It's possible with (2) to do ad-hoc lookups on
> object names in mailing list messages or commit messages and so on.
> Even if it is possible for the new git to answer questions like "does
> this new branch with BLAKE hash X' contain the commit with SHA1 hash
> Y" by implicitly looking up the corresponding BLAKE commit Y' and
> answering the question with reference to Y', this isn't going to help
> if external code does things like "have git compute the merge base of
> X and Y' and check that it is equal to Z". Either the external
> database's record of Z would have to be changed to refer to Z', or the
> external code would have to be taught to apply an object name
> normalisation operation to turn Z into Z' each time.
I think the X->X' conversion on input is easy. The universe of objects
whose sha1 we are about is not getting any bigger. Outputting sha1 Z
instead of BLAKE Z' is a bit harder (and at the very least probably
needs the caller to use a specific command-line option).
> Also, (2) has trouble with shallow clones. This is because it's not
> possible to translate old objects to new ones without descending to
> the roots of the object tree and recursively translating everything
> (or looking up the answer of a previous translation).
Yes. Though if we distribute a partial sha1/BLAKE mapping with the clone
(i.e., just for the objects we're sending) then the client could still
use sha1 names. It not be that big a problem in practice, though. If
you're shallow, then you don't _have_ the old object names to refer to
anyway.
> The main thing here is that, regardless of the choice between (1) or
> (2), we need to choose whether object names specified on the git
> command line, and printed by normal git commands, explicitly identify
> the hash function.
>
> I think there are roughly three options:
>
> (A) Decorate all new hashes with a hash function indication
> (sha256:<hex> or blake_<hex> or H<hex>)
>
> (B) Infer the hash function from the object name length
> (and do some kind of bodge for abbreviated object names).
>
> (C) Hash function is implicit from context. (This is compatible with
> (2) only, because (1) requires any object to be able to refer to
> any hash function.)
>
> I think (A) is best because it means everything is unambiguous, and it
> allows future hash function changes without further difficulty.
For input, I think we should definitely _support_ A, but in practice I
think people would be happy if an undecorated hash (or partial hash)
looks it up as a BLAKE name, and then falls back to the sha1.
In (2) this is obviously the right thing to do, because all of our
output will be BLAKE names.
In (1) it is less clear if we might output sha1 names for old cases. But
I think we're still better off using the stronger hash when possible.
> (A) has the additional advantage that it becomes possible to make
> object names syntactically distinguishable from ref names.
Sort of. Our get_sha1() parser accepts a lot of random syntax. E.g.,
"sha256:1234abcd" is ambiguous with a file inside the tree named by
"sha256". In practice I don't mind carving out a namespace and letting
people with a branch named "sha256" rot.
> We don't know what hash function research will look like in 10-20
> years. We would like to not have a bunch of pain again. So ideally
> we would deploy a framework now that would let us switch hash function
> again without further history-rewriting.
>
> (1)(A) and perhaps (1)(B) are the only options which support this
> well.
Yes, I think planning for another migration is a sensible thing.
I just think we should not sacrifice any other properties to the idea
that people could flip on their bespoke hashes and interoperate with
other users. I.e., "git config core.hash sha3 && git push" should not be
a use case we care about at all, because it creates all sorts of _other_
headaches.
But I have no objection to making the 20-years-from-now migration less
painful, and I agree that (1b) is more along those lines.
-Peff
^ permalink raw reply
* Re: [PATCH v7 3/3] config: add conditional include
From: Jeff King @ 2017-03-03 22:24 UTC (permalink / raw)
To: Junio C Hamano
Cc: Nguyễn Thái Ngọc Duy, git, sschuberth,
Matthieu Moy, Philip Oakley, Ramsay Jones
In-Reply-To: <xmqq4lza9o5r.fsf@gitster.mtv.corp.google.com>
On Fri, Mar 03, 2017 at 11:05:20AM -0800, Junio C Hamano wrote:
> I am not sure about "obviousness", but I agree that we do not know
> that "conditional include" would be the only thing we want for the
> second level for "include.path" directive. "include-if.<cond>.path"
> is better for that reason.
>
> I presume that you could still do
>
> [include "if:gitdir=$path"]
> path = ...
>
> i.e. design the second level to begin with a token that tells
> readers what it means (and assign "if:" token for "conditional
> include"), but I do not think it is worth it.
Yep, all true.
> I also imagine that
>
> [include]
> condition = ...
> path = ...
>
> is easier to read and write by end-users, but it probably is not
> feasible because it requires too invasive change to the current code
> to teach it to grok such construct.
I am against that as it introduces a dependency in the presence and
ordering between two config variables, which can yield some surprises.
> Between "include-if" and "includeIf", if people find the latter not
> too ugly, I'd prefer to keep it the way Duy posted. Because of the
> way "include.path" and "include-if.<cond>.path" work, we can declare
> that they are not like ordinary configuration variable definition
> at all but are higher-level directives and that may be a sufficient
> justification to allow "-" in its name, though, if people find
> "includeIf" too ugly a name to live.
OK. I can live with includeIf.
-Peff
^ permalink raw reply
* Re: [PATCH v7 0/3] Conditional config include
From: Jeff King @ 2017-03-03 22:22 UTC (permalink / raw)
To: Junio C Hamano
Cc: Nguyễn Thái Ngọc Duy, git, sschuberth,
Matthieu Moy, Philip Oakley, Ramsay Jones
In-Reply-To: <xmqqlgsm9q2i.fsf@gitster.mtv.corp.google.com>
On Fri, Mar 03, 2017 at 10:24:05AM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > For those following on the mailing list, there is some discussion at:
> >
> > https://github.com/git/git/commit/484f78e46d00c6d35f20058671a8c76bb924fb33
> >
> > I think that is mostly focused around another failing in the
> > error-handling of the config code, and that does not need to be
> > addressed by this series (though of course I'd welcome any fixes).
>
> Thanks. Without a message like this, the list may have never known
> about the discussion taken elsewhere. I'd appreciate such a report
> to appear on list the next time much earlier ;-)
>
> When built with FREAD_READS_DIRECTORIES=Yes on Linux, the error in
> the test can easily reproduce.
Heh. I had no idea we had FREAD_READS_DIRECTORIES. I think Duy and I
reinvented it in another thread. ;)
I agree that may be worth setting on Linux (though note that we _do_
ignore other stdio read errors in the rest of the code path, which may
be something we want to address).
-Peff
^ permalink raw reply
* Re: [PATCH] t/perf: export variable used in other blocks
From: Jeff King @ 2017-03-03 22:31 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jonathan Tan, git
In-Reply-To: <xmqqbmti9os2.fsf@gitster.mtv.corp.google.com>
On Fri, Mar 03, 2017 at 10:51:57AM -0800, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Thanks, this is obviously the right thing to do, and the mistake is mine
> > from ea97002fc (t/perf: time rev-list with UNINTERESTING commits,
> > 2014-01-20). This is not the first time I've been confused by missing
> > variables in t/perf scripts, since it behaves differently than the
> > normal test suite. I wonder if we should turn on "set -a" during t/perf
> > setup snippets. That's a bit of a blunt tool, but I suspect it would
> > just be easier to work with.
>
> I wonder if we can make t/perf to behave more similar to the normal
> test suite to eliminate the need for this exporting, by the way.
> t/perf/README does not say anything more than "for lack of better
> options" throughout its history, which does not help very much.
The problem is that it has to run each block multiple times under an
external "time" program. If you assume that the shell running the perf
suite is bash, you can do:
i=0
foo() { echo $i; i=$(($i+1)); }
for run in 1 2 3; do time foo; done
which shows that you can get repeated timings from a shell function.
That shell function would run the actual test snippet, and would have to
do some redirect trickery to split the "time" results from the stderr of
the underlying test. And parse bash's time output.
But those are all do-able things. The main thing is throwing out a
dependency on an external "time" for "bash". That may not be a big loss,
though.
-Peff
^ permalink raw reply
* Re: bisect-helper: we do not bisect --objects
From: Christian Couder @ 2017-03-03 23:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqvarq9vzo.fsf@gitster.mtv.corp.google.com>
On Fri, Mar 3, 2017 at 5:16 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Ever since "bisect--helper" was introduced in 1bf072e366
> ("bisect--helper: implement "git bisect--helper"", 2009-03-26),
> after setting up the "rev-list $bad --not $good_ones" machinery, the
> code somehow prepared to mark the trees and blobs at the good boundary
> as uninteresting, only when --objects option was given. This was kept
> across a bit of refactoring done by 2ace9727be ("bisect: move common
> bisect functionality to "bisect_common"", 2009-04-19) and survives
> to this day.
>
> However, "git bisect" does not care about tree/blob object
> reachability at all---it purely works at the commit DAG level and
> nobody passes (and nobody should pass) "--objects" option to the
> underlying rev-list machinery. Remove the dead code.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
> * Christian, do you recall what we were thinking when we added this
> mark_edges_uninteresting() call in this program? If you don't,
> don't worry--this was done more than 8 years ago. I am just
> being curious and also a bit being cautious in case I am missing
> something.
I think I just copy pasted the code from cmd_rev_list() in
builtin-rev-list.c and probably didn't realize that revs->tree_objects
would always be false.
Thanks for spotting this and removing the dead code.
^ permalink raw reply
* What's cooking in git.git (Mar 2017, #02; Fri, 3)
From: Junio C Hamano @ 2017-03-03 23:26 UTC (permalink / raw)
To: git
Here are the topics that have been cooking. Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'. The ones marked with '.' do not appear in any of
the integration branches, but I am still holding onto them.
You can find the changes described here in the integration branches
of the repositories listed at
http://git-blame.blogspot.com/p/git-public-repositories.html
--------------------------------------------------
[New Topics]
* ew/markdown-url-in-readme (2017-03-01) 1 commit
(merged to 'next' on 2017-03-03 at 3d35e3a991)
+ README: create HTTP/HTTPS links from URLs in Markdown
Doc update.
Will merge to 'master'.
* jk/add-i-patch-do-prompt (2017-03-02) 1 commit
- add--interactive: fix missing file prompt for patch mode with "-i"
The patch subcommand of "git add -i" was meant to have paths
selection prompt just like other subcommand, unlike "git add -p"
directly jumps to hunk selection. Recently, this was broken and
"add -i" lost the paths selection dialog, but it now has been
fixed.
Will merge to 'next'.
* ax/line-log-range-merge-fix (2017-03-03) 1 commit
- line-log.c: prevent crash during union of too many ranges
The code to parse "git log -L..." command line was buggy when there
are many ranges specified with -L; overrun of the allocated buffer
has been fixed.
Will merge to 'next'.
* js/early-config (2017-03-03) 11 commits
- t1309: test read_early_config()
- read_early_config(): really discover .git/
- read_early_config(): avoid .git/config hack when unneeded
- setup: make read_early_config() reusable
- setup: export the discover_git_directory() function
- SQUASH??? ERROR: trailing statements should be on next line
- setup_git_directory_1(): avoid changing global state
- setup: prepare setup_discovered_git_directory() the root directory
- SQUASH??? ERROR: trailing statements should be on next line
- setup_git_directory(): use is_dir_sep() helper
- t7006: replace dubious test
The start-up sequence of "git" needs to figure out some configured
settings before it finds and set itself up in the location of the
repository and was quite messy due to its "chicken-and-egg" nature.
The code has been restructured.
Will merge to 'next' after squashing niggle-fixes in.
* jt/perf-updates (2017-03-03) 3 commits
- t/perf: add fallback for pre-bin-wrappers versions of git
- t/perf: use $MODERN_GIT for all repo-copying steps
- t/perf: export variable used in other blocks
The t/perf performance test suite was not prepared to test not so
old versions of Git, but now it covers versions of Git that are not
so ancient.
Will merge to 'next'.
* ss/remote-bzr-hg-placeholder-wo-python (2017-03-03) 1 commit
- contrib: git-remote-{bzr,hg} placeholders don't need Python
There is no need for Python only to give a few messages to the
standard error stream, but we somehow did.
Will merge to 'next'.
--------------------------------------------------
[Stalled]
* nd/worktree-move (2017-01-27) 7 commits
. fixup! worktree move: new command
. worktree remove: new command
. worktree move: refuse to move worktrees with submodules
. worktree move: accept destination as directory
. worktree move: new command
. worktree.c: add update_worktree_location()
. worktree.c: add validate_worktree()
"git worktree" learned move and remove subcommands.
Tentatively ejected as it seems to break 'pu' when merged.
* pb/bisect (2017-02-18) 28 commits
- fixup! bisect--helper: `bisect_next_check` & bisect_voc shell function in C
- bisect--helper: remove the dequote in bisect_start()
- bisect--helper: retire `--bisect-auto-next` subcommand
- bisect--helper: retire `--bisect-autostart` subcommand
- bisect--helper: retire `--bisect-write` subcommand
- bisect--helper: `bisect_replay` shell function in C
- bisect--helper: `bisect_log` shell function in C
- bisect--helper: retire `--write-terms` subcommand
- bisect--helper: retire `--check-expected-revs` subcommand
- bisect--helper: `bisect_state` & `bisect_head` shell function in C
- bisect--helper: `bisect_autostart` shell function in C
- bisect--helper: retire `--next-all` subcommand
- bisect--helper: retire `--bisect-clean-state` subcommand
- bisect--helper: `bisect_next` and `bisect_auto_next` shell function in C
- t6030: no cleanup with bad merge base
- bisect--helper: `bisect_start` shell function partially in C
- bisect--helper: `get_terms` & `bisect_terms` shell function in C
- bisect--helper: `bisect_next_check` & bisect_voc shell function in C
- bisect--helper: `check_and_set_terms` shell function in C
- bisect--helper: `bisect_write` shell function in C
- bisect--helper: `is_expected_rev` & `check_expected_revs` shell function in C
- bisect--helper: `bisect_reset` shell function in C
- wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()
- t6030: explicitly test for bisection cleanup
- bisect--helper: `bisect_clean_state` shell function in C
- bisect--helper: `write_terms` shell function in C
- bisect: rewrite `check_term_format` shell function in C
- bisect--helper: use OPT_CMDMODE instead of OPT_BOOL
Move more parts of "git bisect" to C.
Expecting a reroll.
cf. <CAFZEwPPXPPHi8KiEGS9ggzNHDCGhuqMgH9Z8-Pf9GLshg8+LPA@mail.gmail.com>
cf. <CAFZEwPM9RSTGN54dzaw9gO9iZmsYjJ_d1SjUD4EzSDDbmh-XuA@mail.gmail.com>
cf. <CAFZEwPNUXcNY9Qdz=_B7q2kQuaecPzJtTMGdv8YMUPEz2vnp8A@mail.gmail.com>
* ls/filter-process-delayed (2017-01-08) 1 commit
. convert: add "status=delayed" to filter process protocol
Ejected, as does not build when merged to 'pu'.
* sh/grep-tree-obj-tweak-output (2017-01-20) 2 commits
- grep: use '/' delimiter for paths
- grep: only add delimiter if there isn't one already
"git grep", when fed a tree-ish as an input, shows each hit
prefixed with "<tree-ish>:<path>:<lineno>:". As <tree-ish> is
almost always either a commit or a tag that points at a commit, the
early part of the output "<tree-ish>:<path>" can be used as the
name of the blob and given to "git show". When <tree-ish> is a
tree given in the extended SHA-1 syntax (e.g. "<commit>:", or
"<commit>:<dir>"), however, this results in a string that does not
name a blob (e.g. "<commit>::<path>" or "<commit>:<dir>:<path>").
"git grep" has been taught to be a bit more intelligent about these
cases and omit a colon (in the former case) or use slash (in the
latter case) to produce "<commit>:<path>" and
"<commit>:<dir>/<path>" that can be used as the name of a blob.
Expecting a reroll? Is this good enough with known limitations?
* jc/diff-b-m (2015-02-23) 5 commits
. WIPWIP
. WIP: diff-b-m
- diffcore-rename: allow easier debugging
- diffcore-rename.c: add locate_rename_src()
- diffcore-break: allow debugging
"git diff -B -M" produced incorrect patch when the postimage of a
completely rewritten file is similar to the preimage of a removed
file; such a resulting file must not be expressed as a rename from
other place.
The fix in this patch is broken, unfortunately.
Will discard.
--------------------------------------------------
[Cooking]
* jk/interpret-branch-name (2017-03-02) 9 commits
- checkout: restrict @-expansions when finding branch
- strbuf_check_ref_format(): expand only local branches
- branch: restrict @-expansions when deleting
- t3204: test git-branch @-expansion corner cases
- interpret_branch_name: allow callers to restrict expansions
- strbuf_branchname: add docstring
- strbuf_branchname: drop return value
- interpret_branch_name: move docstring to header file
- interpret_branch_name(): handle auto-namelen for @{-1}
"git branch @" created refs/heads/@ as a branch, and in general the
code that handled @{-1} and @{upstream} was a bit too loose in
disambiguating.
Will merge to 'next'.
* jk/sha1dc (2017-03-01) 7 commits
- Put sha1dc on a diet
- sha1dc: avoid 'for' loop initial decl
- sha1dc: resurrect LICENSE file
- sha1dc: avoid c99 declaration-after-statement
- Makefile: add USE_SHA1DC knob
- sha1dc: adjust header includes for git
- add collision-detecting sha1 implementation
Borrow "detect attempt to create collisions" variant of SHA-1
implementation by Marc Stevens (CWI) and Dan Shumow (Microsoft).
Expecting a cleaned-up reroll after discussion settles.
cf. <CA+55aFxTWqsTTiDKo4DBZT-8Z9t80bGMD3uijzKONa_bYEZABQ@mail.gmail.com>
* js/travis-32bit-linux (2017-03-03) 1 commit
- Travis: also test on 32-bit Linux
Add 32-bit Linux variant to the set of platforms to be tested with
Travis CI.
Will merge to 'next'.
* jt/http-base-url-update-upon-redirect (2017-02-28) 1 commit
(merged to 'next' on 2017-03-03 at 5225bd3ef8)
+ http: attempt updating base URL only if no error
When a redirected http transport gets an error during the
redirected request, we ignored the error we got from the server,
and ended up giving a not-so-useful error message.
Will merge to 'master'.
* jt/mark-tree-uninteresting-for-uninteresting-commit (2017-02-28) 3 commits
- upload-pack: compute blob reachability correctly
- revision: exclude trees/blobs given commit
- revision: unify {tree,blob}_objects in rev_info
The revision/object traversal machinery did not mark all tree and
blob objects that are contained in an uninteresting commit as
uninteresting, because that is quite costly. Instead, it only
marked those that are contained in an uninteresting boundary commit
as uninteresting.
cf. <20170228215937.yd4juycjf7y3vish@sigill.intra.peff.net>
* ps/docs-diffcore (2017-02-28) 2 commits
(merged to 'next' on 2017-03-03 at 9ca5691de2)
+ docs/diffcore: unquote "Complete Rewrites" in headers
+ docs/diffcore: fix grammar in diffcore-rename header
Doc update.
Will merge to 'master'.
* rj/remove-unused-mktemp (2017-02-28) 2 commits
(merged to 'next' on 2017-03-03 at 4512f0c5ab)
+ wrapper.c: remove unused gitmkstemps() function
+ wrapper.c: remove unused git_mkstemp() function
Code cleanup.
Will merge to 'master'.
* sb/submodule-init-url-selection (2017-02-28) 1 commit
(merged to 'next' on 2017-03-03 at 847d1f9a91)
+ submodule init: warn about falling back to a local path
Give a warning when "git submodule init" decides that the submodule
in the working tree is its upstream, as it is not a very common
setup.
Will merge to 'master'.
* jc/diff-populate-filespec-size-only-fix (2017-03-02) 1 commit
- diff: do not short-cut CHECK_SIZE_ONLY check in diff_populate_filespec()
"git diff --quiet" relies on the size field in diff_filespec to be
correctly populated, but diff_populate_filespec() helper function
made an incorrect short-cut when asked only to populate the size
field for paths that need to go through convert_to_git() (e.g. CRLF
conversion).
Will merge to 'next'.
* nd/conditional-config-include (2017-03-03) 5 commits
- SQUASH??? cond config include test
- SQUASH???
- config: add conditional include
- config.txt: reflow the second include.path paragraph
- config.txt: clarify multiple key values in include.path
The configuration file learned a new "includeIf.<condition>.path"
that includes the contents of the given path only when the
condition holds. This allows you to say "include this work-related
bit only in the repositories under my ~/work/ directory".
Will merge to 'next' after squashing niggle-fixes in.
* rs/log-email-subject (2017-03-01) 2 commits
(merged to 'next' on 2017-03-03 at a2ecc84866)
+ pretty: use fmt_output_email_subject()
+ log-tree: factor out fmt_output_email_subject()
Code clean-up.
Will merge to 'master'.
* cc/split-index-config (2017-03-01) 22 commits
- Documentation/git-update-index: explain splitIndex.*
- Documentation/config: add splitIndex.sharedIndexExpire
- read-cache: use freshen_shared_index() in read_index_from()
- read-cache: refactor read_index_from()
- t1700: test shared index file expiration
- read-cache: unlink old sharedindex files
- config: add git_config_get_expiry() from gc.c
- read-cache: touch shared index files when used
- sha1_file: make check_and_freshen_file() non static
- Documentation/config: add splitIndex.maxPercentChange
- t1700: add tests for splitIndex.maxPercentChange
- read-cache: regenerate shared index if necessary
- config: add git_config_get_max_percent_split_change()
- Documentation/git-update-index: talk about core.splitIndex config var
- Documentation/config: add information for core.splitIndex
- t1700: add tests for core.splitIndex
- update-index: warn in case of split-index incoherency
- read-cache: add and then use tweak_split_index()
- split-index: add {add,remove}_split_index() functions
- config: add git_config_get_split_index()
- t1700: change here document style
- config: mark an error message up for translation
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.
I think this is almost ready for 'next'.
* dp/filter-branch-prune-empty (2017-03-03) 4 commits
- p7000: add test for filter-branch with --prune-empty
- filter-branch: fix --prune-empty on parentless commits
- t7003: ensure --prune-empty removes entire branch when applicable
- t7003: ensure --prune-empty can prune root commit
"git filter-branch --prune-empty" drops a single-parent commit that
becomes a no-op, but did not drop a root commit whose tree is empty.
Will merge to 'next'.
* jc/config-case-cmdline-take-2 (2017-02-23) 2 commits
(merged to 'next' on 2017-03-01 at 2e9920eeeb)
+ config: use git_config_parse_key() in git_config_parse_parameter()
+ config: move a few helper functions up
The code to parse "git -c VAR=VAL cmd" and set configuration
variable for the duration of cmd had two small bugs, which have
been fixed.
Will merge to 'master'.
This supersedes jc/config-case-cmdline topic that has been discarded.
* ab/cond-skip-tests (2017-03-01) 3 commits
- gitweb tests: skip tests when we don't have Time::HiRes
- gitweb tests: change confusing "skip_all" phrasing
- cvs tests: skip tests that call "cvs commit" when running as root
A few tests were run conditionally under (rare) conditions where
they cannot be run (like running cvs tests under 'root' account).
Will merge to 'next'.
* jk/interop-test (2017-02-27) 2 commits
- t/interop: add test of old clients against modern git-daemon
- t: add an interoperability test harness
Picking two versions of Git and running tests to make sure the
older one and the newer one interoperate happily has now become
possible.
Needs review.
* jk/parse-config-key-cleanup (2017-02-24) 3 commits
(merged to 'next' on 2017-03-01 at e531d8d3a9)
+ parse_hide_refs_config: tell parse_config_key we don't want a subsection
+ parse_config_key: allow matching single-level config
+ parse_config_key: use skip_prefix instead of starts_with
(this branch uses sb/parse-hide-refs-config-cleanup.)
The "parse_config_key()" API function has been cleaned up.
Will merge to 'master'.
* jk/t6300-cleanup (2017-02-27) 1 commit
(merged to 'next' on 2017-03-02 at 3087521bea)
+ t6300: avoid creating refs/heads/HEAD
A test that creates a confusing branch whose name is HEAD has been
corrected not to do so.
Will merge to 'master'.
* rs/commit-parsing-optim (2017-02-27) 2 commits
(merged to 'next' on 2017-03-02 at 22239f35df)
+ commit: don't check for space twice when looking for header
+ commit: be more precise when searching for headers
The code that parses header fields in the commit object has been
updated for (micro)performance and code hygiene.
Will merge to 'master'.
* rs/sha1-file-plug-fallback-base-leak (2017-02-27) 1 commit
(merged to 'next' on 2017-03-02 at 03344b1119)
+ sha1_file: release fallback base's memory in unpack_entry()
A leak in a codepath to read from a packed object in (rare) cases
has been plugged.
Will merge to 'master'.
* rs/strbuf-add-real-path (2017-02-27) 2 commits
(merged to 'next' on 2017-03-02 at 69191becd6)
+ strbuf: add strbuf_add_real_path()
+ cocci: use ALLOC_ARRAY
An helper function to make it easier to append the result from
real_path() to a strbuf has been added.
Will merge to 'master'.
* sb/parse-hide-refs-config-cleanup (2017-02-24) 1 commit
(merged to 'next' on 2017-03-01 at fd722ba039)
+ refs: parse_hide_refs_config to use parse_config_key
(this branch is used by jk/parse-config-key-cleanup.)
Code clean-up.
Will merge to 'master'.
* sg/clone-refspec-from-command-line-config (2017-02-27) 1 commit
- clone: respect configured fetch respecs during initial fetch
Needs review.
cf. <20170227211217.73gydlxb2qu2sp3m@sigill.intra.peff.net>
* sk/dash-is-previous (2017-03-01) 5 commits
- revert.c: delegate handling of "-" shorthand to setup_revisions
- sha1_name.c: teach get_sha1_1 "-" shorthand for "@{-1}"
- revision.c: args starting with "-" might be a revision
- revision.c: swap if/else blocks
- revision.c: do not update argv with unknown option
A dash "-" can be written to mean "the branch that was previously
checked out" in more places.
Needs review.
cf. <1488007487-12965-1-git-send-email-kannan.siddharth12@gmail.com>
* jh/send-email-one-cc (2017-02-27) 1 commit
(merged to 'next' on 2017-03-02 at 32c0e6ad88)
+ send-email: only allow one address per body tag
"Cc:" on the trailer part does not have to conform to RFC strictly,
unlike in the e-mail header. "git send-email" has been updated to
ignore anything after '>' when picking addresses, to allow non-address
cruft like " # stable 4.4" after the address.
Will merge to 'master'.
* jk/http-auth (2017-02-27) 2 commits
(merged to 'next' on 2017-03-02 at 87f81b4395)
+ http: add an "auto" mode for http.emptyauth
+ http: restrict auth methods to what the server advertises
Reduce authentication round-trip over HTTP when the server supports
just a single authentication method.
Will merge to 'master'.
* jk/ident-empty (2017-02-23) 4 commits
(merged to 'next' on 2017-03-01 at ff80031ce6)
+ ident: do not ignore empty config name/email
+ ident: reject all-crud ident name
+ ident: handle NULL email when complaining of empty name
+ ident: mark error messages for translation
user.email that consists of only cruft chars should consistently
error out, but didn't.
Will merge to 'master'.
* jt/upload-pack-error-report (2017-02-23) 1 commit
(merged to 'next' on 2017-03-01 at aea583dbe5)
+ upload-pack: report "not our ref" to client
"git upload-pack", which is a counter-part of "git fetch", did not
report a request for a ref that was not advertised as invalid.
This is generally not a problem (because "git fetch" will stop
before making such a request), but is the right thing to do.
Will merge to 'master'.
* ah/doc-ls-files-quotepath (2017-03-02) 1 commit
- Documentation: improve description for core.quotePath
Documentation for "git ls-files" did not refer to core.quotePath
Will merge to 'next'.
* jh/memihash-opt (2017-02-17) 5 commits
- name-hash: remember previous dir_entry during lazy_init_name_hash
- name-hash: specify initial size for istate.dir_hash table
- name-hash: precompute hash values during preload-index
- hashmap: allow memihash computation to be continued
- name-hash: eliminate duplicate memihash call
Expecting an update for perf.
cf. <MWHPR03MB29581B0EDDEDCA7D51EC396F8A280@MWHPR03MB2958.namprd03.prod.outlook.com>
* nd/prune-in-worktree (2017-02-19) 15 commits
. rev-list: expose and document --single-worktree
. revision.c: --reflog add HEAD reflog from all worktrees
. files-backend: make reflog iterator go through per-worktree reflog
. refs: add refs_for_each_reflog[_ent]()
. revision.c: --all adds HEAD from all worktrees
. refs: remove dead for_each_*_submodule()
. revision.c: use refs_for_each*() instead of for_each_*_submodule()
. refs: add a refs_for_each_in() and friends
. refs: add refs_for_each_ref()
. refs: add refs_head_ref()
. refs: add refs_read_ref[_full]()
. refs: move submodule slash stripping code to get_submodule_ref_store
. revision.c: --indexed-objects add objects from all worktrees
. revision.c: refactor add_index_objects_to_pending()
. revision.h: new flag in struct rev_info wrt. worktree-related refs
(this branch uses nd/worktree-kill-parse-ref; is tangled with nd/files-backend-git-dir.)
"git gc" and friends when multiple worktrees are used off of a
single repository did not consider the index and per-worktree refs
of other worktrees as the root for reachability traversal, making
objects that are in use only in other worktrees to be subject to
garbage collection.
* mm/fetch-show-error-message-on-unadvertised-object (2017-03-02) 3 commits
- fetch-pack: add specific error for fetching an unadvertised object
- fetch_refs_via_pack: call report_unmatched_refs
- fetch-pack: move code to report unmatched refs to a function
"git fetch" that requests a commit by object name, when the other
side does not allow such an request, failed without much
explanation.
Will merge to 'next'.
* nd/worktree-kill-parse-ref (2017-02-19) 22 commits
. refs: kill set_worktree_head_symref()
. refs: add refs_create_symref()
. worktree.c: kill parse_ref() in favor of refs_resolve_ref_unsafe()
. refs.c: add refs_resolve_ref_unsafe()
. refs: introduce get_worktree_ref_store()
. refs: rename get_ref_store() to get_submodule_ref_store() and make it public
. files-backend: remove submodule_allowed from files_downcast()
. refs: move submodule code out of files-backend.c
. path.c: move some code out of strbuf_git_path_submodule()
. refs.c: make get_main_ref_store() public and use it
. refs.c: kill register_ref_store(), add register_submodule_ref_store()
. refs.c: flatten get_ref_store() a bit
. refs: rename lookup_ref_store() to lookup_submodule_ref_store()
. refs.c: introduce get_main_ref_store()
. files-backend: remove the use of git_path()
. refs.c: share is_per_worktree_ref() to files-backend.c
. files-backend: replace *git_path*() with files_path()
. files-backend: add files_path()
. files-backend: convert git_path() to strbuf_git_path()
. refs-internal.c: make files_log_ref_write() static
. Merge branch 'mh/ref-remove-empty-directory' into nd/files-backend-git-dir
. Merge branch 'mh/submodule-hash' into nd/files-backend-git-dir
(this branch is used by nd/prune-in-worktree; is tangled with nd/files-backend-git-dir.)
(hopefully) a beginning of safer "git worktree" that is resistant
to "gc".
Waiting for nd/files-backend-git-dir to settle.
* nd/files-backend-git-dir (2017-02-22) 26 commits
. t1406: new tests for submodule ref store
. t1405: some basic tests on main ref store
. t/helper: add test-ref-store to test ref-store functions
. refs: delete pack_refs() in favor of refs_pack_refs()
. files-backend: avoid ref api targetting main ref store
. refs: new transaction related ref-store api
. refs: add new ref-store api
. refs: rename get_ref_store() to get_submodule_ref_store() and make it public
. files-backend: replace submodule_allowed check in files_downcast()
. refs: move submodule code out of files-backend.c
. path.c: move some code out of strbuf_git_path_submodule()
. refs.c: make get_main_ref_store() public and use it
. refs.c: kill register_ref_store(), add register_submodule_ref_store()
. refs.c: flatten get_ref_store() a bit
. refs: rename lookup_ref_store() to lookup_submodule_ref_store()
. refs.c: introduce get_main_ref_store()
. files-backend: remove the use of git_path()
. files-backend: add and use files_refname_path()
. files-backend: add and use files_reflog_path()
. files-backend: move "logs/" out of TMP_RENAMED_LOG
. files-backend: convert git_path() to strbuf_git_path()
. files-backend: add and use files_packed_refs_path()
. files-backend: make files_log_ref_write() static
. refs.h: add forward declaration for structs used in this file
. Merge branch 'mh/ref-remove-empty-directory' into nd/files-backend-git-dir
. Merge branch 'mh/submodule-hash' into nd/files-backend-git-dir
(this branch is tangled with nd/prune-in-worktree and nd/worktree-kill-parse-ref.)
The "submodule" specific field in the ref_store structure is
replaced with a more generic "gitdir" that can later be used also
when dealing with ref_store that represents the set of refs visible
from the other worktrees.
* sb/checkout-recurse-submodules (2017-02-23) 15 commits
- builtin/checkout: add --recurse-submodules switch
- entry.c: update submodules when interesting
- read-cache, remove_marked_cache_entries: wipe selected submodules.
- unpack-trees: check if we can perform the operation for submodules
- unpack-trees: pass old oid to verify_clean_submodule
- update submodules: add submodule_move_head
- update submodules: move up prepare_submodule_repo_env
- submodules: introduce check to see whether to touch a submodule
- update submodules: add a config option to determine if submodules are updated
- update submodules: add submodule config parsing
- connect_work_tree_and_git_dir: safely create leading directories
- make is_submodule_populated gently
- lib-submodule-update.sh: define tests for recursing into submodules
- lib-submodule-update.sh: do not use ./. as submodule remote
- lib-submodule-update.sh: reorder create_lib_submodule_repo
"git checkout" is taught --recurse-submodules option.
Reroll exists, but it appears it still needs minor work on top.
cf. <20170302004759.27852-1-sbeller@google.com>
* tg/stash-push (2017-02-28) 6 commits
(merged to 'next' on 2017-03-03 at b50fda0389)
+ stash: allow pathspecs in the no verb form
+ stash: use stash_push for no verb form
+ stash: teach 'push' (and 'create_stash') to honor pathspec
+ stash: refactor stash_create
+ stash: add test for the create command line arguments
+ stash: introduce push verb
Allow "git stash" to take pathspec so that the local changes can be
stashed away only partially.
Will merge to 'master'.
* bc/object-id (2017-02-22) 19 commits
- wt-status: convert to struct object_id
- builtin/merge-base: convert to struct object_id
- Convert object iteration callbacks to struct object_id
- sha1_file: introduce an nth_packed_object_oid function
- refs: simplify parsing of reflog entries
- refs: convert each_reflog_ent_fn to struct object_id
- reflog-walk: convert struct reflog_info to struct object_id
- builtin/replace: convert to struct object_id
- Convert remaining callers of resolve_refdup to object_id
- builtin/merge: convert to struct object_id
- builtin/clone: convert to struct object_id
- builtin/branch: convert to struct object_id
- builtin/grep: convert to struct object_id
- builtin/fmt-merge-message: convert to struct object_id
- builtin/fast-export: convert to struct object_id
- builtin/describe: convert to struct object_id
- builtin/diff-tree: convert to struct object_id
- builtin/commit: convert to struct object_id
- hex: introduce parse_oid_hex
"uchar [40]" to "struct object_id" conversion continues.
Now at v5.
cf. <20170221234737.894681-1-sandals@crustytoothpaste.net>
* jh/mingw-openssl-sha1 (2017-02-09) 1 commit
- mingw: use OpenSSL's SHA-1 routines
Windows port wants to use OpenSSL's implementation of SHA-1
routines, so let them.
Will merge to 'next'.
* sg/completion-refs-speedup (2017-02-13) 13 commits
- squash! completion: fill COMPREPLY directly when completing refs
- completion: fill COMPREPLY directly when completing refs
- completion: list only matching symbolic and pseudorefs when completing refs
- completion: let 'for-each-ref' sort remote branches for 'checkout' DWIMery
- completion: let 'for-each-ref' filter remote branches for 'checkout' DWIMery
- completion: let 'for-each-ref' strip the remote name from remote branches
- completion: let 'for-each-ref' and 'ls-remote' filter matching refs
- completion: don't disambiguate short refs
- completion: don't disambiguate tags and branches
- completion: support excluding full refs
- completion: support completing full refs after '--option=refs/<TAB>'
- completion: wrap __git_refs() for better option parsing
- completion: remove redundant __gitcomp_nl() options from _git_commit()
The refs completion for large number of refs has been sped up,
partly by giving up disambiguating ambiguous refs and partly by
eliminating most of the shell processing between 'git for-each-ref'
and 'ls-remote' and Bash's completion facility.
What's the donness of this topic?
* jk/no-looking-at-dotgit-outside-repo-final (2016-10-26) 1 commit
(merged to 'next' on 2017-02-27 at 7373a1b73d)
+ setup_git_env: avoid blind fall-back to ".git"
This is the endgame of the topic to avoid blindly falling back to
".git" when the setup sequence said we are _not_ in Git repository.
A corner case that happens to work right now may be broken by a
call to die("BUG").
Will cook in 'next'.
* jc/merge-drop-old-syntax (2015-04-29) 1 commit
(merged to 'next' on 2017-02-27 at 2c0f5f73d8)
+ merge: drop 'git merge <message> HEAD <commit>' syntax
Stop supporting "git merge <message> HEAD <commit>" syntax that has
been deprecated since October 2007, and issues a deprecation
warning message since v2.5.0.
Will cook in 'next'.
* jc/bundle (2016-03-03) 6 commits
- index-pack: --clone-bundle option
- Merge branch 'jc/index-pack' into jc/bundle
- bundle v3: the beginning
- bundle: keep a copy of bundle file name in the in-core bundle header
- bundle: plug resource leak
- bundle doc: 'verify' is not about verifying the bundle
The beginning of "split bundle", which could be one of the
ingredients to allow "git clone" traffic off of the core server
network to CDN.
--------------------------------------------------
[Discarded]
* jk/auto-namelen-in-interpret-branch-name (2017-02-27) 1 commit
. interpret_branch_name(): handle auto-namelen for @{-1}
A small bug in the code that parses @{...} has been fixed.
This is now a part of jk/interpret-branch-name topic.
^ permalink raw reply
* Re: bisect-helper: we do not bisect --objects
From: Philip Oakley @ 2017-03-03 23:27 UTC (permalink / raw)
To: Junio C Hamano, git, Christian Couder
In-Reply-To: <xmqqvarq9vzo.fsf@gitster.mtv.corp.google.com>
From: "Junio C Hamano" <gitster@pobox.com>
> Ever since "bisect--helper" was introduced in 1bf072e366
> ("bisect--helper: implement "git bisect--helper"", 2009-03-26),
> after setting up the "rev-list $bad --not $good_ones" machinery, the
> code somehow prepared to mark the trees and blobs at the good boundary
> as uninteresting, only when --objects option was given. This was kept
> across a bit of refactoring done by 2ace9727be ("bisect: move common
> bisect functionality to "bisect_common"", 2009-04-19) and survives
> to this day.
>
> However, "git bisect" does not care about tree/blob object
> reachability at all---it purely works at the commit DAG level and
> nobody passes (and nobody should pass) "--objects" option to the
> underlying rev-list machinery. Remove the dead code.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
> * Christian, do you recall what we were thinking when we added this
> mark_edges_uninteresting() call in this program? If you don't,
> don't worry--this was done more than 8 years ago. I am just
> being curious and also a bit being cautious in case I am missing
> something.
>
Bikeshedding: If the given boundary is a tag, it could be tagging a blob or
tree rather than a commit. Would that be a scenario that reaches this part
of the code? I thought I read previous comments that there is a case in the
Linux tree.
--
Philip
> Thanks.
>
> bisect.c | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/bisect.c b/bisect.c
> index 30808cadf7..86c5929a23 100644
> --- a/bisect.c
> +++ b/bisect.c
> @@ -634,8 +634,6 @@ static void bisect_common(struct rev_info *revs)
> {
> if (prepare_revision_walk(revs))
> die("revision walk setup failed");
> - if (revs->tree_objects)
> - mark_edges_uninteresting(revs, NULL);
> }
>
> static void exit_if_skipped_commits(struct commit_list *tried,
>
^ permalink raw reply
* Re: [PATCH v7 0/3] Conditional config include
From: Junio C Hamano @ 2017-03-03 23:22 UTC (permalink / raw)
To: Jeff King
Cc: Nguyễn Thái Ngọc Duy, git, sschuberth,
Matthieu Moy, Philip Oakley, Ramsay Jones
In-Reply-To: <20170303222215.vwulypicszoh2hds@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> Heh. I had no idea we had FREAD_READS_DIRECTORIES. I think Duy and I
> reinvented it in another thread. ;)
You two were not alone. I was planning to reinvent it before I went
to bed last night and then found it already was there this morning ;-)
> I agree that may be worth setting on Linux (though note that we _do_
> ignore other stdio read errors in the rest of the code path, which may
> be something we want to address).
Yes, we need an error check on fopen() in git_config_from_file()
regardless.
^ permalink raw reply
* Server-side hooks on non-bare repository
From: Mike Lewis @ 2017-03-04 0:10 UTC (permalink / raw)
To: git
Hello,
I’m having some issues with using server-side hooks when pushing to a non-bare repository. In my git config, I have `receive.denyCurrentBranch` set to `updateInstead`, which behaves as expected, and updates the current working tree when the current branch is pushed to. However, attempting to process those changes with pre-receive and post-receive hooks results in some unexpected behavior regarding the current working directory of the scripts and using git commands. I’ve tested these issues using both git 2.11 and 2.12 on various systems (macOS and CentOS), and get the same behavior each time.
Essentially, my problem boils down to two things:
1. When using a non-bare repository, I would expect the the working directory of the hook to be the root directory of the working tree, as this mirrors the behavior of “client”-side hooks like pre-commit. Instead, the working directory is set to the .git directory. That in and of itself is not a huge deal, but it leads into #2:
2. While running the hooks, git treats the repository as being bare, regardless of whether it actually is. For instance, changing the working directory of the scripts to the actual root of the working tree and attempting to run any git commands (for instance, `git rev-parse --abbrev-ref HEAD` to get the current branch name) results in "fatal: Not a git repository: ‘.’” being returned to the client-side `git push` command. I’ve found a workaround to this, which is to explicitly set the “GIT_DIR” to the .git directory, and make sure that is passed to any external scripts as well. However, this is very unintuitive behavior, as the working tree is still there and either unmodified or done being updated, depending on which hook is being called. At the very least, this behavior should be in the documentation somewhere so that users can write their hooks accordingly.
TL;DR: it seems that server-side hooks don’t account for the repository being non-bare, which is no longer a valid assumption with the `receive.denyCurrentBranch=updateInstead` configuration value introduced in 2.3.0.
Thanks, and I’d be happy to provide any other information that anyone needs to take a look at this.
Mike Lewis
^ permalink raw reply
* Re: [PATCH v1] Travis: also test on 32-bit Linux
From: Junio C Hamano @ 2017-03-04 0:03 UTC (permalink / raw)
To: Lars Schneider, Johannes Schindelin; +Cc: git
In-Reply-To: <xmqqh93a9p5r.fsf@gitster.mtv.corp.google.com>
Junio C Hamano <gitster@pobox.com> writes:
> I see he did v2 which you Acked in a different thread. Will replace
> what's been on 'pu' and running with Travis the past few days with
> it. Let's wait for one or more Travis cycles and then merge it to
> 'next'.
https://travis-ci.org/git/git/jobs/207517043 is an output from 'pu'
with the v2 patch; it seems to have fell into a funny state. The
output ends like so:
No output has been received in the last 10m0s, this potentially
indicates a stalled build or something wrong with the build itself.
I only recently started looking at Travis logs, so I cannot tell if
it is just a usual flake (e.g. some builds a few days ago seems to
have failed due to not being able to check out the tree being
tested, which I do not think is our fault) that we shouldn't worry
about, or if it is a sign of a real problem.
Unrelated to linux-32, the same build has hard failure with Apple
clang in t0021 with the rot13-filter.pl thing, by the way.
^ permalink raw reply
* Re: bisect-helper: we do not bisect --objects
From: Junio C Hamano @ 2017-03-03 23:51 UTC (permalink / raw)
To: Philip Oakley; +Cc: git, Christian Couder
In-Reply-To: <D9E30A61FAC84D3B90981B388C9699D0@PhilipOakley>
"Philip Oakley" <philipoakley@iee.org> writes:
> Bikeshedding: If the given boundary is a tag, it could be tagging a
> blob or tree rather than a commit. Would that be a scenario that
> reaches this part of the code?
I do not think it is relevant.
Bisection is an operation over a "bisectable" commit DAG, where
commits can be partitioned into "new" (aka "bad") and "old" (aka
"good") camp, all descendants of a "new" commit are all "new", all
ancestors of an "old" commit are all "old". Think of the "new"-ness
as a 100% inheritable disease that happens spontaneously and without
cure. Once you are infected with "new"-ness, all your descendants
are forever "new". If you know you are free of the "new"-ness, you
know that all your ancestors are not with the "new"-ness, either.
The goal of the operation is to find a "new" commit whose parents
are all "old".
The bisectability of the commit DAG is what allows you to say "this
commit is new" to a commit somewhere in the middle of the history
and avoid having to test any descendants, as they all inherit the
"new"-ness from it (similarly when you have one commit that is
"old", you do not have to test any ancestor), thereby reducing the
number of test from N (all commits in good..bad range) to log(N).
There is no room for a tree or a blob to participate in this graph
partitioning problem. A "bad" tree that is "new" cannot infect its
children with the "new"-ness and a "good" tree cannot guarantee the
lack of "new"-ness of its parents, because a tree or a blob does not
have parent or child commits.
^ permalink raw reply
* RE: [PATCH] Put sha1dc on a diet
From: Dan Shumow @ 2017-03-04 1:07 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, Jeff King, Git Mailing List, Marc Stevens
In-Reply-To: <xmqq1suge1jn.fsf@gitster.mtv.corp.google.com>
From: Junio C Hamano [mailto:gitster@pobox.com]
> As you and Marc seemed to be still working on speeding up, such a customization work to fully adjust your code to our codebase was premature, so I tentatively queued what we saw on the list as-is on our 'pu' branch so that people can have a reference point. Which unfortunately solicited a premature reaction by Johannes. Please do not worry too much about the comment.
> But if you are willing to help us by getting involved in the "customization" part, too, that would be a very welcome news to us.
>In that case, "welcome to the Git development community" ;-)
> So,... from my point of view, we are OK either way. It is OK if you are a third-party upstream that is not particularly interested in Git project's specific requirement. We surely would be happier if you and Marc, the upstream authors of the code in question, also act as participants in the Git development community.
> Either way, thanks for your great help.
You are very welcome. Thank you for the warm welcome. As it turns out, Marc and I are working on the simplifications / removal of c99 and performance upstream in our GitHub repo. I am happy to help for any GitHub specific customizations that are needed as well. But for now, lets see if we can get you everything you want upstream -- I think that's the most simple.
Thanks again,
Dan
^ permalink raw reply
* [PATCH] http: inform about alternates-as-redirects behavior
From: Eric Wong @ 2017-03-04 1:35 UTC (permalink / raw)
To: Junio C Hamano, Jeff King
Cc: Jann Horn, Brandon Williams, git, sbeller, bburky, jrnieder
It is disconcerting for users to not notice the behavior
change in handling alternates from commit cb4d2d35c4622ec2
("http: treat http-alternates like redirects")
Give the user a hint about the config option so they can
see the URL and decide whether or not they want to enable
http.followRedirects in their config.
Signed-off-by: Eric Wong <e@80x24.org>
---
http-walker.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/http-walker.c b/http-walker.c
index b34b6ace7..626badfe6 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -168,6 +168,12 @@ static int is_alternate_allowed(const char *url)
};
int i;
+ if (http_follow_config != HTTP_FOLLOW_ALWAYS) {
+ warning("alternate disabled by http.followRedirects!=true: %s",
+ url);
+ return 0;
+ }
+
for (i = 0; i < ARRAY_SIZE(protocols); i++) {
const char *end;
if (skip_prefix(url, protocols[i], &end) &&
@@ -331,9 +337,6 @@ static void fetch_alternates(struct walker *walker, const char *base)
struct alternates_request alt_req;
struct walker_data *cdata = walker->data;
- if (http_follow_config != HTTP_FOLLOW_ALWAYS)
- return;
-
/*
* If another request has already started fetching alternates,
* wait for them to arrive and return to processing this request's
--
EW
^ permalink raw reply related
* RFC: Another proposed hash function transition plan
From: Jonathan Nieder @ 2017-03-04 1:12 UTC (permalink / raw)
To: git; +Cc: sbeller, bmwill, jonathantanmy, peff, Linus Torvalds
Hi,
This past week we came up with this idea for what a transition to a new
hash function for Git would look like. I'd be interested in your
thoughts (especially if you can make them as comments on the document,
which makes it easier to address them and update the document).
This document is still in flux but I thought it best to send it out
early to start getting feedback.
We tried to incorporate some thoughts from the thread
http://public-inbox.org/git/20170223164306.spg2avxzukkggrpb@kitenet.net
but it is a little long so it is easy to imagine we've missed
some things already discussed there.
You can use the doc URL
https://goo.gl/gh2Mzc
to view the latest version and comment.
Thoughts welcome, as always.
Git hash function transition
============================
Status: Draft
Last Updated: 2017-03-03
Objective
---------
Migrate Git from SHA-1 to a stronger hash function.
Background
----------
The Git version control system can be thought of as a content
addressable filesystem. It uses the SHA-1 hash function to name
content. For example, files, trees, commits are referred to by hash
values unlike in other traditional version control systems where files
or versions are referred to via sequential numbers. The use of a hash
function to address its content delivers a few advantages:
* Integrity checking is easy. Bit flips, for example, are easily
detected, as the hash of corrupted content does not match its name.
Lookup of objects is fast.
Using a cryptographically secure hash function brings additional advantages:
* Object names can be signed and third parties can trust the hash to
address the signed object and all objects it references.
* Communication using Git protocol and out of band communication
methods have a short reliable string that can be used to reliably
address stored content.
Over time some flaws in SHA-1 have been discovered by security
researchers. https://shattered.io demonstrated a practical SHA-1 hash
collision. As a result, SHA-1 cannot be considered cryptographically
secure any more. This impacts the communication of hash values because
we cannot trust that a given hash value represents the known good
version of content that the speaker intended.
SHA-1 still possesses the other properties such as fast object lookup
and safe error checking, but other hash functions are equally suitable
that are believed to be cryptographically secure.
Goals
-----
1. The transition to SHA256 can be done one local repository at a time.
a. Requiring no action by any other party.
b. A SHA256 repository can communicate with SHA-1 Git servers and
clients (push/fetch).
c. Users can use SHA-1 and SHA256 identifiers for objects
interchangeably.
d. New signed objects make use of a stronger hash function than
SHA-1 for their security guarantees.
2. Allow a complete transition away from SHA-1.
a. Local metadata for SHA-1 compatibility can be dropped in a
repository if compatibility with SHA-1 is no longer needed.
3. Maintainability throughout the process.
a. The object format is kept simple and consistent.
b. Creation of a generalized repository conversion tool.
Non-Goals
---------
1. Add SHA256 support to Git protocol. This is valuable and the
logical next step but it is out of scope for this initial design.
2. Transparently improving the security of existing SHA-1 signed
objects.
3. Intermixing objects using multiple hash functions in a single
repository.
4. Taking the opportunity to fix other bugs in git's formats and
protocols.
5. Shallow clones and fetches into a SHA256 repository. (This will
change when we add SHA256 support to Git protocol.)
6. Skip fetching some submodules of a project into a SHA256
repository. (This also depends on SHA256 support in Git protocol.)
Overview
--------
We introduce a new repository format extension `sha256`. Repositories
with this extension enabled use SHA256 instead of SHA-1 to name their
objects. This affects both object names and object content --- both
the names of objects and all references to other objects within an
object are switched to the new hash function.
sha256 repositories cannot be read by older versions of Git.
Alongside the packfile, a sha256 stores a bidirectional mapping
between sha256 and sha1 object names. The mapping is generated locally
and can be verified using "git fsck". Object lookups use this mapping
to allow naming objects using either their sha1 and sha256 names
interchangeably.
"git cat-file" and "git hash-object" gain options to display a sha256
object in its sha1 form and write a sha256 object given its sha1 form.
This requires all objects referenced by that object to be present in
the object database so that they can be named using the appropriate
name (using the bidirectional hash mapping).
Fetches from a SHA-1 based server convert the fetched objects into
sha256 form and record the mapping in the bidirectional mapping table
(see below for details). Pushes to a SHA-1 based server convert the
objects being pushed into sha1 form so the server does not have to be
aware of the hash function the client is using.
Detailed Design
---------------
Object names
~~~~~~~~~~~~
Objects can be named by their 40 hexadecimal digit sha1-name or 64
hexadecimal digit sha256-name, plus names derived from those (see
gitrevisions(7)).
The sha1-name of an object is the SHA-1 of the concatenation of its
type, length, a nul byte, and the object's sha1-content. This is the
traditional <sha1> used in Git to name objects.
The sha256-name of an object is the SHA-256 of the concatenation of
its type, length, a nul byte, and the object's sha256-content.
Object format
~~~~~~~~~~~~~
Objects are stored using a compressed representation of their
sha256-content. The sha256-content of an object is the same as its
sha1-content, except that:
* objects referenced by the object are named using their sha256-names
instead of sha1-names
* signed tags, commits, and merges of signed tags get some additional
fields (see below)
The format allows round-trip conversion between sha256-content and
sha1-content.
Loose objects use zlib compression and packed objects use the packed
format described in Documentation/technical/pack-format.txt, just like
today.
Translation table
~~~~~~~~~~~~~~~~~
A fast bidirectional mapping between sha1-names and sha256-names of
all local objects in the repository is kept on disk. The exact format
of that mapping is to be determined.
All operations that make new objects (e.g., "git commit") add the new
objects to the translation table.
Reading an object's sha1-content
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The sha1-content of an object can be read by converting all
sha256-names its sha256-content references to sha1-names using the
translation table. There is an additional minor transformation needed
for signed tags, commits, and merges (see below).
Fetch
~~~~~
Fetching from a SHA-1 based server requires translating between SHA-1
and SHA-256 based representations on the fly.
SHA-1s named in the ref advertisement can be translated to SHA-256 and
looked up as local objects using the translation table.
Negotiation proceeds as today. Any "have"s or "want"s generated
locally are converted to SHA-1 before being sent to the server, and
SHA-1s mentioned by the server are converted to SHA-256 when looking
them up locally.
After negotiation, the server sends a packfile containing the
requested objects. We convert the packfile to SHA-256 format using the
following steps:
1. index-pack: inflate each object in the packfile and compute its
SHA-1. Objects can contain deltas in OBJ_REF_DELTA format against
objects the client has locally. These objects can be looked up using
the translation table and their sha1-content read as described above
to resolve the deltas.
2. topological sort: starting at the "want"s from the negotiation
phase, walk through objects in the pack and emit a list of them in
topologically sorted order. (This list only contains objects
reachable from the "wants". If the pack from the server contained
additional extraneous objects, then they will be discarded.)
3. convert to sha256: open a new (sha256) packfile. Read the
topologically sorted list just generated in reverse order. For each
object, inflate its sha1-content, convert to sha256-content, and
write it to the sha256 pack. Write an idx file for this pack and
include the new sha1<->sha256 mapping entry in the translation
table.
4. clean up: remove the SHA-1 based pack file, index, and
topologically sorted list obtained from the server and steps 1 and 2.
Step 3 requires every object referenced by the new object to be in the
translation table. This is why the topological sort step is necessary.
As an optimization, step 1 can write a file describing what objects
each object it has inflated from the packfile references. This makes
the topological sort in step 2 possible without inflating the objects
in the packfile for a second time. The objects need to be inflated
again in step 3, for a total of two inflations.
Push
~~~~
Push is simpler than fetch because the objects referenced by the
pushed objects are already in the translation table. The sha1-content
of each object being pushed can be read as described in the "Reading
an object's sha1-content" section to generate the pack written by git
send-pack.
Signed Objects
~~~~~~~~~~~~~~
Commits
^^^^^^^
Commits currently have the following sequence of header lines:
"tree" SP object-name
("parent" SP object-name)*
"author" SP ident
"committer" SP ident
("mergetag" SP object-content)?
("gpgsig" SP pgp-signature)?
We introduce new header lines "hash" and "nohash" that come after the
"gpgsig" field. No "hash" lines may appear unless the "gpgsig" field
is present.
Hash lines have the form
"hash" SP hash-function SP field SP alternate-object-name
Nohash lines have the form
"nohash" SP hash-function
There are only two recognized values of hash-function: "sha1" and
"sha256". "git fsck" will tolerate values of hash-function it does not
recognize, as long as they do not come before either of those two. All
"nohash" lines come before all "hash" lines. Any "hash sha1" lines
must come before all "hash sha256" lines, and likewise for nohash. The
Git project determines any future supported hash-functions that can
come after those two and their order.
There can be at most one "nohash <hash-function>" for one hash
function, indicating that this hash function should not be used when
checking the commit's signature.
There is one "hash <hash-function>" line for each tree or parent field
in the commit object header. The hash lines record object names for
those trees and parents using the indicated hash function, to be used
when checking the commit's signature.
TODO: simplify signature rules, handle the mergetag field better.
sha256-content of signed commits
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The sha256-content of a commit with a "gpgsig" header can include no
hash and nohash lines, a "nohash sha256" line and "hash sha1", or just
a "hash sha1" line.
Examples:
1. tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
2. tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
nohash sha256
hash sha1 tree c7b1cff039a93f3600a1d18b82d26688668c7dea
hash sha1 parent c33429be94b5f2d3ee9b0adad223f877f174b05d
hash sha1 parent 04b871796dc0420f8e7561a895b52484b701d51a
3. tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
hash sha1 tree c7b1cff039a93f3600a1d18b82d26688668c7dea
hash sha1 parent c33429be94b5f2d3ee9b0adad223f877f174b05d
hash sha1 parent 04b871796dc0420f8e7561a895b52484b701d51a
sha1-content of signed commits
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The sha1-content of a commit with a "gpgsig" header can contain a
"nohash sha1" and "hash sha256" line, no hash or nohash lines, or just
a "hash sha256" line.
Examples:
1. tree c7b1cff039a93f3600a1d18b82d26688668c7dea
parent c33429be94b5f2d3ee9b0adad223f877f174b05d
parent 04b871796dc0420f8e7561a895b52484b701d51a
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
nohash sha1
hash sha256 tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
hash sha256 parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
hash sha256 parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
2. tree c7b1cff039a93f3600a1d18b82d26688668c7dea
parent c33429be94b5f2d3ee9b0adad223f877f174b05d
parent 04b871796dc0420f8e7561a895b52484b701d51a
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
3. tree c7b1cff039a93f3600a1d18b82d26688668c7dea
parent c33429be94b5f2d3ee9b0adad223f877f174b05d
parent 04b871796dc0420f8e7561a895b52484b701d51a
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
gpgsig ...
hash sha256 tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
hash sha256 parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
hash sha256 parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
Converting signed commits
^^^^^^^^^^^^^^^^^^^^^^^^^
To convert the sha1-content of a signed commit to its sha256-content:
1. Change "tree" and "parent" lines to use the sha256-names of
referenced objects, as with unsigned commits.
2. If there is a "mergetag" field, convert it from sha1-content to
sha256-content, as with unsigned commits with a mergetag (see the
"Mergetag" section below).
3. Unless there is a "nohash sha1" line, add a full set of "hash sha1
<field> <sha1>" lines indicating the sha1-names of the tree and
parents.
4. Remove any "hash sha256 <field> <sha256>" lines. If no such lines
were present, add a "nohash sha256" line.
Converting the sha256-content of a signed commit to sha1-content uses
the same process with sha1 and sha256 switched.
Verifying signed commit signatures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If the commit has a "hash sha1" line (or is sha1-content without a
"nohash sha1" line): check that the signature matches the sha1-content
with gpgsig field stripped out.
Otherwise: check that the signature matches the sha1-content with
gpgsig, nohash, tree, and parents fields stripped out.
With the examples above, the signed payloads are
1. author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
hash sha256 tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
hash sha256 parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
hash sha256 parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
2. tree c7b1cff039a93f3600a1d18b82d26688668c7dea
parent c33429be94b5f2d3ee9b0adad223f877f174b05d
parent 04b871796dc0420f8e7561a895b52484b701d51a
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
3. tree c7b1cff039a93f3600a1d18b82d26688668c7dea
parent c33429be94b5f2d3ee9b0adad223f877f174b05d
parent 04b871796dc0420f8e7561a895b52484b701d51a
author A U Thor <author@example.com> 1465982009 +0000
committer C O Mitter <committer@example.com> 1465982009 +0000
hash sha1
hash sha256 tree 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
hash sha256 parent e094bc809626f0a401a40d75c56df478e546902ff812772c4594265203b23980
hash sha256 parent 1059dab4748aa33b86dad5ca97357bd322abaa558921255623fbddd066bb3315
Current versions of "git verify-commit" can verify examples (2) and (3)
(but not (1)).
Tags
~~~~
Tags currently have the following sequence of header lines:
"object" SP object-name
"type" SP type
"tag" SP identifier
"tagger" SP ident
A tag's signature, if it exists, is in the message body.
We introduce new header lines "nohash" and "hash" that come after the
"tagger" field. No "nohash" or "hash" lines may appear unless the
message body contains a PGP signature.
As with commits, "nohash" lines have the form "nohash
<hash-function>", indicating that this hash function should not be
used when checking the tag's signature.
"hash" lines have the form
"hash" SP hash-function SP alternate-object-name
This records the pointed-to object name using the indicated hash
function, to be used when checking the tag's signature.
As with commits, "sha1" and "sha256" are the only permitted values of
hash-function and can only appear in that order for a field when they
appear. There can be at most one "nohash" line, and it comes before
any "hash" lines. There can be only one "hash" line for a given hash
function.
sha256-content of signed tags
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The sha256-content of a signed tag can include no "hash" or "nohash"
lines, a "nohash sha256" and "hash sha1 <sha1>" line, or just a "hash
sha1 <sha1>" line.
Examples:
1. object 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
type tree
tag v1.0
tagger C O Mitter <committer@example.com> 1465981006 +0000
Tag Demo v1.0
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJXYRhOAAoJEGEJLoW3InGJklkIAIcnhL7RwEb/+QeX9enkXhxn
...
2. object 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
type tree
tag v1.0
tagger C O Mitter <committer@example.com> 1465981006 +0000
nohash sha256
hash sha1 c7b1cff039a93f3600a1d18b82d26688668c7dea
Tag Demo v1.0
-----BEGIN PGP SIGNATURE-----
...
3. object 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
type tree
tag v1.0
tagger C O Mitter <committer@example.com> 1465981006 +0000
hash sha1 c7b1cff039a93f3600a1d18b82d26688668c7dea
Tag Demo v1.0
...
sha1-content of signed tags
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The sha1-content of a signed tag can include a "nohash sha1" and "hash
sha256" line, no "nohash" or "hash" lines, or just a "hash sha256
<sha256>" line.
Examples:
1. object c7b1cff039a93f3600a1d18b82d26688668c7dea
...
tagger C O Mitter <committer@example.com> 1465981006 +0000
nohash sha1
hash sha256 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
Tag Demo v1.0
-----BEGIN PGP SIGNATURE-----
...
2. object c7b1cff039a93f3600a1d18b82d26688668c7dea
...
tagger C O Mitter <committer@example.com> 1465981006 +0000
Tag Demo v1.0
-----BEGIN PGP SIGNATURE-----
...
3. object c7b1cff039a93f3600a1d18b82d26688668c7dea
...
tagger C O Mitter <committer@example.com> 1465981006 +0000
hash sha256 98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
Tag Demo v1.0
-----BEGIN PGP SIGNATURE-----
...
Signed tags can be converted between sha1-content and sha256-content
using the same process as signed commits.
Verifying signed tags
^^^^^^^^^^^^^^^^^^^^^
As with commits, if the tag has a "hash sha1" (or is sha1-content
without a "nohash sha1" line): check that the signature matches the
sha1-content with PGP signature stripped out.
Otherwise: check that the signature matches the sha1-content with
nohash and object fields and PGP signature stripped out.
Mergetag signatures
~~~~~~~~~~~~~~~~~~~
The mergetag field in the sha1-content of a commit contains the
sha1-content of a tag that was merged by that commit.
The mergetag field in the sha256-content of the same commit contains
the sha256-content of the same tag.
Submodules
~~~~~~~~~~
To convert recorded submodule pointers, you need to have the converted
submodule repository in place. The bidirectional mapping of the
submodule can be used to look up the new hash.
Caveats
-------
Shallow clone and submodules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because this requires all referenced objects to be available in the
locally generated translation table, this design does not support
shallow clone or unfetched submodules.
Protocol improvements might allow lifting this restriction.
Alternatives considered
-----------------------
Upgrading everyone working on a particular project on a flag day
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Projects like the Linux kernel are large and complex enough that
flipping the switch for all projects based on the repository at once
is infeasible.
Not only would all developers and server operators supporting
developers have to switch on the same flag day, but supporting tooling
(continuous integration, code review, bug trackers, etc) would have to
be adapted as well. This also makes it difficult to get early feedback
from some project participants testing before it is time for mass
adoption.
Using hash functions in parallel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(e.g. https://public-inbox.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/ )
Objects newly created would be addressed by the new hash, but inside
such an object (e.g. commit) it is still possible to address objects
using the old hash function.
* You cannot trust its history (needed for bisectability) in the
future without further work
* Maintenance burden as the number of supported hash functions grows
(they will never go away, so they accumulate). In this proposal, by
comparison, converted objects lose all references to SHA-1 except
where needed to verify signatures.
^ permalink raw reply
* [PATCH] http: release strbuf on disabled alternates
From: Eric Wong @ 2017-03-04 1:50 UTC (permalink / raw)
To: Junio C Hamano, Jeff King; +Cc: git
This likely has no real-world impact on memory usage,
but it is cleaner for future readers.
Fixes: abcbdc03895f ("http: respect protocol.*.allow=user for http-alternates")
Signed-off-by: Eric Wong <e@80x24.org>
---
http-walker.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/http-walker.c b/http-walker.c
index b34b6ace7c..2c85316112 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -314,6 +314,8 @@ static void process_alternates_response(void *callback_data)
while (tail->next != NULL)
tail = tail->next;
tail->next = newalt;
+ } else {
+ strbuf_release(&target);
}
}
}
--
EW
^ permalink raw reply related
* Re: [PATCH] http: inform about alternates-as-redirects behavior
From: Jeff King @ 2017-03-04 3:13 UTC (permalink / raw)
To: Eric Wong
Cc: Junio C Hamano, Jann Horn, Brandon Williams, git, sbeller, bburky,
jrnieder
In-Reply-To: <20170304013504.GA27183@untitled>
On Sat, Mar 04, 2017 at 01:35:04AM +0000, Eric Wong wrote:
> It is disconcerting for users to not notice the behavior
> change in handling alternates from commit cb4d2d35c4622ec2
> ("http: treat http-alternates like redirects")
>
> Give the user a hint about the config option so they can
> see the URL and decide whether or not they want to enable
> http.followRedirects in their config.
Yeah, I agree it makes sense to notify the user.
> diff --git a/http-walker.c b/http-walker.c
> index b34b6ace7..626badfe6 100644
> --- a/http-walker.c
> +++ b/http-walker.c
> @@ -168,6 +168,12 @@ static int is_alternate_allowed(const char *url)
> };
> int i;
>
> + if (http_follow_config != HTTP_FOLLOW_ALWAYS) {
> + warning("alternate disabled by http.followRedirects!=true: %s",
> + url);
> + return 0;
> + }
> +
> for (i = 0; i < ARRAY_SIZE(protocols); i++) {
> const char *end;
> if (skip_prefix(url, protocols[i], &end) &&
> @@ -331,9 +337,6 @@ static void fetch_alternates(struct walker *walker, const char *base)
> struct alternates_request alt_req;
> struct walker_data *cdata = walker->data;
>
> - if (http_follow_config != HTTP_FOLLOW_ALWAYS)
> - return;
> -
I was surprised from the description to see not just the addition of a
warning, but a movement of the enforcement code.
I think it's necessary because the original did not bother even fetching
http-alternates if we were not going to respect it. Whereas the new code
will fetch and parse it, and warn only if we actually found something in
it. Which seems reasonable.
The warning itself:
> + warning("alternate disabled by http.followRedirects!=true: %s",
feels like it could use some whitespace around the "!=", but maybe
that's just me.
-Peff
^ permalink raw reply
* Re: [PATCH] http: inform about alternates-as-redirects behavior
From: Jeff King @ 2017-03-04 3:49 UTC (permalink / raw)
To: Eric Wong
Cc: Junio C Hamano, Jann Horn, Brandon Williams, git, sbeller, bburky,
jrnieder
In-Reply-To: <20170304031314.32bta4prahf7pfp7@sigill.intra.peff.net>
On Fri, Mar 03, 2017 at 10:13:14PM -0500, Jeff King wrote:
> > - if (http_follow_config != HTTP_FOLLOW_ALWAYS)
> > - return;
> > -
>
> I was surprised from the description to see not just the addition of a
> warning, but a movement of the enforcement code.
>
> I think it's necessary because the original did not bother even fetching
> http-alternates if we were not going to respect it. Whereas the new code
> will fetch and parse it, and warn only if we actually found something in
> it. Which seems reasonable.
One side effect of this is that it exposes[1] the http-alternates
parsing code to the server's input, even if we aren't planning on using
the result. That code is not very well audited. Just looking at the
context from your patches, I noticed one obvious memory access problem.
The fix is below.
-Peff
[1] Obviously this same code was exposed prior to the redirect-limiting
patches, so it's not like there aren't tons of older clients that
exhibit the same behavior. But IMHO one of the beneficial side
effects of the redirect-limiting was that it avoided this largely
unused and untested code entirely.
-- >8 --
Subject: http-walker: fix buffer underflow processing remote alternates
If we parse a remote alternates (or http-alternates) file,
we expect relative lines like:
../../foo.git/objects
which we convert into "$URL/../foo.git/" (and then use that
as a base for fetching more objects).
But if the remote feeds us nonsense like just:
../
we will try to blindly strip the last 7 characters, assuming
they contain the string "objects". Since we don't _have_ 7
characters at all, this results in feeding a small negative
value to strbuf_add(), which converts it to a size_t,
resulting in a big positive value. This should consistently
fail (since we can't generally allocate the max size_t minus
7 bytes), so there shouldn't be any security implications
(and even if we did allocate, we'd just copy in gigabytes of
garbage, not overflow a buffer).
Let's fix this by using strbuf_strip_suffix() to drop the
characters we want. As a bonus this lets us handle names
that do not end in "objects" (all git repos do, but there is
nothing to say that an alternate object store needs to be a
git repo).
And while we're here, we can add a few other parsing
niceties, like dropping trailing whitespace, and handling
names that do not end in "/".
Signed-off-by: Jeff King <peff@peff.net>
---
http-walker.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/http-walker.c b/http-walker.c
index b34b6ace7..d62ca8953 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -296,11 +296,13 @@ static void process_alternates_response(void *callback_data)
okay = 1;
}
}
- /* skip "objects\n" at end */
if (okay) {
struct strbuf target = STRBUF_INIT;
strbuf_add(&target, base, serverlen);
- strbuf_add(&target, data + i, posn - i - 7);
+ strbuf_add(&target, data + i, posn - i);
+ strbuf_rtrim(&target);
+ strbuf_strip_suffix(&target, "objects");
+ strbuf_complete(&target, '/');
if (is_alternate_allowed(target.buf)) {
warning("adding alternate object store: %s",
--
2.12.0.404.g442e75cca
^ permalink raw reply related
* Re: [PATCH] http: release strbuf on disabled alternates
From: Jeff King @ 2017-03-04 3:42 UTC (permalink / raw)
To: Eric Wong; +Cc: Junio C Hamano, git
In-Reply-To: <20170304015016.GA25408@starla>
On Sat, Mar 04, 2017 at 01:50:16AM +0000, Eric Wong wrote:
> This likely has no real-world impact on memory usage,
> but it is cleaner for future readers.
Yep, this looks good.
-Peff
^ permalink raw reply
* Re: [PATCH v1] Travis: also test on 32-bit Linux
From: Junio C Hamano @ 2017-03-04 4:11 UTC (permalink / raw)
To: Lars Schneider, Johannes Schindelin; +Cc: Git Mailing List
In-Reply-To: <xmqq8tol7vs1.fsf@gitster.mtv.corp.google.com>
On Fri, Mar 3, 2017 at 4:03 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> I only recently started looking at Travis logs, so I cannot tell if
> it is just a usual flake (e.g. some builds a few days ago seems to
> have failed due to not being able to check out the tree being
> tested, which I do not think is our fault) that we shouldn't worry
> about, or if it is a sign of a real problem.
Tonight's pushout also seems to stall the same way. Dscho's
unversioned one didn't exhibit the problem?
https://travis-ci.org/git/git/jobs/206811396
> Unrelated to linux-32, the same build has hard failure with Apple
> clang in t0021 with the rot13-filter.pl thing, by the way.
This one may be a Heisenbug which may indicate some raciness in the
clean/smudge filter protocol.
The latest build of 'pu' https://travis-ci.org/git/git/jobs/207550171 seems to
have passed OK.
^ permalink raw reply
* Re: [PATCH v1 1/1] git diff --quiet exits with 1 on clean tree with CRLF conversions
From: Torsten Bögershausen @ 2017-03-04 6:25 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Mike Crowe, git, Jeff King
In-Reply-To: <xmqq37eu2qxl.fsf@junio-linux.mtv.corp.google.com>
On 2017-03-03 18:47, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@web.de> writes:
>
>> Understood, thanks for the explanation.
>>
>> quiet is not quite any more..
>>
>> Does the following fix help ?
>>
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -2826,6 +2826,8 @@ int diff_populate_filespec(struct diff_filespec *s,
>> unsigned int flags)
>> enum safe_crlf crlf_warn = (safe_crlf == SAFE_CRLF_FAIL
>> ? SAFE_CRLF_WARN
>> : safe_crlf);
>> + if (size_only)
>> + crlf_warn = SAFE_CRLF_FALSE;
>
> If you were to go this route, it may be sufficient to change its
> initialization from WARN to FALSE _unconditionally_, because this
> function uses the convert_to_git() only to _show_ the differences by
> computing canonical form out of working tree contents, and the
> conversion is not done to _write_ into object database to create a
> new object.
Hm, since when (is it not used) ?
I thought that it is needed to support the safecrlf handling introduced in
21e5ad50fc5e7277c74cfbb3cf6502468e840f86
Author: Steffen Prohaska <prohaska@zib.de>
Date: Wed Feb 6 12:25:58 2008 +0100
safecrlf: Add mechanism to warn about irreversible crlf conversions
-------------
The SAFE_CRLF_FAIL was converted into WARN here:
commit 5430bb283b478991a979437a79e10dcbb6f20e28
Author: Junio C Hamano <gitster@pobox.com>
Date: Mon Jun 24 14:35:04 2013 -0700
diff: demote core.safecrlf=true to core.safecrlf=warn
Otherwise the user will not be able to start to guess where in the
contents in the working tree the offending unsafe CR lies.
------------
My understanding is that we don't want to break the safecrlf feature,
but after applying
diff --git a/diff.c b/diff.c
index a628ac3a95..a05d88dd9f 100644
--- a/diff.c
+++ b/diff.c
@@ -2820,12 +2820,10 @@ int diff_populate_filespec(struct diff_filespec *s,
unsigned int flags)
int size_only = flags & CHECK_SIZE_ONLY;
int err = 0;
/*
- * demote FAIL to WARN to allow inspecting the situation
- * instead of refusing.
+ * Don't use FAIL or WARN as this code is not called when _writing_
+ * into object database to create a new object.
*/
- enum safe_crlf crlf_warn = (safe_crlf == SAFE_CRLF_FAIL
- ? SAFE_CRLF_WARN
- : safe_crlf);
+ enum safe_crlf crlf_warn = SAFE_CRLF_FALSE;
None of the test cases in t0020--t0027 fails or complain about missing warnings.
Does this all means that, looking back, 5430bb283b478991 could have been more
aggressive and could have used SAFE_CRLF_FALSE ?
And we can do this change now?
(If the answer is yes, we don't need to deal with the problem below)
> Having size_only here is not a sign of getting --quiet passed from
> the command line, by the way.
>
^ permalink raw reply related
* Re: [PATCH] http: inform about alternates-as-redirects behavior
From: Eric Wong @ 2017-03-04 6:55 UTC (permalink / raw)
To: Jeff King
Cc: Junio C Hamano, Jann Horn, Brandon Williams, git, sbeller, bburky,
jrnieder
In-Reply-To: <20170304031314.32bta4prahf7pfp7@sigill.intra.peff.net>
Jeff King <peff@peff.net> wrote:
> The warning itself:
>
> > + warning("alternate disabled by http.followRedirects!=true: %s",
>
> feels like it could use some whitespace around the "!=", but maybe
> that's just me.
Yeah, I kinda wanted to emulate the command-line syntax.
Maybe rewording it a bit and showing how to enable it will
make more sense:
warning("alternate: %s", url);
warning(" may be enabled by -c http.followRedirects=true");
As well as keeping individual lines shorter and hopefully
easier-to-read.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox