* Re: RFD: leveraging GitHub's asciidoc rendering for our Documentation/
From: Scott Chacon @ 2011-09-09 15:45 UTC (permalink / raw)
To: Michael J Gruber; +Cc: Git Mailing List
In-Reply-To: <4E6A23DB.1040606@drmicha.warpmail.net>
Hey,
On Fri, Sep 9, 2011 at 7:34 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> which has all the renaming (*.txt -> *.asciidoc) and Makefile and script
> changes, but is missing some include changes (because include breaks
> anyway, see below).
I can change this so we can render .asc if that's less ugly. I've
been meaning to do this for a while, but I don't think I ever
incorporated it.
>
> Our own customisation is not loaded (of course) so that, e.g., the
> linkgit macro does not work; and the include statement makes GitHub's
> parser unhappy and choke.
>
> Does anybody feel this is worth pursuing?
>
> + Nicer blob view
> + Simpler way to judge documentation changes
> - Need to get our asciidoc config in there
> - GitHub's parser neeeds to learn include
If this is interesting to people, I can help out. However, you can
also fix rendering issues, as long as they don't introduce security
issues (which 'include' and macros can do) by fixing it here:
https://github.com/github/markup
That is the rendering engine we use. If you make it work how you want
and it's safe and you send us a pull request, we'll happily take fixes
to the asciidoc renderer.
Scott
^ permalink raw reply
* Re: [PATCH 2/2] push -s: skeleton
From: Jeff King @ 2011-09-09 15:34 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Shawn O. Pearce
In-Reply-To: <7v7h5iwub9.fsf@alter.siamese.dyndns.org>
On Thu, Sep 08, 2011 at 03:19:54PM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Yeah, it is a potential problem, but it just seems wrong to put too much
> > policy work onto the server.
>
> My take on it is somewhat different. The only thing in the end result we
> want to see is that the pushed commits are annotated with GPG signatures
> in the notes tree, and there is no reason for us to cast in stone that
> there has to be any significance in the commit history of the notes tree.
Hmm. Is order really irrelevant? If you push a commit to master, moving
it from X to Y, then push-rewind it back to X, then push a new commit Z,
how do I cryptographically determine the correct final state of master?
I'll see two push-certs, one going X..Y and one X..Z. They'll have
timestamps, which can be used for ordering. But what if the first and
third actions above are done by different people. Now you're trusting
that their clocks are synced to order them properly.
Note that simply keeping an unsigned but ordered notes history doesn't
solve this problem, either; you'd probably want a parent pointer in your
push cert saying "and this is the previous push cert I am based on".
And maybe this is a use case we don't care about. Maybe it's enough for
the push-cert to say "at some point in time, I thought it was a good
idea to push these commits into master; signed, me".
But I'm not really clear on exactly what the security goals are. The
series you sent looks interesting, but I haven't seen the verification
side of these signatures. What are they going to be used for? What
guarantees are we attempting to provide? For that matter, what is our
threat model? What are attackers capable of?
> In a busy hosting site that has many branches being pushed simultaneously,
> it is entirely plausible that the server side may just want to store each
> received push certificate in a new flat file in a filesystem, and have
> asynchronous process sweep the new certificates to update the notes tree,
> possibly creating a single notes tree commit that records updates by
> multiple pushes, for performance purposes, in its implementation of
> record_signed_push() in receive-pack.
OK, I see. It is not "the server can do whatever it likes with the
information" as much as "the server can do whatever it likes, but at the
very least should eventually create a notes tree of a given form".
-Peff
^ permalink raw reply
* Re: [PATCH 0/6] Improved infrastructure for refname normalization
From: Michael Haggerty @ 2011-09-09 15:33 UTC (permalink / raw)
To: gitzilla; +Cc: git, Junio C Hamano, cmn
In-Reply-To: <4E6A1D7D.6050602@gmail.com>
On 09/09/2011 04:06 PM, A Large Angry SCM wrote:
> On 09/09/2011 07:46 AM, Michael Haggerty wrote:
>> As a prerequisite to storing references caches hierarchically (itself
>> needed for performance reasons), here is a patch series to help us get
>> refname normalization under control.
>>
>> The problem is that some UI accepts unnormalized reference names (like
>> "/foo/bar" or "foo///bar" instead of "foo/bar") and passes them on to
>> library routines without normalizing them. The library, on the other
>> hand, assumes that the refnames are normalized. Sometimes (mostly in
>> the case of loose references) unnormalized refnames happen to work,
>> but in other cases (like packed references or when looking up refnames
>> in the cache) they silently fail. Given that refnames are sometimes
>> treated as path names, there is a chance that some security-relevant
>> bugs are lurking in this area, if not in git proper then in scripts
>> that interact with git.
>
> Why can't the library do the normalization instead of expecting every
> other component that deals with reference names having to do it for the
> library?
The library could do the normalization, but
1. It would probably cost a lot of redundant checks as reference names
pass in and out of the library and back in again
2. Normalization requires copying or overwriting the incoming string, so
each time a refname crosses the library perimeter there might have to be
an extra memory allocation with the associated headaches of dealing with
the ownership of the memory.
3. The library doesn't encapsulate all uses of reference names; for
example, for_each_ref() invokes a callback function with the refname as
an argument. The callback function is free to do a strcmp() of the
refname (normalized by the library) with some arbitrary string that it
got from the command line. Either the caller has to do the
normalization itself (i.e., outside of the library) or the library has
to learn how to do every possible filtering operation with refnames.
>> * Forbid ".lock" at the end of any refname component, as directories
>> with such names can conflict with attempts to create lock files for
>> other refnames.
>
> I find this overly restrictive. If you need to create a lock based on a
> reference name or component, use a name for the lock object that starts
> with one of the characters that reference names or components are
> already forbidden from starting with.
I agree; this is unpleasantly restrictive.
But please remember that refnames already cannot end in ".lock"
("foo/bar.lock" is already forbidden; this change also prohibits
"foo.lock/bar").
However, your suggested solution would cause problems if two versions of
git are running on the same machine. An old version of git would not
know to respect the new version's lock files. ISTM that this would be
too dangerous. Suggestions welcome.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: [PATCH 2/2] push -s: skeleton
From: Jeff King @ 2011-09-09 15:22 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Junio C Hamano, git
In-Reply-To: <CAJo=hJsQvRN3Z0xJg9q37Km1g_1qUdJKNQ6n8=a9mv3YjugyVw@mail.gmail.com>
On Thu, Sep 08, 2011 at 03:07:25PM -0700, Shawn O. Pearce wrote:
> A notes tree per ref is ugly when you have a lot of branches. Its a
> problem when you merge commits from say "maint" over to "master" 2
> days after they were initially pushed. Ideally the notes tree for
> master also includes what you brought over.
I agree you can end up with a lot of refs if you have a lot of branches,
and that may get a bit unwieldy. I don't see how the merge thing is a
problem, though. If you do the merge at a client, then those commits
will hit master by push, and you'll get entries in the cert tree for
master. If you do the merge locally, then there's not going to be a
signature on those commits hitting the master ref, no matter what the
storage scheme.
> However, maybe it is reasonable for a protocol extension to support
> "auto-notes-merge" on a refs/notes/ ref if the client asks for it in
> the send-pack/receive-pack command stream? Clients could push notes
> and have the server automatically merge the client's pushed commit
> into the notes tree, either by fast-forward or by performing an
> automatic notes merge, with concat being applied to non-identical
> notes on the same SHA-1. This would allow the client to prepare his
> local certificate, and push that notes tree, while still working
> around the race on the server side.
>
> If the server doesn't support the protocol extension, the client can
> still push his signed notes, he just may run into a race with another
> concurrent user pushing into the same repository. Which then means
> that an upgrade of the server is really only important/necessary if
> you have "central repository" model that a lot of users push into. If
> you use the traditional workflow that GitHub encourages of per-user
> repositories, you would never have a race on the server, and wouldn't
> need the upgraded server binary with "auto-notes-merge".
I like this approach, as the protocol extension provides the minimal
building block that can be used to implement this, or any other
notes-related scheme.
-Peff
^ permalink raw reply
* Re: git repository size / compression
From: neubyr @ 2011-09-09 15:09 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Carlos Martín Nieto, git, pjweisberg
In-Reply-To: <m339g5u5pm.fsf@localhost.localdomain>
2011/9/9 Jakub Narebski <jnareb@gmail.com>:
> neubyr <neubyr@gmail.com> writes:
>> On Fri, Sep 9, 2011 at 3:23 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
>> > On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
>
>>>> I have a test git repository with just two files in it. One of the
>>>> file in it has a set of two lines that is repeated n times.
>>>> e.g.:
>>>> {{{
>>>> $ for i in {1..5}; do cat ./lexico.txt>> lexico1.txt && cat
>>>> ./lexico.txt>> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
>>>> }}}
>>>>
>>>
>>> So you've just created some data that can be compressed quite
>>> efficiently.
>>>
>>>> I ran above command few times and performed commit after each run. Now
>>>> disk usage of this repository directory is mentioned below. The 419M
>>>> is working directory size and 2.7M is git repository/database size.
>>>>
>>>> {{{
>>>> $ du -h -d 1 .
>>>> 2.7M ./.git
>>>> 419M .
>>>>
>>>> }}}
>
> Have you tried the same but with
>
> $ git gc --prune=now
>
> before running `du`?
>
Nope, I hadn't run git gc before. Here are du results after running
git gc command. That's about 55% less space now.. Great!
{{{
$ du -d 1 -h
924K ./.git
417M .
}}}
>>>> Is it because of the compression performed by git before storing data
>>>> (or before sending commit)??
>>>
>>> Yes. Git stores its objects (the commit, the snapshot of the files,
>>> etc.) compressed. When these objects are stored in a pack, the size can
>>> be further reduced by storing some objects as deltas which describe the
>>> difference between itself and some other object in the object-db.
>>
>> Does git store deltas for some files? I thought it uses snapshots
>> (exact copy of staged files) only.
>
> When creating packfile from loose objects (e.g. via `git gc`), it
> does perform delta compression.
>
> --
> Jakub Narębski
>
thank you everyone for explaining in detail..
--
neuby.r
^ permalink raw reply
* Re: git repository size / compression
From: neubyr @ 2011-09-09 15:07 UTC (permalink / raw)
To: Carlos Martín Nieto; +Cc: git
In-Reply-To: <1315578547.4377.2.camel@centaur.lab.cmartin.tk>
On Fri, Sep 9, 2011 at 9:28 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
> On Fri, 2011-09-09 at 09:04 -0500, neubyr wrote:
>> On Fri, Sep 9, 2011 at 3:23 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
>> > On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
>> >> I have a test git repository with just two files in it. One of the
>> >> file in it has a set of two lines that is repeated n times.
>> >> e.g.:
>> >> {{{
>> >> $ for i in {1..5}; do cat ./lexico.txt >> lexico1.txt && cat
>> >> ./lexico.txt >> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
>> >> }}}
>> >>
>> >
>> > So you've just created some data that can be compressed quite
>> > efficiently.
>> >
>> >> I ran above command few times and performed commit after each run. Now
>> >> disk usage of this repository directory is mentioned below. The 419M
>> >> is working directory size and 2.7M is git repository/database size.
>> >>
>> >> {{{
>> >> $ du -h -d 1 .
>> >> 2.7M ./.git
>> >> 419M .
>> >>
>> >> }}}
>> >>
>> >> Is it because of the compression performed by git before storing data
>> >> (or before sending commit)??
>> >>
>> >
>> > Yes. Git stores its objects (the commit, the snapshot of the files,
>> > etc.) compressed. When these objects are stored in a pack, the size can
>> > be further reduced by storing some objects as deltas which describe the
>> > difference between itself and some other object in the object-db.
>> >
>>
>> Does git store deltas for some files? I thought it uses snapshots
>> (exact copy of staged files) only.
>
> Yes and no. The data model for git is to always store snapshots, and it
> always expects to have the full files available. In a packfile, however,
> in order to save space, some objects are stored as deltas to other
> objects in the same file.
>
> http://progit.org/book/ch9-4.html
>
Excellent.. That explains compression and deltas really well. Thanks again..
--
neuby.r
^ permalink raw reply
* Re: git repository size / compression
From: Jakub Narebski @ 2011-09-09 14:54 UTC (permalink / raw)
To: neubyr; +Cc: Carlos Martín Nieto, git
In-Reply-To: <CALFxCvxmPN_O_3xpkrGUYtdkVfz5nr7eaucMrAYQ3uvi820FBg@mail.gmail.com>
neubyr <neubyr@gmail.com> writes:
> On Fri, Sep 9, 2011 at 3:23 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
> > On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
>>> I have a test git repository with just two files in it. One of the
>>> file in it has a set of two lines that is repeated n times.
>>> e.g.:
>>> {{{
>>> $ for i in {1..5}; do cat ./lexico.txt>> lexico1.txt && cat
>>> ./lexico.txt>> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
>>> }}}
>>>
>>
>> So you've just created some data that can be compressed quite
>> efficiently.
>>
>>> I ran above command few times and performed commit after each run. Now
>>> disk usage of this repository directory is mentioned below. The 419M
>>> is working directory size and 2.7M is git repository/database size.
>>>
>>> {{{
>>> $ du -h -d 1 .
>>> 2.7M ./.git
>>> 419M .
>>>
>>> }}}
Have you tried the same but with
$ git gc --prune=now
before running `du`?
>>> Is it because of the compression performed by git before storing data
>>> (or before sending commit)??
>>
>> Yes. Git stores its objects (the commit, the snapshot of the files,
>> etc.) compressed. When these objects are stored in a pack, the size can
>> be further reduced by storing some objects as deltas which describe the
>> difference between itself and some other object in the object-db.
>
> Does git store deltas for some files? I thought it uses snapshots
> (exact copy of staged files) only.
When creating packfile from loose objects (e.g. via `git gc`), it
does perform delta compression.
--
Jakub Narębski
^ permalink raw reply
* RFD: leveraging GitHub's asciidoc rendering for our Documentation/
From: Michael J Gruber @ 2011-09-09 14:34 UTC (permalink / raw)
To: Git Mailing List
Hi there,
I've been looking more to GitHub lately and was wondering whether it is
worth to leverage their automatic asciidoc rendering for our asciidoc
files. I have put up a test tree at
https://github.com/gitigit/git/tree/githubtest
which has all the renaming (*.txt -> *.asciidoc) and Makefile and script
changes, but is missing some include changes (because include breaks
anyway, see below).
The simple renaming already gives a rendered display of blobs for
simpler asciidoc files like release notes
https://github.com/gitigit/git/blob/githubtest/Documentation/RelNotes/1.7.7.asciidoc
and api documentation
https://github.com/gitigit/git/blob/githubtest/Documentation/technical/api-credentials.asciidoc
For the man pages, there are several problems as can be seen here:
https://github.com/gitigit/git/blob/githubtest/Documentation/git-blame.asciidoc
Our own customisation is not loaded (of course) so that, e.g., the
linkgit macro does not work; and the include statement makes GitHub's
parser unhappy and choke.
Does anybody feel this is worth pursuing?
+ Nicer blob view
+ Simpler way to judge documentation changes
- Need to get our asciidoc config in there
- GitHub's parser neeeds to learn include
Michael
^ permalink raw reply
* Re: git repository size / compression
From: Carlos Martín Nieto @ 2011-09-09 14:28 UTC (permalink / raw)
To: neubyr; +Cc: git
In-Reply-To: <CALFxCvxmPN_O_3xpkrGUYtdkVfz5nr7eaucMrAYQ3uvi820FBg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2895 bytes --]
On Fri, 2011-09-09 at 09:04 -0500, neubyr wrote:
> On Fri, Sep 9, 2011 at 3:23 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
> > On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
> >> I have a test git repository with just two files in it. One of the
> >> file in it has a set of two lines that is repeated n times.
> >> e.g.:
> >> {{{
> >> $ for i in {1..5}; do cat ./lexico.txt >> lexico1.txt && cat
> >> ./lexico.txt >> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
> >> }}}
> >>
> >
> > So you've just created some data that can be compressed quite
> > efficiently.
> >
> >> I ran above command few times and performed commit after each run. Now
> >> disk usage of this repository directory is mentioned below. The 419M
> >> is working directory size and 2.7M is git repository/database size.
> >>
> >> {{{
> >> $ du -h -d 1 .
> >> 2.7M ./.git
> >> 419M .
> >>
> >> }}}
> >>
> >> Is it because of the compression performed by git before storing data
> >> (or before sending commit)??
> >>
> >
> > Yes. Git stores its objects (the commit, the snapshot of the files,
> > etc.) compressed. When these objects are stored in a pack, the size can
> > be further reduced by storing some objects as deltas which describe the
> > difference between itself and some other object in the object-db.
> >
>
> Does git store deltas for some files? I thought it uses snapshots
> (exact copy of staged files) only.
Yes and no. The data model for git is to always store snapshots, and it
always expects to have the full files available. In a packfile, however,
in order to save space, some objects are stored as deltas to other
objects in the same file.
http://progit.org/book/ch9-4.html
>
>
> >> Following were results with subversion:
> >>
> >> Subversion client (redundant(?) copy exists in .svn/text-base/
> >> directory, hence double size in client):
> >> {{{
> >> $ du -h -d 1
> >> 416M ./.svn
> >> 832M .
> >> }}}
> >
> > Subversion stores the "pristines" (which is the status of the files in
> > the latest revision) inside the .svn directory. I wouldn't call this
> > copy redundant, though, as it allows you to run diff locally. The
> > pristines are stored uncompressed, which is why you half of the space is
> > taken up by the .svn directory.
> >
> >>
> >> Subversion repo/server:
> >> {{{
> >> $ du -h -d 1
> >> 12K ./conf
> >> 1.2M ./db
> >> 36K ./hooks
> >> 8.0K ./locks
> >> 1.2M .
> >> }}}
> >
> > I don't know how the repository is stored in Subversion, but it may also
> > be compressed. You may be able to reduced your git repository size by
> > (re)generating packs with 'git repack' and doing some cleanups with 'git
> > gc', but the repository size is not often a concern.
> >
> > cmn
> >
> >
> >
>
> that's helpful. thanks.
>
> --
> neuby.r
>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply
* Re: git repository size / compression
From: Sverre Rabbelier @ 2011-09-09 14:25 UTC (permalink / raw)
To: neubyr; +Cc: Carlos Martín Nieto, git
In-Reply-To: <CALFxCvxmPN_O_3xpkrGUYtdkVfz5nr7eaucMrAYQ3uvi820FBg@mail.gmail.com>
Heya,
On Fri, Sep 9, 2011 at 16:04, neubyr <neubyr@gmail.com> wrote:
> Does git store deltas for some files? I thought it uses snapshots
> (exact copy of staged files) only.
In packs, yes, it will try to delta objects as efficient as possible.
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: t0300-credentials: poll failed: invalid argument
From: Thomas Rast @ 2011-09-09 14:13 UTC (permalink / raw)
To: Jeff King; +Cc: Brian Gernhardt, Git List
In-Reply-To: <20110829174309.GA11524@sigill.intra.peff.net>
Jeff King wrote:
> On Mon, Aug 29, 2011 at 01:28:05PM -0400, Brian Gernhardt wrote:
>
> > > Ugh, sorry, this is my fault. The check_expiration() function can return
> > > a totally bogus value before we actually get any credentials.
> > >
> > > Does this patch fix it for you?
> >
> > Yes it does! Surprisingly enough, non-bogus parameters keeps poll
> > from erroring with EINVAL. Funny that. ;-)
>
> Great. I'm working on a few more patches on top of that topic, so I'll
> add it to my list to send out in the next day or so.
I'm still seeing this with current pu (from repo.or.cz), but only on
OS X
$ uname -a
Darwin mackeller.inf.ethz.ch 11.1.0 Darwin Kernel Version 11.1.0: Tue Jul 26 16:07:11 PDT 2011; root:xnu-1699.22.81~1/RELEASE_X86_64 x86_64
Where "this" is:
--- expect-stderr 2011-09-09 14:12:13.000000000 +0000
+++ stderr 2011-09-09 14:12:13.000000000 +0000
@@ -1,2 +1,3 @@
askpass: Username:
askpass: Password:
+fatal: poll failed: Invalid argument
for each of the tests 15--19. Is it supposed to be fixed?
I don't have time to look into it without knowing what to search for,
but if you want me to test anything on that OS X just ask.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: [PATCH 0/6] Improved infrastructure for refname normalization
From: A Large Angry SCM @ 2011-09-09 14:06 UTC (permalink / raw)
To: Michael Haggerty; +Cc: git, Junio C Hamano, cmn
In-Reply-To: <1315568778-3592-1-git-send-email-mhagger@alum.mit.edu>
On 09/09/2011 07:46 AM, Michael Haggerty wrote:
> As a prerequisite to storing references caches hierarchically (itself
> needed for performance reasons), here is a patch series to help us get
> refname normalization under control.
>
> The problem is that some UI accepts unnormalized reference names (like
> "/foo/bar" or "foo///bar" instead of "foo/bar") and passes them on to
> library routines without normalizing them. The library, on the other
> hand, assumes that the refnames are normalized. Sometimes (mostly in
> the case of loose references) unnormalized refnames happen to work,
> but in other cases (like packed references or when looking up refnames
> in the cache) they silently fail. Given that refnames are sometimes
> treated as path names, there is a chance that some security-relevant
> bugs are lurking in this area, if not in git proper then in scripts
> that interact with git.
Why can't the library do the normalization instead of expecting every
other component that deals with reference names having to do it for the
library?
[...]
>
> * Forbid ".lock" at the end of any refname component, as directories
> with such names can conflict with attempts to create lock files for
> other refnames.
I find this overly restrictive. If you need to create a lock based on a
reference name or component, use a name for the lock object that starts
with one of the characters that reference names or components are
already forbidden from starting with.
Gitzilla
^ permalink raw reply
* Re: git repository size / compression
From: neubyr @ 2011-09-09 14:04 UTC (permalink / raw)
To: Carlos Martín Nieto; +Cc: git
In-Reply-To: <1315556595.2019.11.camel@bee.lab.cmartin.tk>
On Fri, Sep 9, 2011 at 3:23 AM, Carlos Martín Nieto <cmn@elego.de> wrote:
> On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
>> I have a test git repository with just two files in it. One of the
>> file in it has a set of two lines that is repeated n times.
>> e.g.:
>> {{{
>> $ for i in {1..5}; do cat ./lexico.txt >> lexico1.txt && cat
>> ./lexico.txt >> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
>> }}}
>>
>
> So you've just created some data that can be compressed quite
> efficiently.
>
>> I ran above command few times and performed commit after each run. Now
>> disk usage of this repository directory is mentioned below. The 419M
>> is working directory size and 2.7M is git repository/database size.
>>
>> {{{
>> $ du -h -d 1 .
>> 2.7M ./.git
>> 419M .
>>
>> }}}
>>
>> Is it because of the compression performed by git before storing data
>> (or before sending commit)??
>>
>
> Yes. Git stores its objects (the commit, the snapshot of the files,
> etc.) compressed. When these objects are stored in a pack, the size can
> be further reduced by storing some objects as deltas which describe the
> difference between itself and some other object in the object-db.
>
Does git store deltas for some files? I thought it uses snapshots
(exact copy of staged files) only.
>> Following were results with subversion:
>>
>> Subversion client (redundant(?) copy exists in .svn/text-base/
>> directory, hence double size in client):
>> {{{
>> $ du -h -d 1
>> 416M ./.svn
>> 832M .
>> }}}
>
> Subversion stores the "pristines" (which is the status of the files in
> the latest revision) inside the .svn directory. I wouldn't call this
> copy redundant, though, as it allows you to run diff locally. The
> pristines are stored uncompressed, which is why you half of the space is
> taken up by the .svn directory.
>
>>
>> Subversion repo/server:
>> {{{
>> $ du -h -d 1
>> 12K ./conf
>> 1.2M ./db
>> 36K ./hooks
>> 8.0K ./locks
>> 1.2M .
>> }}}
>
> I don't know how the repository is stored in Subversion, but it may also
> be compressed. You may be able to reduced your git repository size by
> (re)generating packs with 'git repack' and doing some cleanups with 'git
> gc', but the repository size is not often a concern.
>
> cmn
>
>
>
that's helpful. thanks.
--
neuby.r
^ permalink raw reply
* Re: can Git encrypt/decrypt .gpg on push/fetch?
From: Ted Zlatanov @ 2011-09-09 13:52 UTC (permalink / raw)
To: Michael J Gruber; +Cc: Aneesh Bhasin, git
In-Reply-To: <4E6A165D.5010703@drmicha.warpmail.net>
On Fri, 09 Sep 2011 15:36:29 +0200 Michael J Gruber <git@drmicha.warpmail.net> wrote:
MJG> Aneesh Bhasin venit, vidit, dixit 09.09.2011 12:50:
>> Hi Ted,
>>
>>
>> 2011/9/9 Ted Zlatanov <tzz@lifelogs.com>
>>>
>>> I need to store some encrypted files in Git but for some clients with
>>> the right GPG keys, decrypt them on checkout (possibly also encrypt them
>>> back on commit, but that's not as important).
>>>
>>> diff doesn't have to work, this is just for convenience. Can Git do
>>> this (matching only .gpg files) or do I need my own command to run after
>>> the checkout/fetch and before commit? It seems pretty out of Git's
>>> scope but perhaps others have done this before.
>>>
>>
>> Have you looked at git hooks (e.g. here : http://progit.org/book/ch7-3.html).
>>
>> You could do the encryption/decryption in pre-commit and post-checkout
>> hooks scripts respectively...
MJG> I'd recommend textconv for diffing and clean/smudge for plaintext
MJG> checkout. That is, there are two convenient versions:
MJG> A) Keep blobs and checkout encrypted
MJG> - Use an editor which can encrypt/decrypt on the fly (e.g. vim)
MJG> - Use "*.gpg diff=gpg" in your attributes and
MJG> [diff "gpg"]
MJG> textconv = gpg -d
MJG> in your config to have cleartext diffs. Use cachetextconv with caution ;)
MJG> B) Keep blobs encrypted, checkout decrypted
MJG> - Use Use "*.gpg filter=gpg" in your attributes and
MJG> [filter "gpg"]
MJG> smudge = gpg -d
MJG> clean = gpg -e -r yourgpgkey
MJG> in your config.
MJG> I use A on a regular basis. B is untested (but patterned after a similar
MJG> gzip filter I use). You may or may not have better results with "gpg -ea".
MJG> On clients without the keys, you can simply leave out the diff or filter
MJG> config resp. set them to "cat".
That's really helpful, thank you Aneesh and Michael. Exactly what I was
hoping to achieve.
Ted
^ permalink raw reply
* Re: Git is not scalable with too many refs/*
From: Michael Haggerty @ 2011-09-09 13:50 UTC (permalink / raw)
To: Martin Fick; +Cc: git
In-Reply-To: <1315511619144-6773496.post@n2.nabble.com>
On 09/08/2011 09:53 PM, Martin Fick wrote:
> Just thought that I should add some numbers to this thread as it seems that
> the later versions of git are worse off by several orders of magnitude on
> this one.
>
> We have a Gerrit repo with just under 100K refs in refs/changes/*. When I
> fetch them all with git 1.7.6 it does not seem to complete. Even after 5
> days, it is just under half way through the ref #s! [...]
I recently reported very slow performance when doing a "git
filter-branch" involving only about 1000 tags, with hints of O(N^3)
scaling [1]. That could certainly explain enormous runtimes for 100k refs.
References are cached in git in a single linked list, so it is easy to
imagine O(N^2) all over the place (which is bad enough for 100k
references). I am working on improving the situation by reorganizing
how the reference cache is stored in memory, but progress is slow.
I'm not sure whether your problem is related. For example, it is not
obvious to me why the commit that you cite (88a21979) would make the
reference problem so dramatically worse.
I suggest the following experiments to characterize the problem:
1. Fetch the references in batches of a few hundred each, and see if
that dramatically decreases the total time.
2. Same as (1), except run "git pack-refs --all --prune" between the
batches. In my experiments, packing references made a dramatic
difference in runtimes.
3. Try using the --no-replace-objects option (I assume that it can be
used like "git --no-replace-objects fetch ..."). In my case this option
made a dramatic improvement in the runtimes.
4. Try a test using a repository generated something like the test
script that I posted in [1]. If it also gives pathologically bad
performance, then it can serve as a test case to use while we debug the
problem.
Yours,
Michael
[1] http://comments.gmane.org/gmane.comp.version-control.git/177103
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: The imporantance of including http credential caching in 1.7.7
From: Ted Zlatanov @ 2011-09-09 13:31 UTC (permalink / raw)
To: John Szakmeister; +Cc: git, Kyle Neath, Jeff King
In-Reply-To: <CAEBDL5VtVZcmQnj2CH7XzZ0YV_X61gO69-dXriGiYsAqk=NLPg@mail.gmail.com>
On Fri, 9 Sep 2011 06:32:25 -0400 John Szakmeister <john@szakmeister.net> wrote:
JS> [Added back some of the CC's]
JS> Ted: we don't usually cull the CC list on the git mailing list.
Sorry, I followed up via GMane (a NNTP interface to the mailing list).
I'll use `r'eply instead of `f'ollowup.
I actually set
Mail-Copies-To: never
Gmane-Reply-To-List: yes
because I hate getting CC'd on discussions I already follow via GMane.
But that's just my preference :)
JS> 2011/9/9 Ted Zlatanov <tzz@lifelogs.com>:
JS> [snip]
MJG> ... and one for Git on Windows? It seems we're lacking both Win and OS X
MJG> developers here.
>>
>> Windows doesn't have a standard keychain service, does it?
JS> No, it doesn't, but you can use the wincrypt API which allows you to
JS> at least encrypt the password from the user's login credentials. In
JS> particular, CryptProtectData() and CryptUnprotectData(). That way you
JS> can at least have the password stored encrypted on disk.
I don't think that's sufficient but could be wrong.
Ted
^ permalink raw reply
* Re: The imporantance of including http credential caching in 1.7.7
From: Ted Zlatanov @ 2011-09-09 13:33 UTC (permalink / raw)
To: John Szakmeister; +Cc: kusmabite, git, Kyle Neath, Jeff King
In-Reply-To: <CAEBDL5UymsgsQnE9t121omGkR3Zw9BsFqOinTshxEN4WDcOdeA@mail.gmail.com>
On Fri, 9 Sep 2011 06:54:15 -0400 John Szakmeister <john@szakmeister.net> wrote:
JS> On Fri, Sep 9, 2011 at 6:48 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
JS> [snip]
>> Actually, it seems recent Windows versions does have a credential
>> manager, including an API:
>>
>> http://www.yanzzee.com/2009/09/windows-keychain.html
>> http://msdn.microsoft.com/en-us/library/aa374731(v=VS.85).aspx#credentials_management_functions
JS> Yay! It's about time they grew that feature. :-)
That could work. I hope a Windows developer can take a look. That
leaves VMS credential support... Come on guys!
Ted
^ permalink raw reply
* Re: can Git encrypt/decrypt .gpg on push/fetch?
From: Michael J Gruber @ 2011-09-09 13:36 UTC (permalink / raw)
To: Aneesh Bhasin; +Cc: tzz, git
In-Reply-To: <CAGhXAGSw3y=cjAHXtwycDifoBPr13AkYtLHRRXejRKue0vkz7A@mail.gmail.com>
Aneesh Bhasin venit, vidit, dixit 09.09.2011 12:50:
> Hi Ted,
>
>
> 2011/9/9 Ted Zlatanov <tzz@lifelogs.com>
>>
>> I need to store some encrypted files in Git but for some clients with
>> the right GPG keys, decrypt them on checkout (possibly also encrypt them
>> back on commit, but that's not as important).
>>
>> diff doesn't have to work, this is just for convenience. Can Git do
>> this (matching only .gpg files) or do I need my own command to run after
>> the checkout/fetch and before commit? It seems pretty out of Git's
>> scope but perhaps others have done this before.
>>
>
> Have you looked at git hooks (e.g. here : http://progit.org/book/ch7-3.html).
>
> You could do the encryption/decryption in pre-commit and post-checkout
> hooks scripts respectively...
I'd recommend textconv for diffing and clean/smudge for plaintext
checkout. That is, there are two convenient versions:
A) Keep blobs and checkout encrypted
- Use an editor which can encrypt/decrypt on the fly (e.g. vim)
- Use "*.gpg diff=gpg" in your attributes and
[diff "gpg"]
textconv = gpg -d
in your config to have cleartext diffs. Use cachetextconv with caution ;)
B) Keep blobs encrypted, checkout decrypted
- Use Use "*.gpg filter=gpg" in your attributes and
[filter "gpg"]
smudge = gpg -d
clean = gpg -e -r yourgpgkey
in your config.
I use A on a regular basis. B is untested (but patterned after a similar
gzip filter I use). You may or may not have better results with "gpg -ea".
On clients without the keys, you can simply leave out the diff or filter
config resp. set them to "cat".
Michael
^ permalink raw reply
* Re: can Git encrypt/decrypt .gpg on push/fetch?
From: Ted Zlatanov @ 2011-09-09 13:27 UTC (permalink / raw)
To: git
In-Reply-To: <CAGhXAGSw3y=cjAHXtwycDifoBPr13AkYtLHRRXejRKue0vkz7A@mail.gmail.com>
On Fri, 9 Sep 2011 16:20:10 +0530 Aneesh Bhasin <contact.aneesh@gmail.com> wrote:
AB> 2011/9/9 Ted Zlatanov <tzz@lifelogs.com>
>>
>> I need to store some encrypted files in Git but for some clients with
>> the right GPG keys, decrypt them on checkout (possibly also encrypt them
>> back on commit, but that's not as important).
>>
>> diff doesn't have to work, this is just for convenience. Can Git do
>> this (matching only .gpg files) or do I need my own command to run after
>> the checkout/fetch and before commit? It seems pretty out of Git's
>> scope but perhaps others have done this before.
>>
AB> Have you looked at git hooks (e.g. here : http://progit.org/book/ch7-3.html).
AB> You could do the encryption/decryption in pre-commit and post-checkout
AB> hooks scripts respectively...
Yes, thank you. I was wondering if there could be further support so
they are checked out in a binary form on the server side if you don't
have the keys but in text form if you do. So for instance "git log -p"
will DTRT on a client with the keys but not on a client without them.
This could require deep Git changes so I'm wondering if it's even
theoretically possible.
Thanks
Ted
^ permalink raw reply
* Re: git-rebase skips automatically no more needed commits
From: Francis Moreau @ 2011-09-09 13:25 UTC (permalink / raw)
To: Martin von Zweigbergk; +Cc: Junio C Hamano, Michael J Gruber, git
In-Reply-To: <alpine.DEB.2.00.1109090900301.12564@debian>
On Fri, Sep 9, 2011 at 3:06 PM, Martin von Zweigbergk
<martin.von.zweigbergk@gmail.com> wrote:
> On Fri, 9 Sep 2011, Francis Moreau wrote:
>
>> On Fri, Sep 9, 2011 at 4:23 AM, Martin von Zweigbergk
>> <martin.von.zweigbergk@gmail.com> wrote:
>>
>> > you are right that
>> > "git rebase --onto master foo~10 foo" could potentially filter out
>> > patches already in foo..master, without calculating patch-ids for all
>> > commits in master..foo for that matter. I think that would make sense
>>
>> you meant master..foo~10, didn't you ?
>
> Yes. (I actually did mean "all commits in master..foo", but "_any_
> commits in master..foo~10" is more clear and exact.)
>
>> please let me know when you submitting your work, I'm interested to see it.
>
> Will do. It's not really that much work, but I'm not sure when I will
> have time for it. If you or anyone else is interested in doing it, you
> are of course welcome.
I'm a simple mortal user ;)
>
>> > necessary, we could have a flag to disable the filtering e.g. when the
>> > user knows that master is part of a completely separate history from
>> > foo.
>>
>> Can't git figure this out itself ? (I'm not saying the switch is useless)
>
> Yes, but this hypothetical flag would only be to speed things up by
> avoid going to the roots to find out that the histories are disjoint.
You're right but in my (mortal) experience, I'm always rebasing
branches onto another one which is not disjoint, so I'm assuming it's
the (very) common case, but I may be wrong. And what I'm assuming is
correct, I think it's make more sense to do the filtering by default
unless the flag tells to do otherwise.
Thanks
--
Francis
^ permalink raw reply
* Re: git-rebase skips automatically no more needed commits
From: Martin von Zweigbergk @ 2011-09-09 13:06 UTC (permalink / raw)
To: Francis Moreau
Cc: Martin von Zweigbergk, Junio C Hamano, Michael J Gruber, git
In-Reply-To: <CAC9WiBhApjEr-NYm9NkyXaZPNCivoJ65c=xx2bg5Li60kfZMTQ@mail.gmail.com>
On Fri, 9 Sep 2011, Francis Moreau wrote:
> On Fri, Sep 9, 2011 at 4:23 AM, Martin von Zweigbergk
> <martin.von.zweigbergk@gmail.com> wrote:
>
> > you are right that
> > "git rebase --onto master foo~10 foo" could potentially filter out
> > patches already in foo..master, without calculating patch-ids for all
> > commits in master..foo for that matter. I think that would make sense
>
> you meant master..foo~10, didn't you ?
Yes. (I actually did mean "all commits in master..foo", but "_any_
commits in master..foo~10" is more clear and exact.)
> please let me know when you submitting your work, I'm interested to see it.
Will do. It's not really that much work, but I'm not sure when I will
have time for it. If you or anyone else is interested in doing it, you
are of course welcome.
> > necessary, we could have a flag to disable the filtering e.g. when the
> > user knows that master is part of a completely separate history from
> > foo.
>
> Can't git figure this out itself ? (I'm not saying the switch is useless)
Yes, but this hypothetical flag would only be to speed things up by
avoid going to the roots to find out that the histories are disjoint.
Martin
^ permalink raw reply
* Re: merge result
From: Matthieu Moy @ 2011-09-09 13:00 UTC (permalink / raw)
To: Lynn Lin; +Cc: git
In-Reply-To: <CAPgpnMRrSmVrXD__jxv3uNrek8up+scHp+_Kj8+=HX8tfweWzQ@mail.gmail.com>
Lynn Lin <lynn.xin.lin@gmail.com> writes:
> Hi All,
> When I merge branch A back to master branch,if there are same
> commit(developer do double commit) both in master and A branch, there
> will be two same commit in master branch.
They cannot be the "same" commit. They are different commits (i.e.
different sha1 identifier, and probably different trees), even though
they may have the same commit message and represent the same diff.
> 1->2->3-4>5 Master
> |
> 4->6->7 A
A more accurate drawing would be
1->2->3-4>5 Master
|
4'->6->7 A
and after merging, you'd get
1->2->3-4>5-->8 A, master
| /
4'->6->7
with 8 having both 4 and 4' as ancestors. There's nothing wrong with it.
Git cannot remove either 4 or 4' without rewritting history, and "git
merge" does not rewrite history.
If you really really want to avoid this duplication in the history, then
learn about rebase (which is both powerfull and dangerous, you've been
warned).
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
^ permalink raw reply
* Re: merge result
From: Andrew Ardill @ 2011-09-09 12:41 UTC (permalink / raw)
To: Lynn Lin; +Cc: git
In-Reply-To: <CAPgpnMRrSmVrXD__jxv3uNrek8up+scHp+_Kj8+=HX8tfweWzQ@mail.gmail.com>
Hi Lynn,
If you merge two branches together you are merging the state of the
head of those trees, not re-applying commits on top of each other. The
changes introduced in commit 4 will therefore not be applied twice,
but will more-or-less be ignored.
Perhaps you are not trying a merge operation, but something else? Is
this issue hypothetical, or is it something you have experienced??
Regards,
Andrew Ardill
On 9 September 2011 20:54, Lynn Lin <lynn.xin.lin@gmail.com> wrote:
> Hi All,
> When I merge branch A back to master branch,if there are same
> commit(developer do double commit) both in master and A branch, there
> will be two same commit in master branch.For example
>
>
> 1->2->3-4>5 Master
> |
> 4->6->7 A
>
> When I merge A branch into master,the two same 4 commit will present
> in master branch.
>
> Is there any wrong with my operation?
>
> Thanks for your help
> Lynn
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* [PATCH 6/6] Add a REFNAME_ALLOW_UNNORMALIZED flag to check_ref_format()
From: Michael Haggerty @ 2011-09-09 11:46 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, cmn, Michael Haggerty
In-Reply-To: <1315568778-3592-1-git-send-email-mhagger@alum.mit.edu>
Let the callers of check_ref_format() (and normalize_refname()) decide
whether to accept unnormalized refnames via a new
REFNAME_ALLOW_UNNORMALIZED flag. Change callers to set this flag,
which preserves their current behavior. (There are likely places
where this flag can be removed.)
Add analogous options --allow-unnormalized and --no-allow-unnormalized
options to "git check-ref-format" and add tests.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
Documentation/git-check-ref-format.txt | 9 ++++++++-
builtin/check-ref-format.c | 6 +++++-
builtin/checkout.c | 2 +-
builtin/fetch-pack.c | 2 +-
builtin/receive-pack.c | 3 ++-
builtin/replace.c | 2 +-
builtin/show-ref.c | 2 +-
builtin/tag.c | 2 +-
connect.c | 2 +-
environment.c | 2 +-
fast-import.c | 2 +-
notes-merge.c | 4 ++--
refs.c | 11 +++++++----
refs.h | 5 ++++-
remote.c | 8 +++++---
sha1_name.c | 2 +-
t/t1402-check-ref-format.sh | 3 +++
transport.c | 5 ++++-
walker.c | 2 +-
19 files changed, 50 insertions(+), 24 deletions(-)
diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index f2d21c7..fac44ec 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -71,7 +71,7 @@ reference name expressions (see linkgit:gitrevisions[7]):
. at-open-brace `@{` is used as a notation to access a reflog entry.
With the `--print` option, if 'refname' is acceptable, it prints the
-canonicalized name of a hypothetical reference with that name. That is,
+normalized name of a hypothetical reference with that name. That is,
it prints 'refname' with any extra `/` characters removed.
With the `--branch` option, it expands the ``previous branch syntax''
@@ -95,6 +95,13 @@ OPTIONS
of a one full pathname component (e.g., `foo/*/bar` but not
`foo/bar*`).
+--allow-unnormalized::
+--no-allow-unnormalized::
+ Controls whether <refname> is allowed to contain extra `/`
+ characters at the beginning and between name components.
+ These extra slashes are stripped out of the value printed by
+ the `--print` option.) The default is `--allow-unnormalized`.
+
EXAMPLES
--------
diff --git a/builtin/check-ref-format.c b/builtin/check-ref-format.c
index 4c202af..ba0456c 100644
--- a/builtin/check-ref-format.c
+++ b/builtin/check-ref-format.c
@@ -38,7 +38,7 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
{
int i;
int print = 0;
- int flags = 0;
+ int flags = REFNAME_ALLOW_UNNORMALIZED;
if (argc == 2 && !strcmp(argv[1], "-h"))
usage(builtin_check_ref_format_usage);
@@ -55,6 +55,10 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
flags &= ~REFNAME_ALLOW_ONELEVEL;
else if (!strcmp(argv[i], "--refspec-pattern"))
flags |= REFNAME_REFSPEC_PATTERN;
+ else if (!strcmp(argv[i], "--allow-unnormalized"))
+ flags |= REFNAME_ALLOW_UNNORMALIZED;
+ else if (!strcmp(argv[i], "--no-allow-unnormalized"))
+ flags &= ~REFNAME_ALLOW_UNNORMALIZED;
else
usage(builtin_check_ref_format_usage);
}
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2882116..18c3270 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -882,7 +882,7 @@ static int parse_branchname_arg(int argc, const char **argv,
new->name = arg;
setup_branch_path(new);
- if (!check_ref_format(new->path, 0) &&
+ if (!check_ref_format(new->path, REFNAME_ALLOW_UNNORMALIZED) &&
resolve_ref(new->path, branch_rev, 1, NULL))
hashcpy(rev, branch_rev);
else
diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c
index 411aa7d..a4416e4 100644
--- a/builtin/fetch-pack.c
+++ b/builtin/fetch-pack.c
@@ -544,7 +544,7 @@ static void filter_refs(struct ref **refs, int nr_match, char **match)
for (ref = *refs; ref; ref = next) {
next = ref->next;
if (!memcmp(ref->name, "refs/", 5) &&
- check_ref_format(ref->name + 5, 0))
+ check_ref_format(ref->name + 5, REFNAME_ALLOW_UNNORMALIZED))
; /* trash */
else if (args.fetch_all &&
(!args.depth || prefixcmp(ref->name, "refs/tags/") )) {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 4e880ef..e3159c5 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -356,7 +356,8 @@ static const char *update(struct command *cmd)
struct ref_lock *lock;
/* only refs/... are allowed */
- if (prefixcmp(name, "refs/") || check_ref_format(name + 5, 0)) {
+ if (prefixcmp(name, "refs/") ||
+ check_ref_format(name + 5, REFNAME_ALLOW_UNNORMALIZED)) {
rp_error("refusing to create funny ref '%s' remotely", name);
return "funny refname";
}
diff --git a/builtin/replace.c b/builtin/replace.c
index 15f0e5e..4490656 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -94,7 +94,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
"refs/replace/%s",
sha1_to_hex(object)) > sizeof(ref) - 1)
die("replace ref name too long: %.*s...", 50, ref);
- if (check_ref_format(ref, 0))
+ if (check_ref_format(ref, REFNAME_ALLOW_UNNORMALIZED))
die("'%s' is not a valid ref name.", ref);
if (!resolve_ref(ref, prev, 1, NULL))
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index 375a14b..dba92b2 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -145,7 +145,7 @@ static int exclude_existing(const char *match)
if (strncmp(ref, match, matchlen))
continue;
}
- if (check_ref_format(ref, 0)) {
+ if (check_ref_format(ref, REFNAME_ALLOW_UNNORMALIZED)) {
warning("ref '%s' ignored", ref);
continue;
}
diff --git a/builtin/tag.c b/builtin/tag.c
index 7aceaab..50d2018 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -412,7 +412,7 @@ static int strbuf_check_tag_ref(struct strbuf *sb, const char *name)
strbuf_reset(sb);
strbuf_addf(sb, "refs/tags/%s", name);
- return check_ref_format(sb->buf, 0);
+ return check_ref_format(sb->buf, REFNAME_ALLOW_UNNORMALIZED);
}
int cmd_tag(int argc, const char **argv, const char *prefix)
diff --git a/connect.c b/connect.c
index 292a9e2..d50f217 100644
--- a/connect.c
+++ b/connect.c
@@ -22,7 +22,7 @@ static int check_ref(const char *name, int len, unsigned int flags)
len -= 5;
/* REF_NORMAL means that we don't want the magic fake tag refs */
- if ((flags & REF_NORMAL) && check_ref_format(name, 0))
+ if ((flags & REF_NORMAL) && check_ref_format(name, REFNAME_ALLOW_UNNORMALIZED))
return 0;
/* REF_HEADS means that we want regular branch heads */
diff --git a/environment.c b/environment.c
index 8acbb87..2a41c03 100644
--- a/environment.c
+++ b/environment.c
@@ -106,7 +106,7 @@ static char *expand_namespace(const char *raw_namespace)
if (strcmp((*c)->buf, "/") != 0)
strbuf_addf(&buf, "refs/namespaces/%s", (*c)->buf);
strbuf_list_free(components);
- if (check_ref_format(buf.buf, 0))
+ if (check_ref_format(buf.buf, REFNAME_ALLOW_UNNORMALIZED))
die("bad git namespace path \"%s\"", raw_namespace);
strbuf_addch(&buf, '/');
return strbuf_detach(&buf, NULL);
diff --git a/fast-import.c b/fast-import.c
index 4d55ee6..d8e9fe2 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -722,7 +722,7 @@ static struct branch *new_branch(const char *name)
if (b)
die("Invalid attempt to create duplicate branch: %s", name);
- if (check_ref_format(name, REFNAME_ALLOW_ONELEVEL))
+ if (check_ref_format(name, REFNAME_ALLOW_ONELEVEL|REFNAME_ALLOW_UNNORMALIZED))
die("Branch name doesn't conform to GIT standards: %s", name);
b = pool_calloc(1, sizeof(struct branch));
diff --git a/notes-merge.c b/notes-merge.c
index bb8d7c8..c09ce7c 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -570,7 +570,7 @@ int notes_merge(struct notes_merge_options *o,
/* Dereference o->local_ref into local_sha1 */
if (!resolve_ref(o->local_ref, local_sha1, 0, NULL))
die("Failed to resolve local notes ref '%s'", o->local_ref);
- else if (!check_ref_format(o->local_ref, 0) &&
+ else if (!check_ref_format(o->local_ref, REFNAME_ALLOW_UNNORMALIZED) &&
is_null_sha1(local_sha1))
local = NULL; /* local_sha1 == null_sha1 indicates unborn ref */
else if (!(local = lookup_commit_reference(local_sha1)))
@@ -584,7 +584,7 @@ int notes_merge(struct notes_merge_options *o,
* Failed to get remote_sha1. If o->remote_ref looks like an
* unborn ref, perform the merge using an empty notes tree.
*/
- if (!check_ref_format(o->remote_ref, 0)) {
+ if (!check_ref_format(o->remote_ref, REFNAME_ALLOW_UNNORMALIZED)) {
hashclr(remote_sha1);
remote = NULL;
} else {
diff --git a/refs.c b/refs.c
index 6985a3f..c2a7c01 100644
--- a/refs.c
+++ b/refs.c
@@ -879,8 +879,11 @@ int normalize_refname(char *dst, int dstlen, const char *ref, int flags)
ch = *cp;
do {
- while (ch == '/')
- ch = *++cp; /* tolerate leading and repeated slashes */
+ if (flags && REFNAME_ALLOW_UNNORMALIZED) {
+ /* tolerate leading and repeated slashes */
+ while (ch == '/')
+ ch = *++cp;
+ }
/*
* We are at the start of a path component. Record
@@ -1132,7 +1135,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *ref, const unsigned char
struct ref_lock *lock_ref_sha1(const char *ref, const unsigned char *old_sha1)
{
char refpath[PATH_MAX];
- if (check_ref_format(ref, 0))
+ if (check_ref_format(ref, REFNAME_ALLOW_UNNORMALIZED))
return NULL;
strcpy(refpath, mkpath("refs/%s", ref));
return lock_ref_sha1_basic(refpath, old_sha1, 0, NULL);
@@ -1140,7 +1143,7 @@ struct ref_lock *lock_ref_sha1(const char *ref, const unsigned char *old_sha1)
struct ref_lock *lock_any_ref_for_update(const char *ref, const unsigned char *old_sha1, int flags)
{
- if (check_ref_format(ref, REFNAME_ALLOW_ONELEVEL))
+ if (check_ref_format(ref, REFNAME_ALLOW_ONELEVEL|REFNAME_ALLOW_UNNORMALIZED))
return NULL;
return lock_ref_sha1_basic(ref, old_sha1, flags, NULL);
}
diff --git a/refs.h b/refs.h
index 8a15f83..f203726 100644
--- a/refs.h
+++ b/refs.h
@@ -99,6 +99,7 @@ extern int for_each_reflog(each_ref_fn, void *);
#define REFNAME_ALLOW_ONELEVEL 1
#define REFNAME_REFSPEC_PATTERN 2
+#define REFNAME_ALLOW_UNNORMALIZED 4
/*
* Check that ref is a valid refname according to the rules described
@@ -110,7 +111,9 @@ extern int for_each_reflog(each_ref_fn, void *);
* OK and fit into dst. If REFNAME_ALLOW_ONELEVEL is set in flags,
* then accept one-level reference names. If REFNAME_REFSPEC_PATTERN
* is set in flags, then allow a "*" wildcard characters in place of
- * one of the name components.
+ * one of the name components. If REFNAME_ALLOW_UNNORMALIZED is set
+ * in flags, then allow extra "/" characters at the start of the
+ * refname or between name components.
*/
extern int normalize_refname(char *dst, int dstlen, const char *ref, int flags);
diff --git a/remote.c b/remote.c
index 7059885..87da1e9 100644
--- a/remote.c
+++ b/remote.c
@@ -559,7 +559,9 @@ static struct refspec *parse_refspec_internal(int nr_refspec, const char **refsp
rs[i].pattern = is_glob;
rs[i].src = xstrndup(lhs, llen);
- flags = REFNAME_ALLOW_ONELEVEL | (is_glob ? REFNAME_REFSPEC_PATTERN : 0);
+ flags = REFNAME_ALLOW_ONELEVEL |
+ (is_glob ? REFNAME_REFSPEC_PATTERN : 0) |
+ REFNAME_ALLOW_UNNORMALIZED;
if (fetch) {
/*
@@ -1403,7 +1405,7 @@ int get_fetch_map(const struct ref *remote_refs,
for (rmp = &ref_map; *rmp; ) {
if ((*rmp)->peer_ref) {
if (check_ref_format((*rmp)->peer_ref->name + 5,
- REFNAME_ALLOW_ONELEVEL)) {
+ REFNAME_ALLOW_ONELEVEL|REFNAME_ALLOW_UNNORMALIZED)) {
struct ref *ignore = *rmp;
error("* Ignoring funny ref '%s' locally",
(*rmp)->peer_ref->name);
@@ -1595,7 +1597,7 @@ static int one_local_ref(const char *refname, const unsigned char *sha1, int fla
int len;
/* we already know it starts with refs/ to get here */
- if (check_ref_format(refname + 5, 0))
+ if (check_ref_format(refname + 5, REFNAME_ALLOW_UNNORMALIZED))
return 0;
len = strlen(refname) + 1;
diff --git a/sha1_name.c b/sha1_name.c
index 975ec3b..6a5d104 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -974,7 +974,7 @@ int strbuf_check_branch_ref(struct strbuf *sb, const char *name)
if (name[0] == '-')
return -1;
strbuf_splice(sb, 0, 0, "refs/heads/", 11);
- return check_ref_format(sb->buf, 0);
+ return check_ref_format(sb->buf, REFNAME_ALLOW_UNNORMALIZED);
}
/*
diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh
index 6848bfb..a1d4c5b 100755
--- a/t/t1402-check-ref-format.sh
+++ b/t/t1402-check-ref-format.sh
@@ -27,9 +27,12 @@ invalid_ref() {
valid_ref 'foo/bar/baz'
valid_ref 'refs///heads/foo'
+invalid_ref 'refs///heads/foo' --no-allow-unnormalized
invalid_ref 'heads/foo/'
valid_ref '/heads/foo'
+invalid_ref '/heads/foo' --no-allow-unnormalized
valid_ref '///heads/foo'
+invalid_ref '///heads/foo' --no-allow-unnormalized
invalid_ref './foo'
invalid_ref './foo/bar'
invalid_ref 'foo/./bar'
diff --git a/transport.c b/transport.c
index 225d9b8..65209af 100644
--- a/transport.c
+++ b/transport.c
@@ -754,7 +754,10 @@ void transport_verify_remote_names(int nr_heads, const char **heads)
continue;
remote = remote ? (remote + 1) : local;
- if (check_ref_format(remote, REFNAME_ALLOW_ONELEVEL|REFNAME_REFSPEC_PATTERN))
+ if (check_ref_format(remote,
+ REFNAME_ALLOW_ONELEVEL|
+ REFNAME_REFSPEC_PATTERN|
+ REFNAME_ALLOW_UNNORMALIZED))
die("remote part of refspec is not a valid name in %s",
heads[i]);
}
diff --git a/walker.c b/walker.c
index e5d8eb2..f0dbe58 100644
--- a/walker.c
+++ b/walker.c
@@ -190,7 +190,7 @@ static int interpret_target(struct walker *walker, char *target, unsigned char *
{
if (!get_sha1_hex(target, sha1))
return 0;
- if (!check_ref_format(target, 0)) {
+ if (!check_ref_format(target, REFNAME_ALLOW_UNNORMALIZED)) {
struct ref *ref = alloc_ref(target);
if (!walker->fetch_ref(walker, ref)) {
hashcpy(sha1, ref->old_sha1);
--
1.7.6.8.gd2879
^ permalink raw reply related
* [PATCH 5/6] Do not allow ".lock" at the end of any refname component
From: Michael Haggerty @ 2011-09-09 11:46 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, cmn, Michael Haggerty
In-Reply-To: <1315568778-3592-1-git-send-email-mhagger@alum.mit.edu>
Allowing any refname component to end with ".lock" is looking for
trouble; for example,
$ git br foo.lock/bar
$ git br foo
fatal: Unable to create '[...]/.git/refs/heads/foo.lock': File exists.
Therefore, do not allow any refname component to end with ".lock".
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
Documentation/git-check-ref-format.txt | 4 +---
refs.c | 6 +++---
t/t1402-check-ref-format.sh | 4 ++++
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/Documentation/git-check-ref-format.txt b/Documentation/git-check-ref-format.txt
index 3ab22b9..f2d21c7 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -28,7 +28,7 @@ git imposes the following rules on how references are named:
. They can include slash `/` for hierarchical (directory)
grouping, but no slash-separated component can begin with a
- dot `.`.
+ dot `.` or end with the sequence `.lock`.
. They must contain at least one `/`. This enforces the presence of a
category like `heads/`, `tags/` etc. but the actual names are not
@@ -47,8 +47,6 @@ git imposes the following rules on how references are named:
. They cannot end with a slash `/` nor a dot `.`.
-. They cannot end with the sequence `.lock`.
-
. They cannot contain a sequence `@{`.
. They cannot contain a `\`.
diff --git a/refs.c b/refs.c
index 372350e..6985a3f 100644
--- a/refs.c
+++ b/refs.c
@@ -933,14 +933,14 @@ int normalize_refname(char *dst, int dstlen, const char *ref, int flags)
if (component[0] == '.')
/* Components must not start with '.'. */
return -1;
+ if (component_len >= 5 && !memcmp(&component[component_len - 5], ".lock", 5))
+ /* Components must not end with ".lock". */
+ return -1;
} while (ch != 0);
if (last == '.')
/* Refname must not end with '.'. */
return -1;
- if (component_len >= 5 && !memcmp(&component[component_len - 5], ".lock", 5))
- /* Refname must not end with ".lock". */
- return -1;
if (!(flags & REFNAME_ALLOW_ONELEVEL) && component_count < 2)
/* Refname must have at least two components. */
return -1;
diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh
index 20a7782..6848bfb 100755
--- a/t/t1402-check-ref-format.sh
+++ b/t/t1402-check-ref-format.sh
@@ -40,6 +40,8 @@ invalid_ref 'heads/foo?bar'
valid_ref 'foo./bar'
invalid_ref 'heads/foo.lock'
invalid_ref 'heads///foo.lock'
+invalid_ref 'foo.lock/bar'
+invalid_ref 'foo.lock///bar'
valid_ref 'heads/foo@bar'
invalid_ref 'heads/v@{ation'
invalid_ref 'heads/foo\bar'
@@ -155,5 +157,7 @@ invalid_ref_normalized 'heads/./foo'
invalid_ref_normalized 'heads\foo'
invalid_ref_normalized 'heads/foo.lock'
invalid_ref_normalized 'heads///foo.lock'
+invalid_ref_normalized 'foo.lock/bar'
+invalid_ref_normalized 'foo.lock///bar'
test_done
--
1.7.6.8.gd2879
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox