Unresolved issues #3

Unresolved issues #3

[Junio C Hamano]

As everybody has probably noticed already, I am terrible at
maintaining "the current issues" list.  The most recent issue of
this series was done when?  Back on May 4th this year.

Shame on me.


Here is a list of topics in the recent git traffic that I feel
inadequately addressed, in no particular order.  I've commented
on some of them to give people a feel for what my priorities
are.  Somebody might want to rehash the ones low on my priority
list to conclusion with a concrete proposal if they cared about
them enough.


* Mozilla import team seems to be making an interesting set of
  discoveries around the area of scalability, although I haven't
  personally looked at any of the tools they are using so far.

  Continued discussion is encouraged, and I am looking forward
  to see the fruits of this effort.  Very much appreciated.

* Eric W Biederman outlined an alternative workflow to track
  history of the tip of a branch that does not use ref-log
  facility.

  Message-ID: <m1mzakpam8.fsf@ebiederm.dsl.xmission.com>

  I think this sort-of makes sense when everybody involved does
  not misuse the fake parallel branch for purposes other than
  tracking the tip of the other, "true", branch, but I fear it
  would confuse people quite a bit.  Eric talks about making
  rewinding and rebasing in distributed manner possible, but I
  do not know what merging between the fake branches would mean,
  for example.

* Matthias Lederhofer's helper for stand alone gitweb test looked
  very promising:

  Message-ID: <20060806165151.GB9548@moooo.ath.cx>

  With the recent flurry of gitweb changes, we really need a
  test suite to catch obvious breakages in the t/ hierarchy, and
  something like this patch would help.

  I'd like to apply this patch, if somebody wants to add gitweb
  tests.

* Jeff King sent a patch to color git-status output

  Message-ID: <20060805031418.GA11102@coredump.intra.peff.net>

  But later he started working on rewriting one core function
  from git-status in C, which I think makes a lot of sense.
  That would make this patch unnecessary.  A thread related to
  this effort is this:
  
  Message-ID: <20060810082455.GA30739@coredump.intra.peff.net>

  Hoping to see the C implementation of run_status but I am in
  no rush myself.

  I vaguely recall there was a companion patch to add vim
  colorizer for the current git-status output, meant for
  contrib/vim, but I lost it.  If somebody cares deeply enough
  please send it over.

* Alex Riesen still has problems with Git.pm/Git.xs on Windows
  with ActiveState Perl.

  Message-ID: <81b0412b0608020702q2fd4ec83ga43714c15538f7ad@mail.gmail.com>

  There was a workaround patch by Alex to disable certain
  commands that happen to use Git.pm:

  Message-ID: <81b0412b0608040640s44c0d84et94871bce0271b047@mail.gmail.com>

  But I sense this would make us think twice before using Git.pm
  in new programs (i.e. "Can Alex live without this new nice
  program?  If not maybe we should write it without Git.pm"),
  which feels backwards and I would like to avoid it if we can.

  I am not sure what the resolution of this should be.

* Michael S. Tsirkin discovered that we have trouble dealing
  with the same remote ref tracked by multiple tracking
  branches.

  Message-ID: <20060807125116.GA28658@mellanox.co.il>

  Since I do not see a valid use case that _must_ use more than
  one local tracking branch to track one remote ref, I think it
  is Ok to declare it an error to do so.  However, at least, we
  would need a better error message even then.

* Jeff Garzik reports that the summary page of gitweb does not
  look at anything other than "master" which is not appropriate
  for his tree.

  Message-ID: <44D874F0.6000907@garzik.org>

  I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
  Aneesh) about this.

* gitk bottom-left pane layout is reported to be broken for some
  people.

  Message-ID: <20060802192922.GA30539@prophet.net-ronin.org>

  "carbonated beverage" tried some random hacks to work it
  around but I do not think we have gotten anywhere, and the
  discussion fizzled out.

  I vaguely recall paulus saying it was a Tk bug but do not
  recall hearing nor understanding any details myself.

* "A Large Angry SCM" wrote a nice summary, "Git files data
  formats documentation".

  Message-ID: <44D51D47.9090700@gmail.com>

  With one final update by Nico yesterday, I think it is ready
  for inclusion.

  Does somebody care to make a patch out of it to add it to
  Documentation/technical/, maybe removing pack-format.txt there
  after making sure all it talks about is covered by the new
  documentation?

  I do not have enough "virginity" to spot omissions in the
  description anymore, so comments from somebody new to the
  system are very much appreciated.

* Martin Langhoff proposed git-xxdiff as a helper after a failed
  merge.

  Message-ID: <11546492331601-git-send-email-martin@catalyst.net.nz>

  I like the general idea of this a lot, but am having a bit of
  trouble envisioning how we can integrate this while making
  sure mergers other than xxdiff can be added easily without
  disrupting the end user experience.

* Shawn Pearce noticed that fsck-objects do not fully check some
  fields in commits:

  Message-ID: <20060814062830.GF18667@spearce.org>

* Although "annotate" had some big fixes just before 1.4.2, Ryan
  seems to feel "blame" has already won.

  Message-ID: <20060807194539.GD15477@h4x0r5.com>

  Is there still an area "annotate" shines more than "blame -c"?

* "Michael barra_cuda" noticed that an option "--no" is
  ambiguous to git-revert:

  Message-ID: <200608031742.23170.barra_cuda@katamail.com>

  I need to fix this.

* Cherry-pick should not require -r to suppress "cherry-picked
  from" message.

  Message-ID: <Pine.LNX.4.64.0607120834200.5623@g5.osdl.org>

  This was requested by Linus, which I haven't done anything
  about yet.  Maybe making -r a no-op, defaulting not to add the
  message, and introducing --record-original flag to add it
  would be a way to go?  My fingers are trained to automatically
  type -r when doing cherry-pick which is a very good indication
  that the flag was a mistake.

* Martin Waitz has a patch to accept "end-of-cover-letter"
  marker in patch messages.

  Message-ID: <20060723214524.GC20068@admingilde.org>

  I did not take the patch, primarily because I do not want to
  encourage "cover letter then log then patch" format, and also
  the proposed marker "+++" looked somewhat ugly (see Message-ID:
  <20060801193408.GF16364@admingilde.org> for an example).

  If somebody wants to rehash this I am still open to reconsider
  it, but I sense a veto from Linus coming...

Re: Unresolved issues #3

[A Large Angry SCM]

Junio C Hamano wrote:
...
> * "A Large Angry SCM" wrote a nice summary, "Git files data
>   formats documentation".
> 
>   Message-ID: <44D51D47.9090700@gmail.com>
> 
>   With one final update by Nico yesterday, I think it is ready
>   for inclusion.
> 
>   Does somebody care to make a patch out of it to add it to
>   Documentation/technical/, maybe removing pack-format.txt there
>   after making sure all it talks about is covered by the new
>   documentation?
> 
>   I do not have enough "virginity" to spot omissions in the
>   description anymore, so comments from somebody new to the
>   system are very much appreciated.
> 

Two things:

1) I disagree with Nico's assessment that, other than his, there can not
exist any type 2 packs that have bit 6 set to mean copy from result.

2) The document, if included into the core Git documentation, would be
viewed as more authoritative than it actually is. It's really just a set
of (structured) notes on various aspects of the files Git reads and
writes, pulled from various sources. Little or no attempt was made to
separate "implementation artifact" from intended structure.

Re: Unresolved issues #3

[Jeff King]

On Thu, Aug 17, 2006 at 09:09:03PM -0700, Junio C Hamano wrote:

> * Jeff King sent a patch to color git-status output
[...]
>   Hoping to see the C implementation of run_status but I am in
>   no rush myself.

I am working on this but got side-tracked by "real" work. The current
state is that I have a simplistic working C run_status, but I'm still
hoping to hack the diff code to simultaneously do the tree<->index and
index<->files diffs. I will try to send out something next week.

>   I vaguely recall there was a companion patch to add vim
>   colorizer for the current git-status output, meant for
>   contrib/vim, but I lost it.  If somebody cares deeply enough
>   please send it over.

It's in <20060805032135.GA11244@coredump.intra.peff.net>. However, if I
can get the multi-diff support working, then the status format will
change. I will wait until that is resolved before submitting a patch to
put vim highlighting into contrib/.

-Peff

Re: Unresolved issues #3

[Catalin Marinas]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

Junio C Hamano <junkio@cox.net> wrote:
> * Martin Langhoff proposed git-xxdiff as a helper after a failed
>   merge.
>
>   Message-ID: <11546492331601-git-send-email-martin@catalyst.net.nz>
>
>   I like the general idea of this a lot, but am having a bit of
>   trouble envisioning how we can integrate this while making
>   sure mergers other than xxdiff can be added easily without
>   disrupting the end user experience.

In StGIT I can configure the merge command and I currently use the
attached script (I need to add it to the StGIT repository). It tries
diff3 first and, if that fails, invokes emacs' merge (can use xxdiff
as well). It also checks whether the file was modified in case I want
to exit and solve the conflict later (maybe after getting conflict
information for the other files).

-- 
Catalin


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: merge-interactive.py --]
[-- Type: text/x-python, Size: 1950 bytes --]

#!/usr/bin/env python
"""Run diff3 and an interactive tool if this fails.
"""

__copyright__ = """
Copyright (C) 2006, Catalin Marinas <catalin.marinas@gmail.com>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License version 2 as
published by the Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""

import sys, os

if len(sys.argv) != 5:
    print >> sys.stderr, 'Usage: %s ancestor branch1 branch2 output' \
          % sys.argv[0]
    sys.exit(1)

ancestor = sys.argv[1]
branch1  = sys.argv[2]
branch2  = sys.argv[3]
output   = sys.argv[4]

# default merger
if os.system('diff3 -L current -L ancestor -L patched -m -E '
             '"%s" "%s" "%s" > "%s"'
             % (branch1, ancestor, branch2, output)):
    # interactive merge
    if os.path.exists(output):
        mtime = os.path.getmtime(output)
    else:
        mtime = 0

    ret = os.system('emacs --eval \'(ediff-merge-files-with-ancestor '
                    '"%s" "%s" "%s" nil "%s")\''
                    % (branch1, branch2, ancestor, output))

    #ret = os.system(
    #    'xxdiff --title1 current --title2 ancestor --title3 patched '
    #    '--show-merged-pane -m -E -O -X -M "%s" "%s" "%s" "%s"'
    #    % (output, branch1, ancestor, branch2))

    # error in the interactive merger, just exit
    if ret:
        sys.exit(2)

    # check for file modification
    if not os.path.exists(output) or mtime == os.path.getmtime(output):
        sys.exit(3)

# everything's fine
sys.exit(0)

Re: Unresolved issues #3

[Jakub Narebski]

Junio C Hamano wrote:

> * Jeff Garzik reports that the summary page of gitweb does not
>   look at anything other than "master" which is not appropriate
>   for his tree.
> 
>   Message-ID: <44D874F0.6000907@garzik.org>
> 
>   I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
>   Aneesh) about this.

For performance reasons this should probably wait for functioning
git-show-refs command. BTW. summary page doesn't show "Last Updated" based
on "master", but based on "HEAD" (current branch)... which usually is
"master" I guess in typical usage.

Another thing that I think is/might be important to gitweb, is to rewrite
commitdiff and blobdiff to not use external diff, but builtin
git-diff-tree... this probably would lead to changes in the output.
I'm thinking about "-B" without "-C" nor "-M" for plain output (is this
no-op for this combination?), and "-C", "-M" for HTML output.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Unresolved issues #3

[Junio C Hamano]

Catalin Marinas <catalin.marinas@arm.com> writes:

> Junio C Hamano <junkio@cox.net> wrote:
>> * Martin Langhoff proposed git-xxdiff as a helper after a failed
>>   merge.
>>
>>   Message-ID: <11546492331601-git-send-email-martin@catalyst.net.nz>
>>
>>   I like the general idea of this a lot, but am having a bit of
>>   trouble envisioning how we can integrate this while making
>>   sure mergers other than xxdiff can be added easily without
>>   disrupting the end user experience.
>
> In StGIT I can configure the merge command and I currently use the
> attached script (I need to add it to the StGIT repository). It tries
> diff3 first and, if that fails, invokes emacs' merge (can use xxdiff
> as well). It also checks whether the file was modified in case I want
> to exit and solve the conflict later (maybe after getting conflict
> information for the other files).

Thanks.  That is interesting.  However, the workflow Martin
wants to use this is a bit different from this.  He wants
regular diff3/merge to leave conflicted result and have "xxdiff
-U" to sort out the resulting mess (I do that manually myself
but without any wrapper).

Having said that...

> # default merger
> if os.system('diff3 -L current -L ancestor -L patched -m -E '
>              '"%s" "%s" "%s" > "%s"'
>              % (branch1, ancestor, branch2, output)):
>     # interactive merge
>     if os.path.exists(output):
>         mtime = os.path.getmtime(output)
>     else:
>         mtime = 0

This reminds me that I should make the git barebone
porcelain-ish to use diff3 directly without using merge
wrapper.

I wonder why you do not error out when "if os.path.exists(output)"
does not hold true; you are redirecting into it, so the possibilities
are:

 - the filesystem is full;
 - your quota ran out;
 - you do not have write permission to the directory "output"
   wants to be in;
 - the user gave 'fooled you"; rm -fr . "bye' as the value of
   sys.argv[4];

etc.  No matter what the reason, the fall-back interactive merge
would not be able to write into output either anyway, no?

Also I wonder what would happen upon a user error where an
existing file that is unwritable by the user is specified as
output.

Re: Unresolved issues #3

[Catalin Marinas]

On 18/08/06, Junio C Hamano <junkio@cox.net> wrote:
> Catalin Marinas <catalin.marinas@arm.com> writes:
> > In StGIT I can configure the merge command and I currently use the
> > attached script (I need to add it to the StGIT repository). It tries
> > diff3 first and, if that fails, invokes emacs' merge (can use xxdiff
> > as well). It also checks whether the file was modified in case I want
> > to exit and solve the conflict later (maybe after getting conflict
> > information for the other files).
>
> Thanks.  That is interesting.  However, the workflow Martin
> wants to use this is a bit different from this.  He wants
> regular diff3/merge to leave conflicted result and have "xxdiff
> -U" to sort out the resulting mess (I do that manually myself
> but without any wrapper).

The problem with this approach is that you lose the ancestor
information. You want "diff3 -E" command to not report conflicts if
the changes are the same but the output no longer has the ancestor
information. The -A option would keep the ancestor (and use "xxdiff
--unmerge3" instead) but it always reports a conflict even if the
changes are the same. Are there any diff3 options I missed?

That's why I prefer to explicitely pass the 3 files to either emacs or xxdiff.

> > # default merger
> > if os.system('diff3 -L current -L ancestor -L patched -m -E '
> >              '"%s" "%s" "%s" > "%s"'
> >              % (branch1, ancestor, branch2, output)):
> >     # interactive merge
> >     if os.path.exists(output):
> >         mtime = os.path.getmtime(output)
> >     else:
> >         mtime = 0
>
> I wonder why you do not error out when "if os.path.exists(output)"
> does not hold true; you are redirecting into it, so the possibilities
> are:

You are right. I was thinking about the file removed in the current
branch but modified in the other. However, this would never invoke
diff3.

Thanks.

-- 
Catalin

Re: Unresolved issues #3

[Nicolas Pitre]

On Thu, 17 Aug 2006, A Large Angry SCM wrote:

> Junio C Hamano wrote:
> ...
> > * "A Large Angry SCM" wrote a nice summary, "Git files data
> >   formats documentation".
> > 
> >   Message-ID: <44D51D47.9090700@gmail.com>
> > 
> >   With one final update by Nico yesterday, I think it is ready
> >   for inclusion.
> > 
> >   Does somebody care to make a patch out of it to add it to
> >   Documentation/technical/, maybe removing pack-format.txt there
> >   after making sure all it talks about is covered by the new
> >   documentation?
> > 
> >   I do not have enough "virginity" to spot omissions in the
> >   description anymore, so comments from somebody new to the
> >   system are very much appreciated.
> > 
> 
> Two things:
> 
> 1) I disagree with Nico's assessment that, other than his, there can not
> exist any type 2 packs that have bit 6 set to mean copy from result.

Care to explain why?

Since this code is mine I can tell you that no official GIT version ever 
produced such a pack.  The code to make use of that bit was quite 
involving and the end result wasn't great at all so I never published 
said code.  This is also why current GIT accepts both pack version 2 and 
3 without any distinction using the same code in patch-delta.c on the 
basis that no version 2 packs ever used that bit.


Nicolas

Re: Unresolved issues #3

[A Large Angry SCM]

Nicolas Pitre wrote:
> On Thu, 17 Aug 2006, A Large Angry SCM wrote:
...
>>
>> 1) I disagree with Nico's assessment that, other than his, there can not
>> exist any type 2 packs that have bit 6 set to mean copy from result.
> 
> Care to explain why?
> 
> Since this code is mine I can tell you that no official GIT version ever 
                                              ^^^^^^^^^^^^^^^^^^^^^^^
> produced such a pack.  The code to make use of that bit was quite 
> involving and the end result wasn't great at all so I never published 
> said code.  This is also why current GIT accepts both pack version 2 and 
> 3 without any distinction using the same code in patch-delta.c on the 
> basis that no version 2 packs ever used that bit.

That doesn't prove the non-existence of other code to do it.

Re: Unresolved issues #3

[Nicolas Pitre]

On Fri, 18 Aug 2006, A Large Angry SCM wrote:

> Nicolas Pitre wrote:
> > On Thu, 17 Aug 2006, A Large Angry SCM wrote:
> ...
> >>
> >> 1) I disagree with Nico's assessment that, other than his, there can not
> >> exist any type 2 packs that have bit 6 set to mean copy from result.
> > 
> > Care to explain why?
> > 
> > Since this code is mine I can tell you that no official GIT version ever 
>                                               ^^^^^^^^^^^^^^^^^^^^^^^
> > produced such a pack.  The code to make use of that bit was quite 
> > involving and the end result wasn't great at all so I never published 
> > said code.  This is also why current GIT accepts both pack version 2 and 
> > 3 without any distinction using the same code in patch-delta.c on the 
> > basis that no version 2 packs ever used that bit.
> 
> That doesn't prove the non-existence of other code to do it.

So?  If the official and primary code for GIT doesn't support it, what 
is the point?  I'm telling you that if such packs exist they will simply 
barf with all official GIT releases later than v1.1.6 making your 
argument pointless.

I don't mind you documenting that historic intent for a bit that was 
never officially used, but at least let's document it right.


Nicolas

Re: Unresolved issues #3

[Aneesh Kumar K.V]

Junio C Hamano wrote:

> * Jeff Garzik reports that the summary page of gitweb does not
>   look at anything other than "master" which is not appropriate
>   for his tree.
> 
>   Message-ID: <44D874F0.6000907@garzik.org>
> 
>   I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
>   Aneesh) about this.
> 


I just tried editing HEAD. For the project 

http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary

$more HEAD
ref: refs/heads/from-linus
$

Is this solution fine ?. Or do we want to add a git-rep-config variable to indicate which branch to show.

-aneesh 

Re: Unresolved issues #3

[Jakub Narebski]

Aneesh Kumar K.V wrote:

> Junio C Hamano wrote:
> 
>> * Jeff Garzik reports that the summary page of gitweb does not
>>   look at anything other than "master" which is not appropriate
>>   for his tree.
>> 
>>   Message-ID: <44D874F0.6000907@garzik.org>
>> 
>>   I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
>>   Aneesh) about this.
> 
> I just tried editing HEAD. For the project 
> 
> http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary
> 
> $more HEAD
> ref: refs/heads/from-linus
> $
> 
> Is this solution fine ?. Or do we want to add a git-rep-config 
> variable to indicate which branch to show. 

Err, of course gitweb shows "Last Change" for HEAD, which usually is master.
The solution would be to show "Last Change" date to be the date of last
change of all/any branch.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Unresolved issues #3

[Aneesh Kumar K.V]

Jakub Narebski wrote:
> Aneesh Kumar K.V wrote:
> 
>> Junio C Hamano wrote:
>>
>>> * Jeff Garzik reports that the summary page of gitweb does not
>>>   look at anything other than "master" which is not appropriate
>>>   for his tree.
>>>
>>>   Message-ID: <44D874F0.6000907@garzik.org>
>>>
>>>   I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
>>>   Aneesh) about this.
>> I just tried editing HEAD. For the project 
>>
>> http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary
>>
>> $more HEAD
>> ref: refs/heads/from-linus
>> $
>>
>> Is this solution fine ?. Or do we want to add a git-rep-config 
>> variable to indicate which branch to show. 
> 
> Err, of course gitweb shows "Last Change" for HEAD, which usually is master.
> The solution would be to show "Last Change" date to be the date of last
> change of all/any branch.
> 

I didn't quiet understand that. AFAIU what jeff wanted is to make gitweb show some branch other than master
by default in the summary page.  I guess editing HEAD enables that.

-aneesh

Re: Unresolved issues #3

[Jakub Narebski]

Aneesh Kumar K.V wrote:

> Jakub Narebski wrote:
>> Aneesh Kumar K.V wrote:
>> 
>>> Junio C Hamano wrote:
>>>
>>>> * Jeff Garzik reports that the summary page of gitweb does not
>>>>   look at anything other than "master" which is not appropriate
>>>>   for his tree.
>>>>
>>>>   Message-ID: <44D874F0.6000907@garzik.org>
>>>>
>>>>   I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
>>>>   Aneesh) about this.
>>> I just tried editing HEAD. For the project 
>>>
>>> http://git.openssi.org/~kvaneesh/gitweb.cgi?p=ci-to-linus.git;a=summary
>>>
>>> $more HEAD
>>> ref: refs/heads/from-linus
>>> $
>>>
>>> Is this solution fine ?. Or do we want to add a git-rep-config 
>>> variable to indicate which branch to show. 
>> 
>> Err, of course gitweb shows "Last Change" for HEAD, which usually is
>> master.  The solution would be to show "Last Change" date to be the date
>> of last change of all/any branch.
> 
> I didn't quiet understand that. AFAIU what jeff wanted is to make 
> gitweb show some branch other than master by default in the summary page.
> I guess editing HEAD enables that. 

Editing head, well, having other branch checked out would be enough (checked
out before push would be enough I guess for gitweb repository being
"published repository", other that the copy you are working on).

AFAIR the problem was that development was not made at "master", so "Last
Changes" were last changes in master, no last changes as a whole. Last
changes date as a whole would be last change date of latest edited branch.
And for this I'd rather have functioning git-show-refs, for performance (at
least with larger number of heads...). 

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Unresolved issues #3

[Jon Loeliger]

On Thu, 2006-08-17 at 23:09, Junio C Hamano wrote:
> As everybody has probably noticed already, I am terrible at
> maintaining "the current issues" list.  The most recent issue of
> this series was done when?  Back on May 4th this year.
> 
> Shame on me.

I continue to think you are doing a fabulous job nonetheless!

> 
> Here is a list of topics in the recent git traffic that I feel
> inadequately addressed, in no particular order.  I've commented
> on some of them to give people a feel for what my priorities
> are.  Somebody might want to rehash the ones low on my priority
> list to conclusion with a concrete proposal if they cared about
> them enough.

I have another:

git-daemon virtualization so that consistent HTTP and
native git protocols can appear to use consistent URLs
even in the face of HTTP configurations aliasing them
to somewhere else on the filesystem and for multiple
virtually hosted domain names.


Thanks,
jdl

Re: Unresolved issues #3

[A Large Angry SCM]

Nicolas Pitre wrote:
> On Fri, 18 Aug 2006, A Large Angry SCM wrote:
> 
>> Nicolas Pitre wrote:
>>> On Thu, 17 Aug 2006, A Large Angry SCM wrote:
>> ...
>>>> 1) I disagree with Nico's assessment that, other than his, there can not
>>>> exist any type 2 packs that have bit 6 set to mean copy from result.
>>> Care to explain why?
>>>
>>> Since this code is mine I can tell you that no official GIT version ever 
>>                                               ^^^^^^^^^^^^^^^^^^^^^^^
>>> produced such a pack.  The code to make use of that bit was quite 
>>> involving and the end result wasn't great at all so I never published 
>>> said code.  This is also why current GIT accepts both pack version 2 and 
>>> 3 without any distinction using the same code in patch-delta.c on the 
>>> basis that no version 2 packs ever used that bit.
>> That doesn't prove the non-existence of other code to do it.
> 
> So?  If the official and primary code for GIT doesn't support it, what 
> is the point?  I'm telling you that if such packs exist they will simply 
> barf with all official GIT releases later than v1.1.6 making your 
> argument pointless.
> 
> I don't mind you documenting that historic intent for a bit that was 
> never officially used, but at least let's document it right.

Historic fact. Between Thu May 19 08:56:22 2005 and Thu Feb  9 21:06:38
2006 bit 6 of the first byte of a delta hunk was interpreted to mean
that the source of the copy was the result buffer. From Thu May 19
08:56:22 2005 on, the code to decode delta hunks in type 2 packs was
available to everyone and anyone interested could make a pack encoder
that would create packs that the core Git code would correctly read. The
commit of Thu Feb  9 21:06:38 2006, d60fc, actually introduced a bug
that would treat valid type 2 packs as invalid.

Since there was not any documentation that declared the bit as reserved,
the code was the documentation and it specified that bit 6 of the first
byte of a delta hunk was to be interpreted as meaning the the copy
source in the result buffer. The code did not and does not document intent.

Re: Unresolved issues #3

[Junio C Hamano]

Jon Loeliger <jdl@freescale.com> writes:

> I have another:
>
> git-daemon virtualization so that consistent HTTP and
> native git protocols can appear to use consistent URLs
> even in the face of HTTP configurations aliasing them
> to somewhere else on the filesystem and for multiple
> virtually hosted domain names.

Good point.

Here are a handful others I have about git-daemon:

 * Possibly add --strict-symlink option we discussed a long time
   ago that prevents symbolic links inside the named whitelisted
   directory hierarchies to step outside.

   Nobody actually jumped up-and-down asking for it since we
   initially discussed this, so we may not need it after all.

 * The whitelist directories need to be specified with full path
   even when --base-path is used, which somewhat felt wrong.  I
   do not have strong feeling about this anymore, though.

 * Extend $GIT_DIR/git-daemon-export-ok mechanism and have it
   read from $GIT_DIR/config.

 * Also extend the same in a way similar to the discussion we
   had between Aneesh, me and Jakub to control 'blame' and
   'snapshot' in gitweb.  I.e. allow the site-wide configuration
   to specify which services are enabled by default and which
   services can be overridden per repository, and allow per
   repository configuration to specify what's enabled and what's
   not.

 * Add git-upload-tar as a potential program that can be
   executed from git-daemon.

If we are to do the last three items, my thinking is to do
something like this:

  * The config file can have:

       daemon.upload-pack = yes | no
       daemon.upload-tar = yes | no

      to enable or disable the services individually.

  * $GIT_DIR/git-daemon-export-ok is kept as a backward
    compatibility measure; it means upload-pack is enabled.

If we were to allow git-upload-tar from git-daemon:

 * Fix protocol exchange between git-upload-tar and git-tar-tree
   so that the downloader can name the <tree-ish> symbolically
   like this:

	git tar-tree --remote=git://git.kernel.org/pub/scm/git/git next

   Currently the downloader has to get the value of "next" at
   the remote server out-of-band and give an explicit object
   name.

Re: Unresolved issues #3

[Nicolas Pitre]

On Fri, 18 Aug 2006, A Large Angry SCM wrote:

> Nicolas Pitre wrote:
> > On Fri, 18 Aug 2006, A Large Angry SCM wrote:
> >> That doesn't prove the non-existence of other code to do it.
> > 
> > So?  If the official and primary code for GIT doesn't support it, what 
> > is the point?  I'm telling you that if such packs exist they will simply 
> > barf with all official GIT releases later than v1.1.6 making your 
> > argument pointless.
> > 
> > I don't mind you documenting that historic intent for a bit that was 
> > never officially used, but at least let's document it right.
> 
> Historic fact. Between Thu May 19 08:56:22 2005 and Thu Feb  9 21:06:38
> 2006 bit 6 of the first byte of a delta hunk was interpreted to mean
> that the source of the copy was the result buffer. From Thu May 19
> 08:56:22 2005 on, the code to decode delta hunks in type 2 packs was
> available to everyone and anyone interested could make a pack encoder
> that would create packs that the core Git code would correctly read. The
> commit of Thu Feb  9 21:06:38 2006, d60fc, actually introduced a bug
> that would treat valid type 2 packs as invalid.

The "actually introduced a bug" sentence is your own interpretation not 
a _fact_.

And I simply disagree with that interpretation of yours.

I don't think this is worth arguing any further.


Nicolas

Re: Unresolved issues #3

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> On Fri, 18 Aug 2006, A Large Angry SCM wrote:
>
>> Historic fact. Between Thu May 19 08:56:22 2005 and Thu Feb  9 21:06:38
>> 2006 bit 6 of the first byte of a delta hunk was interpreted to mean
>> that the source of the copy was the result buffer. From Thu May 19
>> 08:56:22 2005 on, the code to decode delta hunks in type 2 packs was
>> available to everyone and anyone interested could make a pack encoder
>> that would create packs that the core Git code would correctly read. The
>> commit of Thu Feb  9 21:06:38 2006, d60fc, actually introduced a bug
>> that would treat valid type 2 packs as invalid.

It is more like the said commit made the pack format extensible
by declaring the bit reserved for the future use, by declaring
retroactively that a type 2 pack that used that bit invalid.
And it was deemed a reasonable and safe decision because no
official git ever produced a type 2 pack that used that bit,

Yes, that was a backward incompatible change, strictly speaking,
and probably I should have made an announcement that looked
similar to this by Linus:

        From: Linus Torvalds <torvalds@osdl.org>
        Subject: CAREFUL! No more delta object support!
        Date: Mon, 27 Jun 2005 18:14:40 -0700 (PDT)
        Message-ID: <Pine.LNX.4.58.0506271755140.19755@ppc970.osdl.org>
        To: Git Mailing List <git@vger.kernel.org>

So you could argue I was incompetent not to make a big fuss
about this backward incompatibility back then, if you like.

I did not think it was worth it back then, and I do not think it
is worth it now, either.  But if it makes you feel better, I
could retroactively make such an announcement about the
unofficial bit 6.

The announcement would have read like this:

    The current git code does not support type #2 packs that
    uses delta with bit 6 to mean "copy inside destination
    buffer".  Although the code that interpreted delta data
    supported bit 6 that way for a brief period of time, no
    official git ever released produced delta that used the
    bit that way.

    In other words, if you have created packs with your own,
    modified git, that took advantage of "copy inside
    destination buffer" feature in the delta interpretation
    code, such packs are not usable by the official git, so
    you need to unpack them using your own version of git
    and then repack with the official version of git.

[PATCH] daemon: prepare for multiple services.

[Junio C Hamano]

This adds an infrastructure to selectively enable and disable
more than one services in git-daemon.  Currently upload-pack
service, which serves the git-fetch-pack and git-peek-remote
clients, is the only service that is defined.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 daemon.c |  113 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 106 insertions(+), 7 deletions(-)

diff --git a/daemon.c b/daemon.c
index 012936f..0a6cbb4 100644
--- a/daemon.c
+++ b/daemon.c
@@ -229,13 +229,42 @@ static char *path_ok(char *dir)
 	return NULL;		/* Fallthrough. Deny by default */
 }
 
-static int upload(char *dir)
+typedef int (*daemon_service_fn)(void);
+struct daemon_service {
+	const char *name;
+	const char *config_name;
+	daemon_service_fn fn;
+	int enabled;
+	int overridable;
+};
+
+static struct daemon_service *service_looking_at;
+static int service_enabled;
+
+static int git_daemon_config(const char *var, const char *value)
+{
+	if (!strncmp(var, "daemon.", 7) &&
+	    !strcmp(var + 7, service_looking_at->config_name)) {
+		service_enabled = git_config_bool(var, value);
+		return 0;
+	}
+
+	/* we are not interested in parsing any other configuration here */
+	return 0;
+}
+
+static int run_service(char *dir, struct daemon_service *service)
 {
-	/* Timeout as string */
-	char timeout_buf[64];
 	const char *path;
+	int enabled = service->enabled;
+
+	loginfo("Request %s for '%s'", service->name, dir);
 
-	loginfo("Request for '%s'", dir);
+	if (!enabled && !service->overridable) {
+		logerror("'%s': service not enabled.", service->name);
+		errno = EACCES;
+		return -1;
+	}
 
 	if (!(path = path_ok(dir)))
 		return -1;
@@ -257,12 +286,34 @@ static int upload(char *dir)
 		return -1;
 	}
 
+	if (service->overridable) {
+		service_looking_at = service;
+		service_enabled = -1;
+		git_config(git_daemon_config);
+		if (0 <= service_enabled)
+			enabled = service_enabled;
+	}
+	if (!enabled) {
+		logerror("'%s': service not enabled for '%s'",
+			 service->name, path);
+		errno = EACCES;
+		return -1;
+	}
+
 	/*
 	 * We'll ignore SIGTERM from now on, we have a
 	 * good client.
 	 */
 	signal(SIGTERM, SIG_IGN);
 
+	return service->fn();
+}
+
+static int upload_pack(void)
+{
+	/* Timeout as string */
+	char timeout_buf[64];
+
 	snprintf(timeout_buf, sizeof timeout_buf, "--timeout=%u", timeout);
 
 	/* git-upload-pack only ever reads stuff, so this is safe */
@@ -270,10 +321,36 @@ static int upload(char *dir)
 	return -1;
 }
 
+static struct daemon_service daemon_service[] = {
+	{ "upload-pack", "uploadpack", upload_pack, 1, 1 },
+};
+
+static void enable_service(const char *name, int ena) {
+	int i;
+	for (i = 0; i < ARRAY_SIZE(daemon_service); i++) {
+		if (!strcmp(daemon_service[i].name, name)) {
+			daemon_service[i].enabled = ena;
+			return;
+		}
+	}
+	die("No such service %s", name);
+}
+
+static void make_service_overridable(const char *name, int ena) {
+	int i;
+	for (i = 0; i < ARRAY_SIZE(daemon_service); i++) {
+		if (!strcmp(daemon_service[i].name, name)) {
+			daemon_service[i].overridable = ena;
+			return;
+		}
+	}
+	die("No such service %s", name);
+}
+
 static int execute(struct sockaddr *addr)
 {
 	static char line[1000];
-	int pktlen, len;
+	int pktlen, len, i;
 
 	if (addr) {
 		char addrbuf[256] = "";
@@ -310,8 +387,14 @@ #endif
 	if (len && line[len-1] == '\n')
 		line[--len] = 0;
 
-	if (!strncmp("git-upload-pack ", line, 16))
-		return upload(line+16);
+	for (i = 0; i < ARRAY_SIZE(daemon_service); i++) {
+		struct daemon_service *s = &(daemon_service[i]);
+		int namelen = strlen(s->name);
+		if (!strncmp("git-", line, 4) &&
+		    !strncmp(s->name, line + 4, namelen) &&
+		    line[namelen + 4] == ' ')
+			return run_service(line + namelen + 5, s);
+	}
 
 	logerror("Protocol error: '%s'", line);
 	return -1;
@@ -791,6 +874,22 @@ int main(int argc, char **argv)
 			log_syslog = 1;
 			continue;
 		}
+		if (!strncmp(arg, "--enable=", 9)) {
+			enable_service(arg + 9, 1);
+			continue;
+		}
+		if (!strncmp(arg, "--disable=", 10)) {
+			enable_service(arg + 10, 0);
+			continue;
+		}
+		if (!strncmp(arg, "--enable-override=", 18)) {
+			make_service_overridable(arg + 18, 1);
+			continue;
+		}
+		if (!strncmp(arg, "--disable-override=", 19)) {
+			make_service_overridable(arg + 19, 0);
+			continue;
+		}
 		if (!strcmp(arg, "--")) {
 			ok_paths = &argv[i+1];
 			break;
-- 
1.4.2.g0cac8

[PATCH] daemon: add upload-tar service.

[Junio C Hamano]

This allows clients to ask for tarballs with:

	git tar-tree --remote=git://server/repo refname

By default, the upload-tar service is not enabled.  To enable
it server-wide, the server can be started with:

	git-daemon --enable=upload-tar

This service is by default overridable per repostiory, so
alternatively, a repository can define "daemon.uploadtar = true"
to enable it.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 daemon.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/daemon.c b/daemon.c
index 0a6cbb4..5549014 100644
--- a/daemon.c
+++ b/daemon.c
@@ -321,8 +321,15 @@ static int upload_pack(void)
 	return -1;
 }
 
+static int upload_tar(void)
+{
+	execl_git_cmd("upload-tar", ".", NULL);
+	return -1;
+}
+
 static struct daemon_service daemon_service[] = {
 	{ "upload-pack", "uploadpack", upload_pack, 1, 1 },
+	{ "upload-tar", "uploadtar", upload_tar, 0, 1 },
 };
 
 static void enable_service(const char *name, int ena) {
-- 
1.4.2.g0cac8

Re: Unresolved issues #3

[A Large Angry SCM]

Junio C Hamano wrote:
> Nicolas Pitre <nico@cam.org> writes:
> 
>> On Fri, 18 Aug 2006, A Large Angry SCM wrote:
>>
>>> Historic fact. Between Thu May 19 08:56:22 2005 and Thu Feb  9 21:06:38
>>> 2006 bit 6 of the first byte of a delta hunk was interpreted to mean
>>> that the source of the copy was the result buffer. From Thu May 19
>>> 08:56:22 2005 on, the code to decode delta hunks in type 2 packs was
>>> available to everyone and anyone interested could make a pack encoder
>>> that would create packs that the core Git code would correctly read. The
>>> commit of Thu Feb  9 21:06:38 2006, d60fc, actually introduced a bug
>>> that would treat valid type 2 packs as invalid.
> 
> It is more like the said commit made the pack format extensible
> by declaring the bit reserved for the future use, by declaring
> retroactively that a type 2 pack that used that bit invalid.
> And it was deemed a reasonable and safe decision because no
> official git ever produced a type 2 pack that used that bit,
> 
> Yes, that was a backward incompatible change, strictly speaking,
> and probably I should have made an announcement that looked
> similar to this by Linus:
> 
>         From: Linus Torvalds <torvalds@osdl.org>
>         Subject: CAREFUL! No more delta object support!
>         Date: Mon, 27 Jun 2005 18:14:40 -0700 (PDT)
>         Message-ID: <Pine.LNX.4.58.0506271755140.19755@ppc970.osdl.org>
>         To: Git Mailing List <git@vger.kernel.org>
> 
> So you could argue I was incompetent not to make a big fuss
> about this backward incompatibility back then, if you like.
> 
> I did not think it was worth it back then, and I do not think it
> is worth it now, either.  But if it makes you feel better, I
> could retroactively make such an announcement about the
> unofficial bit 6.
> 
> The announcement would have read like this:
> 
>     The current git code does not support type #2 packs that
>     uses delta with bit 6 to mean "copy inside destination
>     buffer".  Although the code that interpreted delta data
>     supported bit 6 that way for a brief period of time, no
>     official git ever released produced delta that used the
>     bit that way.
> 
>     In other words, if you have created packs with your own,
>     modified git, that took advantage of "copy inside
>     destination buffer" feature in the delta interpretation
>     code, such packs are not usable by the official git, so
>     you need to unpack them using your own version of git
>     and then repack with the official version of git.

Please read the commit message for commit d60fc. It's type _3_ pack
files that redefined bit 6 to add the extra byte of copy length, not
type 2. Thus, no need to retroactively invalidate the type 2 pack files
that used copy from result.

Re: Unresolved issues #3

[Martin Langhoff]

On 8/18/06, Junio C Hamano <junkio@cox.net> wrote:
> As everybody has probably noticed already, I am terrible at
> maintaining "the current issues" list.  The most recent issue of
> this series was done when?  Back on May 4th this year.
>
> Shame on me.

Hey, I think you're doing a great job. A small improvement may be to
copy these to the wiki for longer term, but we're all lazy about
maintaining them.

Perhaps at least have a TODO page? How about what I just placed in
http://git.or.cz/gitwiki/ToDo ? A link to the 'todo' head, and to a
search for 'Unresolved Issues' in the list archives...

Should give newcomers a hint of where things are, and how to gauge
directions and interests...



martin

Re: Unresolved issues #3

[Jakub Narebski]

There was a thread about system-wide and per-user git configuration. Some
patches were accepted... but it is nowhere documented. Not all ideas are
incorporated in git.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Unresolved issues #4

[Junio C Hamano]

Recent issues I am aware of but haven't kept track of closely
enough and as a result do not have a clue about X-<.

From: A Large Angry SCM <gitzilla@gmail.com>
Subject: Notes on Using Git with Subprojects
Message-ID: <45196628.9010107@gmail.com>

[jc: a very nice write-up of a subprojects workflow. I do not
 remember if it produced any actionable items, though]

From: Franck Bui-Huu <vagabon.xyz@gmail.com>
Message-ID: <450EABD0.1040102@innova-card.com>

Repeated requests against git-daemon makes it stuck

[jc: does not reproduce easily for me; has anybody seen it?]

From: Shawn Pearce <spearce@spearce.org>
Message-ID: <20060926215745.GC8177@spearce.org>

git-mirror (reverse of git-push --all).

[jc: any progress?]

From: Junio C Hamano <junkio@cox.net>
Message-ID: <7v7izrzpk2.fsf@assigned-by-dhcp.cox.net>

Deal with rfc2822-invalid author mail address in send-email.

[jc: forgot to apply?]

From: Nicolas Pitre <nico@cam.org>
Subject: [PATCH 8/6] let the GIT native protocol use offsets to delta base when

[jc: applied all but I suspect git-push side hasn't been converted?]

From: Shawn Pearce <spearce@spearce.org>
Message-ID: <20060930045037.GB18479@spearce.org>

"git ref-log" command to interact with ref-log?

[jc: not much interest from users?]

From: Stefan Richter <stefanr@s5r6.in-berlin.de>
Message-ID: <4523EC14.6070806@s5r6.in-berlin.de>

AsciiDoc 8 does not grok documents written for AsciiDoc 7 out of
the box.

[jc: status?]

From:	Josh Triplett <josh@freedesktop.org>
Message-ID: <451A30E4.50801@freedesktop.org>

git-split

[jc: no response to the initial review comments]

Re: Unresolved issues #4

[Jakub Narebski]

Junio C Hamano wrote:

> Recent issues I am aware of but haven't kept track of closely
> enough and as a result do not have a clue about X-<.
> 
> From: A Large Angry SCM <gitzilla@gmail.com>
> Subject: Notes on Using Git with Subprojects
> Message-ID: <45196628.9010107@gmail.com>
> 
> [jc: a very nice write-up of a subprojects workflow. I do not
>  remember if it produced any actionable items, though]

It did produce two workflows: the original "build time"
by A Large Angry SCM/Gitzilla, which I think is/will be
available at http://git.rsbx.net/Notes/Git_Subprojects.txt
and second using linked objects and refs. Both I think not
quite finished; perhaps some of the code could be put
in 'pu'.

Perhaps to be added to Subpro.txt in 'todo' branch,
and/or to git wiki...

> From: Shawn Pearce <spearce@spearce.org>
> Message-ID: <20060930045037.GB18479@spearce.org>
> 
> "git ref-log" command to interact with ref-log?
> 
> [jc: not much interest from users?]

I thing it can increase number of reflog users. 

> From: Josh Triplett <josh@freedesktop.org>
> Message-ID: <451A30E4.50801@freedesktop.org>
> 
> git-split

Should be git-split-hierarchy or git-split-by-directory
I think. We could have also git-split-history, which would
split current history into archive repository, and active
repository with archive repository grafted in; or add to
archive repository if it exists, regraft active (current work)
repository and prune below new grafts. 

> [jc: no response to the initial review comments]

It is "done once" problem, and usable only for some repositories,
so perhaps this is the cause of not many responses.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Unresolved issues #4

[Jon Loeliger]

On Fri, 2006-10-06 at 01:26, Junio C Hamano wrote:

> From: Franck Bui-Huu <vagabon.xyz@gmail.com>
> Message-ID: <450EABD0.1040102@innova-card.com>
> 
> Repeated requests against git-daemon makes it stuck
> 
> [jc: does not reproduce easily for me; has anybody seen it?]

I've not seen that behavior either.  However, I
suspect that this might be a poorly described
issue with _clients_ that are out of date.
Specifically, if a client is older than 1.4.2.1,
it shows up in the git-daemon log as just a
single "connection from <ipaddr>" line and no
actual transfer then happens, making it perhaps
appear "stuck".

jdl

Re: Unresolved issues #4

[Shawn Pearce]

Jakub Narebski <jnareb@gmail.com> wrote:
> > From: Shawn Pearce <spearce@spearce.org>
> > Message-ID: <20060930045037.GB18479@spearce.org>
> > 
> > "git ref-log" command to interact with ref-log?
> > 
> > [jc: not much interest from users?]
> 
> I thing it can increase number of reflog users. 

Agreed and I'd really like to see that command but I have not had the
time to do code it myself.  What time I am able to make available
for Git in the near future needs to be for the window mmap rewrite
ontop of Nico's delta offset changes that are currently in next.

If nobody gets around to it I'll likely try to pick it up and do it,
but that probably won't be for at least a month, maybe two.

-- 
Shawn.

Re: Unresolved issues #4

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> From: Shawn Pearce <spearce@spearce.org>
> Message-ID: <20060926215745.GC8177@spearce.org>
> 
> git-mirror (reverse of git-push --all).
> 
> [jc: any progress?]

No.  Back burner.  Waaaaaaay back.  I have your comments and I'd
like to incorporate them and resubmit but I've had other things
demanding my time.

-- 
Shawn.

Re: Unresolved issues #4

[A Large Angry SCM]

Junio C Hamano wrote:
> Recent issues I am aware of but haven't kept track of closely
> enough and as a result do not have a clue about X-<.
> 
> From: A Large Angry SCM <gitzilla@gmail.com>
> Subject: Notes on Using Git with Subprojects
> Message-ID: <45196628.9010107@gmail.com>
> 
> [jc: a very nice write-up of a subprojects workflow. I do not
>  remember if it produced any actionable items, though]
> 

No actionable items that I'm aware of.