Git development
 help / color / mirror / Atom feed
* Re: Symbolic refs and the git protocol?
From: Sitaram Chamarty @ 2011-09-26 14:23 UTC (permalink / raw)
  To: Avi Kivity; +Cc: Git Mailing List, John Hawley
In-Reply-To: <CAMK1S_g1Co8vogBuObQVsr+nT09urYs+qifJn1X+eGrkgUQZCA@mail.gmail.com>

On Sun, Sep 25, 2011 at 7:40 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:
> On Sun, Sep 25, 2011 at 7:18 PM, Avi Kivity <avi@redhat.com> wrote:
>> On 09/25/2011 04:42 PM, Sitaram Chamarty wrote:
>>>
>>> On Sun, Sep 25, 2011 at 6:47 PM, Avi Kivity<avi@redhat.com>  wrote:
>>> >  With the old kernel.org setup, practically the only think I needed to
>>> > do on
>>> >  the server itself is the 'git symbolic-ref' command (used to point
>>> >  refs/heads/linux-next at the correct stream).
>>> >
>>> >  Now it looks like kernel.org will no longer offer shell access.  Is it
>>> >  possible to support symbolic-ref via 'git push'?
>>>
>>> Can I assume you mean specifically for HEAD, as in 'git symbolic-ref
>>> HEAD refs/heads/some-branch'?
>>
>> No, I use them for ordinary branches, for example
>>
>>  git symbolic-ref refs/heads/linux-next refs/heads/kvm-updates/3.2
>>
>> from this point on, any update to kvm-updates/3.2 will be reflected in the
>> linux-next ref.
>>
>>> Although gitolite disables shell access by design, it provides several
>>> (optionally installed) features to allow specific shell commands to be
>>> executed.  One such command is "set-head", which does exactly what you
>>> ask.  You run it like 'ssh git@server set-head reponame.git
>>> refs/heads/foo' and it ends up running 'git symbolic-ref HEAD
>>> refs/heads/foo' in the repo.
>>>
>>> I realise that does not answer "can it be done via git push" but I
>>> thought it might help...
>>
>> Well, if set-head can be extended a bit, it would suit me perfectly.
>
> no reason it can't; shorn of all the argument/access checking, it's a
> very simple shell script

Done.

https://github.com/sitaramc/gitolite/commit/a2fd597c51a086a0f7298c1641337faa0f296a62

I like this much better actually, so thanks for the idea.

^ permalink raw reply

* Re: subversion-perl missing
From: Christian Couder @ 2011-09-26 14:48 UTC (permalink / raw)
  To: Georg-Johann Lay; +Cc: git
In-Reply-To: <4E8066AB.7000208@gjlay.de>

Hi,

On Mon, Sep 26, 2011 at 1:48 PM, Georg-Johann Lay <avr@gjlay.de> wrote:
>
> Where do I have to go to get the svn+perl stuff? Couldn't locate anything like
> that in subversion.tigris.org, so that I am stuck and need some hints on how to
> proceed.

Perhaps you can try to install this module from CPAN:

http://search.cpan.org/~mschwern/Alien-SVN-v1.6.12.1/

or from it's github repo:

https://github.com/schwern/alien-svn

Regards,
Christian.

^ permalink raw reply

* Re: Symbolic refs and the git protocol?
From: Avi Kivity @ 2011-09-26 14:49 UTC (permalink / raw)
  To: Sitaram Chamarty; +Cc: Git Mailing List, John Hawley
In-Reply-To: <CAMK1S_gfw4WUdhdCzFMNbJXJP7762tp62fpmQ_-qJLwvD1hR-A@mail.gmail.com>

On 09/26/2011 05:23 PM, Sitaram Chamarty wrote:
> On Sun, Sep 25, 2011 at 7:40 PM, Sitaram Chamarty<sitaramc@gmail.com>  wrote:
> >  On Sun, Sep 25, 2011 at 7:18 PM, Avi Kivity<avi@redhat.com>  wrote:
> >>  On 09/25/2011 04:42 PM, Sitaram Chamarty wrote:
> >>>
> >>>  On Sun, Sep 25, 2011 at 6:47 PM, Avi Kivity<avi@redhat.com>    wrote:
> >>>  >    With the old kernel.org setup, practically the only think I needed to
> >>>  >  do on
> >>>  >    the server itself is the 'git symbolic-ref' command (used to point
> >>>  >    refs/heads/linux-next at the correct stream).
> >>>  >
> >>>  >    Now it looks like kernel.org will no longer offer shell access.  Is it
> >>>  >    possible to support symbolic-ref via 'git push'?
> >>>
> >>>  Can I assume you mean specifically for HEAD, as in 'git symbolic-ref
> >>>  HEAD refs/heads/some-branch'?
> >>
> >>  No, I use them for ordinary branches, for example
> >>
> >>    git symbolic-ref refs/heads/linux-next refs/heads/kvm-updates/3.2
> >>
> >>  from this point on, any update to kvm-updates/3.2 will be reflected in the
> >>  linux-next ref.
> >>
> >>>  Although gitolite disables shell access by design, it provides several
> >>>  (optionally installed) features to allow specific shell commands to be
> >>>  executed.  One such command is "set-head", which does exactly what you
> >>>  ask.  You run it like 'ssh git@server set-head reponame.git
> >>>  refs/heads/foo' and it ends up running 'git symbolic-ref HEAD
> >>>  refs/heads/foo' in the repo.
> >>>
> >>>  I realise that does not answer "can it be done via git push" but I
> >>>  thought it might help...
> >>
> >>  Well, if set-head can be extended a bit, it would suit me perfectly.
> >
> >  no reason it can't; shorn of all the argument/access checking, it's a
> >  very simple shell script
>
> Done.
>
> https://github.com/sitaramc/gitolite/commit/a2fd597c51a086a0f7298c1641337faa0f296a62
>
> I like this much better actually, so thanks for the idea.

Thanks very much for the rapid response and update!

John, can we get this into the new kernel.org infrastructure?

-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply

* Re: subversion-perl missing
From: Georg-Johann Lay @ 2011-09-26 14:51 UTC (permalink / raw)
  To: Michael J Gruber; +Cc: git
In-Reply-To: <4E8089BC.6030000@drmicha.warpmail.net>

Michael J Gruber schrieb:
> Georg-Johann Lay venit, vidit, dixit 26.09.2011 15:41:
>> Michael J Gruber schrieb:
>>> Georg-Johann Lay venit, vidit, dixit 26.09.2011 13:48:
>>>> Hi, I just downloaded git and built from source, version is
>>>>
>>>> git version 1.7.6.4
>>>>
>>>> I am particularly interested in git-svn, but running it I get complaints
>>>>
>>>> Initialized empty Git repository in /local/Downloads/v4.5.x/.git/
>>>> Can't locate SVN/Core.pm in @INC (@INC contains:
>>>> /usr/local/lib/perl5/site_perl/5.10.0/i586-linux-thread-multi
>>>> /usr/local/lib/perl5/site_perl/5.10.0
>>>> /usr/lib/perl5/5.10.0/i586-linux-thread-multi /usr/lib/perl5/5.10.0
>>>> /usr/lib/perl5/site_perl/5.10.0/i586-linux-thread-multi
>>>> /usr/lib/perl5/site_perl/5.10.0
>>>> /usr/lib/perl5/vendor_perl/5.10.0/i586-linux-thread-multi
>>>> /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl .) at
>>>> /usr/local/libexec/git-core/git-svn line 42
>>>>
>>>> I found out that a module like subversion-perl is missing, but installing it
>>>> via OpenSuse 1.11 would downgrade my SVN installation, so I skipped that.
>>> So, where did you get your svn from? If you compiled it yourself, you
>>> should rebuild it with the perl bindings. Make sure you note where you
>>> install this to - you may have to adjust your PERL5LIB.
>> Hi Michael
>>
>> svn identifies itself as
>>
>>  > svn --version
>>
>> svn, version 1.6.2 (r37639)
>>    compiled Jun 19 2009, 12:21:15
>>
>> Copyright (C) 2000-2009 CollabNet.
>> Subversion is open source software, see http://subversion.tigris.org/
>> This product includes software developed by CollabNet (http://www.Collab.Net/).
>>
>> The following repository access (RA) modules are available:
>>
>> * ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
>>   - handles 'http' scheme
>>   - handles 'https' scheme
>> * ra_svn : Module for accessing a repository using the svn network protocol.
>>   - with Cyrus SASL authentication
>>   - handles 'svn' scheme
>> * ra_local : Module for accessing a repository on local disk.
>>   - handles 'file' scheme
>>
>> and its installed in
>>
>>  > which svn
>>
>> /usr/bin/svn
>>
>>  > echo $PERL5LIB
>>
>> <empty>
>>
>> so looks like plain vanilla installation/build.
>>
>> Is "perl bindings" the "subversion-perl" package (which likes to downgrade my
>> svn upon installing in yast)?
> 
> Yes, subversion-perl are the perl bindings, and if your distro has
> mismatching versions of subversion and subversion perl, then they have a
> problem, sorry :|
>> Is it a part of SVN (couln't find it there) or of perl?


I could find
http://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/swig/INSTALL
and hope that explains the trick so that a noob like me doesn't get lost...

Many, many thanks for your help Michael!

Johann

^ permalink raw reply

* Re: subversion-perl missing
From: Michael J Gruber @ 2011-09-26 14:54 UTC (permalink / raw)
  To: Georg-Johann Lay; +Cc: git
In-Reply-To: <4E80916A.9060809@gjlay.de>

Georg-Johann Lay venit, vidit, dixit 26.09.2011 16:51:
> Michael J Gruber schrieb:
>> Georg-Johann Lay venit, vidit, dixit 26.09.2011 15:41:
>>> Michael J Gruber schrieb:
>>>> Georg-Johann Lay venit, vidit, dixit 26.09.2011 13:48:
>>>>> Hi, I just downloaded git and built from source, version is
>>>>>
>>>>> git version 1.7.6.4
>>>>>
>>>>> I am particularly interested in git-svn, but running it I get complaints
>>>>>
>>>>> Initialized empty Git repository in /local/Downloads/v4.5.x/.git/
>>>>> Can't locate SVN/Core.pm in @INC (@INC contains:
>>>>> /usr/local/lib/perl5/site_perl/5.10.0/i586-linux-thread-multi
>>>>> /usr/local/lib/perl5/site_perl/5.10.0
>>>>> /usr/lib/perl5/5.10.0/i586-linux-thread-multi /usr/lib/perl5/5.10.0
>>>>> /usr/lib/perl5/site_perl/5.10.0/i586-linux-thread-multi
>>>>> /usr/lib/perl5/site_perl/5.10.0
>>>>> /usr/lib/perl5/vendor_perl/5.10.0/i586-linux-thread-multi
>>>>> /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl .) at
>>>>> /usr/local/libexec/git-core/git-svn line 42
>>>>>
>>>>> I found out that a module like subversion-perl is missing, but installing it
>>>>> via OpenSuse 1.11 would downgrade my SVN installation, so I skipped that.
>>>> So, where did you get your svn from? If you compiled it yourself, you
>>>> should rebuild it with the perl bindings. Make sure you note where you
>>>> install this to - you may have to adjust your PERL5LIB.
>>> Hi Michael
>>>
>>> svn identifies itself as
>>>
>>>  > svn --version
>>>
>>> svn, version 1.6.2 (r37639)
>>>    compiled Jun 19 2009, 12:21:15
>>>
>>> Copyright (C) 2000-2009 CollabNet.
>>> Subversion is open source software, see http://subversion.tigris.org/
>>> This product includes software developed by CollabNet (http://www.Collab.Net/).
>>>
>>> The following repository access (RA) modules are available:
>>>
>>> * ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
>>>   - handles 'http' scheme
>>>   - handles 'https' scheme
>>> * ra_svn : Module for accessing a repository using the svn network protocol.
>>>   - with Cyrus SASL authentication
>>>   - handles 'svn' scheme
>>> * ra_local : Module for accessing a repository on local disk.
>>>   - handles 'file' scheme
>>>
>>> and its installed in
>>>
>>>  > which svn
>>>
>>> /usr/bin/svn
>>>
>>>  > echo $PERL5LIB
>>>
>>> <empty>
>>>
>>> so looks like plain vanilla installation/build.
>>>
>>> Is "perl bindings" the "subversion-perl" package (which likes to downgrade my
>>> svn upon installing in yast)?
>>
>> Yes, subversion-perl are the perl bindings, and if your distro has
>> mismatching versions of subversion and subversion perl, then they have a
>> problem, sorry :|
>>> Is it a part of SVN (couln't find it there) or of perl?
> 
> 
> I could find
> http://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/swig/INSTALL
> and hope that explains the trick so that a noob like me doesn't get lost...

That requires you to build swig and subversion, which I thought you
wanted to avoid. But if that does not scare you - by all means go ahead :)

> Many, many thanks for your help Michael!
> 
> Johann

Cheers,
Michael

^ permalink raw reply

* Re: How to use git attributes to configure server-side checks?
From: Michael Haggerty @ 2011-09-26 15:11 UTC (permalink / raw)
  To: Jeff King
  Cc: Jay Soffian, Junio C Hamano, git discussion list, Jakub Narebski
In-Reply-To: <20110926110552.GA20796@sigill.intra.peff.net>

On 09/26/2011 01:05 PM, Jeff King wrote:
> On Sat, Sep 24, 2011 at 01:03:50PM +0200, Michael Haggerty wrote:
>> There are obviously config options that should not be versioned.  These
>> should not be stored in the tree.  (Perhaps they should be stored in
>> another branch, as you suggested.)  But there's no need to prevent
>> people from versioning *any* config options just because they shouldn't
>> version *all* of them.
> 
> Do you have an example of a config option whose old value should be
> respected when looking at an old version? In general, the split between
> attributes and config was meant to split what is tied to a specific
> version, and what should apply to all versions. We may not have followed
> that in all cases, but if so, the right solution may actually be
> migrating the functionality into .gitattributes.

I hadn't realized this point about the split between gitconfig and
gitattributes.  It makes sense.  I'd been thinking that it would be nice
to be able to share changes to the git config the same way that source
code is shared, but you are right that the git config is versioned
differently than the source code.  I can't think of any counterexamples
right now.

>> While we are on the topic of config settings, I have often thought that
>> it would be nice for git's default settings to be set via a
>> well-commented config file, installed along with git, rather than via
>> values compiled into the code.  This file and Documentation/config.txt
>> could be generated from a single source file as part of the build
>> process.
> 
> I think that can be a nice piece of documentation, but there may be some
> complications.  I seem to recall that there may be one or two options
> whose builtin values cannot be replicated via config (i.e., the "unset"
> state means something). But I may be misremembering.

This is more a reason to implement a way to "unset" options than an
argument against storing git's builtin defaults in a gitconfig-formatted
file.

> However, I'm not sure what you mean by "rather than via values compiled
> into the code".  Would you somehow generate code that sets the default
> according to your master file? Would git fail to start if the file is
> missing? Or if a specific config option is missing? How would you track
> that?

I think that git should read its "as-installed" config file at runtime
to maximize the transparency.  If the file is missing then git should
fail, just as it would if some other important part of git were missing
from an install.  The absence of a required config option would be a bug
and might cause git to abort just like any other bug.

I probably shouldn't mention this lest people realize that I'm stupid
and ugly, but...

Subversion installs a file with all of its defaults into your
~/.subversion directory the first time that it runs.  I find this
feature to be very helpful as a source of examples and to learn new
features.  But they do it a bit differently than I am proposing for
git--the file is written to the user's ~/.subversion, and is
subsequently edited by the user.  Because the file lives across
Subversion upgrades, the options in this file are *commented out* to
prevent them causing trouble in the future; i.e., they are for human
consumption only.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-26 15:15 UTC (permalink / raw)
  To: git, Julian Phillips
In-Reply-To: <201109251443.28243.mfick@codeaurora.org>

OK, I have found what I believe is another performance 
regression for large ref counts (~100K).  

When I run git br on my repo which only has one branch, but 
has ~100K refs under ref/changes (a gerrit repo), it takes 
normally 3-6mins depending on whether my caches are fresh or 
not.  After bisecting some older changes, I noticed that 
this ref seems to be where things start to get slow: 
c774aab98ce6c5ef7aaacbef38da0a501eb671d4


commit c774aab98ce6c5ef7aaacbef38da0a501eb671d4
Author: Julian Phillips <julian@quantumfyre.co.uk>
Date:   Tue Apr 17 02:42:50 2007 +0100

    refs.c: add a function to sort a ref list, rather then 
sorting on add

    Rather than sorting the refs list while building it, 
sort in one
    go after it is built using a merge sort.  This has a 
large
    performance boost with large numbers of refs.

    It shouldn't happen that we read duplicate entries into 
the same
    list, but just in case sort_ref_list drops them if the 
SHA1s are
    the same, or dies, as we have no way of knowing which 
one is the
    correct one.

    Signed-off-by: Julian Phillips 
<julian@quantumfyre.co.uk>
    Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>



which is a bit strange since that commit's purpose was to 
actually speed things up in the case of many refs.  Just to 
verify, I reverted the commit on 1.7.7.rc0.73 and sure 
enough, things speed up down to the 14-20s range depending 
on caching.

If this change does not actually speed things up, should it 
be reverted?  Or was there a bug in the change that makes it 
not do what it was supposed to do?

Thanks,

-Martin

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Sverre Rabbelier @ 2011-09-26 15:21 UTC (permalink / raw)
  To: Martin Fick; +Cc: git, Julian Phillips
In-Reply-To: <201109260915.29285.mfick@codeaurora.org>

Heya,

On Mon, Sep 26, 2011 at 17:15, Martin Fick <mfick@codeaurora.org> wrote:
> If this change does not actually speed things up, should it
> be reverted?  Or was there a bug in the change that makes it
> not do what it was supposed to do?

It probably looks at the refs in refs/changes while it shouldn't,
hence worsening your performance compared to not looking at those
refs. I assume that it does improve your situation if you have all
those refs under say refs/heads.

-- 
Cheers,

Sverre Rabbelier

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Michael Haggerty @ 2011-09-26 15:32 UTC (permalink / raw)
  To: Martin Fick; +Cc: git, Christian Couder
In-Reply-To: <201109251443.28243.mfick@codeaurora.org>

On 09/25/2011 10:43 PM, Martin Fick wrote:
> A coworker of mine pointed out to me that a simple
> 
>   git checkout 
> 
> can also take rather long periods of time > 3 mins when run 
> on a repo with ~100K refs.  
> 
> While this is not massive like the other problem I reported, 
> it still seems like it is more than one would expect.  So, I 
> tried an older version of git, and to my surprise/delight, 
> it was much faster (.2s).  So, I bisected this issue also, 
> and it seems that the "offending" commit is 
> 680955702990c1d4bfb3c6feed6ae9c6cb5c3c07:

I'm still working on changes to store references hierarchically in the
cache and read them lazily.  I hope that it will help some scaling
problems with large number of refs.

Unfortunately I keep getting tangled up in side issues, so it is taking
a lot longer than expected.  But there's still hope.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-26 15:42 UTC (permalink / raw)
  To: Michael Haggerty; +Cc: git, Christian Couder
In-Reply-To: <4E809AFE.6010901@alum.mit.edu>

On Monday, September 26, 2011 09:32:14 am Michael Haggerty 
wrote:
> On 09/25/2011 10:43 PM, Martin Fick wrote:
> > A coworker of mine pointed out to me that a simple
> > 
> >   git checkout
> > 
> > can also take rather long periods of time > 3 mins when
> > run on a repo with ~100K refs.
> > 
> > While this is not massive like the other problem I
> > reported, it still seems like it is more than one
> > would expect.  So, I tried an older version of git,
> > and to my surprise/delight, it was much faster (.2s). 
> > So, I bisected this issue also, and it seems that the
> > "offending" commit is
> 
> > 680955702990c1d4bfb3c6feed6ae9c6cb5c3c07:
> I'm still working on changes to store references
> hierarchically in the cache and read them lazily.  I
> hope that it will help some scaling problems with large
> number of refs.
> 
> Unfortunately I keep getting tangled up in side issues,
> so it is taking a lot longer than expected.  But there's
> still hope.
> 
> Michael

Thanks Michael, I look forward to those changes.  In the 
meantime however, I will try to take advantage of the 
current inefficiencies of large ref counts to attempt to 
find places where there are obvious problems in the code 
paths.  I suspect that there are several commands in git 
which inadvertently scan all the refs when they probably 
shouldn't.  Since this is likely very slow now, it should be 
easy to find those, if it were faster, this might get 
overlooked.  I feel like git checkout is one of those cases, 
it does not seem like git checkout should be affected by the 
number of refs in a repo?

-Martin

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-26 15:48 UTC (permalink / raw)
  To: Sverre Rabbelier; +Cc: git, Julian Phillips
In-Reply-To: <CAGdFq_iuY0-PBDOmtz1pRvh6J9EDRiRJHsWkTN_cHjGU20PYTQ@mail.gmail.com>

On Monday, September 26, 2011 09:21:30 am Sverre Rabbelier 
wrote:
> Heya,
> 
> On Mon, Sep 26, 2011 at 17:15, Martin Fick 
<mfick@codeaurora.org> wrote:
> > If this change does not actually speed things up,
> > should it be reverted?  Or was there a bug in the
> > change that makes it not do what it was supposed to
> > do?
> 
> It probably looks at the refs in refs/changes while it
> shouldn't, hence worsening your performance compared to
> not looking at those refs. I assume that it does improve
> your situation if you have all those refs under say
> refs/heads.

Hmm, I was thinking that too, and I just did a test. 

Instead of storing the changes under refs/changes, I fetched 
them under refs/heads/changes and then ran git 1.7.6 and it 
took about 3 mins.  Then, I ran the 1.7.7.rc0.73 with 
c774aab98ce6c5ef7aaacbef38da0a501eb671d4 reverted and it 
only took 13s!  So, if this indeed tests what you were 
suggesting, I think it shows that even in the intended case 
this change slowed things down?

-Martin

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Sverre Rabbelier @ 2011-09-26 15:56 UTC (permalink / raw)
  To: Martin Fick; +Cc: git, Julian Phillips
In-Reply-To: <201109260948.25312.mfick@codeaurora.org>

Heya,

On Mon, Sep 26, 2011 at 17:48, Martin Fick <mfick@codeaurora.org> wrote:
> Hmm, I was thinking that too, and I just did a test.
>
> Instead of storing the changes under refs/changes, I fetched
> them under refs/heads/changes and then ran git 1.7.6 and it
> took about 3 mins.  Then, I ran the 1.7.7.rc0.73 with
> c774aab98ce6c5ef7aaacbef38da0a501eb671d4 reverted and it
> only took 13s!  So, if this indeed tests what you were
> suggesting, I think it shows that even in the intended case
> this change slowed things down?

And if you run 1.7.7 without that commit reverted?

-- 
Cheers,

Sverre Rabbelier

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Thomas Rast @ 2011-09-26 16:25 UTC (permalink / raw)
  To: Martin Fick; +Cc: Michael Haggerty, git, Christian Couder
In-Reply-To: <201109260942.08299.mfick@codeaurora.org>

Martin Fick wrote:
> 
> I suspect that there are several commands in git 
> which inadvertently scan all the refs when they probably 
> shouldn't. [...] I feel like git checkout is one of those cases, 
> it does not seem like git checkout should be affected by the 
> number of refs in a repo?

git-checkout checks whether you are leaving any unreferenced
(orphaned) commits behind when you leave a detached HEAD, which
requires that it scan the history of all refs for the commit you just
left.

So unless you disable that warning it'll be pretty expensive
regardless.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

^ permalink raw reply

* Re: [PATCH] send-email: auth plain/login fix
From: Junio C Hamano @ 2011-09-26 16:38 UTC (permalink / raw)
  To: Zbigniew Jędrzejewski-Szmek; +Cc: joe, git
In-Reply-To: <1316879367-1182-1-git-send-email-zbyszek@in.waw.pl>

Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> writes:

> git send-email was not authenticating properly when communicating over
> TLS with a server supporting only AUTH PLAIN and AUTH LOGIN. This is
> e.g. the standard server setup under debian with exim4 and probably
> everywhere where system accounts are used.

Now that's a solution that makes me feel less dirty than

  http://thread.gmane.org/gmane.comp.version-control.git/178818/focus=178824

even though, by forcing Authen::SASL::Perl to be used bypassing XS and
Cyrus variants, this _might_ be introducing regression for others. We'll
find out soon enough if anybody screams ;-)

I see you already researched previous discussion and have Joe Perches in
the loop; Thanks.

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-26 16:38 UTC (permalink / raw)
  To: Sverre Rabbelier; +Cc: git, Julian Phillips
In-Reply-To: <CAGdFq_hRmSif4=V+9h8=S1fWfPCj+meRY8xGyfgv=SWk+DrBQw@mail.gmail.com>

On Monday, September 26, 2011 09:56:50 am Sverre Rabbelier 
wrote:
> Heya,
> 
> On Mon, Sep 26, 2011 at 17:48, Martin Fick 
<mfick@codeaurora.org> wrote:
> > Hmm, I was thinking that too, and I just did a test.
> > 
> > Instead of storing the changes under refs/changes, I
> > fetched them under refs/heads/changes and then ran git
> > 1.7.6 and it took about 3 mins.  Then, I ran the
> > 1.7.7.rc0.73 with
> > c774aab98ce6c5ef7aaacbef38da0a501eb671d4 reverted and
> > it only took 13s!  So, if this indeed tests what you
> > were suggesting, I think it shows that even in the
> > intended case this change slowed things down?
> 
> And if you run 1.7.7 without that commit reverted?

Sorry, I probably confused things by mentioning 1.7.6, the 
bad commit was way before that early 1.5 days...  

As for 1.7.7, I don't think that exists yet, so did you mean 
the 1.7.7.rc0.73 version that I mentioned above without the 
revert?  Strangely enough, that ends up being 
1.7.7.rc0.72.g4b5ea.  That is also slow with 
refs/heads/changes > 3mins.

-Martin

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum

^ permalink raw reply

* Re: [PATCH] mergetool: Use args as pathspec to unmerged files
From: Junio C Hamano @ 2011-09-26 16:40 UTC (permalink / raw)
  To: David Aguilar; +Cc: Jonathon Mah, git, Dan McGee
In-Reply-To: <20110925004459.GA19780@gmail.com>

David Aguilar <davvid@gmail.com> writes:

>> Ack from mergetool/difftool folks?
> ...
> Acked-by: David Aguilar <davvid@gmail.com>
>
> Being able to do "git mergetool -- subdir" is very nice!
> Thanks guys,

Thanks.

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Julian Phillips @ 2011-09-26 16:49 UTC (permalink / raw)
  To: Martin Fick; +Cc: Sverre Rabbelier, git
In-Reply-To: <201109261038.34527.mfick@codeaurora.org>

On Mon, 26 Sep 2011 10:38:34 -0600, Martin Fick wrote:
> On Monday, September 26, 2011 09:56:50 am Sverre Rabbelier
> wrote:
>> Heya,
>>
>> On Mon, Sep 26, 2011 at 17:48, Martin Fick
> <mfick@codeaurora.org> wrote:
>> > Hmm, I was thinking that too, and I just did a test.
>> >
>> > Instead of storing the changes under refs/changes, I
>> > fetched them under refs/heads/changes and then ran git
>> > 1.7.6 and it took about 3 mins.  Then, I ran the
>> > 1.7.7.rc0.73 with
>> > c774aab98ce6c5ef7aaacbef38da0a501eb671d4 reverted and
>> > it only took 13s!  So, if this indeed tests what you
>> > were suggesting, I think it shows that even in the
>> > intended case this change slowed things down?
>>
>> And if you run 1.7.7 without that commit reverted?
>
> Sorry, I probably confused things by mentioning 1.7.6, the
> bad commit was way before that early 1.5 days...
>
> As for 1.7.7, I don't think that exists yet, so did you mean
> the 1.7.7.rc0.73 version that I mentioned above without the
> revert?  Strangely enough, that ends up being
> 1.7.7.rc0.72.g4b5ea.  That is also slow with
> refs/heads/changes > 3mins.

Hmm ... something interesting is going on.

I created a little test repo with ~100k unpacked refs.

I tried "time git branch" with three versions of git, and I got (hot 
cache times):

git version 1.7.6.1: ~1.2s
git version 1.7.7.rc3: ~1.2s
git version 1.7.7.rc3.1.gbc93f: ~40s

Where the third was with the commit reverted.  That was almost 40s of 
100% CPU - my poor laptop had to turn the fans up to noisy ...

> -Martin

-- 
Julian

^ permalink raw reply

* Re: subversion-perl missing
From: Georg-Johann Lay @ 2011-09-26 16:49 UTC (permalink / raw)
  To: Christian Couder; +Cc: git
In-Reply-To: <CAP8UFD0PiiA9SxvOuuh4t8P96v+iaJ0TfcBr1-xjMh4TXBEONQ@mail.gmail.com>

Christian Couder schrieb:
> Hi,
> 
> On Mon, Sep 26, 2011 at 1:48 PM, Georg-Johann Lay <spam@spam.com> wrote:
>> Where do I have to go to get the svn+perl stuff? Couldn't locate anything like
>> that in subversion.tigris.org, so that I am stuck and need some hints on how to
>> proceed.
> 
> Perhaps you can try to install this module from CPAN:
> 
> http://search.cpan.org/~mschwern/Alien-SVN-v1.6.12.1/
> 
> or from it's github repo:
> 
> https://github.com/schwern/alien-svn

Hi Christian,

thanks for that hint.

After getting that piece of software, running ./Build.PL and ./Build I just
don't know what to do next.  The README doesn't say anything about how to use
the built modules or how to do an install.

As git-svn complains

...Can't locate SVN/Core.pm in @INC (@INC contains: ...

and

 schwern-alien-svn-9298884>find . -name 'Core.pm'
./src/subversion/subversion/bindings/swig/perl/native/Core.pm
./src/subversion/subversion/bindings/swig/perl/native/blib/lib/SVN/Core.pm

the question is how to do the final trick of telling git how to use Core.pm
(and which of the two?) and do the connexion between git and alien-svn.

Sorry for the noob questions, never used perl or git before...

Johann

> Regards,
> Christian.

p.s. is it possible not to open quote e-mail addresses in mailing lists?

^ permalink raw reply

* Re: [PATCH/RFC 1/2] is_url: Remove redundant assignment
From: Junio C Hamano @ 2011-09-26 16:52 UTC (permalink / raw)
  To: Tay Ray Chuan
  Cc: Pang Yan Han, git, Shawn O. Pearce, Jeff King, Sitaram Chamarty,
	Johannes Schindelin
In-Reply-To: <CALUzUxp-++A7azzimE1mjvPVoGxRM4rzYt9z66CmFb0B3vi7dQ@mail.gmail.com>

Tay Ray Chuan <rctay89@gmail.com> writes:

> On Sun, Sep 25, 2011 at 1:06 PM, Pang Yan Han <pangyanhan@gmail.com> wrote:
>> Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
>> ---
>>  url.c |    1 -
>>  1 files changed, 0 insertions(+), 1 deletions(-)
>>
>> diff --git a/url.c b/url.c
>> index 3e06fd3..d2e17e6 100644
>> --- a/url.c
>> +++ b/url.c
>> @@ -22,7 +22,6 @@ int is_url(const char *url)
>>
>>        if (!url)
>>                return 0;
>> -       url2 = url;
>>        first_slash = strchr(url, '/');
>>
>>        /* Input with no slash at all or slash first can't be URL. */
>
> Looks correct. Perhaps you could mention in the patch message that
>
>   There are no operations on url2 until another assignment to it later
> at line 41.

The looks correct, so I'll queue it, but it looks like that the function
is implemented in an overly complicated way.

Why aren't we checking from left to right in a single pass, perhaps like
this?

	/* Make sure it is of form "scheme://something" */
	int is_url(const char *url)
	{
		/* Is "scheme" part reasonable? */
		if (!url || !is_urlschemechar(1, *url++))
	        	return 0;
		while (*url && *url != ':') {
			if (!is_urlschemechar(0, *url++))
				return 0;
		}
		/* We've seen "scheme"; we want colon-slash-slash */
		return (url[0] == ':' && url[1] == '/' && url[2] == '/');
	}

^ permalink raw reply

* Re: [PATCH] send-email: auth plain/login fix
From: Joe Perches @ 2011-09-26 16:59 UTC (permalink / raw)
  To: Junio C Hamano, Graham Barr; +Cc: Zbigniew Jędrzejewski-Szmek, git
In-Reply-To: <7v8vpb5kea.fsf@alter.siamese.dyndns.org>

On Mon, 2011-09-26 at 09:38 -0700, Junio C Hamano wrote:
> Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> writes:
> 
> > git send-email was not authenticating properly when communicating over
> > TLS with a server supporting only AUTH PLAIN and AUTH LOGIN. This is
> > e.g. the standard server setup under debian with exim4 and probably
> > everywhere where system accounts are used.
> 
> Now that's a solution that makes me feel less dirty than
> 
>   http://thread.gmane.org/gmane.comp.version-control.git/178818/focus=178824

Dirty is dirty.  I don't see this as better or worse really,
but thanks Zbigniew for bringing it up again.

It'd be best if Graham Barr, the module author (cc'd),
could respond to the issue.

cheers, Joe

> even though, by forcing Authen::SASL::Perl to be used bypassing XS and
> Cyrus variants, this _might_ be introducing regression for others. We'll
> find out soon enough if anybody screams ;-)
> 
> I see you already researched previous discussion and have Joe Perches in
> the loop; Thanks.

^ permalink raw reply

* emacs as mergetool
From: Thomas Koch @ 2011-09-26 16:21 UTC (permalink / raw)
  To: git

Hi,

would you be so kind and give an emacs newbie (and vim refugee) a hint about 
using emacs as mergetool?

- Which one? ediff vs. emerge (I don't bother learning, just give me the best)
- I'd like to have a setup, where I can run mergetool with different options 
to
  - connect to a running emacs server
  - run emacs in X
  - or in the terminal (default)
- Is it possible to resolve all conflicting files in one session instead of 
having emacs called again for every file?
- Do you know a tutorial on emacs as a mergetool?

Thank you,

Thomas Koch, http://www.koch.ro

^ permalink raw reply

* Re: subversion-perl missing
From: Jakub Narebski @ 2011-09-26 17:17 UTC (permalink / raw)
  To: Georg-Johann Lay; +Cc: Christian Couder, git
In-Reply-To: <4E80AD1B.3080700@gjlay.de>

Georg-Johann Lay writes:
> Christian Couder schrieb:
> > On Mon, Sep 26, 2011 at 1:48 PM, Georg-Johann Lay <spam@spam.com> wrote:

> > > Where do I have to go to get the svn+perl stuff? Couldn't locate anything like
> > > that in subversion.tigris.org, so that I am stuck and need some hints on how to
> > > proceed.
> > 
> > Perhaps you can try to install this module from CPAN:
> > 
> > http://search.cpan.org/~mschwern/Alien-SVN-v1.6.12.1/
> > 
> > or from it's github repo:
> > 
> > https://github.com/schwern/alien-svn
> 
> Hi Christian,
> 
> thanks for that hint.
> 
> After getting that piece of software, running ./Build.PL and ./Build I just
> don't know what to do next.  The README doesn't say anything about how to use
> the built modules or how to do an install.

If you don't have admin privileges on the box, the best solution would
be to use local::lib[1] to install Perl modules in your home directory
for you (#perl IRC channel on FreeNode, or perlmonks are really helpful).

I really recommend it, together with cpanm[2] (which can configure
local::lib for you, via --local-lib option), though the latter is not
necessary.

[1]: http://search.cpan.org/~apeiron/local-lib-1.008004/lib/local/lib.pm
[2]: http://search.cpan.org/~miyagawa/App-cpanminus-1.4008/bin/cpanm


If you have admin privileges, I think the best solution would be
either find repository which has subversion-perl that matches your
version of Subversion, or compile subversion-perl from a source
package... though this might be more difficult, it has advantage of
being in package management.
 
> As git-svn complains
> 
> ...Can't locate SVN/Core.pm in @INC (@INC contains: ...
> 
> and
> 
>  schwern-alien-svn-9298884>find . -name 'Core.pm'
> ./src/subversion/subversion/bindings/swig/perl/native/Core.pm
> ./src/subversion/subversion/bindings/swig/perl/native/blib/lib/SVN/Core.pm
> 
> the question is how to do the final trick of telling git how to use Core.pm
> (and which of the two?) and do the connexion between git and alien-svn.
> 
> Sorry for the noob questions, never used perl or git before...

You have to install subversion-perl somewhere, not only build it.
Though you could fiddle with PERL5LIB (which is PATH-like environmtne
variable to specify where to search for Perl modules) to make Perl
find your built but not installed Alien::SVN.

HTH
-- 
Jakub Narębski

^ permalink raw reply

* Re: [PATCH] send-email: auth plain/login fix
From: Jakub Narebski @ 2011-09-26 17:24 UTC (permalink / raw)
  To: Zbigniew Jędrzejewski-Szmek; +Cc: gitster, joe, git
In-Reply-To: <1316879367-1182-1-git-send-email-zbyszek@in.waw.pl>

Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> writes:

> git send-email was not authenticating properly when communicating over
> TLS with a server supporting only AUTH PLAIN and AUTH LOGIN. This is
> e.g. the standard server setup under debian with exim4 and probably
> everywhere where system accounts are used.
> 
> The solution comes from this forum thread:
> http://www.perlmonks.org/?node_id=904354

Signoff?

> ---
>  git-send-email.perl |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git git-send-email.perl git-send-email.perl
> index 37dfbe7..100fbd9 100755
> --- git-send-email.perl
> +++ git-send-email.perl
> @@ -27,6 +27,7 @@ use Term::ANSIColor;
>  use File::Temp qw/ tempdir tempfile /;
>  use File::Spec::Functions qw(catfile);
>  use Error qw(:try);
> +use Authen::SASL qw(Perl);
>  use Git;
>  use MIME::Base64;

Shouldn't we load Authen::SASL only when it is necessary (on demand),
rather than forcing everybody who use send-email (even if via
sendmail, or with other authentication support)?

We load Net::SMTP::SSL only on demand.  

It would be:

   require Authen::SASL;
   Authen::SASL->import(qw(Perl));

-- 
Jakub Narębski

^ permalink raw reply

* Re: emacs as mergetool
From: Rémi Vanicat @ 2011-09-26 17:28 UTC (permalink / raw)
  To: git
In-Reply-To: <201109261821.28382.thomas@koch.ro>

Thomas Koch <thomas@koch.ro> writes:

> Hi,
>
> would you be so kind and give an emacs newbie (and vim refugee) a hint about 
> using emacs as mergetool?

In fact, I use magit like that: after the merge, if there was a
conflict, I launch magit (M-x magit) and go on the unmerged file, then I
use "e" (magit-ediff) to merge them. ediff merge tool is self
documented, and I never needed more than its documentation.

Of course for this you need to install magit, and you don't really use
Emacs as mergetool...

-- 
Rémi Vanicat

^ permalink raw reply

* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-26 17:47 UTC (permalink / raw)
  To: Christian Couder; +Cc: git, Christian Couder
In-Reply-To: <CAP8UFD3TWQHU0wLPuxMDnc3bRSz90Yd+yDMBe03kofeo-nr7yA@mail.gmail.com>

On Monday, September 26, 2011 06:41:04 am Christian Couder 
wrote:
> On Sun, Sep 25, 2011 at 10:43 PM, Martin Fick 
<mfick@codeaurora.org> wrote:
> > A coworker of mine pointed out to me that a simple
> > 
> >  git checkout
> > 
> > can also take rather long periods of time > 3 mins when
> > run on a repo with ~100K refs.
> 
> Are all these refs packed?

I think so, is there a way to find out for sure?

-Martin

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox