Re: Any chance for a Git v2.1.5 release?

Re: Any chance for a Git v2.1.5 release?

[Kyle J. McKay]

On Feb 24, 2015, at 11:52, Junio C Hamano wrote:

> Kyle J. McKay <mackyle@gmail.com> writes:
>
>> Which brings us back to the subject of this email, is there any  
>> chance
>> for a v2.1.5 release?
>> ...
>> It appears that the average support lifespan of a Git release from
>> initial release date through last released maintenance update is
>> approximately 2-3 months with the 1.7.6 release being an exception at
>> a little over 7 months.
>
> That matches my expectation.
>
> A typical cycle lasts for 8-12 weeks, and during that time, topics
> that are bugfixes that have graduated to the 'master' branch are
> merged to the 'maint' branch with some lag and then the tip of
> 'maint' gets tagged as a maintenance release from time to time.
> Some important but trivial fixes are further merged to older
> maitenance tracks like 'maint-2.2', 'maint-2.1', etc.
>
> But these topics downmerged to older maint-* branches have to be
> very trivial for an obvious reason: there are only 24 hours a day
> and 7 days in a week, and bugs that affect real world use cases are
> found by using the software in real world use cases.  Usually I use
> something a bit ahead of 'next' exactly for this reason---we would
> want to catch bugs before topics are merged to 'master'.  Although I
> sometimes have "let's use 'maint' for my work" day once or twice
> every month, I cannot afford to do that for anything older than the
> tip of 'maint' myself.

Obviously there would have to actually be some interest in having an  
older long-term-support release and some folks willing to exercise  
such a thing.  Unless we can figure out a way to clone you. ;) ;)

> The consequence of the above is this.  v2.1.1 may be more stable
> than v2.1.0 and v2.1.2 may be more stable than v2.1.1, but later
> tagged versions on older maintenance tracks are made by merging
> topics only after "ah, these are obvious enough" eyeballing without
> real use (at least by me), once newer feature release is made and
> there is a newer maintenance track.  I would not be surprised if
> v2.1.5, if it is made, has hidden interactions between the changes
> since v2.1.4 and the older codebase to cause unforeseen bugs.
>
> When I say "the tip of 'master' is meant to be more stable than any
> tagged versions", I do mean it.

Some fixes would likely not be back portable (e.g. to fix X you first  
need change Y which needs change Z which needs ...), not without  
ending up pulling in things that exceed the scope of a "maintenance"  
update.

> Having said all that, if I were to tag maint-2.1 branch as 2.1.5
> today, we would have
>
>    6aaf956 is_hfs_dotgit: loosen over-eager match of \u{..47}
>
> that does not exist in 2.1.4.  Is that what you want?

I suppose in that it fixes "false positives" it is a regression fix,  
but if that's all that showed up in v2.1.5, no, that wouldn't make it  
worthwhile.

>> If a v2.1.5 release is out of the question, would it be possible to
>> periodically designate certain Git releases as "long term support"
>> releases?
>
> I can designate ;-), but I do not think I'd be the right person to
> maintain or long-term-support it.  Are you volunteering to oversee
> the "LTS team"?

I could not promise a team of more than one member.  And that would  
not be full-time 24/7 either.

> It would involve:
>
>    - Monitor "git log --first-parent maint-lts..master" and find
>      the tip of topic branches that need to be down-merged;
>
>    - Down-merge such topics to maint-lts; this might involve
>      cherry-picking instead of merge, as the bugfix topics may
>      originally be done on the codebase newer than maint-lts;

I've been cherry-picking fixes for a while now onto older releases.  I  
don't suppose down-merging would be that much more difficult with a  
fallback to cherry-picking.

>    - Use the tip of the maint-lts branch in everyday work.
>
> The last item is the most important of the above, because I do not
> have time for that.  I can help with the first two to some degree,
> though.

That's pretty much all I use at this point -- a slightly older release  
with cherry-picked fixes.  While it did cause me to find the problem  
with the first version of the loose alternates fix, having only one  
person use such a release doesn't provide that much coverage.

> If the lts releases need to be tagged and published by me, then lts
> team can have me pull from the tip of maint-lts they are confident
> with and have me sign it and push it out.

It occurs to me that if the "maint-lts" updates were limited to crash  
fixes, regressions and security issues then often the pre-built man  
pages and docs from the release it's based on could be used as-is with  
the exception of the new release notes which might save some time.

> If the primary concern you have with the currently maintained
> releases is git-svn, perhaps a better way forward for you,
> especially if you are willing to maintain your own "release plus
> patches",

That was the discussion the admins had that we prefer to avoid rolling  
our own, but if there's no other option then we will do that.  Some  
other Git hosters obviously do not have the same reticence about  
rolling their own Git release (as can be seen from the capabilities  
returned).  There are several fixes not yet in a release that are  
needed besides just git-svn, but since the other fixes are all in  
master now they'll be out in the next release.  But, picking up that  
release to get those fixes also exposes us to new code that has not  
yet had widespread use and a "maint-lts" would hopefully pick up the  
fixes without doing that.

> might be to keep a patch that reverts recent git-svn
> changes back to v2.1.whatever and deploy the up-to-date release
> and that patch on top?

That has occurred to me assuming there's not some dependency issue  
there that causes a cascade of reversions.

>> Nice to have: The CVE-2014-9390 fix, but repo.or.cz does not create
>> any working trees so it's not mandatory.
>
> Even if you do not have a working tree, people who use you as the
> meeting place and push into you and fetch from you do, so I do not
> necessarily agree with your assessment.

Having the feature does no good unless it's turned on.  :)  How  
expensive would it be to turn such a feature on in terms of CPU  
usage?  Running `git fsck` is rather expensive and the name of the  
config option suggests it's related, is it also as expensive?  When  
someone pushes a 2GB pack, what impact does that option have on server  
load?  All existing repos would have to be checked first.  Do we eject  
any that fail?  What about mirrors?  Do we refuse to mirror from a  
source that fails the check?  As long as it's not too CPU expensive, I  
could see turning it on immediately for new push repos and after  
checking, existing push and mirror repos that pass.  Hopefully all of  
them just pass the check.  So we cannot just deploy a CVE-2014-9390  
fix even if we had it without first doing some work assuming the CPU  
load is not an issue.  But it would be nice to have so we *could*  
proceed in that direction.

-Kyle

Re: Any chance for a Git v2.1.5 release?

[Junio C Hamano]

"Kyle J. McKay" <mackyle@gmail.com> writes:

>> I can designate ;-), but I do not think I'd be the right person to
>> maintain or long-term-support it.  Are you volunteering to oversee
>> the "LTS team"?
>
> I could not promise a team of more than one member.  And that would
> not be full-time 24/7 either.

Heh. Making noises to find like-minded people would be the first
step to build a viable team, and hopefully you are already doing a
good job here ;-)

>> It would involve:
>>
>>    - Monitor "git log --first-parent maint-lts..master" and find
>>      the tip of topic branches that need to be down-merged;
>>
>>    - Down-merge such topics to maint-lts; this might involve
>>      cherry-picking instead of merge, as the bugfix topics may
>>      originally be done on the codebase newer than maint-lts;
>
> I've been cherry-picking fixes for a while now onto older releases.  I
> don't suppose down-merging would be that much more difficult with a
> fallback to cherry-picking.
>
>>    - Use the tip of the maint-lts branch in everyday work.
>>
>> The last item is the most important of the above, because I do not
>> have time for that.  I can help with the first two to some degree,
>> though.
>
> That's pretty much all I use at this point -- a slightly older release
> with cherry-picked fixes.  While it did cause me to find the problem
> with the first version of the loose alternates fix, having only one
> person use such a release doesn't provide that much coverage.

That is why I used the word "team".

> It occurs to me that if the "maint-lts" updates were limited to crash
> fixes, regressions and security issues then often the pre-built man
> pages and docs from the release it's based on could be used as-is with
> the exception of the new release notes which might save some time.

Cutting release tarballs including the pre-formatting docs might
consume a lot of machine time, but it does not cost me time at all.
I have Makefile for that ;-)

Judging which fixes that have proven themselves to be safe and sound
(by being in 'next', 'master' and hopefully 'maint' for some time)
are worthy enough of down-merging to the LTS track is something I'd
want to farm out to the LTS team.  I am already doing the "safe and
sound" part by deciding which topics to merge to 'maint' among the
topics that have gone through 'pu' to 'next' to 'master' branches,
but not all that are worthy enough to be merged to 'maint' may be
important enough to bother downmerging and updating LTS track with,
and picking which ones matter to the LTS users is what the folks who
are interested in the LTS can help.

Re: Any chance for a Git v2.1.5 release?

[Kyle J. McKay]

On Feb 24, 2015, at 21:13, Junio C Hamano wrote:

> "Kyle J. McKay" <mackyle@gmail.com> writes:
>
>>> I can designate ;-), but I do not think I'd be the right person to
>>> maintain or long-term-support it.  Are you volunteering to oversee
>>> the "LTS team"?
>>
>> I could not promise a team of more than one member.  And that would
>> not be full-time 24/7 either.
>
> Heh. Making noises to find like-minded people would be the first
> step to build a viable team, and hopefully you are already doing a
> good job here ;-)

Doesn't seem to be catching much interest though.  Maybe they're all  
just watching silently peering through the blinds waiting to see how  
it turns out.  ;)

>>> It would involve:
>>>
>>>   - Monitor "git log --first-parent maint-lts..master" and find
>>>     the tip of topic branches that need to be down-merged;
>>>
>>>   - Down-merge such topics to maint-lts; this might involve
>>>     cherry-picking instead of merge, as the bugfix topics may
>>>     originally be done on the codebase newer than maint-lts;
>>
>> I've been cherry-picking fixes for a while now onto older  
>> releases.  I
>> don't suppose down-merging would be that much more difficult with a
>> fallback to cherry-picking.
>>
>>>   - Use the tip of the maint-lts branch in everyday work.
>>>
>>> The last item is the most important of the above, because I do not
>>> have time for that.  I can help with the first two to some degree,
>>> though.
>>
>> That's pretty much all I use at this point -- a slightly older  
>> release
>> with cherry-picked fixes.  While it did cause me to find the problem
>> with the first version of the loose alternates fix, having only one
>> person use such a release doesn't provide that much coverage.
>
> That is why I used the word "team".
>
>> It occurs to me that if the "maint-lts" updates were limited to crash
>> fixes, regressions and security issues then often the pre-built man
>> pages and docs from the release it's based on could be used as-is  
>> with
>> the exception of the new release notes which might save some time.
>
> Cutting release tarballs including the pre-formatting docs might
> consume a lot of machine time, but it does not cost me time at all.
> I have Makefile for that ;-)
>
> Judging which fixes that have proven themselves to be safe and sound
> (by being in 'next', 'master' and hopefully 'maint' for some time)
> are worthy enough of down-merging to the LTS track is something I'd
> want to farm out to the LTS team.  I am already doing the "safe and
> sound" part by deciding which topics to merge to 'maint' among the
> topics that have gone through 'pu' to 'next' to 'master' branches,
> but not all that are worthy enough to be merged to 'maint' may be
> important enough to bother downmerging and updating LTS track with,
> and picking which ones matter to the LTS users is what the folks who
> are interested in the LTS can help.

If I were to keep a maint-lts up-to-date somewhere (with suitable down- 
merges matching the manner in which you maintain your tree) that you  
could pull from and potentially make releases from.  I would not want  
to pick up anything that hadn't already made it into master or maint  
and I don't think any actual release based off maint-lts should have  
any fix that has not already appeared in another non-maint-lts  
release.  So any given maint-lts release date would be expected to lag  
behind the corresponding master/maint release date that contained the  
same fixes (except possibly for a coordinated security fix release).

That said, there's no reason I couldn't also keep a pu-lts up-to-date  
somewhere external to your tree that interested parties could grab  
that would include fixes making their way into maint/master that I  
believe would be candidates for inclusion in maint-lts once they  
graduated.

I would like to better understand how the various heads are  
maintained.  I've read MaintNotes and I've got the concepts, but I'm  
still a little fuzzy on some details.  It looks to me like all topics  
still only in pu after master has been updated are then rebased onto  
the new master and then pu is rebuilt.  MaintNotes doesn't get into  
the rebasing details.  None of the graphic log viewers I've tried are  
much help -- too many lines running around although the graphiclog on  
repo.or.cz helps a bit as it shows --first-parent links as a bolder  
line, but still after the first page that's not much help either.

I vaguely recall you may have explained some of this in more detail in  
the context of explaining how you used the scripts in todo to put  
everything together when someone asked a question about it on the  
list.  But I can't seem to find that message anymore.  :(

I think I mostly understand how master, next and pu are maintained.   
But MaintNotes says "Whenever a feature release is made, 'maint'  
branch is forked off from 'master' at that point."  What happens to  
the previous maint at that time?  Is it just renamed to maint-X.X?

Also, how do you handle a down merge to maint when you have this:

* (master)
* merge topic foo
|\
| * topic foo
|/
* c
* b
* a
* (tag: vX.X.X, maint)
*

And you want to down merge topic foo into maint (where foo is a topic  
that didn't even first enter pu until some time after the vX.X.X  
release).  Won't doing 'checkout maint', 'merge ":/^topic foo"' pull  
in commits a, b and c as well?  I can think of several things to do  
here, but how do you normally handle this situation?

-Kyle

Re: Any chance for a Git v2.1.5 release?

[Junio C Hamano]

"Kyle J. McKay" <mackyle@gmail.com> writes:

> I would like to better understand how the various heads are
> maintained.  I've read MaintNotes and I've got the concepts, but I'm
> still a little fuzzy on some details.  It looks to me like all topics
> still only in pu after master has been updated are then rebased onto
> the new master and then pu is rebuilt.

Topics in 'pu' not yet in 'next' _can_ be rebased, but unless there
is a good reason to do so, I wouldn't spend extra cycles necessary
to do such thing.  I even try to keep the same base when replacing
the contents of a branch with a rerolled series.  For example, today
I have this:

    $ git config alias.lgf
    log --oneline --boundary --first-parent
    $ git lgf master..nd/slim-index-pack-memory-usage
    3538291 index-pack: kill union delta_base to save memory
    7b4ff41 FIXUP
    45b6b71 index-pack: reduce memory footprint a bit
    - 9874fca Git 2.3

and Duy has a newer iteration of it starting at $gmane/264429.  What
I would do, after saving these patches in a mbox +nd-index-pack,
would be to

    $ git checkout 9874fca
    $ git am -s3c ./+nd-index-pack
    $ git show-branch HEAD nd/slim-index-pack-memory-usage
    ... compare corresponding changes between the old and the new
    ... until I am satisified; otherwise I may tweak the new one
    $ git rebase -i 9874fca
    ... and then finally
    $ git branch -f nd/slim-index-pack-memory-usage HEAD

to update the topic.  Of course, it would be necessary to rebuild
'pu' by merging all the topics that are in it on top of 'master'.

https://github.com/gitster/git.git has these topic branches broken
out and you can observe how they change over time from your local
reflog for refs/remotes/<that repository>/<topic branches>.

> I vaguely recall you may have explained some of this in more detail in
> the context of explaining how you used the scripts in todo to put
> everything together when someone asked a question about it on the
> list.  But I can't seem to find that message anymore.  :(

There may be a copy in Documentaiton/howto/ somewhere.

> I think I mostly understand how master, next and pu are maintained.
> But MaintNotes says "Whenever a feature release is made, 'maint'
> branch is forked off from 'master' at that point."  What happens to
> the previous maint at that time?  Is it just renamed to maint-X.X?

After finishing 2.3.0 release, at some point while 'master' is still
at 2.3.0, something like this would happen:

    $ git branch -m maint maint-2.2
    $ git branch maint master

> Also, how do you handle a down merge to maint when you have this:
>
> * (master)
> * merge topic foo
> |\
> | * topic foo
> |/
> * c
> * b
> * a
> * (tag: vX.X.X, maint)
> *

I don't, and this is done by making sure I do not get into such a
situation in the first place.

A general rule of thumb when applying a set of patches that are
fixes eventually worth having in older maintenance tracks is to find
the oldest maintenance branch and apply them there.

Re: Any chance for a Git v2.1.5 release?

[Kyle J. McKay]

On Feb 26, 2015, at 12:54, Junio C Hamano wrote:

> "Kyle J. McKay" <mackyle@gmail.com> writes:
>
>> I would like to better understand how the various heads are
>> maintained.  I've read MaintNotes and I've got the concepts, but I'm
>> still a little fuzzy on some details.  It looks to me like all topics
>> still only in pu after master has been updated are then rebased onto
>> the new master and then pu is rebuilt.
>
> Topics in 'pu' not yet in 'next' _can_ be rebased, but unless there
> is a good reason to do so, I wouldn't spend extra cycles necessary
> to do such thing.  I even try to keep the same base when replacing
> the contents of a branch with a rerolled series.  For example, today
> I have this:
>
>    $ git config alias.lgf
>    log --oneline --boundary --first-parent
>    $ git lgf master..nd/slim-index-pack-memory-usage
>    3538291 index-pack: kill union delta_base to save memory
>    7b4ff41 FIXUP
>    45b6b71 index-pack: reduce memory footprint a bit
>    - 9874fca Git 2.3
>
> and Duy has a newer iteration of it starting at $gmane/264429.  What
> I would do, after saving these patches in a mbox +nd-index-pack,
> would be to
>
>    $ git checkout 9874fca
>    $ git am -s3c ./+nd-index-pack
>    $ git show-branch HEAD nd/slim-index-pack-memory-usage
>    ... compare corresponding changes between the old and the new
>    ... until I am satisified; otherwise I may tweak the new one
>    $ git rebase -i 9874fca
>    ... and then finally
>    $ git branch -f nd/slim-index-pack-memory-usage HEAD
>
> to update the topic.  Of course, it would be necessary to rebuild
> 'pu' by merging all the topics that are in it on top of 'master'.

Thanks.  That's helpful.

> After finishing 2.3.0 release, at some point while 'master' is still
> at 2.3.0, something like this would happen:
>
>    $ git branch -m maint maint-2.2
>    $ git branch maint master

So the reason I don't notice force-updates to maint when this happens  
is because of the "Sync with maint" commits that make sure the new  
maint contains the old one.

>> Also, how do you handle a down merge to maint when you have this:
>>
>> * (master)
>> * merge topic foo
>> |\
>> | * topic foo
>> |/
>> * c
>> * b
>> * a
>> * (tag: vX.X.X, maint)
>> *
>
> I don't, and this is done by making sure I do not get into such a
> situation in the first place.
>
> A general rule of thumb when applying a set of patches that are
> fixes eventually worth having in older maintenance tracks is to find
> the oldest maintenance branch and apply them there.

If I were to keep a maint-lts branch somewhere I would need to cherry- 
pick topics with desired fixes that fall into that situation.  That's  
what I expected but when you mentioned down merging the fixes I wanted  
to make sure I wasn't overlooking something.

I'll see about setting up a maint-lts in a local git repository clone  
and tracking LTS fixes.  If I'm able to keep that going without it  
becoming a black-hole of temporal need that sucks the life right out  
of me  ;)  then perhaps we can have a discussion at a future date  
about what would be needed for you to consider pulling from it and  
issuing LTS releases off it.  :)

-Kyle

Re: Any chance for a Git v2.1.5 release?

[Junio C Hamano]

"Kyle J. McKay" <mackyle@gmail.com> writes:

>> After finishing 2.3.0 release, at some point while 'master' is still
>> at 2.3.0, something like this would happen:
>>
>>    $ git branch -m maint maint-2.2
>>    $ git branch maint master
>
> So the reason I don't notice force-updates to maint when this happens
> is because of the "Sync with maint" commits that make sure the new
> maint contains the old one.

I could also do

    git branch maint-2.2 maint
    git push . master:maint ;# not +master:maint

to make sure that I won't rewind 'maint', but this works because
'master' is designed to be always a super-set of 'maint'.

> If I were to keep a maint-lts branch somewhere I would need to cherry- 
> pick topics with desired fixes that fall into that situation.  That's
> what I expected but when you mentioned down merging the fixes I wanted
> to make sure I wasn't overlooking something.

Yup.  I _can_ become sloppy especially late in the release cycle and
end up doing something silly like this:

    - Fork km/svn-fix from somewhere it _could_ be merged to 'maint'
      (e.g. "the last big release", e.g. v2.3.0).

    - Merge km/svn-fix to 'master' early in a cycle;

    - A mistake is found in the topic late in the cycle; as the next
      release will happen soon _anyway_, and I do not happen to
      consider km/svn-fix is so important a fix to be merged to
      'maint', I apply a fix-up patch directly on top of 'master';

    - A release candidate is tagged from 'master'.

Then even though you can easily grab a broken-out tree at
github.com:gitster/git, km/svn-fix topic alone cannot be merged to
'maint' as it would lack the late fix.  I've been trying to be
careful but I cannot promise to be perfect X-<.

Re: Any chance for a Git v2.1.5 release?

[Philip Oakley]

From: "Kyle J. McKay" <mackyle@gmail.com>
> On Feb 26, 2015, at 12:54, Junio C Hamano wrote:
>
>> "Kyle J. McKay" <mackyle@gmail.com> writes:
>>
>>> I would like to better understand how the various heads are
>>> maintained.  I've read MaintNotes and I've got the concepts, but I'm
>>> still a little fuzzy on some details.  It looks to me like all 
>>> topics
>>> still only in pu after master has been updated are then rebased onto
>>> the new master and then pu is rebuilt.
>>
>> Topics in 'pu' not yet in 'next' _can_ be rebased, but unless there
>> is a good reason to do so, I wouldn't spend extra cycles necessary
>> to do such thing.  I even try to keep the same base when replacing
>> the contents of a branch with a rerolled series.  For example, today
>> I have this:
>>
>>    $ git config alias.lgf
>>    log --oneline --boundary --first-parent
>>    $ git lgf master..nd/slim-index-pack-memory-usage
>>    3538291 index-pack: kill union delta_base to save memory
>>    7b4ff41 FIXUP
>>    45b6b71 index-pack: reduce memory footprint a bit
>>    - 9874fca Git 2.3
>>
>> and Duy has a newer iteration of it starting at $gmane/264429.  What
>> I would do, after saving these patches in a mbox +nd-index-pack,
>> would be to
>>
>>    $ git checkout 9874fca
>>    $ git am -s3c ./+nd-index-pack
>>    $ git show-branch HEAD nd/slim-index-pack-memory-usage
>>    ... compare corresponding changes between the old and the new
>>    ... until I am satisified; otherwise I may tweak the new one
>>    $ git rebase -i 9874fca
>>    ... and then finally
>>    $ git branch -f nd/slim-index-pack-memory-usage HEAD
>>
>> to update the topic.  Of course, it would be necessary to rebuild
>> 'pu' by merging all the topics that are in it on top of 'master'.
>
> Thanks.  That's helpful.
>
>> After finishing 2.3.0 release, at some point while 'master' is still
>> at 2.3.0, something like this would happen:
>>
>>    $ git branch -m maint maint-2.2
>>    $ git branch maint master
>
> So the reason I don't notice force-updates to maint when this happens 
> is because of the "Sync with maint" commits that make sure the new 
> maint contains the old one.
>
>>> Also, how do you handle a down merge to maint when you have this:
>>>
>>> * (master)
>>> * merge topic foo
>>> |\
>>> | * topic foo
>>> |/
>>> * c
>>> * b
>>> * a
>>> * (tag: vX.X.X, maint)
>>> *
>>
>> I don't, and this is done by making sure I do not get into such a
>> situation in the first place.
>>
>> A general rule of thumb when applying a set of patches that are
>> fixes eventually worth having in older maintenance tracks is to find
>> the oldest maintenance branch and apply them there.
>
> If I were to keep a maint-lts branch somewhere I would need to cherry- 
> pick topics with desired fixes that fall into that situation.  That's 
> what I expected but when you mentioned down merging the fixes I wanted 
> to make sure I wasn't overlooking something.

<minor bikeshed>
The announcement a few days ago by Adam Spiers ($gmane/264405) of a new 
video of his git-deps tool ($gmane/262606) may be of interest for 
determining the dependency charts. It looked interesting for this type 
of issue.

His original blog is http://blog.adamspiers.org/2015/01/19/git-deps/ 
with repo at https://github.com/aspiers/git-deps
<\minor bikeshed>
>
> I'll see about setting up a maint-lts in a local git repository clone 
> and tracking LTS fixes.  If I'm able to keep that going without it 
> becoming a black-hole of temporal need that sucks the life right out 
> of me  ;)  then perhaps we can have a discussion at a future date 
> about what would be needed for you to consider pulling from it and 
> issuing LTS releases off it.  :)
>
> -Kyle
> --
Philip 

Re: Any chance for a Git v2.1.5 release?

[Kyle J. McKay]

The first two messages in this thread were dropped by the vger mailing  
list for some unknown reason.

In an attempt to make the mailing list archives of this thread  
complete, the original two messages in this thread are included below.

-Kyle

---- BEGIN FIRST MESSAGE ----
From: Kyle J. McKay <mackyle@gmail.com>
Date: February 24, 2015 09:16:05 PST
To: Junio C Hamano <gitster@pobox.com>
Cc: Git Mailing List <git@vger.kernel.org>
Subject: Any chance for a Git v2.1.5 release?
Message-Id: <C5211E53-8905-41C9-9D28-26D7BB51E76A@gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes

As you know, I help out with repo.or.cz.  Recently the admins have  
been discussing upgrading the version of Git we use in order to  
support newer features such as pack bitmaps.  While we do use a  
slightly customized gitweb, we have always stuck to an official Git  
release for everything else.

Repo.or.cz provides fetch (git, http, https, ssh), browsing (gitweb)  
and push (ssh, https).  Additionally, created repositories can be  
mirrors (no push allowed) of other repositories (including svn via git- 
svn).  Email notification of ref changes (along with diffs) can also  
be requested.

We are finding that in order to upgrade Git at this point we would  
need to build a custom Git with cherry picked fixes for various issues  
that have come up or they would likely be triggered by one of the  
services repo.or.cz provides.  In addition, as there are numerous  
reports of an unresolved issue with git-svn starting with v2.2.0  
upgrading to v2.2.0 or later presents a problem since we have several  
mirrors that rely on git-svn.

Which brings us back to the subject of this email, is there any chance  
for a v2.1.5 release?

After reviewing a few release dates:

2011-06-26T12:41:26-07:00 v1.7.6
2012-02-05T23:51:07-08:00 v1.7.6.6

2013-11-27T12:14:52-08:00 v1.8.5
2014-02-13T13:42:01-08:00 v1.8.5.5

2014-02-14T11:36:11-08:00 v1.9.0
2014-05-30T10:15:10-07:00 v1.9.4

2014-05-28T11:04:29-07:00 v2.0.0
2014-07-30T14:20:01-07:00 v2.0.4

2014-08-15T15:09:28-07:00 v2.1.0
2014-10-29T10:48:57-07:00 v2.1.3

2014-11-26T13:18:43-08:00 v2.2.0
2015-01-12T14:06:20-08:00 v2.2.2

2015-02-05T13:24:05-08:00 v2.3.0

(I have omitted the dates of the 5 security releases on 2014-12-17 as  
that seems like an extraordinary event unlikely to be repeated.)

It appears that the average support lifespan of a Git release from  
initial release date through last released maintenance update is  
approximately 2-3 months with the 1.7.6 release being an exception at  
a little over 7 months.

If a v2.1.5 release is out of the question, would it be possible to  
periodically designate certain Git releases as "long term support"  
releases?  Meaning that they would receive maintenance updates (e.g.  
fixes for invalid memory accesses/crashes, regressions or security  
issues) for an extended period of time, say at least 6 months?


Here's the analysis that led to this request:


Goal 1: add support for symref=HEAD:refs/... capability

Goal 2: add support for pack bitmaps

Nice to have: The CVE-2014-9390 fix, but repo.or.cz does not create  
any working trees so it's not mandatory.


Goal 1:

symref=HEAD:refs/... requires at least Git 1.8.4.3.  However,  
repo.or.cz runs git-daemon with read-only access to the repositories  
and since Git 1.8.4.2 shallow clones require write access.

This was corrected in v2.0.0.  So at least v2.0.0 would be needed for  
symref=HEAD:refs/....


Goal 2:

Pack bitmap support was added in v2.0.0, but it's probably not a good  
idea to run without 21134714 (pack-objects: turn off bitmaps when we  
split packs) just in case which requires at least v2.1.3.


Nice to have:

If at least v2.1.3 is required, then we might as well use v2.1.4 since  
the primary change from v2.1.3 to v2.1.4 is the addition of the  
CVE-2014-9390 fix.


What about a later release, v2.2.0 or later?


git-svn is reported to suffer from occasional ".git/ 
Git_svn_hash_XXXXXX: Bad file descriptor" errors since v2.2.0 making  
that a non-starter.  No fix is currently available in the Git  
repository.

Since 660c889e (sha1_file: add for_each iterators for loose and packed  
objects) loose objects in alternates directories may not be found when  
pruning.  This affects v2.2.0 and later.  A fix is currently in  
master.  This is an absolute must have for repo.or.cz as alternates  
are used to implement repository forks.

Since d38379e (make update-server-info more robust), affecting v2.2.x,  
the files used by non-smart HTTP clients could have the wrong  
permissions.  This might preclude them from being updated correctly on  
repo.or.cz.  It would require research to see if repo.or.cz is  
affected.  The fix for this d91175b2 (update-server-info: create info/ 
* with mode 0666) was released in v2.3.0.


So why not v2.1.4 then?


There's an XDL_FAST_HASH performance regression that affects v1.7.11  
and later [1].  But that can be turned off in config.mak.

New repositories have an extra +x on the config file since v2.1.0  
(fixed in v2.2.2), but we can compensate with a chmod after creating a  
new repository.

Git blame has a crashing regression since ea02ffa3 (mailmap: simplify  
map_user() interface) that affects v1.8.2 and later. The fix is  
currently in master.  repo.or.cz enables blame in the gitweb interface  
so that would be a serious regression.

There's an issue with ungetting EOF.  That would seem unlikely to  
affect repo.or.cz, but, I wonder if any of the displays used by gitweb  
could trigger attempts to read a .mailmap or .gitattributes blob?  I  
don't know.  Would require research to be certain or just picking up  
the fix which is currently in master.

There's a decimal_width overflow regression since dc801e7 (diff -- 
stat: use less columns for change counts) affecting v1.7.11 and later  
that can lead to an infinite loop in some cases.  Presumably either  
gitweb or the diffs sent out by the ref update email notification  
mechanism could trigger this.  A fix is currently in master.

There is a regression since v1.7.1.1 where a fetch from a non-smart  
HTTP source could potentially fail to retrieve the correct objects.   
As repo.or.cz has several mirrors set up from non-smart HTTP sources a  
fix for this (currently in master) would be nice but is probably not  
mandatory since repo.or.cz is currently running a version of Git with  
this regression and has not received any reports of failures that  
could be related to this.

Since fbd4a703 (list-objects: mark more commits as edges in  
mark_edges_uninteresting) affecting v1.8.4.2 and later combined with  
200abe74 (list-objects: only look at cmdline trees with edge_hint)  
affecting v1.9.0 and later there is a performance regression when  
pushing with lots of refs to non-shallow repos.  The fix 2dacf26d  
(pack-objects: use --objects-edge-aggressive for shallow repos) was  
released in v2.3.0.


-Kyle

[1] http://thread.gmane.org/gmane.comp.version-control.git/261638
---- END FIRST MESSAGE ----

---- BEGIN SECOND MESSAGE ----
From: Junio C Hamano <gitster@pobox.com>
Date: February 24, 2015 11:52:03 PST
To: Kyle J. McKay <mackyle@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>
Subject: Re: Any chance for a Git v2.1.5 release?
Message-Id: <xmqqk2z7qe8s.fsf@gitster.dls.corp.google.com>
Content-Type: text/plain

Kyle J. McKay <mackyle@gmail.com> writes:

> Which brings us back to the subject of this email, is there any chance
> for a v2.1.5 release?
> ...
> It appears that the average support lifespan of a Git release from
> initial release date through last released maintenance update is
> approximately 2-3 months with the 1.7.6 release being an exception at
> a little over 7 months.

That matches my expectation.

A typical cycle lasts for 8-12 weeks, and during that time, topics
that are bugfixes that have graduated to the 'master' branch are
merged to the 'maint' branch with some lag and then the tip of
'maint' gets tagged as a maintenance release from time to time.
Some important but trivial fixes are further merged to older
maitenance tracks like 'maint-2.2', 'maint-2.1', etc.

But these topics downmerged to older maint-* branches have to be
very trivial for an obvious reason: there are only 24 hours a day
and 7 days in a week, and bugs that affect real world use cases are
found by using the software in real world use cases.  Usually I use
something a bit ahead of 'next' exactly for this reason---we would
want to catch bugs before topics are merged to 'master'.  Although I
sometimes have "let's use 'maint' for my work" day once or twice
every month, I cannot afford to do that for anything older than the
tip of 'maint' myself.

The consequence of the above is this.  v2.1.1 may be more stable
than v2.1.0 and v2.1.2 may be more stable than v2.1.1, but later
tagged versions on older maintenance tracks are made by merging
topics only after "ah, these are obvious enough" eyeballing without
real use (at least by me), once newer feature release is made and
there is a newer maintenance track.  I would not be surprised if
v2.1.5, if it is made, has hidden interactions between the changes
since v2.1.4 and the older codebase to cause unforeseen bugs.

When I say "the tip of 'master' is meant to be more stable than any
tagged versions", I do mean it.

Having said all that, if I were to tag maint-2.1 branch as 2.1.5
today, we would have

   6aaf956 is_hfs_dotgit: loosen over-eager match of \u{..47}

that does not exist in 2.1.4.  Is that what you want?

> If a v2.1.5 release is out of the question, would it be possible to
> periodically designate certain Git releases as "long term support"
> releases?

I can designate ;-), but I do not think I'd be the right person to
maintain or long-term-support it.  Are you volunteering to oversee
the "LTS team"?  It would involve:

   - Monitor "git log --first-parent maint-lts..master" and find
     the tip of topic branches that need to be down-merged;

   - Down-merge such topics to maint-lts; this might involve
     cherry-picking instead of merge, as the bugfix topics may
     originally be done on the codebase newer than maint-lts;

   - Use the tip of the maint-lts branch in everyday work.

The last item is the most important of the above, because I do not
have time for that.  I can help with the first two to some degree,
though.

If the lts releases need to be tagged and published by me, then lts
team can have me pull from the tip of maint-lts they are confident
with and have me sign it and push it out.

If the primary concern you have with the currently maintained
releases is git-svn, perhaps a better way forward for you,
especially if you are willing to maintain your own "release plus
patches", might be to keep a patch that reverts recent git-svn
changes back to v2.1.whatever and deploy the up-to-date release
and that patch on top?

> Nice to have: The CVE-2014-9390 fix, but repo.or.cz does not create
> any working trees so it's not mandatory.

Even if you do not have a working tree, people who use you as the
meeting place and push into you and fetch from you do, so I do not
necessarily agree with your assessment.
---- END SECOND MESSAGE ----