how to determine oldest supported version of git

how to determine oldest supported version of git

[Neal Kreitzinger]

What is the best way for me (a git user) to determine what is currently the 
oldest supported version of git (the oldest version still getting bugfixes)? 
IOW, when can I tell that my version of git is no longer supported?

v/r,
neal 

Re: how to determine oldest supported version of git

[Jonathan Nieder]

Hi Neal,

Neal Kreitzinger wrote:

> What is the best way for me (a git user) to determine what is currently the 
> oldest supported version of git (the oldest version still getting bugfixes)? 
> IOW, when can I tell that my version of git is no longer supported?

It depends what supported means.  Even very old git releases might get
point updates to fix major problems such as security bugs.

If you want to see which branches Junio is actively maintaining,
looking at the last commit date from the maint-* branches on [1] is
one way.

However, in my experience people interested in product lifetimes more
often mean "versions the vendor will respond to bug reports about"
rather than "versions getting updates".  If you have discovered a bug
in an old version of git, even if it is only a couple of major
releases ago, a good debugging strategy is almost always to try with
the newest release and see if it still exhibits the bug.  If you don't
try that, people on this list might just try it themselves.  If it
doesn't affect recent releases, I would not be surprised if people on
this list do not necessarily care much.  One can more easily interest
me at least by pointing out which regression is making it hard to
upgrade instead.

Thanks,
Jonathan

[1] git://github.com/gitster/git.git

Re: how to determine oldest supported version of git

[Jeff King]

On Thu, Feb 02, 2012 at 01:23:40PM -0600, Jonathan Nieder wrote:

> However, in my experience people interested in product lifetimes more
> often mean "versions the vendor will respond to bug reports about"
> rather than "versions getting updates".  If you have discovered a bug
> in an old version of git, even if it is only a couple of major
> releases ago, a good debugging strategy is almost always to try with
> the newest release and see if it still exhibits the bug.  If you don't
> try that, people on this list might just try it themselves.  If it
> doesn't affect recent releases, I would not be surprised if people on
> this list do not necessarily care much.  One can more easily interest
> me at least by pointing out which regression is making it hard to
> upgrade instead.

Agreed. It is very annoying to have somebody report a bug, I (or another
dev) spends time trying to reproduce, and then we find out that it was
actually fixed a year ago.

However, I am much happier if a submitter does that leg-work themselves,
and posts to the list something like:

  I am using version a.b.c. It has bug $FOO, which was fixed by $COMMIT
  and released in d.e.f [or even "I tried d.e.f and it does not exhibit
  the bug"]. This bug fix should get cherry-picked back to a.b.c,
  because {it is more important than usual for reason X, upgrading past
  a.b.c is not feasible for reason Y, etc}.

Nobody wastes time tracking down the already-fixed bug, and it's
relatively easy to decide whether the cherry-pick is worth the effort
based on the reasoning given.

I know not everybody is capable of complex bisection or writing a
succinct test case. But they can at least try to reproduce with the
latest version and convert "there's a bug in git" to "there's a bug in
this old version of git".

-Peff

Re: how to determine oldest supported version of git

[Junio C Hamano]

"Neal Kreitzinger" <neal@rsss.com> writes:

> What is the best way for me (a git user) to determine what is currently
> the oldest supported version of git (the oldest version still getting
> bugfixes)?  IOW, when can I tell that my version of git is no longer
> supported?

"A note from the maintainer" only promises that the latest major release
(as of this writing, 1.7.9) gets regular maintenance releases until the
next major release happens.

When queuing a fix to an old bug, however, I try to build a topic branch
for that fix from as old an release as practical, in order to make sure
that older maintenance tracks could benefit, and I do give updates for
older maintenance tracks when able (but no promises).

For example, during the last cycle leading to 1.7.9, in other words, back
when 1.7.8 was the latest major release, in addition to the maintenance
releases 1.7.8.1, 1.7.8.2, 1.7.8.3 and 1.7.8.4, maintenance releases for
older version of Git were tagged (1.7.6.5, 1.7.7.5, and 1.7.7.6).  Note
that 1.7.6 was originally released on June 26th, 2011.

One cycle of major release development is expected to last between 8 to 10
weeks, so keeping two stale maintenance tracks in addition to the latest
maintenance track alive would roughly translate to 6 months shelf life for
an ancient release.

As other people mentioned, if you are on a (probably paid) support plan
from a(n enterprise) distro, asking them would be the best way, and if you
are running Git supplied as part of a distro, the distro would dictate the
version it supplies to you, so asking here would not help very much.

Re: how to determine oldest supported version of git

[Neal Kreitzinger]

On 2/2/2012 10:52 PM, Junio C Hamano wrote:
> As other people mentioned, if you are on a (probably paid) support plan
> from a(n enterprise) distro, asking them would be the best way, and if you
> are running Git supplied as part of a distro, the distro would dictate the
> version it supplies to you, so asking here would not help very much.
We compile our git from you (and your cohort).  (Our paid distro does 
not keep up.)  Your replies have been *very* helpful.

thanks!

v/r,
neal

Re: how to determine oldest supported version of git

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "Neal Kreitzinger" <neal@rsss.com> writes:
>
>> What is the best way for me (a git user) to determine what is currently
>> the oldest supported version of git (the oldest version still getting
>> bugfixes)?  IOW, when can I tell that my version of git is no longer
>> supported?
>
> "A note from the maintainer" only promises that the latest major release
> (as of this writing, 1.7.9) gets regular maintenance releases until the
> next major release happens.
>
> When queuing a fix to an old bug, however, I try to build a topic branch
> for that fix from as old an release as practical, in order to make sure
> that older maintenance tracks could benefit, and I do give updates for
> older maintenance tracks when able (but no promises).
> ...
> One cycle of major release development is expected to last between 8 to 10
> weeks, so keeping two stale maintenance tracks in addition to the latest
> maintenance track alive would roughly translate to 6 months shelf life for
> an ancient release.

Having said all that, I am starting to doubt what the point of all
this is.

Maybe I am being slow to come to this realization after having done this
for all these years, but Git is not like the Linux kernel, where hundreds
of companies maintain their own internal forks, e.g. out of tree drivers,
filesystem tweaks or scheduler tweaks, all tied to a specific version of
the internal API that is a rapidly moving target and cannot afford to
adjust to the bleeding edge.  Also our strict no regression policy means
that it is not like an option --foo in one version of Git changes its
meaning from X to Y across version boundaries, and even if on rare
occasions we need to introduce incompatibilities to improve the system, we
give enough advance warning and execute careful migration plans to ensure
that third-parties can keep up, and the "fix at the oldest branch and
merge upwards" policy means fixes to important bugs will be in all the
maintenance tracks, including the 'master' version.

So in practical terms, once 1.7.9 is out, there is *no* practical reason
for anybody to use 1.7.8.x or anything older. The only two things people
are gaining by sticking to an older version are that (1) they do not have
a way to use new features, and that (2) they get fixes that are less
rigorously tested, because the testing happens mostly in the context of
the 'next' branch and then subsequently in the 'master' branch, and fixes
cooked in these two contexts may have unintended consequences that will
never be discovered until they are merged down to the older maintenance
tracks.

If we only released the feature releases without _any_ maintenance
releases, distros no longer have an excuse to stick to older maintenance
tracks ("For the upcoming Zesty Zebra LTS, Git will stay at 1.7.7.x and
never updated to any newer major version.")  This in turn removes the need
for the third-party tools that support wider Git ecosystem to worry about
their users who are kept on very stale versions of Git by distros, because
any reasonably maintained distro will not pin their users to an ancient
version of Git.  If we can change the distro's idea of what constitutes a
release of Git that is on a single maintenance track from the current
"1.7.7.3 and newer, but not anything that does not begin with 1.7.7."  to
"1.7.9 and newer, but not anything that does not begin with 1.7.", that
would be a major win, I would imagine.

And dropping the maintenance tracks for older major releases may be a good
first step in that right direction.

With that in mind, the real answer to the original question in this thread
may be "the oldest supported version is the current one. stay at the
latest major release, in other words, do not even ask that question".

Thoughts?

Re: how to determine oldest supported version of git

[Jeff King]

On Fri, Feb 10, 2012 at 11:42:45AM -0800, Junio C Hamano wrote:

> Also our strict no regression policy means that it is not like an
> option --foo in one version of Git changes its meaning from X to Y
> across version boundaries, and even if on rare occasions we need to
> introduce incompatibilities to improve the system, we give enough
> advance warning and execute careful migration plans to ensure that
> third-parties can keep up, and the "fix at the oldest branch and merge
> upwards" policy means fixes to important bugs will be in all the
> maintenance tracks, including the 'master' version.

Just because we have a no-regression policy does not mean that we always
succeed. Bugs happen in new features. Hopefully the bugs only exist
for users of the new features, but occasionally they creep into
everybody's workflow.

If you are running v1.7.8.1 now, even if v1.7.9 is out, it is less risky
to move to v1.7.8.2 than to move to v1.7.9. Even though the risk of
moving to v1.7.9 is small, which I think is what you are arguing, it is
still greater than moving to a branch on which a release engineer (read:
you) has cherry-picked only ultra-safe bugfixes.

I think you are perhaps arguing that we are so safe that the difference
in risk is negligible. I do think we do better than many other projects,
but as a data storage project, I think it's nice to have a very
conservative branch (OTOH, data-loss bugs have historically been
extremely rare for git).

> So in practical terms, once 1.7.9 is out, there is *no* practical reason
> for anybody to use 1.7.8.x or anything older.

Wouldn't that also mean that v1.7.8.x has no reason to exist in the
first place? That is, if it is truly safe to move to v1.7.9, then why
did we bother making maintenance releases in the first place? Why not
just release the tip of master more frequently?

I think the answer is that v1.7.9 is _not_ as safe. But v1.7.9.4
probably _is_. That is, v1.7.9 will introduce new features, and
hopefully bugs will be shaken out while topics cook in master and next.
But once v1.7.9 is released, we get a much wider audience, and we will
probably find a few new bugs and some accidental regressions. Those will
slowly get fixed and we will get new minor v1.7.9.x releases, until at
some point the v1.7.9.x series gets just as solid as the v1.7.8.x series
was.

IOW, if you are on v1.7.8.2 and a we have a new bugfix, whether you want
a v1.7.8.3 or whether you jump to v1.7.9 should depend on where the
v1.7.9 series is in terms of maturity.

Which implies to me that in an ideal world, there would be maint
releases for the current series (i.e., v1.7.9.x now) and the previous
one (v1.7.8.x now). Somewhere around v1.7.9.3 (or after 3 months, or
whatever), stop bothering with v1.7.8.x releases.

Of course that takes time and effort (from you, mostly). If we dropped
maint releases entirely, then nobody would have to bother with the
release engineering task of deciding which topics were safe, and which
were not.

> If we only released the feature releases without _any_ maintenance
> releases, distros no longer have an excuse to stick to older maintenance
> tracks ("For the upcoming Zesty Zebra LTS, Git will stay at 1.7.7.x and
> never updated to any newer major version.")

I suspect that distros would not simply keep updating. Conservative
distros like Debian and RedHat (and probably Ubuntu LTSs, though I have
no experience with that) have policies in place about what can and
should go into updates. And they will end up doing the release
engineering themselves, staying on v1.7.7 + a bunch of cherry-picked
patches.

In some ways, that's a good thing; they can deal with the release
management work. OTOH, it's duplicated effort, and done by people who
are not as intimately familiar with git. A distro can probably pick up
your v1.7.8.x for an interim stable release without having to look
through all of the patches themselves.

Of course, the distros end up doing some of that release management
themselves, anyway. The git project's idea of "old" is 6-12 months, and
distros with long-term releases operate on much larger scales. So long
after we are tired of v1.7.8.x, they will probably still be
cherry-picking patches onto it. So maybe it is not worth caring about
distro's release management, as we are only helping them for a few
months, anyway.

-Peff

Re: how to determine oldest supported version of git

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> If you are running v1.7.8.1 now, even if v1.7.9 is out, it is less risky
> to move to v1.7.8.2 than to move to v1.7.9.

That has been the illusion I have been feeding the users.  Unfortunately,
of all people even you started to believe it, because I have been doing
such a good job in giving the illusion, not necessarily in actually
keeping the promise.

But think again, with the intimate knowledge of how these bugfix topics
are merged down to older maintenance tracks.

Typically, a new patch that looks reasonable lives in 'pu' only for a day
or two, cook in 'next' for a couple of days to a few weeks, merged to
'master' and then after several days to a week or two, further merged down
to 'maint' and older maintenance tracks.

I typically run 'next' and I presume so do many other Git people, so this
procedure has proven to be a reasonably robust way to ensure that by the
time the change hits 'master', it has been used in a context that is
reasonably close to the surrounding code (by this word, I do not mean
"textually surrounding" code here; what I mean are the things like the
internal state of the process prepared by existing code before it calls
the updated code, and the way the existing code uses data returned by the
new code) in its final destination that is 'master' for at least a week,
preferrably longer. That gives us a stable 'master'.

But nobody in the development community rebuilds 'maint' every time it is
updated and runs the result as his or her primary production version. Even
I do not do that (remember, I run 'next'). I only build and run full test
suite. Older maintenance tracks are worse. I do not think anybody runs
them before they are tagged and released.

And because 'maint' is deliberately kept behind to exclude all the new
features, and occasional code restructuring associated with some of the
new features only happens in 'master' but never merged down to 'maint',
the surrounding code between 'master' and 'maint' can become vastly
different, making the risk of potential breakage coming from impedance
mismatch higher. I would be very surprised if moving from 1.7.8.3 to
1.7.8.4 were less risky than moving from 1.7.8.3 to 1.7.9.1.  You simply
do not really know what you are getting if you moved from 1.7.6.1 (which
has fixes proven in early 'master' of post 1.7.6 cycle, that was similar
to 1.7.6) to 1.7.6.6 (which merges down fixes that were primarily tested
only in the context of post 1.7.8 cycle), so such a move is even riskier,
simply the base code has diverged too much.

> moving to v1.7.9 is small, which I think is what you are arguing, it is
> still greater than moving to a branch on which a release engineer (read:
> you) has cherry-picked only ultra-safe bugfixes.

The key thing to realize is that they are ultra-safe in the context it has
been cooking in.

> I think you are perhaps arguing that we are so safe that the difference
> in risk is negligible.

Quite the contrary, I am saying that older and untested down-merges are
much riskier, and the difference in risk should not be ignored.

> Which implies to me that in an ideal world, there would be maint
> releases for the current series (i.e., v1.7.9.x now) and the previous
> one (v1.7.8.x now). Somewhere around v1.7.9.3 (or after 3 months, or
> whatever), stop bothering with v1.7.8.x releases.

Actually what I was thinking was to restructure the release schedule
slightly so that

 * We do not merge to 'master' anything but bugfix patches to regressions
   introduced by 1.7.10 or to new features introduced by 1.7.10, for two
   weeks after it ships;

 * During that time, if an urgent fix is needed, 'maint' is directly
   patched to produce 1.7.9.X, and it is merged upward to 'next';

 * After finishing applying the early fixes to 1.7.10 to 'master', we tag
   the tip of 'master' as 1.7.10.1 and fork 'maint' from there;

 * At that point, old 'maint' and 1.7.9.X track cease to receive updates,
   as there is no point maintaining them. It only encourages distros to
   stay behind, adding unnecesary maintenance burden to us.

> In some ways, that's a good thing; they can deal with the release
> management work. OTOH, it's duplicated effort, and done by people who
> are not as intimately familiar with git.

Yes, that's the crucial observation to make.  Cherry-picking or down
merging fixes tested in a new context to older codebase that is not
actively used by the person who is cherry-picking does not produce a
stable end product. It only produces stale end product.  It makes it
slightly scarier to imagine that the cherry-pick is done by people who
may not be as familiar with the codebase as us, but on the other hand,
they might be using that old codebase for their day-to-day work, and may
have better luck hitting issues that did not manifest themselves in the
context of 'master' and 'next.

Re: how to determine oldest supported version of git

[Jeff King]

On Tue, Feb 14, 2012 at 10:36:32PM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > If you are running v1.7.8.1 now, even if v1.7.9 is out, it is less risky
> > to move to v1.7.8.2 than to move to v1.7.9.
> [...]
> But nobody in the development community rebuilds 'maint' every time it is
> updated and runs the result as his or her primary production version. Even
> I do not do that (remember, I run 'next'). I only build and run full test
> suite. Older maintenance tracks are worse. I do not think anybody runs
> them before they are tagged and released.

That's a good point. Maint releases are not well tested before release.
However, I think there are two things that help balance that:

  1. The commits that go onto them are "obviously" correct. I put
     obviously in quotes, because of course we can make a mistake. But
     just looking at the commits that end up in a typical maint release,
     they're usually quite conservative.

     So in deciding between v1.7.8.4 or v1.7.9, it is a question of
     whether you are more likely to be screwed by a conservative commit
     that has gotten less testing, or by one of the host of
     non-conservative commits that have gotten more testing.

     I don't think we have numbers on the historical balance (and going
     forward, it would really depend on qualitative factors like "how
     conservative" anyway).

  2. The commits on maint are often tested in isolation. That is, they
     are not generally cherry-picks, but were rather branched and
     developed from a maint-worthy version, and then forward ported into
     master (where forward porting for us is basically merging plus
     adding any required tweaks). So in an ideal world, the developer
     considers the fix looking at the maint code, and then the risky
     forward-porting happens in master (where we have time to cook and
     squash bugs).

     On the other hand, just because the developer comes up with the fix
     based on an old version doesn't mean that they don't make a
     mistake. And with nobody running it, those mistakes may slip
     through. Also, we do sometimes base bugfixes on the source of a
     bug, which is ancient, and then merge up not only to master but
     also to maint.  The right fix at the source of the bug may be
     different than what is needed at maint, which is different than
     what is needed at master.

Hmm. I really wish we had some numbers, because it's very unclear to me
which factor dominates. My gut says that the maint releases are still
safer, even with the problems you listed. I recall multiple bugs in
feature releases that caused quick bugfix releases. I don't recall
offhand having to quickly issue a fix for a maintenance release.

> > Which implies to me that in an ideal world, there would be maint
> > releases for the current series (i.e., v1.7.9.x now) and the previous
> > one (v1.7.8.x now). Somewhere around v1.7.9.3 (or after 3 months, or
> > whatever), stop bothering with v1.7.8.x releases.
> 
> Actually what I was thinking was to restructure the release schedule
> slightly so that
> 
>  * We do not merge to 'master' anything but bugfix patches to regressions
>    introduced by 1.7.10 or to new features introduced by 1.7.10, for two
>    weeks after it ships;
> 
>  * During that time, if an urgent fix is needed, 'maint' is directly
>    patched to produce 1.7.9.X, and it is merged upward to 'next';
> 
>  * After finishing applying the early fixes to 1.7.10 to 'master', we tag
>    the tip of 'master' as 1.7.10.1 and fork 'maint' from there;
> 
>  * At that point, old 'maint' and 1.7.9.X track cease to receive updates,
>    as there is no point maintaining them. It only encourages distros to
>    stay behind, adding unnecesary maintenance burden to us.

I think that is not so far from what I proposed (except that my "3
months or whatever" is your "2 weeks and one version").

> Yes, that's the crucial observation to make.  Cherry-picking or down
> merging fixes tested in a new context to older codebase that is not
> actively used by the person who is cherry-picking does not produce a
> stable end product. It only produces stale end product.  It makes it
> slightly scarier to imagine that the cherry-pick is done by people who
> may not be as familiar with the codebase as us, but on the other hand,
> they might be using that old codebase for their day-to-day work, and may
> have better luck hitting issues that did not manifest themselves in the
> context of 'master' and 'next.

Good point. I thought of them as less-qualified, but in many ways they
are more so.

Hmph. You've certainly given me something to think about. I joined this
thread thinking you were a little bit crazy, but now I think you are
starting to convince me. :)

-Peff

Re: how to determine oldest supported version of git

[Jonathan Nieder]

Junio C Hamano wrote:

> But think again, with the intimate knowledge of how these bugfix topics
> are merged down to older maintenance tracks.
[...]
> But nobody in the development community rebuilds 'maint' every time it is
> updated and runs the result as his or her primary production version. Even
> I do not do that (remember, I run 'next'). I only build and run full test
> suite. Older maintenance tracks are worse. I do not think anybody runs
> them before they are tagged and released.

I can offer one data point.  In the context of Debian sid, Gerrit and
I do test each version in daily work before uploading it.  I generally
build from and test whatever track is going to be used for the next
upload (usually plus a few extra features I am interested in for
private use) a little while before the release, to be prepared.
Anders sometimes uploads to the Ubuntu PPA which brings more testers.
After the upload, users running "sid" test for about a week before
even more users running "testing" get to take a look at it and test
for the sake of later users who will run "stable".

So little bugs get discovered, with time to fix them.

Even with this, the extra time to migrate from 1.7.6 to 1.7.7, for
example, was very helpful in the context of Debian sid.  Like it or
not, new features *do* come with minor regressions, and it helps the
sanity of a package maintainer to not have to suffer people who did
not request a bleeding-edge release complaining about regressions
until there has been time to fix them.

Of course, this has nothing to do with Debian stable, which is an
orthogonal story.  I'll discuss that below.

[...]
>  * At that point, old 'maint' and 1.7.9.X track cease to receive updates,
>    as there is no point maintaining them. It only encourages distros to
>    stay behind, adding unnecesary maintenance burden to us.

If you are thinking of distros like Debian stable, then that is just
wishful thinking.  Dropping support for old releases does not have any
effect except to cause patches to be missed there.  (See iceweasel and
chromium-browser for examples where using the version in Debian stable
is something I would usually not recommend.)

This may seem weird, but keep in mind that people like you and me are
not the target audience for the git package in Debian stable.  We use
git heavily.  If I am on a machine running stable or RHEL, I will
build a private copy of git in $HOME or ask the sysadmin to install a
more recent git as the first thing I do.

The reason that packages go into Debian stable and then just _don't
change_ is that the target users are not using those packages heavily.
If a new feature (e.g., "signatures from tags get incorporated into
the merge commit from pulling them") causes a regression (e.g., "the
script I used to run every week that pulls my favorite software
package and builds it just stopped working"), then these people get
zero benefit, for a sizable cost.

Though that's a digression.  The relevant detail to mention here is
that there is real demand on downstreams to continue to maintain
packages without adding new features.  They will help to maintain
old releases if you want.  If we want to influence that maintainance,
for example to ensure security bugs are fixed correctly and in the
same way everywhere, a good way is to keep a maintenance branch.

Hoping that clarifies a little.
Jonathan

Re: how to determine oldest supported version of git

[Jonathan Nieder]

Jonathan Nieder wrote:

> Even with this, the extra time to migrate from 1.7.6 to 1.7.7, for
> example, was very helpful in the context of Debian sid.

Whoops, off by one error.  The extra time to move from 1.7.4 to 1.7.5
and 1.7.5 to 1.7.6 was helpful.  1.7.7 was actually pretty painless,
so sid moved to it right away. ;-)