Instituting feature and infrastructure enhancement proposal window?

Instituting feature and infrastructure enhancement proposal window?

[Junio C Hamano]

As I mentioned, I'd like to keep each cycle manageably shorter
than 1.5.4 (which took 5 months that is 2 months too many).

This is still "a weather balloon", but I'd like to establish for
each development cycle a feature and infrastructure enhancement
proposal window, just like the kernel folks have "merge window".

 * These are outside of the scope:

   - Well-contained minor bugfixes,
   - Documentation updates,
   - Additional tests.
   - Updates to contrib/ and (what I consider) non-core parts,
     like gitweb.

 * Fixes to serious issues can delay and prolong the cycle.

 * The mailing list discussions and patch reviews are expected
   to continue as before, however:

   - Isolated new features that come after the window closes may
     be merged to "next" but won't graduate until the new release.

   - Infrastructure changes that have wider impact that come
     after the window closes won't even hit "next" until the
     new release.

If this proposal is accepted favorably by the list and is
implemented, a release cycle will look like this:

 * A release from 'master' is made (e.g. v1.5.4 gets released on
   Feb 1st).

 * The release is merged to 'maint'.

 * Fix-ups to v1.5.4 start flowing and get applied to 'maint',
   and merged to 'master'.  At some point, the first maintenance
   release is made (e.g. v1.5.4.1 was released on Feb 10th).

 * The topics that have been cooking in 'next' may be rebased to
   v1.5.4, and 'next' is rebuilt on top of 'master' with them
   (e.g. this happened on Feb 17th for this cycle).

 * New topics start appearing in 'next', cook and graduate to
   'master' as before.

 * The window closes.  We pick what topics we should have in the
   new release.

 * We continue as before, but with the understanding that some
   topics in 'next' won't graduate before the new release.

 * Tag -rc1.  'next' really closes and we go into feature
   freeze.

 * RC cycle continues.  Perhaps one or two RCs every week, as before.

 * The new release is made (hopefully within 3 months since the
   beginning of the cycle).

The important dates in the above would be (in parentheses are my
straw-mans):

 * The first maintenance release (+1 week from the last release)

 * The rebuilding of 'next' (immediately after the first maint)

 * The close of the window (+6 weeks from the last release)

 * The -rc1 (+8 weeks from the last release)

 * The new release (+12 weeks from the last release)

For 1.5.5, I'd like to make the above +6 weeks a bit short and
make it by the end of this month, though, so upcoming dates
would roughly be:

 - By the end of February, the 1.5.5 window closes;
 - By mid March, 1.5.5-rc1 is tagged;
 - By early April, 1.5.5 is released.

Which would make this week for the discussion to pick which
topic should be in 1.5.5.

Here are the current candidates I personally have in mind for
topics that should be in 1.5.5:

 * describe --exact (Shawn)
 * checkout in C (Daniel)
 * format-patch --cover-letter (Daniel)
 * autodetect ncpu in threaded pack (Andreas Ericsson)
 * stash delete / stash pop (Dscho, Brandon Casey)
 * run-command API clean-ups (Johannes Sixt)
 * local branch tracking (Jay Soffian)
 * url rewriting (Daniel)
 * low-level merge improvements (Dscho)
 * apply --whitespace=fix improvements (me)
 * diff --relative (me)
 * diff --dirstat (Linus)
 - mergetool updates (Charles Bailey)
 - bisect vs cg-seek (Carl Worth)
 - worktree setup (Nguyễn Thái Ngọc Duy)
 - tightened object parser and walker safety (Martin Koegler)

One topic that is missing from the above list, which is queued
in 'pu', worth mentioning is "gitdir: $path in .git" by Lars
Hjemli.  The situation is like Dscho's "reflog delete" before we
did v1.5.4 in that it is waiting for a real user to prove this
is a good enhancement.  I am sure that later rounds would build
enhancements to "git submodule" on top of this, and I think it
is better to wait until that effort starts cooking.

Thoughts?

Re: Instituting feature and infrastructure enhancement proposal window?

[Linus Torvalds]

On Sun, 24 Feb 2008, Junio C Hamano wrote:
>
> The important dates in the above would be (in parentheses are my
> straw-mans):

On the merge window issue: I can heartily recommend it, but for the kernel 
we have noticed:

 - people get really antsy it the releases are more than two months apart. 
   Quite frankly, six weeks would be better, but it's just not realistic.

 - the kernel has to have a long stabilization phase, and I largely blame 
   the fact that we have so much different hardware (and driver and arch 
   changes are generally much more than 50% of all changes anyway). So we 
   arguably need wider testing than projects that are more "repeatable" on 
   a smaller set of machines.

So I think the merge window has worked out beautifully, but in a perfect 
world, I'd make the window be roughly half the time of the release (rather 
than about 20% as it is for the kernel), and aiming for a six-week release 
timing. And I think both of those should be reasonably easy to hit within 
the git development model.

So I would basically cut you target timeframes into half (except for the 
first -rc release - there's no point in making that less than a week). 
Three months is just going to make people who miss the release window 
antsy. It's what the kernel has in practice, and I would *not* advocate it 
as ideal - I actually aim for 2 months and we then invariably slip a bit..

So I'd suggest:
 - first -rc in 1 week
 - window closes in 3 weeks
 - next release in 6 weeks
 - rinse and repeat

and if you worry that there won't be enough changes to merit a release, 
trust me, that's the *good* schenario. Nobody will ever mind releases that 
aren't disruptive, and you'll have an easy case with making them too. 
There really are no downsides to having an easy release cycle with not a 
lot of big changes. I _wish_ I had more of those.

And on exactly that note:

>  * diff --dirstat (Linus)

I actually actively use this (not just for -rc releases, but I find it 
nice to do occasionally in other cases too), so it would be really nice if 
at least the initial version got merged soon. Even if it gets the binary 
case wrong, and even if the statistics would arguably be better based on 
"damage" instead of just number of lines.

Of course, I have it in my tree regardless, but especially with me then 
publicising the feature in my -rc releases (another one coming up today), 
I'd hope that you'd merge it so that others can use it too.. If this was 
at the end of a merge window, I'd hate to have to wait 1.5 months for the 
next one to open.

			Linus

Re: Instituting feature and infrastructure enhancement proposal window?

[Shawn O. Pearce]

Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Sun, 24 Feb 2008, Junio C Hamano wrote:
> >
> > The important dates in the above would be (in parentheses are my
> > straw-mans):
> 
> So I would basically cut you target timeframes into half (except for the 
> first -rc release - there's no point in making that less than a week). 
> Three months is just going to make people who miss the release window 
> antsy. It's what the kernel has in practice, and I would *not* advocate it 
> as ideal - I actually aim for 2 months and we then invariably slip a bit..
> 
> So I'd suggest:
>  - first -rc in 1 week
>  - window closes in 3 weeks
>  - next release in 6 weeks
>  - rinse and repeat

Yea, I agree with Linus suggestion of cutting the time windows
down to closer to 2 months per release.  1.5.4 took a long time.
It doesn't really affect me as I usually run the bleeding edge
'next', but I think most of our users feel warm and fuzzy about
running a released version.  As a community we need to help Junio
get stable releases out more often, so users can benefit from the
improvements we've made.

-- 
Shawn.

Re: Instituting feature and infrastructure enhancement proposal window?

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> And on exactly that note:
>
>>  * diff --dirstat (Linus)
>
> I actually actively use this (not just for -rc releases, but I find it 
> nice to do occasionally in other cases too), so it would be really nice if 
> at least the initial version got merged soon. Even if it gets the binary 
> case wrong,...

I am planning to do this (dropping the "damange" one I
previously showed and queued in 'pu') and merge to 'master' for
now.

-- >8 --
[PATCH] diff --dirstat: saner handling of binary and unmerged files

We do not account binary nor unmerged files when --shortstat is
asked for (or the summary stat at the end of --stat).  

The new option --dirstat should work the same way as it is about
summarizing the changes of multiple files by adding them up.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/diff.c b/diff.c
index dd374d4..bcc323f 100644
--- a/diff.c
+++ b/diff.c
@@ -1016,7 +1016,10 @@ static long gather_dirstat(struct diffstat_dir *dir, unsigned long changed, cons
 			this = gather_dirstat(dir, changed, f->name, newbaselen);
 			sources++;
 		} else {
-			this = f->added + f->deleted;
+			if (f->is_unmerged || f->is_binary)
+				this = 0;
+			else
+				this = f->added + f->deleted;
 			dir->files++;
 			dir->nr--;
 			sources += 2;
@@ -1053,6 +1056,8 @@ static void show_dirstat(struct diffstat_t *data, struct diff_options *options)
 	/* Calculate total changes */
 	changed = 0;
 	for (i = 0; i < data->nr; i++) {
+		if (data->files[i]->is_binary || data->files[i]->is_unmerged)
+			continue;
 		changed += data->files[i]->added;
 		changed += data->files[i]->deleted;
 	}

Re: Instituting feature and infrastructure enhancement proposal window?

[Daniel Barkalow]

On Sun, 24 Feb 2008, Junio C Hamano wrote:

> If this proposal is accepted favorably by the list and is
> implemented, a release cycle will look like this:
> 
>  * A release from 'master' is made (e.g. v1.5.4 gets released on
>    Feb 1st).
> 
>  * The release is merged to 'maint'.
> 
>  * Fix-ups to v1.5.4 start flowing and get applied to 'maint',
>    and merged to 'master'.  At some point, the first maintenance
>    release is made (e.g. v1.5.4.1 was released on Feb 10th).
> 
>  * The topics that have been cooking in 'next' may be rebased to
>    v1.5.4, and 'next' is rebuilt on top of 'master' with them
>    (e.g. this happened on Feb 17th for this cycle).

I'd actually like to have topics that don't make a release reverted in 
'next' and reapplied in 'pu'. That way, from the release to the rebuild of 
'next', 'next' and 'master' have identical trees, and it's therefore 
trivial to rebase off of 'next'. Topics that were moved out to 'pu' in 
this period would presumably be merged back into the rebased 'next' early.

Or maybe, when 'next' closes, anything not in 'master' moves out to 'pu' 
until the rebuild.

>  * New topics start appearing in 'next', cook and graduate to
>    'master' as before.
> 
>  * The window closes.  We pick what topics we should have in the
>    new release.
> 
>  * We continue as before, but with the understanding that some
>    topics in 'next' won't graduate before the new release.
> 
>  * Tag -rc1.  'next' really closes and we go into feature
>    freeze.
> 
>  * RC cycle continues.  Perhaps one or two RCs every week, as before.
> 
>  * The new release is made (hopefully within 3 months since the
>    beginning of the cycle).

So what would be the 'pu' policy through this? I think one of the reasons 
that the kernel manages to do well with a short cycle is that stuff sits 
in -mm for some time, in general, before hitting mainline. This both means 
that people can get their code exposure without hurrying it into the merge 
window and that code can cook as long as necessary before starting to 
impact end users.

In any case, I like the short cycle idea.

	-Daniel
*This .sig left intentionally blank*