Re: [1.8.0] reorganize the mess that the source tree has become

Re: [1.8.0] reorganize the mess that the source tree has become

[George Spelvin]

For what it's worth, I don't see the "cleanup".

If it significantly reduced the size of the largest directory,
that would be a win.  But moving everything into src/ doesn't
do that.

If there's a way to divide the source into cohesive subunits, that
would be great.  A programmer could ignore whole subdirectories
when working on something.

But just moving the whole existing pile into a subdirectory "because
everyone else does it" is not a reason; that's superstition.

Having to type "src/" a lot more often is definitely a downside.

Heck, that's one thing I actively dislike about GNU autoconf conventions.

If there's a compelling reason to change, could someone please describe it?

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Wed, Feb 2, 2011 at 3:29 AM, George Spelvin <linux@horizon.com> wrote:
> If there's a compelling reason to change, could someone please describe it?

I think the most compelling argument I can think of is that it makes
top-level entries like Documentation, RelNotes, COPYING, README and
INSTALL easier to spot when doing "ls". For some users (first-time
hackers etc) that's a pretty big plus, I'd think.

Re: [1.8.0] reorganize the mess that the source tree has become

[Pascal Obry]

Le 02/02/2011 09:31, Erik Faye-Lund a écrit :
> On Wed, Feb 2, 2011 at 3:29 AM, George Spelvin <linux@horizon.com> wrote:
>> If there's a compelling reason to change, could someone please describe it?
> 
> I think the most compelling argument I can think of is that it makes
> top-level entries like Documentation, RelNotes, COPYING, README and
> INSTALL easier to spot when doing "ls". For some users (first-time
> hackers etc) that's a pretty big plus, I'd think.

I was about to say that too. That's an argument strong enough I would
say. Today it is hard to see the structure.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Tue, 1 Feb 2011, George Spelvin wrote:

> For what it's worth, I don't see the "cleanup".
> 
> If it significantly reduced the size of the largest directory,
> that would be a win.  But moving everything into src/ doesn't
> do that.
> 
> If there's a way to divide the source into cohesive subunits, that
> would be great.  A programmer could ignore whole subdirectories
> when working on something.
> 
> But just moving the whole existing pile into a subdirectory "because
> everyone else does it" is not a reason; that's superstition.

There is no superstition here, simply basic elegance.

When you pick up a book from a shelf, do you see the actual content of 
the book printed right from the inside of the cover page, and the table 
of content tossed in the margin?  Would you construct a book yourself 
that way?

A nice source tree should be organized with a minimum of hierarchical 
structure.  To a newbie wanting to contribute to Git, it is rather 
frightening to cd into the git directory and see that ls generates more 
than 280 entries.  That simply looks sloppy.  And this gets much worse 
after a make.

The top directory should make different things stand out much more 
clearly, like a preface and a table of content.  You have the 
documentation here, the source there, the tests there, a clearly visible 
README file, etc.  If the src directory has about the same relative 
number of files after a move that's fine.  At least you should expect 
_only_ source files in there (and possibly their by-products), and not 
other types of data buried into the mix.

> Having to type "src/" a lot more often is definitely a downside.

Come on.  This is a rather egocentric argument without much substance.

> Heck, that's one thing I actively dislike about GNU autoconf conventions.

This has _nothing_ about any autoconf convention.  GNU autoconf requires 
stupid things like having a bunch of files such as CREDITS, INSTALL, 
CHANGELOG, and other whatnots even if you have nothing to put in them, 
in which case they still have to be there but empty.  It also dictates 
the exact name your directories must have, etc.

I'm not proposing a tree reorganization because GNU autoconf commands 
it, but rather because this is a sensible thing to do.

> If there's a compelling reason to change, could someone please describe it?

It's about the third time I'm putting forward arguments for this.  
Please see the list archive.

P.S. the netiquette on busy mailing lists recommends that you preserve 
all the email addresses that were listed as recipients on the message 
you reply to.  That would be highly appreciated.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Miles Bader]

Nicolas Pitre <nico@fluxnic.net> writes:
>> But just moving the whole existing pile into a subdirectory "because
>> everyone else does it" is not a reason; that's superstition.
>
> There is no superstition here, simply basic elegance.

"basic elegance" is hardly well-defined, and although there are probably
issues on which there's general agreement, this doesn't appear to be one
of them.

>> Having to type "src/" a lot more often is definitely a downside.
>
> Come on.  This is a rather egocentric argument without much substance.

It certainly has more substance than hand-waving like "basic elegance"
though...

Some slightly more concrete arguments have been:

  Pro-src:  Big top-level dir scares newbs
  Anti-src: Extra typing is annoying

I'm not really against a "src/" subdir, but it seems mostly a matter of
taste, and I've seen plenty of projects where the src/ directory seemed
pretty pointless...

-Miles

-- 
Politics, n. A strife of interests masquerading as a contest of
principles. The conduct of public affairs for private advantage.

Re: [1.8.0] reorganize the mess that the source tree has become

[Andreas Schwab]

Nicolas Pitre <nico@fluxnic.net> writes:

> This has _nothing_ about any autoconf convention.  GNU autoconf requires 
> stupid things like having a bunch of files such as CREDITS, INSTALL, 
> CHANGELOG, and other whatnots even if you have nothing to put in them, 
> in which case they still have to be there but empty.  It also dictates 
> the exact name your directories must have, etc.

That's automake, not autoconf (and only the default automake operation).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [1.8.0] reorganize the mess that the source tree has become

[Eugene Sajine]

On Thu, Feb 3, 2011 at 1:16 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Tue, 1 Feb 2011, George Spelvin wrote:
>
>> For what it's worth, I don't see the "cleanup".
>>
>> If it significantly reduced the size of the largest directory,
>> that would be a win.  But moving everything into src/ doesn't
>> do that.
>>
>> If there's a way to divide the source into cohesive subunits, that
>> would be great.  A programmer could ignore whole subdirectories
>> when working on something.
>>
>> But just moving the whole existing pile into a subdirectory "because
>> everyone else does it" is not a reason; that's superstition.
>
> There is no superstition here, simply basic elegance.
>
> When you pick up a book from a shelf, do you see the actual content of
> the book printed right from the inside of the cover page, and the table
> of content tossed in the margin?  Would you construct a book yourself
> that way?
>
> A nice source tree should be organized with a minimum of hierarchical
> structure.  To a newbie wanting to contribute to Git, it is rather
> frightening to cd into the git directory and see that ls generates more
> than 280 entries.  That simply looks sloppy.  And this gets much worse
> after a make.
>
> The top directory should make different things stand out much more
> clearly, like a preface and a table of content.  You have the
> documentation here, the source there, the tests there, a clearly visible
> README file, etc.  If the src directory has about the same relative
> number of files after a move that's fine.  At least you should expect
> _only_ source files in there (and possibly their by-products), and not
> other types of data buried into the mix.
>
>> Having to type "src/" a lot more often is definitely a downside.
>
> Come on.  This is a rather egocentric argument without much substance.
>
>> Heck, that's one thing I actively dislike about GNU autoconf conventions.
>
> This has _nothing_ about any autoconf convention.  GNU autoconf requires
> stupid things like having a bunch of files such as CREDITS, INSTALL,
> CHANGELOG, and other whatnots even if you have nothing to put in them,
> in which case they still have to be there but empty.  It also dictates
> the exact name your directories must have, etc.
>
> I'm not proposing a tree reorganization because GNU autoconf commands
> it, but rather because this is a sensible thing to do.
>
>> If there's a compelling reason to change, could someone please describe it?
>
> It's about the third time I'm putting forward arguments for this.
> Please see the list archive.
>
> P.S. the netiquette on busy mailing lists recommends that you preserve
> all the email addresses that were listed as recipients on the message
> you reply to.  That would be highly appreciated.
>
>
> Nicolas
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I'm not a hacker, but a user who had sometimes peeked into the git
sources. Unbelievable mess... Impossible to see the structure in
command line interface.
I totally agree with Nicolas here.
Folders were invented for a reason.

IMHO
src for source code
build for build by-products
tests for tests

Come on, give us some love, please!;)

Thanks,
Eugene

Re: [1.8.0] reorganize the mess that the source tree has become

[Hilco Wijbenga]

On 3 February 2011 10:46, Eugene Sajine <euguess@gmail.com> wrote:
> On Thu, Feb 3, 2011 at 1:16 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
>> On Tue, 1 Feb 2011, George Spelvin wrote:
>>
>>> For what it's worth, I don't see the "cleanup".
>>>
>>> If it significantly reduced the size of the largest directory,
>>> that would be a win.  But moving everything into src/ doesn't
>>> do that.
>>>
>>> If there's a way to divide the source into cohesive subunits, that
>>> would be great.  A programmer could ignore whole subdirectories
>>> when working on something.
>>>
>>> But just moving the whole existing pile into a subdirectory "because
>>> everyone else does it" is not a reason; that's superstition.
>>
>> There is no superstition here, simply basic elegance.
>>
>> When you pick up a book from a shelf, do you see the actual content of
>> the book printed right from the inside of the cover page, and the table
>> of content tossed in the margin?  Would you construct a book yourself
>> that way?
>>
>> A nice source tree should be organized with a minimum of hierarchical
>> structure.  To a newbie wanting to contribute to Git, it is rather
>> frightening to cd into the git directory and see that ls generates more
>> than 280 entries.  That simply looks sloppy.  And this gets much worse
>> after a make.
>>
>> The top directory should make different things stand out much more
>> clearly, like a preface and a table of content.  You have the
>> documentation here, the source there, the tests there, a clearly visible
>> README file, etc.  If the src directory has about the same relative
>> number of files after a move that's fine.  At least you should expect
>> _only_ source files in there (and possibly their by-products), and not
>> other types of data buried into the mix.
>>
>>> Having to type "src/" a lot more often is definitely a downside.
>>
>> Come on.  This is a rather egocentric argument without much substance.
>>
>>> Heck, that's one thing I actively dislike about GNU autoconf conventions.
>>
>> This has _nothing_ about any autoconf convention.  GNU autoconf requires
>> stupid things like having a bunch of files such as CREDITS, INSTALL,
>> CHANGELOG, and other whatnots even if you have nothing to put in them,
>> in which case they still have to be there but empty.  It also dictates
>> the exact name your directories must have, etc.
>>
>> I'm not proposing a tree reorganization because GNU autoconf commands
>> it, but rather because this is a sensible thing to do.
>>
>>> If there's a compelling reason to change, could someone please describe it?
>>
>> It's about the third time I'm putting forward arguments for this.
>> Please see the list archive.
>>
>> P.S. the netiquette on busy mailing lists recommends that you preserve
>> all the email addresses that were listed as recipients on the message
>> you reply to.  That would be highly appreciated.
>>
>>
>> Nicolas
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> I'm not a hacker, but a user who had sometimes peeked into the git
> sources. Unbelievable mess... Impossible to see the structure in
> command line interface.
> I totally agree with Nicolas here.
> Folders were invented for a reason.
>
> IMHO
> src for source code
> build for build by-products
> tests for tests
>
> Come on, give us some love, please!;)

Another one from the peanut gallery. :-) I wholeheartedly agree with
both Nicolas and Eugene.

Quite frankly, I'm surprised there are (presumably experienced)
developers who do not immediately see the value of a little
organization. Surely, given the use of code conventions, formatting
rules, etcetera, the obvious one step further is to also organize
where the files go?

(Given that I'm just a lurker I promise to leave it at this. I just
wanted to show Nicolas a little support.)

Re: [1.8.0] reorganize the mess that the source tree has become

[Miles Bader]

Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> Quite frankly, I'm surprised there are (presumably experienced)
> developers who do not immediately see the value of a little
> organization. Surely, given the use of code conventions, formatting
> rules, etcetera, the obvious one step further is to also organize
> where the files go?

I think one of the problems is that what's been suggested seems like
window-dressing.  Moving everything into src/ and calling it "organized"
doesn't actually accomplish much other than perhaps making the README
file more visible to newbs; things are _still_ a mess, just a mess with
four more letters...

-Miles

-- 
Back, n. That part of your friend which it is your privilege to contemplate in
your adversity.

Re: [1.8.0] reorganize the mess that the source tree has become

[Tor Arntsen]

On Fri, Feb 4, 2011 at 03:06, Miles Bader <miles@gnu.org> wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>> Quite frankly, I'm surprised there are (presumably experienced)
>> developers who do not immediately see the value of a little
>> organization. Surely, given the use of code conventions, formatting
>> rules, etcetera, the obvious one step further is to also organize
>> where the files go?
>
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

What Miles says is my feeling as well. And having a 'bin/' as was suggested
in one post doesn't make much sense to me either - if you want your compiled
output to go elsewhere than the source directory then the normal way of doing
that is to do and out-of-tree build (so if that's not working - I have
not checked -
then that's something which would be worth looking into.)

-Tor

Re: [1.8.0] reorganize the mess that the source tree has become

[Jakub Narebski]

Tor Arntsen <tor@spacetec.no> writes:
> On Fri, Feb 4, 2011 at 03:06, Miles Bader <miles@gnu.org> wrote:
> > Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>>>
>>> Quite frankly, I'm surprised there are (presumably experienced)
>>> developers who do not immediately see the value of a little
>>> organization. Surely, given the use of code conventions, formatting
>>> rules, etcetera, the obvious one step further is to also organize
>>> where the files go?
>>
>> I think one of the problems is that what's been suggested seems like
>> window-dressing.  Moving everything into src/ and calling it "organized"
>> doesn't actually accomplish much other than perhaps making the README
>> file more visible to newbs; things are _still_ a mess, just a mess with
>> four more letters...
> 
> What Miles says is my feeling as well. And having a 'bin/' as was suggested
> in one post doesn't make much sense to me either - if you want your compiled
> output to go elsewhere than the source directory then the normal way of doing
> that is to do and out-of-tree build (so if that's not working - I have
> not checked - then that's something which would be worth looking into.)

It is about supporting 'srcdir', isn't it?

BTW. what about using 'lib/' directory?
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Fri, Feb 4, 2011 at 3:06 AM, Miles Bader <miles@gnu.org> wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>> Quite frankly, I'm surprised there are (presumably experienced)
>> developers who do not immediately see the value of a little
>> organization. Surely, given the use of code conventions, formatting
>> rules, etcetera, the obvious one step further is to also organize
>> where the files go?
>
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

FWIW, I don't quite see what's wrong with "window dressing" here.
Making those files more visible is a good thing, IMO.

But I'm not so sure I agree that the rest of the source tree is such a
mess that everyone makes it out to be. OK, there's a lot of
source-files on the top-level (which would be the src-level with this
change), but why is that such a bad thing? And if this is a big deal,
perhaps moving libgit-sources to a separate folder would help?

[1.8.0] Re: reorganize the mess that the source tree has become

[Nicolas Sebrecht]

The 04/02/11, Miles Bader wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> > Quite frankly, I'm surprised there are (presumably experienced)
> > developers who do not immediately see the value of a little
> > organization. Surely, given the use of code conventions, formatting
> > rules, etcetera, the obvious one step further is to also organize
> > where the files go?
> 
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

So it would be an ordered mess, at least. The current amount of files in
the root directory do make things harder for people not already familiar
with the content. FMHO, moving the source files into a subdirectory
could be only a first step to the good direction.

-- 
Nicolas Sebrecht

Re: [1.8.0] Re: reorganize the mess that the source tree has become

[Drew Northup]

On Fri, 2011-02-04 at 19:15 +0100, Nicolas Sebrecht wrote:
> The 04/02/11, Miles Bader wrote:
> > Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> > > Quite frankly, I'm surprised there are (presumably experienced)
> > > developers who do not immediately see the value of a little
> > > organization. Surely, given the use of code conventions, formatting
> > > rules, etcetera, the obvious one step further is to also organize
> > > where the files go?
> > 
> > I think one of the problems is that what's been suggested seems like
> > window-dressing.  Moving everything into src/ and calling it "organized"
> > doesn't actually accomplish much other than perhaps making the README
> > file more visible to newbs; things are _still_ a mess, just a mess with
> > four more letters...
> 
> So it would be an ordered mess, at least. The current amount of files in
> the root directory do make things harder for people not already familiar
> with the content. FMHO, moving the source files into a subdirectory
> could be only a first step to the good direction.

Nicolas,
Having once upon a time (in CVS days) taken over a project that was
neatly organized into tons of folders I can say that more folders is not
always better.
If you are organizing things into modules by folders, and those things
are mutually exclusive pre-compilation then doing so may make sense. If
the folders ADD value to the project by adding organization--as opposed
to hiding disorganization--then they may have value.
>From my meager hacking thus far (working on making utf-16 a more
user-friendly experience out-of-the-box) I have found that none of this
is true (thus far) with the git codebase. In fact the one thing that
would have been useful is more in-code--and/or separate
API-ish--documentation (it took me waaaay too long to figure out how git
add, aka git-add, works), but I am too much of a realist to expect that
to change much. I most certainly DO NOT recommend that a mess of patches
be submitted to Junio to fix it (document what you are working on as you
see fit; I work on too many things to not document fairly extensively).
I approach codebase reorganization the same way. I have seen the
destructive things it can do to a project when it becomes an end unto
itself separate from the primary focus. In fact, what killed that first
project I mentioned was an argument about something that was not part of
the primary purpose of the project which exploded with a fury resembling
a religious confrontation. I do not want to see that happen to Git. I
didn't want to see that project die either, but when you exasperate
enough of the core developers that's what happens. 

(My first job as project leader was to get it off of our CVS host, the
second was to find it a nice grave over at nongnu.org. I never did get
CVS import to work. I never did convince any of the core developers that
it was still worth working on.)

-- 
-Drew Northup
________________________________________________
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59

[1.8.0] Re: reorganize the mess that the source tree has become

[Nicolas Sebrecht]

The 04/02/11, Drew Northup wrote:

> Having once upon a time (in CVS days) taken over a project that was
> neatly organized into tons of folders I can say that more folders is not
> always better.
> If you are organizing things into modules by folders, and those things
> are mutually exclusive pre-compilation then doing so may make sense. If
> the folders ADD value to the project by adding organization--as opposed
> to hiding disorganization--then they may have value.

This _is_ what we are talking about. Not tons of folders or whathever
you might think.

> destructive things
>                                                       killed that first
> project
>                            project       exploded with a fury resembling
> a religious confrontation.                  see that happen to Git
>                see that project die                  you exasperate
> enough of the core developers

Read again. I'm pretty sure this was not your goal but this almostly
looks like FUD to me. So, I don't think I'll involve to more discussion
in this subthread.

-- 
Nicolas Sebrecht

What's cooking in git.git (Jan 2011, #06; Sun, 30)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with '-' are
only in 'pu' while commits prefixed with '+' are in 'next'.

1.7.4 is out. I'd like to stop and calm the tree down for a few days
so that we can catch any brown-paper-bag bugs before moving things
forward, and then open the floodgate for the next cycle, which I am
inclined to designate as "We would have done these differently if we
were creating git from scratch with the experience we have and wisdom
we have gained" cycle, allowing minor backward incompatibilities,
somewhat like 1.6.0 but not so drastic.  The result would probably
be called 1.8.0--the details in a separate message.

--------------------------------------------------
[New Topics]

* jc/fsck-fixes (2011-01-26) 2 commits
 - fsck: do not give up too early in fsck_dir()
 - fsck: drop unused parameter from traverse_one_object()

--------------------------------------------------
[Stalled]

* nd/index-doc (2010-09-06) 1 commit
 - doc: technical details about the index file format

Half-written but it is a good start.  I may need to give some help in
describing more recent index extensions.

* cb/ignored-paths-are-precious (2010-08-21) 1 commit
 - checkout/merge: optionally fail operation when ignored files need to be overwritten

This needs tests; also we know of longstanding bugs in related area that
needs to be addressed---they do not have to be part of this series but
their reproduction recipe would belong to the test script for this topic.

It would hurt users to make the new feature on by default, especially the
ones with subdirectories that come and go.

* jk/tag-contains (2010-07-05) 4 commits
 - Why is "git tag --contains" so slow?
 - default core.clockskew variable to one day
 - limit "contains" traversals based on commit timestamp
 - tag: speed up --contains calculation

The idea of the bottom one is probably Ok, except that the use of object
flags needs to be rethought, or at least the helper needs to be moved to
builtin/tag.c to make it clear that it should not be used outside the
current usage context.

* jc/rename-degrade-cc-to-c (2011-01-06) 3 commits
 - diffcore-rename: fall back to -C when -C -C busts the rename limit
 - diffcore-rename: record filepair for rename src
 - diffcore-rename: refactor "too many candidates" logic

IIRC, this was a weather-baloon "if you wanted to, this may be how you
would do it" without test updates.  People who care need to help moving
things forward.

* jc/rerere-remaining (2011-01-06) 1 commit
 - rerere "remaining"

Just a handful of weatherballoon patches without proper tests, in response
to feature/minor fix requests.

* ab/p4 (2011-01-11) 1 commit
 - git-p4: correct indenting and formatting

Lacks sign-off but is trivial.

* tr/maint-branch-no-track-head (2010-12-14) 1 commit
 - branch: do not attempt to track HEAD implicitly

Probably needs a re-roll to exclude either (1) any ref outside the
hierarchies for branches (i.e. refs/{heads,remotes}/), or (2) only refs
outside refs/ hierarchies (e.g. HEAD, ORIG_HEAD, ...).  The latter feels
safer and saner.

* hv/mingw-fs-funnies (2010-12-14) 5 commits
 - mingw_rmdir: set errno=ENOTEMPTY when appropriate
 - mingw: add fallback for rmdir in case directory is in use
 - mingw: make failures to unlink or move raise a question
 - mingw: work around irregular failures of unlink on windows
 - mingw: move unlink wrapper to mingw.c

Will be rerolled (Heiko, 2010-12-23)

* mz/rebase (2010-12-28) 31 commits
 - rebase -i: remove unnecessary state rebase-root
 - rebase -i: don't read unused variable preserve_merges
 - git-rebase--am: remove unnecessary --3way option
 - rebase -m: don't print exit code 2 when merge fails
 - rebase -m: remember allow_rerere_autoupdate option
 - rebase: remember strategy and strategy options
 - rebase: remember verbose option
 - rebase: extract code for writing basic state
 - rebase: factor out sub command handling
 - rebase: make -v a tiny bit more verbose
 - rebase -i: align variable names
 - rebase: show consistent conflict resolution hint
 - rebase: extract am code to new source file
 - rebase: extract merge code to new source file
 - rebase: remove $branch as synonym for $orig_head
 - rebase -i: support --stat
 - rebase: factor out call to pre-rebase hook
 - rebase: factor out clean work tree check
 - rebase: factor out reference parsing
 - rebase: reorder validation steps
 - rebase -i: remove now unnecessary directory checks
 - rebase: factor out command line option processing
 - rebase: align variable content
 - rebase: align variable names
 - rebase: stricter check of standalone sub command
 - rebase: act on command line outside parsing loop
 - rebase: improve detection of rebase in progress
 - rebase: remove unused rebase state 'prev_head'
 - rebase: read state outside loop
 - rebase: refactor reading of state
 - rebase: clearer names for directory variables

Will be rerolled (Martin, Jan 28).

--------------------------------------------------
[Cooking]

* jn/unpack-lstat-failure-report (2011-01-12) 2 commits
  (merged to 'next' on 2011-01-24 at 1245180)
 + unpack-trees: handle lstat failure for existing file
 + unpack-trees: handle lstat failure for existing directory

* rr/fi-import-marks-if-exists (2011-01-15) 1 commit
 - fast-import: Introduce --import-marks-if-exists

Looked reasonable.

* tr/diff-words-test (2011-01-18) 4 commits
 - t4034 (diff --word-diff): add a minimum Perl drier test vector
 - t4034 (diff --word-diff): style suggestions
 - userdiff: simplify word-diff safeguard
 - t4034: bulk verify builtin word regex sanity

I thought this was Ok; further comments, anybody?

* ef/alias-via-run-command (2011-01-07) 1 commit
  (merged to 'next' on 2011-01-06 at 1fbd4a0)
 + alias: use run_command api to execute aliases

* uk/checkout-ambiguous-ref (2011-01-11) 1 commit
  (merged to 'next' on 2011-01-11 at 2aa30de)
 + checkout: fix bug with ambiguous refs

* cb/setup (2010-12-27) 1 commit
  (merged to 'next' on 2011-01-05 at 790b288)
 + setup: translate symlinks in filename when using absolute paths

* ae/better-template-failure-report (2010-12-18) 1 commit
  (merged to 'next' on 2011-01-05 at d3f9142)
 + Improve error messages when temporary file creation fails

* jc/unpack-trees (2010-12-22) 2 commits
 - unpack_trees(): skip trees that are the same in all input
 - unpack-trees.c: cosmetic fix

* jn/cherry-pick-strategy-option (2010-12-10) 1 commit
  (merged to 'next' on 2011-01-05 at 3ccc590)
 + cherry-pick/revert: add support for -X/--strategy-option

* nd/struct-pathspec (2010-12-15) 21 commits
  (merged to 'next' on 2011-01-24 at 08f1774)
 + t7810: overlapping pathspecs and depth limit
 + grep: drop pathspec_matches() in favor of tree_entry_interesting()
 + grep: use writable strbuf from caller for grep_tree()
 + grep: use match_pathspec_depth() for cache/worktree grepping
 + grep: convert to use struct pathspec
 + Convert ce_path_match() to use match_pathspec_depth()
 + Convert ce_path_match() to use struct pathspec
 + struct rev_info: convert prune_data to struct pathspec
 + pathspec: add match_pathspec_depth()
 + tree_entry_interesting(): optimize wildcard matching when base is matched
 + tree_entry_interesting(): support wildcard matching
 + tree_entry_interesting(): fix depth limit with overlapping pathspecs
 + tree_entry_interesting(): support depth limit
 + tree_entry_interesting(): refactor into separate smaller functions
 + diff-tree: convert base+baselen to writable strbuf
 + glossary: define pathspec
 + Move tree_entry_interesting() to tree-walk.c and export it
 + tree_entry_interesting(): remove dependency on struct diff_options
 + Convert struct diff_options to use struct pathspec
 + diff-no-index: use diff_tree_setup_paths()
 + Add struct pathspec
 (this branch is used by en/object-list-with-pathspec.)

* en/object-list-with-pathspec (2010-09-20) 2 commits
  (merged to 'next' on 2011-01-24 at 134f65c)
 + Add testcases showing how pathspecs are handled with rev-list --objects
 + Make rev-list --objects work together with pathspecs
 (this branch uses nd/struct-pathspec.)

I've been toying with the above two topics and am reasonably happy.
I suspect that there could be (and probably need to be) further
consolidation of the two remaining pathspec API, but this seems to
be already usable.

* tr/merge-unborn-clobber (2010-08-22) 1 commit
 - Exhibit merge bug that clobbers index&WT

* ab/i18n (2010-10-07) 161 commits
 - po/de.po: complete German translation
 - po/sv.po: add Swedish translation
 - gettextize: git-bisect bisect_next_check "You need to" message
 - gettextize: git-bisect [Y/n] messages
 - gettextize: git-bisect bisect_replay + $1 messages
 - gettextize: git-bisect bisect_reset + $1 messages
 - gettextize: git-bisect bisect_run + $@ messages
 - gettextize: git-bisect die + eval_gettext messages
 - gettextize: git-bisect die + gettext messages
 - gettextize: git-bisect echo + eval_gettext message
 - gettextize: git-bisect echo + gettext messages
 - gettextize: git-bisect gettext + echo message
 - gettextize: git-bisect add git-sh-i18n
 - gettextize: git-stash drop_stash say/die messages
 - gettextize: git-stash "unknown option" message
 - gettextize: git-stash die + eval_gettext $1 messages
 - gettextize: git-stash die + eval_gettext $* messages
 - gettextize: git-stash die + eval_gettext messages
 - gettextize: git-stash die + gettext messages
 - gettextize: git-stash say + gettext messages
 - gettextize: git-stash echo + gettext message
 - gettextize: git-stash add git-sh-i18n
 - gettextize: git-submodule "blob" and "submodule" messages
 - gettextize: git-submodule "path not initialized" message
 - gettextize: git-submodule "[...] path is ignored" message
 - gettextize: git-submodule "Entering [...]" message
 - gettextize: git-submodule $errmsg messages
 - gettextize: git-submodule "Submodule change[...]" messages
 - gettextize: git-submodule "cached cannot be used" message
 - gettextize: git-submodule $update_module say + die messages
 - gettextize: git-submodule die + eval_gettext messages
 - gettextize: git-submodule say + eval_gettext messages
 - gettextize: git-submodule echo + eval_gettext messages
 - gettextize: git-submodule add git-sh-i18n
 - gettextize: git-pull "rebase against" / "merge with" messages
 - gettextize: git-pull "[...] not currently on a branch" message
 - gettextize: git-pull "You asked to pull" message
 - gettextize: git-pull split up "no candidate" message
 - gettextize: git-pull eval_gettext + warning message
 - gettextize: git-pull eval_gettext + die message
 - gettextize: git-pull die messages
 - gettextize: git-pull add git-sh-i18n
 - gettext docs: add "Testing marked strings" section to po/README
 - gettext docs: the Git::I18N Perl interface
 - gettext docs: the git-sh-i18n.sh Shell interface
 - gettext docs: the gettext.h C interface
 - gettext docs: add "Marking strings for translation" section in po/README
 - gettext docs: add a "Testing your changes" section to po/README
 - po/pl.po: add Polish translation
 - po/hi.po: add Hindi Translation
 - po/en_GB.po: add British English translation
 - po/de.po: add German translation
 - Makefile: only add gettext tests on XGETTEXT_INCLUDE_TESTS=YesPlease
 - gettext docs: add po/README file documenting Git's gettext
 - gettextize: git-am printf(1) message to eval_gettext
 - gettextize: git-am core say messages
 - gettextize: git-am "Apply?" message
 - gettextize: git-am clean_abort messages
 - gettextize: git-am cannot_fallback messages
 - gettextize: git-am die messages
 - gettextize: git-am eval_gettext messages
 - gettextize: git-am multi-line getttext $msg; echo
 - gettextize: git-am one-line gettext $msg; echo
 - gettextize: git-am add git-sh-i18n
 - gettext tests: add GETTEXT_POISON tests for shell scripts
 - gettext tests: add GETTEXT_POISON support for shell scripts
 - Makefile: MSGFMT="msgfmt --check" under GNU_GETTEXT
 - Makefile: add GNU_GETTEXT, set when we expect GNU gettext
 - gettextize: git-shortlog basic messages
 - gettextize: git-revert split up "could not revert/apply" message
 - gettextize: git-revert literal "me" messages
 - gettextize: git-revert "Your local changes" message
 - gettextize: git-revert basic messages
 - gettextize: git-notes "Refusing to %s notes in %s" message
 - gettextize: git-notes GIT_NOTES_REWRITE_MODE error message
 - gettextize: git-notes basic commands
 - gettextize: git-gc "Auto packing the repository" message
 - gettextize: git-gc basic messages
 - gettextize: git-describe basic messages
 - gettextize: git-clean clean.requireForce messages
 - gettextize: git-clean basic messages
 - gettextize: git-bundle basic messages
 - gettextize: git-archive basic messages
 - gettextize: git-status "renamed: " message
 - gettextize: git-status "Initial commit" message
 - gettextize: git-status "Changes to be committed" message
 - gettextize: git-status shortstatus messages
 - gettextize: git-status "nothing to commit" messages
 - gettextize: git-status basic messages
 - gettextize: git-push "prevent you from losing" message
 - gettextize: git-push basic messages
 - gettextize: git-tag tag_template message
 - gettextize: git-tag basic messages
 - gettextize: git-reset "Unstaged changes after reset" message
 - gettextize: git-reset reset_type_names messages
 - gettextize: git-reset basic messages
 - gettextize: git-rm basic messages
 - gettextize: git-mv "bad" messages
 - gettextize: git-mv basic messages
 - gettextize: git-merge "Wonderful" message
 - gettextize: git-merge "You have not concluded your merge" messages
 - gettextize: git-merge "Updating %s..%s" message
 - gettextize: git-merge basic messages
 - gettextize: git-log "--OPT does not make sense" messages
 - gettextize: git-log basic messages
 - gettextize: git-grep "--open-files-in-pager" message
 - gettextize: git-grep basic messages
 - gettextize: git-fetch split up "(non-fast-forward)" message
 - gettextize: git-fetch update_local_ref messages
 - gettextize: git-fetch formatting messages
 - gettextize: git-fetch basic messages
 - gettextize: git-diff basic messages
 - gettextize: git-commit advice messages
 - gettextize: git-commit "enter the commit message" message
 - gettextize: git-commit print_summary messages
 - gettextize: git-commit formatting messages
 - gettextize: git-commit "middle of a merge" message
 - gettextize: git-commit basic messages
 - gettextize: git-checkout "Switched to a .. branch" message
 - gettextize: git-checkout "HEAD is now at" message
 - gettextize: git-checkout describe_detached_head messages
 - gettextize: git-checkout: our/their version message
 - gettextize: git-checkout basic messages
 - gettextize: git-branch "(no branch)" message
 - gettextize: git-branch "git branch -v" messages
 - gettextize: git-branch "Deleted branch [...]" message
 - gettextize: git-branch "remote branch '%s' not found" message
 - gettextize: git-branch basic messages
 - gettextize: git-add refresh_index message
 - gettextize: git-add "remove '%s'" message
 - gettextize: git-add "pathspec [...] did not match" message
 - gettextize: git-add "Use -f if you really want" message
 - gettextize: git-add "no files added" message
 - gettextize: git-add basic messages
 - gettextize: git-clone "Cloning into" message
 - gettextize: git-clone basic messages
 - gettext tests: test message re-encoding under C
 - po/is.po: add Icelandic translation
 - gettext tests: mark a test message as not needing translation
 - gettext tests: test re-encoding with a UTF-8 msgid under Shell
 - gettext tests: test message re-encoding under Shell
 - gettext tests: add detection for is_IS.ISO-8859-1 locale
 - gettext tests: test if $VERSION exists before using it
 - gettextize: git-init "Initialized [...] repository" message
 - gettextize: git-init basic messages
 - gettext tests: skip lib-gettext.sh tests under GETTEXT_POISON
 - gettext tests: add GETTEXT_POISON=YesPlease Makefile parameter
 - gettext.c: use libcharset.h instead of langinfo.h when available
 - gettext.c: work around us not using setlocale(LC_CTYPE, "")
 - builtin.h: Include gettext.h
 - Makefile: use variables and shorter lines for xgettext
 - Makefile: tell xgettext(1) that our source is in UTF-8
 - Makefile: provide a --msgid-bugs-address to xgettext(1)
 - Makefile: A variable for options used by xgettext(1) calls
 - gettext tests: locate i18n lib&data correctly under --valgrind
 - gettext: setlocale(LC_CTYPE, "") breaks Git's C function assumptions
 - gettext tests: rename test to work around GNU gettext bug
 - gettext: add infrastructure for translating Git with gettext
 - builtin: use builtin.h for all builtin commands
 - tests: use test_cmp instead of piping to diff(1)
 - t7004-tag.sh: re-arrange git tag comment for clarity

It is getting ridiculously painful to keep re-resolving the conflicts with
other topics in flight, even with the help with rerere.

Needs a bit more minor work to get the basic code structure right.

Planning for 1.7.5 and 1.8.0

[Junio C Hamano]

Now the 1.7.4 release is out, I'd like people to help thinking about the
next cycle(s).

As a discussion-starter, here are my random wishes.  Even though this does
not attempt to be exhaustive, keeping the number of goals manageably small
may help us focus.

 * The i18n effort Ævar Arnfjörð Bjarmason started two cycles ago has
   stalled. If enough people feel i18n's Porcelain UI is worth having, I
   think we would need a brief calming period in the entire tree in order
   for us to get the minimum support (definition of _() macro that is
   empty is a good start) and _() mark-up of existing strings in, and then
   ask everybody to rebase their ongoing work on top of it.

 * There was a discussion on documentation updates to reduce "here we tell
   you only the basics; see elsewhere for details", and consolidating the
   description of configuration options in one place.

 * Nguyễn has been scratching my longstanding itch by attempting to unify
   two pathspec semantics (the ones based on tree-diff matches only
   leading path while others know globs) to reduce inconsistencies. I
   would really want to see this polished and in the main release.

 * Elijah's fix to "rev-list --objects", together with the updated
   pathspec semantics will allow us to cleanly implement narrow cloning
   (possibly deprecating and replacing the narrow checkout in the
   future).  I am hoping that we can lay groundwork on this inside one
   cycle and the initial end-to-end implementation in another.

 * Shawn Pearce says that the diff implementation JGit uses (histogram
   diff) performs way better than the xdiff implementation we use by
   default. It would be great if somebody can spend time taking a look at
   it and possibly port it back to C-git.


Over the time we have discussed minor glitches and inconsistencies that we
all (or at least most of us anyway) agreed we would have done differently
if we were writing Git from scratch, yet we cannot retroactively introduce
differences not to harm existing users.  We may also want to revisit these
discussions during this round--if there are reasonable number of them that
we can agree the benefit of tweaked semantics/behaviour outweighs the risk
of breaking and having to update ancient scripts that exploited obscure
corner case behaviour of Git, we would want warn the users loudly, bite
the bullet and break them so that we can move forward.  We would however
need to roll such potentially disruptive changes into a big single cycle,
like we did in 1.6.0.

I'll follow-up this message with a couple of example proposals.  Please
send your own, imitating the format of the message, as a reply to this
message.  Do not forget to retitle your message when you do so (iow, I
don't want to see "Re: Planning for 1.7.5 and 1.8.0").

Thanks.

[1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

To me the source tree looks highly untidy to me.

We do have subdirectories for documentation, tests, contributions, etc.  
But a sizeable part of the tree is just a big splat of source files 
dumped right in the root of the tree.

So I'd suggest doing the following:

1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
   the builtin directory from the root directory to it.

2) Create a build/ directory, or bin/ if prefered, to hold the result of 
   the build.

3) Consider dropping the ppc/ directory.  Unless someone really cares 
   deeply, it would be nice to simply always use the block-sha1 code and 
   move it straight into src/.

4) Consider moving some more directories into src/ such as xdiff/.
   I'd leave compat/ outside src/ to make it more explicit that this is 
   not about Git proper.

5) Rename t/ to testsuite/ so this doesn't look like some garbage 
   leftover.

6) And fix up all the Makefiles to cope with the above movements.

What do you think?


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Nicolas Pitre <nico@fluxnic.net> writes:

> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
>    the builtin directory from the root directory to it.
>
> 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
>    the build.
>...
> 6) And fix up all the Makefiles to cope with the above movements.
>
> What do you think?

Knee-jerk reaction: not very motivated to make the top-level directory
just a skeleton that holds various directories with a handful of
administrative files like Makefile, README, etc.  Under your proposal, the
bulk of the current content at the top would simply move to another single
directory anyway, so I don't immediately see much point of such a move,
other than adding merge burden on me and rebase burden on others, that
is.

But that is just a knee-jerk reaction, just to fill the "Risks:" section
you didn't fill.  Your missing "Migration Plans" section might outline a
clever approach to lessen the interim hurt while merging in-flight topics.

Re: [1.8.0] reorganize the mess that the source tree has become

[Matthieu Moy]

Junio C Hamano <gitster@pobox.com> writes:

> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc.  Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,

There would be at least one obvious benefit: currently, we have this

git$ ls | wc -l
623

(that's after a build)

It's a bit hard to find the interesting bits (README, Documentation/,
contrib/ for example) in the output of "ls".

> other than adding merge burden on me and rebase burden on others, that
> is.

That can be seen as a test of how good Git is at bulk rename
management ;-).

All that said, I cannot really say whether the benefit is higher than
the cost.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Matthieu Moy wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > other than adding merge burden on me and rebase burden on others, that
> > is.
> 
> That can be seen as a test of how good Git is at bulk rename
> management ;-).
> 
> All that said, I cannot really say whether the benefit is higher than
> the cost.

There is a huge value in inflicting on ourselves such a test case for 
the tool we produce.  That helps avoiding the ivory tower syndrome.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Junio C Hamano wrote:

> Nicolas Pitre <nico@fluxnic.net> writes:
> 
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
> >    the builtin directory from the root directory to it.
> >
> > 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
> >    the build.
> >...
> > 6) And fix up all the Makefiles to cope with the above movements.
> >
> > What do you think?
> 
> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc.  Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,
> other than adding merge burden on me and rebase burden on others, that
> is.

I really think that the top directory is not the proper place for source 
files to live, especially considering how big a project Git is now.  
The top directory should be like a table of content and not the content 
itself.

But if you the maintainer doesn't see a long-term value in this to be 
greater than the one-time burden, then I'm afraid there's nothing I can 
do to help it.

> But that is just a knee-jerk reaction, just to fill the "Risks:" section
> you didn't fill.  Your missing "Migration Plans" section might outline a
> clever approach to lessen the interim hurt while merging in-flight topics.

Well, there is no such plan.  Given that 1.8 is meant to be an inflexion 
point for users, it could as well be for developers the best time to 
clean up this mess too.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:

> We do have subdirectories for documentation, tests, contributions, etc.  
> But a sizeable part of the tree is just a big splat of source files 
> dumped right in the root of the tree.
> 
> So I'd suggest doing the following:
> 
> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
>    the builtin directory from the root directory to it.

Wouldn't this just be the same giant splat of source files, but in a
different tree? I don't really see the advantage, and it seems like an
extra annoyance. Besides being just one more directory to go up and
down, it does make history browsing more annoying. As much as I love
git's "don't record renames" philosophy, our handling of renames on the
viewing side is often annoying. I already get annoyed sometimes
following stuff across the s!builtin-!builtin/! change. This would be
like that but more so.

Or maybe it is a good thing for that reason, as we will eat our own
rename dogfood. :)

> 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
>    leftover.

Ugh, more typing. :P

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
> 
> > We do have subdirectories for documentation, tests, contributions, etc.  
> > But a sizeable part of the tree is just a big splat of source files 
> > dumped right in the root of the tree.
> > 
> > So I'd suggest doing the following:
> > 
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
> >    the builtin directory from the root directory to it.
> 
> Wouldn't this just be the same giant splat of source files, but in a
> different tree? I don't really see the advantage, and it seems like an
> extra annoyance. 

Like I said to Junio, if you don't see the advantage, there's nothing I 
can do for you.  To me this is simple good source code hygiene.

> Besides being just one more directory to go up and down, it does make 
> history browsing more annoying. As much as I love git's "don't record 
> renames" philosophy, our handling of renames on the viewing side is 
> often annoying. I already get annoyed sometimes following stuff across 
> the s!builtin-!builtin/! change. This would be like that but more so.

So... we do suck at something?  So why not take this opportunity to 
shake yourself out of this easy comfort and improve Git as a result on 
both front?  :-)

> Or maybe it is a good thing for that reason, as we will eat our own
> rename dogfood. :)

Exactly!  And maybe we'll make Git even more useful in the process.

> > 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
> >    leftover.
> 
> Ugh, more typing. :P

Come on!  You sound like an old fart now!  ;-)


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Nicolas Pitre <nico@fluxnic.net> writes:

> On Mon, 31 Jan 2011, Jeff King wrote:
>
>> Besides being just one more directory to go up and down, it does make 
>> history browsing more annoying. As much as I love git's "don't record 
>> renames" philosophy, our handling of renames on the viewing side is 
>> often annoying. I already get annoyed sometimes following stuff across 
>> the s!builtin-!builtin/! change. This would be like that but more so.
>
> So... we do suck at something?  So why not take this opportunity to 
> shake yourself out of this easy comfort and improve Git as a result on 
> both front?  :-)
>
>> Or maybe it is a good thing for that reason, as we will eat our own
>> rename dogfood. :)
>
> Exactly!  And maybe we'll make Git even more useful in the process.

This part I _could_ actually buy; even though I do not think moving files
without much reason is a good project hygine, it does happen in real life,
and we would want to keep things smooth for real people.

>> > 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
>> >    leftover.

I am not sure about this "t/" vs "testsuite/".

>> Ugh, more typing. :P
>
> Come on!  You sound like an old fart now!  ;-)

If we make the top-level directory lean enough, we probably can tab
complete after typing just "cd t" to go to testsuite/ or tests/ or
whatever you come up with, so "more typing" is not a huge issue to me
personally.

I however think the directory name "t/" is not our invention but what we
took from somebody else (perhaps Perl?), and I suspect some people expect
to find tests under there since we have had them there for a long time.

Re: [1.8.0] reorganize the mess that the source tree has become

[João P. Sampaio]

I think a good code hygiene is important, and some suggestions here
are relevant:

1) I also think there should be a directory for the source code
(namely, src/), and the top level should act as a table of contents.
As a newcomer myself who's trying to grasp Git, I can say an organized
project makes people more inclined join. Therefore, directories should
be named as clearly as possible: see item 2;

2) For item 1, t/ should be renamed to testsuite/. As Junio said, if
we get a more organized project, people could just "cd t" and tab to
autocomplete, or "cd tes" at the worst scenario, which is not such a
big hassle. About people expecting the testsuite to be inside t/, once
they type "cd t" and get an error, most people would look for an
alternative and eventually find the correct folder, or tabbing would
just suggest the name.

3) The top level should hold files that point people towards where
they want to go, helpful files like README and even some
Documentation/ files could get a promotion.

3) As 1.8.0 can be an inflexion for users, so could be for the
developers as well.

-- 
João Paulo Melo de Sampaio
Computer Engineering Student @ UFSCar
Website: http://www.jpmelos.com
Twitter: twitter.com/jpmelos (@jpmelos)

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Junio C Hamano wrote:

> If we make the top-level directory lean enough, we probably can tab
> complete after typing just "cd t" to go to testsuite/ or tests/ or
> whatever you come up with, so "more typing" is not a huge issue to me
> personally.
> 
> I however think the directory name "t/" is not our invention but what we
> took from somebody else (perhaps Perl?), and I suspect some people expect
> to find tests under there since we have had them there for a long time.

If those people are not able to figure out that "testsuite" means where 
tests are, especially within a lean top directory, then we might 
question the reliability of the tests they might contribute.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:

> > Besides being just one more directory to go up and down, it does make 
> > history browsing more annoying. As much as I love git's "don't record 
> > renames" philosophy, our handling of renames on the viewing side is 
> > often annoying. I already get annoyed sometimes following stuff across 
> > the s!builtin-!builtin/! change. This would be like that but more so.
> 
> So... we do suck at something?  So why not take this opportunity to 
> shake yourself out of this easy comfort and improve Git as a result on 
> both front?  :-)

Yes, we do suck at rename following. The problem is that it is partially
an implementation issue, and partially a fundamental issue. Obviously
--follow sucks pretty hard right now, and that could be fixed. Namely it
follows only a single file, and it interacts very badly with history
simplification.

But even with those things fixed, there will still be annoyances.

It will still be _slower_ to turn it on all the time, for one[1]. And
that's due to fundamental design decisions of the git data structure.
And I'm not knocking those decisions; I think they made the right
tradeoff. But that doesn't mean we don't pay the cost for that tradeoff.

And no matter what your model, renames can be annoying. On-going topics
will have a painful rebase or merge. And people looking at history will
have to deal with the code-base having different names at different
points. Yeah, you can say it's all just "content", but the filenames we
put things in are actually useful.

So I don't think it's wrong to say "renames are a pain, and so should
not be done lightly". I do think it's wrong to say "renames can't be
done"; I just the cost needs to be considered.

-Peff

[1] I'd be interested to see how much we can get around that slowness
using a notes-cache.

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:
> 
> > So... we do suck at something?  So why not take this opportunity to 
> > shake yourself out of this easy comfort and improve Git as a result on 
> > both front?  :-)
> 
> Yes, we do suck at rename following. The problem is that it is partially
> an implementation issue, and partially a fundamental issue. Obviously
> --follow sucks pretty hard right now, and that could be fixed. Namely it
> follows only a single file, and it interacts very badly with history
> simplification.

This is no excuse not to do proper source tree reorganization.

Telling people not to move files around because Git sucks at tracking 
them is also the wrong answer.

> But even with those things fixed, there will still be annoyances.
> 
> It will still be _slower_ to turn it on all the time, for one[1]. And
> that's due to fundamental design decisions of the git data structure.
> And I'm not knocking those decisions; I think they made the right
> tradeoff. But that doesn't mean we don't pay the cost for that tradeoff.

I agree.  However sitting on our back and resisting a cleanup just 
because our very tool does poorly in that scenario is just like putting 
our heads in the sand and pretend that the problem doesn't exist.  
Better do what most people without internal knowledge of Git would do 
and just clean up the tree, and then benefit from this extraordinary 
opportunity of having this environment right at home where Git 
developers have a much greater incentive to work on this issue and 
improve things.

> And no matter what your model, renames can be annoying. On-going topics
> will have a painful rebase or merge. And people looking at history will
> have to deal with the code-base having different names at different
> points. Yeah, you can say it's all just "content", but the filenames we
> put things in are actually useful.

Of course.  But such is life.  Many projects out there are just like 
that, and facing this situation ourselves will just help us figure out 
ways to make Git even more useful to more people.

> So I don't think it's wrong to say "renames are a pain, and so should
> not be done lightly".

I disagree.  This is like saying: "renames are not well supported, so 
let's avoid them while using Git."  People used to say that of merges 
with CVS.  Are we going to follow suit de facto?  Imagine the Git 
detractors taking our source tree mess to exemplify this Git flaw since 
"Git developers themselves are unwilling to move files around because 
Git sucks at it".

> I do think it's wrong to say "renames can't be
> done"; I just the cost needs to be considered.

Instead, why not saying: "Rename tracking is not as optimal as it could 
be, so let's work it out." ?


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:

> > Yes, we do suck at rename following. The problem is that it is partially
> [...]
> This is no excuse not to do proper source tree reorganization.

I think this is the crux of our disagreement. I don't agree that your
proposal is any way more "proper" than what is there now. Leaving the
rename issue aside (i.e., if we were starting a new project), I would
still be slightly against a src/ directory. I find them annoying.

But I don't care _that_ much, and I would rather not waste either of our
time debating it more. I would much rather you spend your time on
pack v4. :)

> I disagree.  This is like saying: "renames are not well supported, so 
> let's avoid them while using Git."  People used to say that of merges 
> with CVS.  Are we going to follow suit de facto?  Imagine the Git 
> detractors taking our source tree mess to exemplify this Git flaw since 
> "Git developers themselves are unwilling to move files around because 
> Git sucks at it".

For the record, part of my argument was that renaming is annoying to
some degree in _all_ systems, not just git.

> > I do think it's wrong to say "renames can't be
> > done"; I just the cost needs to be considered.
> 
> Instead, why not saying: "Rename tracking is not as optimal as it could 
> be, so let's work it out." ?

I did also say that. :)

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:
> 
> > This is no excuse not to do proper source tree reorganization.
> 
> I think this is the crux of our disagreement. I don't agree that your
> proposal is any way more "proper" than what is there now. Leaving the
> rename issue aside (i.e., if we were starting a new project), I would
> still be slightly against a src/ directory. I find them annoying.

Let's agree to disagree then.  What I see in the root of the Git source 
tree is a huge clutter of source files, binary files, scripts, and 
subdirectories all mixed together.  If you know by hart where things are 
because you've been hacking on them for the last 5 years then of course 
you might not see the point.  But since I didn't work much on Git 
lately, things are not as obvious to me as they used to be.  Looking 
back at it now with some distance, this tree looks like a mess and it is 
really annoying to work with.

> But I don't care _that_ much, and I would rather not waste either of our
> time debating it more. I would much rather you spend your time on
> pack v4. :)

I wish... I wish.  But I have a plan which might involve taking some 
vacation from $day_job in the Caribbeans with $wife and no kids, where 
$wife is going to do scuba diving with her club mates while I'll be 
alone with a laptop and no net connection and therefore nothing else to 
do for a week.  I've been craving for such free time for quite a while 
now.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Thomas Rast]

Nicolas Pitre wrote:
> What I see in the root of the Git source 
> tree is a huge clutter of source files, binary files, scripts, and 
> subdirectories all mixed together.  If you know by hart where things are 
> because you've been hacking on them for the last 5 years then of course 
> you might not see the point.  But since I didn't work much on Git 
> lately, things are not as obvious to me as they used to be.  Looking 
> back at it now with some distance, this tree looks like a mess and it is 
> really annoying to work with.

But judging by that assessment, shouldn't we strive to make it
*easier* to find things?

In particular a prospective git hacker would not care whether
something is a source file or a script (you seem to imply the
opposite).  He would instead expect to find git-foo implemented in
something named of that sort, so we could probably help him by mapping

  git-foo.sh      ->   git-foo.sh
  builtin/bar.c   ->   git-bar.c
  baz.c           ->   lib/baz.c
  baz.o           ->   build/baz.o (or whatever, just elsewhere)
  baz.gcov        ->   build/baz.gcov (ditto)


(I'm no huge fan of src/ either, but this should be orthogonal.)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: [1.8.0] reorganize the mess that the source tree has become

[Jonathan Nieder]

Thomas Rast wrote:

> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect to find git-foo implemented in
> something named of that sort, so we could probably help him by mapping
> 
>   git-foo.sh      ->   git-foo.sh
>   builtin/bar.c   ->   git-bar.c
>   baz.c           ->   lib/baz.c

I agree.  This sets off my "time to resist change" alarms much
less than "git mv *.c *.sh src/", for what it's worth.

>   baz.o           ->   build/baz.o (or whatever, just elsewhere)
>   baz.gcov        ->   build/baz.gcov (ditto)

Maybe something like this to start?

-- 8< --
Subject: Makefile: basic support for separate build dir

 - python and perl machinery haven't been tweaked yet
 - requires good VPATH support
 - relies on COMPUTE_HEADER_DIRECTORIES to make the object file
   directories
 - does not support paths with spaces

Usage:

	mkdir output
	cd output
	echo COMPUTE_HEADER_DIRECTORIES=1 >config.mak
	make -f ../Makefile GIT_SRC=$(pwd)/../ -j2

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 Makefile            |   41 ++++++++++++++++++++++++++---------------
 generate-cmdlist.sh |    4 ++--
 perl/Makefile       |    2 +-
 3 files changed, 29 insertions(+), 18 deletions(-)

diff --git a/Makefile b/Makefile
index 775ee83..b258a24 100644
--- a/Makefile
+++ b/Makefile
@@ -243,8 +243,18 @@ all::
 #
 # Define NATIVE_CRLF if your platform uses CRLF for line endings.
 
+# Absolute path to the toplevel of the git sources, with trailing /.
+# Leave empty for an in-place build.
+GIT_SRC =
+ifdef GIT_SRC
+	NO_PYTHON = YesPlease
+	NO_PERL_MAKEMAKER = YesPlease
+endif
+
+VPATH := $(if $(GIT_SRC),$(GIT_SRC),$(CURDIR))
+
 GIT-VERSION-FILE: FORCE
-	@$(SHELL_PATH) ./GIT-VERSION-GEN
+	@$(SHELL_PATH) $(GIT_SRC)/GIT-VERSION-GEN
 -include GIT-VERSION-FILE
 
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
@@ -305,6 +315,7 @@ lib = lib
 pathsep = :
 
 export prefix bindir sharedir sysconfdir gitwebdir
+export GIT_SRC
 
 CC = gcc
 AR = ar
@@ -333,7 +344,7 @@ SPARSE_FLAGS = -D__BIG_ENDIAN__ -D__powerpc__
 # Those must not be GNU-specific; they are shared with perl/ which may
 # be built by a different compiler. (Note that this is an artifact now
 # but it still might be nice to keep that distinction.)
-BASIC_CFLAGS = -I.
+BASIC_CFLAGS = -I$(GIT_SRC). -I.
 BASIC_LDFLAGS =
 
 # Guard against environment variables
@@ -1562,7 +1573,7 @@ ifeq ($(PYTHON_PATH),)
 NO_PYTHON=NoThanks
 endif
 
-QUIET_SUBDIR0  = +$(MAKE) -C # space to separate -C and subdir
+QUIET_SUBDIR0  = +$(MAKE) -C $(GIT_SRC)# no space before subdir
 QUIET_SUBDIR1  =
 
 ifneq ($(findstring $(MAKEFLAGS),w),w)
@@ -1582,7 +1593,7 @@ ifndef V
 	QUIET_GCOV     = @echo '   ' GCOV $@;
 	QUIET_SUBDIR0  = +@subdir=
 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
-			 $(MAKE) $(PRINT_DIR) -C $$subdir
+			 $(MAKE) $(PRINT_DIR) -C $(GIT_SRC)$$subdir
 	export V
 	export QUIET_GEN
 	export QUIET_BUILT_IN
@@ -1696,10 +1707,10 @@ $(BUILT_INS): git$X
 	ln -s git$X $@ 2>/dev/null || \
 	cp git$X $@
 
-common-cmds.h: ./generate-cmdlist.sh command-list.txt
+common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)./generate-cmdlist.sh > $@+ && mv $@+ $@
+	$(QUIET_GEN)$(GIT_SRC)./generate-cmdlist.sh > $@+ && mv $@+ $@
 
 define cmd_munge_script
 $(RM) $@ $@+ && \
@@ -1709,7 +1720,7 @@ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
     -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
     -e 's/@@NO_CURL@@/$(NO_CURL)/g' \
     -e $(BROKEN_PATH_FIX) \
-    $@.sh >$@+
+    $(GIT_SRC)$@.sh >$@+
 endef
 
 $(patsubst %.sh,%,$(SCRIPT_SH)) : % : %.sh
@@ -1729,7 +1740,7 @@ perl/perl.mak: GIT-CFLAGS perl/Makefile perl/Makefile.PL
 
 $(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl
 	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C $(GIT_SRC)perl -s --no-print-directory instlibdir` && \
 	sed -e '1{' \
 	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
 	    -e '	h' \
@@ -1738,7 +1749,7 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl
 	    -e '	x' \
 	    -e '}' \
 	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-	    $@.perl >$@+ && \
+	    $(GIT_SRC)$@.perl >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 
@@ -1780,7 +1791,7 @@ git-instaweb: git-instaweb.sh gitweb/gitweb.cgi gitweb/static/gitweb.css gitweb/
 	    -e 's/@@NO_CURL@@/$(NO_CURL)/g' \
 	    -e 's|@@GITWEBDIR@@|$(gitwebdir_SQ)|g' \
 	    -e 's|@@PERL@@|$(PERL_PATH_SQ)|g' \
-	    $@.sh > $@+ && \
+	    $(GIT_SRC)$@.sh > $@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
@@ -1788,7 +1799,7 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
 	$(QUIET_GEN)$(RM) $@ $@+ && \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
 	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
-	    unimplemented.sh >$@+ && \
+	    $(GIT_SRC)unimplemented.sh >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 endif # NO_PERL
@@ -1803,7 +1814,7 @@ $(patsubst %.py,%,$(SCRIPT_PYTHON)): % : %.py
 	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
 	    -e 's|\(os\.getenv("GITPYTHONLIB"\)[^)]*)|\1,"@@INSTLIBDIR@@")|' \
 	    -e 's|@@INSTLIBDIR@@|'"$$INSTLIBDIR"'|g' \
-	    $@.py >$@+ && \
+	    $(GIT_SRC)$@.py >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PYTHON
@@ -1811,7 +1822,7 @@ $(patsubst %.py,%,$(SCRIPT_PYTHON)): % : unimplemented.sh
 	$(QUIET_GEN)$(RM) $@ $@+ && \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
 	    -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \
-	    unimplemented.sh >$@+ && \
+	    $(GIT_SRC)unimplemented.sh >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 endif # NO_PYTHON
@@ -2142,7 +2153,7 @@ test-%$X: test-%.o $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)
 
 check-sha1:: test-sha1$X
-	./test-sha1.sh
+	$(GIT_SRC)./test-sha1.sh
 
 check: common-cmds.h
 	if sparse; \
@@ -2229,7 +2240,7 @@ endif
 		ln -s "git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
 		cp "$$execdir/git-remote-http$X" "$$execdir/$$p" || exit; \
 	done && \
-	./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
+	$(GIT_SRC)./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
 
 install-gitweb:
 	$(MAKE) -C gitweb install
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 75c68d9..f718633 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -9,7 +9,7 @@ struct cmdname_help
 
 static struct cmdname_help common_cmds[] = {"
 
-sed -n -e 's/^git-\([^ 	]*\)[ 	].* common.*/\1/p' command-list.txt |
+sed -n -e 's/^git-\([^ 	]*\)[ 	].* common.*/\1/p' "$GIT_SRC"command-list.txt |
 sort |
 while read cmd
 do
@@ -19,6 +19,6 @@ do
             x
             s/.*git-'"$cmd"' - \(.*\)/  {"'"$cmd"'", "\1"},/
 	    p
-     }' "Documentation/git-$cmd.txt"
+     }' "${GIT_SRC}Documentation/git-$cmd.txt"
 done
 echo "};"
diff --git a/perl/Makefile b/perl/Makefile
index a2ffb64..7c3a82a 100644
--- a/perl/Makefile
+++ b/perl/Makefile
@@ -44,4 +44,4 @@ endif
 # this is just added comfort for calling make directly in perl dir
 # (even though GIT-CFLAGS aren't used yet. If ever)
 ../GIT-CFLAGS:
-	$(MAKE) -C .. GIT-CFLAGS
+	$(MAKE) -C $(GIT_SRC).. GIT-CFLAGS
-- 
1.7.2.3

Re: [1.8.0] reorganize the mess that the source tree has become

[Jonathan Nieder]

Jonathan Nieder wrote:

> 	mkdir output
> 	cd output
> 	echo COMPUTE_HEADER_DIRECTORIES=1 >config.mak
> 	make -f ../Makefile GIT_SRC=$(pwd)/../ -j2

Erm, COMPUTE_HEADER_DEPENDENCIES.  Anyway, if someone wants to make
setting GIT_SRC take care of making the directories automatically, I
wouldn't mind. ;-)

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Tue, 1 Feb 2011, Thomas Rast wrote:

> Nicolas Pitre wrote:
> > What I see in the root of the Git source 
> > tree is a huge clutter of source files, binary files, scripts, and 
> > subdirectories all mixed together.  If you know by hart where things are 
> > because you've been hacking on them for the last 5 years then of course 
> > you might not see the point.  But since I didn't work much on Git 
> > lately, things are not as obvious to me as they used to be.  Looking 
> > back at it now with some distance, this tree looks like a mess and it is 
> > really annoying to work with.
> 
> But judging by that assessment, shouldn't we strive to make it
> *easier* to find things?
> 
> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect to find git-foo implemented in
> something named of that sort, so we could probably help him by mapping
> 
>   git-foo.sh      ->   git-foo.sh
>   builtin/bar.c   ->   git-bar.c
>   baz.c           ->   lib/baz.c
>   baz.o           ->   build/baz.o (or whatever, just elsewhere)
>   baz.gcov        ->   build/baz.gcov (ditto)

I'm not proposing to go that far, especially given the current 
resistance to any changes.  IMHO anything that unclutters the top 
directory is good.

Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Nguyen Thai Ngoc Duy]

On Tue, Feb 1, 2011 at 7:42 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> Nicolas Pitre wrote:
>> What I see in the root of the Git source
>> tree is a huge clutter of source files, binary files, scripts, and
>> subdirectories all mixed together.  If you know by hart where things are
>> because you've been hacking on them for the last 5 years then of course
>> you might not see the point.  But since I didn't work much on Git
>> lately, things are not as obvious to me as they used to be.  Looking
>> back at it now with some distance, this tree looks like a mess and it is
>> really annoying to work with.
>
> But judging by that assessment, shouldn't we strive to make it
> *easier* to find things?
>
> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect ...

A hacker is expected to RTFM first, IMO. Put up a document describing
how things are organized in git and we're good. git-grep will take
care from there.
-- 
Duy

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:
>
>> > Yes, we do suck at rename following. The problem is that it is partially
>> [...]
>> This is no excuse not to do proper source tree reorganization.
>
> I think this is the crux of our disagreement. I don't agree that your
> proposal is any way more "proper" than what is there now. Leaving the
> rename issue aside (i.e., if we were starting a new project), I would
> still be slightly against a src/ directory. I find them annoying.
>
> But I don't care _that_ much, and I would rather not waste either of our
> time debating it more. I would much rather you spend your time on
> pack v4. :)

I am with you, both counts on this topic.  I don't necessarily agree that
having sources at the top-level is bad, I don't want to see Nico wasting
his time arguing, and I do see some value in the proposal that gives us an
opportunity for dogfooding (but we already have done so with builtin/ and
it was not all that annoying---I think the timing was rather good and the
tree was semi-quiescent).

Ehh, that makes it not "both" but "two and half" counts ;-).

As long as the new directories are named sanely (one of the things I
detest is abbreviated uppercase silliness like "Src", "Lib"), I am fine
with the proposal.  Also I have a mild preference to keep build-products
next to the source (i.e. no separate "obj" directory).

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Tue, Feb 1, 2011 at 12:12 AM, Jeff King <peff@peff.net> wrote:
> On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:
>
>> > Besides being just one more directory to go up and down, it does make
>> > history browsing more annoying. As much as I love git's "don't record
>> > renames" philosophy, our handling of renames on the viewing side is
>> > often annoying. I already get annoyed sometimes following stuff across
>> > the s!builtin-!builtin/! change. This would be like that but more so.
>>
>> So... we do suck at something?  So why not take this opportunity to
>> shake yourself out of this easy comfort and improve Git as a result on
>> both front?  :-)
>
> Yes, we do suck at rename following. The problem is that it is partially
> an implementation issue, and partially a fundamental issue. Obviously
> --follow sucks pretty hard right now, and that could be fixed. Namely it
> follows only a single file, and it interacts very badly with history
> simplification.
>
> But even with those things fixed, there will still be annoyances.
>
> It will still be _slower_ to turn it on all the time, for one[1]. And
> that's due to fundamental design decisions of the git data structure.

Does it really have to be? I mean, for whole-file rename-detection, if
we say that we automatically enable rename-detection (by default) as
we reach the first commit that doesn't have a given tree-entry, then
it would only kick in as we're about to terminate the log-output. And
since we're usually reading through a pager, it should means that it
takes a little bit more time before the user knows he's at the end of
the log. It shouldn't really affect the throughput of the data before
the point that becomes an annoyance, right?

At least that's the part that I find the most annoying with the
current rename detection; having to enable the flag as I reach the end
of the history for a file, often having to search through a lot of
commits again.

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Tue, Feb 01, 2011 at 01:35:16AM +0100, Erik Faye-Lund wrote:

> > It will still be _slower_ to turn it on all the time, for one[1]. And
> > that's due to fundamental design decisions of the git data structure.
> 
> Does it really have to be? I mean, for whole-file rename-detection, if
> we say that we automatically enable rename-detection (by default) as
> we reach the first commit that doesn't have a given tree-entry, then
> it would only kick in as we're about to terminate the log-output. And
> since we're usually reading through a pager, it should means that it
> takes a little bit more time before the user knows he's at the end of
> the log. It shouldn't really affect the throughput of the data before
> the point that becomes an annoyance, right?

That's not quite true. You may be following not just a single file, but
some arbitrary pathspec. Any time there is a file creation event that
matches that pathspec, you will want to rename-detect any possible
sources for that created file, and add them to the list of interesting
paths.

But yeah, we don't have to do rename detection for every single case. So
the slowness may turn out to be not that bad. When I had
arbitrary-pathspec following this summer I thought I did some numbers,
but I don't remember the results and I can't find them on the list.

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Sverre Rabbelier]

Heya,

On Tue, Feb 1, 2011 at 00:12, Jeff King <peff@peff.net> wrote:
> [1] I'd be interested to see how much we can get around that slowness
> using a notes-cache.

Do you mean something like a refs/notes/renames namespace in which we
stick notes on commits indicating that a rename indicated at that
commit, with an option of the user, after-the-fact, adding this
information manually?

-- 
Cheers,

Sverre Rabbelier

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Tue, Feb 01, 2011 at 02:00:05AM +0100, Sverre Rabbelier wrote:

> Heya,
> 
> On Tue, Feb 1, 2011 at 00:12, Jeff King <peff@peff.net> wrote:
> > [1] I'd be interested to see how much we can get around that slowness
> > using a notes-cache.
> 
> Do you mean something like a refs/notes/renames namespace in which we
> stick notes on commits indicating that a rename indicated at that
> commit, with an option of the user, after-the-fact, adding this
> information manually?

Yes, without the "option of the user..." bit. Basically just cache the
list of renames for a given commit against its parent (which should be
immutable[1]), under the assumption that it is cheaper to look up the
note than it is to calculate the renames.

But I would make it purely a cache, not some place for users to stick
their own generated rename information (if people really want to do
that, I would much rather see it go into the commit itself as a
pseudo-header).

-Peff

[1] It's technically not immutable if you limit the pathspec, or if you
have non-standard rename options. But you could define some "canonical"
rename set, like all of the pairs found doing rename detection with -M90
when considering the whole set of removed files as sources and added
files as destinations. That would cover the common case of people
running "git log", and then more specialized detection would not use the
cache.

Re: [1.8.0] reorganize the mess that the source tree has become

[Jay Soffian]

On Mon, Jan 31, 2011 at 6:12 PM, Jeff King <peff@peff.net> wrote:
> And no matter what your model, renames can be annoying. On-going topics
> will have a painful rebase or merge. And people looking at history will
> have to deal with the code-base having different names at different
> points. Yeah, you can say it's all just "content", but the filenames we
> put things in are actually useful.

I have been dealing with this quite a bit lately as Chromium has been
doing mass renaming. It's no small project and sometimes those merges
are big:

[diff]
	renames = copies
	renameLimit = 2000

:-)

What I can say is, yes, it's annoying, but also: git does quite a
decent job of it. I've found myself having to do this occasionally,
but not too often:

  git diff ...MERGE_HEAD -- /path/to/old/name | patch /path/to/new/name

(Typically when a header is renamed, a stub is left in place at the
old name which just includes the new name, and the local changes don't
come across to the new name 100% cleanly.)

Anyway, I would welcome git.git getting a little taste of that. :-) :-) :-)

j.

Re: [1.8.0] reorganize the mess that the source tree has become

[Andreas Ericsson]

On 01/31/2011 10:28 PM, Nicolas Pitre wrote:
> On Mon, 31 Jan 2011, Jeff King wrote:
> 
>> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
>>
>>> We do have subdirectories for documentation, tests, contributions, etc.
>>> But a sizeable part of the tree is just a big splat of source files
>>> dumped right in the root of the tree.
>>>
>>> So I'd suggest doing the following:
>>>
>>> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
>>>     the builtin directory from the root directory to it.
>>
>> Wouldn't this just be the same giant splat of source files, but in a
>> different tree? I don't really see the advantage, and it seems like an
>> extra annoyance.
> 
> Like I said to Junio, if you don't see the advantage, there's nothing I
> can do for you.  To me this is simple good source code hygiene.
> 
>> Besides being just one more directory to go up and down, it does make
>> history browsing more annoying. As much as I love git's "don't record
>> renames" philosophy, our handling of renames on the viewing side is
>> often annoying. I already get annoyed sometimes following stuff across
>> the s!builtin-!builtin/! change. This would be like that but more so.
> 
> So... we do suck at something?  So why not take this opportunity to
> shake yourself out of this easy comfort and improve Git as a result on
> both front?  :-)
> 
>> Or maybe it is a good thing for that reason, as we will eat our own
>> rename dogfood. :)
> 
> Exactly!  And maybe we'll make Git even more useful in the process.
> 
>>> 5) Rename t/ to testsuite/ so this doesn't look like some garbage
>>>     leftover.
>>
>> Ugh, more typing. :P
> 
> Come on!  You sound like an old fart now!  ;-)
> 

Personally, I kinda like the capital D in Documentation for tab
completion reasons. Keeping frequently used files and directories
with short unique prefixes makes perfect sense from a typing point
of view. Using longer mnemonic names makes perfect sense from a
regex search/replace point of view.

I'm kinda with Junio on the ./*.[ch] -> src/*.[ch] move though, but
perhaps that's just because I hate autoconf projects which generate
a ton of cruft in the root before one can even start building it.

It would probably help matters along if buildproducts ended up in
their own directory though. That way .gitignore won't have so many
extra commits when new source files are added, and 'make clean'
gets easier to maintain.

So to sum up what I'm for;

t/ -> Test/ (or Testsuite, but some more mnemonic name anyways
with a short unique prefix for tab completion). This would also
exercise our rename machinery quite a bit, altohugh not to the
point where people get annoyed if it gets sort-of-broken.

buildproducts to Build/ (capital B to avoid completing against
builtin*). This also has the benefit that %.o: %.c rules makes
tab completion work better when object files are already built,
and "git add git-foo.*" doesn't throw the "git-foo.o is ignored"
error and forces one to re-type it as "git-foo.[ch]"

The ppc stuff I don't really care about and it wouldn't be hard
to resurrect if we remove it and people complain. Everything
will also still work if we do, although with possibly a slight
decrease in performance.

Bulk of source-files stay as ./*.[ch]. I see absolutely no
benefit to moving them, but two potential drawbacks. One is that
it'll cause more typing for those of us who use console-based
editors and run 'make' manually (yes, we do exist). The afore-
mentioned merge+rebase hell is also a considerable drawback,
even though that's primarily an issue for maint releases.

Risks: Well... dunno about that really. Mucking up the build and
test systems, I suppose, but it's easy enough to test.

Cons: Rebasing and merging stuff to the Makefile and test-stuff
will suck a bit, but as has been pointed out that's not only a
bad thing.

Pros: Less typing all-round. Simpler "make clean" rules. Fewer
tacked-on .gitignore patches for new commands. We (well, Junio)
get to experience first-hand the problems of directory renames
across releases but on a smaller scale than moving *everything*
around in one go.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: [1.8.0] reorganize the mess that the source tree has become

[Martin von Zweigbergk]

On Mon, 31 Jan 2011, Nicolas Pitre wrote:

> 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
>    the build.

I don't care much about the other items on the list, but I do agree
with this one. The biggest reason I like this is because it makes it
easier to tab complete. In all the cases so far that I have tab
completed "git-rebase--i" to open it in an editor or to run some git
command on it, I have wanted "git-rebase--interactive.sh"; I have
never wanted the build result.

It is also nice to have one less file to edit (.gitignore) when you
add a new source file, but that is of course much less important.

Are there any arguments against this part of Nicolas's proposal?

Btw, this is not really related to 1.8.0, is it? It seems to me like
it could be done at any time...


/Martin