[1.8.0] reorganize the mess that the source tree has become

[1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

To me the source tree looks highly untidy to me.

We do have subdirectories for documentation, tests, contributions, etc.  
But a sizeable part of the tree is just a big splat of source files 
dumped right in the root of the tree.

So I'd suggest doing the following:

1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
   the builtin directory from the root directory to it.

2) Create a build/ directory, or bin/ if prefered, to hold the result of 
   the build.

3) Consider dropping the ppc/ directory.  Unless someone really cares 
   deeply, it would be nice to simply always use the block-sha1 code and 
   move it straight into src/.

4) Consider moving some more directories into src/ such as xdiff/.
   I'd leave compat/ outside src/ to make it more explicit that this is 
   not about Git proper.

5) Rename t/ to testsuite/ so this doesn't look like some garbage 
   leftover.

6) And fix up all the Makefiles to cope with the above movements.

What do you think?


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Nicolas Pitre <nico@fluxnic.net> writes:

> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
>    the builtin directory from the root directory to it.
>
> 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
>    the build.
>...
> 6) And fix up all the Makefiles to cope with the above movements.
>
> What do you think?

Knee-jerk reaction: not very motivated to make the top-level directory
just a skeleton that holds various directories with a handful of
administrative files like Makefile, README, etc.  Under your proposal, the
bulk of the current content at the top would simply move to another single
directory anyway, so I don't immediately see much point of such a move,
other than adding merge burden on me and rebase burden on others, that
is.

But that is just a knee-jerk reaction, just to fill the "Risks:" section
you didn't fill.  Your missing "Migration Plans" section might outline a
clever approach to lessen the interim hurt while merging in-flight topics.

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:

> We do have subdirectories for documentation, tests, contributions, etc.  
> But a sizeable part of the tree is just a big splat of source files 
> dumped right in the root of the tree.
> 
> So I'd suggest doing the following:
> 
> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
>    the builtin directory from the root directory to it.

Wouldn't this just be the same giant splat of source files, but in a
different tree? I don't really see the advantage, and it seems like an
extra annoyance. Besides being just one more directory to go up and
down, it does make history browsing more annoying. As much as I love
git's "don't record renames" philosophy, our handling of renames on the
viewing side is often annoying. I already get annoyed sometimes
following stuff across the s!builtin-!builtin/! change. This would be
like that but more so.

Or maybe it is a good thing for that reason, as we will eat our own
rename dogfood. :)

> 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
>    leftover.

Ugh, more typing. :P

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Matthieu Moy]

Junio C Hamano <gitster@pobox.com> writes:

> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc.  Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,

There would be at least one obvious benefit: currently, we have this

git$ ls | wc -l
623

(that's after a build)

It's a bit hard to find the interesting bits (README, Documentation/,
contrib/ for example) in the output of "ls".

> other than adding merge burden on me and rebase burden on others, that
> is.

That can be seen as a test of how good Git is at bulk rename
management ;-).

All that said, I cannot really say whether the benefit is higher than
the cost.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Junio C Hamano wrote:

> Nicolas Pitre <nico@fluxnic.net> writes:
> 
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
> >    the builtin directory from the root directory to it.
> >
> > 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
> >    the build.
> >...
> > 6) And fix up all the Makefiles to cope with the above movements.
> >
> > What do you think?
> 
> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc.  Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,
> other than adding merge burden on me and rebase burden on others, that
> is.

I really think that the top directory is not the proper place for source 
files to live, especially considering how big a project Git is now.  
The top directory should be like a table of content and not the content 
itself.

But if you the maintainer doesn't see a long-term value in this to be 
greater than the one-time burden, then I'm afraid there's nothing I can 
do to help it.

> But that is just a knee-jerk reaction, just to fill the "Risks:" section
> you didn't fill.  Your missing "Migration Plans" section might outline a
> clever approach to lessen the interim hurt while merging in-flight topics.

Well, there is no such plan.  Given that 1.8 is meant to be an inflexion 
point for users, it could as well be for developers the best time to 
clean up this mess too.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
> 
> > We do have subdirectories for documentation, tests, contributions, etc.  
> > But a sizeable part of the tree is just a big splat of source files 
> > dumped right in the root of the tree.
> > 
> > So I'd suggest doing the following:
> > 
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and 
> >    the builtin directory from the root directory to it.
> 
> Wouldn't this just be the same giant splat of source files, but in a
> different tree? I don't really see the advantage, and it seems like an
> extra annoyance. 

Like I said to Junio, if you don't see the advantage, there's nothing I 
can do for you.  To me this is simple good source code hygiene.

> Besides being just one more directory to go up and down, it does make 
> history browsing more annoying. As much as I love git's "don't record 
> renames" philosophy, our handling of renames on the viewing side is 
> often annoying. I already get annoyed sometimes following stuff across 
> the s!builtin-!builtin/! change. This would be like that but more so.

So... we do suck at something?  So why not take this opportunity to 
shake yourself out of this easy comfort and improve Git as a result on 
both front?  :-)

> Or maybe it is a good thing for that reason, as we will eat our own
> rename dogfood. :)

Exactly!  And maybe we'll make Git even more useful in the process.

> > 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
> >    leftover.
> 
> Ugh, more typing. :P

Come on!  You sound like an old fart now!  ;-)


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Matthieu Moy wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > other than adding merge burden on me and rebase burden on others, that
> > is.
> 
> That can be seen as a test of how good Git is at bulk rename
> management ;-).
> 
> All that said, I cannot really say whether the benefit is higher than
> the cost.

There is a huge value in inflicting on ourselves such a test case for 
the tool we produce.  That helps avoiding the ivory tower syndrome.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Nicolas Pitre <nico@fluxnic.net> writes:

> On Mon, 31 Jan 2011, Jeff King wrote:
>
>> Besides being just one more directory to go up and down, it does make 
>> history browsing more annoying. As much as I love git's "don't record 
>> renames" philosophy, our handling of renames on the viewing side is 
>> often annoying. I already get annoyed sometimes following stuff across 
>> the s!builtin-!builtin/! change. This would be like that but more so.
>
> So... we do suck at something?  So why not take this opportunity to 
> shake yourself out of this easy comfort and improve Git as a result on 
> both front?  :-)
>
>> Or maybe it is a good thing for that reason, as we will eat our own
>> rename dogfood. :)
>
> Exactly!  And maybe we'll make Git even more useful in the process.

This part I _could_ actually buy; even though I do not think moving files
without much reason is a good project hygine, it does happen in real life,
and we would want to keep things smooth for real people.

>> > 5) Rename t/ to testsuite/ so this doesn't look like some garbage 
>> >    leftover.

I am not sure about this "t/" vs "testsuite/".

>> Ugh, more typing. :P
>
> Come on!  You sound like an old fart now!  ;-)

If we make the top-level directory lean enough, we probably can tab
complete after typing just "cd t" to go to testsuite/ or tests/ or
whatever you come up with, so "more typing" is not a huge issue to me
personally.

I however think the directory name "t/" is not our invention but what we
took from somebody else (perhaps Perl?), and I suspect some people expect
to find tests under there since we have had them there for a long time.

Re: [1.8.0] reorganize the mess that the source tree has become

[João P. Sampaio]

I think a good code hygiene is important, and some suggestions here
are relevant:

1) I also think there should be a directory for the source code
(namely, src/), and the top level should act as a table of contents.
As a newcomer myself who's trying to grasp Git, I can say an organized
project makes people more inclined join. Therefore, directories should
be named as clearly as possible: see item 2;

2) For item 1, t/ should be renamed to testsuite/. As Junio said, if
we get a more organized project, people could just "cd t" and tab to
autocomplete, or "cd tes" at the worst scenario, which is not such a
big hassle. About people expecting the testsuite to be inside t/, once
they type "cd t" and get an error, most people would look for an
alternative and eventually find the correct folder, or tabbing would
just suggest the name.

3) The top level should hold files that point people towards where
they want to go, helpful files like README and even some
Documentation/ files could get a promotion.

3) As 1.8.0 can be an inflexion for users, so could be for the
developers as well.

-- 
João Paulo Melo de Sampaio
Computer Engineering Student @ UFSCar
Website: http://www.jpmelos.com
Twitter: twitter.com/jpmelos (@jpmelos)

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Junio C Hamano wrote:

> If we make the top-level directory lean enough, we probably can tab
> complete after typing just "cd t" to go to testsuite/ or tests/ or
> whatever you come up with, so "more typing" is not a huge issue to me
> personally.
> 
> I however think the directory name "t/" is not our invention but what we
> took from somebody else (perhaps Perl?), and I suspect some people expect
> to find tests under there since we have had them there for a long time.

If those people are not able to figure out that "testsuite" means where 
tests are, especially within a lean top directory, then we might 
question the reliability of the tests they might contribute.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:

> > Besides being just one more directory to go up and down, it does make 
> > history browsing more annoying. As much as I love git's "don't record 
> > renames" philosophy, our handling of renames on the viewing side is 
> > often annoying. I already get annoyed sometimes following stuff across 
> > the s!builtin-!builtin/! change. This would be like that but more so.
> 
> So... we do suck at something?  So why not take this opportunity to 
> shake yourself out of this easy comfort and improve Git as a result on 
> both front?  :-)

Yes, we do suck at rename following. The problem is that it is partially
an implementation issue, and partially a fundamental issue. Obviously
--follow sucks pretty hard right now, and that could be fixed. Namely it
follows only a single file, and it interacts very badly with history
simplification.

But even with those things fixed, there will still be annoyances.

It will still be _slower_ to turn it on all the time, for one[1]. And
that's due to fundamental design decisions of the git data structure.
And I'm not knocking those decisions; I think they made the right
tradeoff. But that doesn't mean we don't pay the cost for that tradeoff.

And no matter what your model, renames can be annoying. On-going topics
will have a painful rebase or merge. And people looking at history will
have to deal with the code-base having different names at different
points. Yeah, you can say it's all just "content", but the filenames we
put things in are actually useful.

So I don't think it's wrong to say "renames are a pain, and so should
not be done lightly". I do think it's wrong to say "renames can't be
done"; I just the cost needs to be considered.

-Peff

[1] I'd be interested to see how much we can get around that slowness
using a notes-cache.

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:
> 
> > So... we do suck at something?  So why not take this opportunity to 
> > shake yourself out of this easy comfort and improve Git as a result on 
> > both front?  :-)
> 
> Yes, we do suck at rename following. The problem is that it is partially
> an implementation issue, and partially a fundamental issue. Obviously
> --follow sucks pretty hard right now, and that could be fixed. Namely it
> follows only a single file, and it interacts very badly with history
> simplification.

This is no excuse not to do proper source tree reorganization.

Telling people not to move files around because Git sucks at tracking 
them is also the wrong answer.

> But even with those things fixed, there will still be annoyances.
> 
> It will still be _slower_ to turn it on all the time, for one[1]. And
> that's due to fundamental design decisions of the git data structure.
> And I'm not knocking those decisions; I think they made the right
> tradeoff. But that doesn't mean we don't pay the cost for that tradeoff.

I agree.  However sitting on our back and resisting a cleanup just 
because our very tool does poorly in that scenario is just like putting 
our heads in the sand and pretend that the problem doesn't exist.  
Better do what most people without internal knowledge of Git would do 
and just clean up the tree, and then benefit from this extraordinary 
opportunity of having this environment right at home where Git 
developers have a much greater incentive to work on this issue and 
improve things.

> And no matter what your model, renames can be annoying. On-going topics
> will have a painful rebase or merge. And people looking at history will
> have to deal with the code-base having different names at different
> points. Yeah, you can say it's all just "content", but the filenames we
> put things in are actually useful.

Of course.  But such is life.  Many projects out there are just like 
that, and facing this situation ourselves will just help us figure out 
ways to make Git even more useful to more people.

> So I don't think it's wrong to say "renames are a pain, and so should
> not be done lightly".

I disagree.  This is like saying: "renames are not well supported, so 
let's avoid them while using Git."  People used to say that of merges 
with CVS.  Are we going to follow suit de facto?  Imagine the Git 
detractors taking our source tree mess to exemplify this Git flaw since 
"Git developers themselves are unwilling to move files around because 
Git sucks at it".

> I do think it's wrong to say "renames can't be
> done"; I just the cost needs to be considered.

Instead, why not saying: "Rename tracking is not as optimal as it could 
be, so let's work it out." ?


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Tue, Feb 1, 2011 at 12:12 AM, Jeff King <peff@peff.net> wrote:
> On Mon, Jan 31, 2011 at 04:28:49PM -0500, Nicolas Pitre wrote:
>
>> > Besides being just one more directory to go up and down, it does make
>> > history browsing more annoying. As much as I love git's "don't record
>> > renames" philosophy, our handling of renames on the viewing side is
>> > often annoying. I already get annoyed sometimes following stuff across
>> > the s!builtin-!builtin/! change. This would be like that but more so.
>>
>> So... we do suck at something?  So why not take this opportunity to
>> shake yourself out of this easy comfort and improve Git as a result on
>> both front?  :-)
>
> Yes, we do suck at rename following. The problem is that it is partially
> an implementation issue, and partially a fundamental issue. Obviously
> --follow sucks pretty hard right now, and that could be fixed. Namely it
> follows only a single file, and it interacts very badly with history
> simplification.
>
> But even with those things fixed, there will still be annoyances.
>
> It will still be _slower_ to turn it on all the time, for one[1]. And
> that's due to fundamental design decisions of the git data structure.

Does it really have to be? I mean, for whole-file rename-detection, if
we say that we automatically enable rename-detection (by default) as
we reach the first commit that doesn't have a given tree-entry, then
it would only kick in as we're about to terminate the log-output. And
since we're usually reading through a pager, it should means that it
takes a little bit more time before the user knows he's at the end of
the log. It shouldn't really affect the throughput of the data before
the point that becomes an annoyance, right?

At least that's the part that I find the most annoying with the
current rename detection; having to enable the flag as I reach the end
of the history for a file, often having to search through a lot of
commits again.

Re: [1.8.0] reorganize the mess that the source tree has become

[Sverre Rabbelier]

Heya,

On Tue, Feb 1, 2011 at 00:12, Jeff King <peff@peff.net> wrote:
> [1] I'd be interested to see how much we can get around that slowness
> using a notes-cache.

Do you mean something like a refs/notes/renames namespace in which we
stick notes on commits indicating that a rename indicated at that
commit, with an option of the user, after-the-fact, adding this
information manually?

-- 
Cheers,

Sverre Rabbelier

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:

> > Yes, we do suck at rename following. The problem is that it is partially
> [...]
> This is no excuse not to do proper source tree reorganization.

I think this is the crux of our disagreement. I don't agree that your
proposal is any way more "proper" than what is there now. Leaving the
rename issue aside (i.e., if we were starting a new project), I would
still be slightly against a src/ directory. I find them annoying.

But I don't care _that_ much, and I would rather not waste either of our
time debating it more. I would much rather you spend your time on
pack v4. :)

> I disagree.  This is like saying: "renames are not well supported, so 
> let's avoid them while using Git."  People used to say that of merges 
> with CVS.  Are we going to follow suit de facto?  Imagine the Git 
> detractors taking our source tree mess to exemplify this Git flaw since 
> "Git developers themselves are unwilling to move files around because 
> Git sucks at it".

For the record, part of my argument was that renaming is annoying to
some degree in _all_ systems, not just git.

> > I do think it's wrong to say "renames can't be
> > done"; I just the cost needs to be considered.
> 
> Instead, why not saying: "Rename tracking is not as optimal as it could 
> be, so let's work it out." ?

I did also say that. :)

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Tue, Feb 01, 2011 at 01:35:16AM +0100, Erik Faye-Lund wrote:

> > It will still be _slower_ to turn it on all the time, for one[1]. And
> > that's due to fundamental design decisions of the git data structure.
> 
> Does it really have to be? I mean, for whole-file rename-detection, if
> we say that we automatically enable rename-detection (by default) as
> we reach the first commit that doesn't have a given tree-entry, then
> it would only kick in as we're about to terminate the log-output. And
> since we're usually reading through a pager, it should means that it
> takes a little bit more time before the user knows he's at the end of
> the log. It shouldn't really affect the throughput of the data before
> the point that becomes an annoyance, right?

That's not quite true. You may be following not just a single file, but
some arbitrary pathspec. Any time there is a file creation event that
matches that pathspec, you will want to rename-detect any possible
sources for that created file, and add them to the list of interesting
paths.

But yeah, we don't have to do rename detection for every single case. So
the slowness may turn out to be not that bad. When I had
arbitrary-pathspec following this summer I thought I did some numbers,
but I don't remember the results and I can't find them on the list.

-Peff

Re: [1.8.0] reorganize the mess that the source tree has become

[Jeff King]

On Tue, Feb 01, 2011 at 02:00:05AM +0100, Sverre Rabbelier wrote:

> Heya,
> 
> On Tue, Feb 1, 2011 at 00:12, Jeff King <peff@peff.net> wrote:
> > [1] I'd be interested to see how much we can get around that slowness
> > using a notes-cache.
> 
> Do you mean something like a refs/notes/renames namespace in which we
> stick notes on commits indicating that a rename indicated at that
> commit, with an option of the user, after-the-fact, adding this
> information manually?

Yes, without the "option of the user..." bit. Basically just cache the
list of renames for a given commit against its parent (which should be
immutable[1]), under the assumption that it is cheaper to look up the
note than it is to calculate the renames.

But I would make it purely a cache, not some place for users to stick
their own generated rename information (if people really want to do
that, I would much rather see it go into the commit itself as a
pseudo-header).

-Peff

[1] It's technically not immutable if you limit the pathspec, or if you
have non-standard rename options. But you could define some "canonical"
rename set, like all of the pairs found doing rename detection with -M90
when considering the whole set of removed files as sources and added
files as destinations. That would cover the common case of people
running "git log", and then more specialized detection would not use the
cache.

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Mon, 31 Jan 2011, Jeff King wrote:

> On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:
> 
> > This is no excuse not to do proper source tree reorganization.
> 
> I think this is the crux of our disagreement. I don't agree that your
> proposal is any way more "proper" than what is there now. Leaving the
> rename issue aside (i.e., if we were starting a new project), I would
> still be slightly against a src/ directory. I find them annoying.

Let's agree to disagree then.  What I see in the root of the Git source 
tree is a huge clutter of source files, binary files, scripts, and 
subdirectories all mixed together.  If you know by hart where things are 
because you've been hacking on them for the last 5 years then of course 
you might not see the point.  But since I didn't work much on Git 
lately, things are not as obvious to me as they used to be.  Looking 
back at it now with some distance, this tree looks like a mess and it is 
really annoying to work with.

> But I don't care _that_ much, and I would rather not waste either of our
> time debating it more. I would much rather you spend your time on
> pack v4. :)

I wish... I wish.  But I have a plan which might involve taking some 
vacation from $day_job in the Caribbeans with $wife and no kids, where 
$wife is going to do scuba diving with her club mates while I'll be 
alone with a laptop and no net connection and therefore nothing else to 
do for a week.  I've been craving for such free time for quite a while 
now.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Jay Soffian]

On Mon, Jan 31, 2011 at 6:12 PM, Jeff King <peff@peff.net> wrote:
> And no matter what your model, renames can be annoying. On-going topics
> will have a painful rebase or merge. And people looking at history will
> have to deal with the code-base having different names at different
> points. Yeah, you can say it's all just "content", but the filenames we
> put things in are actually useful.

I have been dealing with this quite a bit lately as Chromium has been
doing mass renaming. It's no small project and sometimes those merges
are big:

[diff]
	renames = copies
	renameLimit = 2000

:-)

What I can say is, yes, it's annoying, but also: git does quite a
decent job of it. I've found myself having to do this occasionally,
but not too often:

  git diff ...MERGE_HEAD -- /path/to/old/name | patch /path/to/new/name

(Typically when a header is renamed, a stub is left in place at the
old name which just includes the new name, and the local changes don't
come across to the new name 100% cleanly.)

Anyway, I would welcome git.git getting a little taste of that. :-) :-) :-)

j.

Re: [1.8.0] reorganize the mess that the source tree has become

[Jonathan Nieder]

Thomas Rast wrote:

> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect to find git-foo implemented in
> something named of that sort, so we could probably help him by mapping
> 
>   git-foo.sh      ->   git-foo.sh
>   builtin/bar.c   ->   git-bar.c
>   baz.c           ->   lib/baz.c

I agree.  This sets off my "time to resist change" alarms much
less than "git mv *.c *.sh src/", for what it's worth.

>   baz.o           ->   build/baz.o (or whatever, just elsewhere)
>   baz.gcov        ->   build/baz.gcov (ditto)

Maybe something like this to start?

-- 8< --
Subject: Makefile: basic support for separate build dir

 - python and perl machinery haven't been tweaked yet
 - requires good VPATH support
 - relies on COMPUTE_HEADER_DIRECTORIES to make the object file
   directories
 - does not support paths with spaces

Usage:

	mkdir output
	cd output
	echo COMPUTE_HEADER_DIRECTORIES=1 >config.mak
	make -f ../Makefile GIT_SRC=$(pwd)/../ -j2

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 Makefile            |   41 ++++++++++++++++++++++++++---------------
 generate-cmdlist.sh |    4 ++--
 perl/Makefile       |    2 +-
 3 files changed, 29 insertions(+), 18 deletions(-)

diff --git a/Makefile b/Makefile
index 775ee83..b258a24 100644
--- a/Makefile
+++ b/Makefile
@@ -243,8 +243,18 @@ all::
 #
 # Define NATIVE_CRLF if your platform uses CRLF for line endings.
 
+# Absolute path to the toplevel of the git sources, with trailing /.
+# Leave empty for an in-place build.
+GIT_SRC =
+ifdef GIT_SRC
+	NO_PYTHON = YesPlease
+	NO_PERL_MAKEMAKER = YesPlease
+endif
+
+VPATH := $(if $(GIT_SRC),$(GIT_SRC),$(CURDIR))
+
 GIT-VERSION-FILE: FORCE
-	@$(SHELL_PATH) ./GIT-VERSION-GEN
+	@$(SHELL_PATH) $(GIT_SRC)/GIT-VERSION-GEN
 -include GIT-VERSION-FILE
 
 uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
@@ -305,6 +315,7 @@ lib = lib
 pathsep = :
 
 export prefix bindir sharedir sysconfdir gitwebdir
+export GIT_SRC
 
 CC = gcc
 AR = ar
@@ -333,7 +344,7 @@ SPARSE_FLAGS = -D__BIG_ENDIAN__ -D__powerpc__
 # Those must not be GNU-specific; they are shared with perl/ which may
 # be built by a different compiler. (Note that this is an artifact now
 # but it still might be nice to keep that distinction.)
-BASIC_CFLAGS = -I.
+BASIC_CFLAGS = -I$(GIT_SRC). -I.
 BASIC_LDFLAGS =
 
 # Guard against environment variables
@@ -1562,7 +1573,7 @@ ifeq ($(PYTHON_PATH),)
 NO_PYTHON=NoThanks
 endif
 
-QUIET_SUBDIR0  = +$(MAKE) -C # space to separate -C and subdir
+QUIET_SUBDIR0  = +$(MAKE) -C $(GIT_SRC)# no space before subdir
 QUIET_SUBDIR1  =
 
 ifneq ($(findstring $(MAKEFLAGS),w),w)
@@ -1582,7 +1593,7 @@ ifndef V
 	QUIET_GCOV     = @echo '   ' GCOV $@;
 	QUIET_SUBDIR0  = +@subdir=
 	QUIET_SUBDIR1  = ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
-			 $(MAKE) $(PRINT_DIR) -C $$subdir
+			 $(MAKE) $(PRINT_DIR) -C $(GIT_SRC)$$subdir
 	export V
 	export QUIET_GEN
 	export QUIET_BUILT_IN
@@ -1696,10 +1707,10 @@ $(BUILT_INS): git$X
 	ln -s git$X $@ 2>/dev/null || \
 	cp git$X $@
 
-common-cmds.h: ./generate-cmdlist.sh command-list.txt
+common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)./generate-cmdlist.sh > $@+ && mv $@+ $@
+	$(QUIET_GEN)$(GIT_SRC)./generate-cmdlist.sh > $@+ && mv $@+ $@
 
 define cmd_munge_script
 $(RM) $@ $@+ && \
@@ -1709,7 +1720,7 @@ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
     -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
     -e 's/@@NO_CURL@@/$(NO_CURL)/g' \
     -e $(BROKEN_PATH_FIX) \
-    $@.sh >$@+
+    $(GIT_SRC)$@.sh >$@+
 endef
 
 $(patsubst %.sh,%,$(SCRIPT_SH)) : % : %.sh
@@ -1729,7 +1740,7 @@ perl/perl.mak: GIT-CFLAGS perl/Makefile perl/Makefile.PL
 
 $(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl
 	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C $(GIT_SRC)perl -s --no-print-directory instlibdir` && \
 	sed -e '1{' \
 	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
 	    -e '	h' \
@@ -1738,7 +1749,7 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl
 	    -e '	x' \
 	    -e '}' \
 	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-	    $@.perl >$@+ && \
+	    $(GIT_SRC)$@.perl >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 
@@ -1780,7 +1791,7 @@ git-instaweb: git-instaweb.sh gitweb/gitweb.cgi gitweb/static/gitweb.css gitweb/
 	    -e 's/@@NO_CURL@@/$(NO_CURL)/g' \
 	    -e 's|@@GITWEBDIR@@|$(gitwebdir_SQ)|g' \
 	    -e 's|@@PERL@@|$(PERL_PATH_SQ)|g' \
-	    $@.sh > $@+ && \
+	    $(GIT_SRC)$@.sh > $@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
@@ -1788,7 +1799,7 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
 	$(QUIET_GEN)$(RM) $@ $@+ && \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
 	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
-	    unimplemented.sh >$@+ && \
+	    $(GIT_SRC)unimplemented.sh >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 endif # NO_PERL
@@ -1803,7 +1814,7 @@ $(patsubst %.py,%,$(SCRIPT_PYTHON)): % : %.py
 	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
 	    -e 's|\(os\.getenv("GITPYTHONLIB"\)[^)]*)|\1,"@@INSTLIBDIR@@")|' \
 	    -e 's|@@INSTLIBDIR@@|'"$$INSTLIBDIR"'|g' \
-	    $@.py >$@+ && \
+	    $(GIT_SRC)$@.py >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PYTHON
@@ -1811,7 +1822,7 @@ $(patsubst %.py,%,$(SCRIPT_PYTHON)): % : unimplemented.sh
 	$(QUIET_GEN)$(RM) $@ $@+ && \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
 	    -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \
-	    unimplemented.sh >$@+ && \
+	    $(GIT_SRC)unimplemented.sh >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
 endif # NO_PYTHON
@@ -2142,7 +2153,7 @@ test-%$X: test-%.o $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)
 
 check-sha1:: test-sha1$X
-	./test-sha1.sh
+	$(GIT_SRC)./test-sha1.sh
 
 check: common-cmds.h
 	if sparse; \
@@ -2229,7 +2240,7 @@ endif
 		ln -s "git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
 		cp "$$execdir/git-remote-http$X" "$$execdir/$$p" || exit; \
 	done && \
-	./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
+	$(GIT_SRC)./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
 
 install-gitweb:
 	$(MAKE) -C gitweb install
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 75c68d9..f718633 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -9,7 +9,7 @@ struct cmdname_help
 
 static struct cmdname_help common_cmds[] = {"
 
-sed -n -e 's/^git-\([^ 	]*\)[ 	].* common.*/\1/p' command-list.txt |
+sed -n -e 's/^git-\([^ 	]*\)[ 	].* common.*/\1/p' "$GIT_SRC"command-list.txt |
 sort |
 while read cmd
 do
@@ -19,6 +19,6 @@ do
             x
             s/.*git-'"$cmd"' - \(.*\)/  {"'"$cmd"'", "\1"},/
 	    p
-     }' "Documentation/git-$cmd.txt"
+     }' "${GIT_SRC}Documentation/git-$cmd.txt"
 done
 echo "};"
diff --git a/perl/Makefile b/perl/Makefile
index a2ffb64..7c3a82a 100644
--- a/perl/Makefile
+++ b/perl/Makefile
@@ -44,4 +44,4 @@ endif
 # this is just added comfort for calling make directly in perl dir
 # (even though GIT-CFLAGS aren't used yet. If ever)
 ../GIT-CFLAGS:
-	$(MAKE) -C .. GIT-CFLAGS
+	$(MAKE) -C $(GIT_SRC).. GIT-CFLAGS
-- 
1.7.2.3

Re: [1.8.0] reorganize the mess that the source tree has become

[Jonathan Nieder]

Jonathan Nieder wrote:

> 	mkdir output
> 	cd output
> 	echo COMPUTE_HEADER_DIRECTORIES=1 >config.mak
> 	make -f ../Makefile GIT_SRC=$(pwd)/../ -j2

Erm, COMPUTE_HEADER_DEPENDENCIES.  Anyway, if someone wants to make
setting GIT_SRC take care of making the directories automatically, I
wouldn't mind. ;-)

Re: [1.8.0] reorganize the mess that the source tree has become

[Thomas Rast]

Nicolas Pitre wrote:
> What I see in the root of the Git source 
> tree is a huge clutter of source files, binary files, scripts, and 
> subdirectories all mixed together.  If you know by hart where things are 
> because you've been hacking on them for the last 5 years then of course 
> you might not see the point.  But since I didn't work much on Git 
> lately, things are not as obvious to me as they used to be.  Looking 
> back at it now with some distance, this tree looks like a mess and it is 
> really annoying to work with.

But judging by that assessment, shouldn't we strive to make it
*easier* to find things?

In particular a prospective git hacker would not care whether
something is a source file or a script (you seem to imply the
opposite).  He would instead expect to find git-foo implemented in
something named of that sort, so we could probably help him by mapping

  git-foo.sh      ->   git-foo.sh
  builtin/bar.c   ->   git-bar.c
  baz.c           ->   lib/baz.c
  baz.o           ->   build/baz.o (or whatever, just elsewhere)
  baz.gcov        ->   build/baz.gcov (ditto)


(I'm no huge fan of src/ either, but this should be orthogonal.)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Tue, 1 Feb 2011, Thomas Rast wrote:

> Nicolas Pitre wrote:
> > What I see in the root of the Git source 
> > tree is a huge clutter of source files, binary files, scripts, and 
> > subdirectories all mixed together.  If you know by hart where things are 
> > because you've been hacking on them for the last 5 years then of course 
> > you might not see the point.  But since I didn't work much on Git 
> > lately, things are not as obvious to me as they used to be.  Looking 
> > back at it now with some distance, this tree looks like a mess and it is 
> > really annoying to work with.
> 
> But judging by that assessment, shouldn't we strive to make it
> *easier* to find things?
> 
> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect to find git-foo implemented in
> something named of that sort, so we could probably help him by mapping
> 
>   git-foo.sh      ->   git-foo.sh
>   builtin/bar.c   ->   git-bar.c
>   baz.c           ->   lib/baz.c
>   baz.o           ->   build/baz.o (or whatever, just elsewhere)
>   baz.gcov        ->   build/baz.gcov (ditto)

I'm not proposing to go that far, especially given the current 
resistance to any changes.  IMHO anything that unclutters the top 
directory is good.

Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Andreas Ericsson]

On 01/31/2011 10:28 PM, Nicolas Pitre wrote:
> On Mon, 31 Jan 2011, Jeff King wrote:
> 
>> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
>>
>>> We do have subdirectories for documentation, tests, contributions, etc.
>>> But a sizeable part of the tree is just a big splat of source files
>>> dumped right in the root of the tree.
>>>
>>> So I'd suggest doing the following:
>>>
>>> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
>>>     the builtin directory from the root directory to it.
>>
>> Wouldn't this just be the same giant splat of source files, but in a
>> different tree? I don't really see the advantage, and it seems like an
>> extra annoyance.
> 
> Like I said to Junio, if you don't see the advantage, there's nothing I
> can do for you.  To me this is simple good source code hygiene.
> 
>> Besides being just one more directory to go up and down, it does make
>> history browsing more annoying. As much as I love git's "don't record
>> renames" philosophy, our handling of renames on the viewing side is
>> often annoying. I already get annoyed sometimes following stuff across
>> the s!builtin-!builtin/! change. This would be like that but more so.
> 
> So... we do suck at something?  So why not take this opportunity to
> shake yourself out of this easy comfort and improve Git as a result on
> both front?  :-)
> 
>> Or maybe it is a good thing for that reason, as we will eat our own
>> rename dogfood. :)
> 
> Exactly!  And maybe we'll make Git even more useful in the process.
> 
>>> 5) Rename t/ to testsuite/ so this doesn't look like some garbage
>>>     leftover.
>>
>> Ugh, more typing. :P
> 
> Come on!  You sound like an old fart now!  ;-)
> 

Personally, I kinda like the capital D in Documentation for tab
completion reasons. Keeping frequently used files and directories
with short unique prefixes makes perfect sense from a typing point
of view. Using longer mnemonic names makes perfect sense from a
regex search/replace point of view.

I'm kinda with Junio on the ./*.[ch] -> src/*.[ch] move though, but
perhaps that's just because I hate autoconf projects which generate
a ton of cruft in the root before one can even start building it.

It would probably help matters along if buildproducts ended up in
their own directory though. That way .gitignore won't have so many
extra commits when new source files are added, and 'make clean'
gets easier to maintain.

So to sum up what I'm for;

t/ -> Test/ (or Testsuite, but some more mnemonic name anyways
with a short unique prefix for tab completion). This would also
exercise our rename machinery quite a bit, altohugh not to the
point where people get annoyed if it gets sort-of-broken.

buildproducts to Build/ (capital B to avoid completing against
builtin*). This also has the benefit that %.o: %.c rules makes
tab completion work better when object files are already built,
and "git add git-foo.*" doesn't throw the "git-foo.o is ignored"
error and forces one to re-type it as "git-foo.[ch]"

The ppc stuff I don't really care about and it wouldn't be hard
to resurrect if we remove it and people complain. Everything
will also still work if we do, although with possibly a slight
decrease in performance.

Bulk of source-files stay as ./*.[ch]. I see absolutely no
benefit to moving them, but two potential drawbacks. One is that
it'll cause more typing for those of us who use console-based
editors and run 'make' manually (yes, we do exist). The afore-
mentioned merge+rebase hell is also a considerable drawback,
even though that's primarily an issue for maint releases.

Risks: Well... dunno about that really. Mucking up the build and
test systems, I suppose, but it's easy enough to test.

Cons: Rebasing and merging stuff to the Makefile and test-stuff
will suck a bit, but as has been pointed out that's not only a
bad thing.

Pros: Less typing all-round. Simpler "make clean" rules. Fewer
tacked-on .gitignore patches for new commands. We (well, Junio)
get to experience first-hand the problems of directory renames
across releases but on a smaller scale than moving *everything*
around in one go.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: [1.8.0] reorganize the mess that the source tree has become

[Nguyen Thai Ngoc Duy]

On Tue, Feb 1, 2011 at 7:42 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> Nicolas Pitre wrote:
>> What I see in the root of the Git source
>> tree is a huge clutter of source files, binary files, scripts, and
>> subdirectories all mixed together.  If you know by hart where things are
>> because you've been hacking on them for the last 5 years then of course
>> you might not see the point.  But since I didn't work much on Git
>> lately, things are not as obvious to me as they used to be.  Looking
>> back at it now with some distance, this tree looks like a mess and it is
>> really annoying to work with.
>
> But judging by that assessment, shouldn't we strive to make it
> *easier* to find things?
>
> In particular a prospective git hacker would not care whether
> something is a source file or a script (you seem to imply the
> opposite).  He would instead expect ...

A hacker is expected to RTFM first, IMO. Put up a document describing
how things are organized in git and we're good. git-grep will take
care from there.
-- 
Duy

Re: [1.8.0] reorganize the mess that the source tree has become

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Mon, Jan 31, 2011 at 07:29:54PM -0500, Nicolas Pitre wrote:
>
>> > Yes, we do suck at rename following. The problem is that it is partially
>> [...]
>> This is no excuse not to do proper source tree reorganization.
>
> I think this is the crux of our disagreement. I don't agree that your
> proposal is any way more "proper" than what is there now. Leaving the
> rename issue aside (i.e., if we were starting a new project), I would
> still be slightly against a src/ directory. I find them annoying.
>
> But I don't care _that_ much, and I would rather not waste either of our
> time debating it more. I would much rather you spend your time on
> pack v4. :)

I am with you, both counts on this topic.  I don't necessarily agree that
having sources at the top-level is bad, I don't want to see Nico wasting
his time arguing, and I do see some value in the proposal that gives us an
opportunity for dogfooding (but we already have done so with builtin/ and
it was not all that annoying---I think the timing was rather good and the
tree was semi-quiescent).

Ehh, that makes it not "both" but "two and half" counts ;-).

As long as the new directories are named sanely (one of the things I
detest is abbreviated uppercase silliness like "Src", "Lib"), I am fine
with the proposal.  Also I have a mild preference to keep build-products
next to the source (i.e. no separate "obj" directory).

Re: [1.8.0] reorganize the mess that the source tree has become

[George Spelvin]

For what it's worth, I don't see the "cleanup".

If it significantly reduced the size of the largest directory,
that would be a win.  But moving everything into src/ doesn't
do that.

If there's a way to divide the source into cohesive subunits, that
would be great.  A programmer could ignore whole subdirectories
when working on something.

But just moving the whole existing pile into a subdirectory "because
everyone else does it" is not a reason; that's superstition.

Having to type "src/" a lot more often is definitely a downside.

Heck, that's one thing I actively dislike about GNU autoconf conventions.

If there's a compelling reason to change, could someone please describe it?

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Wed, Feb 2, 2011 at 3:29 AM, George Spelvin <linux@horizon.com> wrote:
> If there's a compelling reason to change, could someone please describe it?

I think the most compelling argument I can think of is that it makes
top-level entries like Documentation, RelNotes, COPYING, README and
INSTALL easier to spot when doing "ls". For some users (first-time
hackers etc) that's a pretty big plus, I'd think.

Re: [1.8.0] reorganize the mess that the source tree has become

[Pascal Obry]

Le 02/02/2011 09:31, Erik Faye-Lund a écrit :
> On Wed, Feb 2, 2011 at 3:29 AM, George Spelvin <linux@horizon.com> wrote:
>> If there's a compelling reason to change, could someone please describe it?
> 
> I think the most compelling argument I can think of is that it makes
> top-level entries like Documentation, RelNotes, COPYING, README and
> INSTALL easier to spot when doing "ls". For some users (first-time
> hackers etc) that's a pretty big plus, I'd think.

I was about to say that too. That's an argument strong enough I would
say. Today it is hard to see the structure.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: [1.8.0] reorganize the mess that the source tree has become

[Nicolas Pitre]

On Tue, 1 Feb 2011, George Spelvin wrote:

> For what it's worth, I don't see the "cleanup".
> 
> If it significantly reduced the size of the largest directory,
> that would be a win.  But moving everything into src/ doesn't
> do that.
> 
> If there's a way to divide the source into cohesive subunits, that
> would be great.  A programmer could ignore whole subdirectories
> when working on something.
> 
> But just moving the whole existing pile into a subdirectory "because
> everyone else does it" is not a reason; that's superstition.

There is no superstition here, simply basic elegance.

When you pick up a book from a shelf, do you see the actual content of 
the book printed right from the inside of the cover page, and the table 
of content tossed in the margin?  Would you construct a book yourself 
that way?

A nice source tree should be organized with a minimum of hierarchical 
structure.  To a newbie wanting to contribute to Git, it is rather 
frightening to cd into the git directory and see that ls generates more 
than 280 entries.  That simply looks sloppy.  And this gets much worse 
after a make.

The top directory should make different things stand out much more 
clearly, like a preface and a table of content.  You have the 
documentation here, the source there, the tests there, a clearly visible 
README file, etc.  If the src directory has about the same relative 
number of files after a move that's fine.  At least you should expect 
_only_ source files in there (and possibly their by-products), and not 
other types of data buried into the mix.

> Having to type "src/" a lot more often is definitely a downside.

Come on.  This is a rather egocentric argument without much substance.

> Heck, that's one thing I actively dislike about GNU autoconf conventions.

This has _nothing_ about any autoconf convention.  GNU autoconf requires 
stupid things like having a bunch of files such as CREDITS, INSTALL, 
CHANGELOG, and other whatnots even if you have nothing to put in them, 
in which case they still have to be there but empty.  It also dictates 
the exact name your directories must have, etc.

I'm not proposing a tree reorganization because GNU autoconf commands 
it, but rather because this is a sensible thing to do.

> If there's a compelling reason to change, could someone please describe it?

It's about the third time I'm putting forward arguments for this.  
Please see the list archive.

P.S. the netiquette on busy mailing lists recommends that you preserve 
all the email addresses that were listed as recipients on the message 
you reply to.  That would be highly appreciated.


Nicolas

Re: [1.8.0] reorganize the mess that the source tree has become

[Miles Bader]

Nicolas Pitre <nico@fluxnic.net> writes:
>> But just moving the whole existing pile into a subdirectory "because
>> everyone else does it" is not a reason; that's superstition.
>
> There is no superstition here, simply basic elegance.

"basic elegance" is hardly well-defined, and although there are probably
issues on which there's general agreement, this doesn't appear to be one
of them.

>> Having to type "src/" a lot more often is definitely a downside.
>
> Come on.  This is a rather egocentric argument without much substance.

It certainly has more substance than hand-waving like "basic elegance"
though...

Some slightly more concrete arguments have been:

  Pro-src:  Big top-level dir scares newbs
  Anti-src: Extra typing is annoying

I'm not really against a "src/" subdir, but it seems mostly a matter of
taste, and I've seen plenty of projects where the src/ directory seemed
pretty pointless...

-Miles

-- 
Politics, n. A strife of interests masquerading as a contest of
principles. The conduct of public affairs for private advantage.

Re: [1.8.0] reorganize the mess that the source tree has become

[Andreas Schwab]

Nicolas Pitre <nico@fluxnic.net> writes:

> This has _nothing_ about any autoconf convention.  GNU autoconf requires 
> stupid things like having a bunch of files such as CREDITS, INSTALL, 
> CHANGELOG, and other whatnots even if you have nothing to put in them, 
> in which case they still have to be there but empty.  It also dictates 
> the exact name your directories must have, etc.

That's automake, not autoconf (and only the default automake operation).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [1.8.0] reorganize the mess that the source tree has become

[Eugene Sajine]

On Thu, Feb 3, 2011 at 1:16 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Tue, 1 Feb 2011, George Spelvin wrote:
>
>> For what it's worth, I don't see the "cleanup".
>>
>> If it significantly reduced the size of the largest directory,
>> that would be a win.  But moving everything into src/ doesn't
>> do that.
>>
>> If there's a way to divide the source into cohesive subunits, that
>> would be great.  A programmer could ignore whole subdirectories
>> when working on something.
>>
>> But just moving the whole existing pile into a subdirectory "because
>> everyone else does it" is not a reason; that's superstition.
>
> There is no superstition here, simply basic elegance.
>
> When you pick up a book from a shelf, do you see the actual content of
> the book printed right from the inside of the cover page, and the table
> of content tossed in the margin?  Would you construct a book yourself
> that way?
>
> A nice source tree should be organized with a minimum of hierarchical
> structure.  To a newbie wanting to contribute to Git, it is rather
> frightening to cd into the git directory and see that ls generates more
> than 280 entries.  That simply looks sloppy.  And this gets much worse
> after a make.
>
> The top directory should make different things stand out much more
> clearly, like a preface and a table of content.  You have the
> documentation here, the source there, the tests there, a clearly visible
> README file, etc.  If the src directory has about the same relative
> number of files after a move that's fine.  At least you should expect
> _only_ source files in there (and possibly their by-products), and not
> other types of data buried into the mix.
>
>> Having to type "src/" a lot more often is definitely a downside.
>
> Come on.  This is a rather egocentric argument without much substance.
>
>> Heck, that's one thing I actively dislike about GNU autoconf conventions.
>
> This has _nothing_ about any autoconf convention.  GNU autoconf requires
> stupid things like having a bunch of files such as CREDITS, INSTALL,
> CHANGELOG, and other whatnots even if you have nothing to put in them,
> in which case they still have to be there but empty.  It also dictates
> the exact name your directories must have, etc.
>
> I'm not proposing a tree reorganization because GNU autoconf commands
> it, but rather because this is a sensible thing to do.
>
>> If there's a compelling reason to change, could someone please describe it?
>
> It's about the third time I'm putting forward arguments for this.
> Please see the list archive.
>
> P.S. the netiquette on busy mailing lists recommends that you preserve
> all the email addresses that were listed as recipients on the message
> you reply to.  That would be highly appreciated.
>
>
> Nicolas
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I'm not a hacker, but a user who had sometimes peeked into the git
sources. Unbelievable mess... Impossible to see the structure in
command line interface.
I totally agree with Nicolas here.
Folders were invented for a reason.

IMHO
src for source code
build for build by-products
tests for tests

Come on, give us some love, please!;)

Thanks,
Eugene

Re: [1.8.0] reorganize the mess that the source tree has become

[Hilco Wijbenga]

On 3 February 2011 10:46, Eugene Sajine <euguess@gmail.com> wrote:
> On Thu, Feb 3, 2011 at 1:16 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
>> On Tue, 1 Feb 2011, George Spelvin wrote:
>>
>>> For what it's worth, I don't see the "cleanup".
>>>
>>> If it significantly reduced the size of the largest directory,
>>> that would be a win.  But moving everything into src/ doesn't
>>> do that.
>>>
>>> If there's a way to divide the source into cohesive subunits, that
>>> would be great.  A programmer could ignore whole subdirectories
>>> when working on something.
>>>
>>> But just moving the whole existing pile into a subdirectory "because
>>> everyone else does it" is not a reason; that's superstition.
>>
>> There is no superstition here, simply basic elegance.
>>
>> When you pick up a book from a shelf, do you see the actual content of
>> the book printed right from the inside of the cover page, and the table
>> of content tossed in the margin?  Would you construct a book yourself
>> that way?
>>
>> A nice source tree should be organized with a minimum of hierarchical
>> structure.  To a newbie wanting to contribute to Git, it is rather
>> frightening to cd into the git directory and see that ls generates more
>> than 280 entries.  That simply looks sloppy.  And this gets much worse
>> after a make.
>>
>> The top directory should make different things stand out much more
>> clearly, like a preface and a table of content.  You have the
>> documentation here, the source there, the tests there, a clearly visible
>> README file, etc.  If the src directory has about the same relative
>> number of files after a move that's fine.  At least you should expect
>> _only_ source files in there (and possibly their by-products), and not
>> other types of data buried into the mix.
>>
>>> Having to type "src/" a lot more often is definitely a downside.
>>
>> Come on.  This is a rather egocentric argument without much substance.
>>
>>> Heck, that's one thing I actively dislike about GNU autoconf conventions.
>>
>> This has _nothing_ about any autoconf convention.  GNU autoconf requires
>> stupid things like having a bunch of files such as CREDITS, INSTALL,
>> CHANGELOG, and other whatnots even if you have nothing to put in them,
>> in which case they still have to be there but empty.  It also dictates
>> the exact name your directories must have, etc.
>>
>> I'm not proposing a tree reorganization because GNU autoconf commands
>> it, but rather because this is a sensible thing to do.
>>
>>> If there's a compelling reason to change, could someone please describe it?
>>
>> It's about the third time I'm putting forward arguments for this.
>> Please see the list archive.
>>
>> P.S. the netiquette on busy mailing lists recommends that you preserve
>> all the email addresses that were listed as recipients on the message
>> you reply to.  That would be highly appreciated.
>>
>>
>> Nicolas
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> I'm not a hacker, but a user who had sometimes peeked into the git
> sources. Unbelievable mess... Impossible to see the structure in
> command line interface.
> I totally agree with Nicolas here.
> Folders were invented for a reason.
>
> IMHO
> src for source code
> build for build by-products
> tests for tests
>
> Come on, give us some love, please!;)

Another one from the peanut gallery. :-) I wholeheartedly agree with
both Nicolas and Eugene.

Quite frankly, I'm surprised there are (presumably experienced)
developers who do not immediately see the value of a little
organization. Surely, given the use of code conventions, formatting
rules, etcetera, the obvious one step further is to also organize
where the files go?

(Given that I'm just a lurker I promise to leave it at this. I just
wanted to show Nicolas a little support.)

Re: [1.8.0] reorganize the mess that the source tree has become

[Miles Bader]

Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> Quite frankly, I'm surprised there are (presumably experienced)
> developers who do not immediately see the value of a little
> organization. Surely, given the use of code conventions, formatting
> rules, etcetera, the obvious one step further is to also organize
> where the files go?

I think one of the problems is that what's been suggested seems like
window-dressing.  Moving everything into src/ and calling it "organized"
doesn't actually accomplish much other than perhaps making the README
file more visible to newbs; things are _still_ a mess, just a mess with
four more letters...

-Miles

-- 
Back, n. That part of your friend which it is your privilege to contemplate in
your adversity.

Re: [1.8.0] reorganize the mess that the source tree has become

[Tor Arntsen]

On Fri, Feb 4, 2011 at 03:06, Miles Bader <miles@gnu.org> wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>> Quite frankly, I'm surprised there are (presumably experienced)
>> developers who do not immediately see the value of a little
>> organization. Surely, given the use of code conventions, formatting
>> rules, etcetera, the obvious one step further is to also organize
>> where the files go?
>
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

What Miles says is my feeling as well. And having a 'bin/' as was suggested
in one post doesn't make much sense to me either - if you want your compiled
output to go elsewhere than the source directory then the normal way of doing
that is to do and out-of-tree build (so if that's not working - I have
not checked -
then that's something which would be worth looking into.)

-Tor

Re: [1.8.0] reorganize the mess that the source tree has become

[Jakub Narebski]

Tor Arntsen <tor@spacetec.no> writes:
> On Fri, Feb 4, 2011 at 03:06, Miles Bader <miles@gnu.org> wrote:
> > Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>>>
>>> Quite frankly, I'm surprised there are (presumably experienced)
>>> developers who do not immediately see the value of a little
>>> organization. Surely, given the use of code conventions, formatting
>>> rules, etcetera, the obvious one step further is to also organize
>>> where the files go?
>>
>> I think one of the problems is that what's been suggested seems like
>> window-dressing.  Moving everything into src/ and calling it "organized"
>> doesn't actually accomplish much other than perhaps making the README
>> file more visible to newbs; things are _still_ a mess, just a mess with
>> four more letters...
> 
> What Miles says is my feeling as well. And having a 'bin/' as was suggested
> in one post doesn't make much sense to me either - if you want your compiled
> output to go elsewhere than the source directory then the normal way of doing
> that is to do and out-of-tree build (so if that's not working - I have
> not checked - then that's something which would be worth looking into.)

It is about supporting 'srcdir', isn't it?

BTW. what about using 'lib/' directory?
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [1.8.0] reorganize the mess that the source tree has become

[Erik Faye-Lund]

On Fri, Feb 4, 2011 at 3:06 AM, Miles Bader <miles@gnu.org> wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>> Quite frankly, I'm surprised there are (presumably experienced)
>> developers who do not immediately see the value of a little
>> organization. Surely, given the use of code conventions, formatting
>> rules, etcetera, the obvious one step further is to also organize
>> where the files go?
>
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

FWIW, I don't quite see what's wrong with "window dressing" here.
Making those files more visible is a good thing, IMO.

But I'm not so sure I agree that the rest of the source tree is such a
mess that everyone makes it out to be. OK, there's a lot of
source-files on the top-level (which would be the src-level with this
change), but why is that such a bad thing? And if this is a big deal,
perhaps moving libgit-sources to a separate folder would help?

[1.8.0] Re: reorganize the mess that the source tree has become

[Nicolas Sebrecht]

The 04/02/11, Miles Bader wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> > Quite frankly, I'm surprised there are (presumably experienced)
> > developers who do not immediately see the value of a little
> > organization. Surely, given the use of code conventions, formatting
> > rules, etcetera, the obvious one step further is to also organize
> > where the files go?
> 
> I think one of the problems is that what's been suggested seems like
> window-dressing.  Moving everything into src/ and calling it "organized"
> doesn't actually accomplish much other than perhaps making the README
> file more visible to newbs; things are _still_ a mess, just a mess with
> four more letters...

So it would be an ordered mess, at least. The current amount of files in
the root directory do make things harder for people not already familiar
with the content. FMHO, moving the source files into a subdirectory
could be only a first step to the good direction.

-- 
Nicolas Sebrecht

Re: [1.8.0] Re: reorganize the mess that the source tree has become

[Drew Northup]

On Fri, 2011-02-04 at 19:15 +0100, Nicolas Sebrecht wrote:
> The 04/02/11, Miles Bader wrote:
> > Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> > > Quite frankly, I'm surprised there are (presumably experienced)
> > > developers who do not immediately see the value of a little
> > > organization. Surely, given the use of code conventions, formatting
> > > rules, etcetera, the obvious one step further is to also organize
> > > where the files go?
> > 
> > I think one of the problems is that what's been suggested seems like
> > window-dressing.  Moving everything into src/ and calling it "organized"
> > doesn't actually accomplish much other than perhaps making the README
> > file more visible to newbs; things are _still_ a mess, just a mess with
> > four more letters...
> 
> So it would be an ordered mess, at least. The current amount of files in
> the root directory do make things harder for people not already familiar
> with the content. FMHO, moving the source files into a subdirectory
> could be only a first step to the good direction.

Nicolas,
Having once upon a time (in CVS days) taken over a project that was
neatly organized into tons of folders I can say that more folders is not
always better.
If you are organizing things into modules by folders, and those things
are mutually exclusive pre-compilation then doing so may make sense. If
the folders ADD value to the project by adding organization--as opposed
to hiding disorganization--then they may have value.
>From my meager hacking thus far (working on making utf-16 a more
user-friendly experience out-of-the-box) I have found that none of this
is true (thus far) with the git codebase. In fact the one thing that
would have been useful is more in-code--and/or separate
API-ish--documentation (it took me waaaay too long to figure out how git
add, aka git-add, works), but I am too much of a realist to expect that
to change much. I most certainly DO NOT recommend that a mess of patches
be submitted to Junio to fix it (document what you are working on as you
see fit; I work on too many things to not document fairly extensively).
I approach codebase reorganization the same way. I have seen the
destructive things it can do to a project when it becomes an end unto
itself separate from the primary focus. In fact, what killed that first
project I mentioned was an argument about something that was not part of
the primary purpose of the project which exploded with a fury resembling
a religious confrontation. I do not want to see that happen to Git. I
didn't want to see that project die either, but when you exasperate
enough of the core developers that's what happens. 

(My first job as project leader was to get it off of our CVS host, the
second was to find it a nice grave over at nongnu.org. I never did get
CVS import to work. I never did convince any of the core developers that
it was still worth working on.)

-- 
-Drew Northup
________________________________________________
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59

Re: [1.8.0] reorganize the mess that the source tree has become

[Martin von Zweigbergk]

On Mon, 31 Jan 2011, Nicolas Pitre wrote:

> 2) Create a build/ directory, or bin/ if prefered, to hold the result of 
>    the build.

I don't care much about the other items on the list, but I do agree
with this one. The biggest reason I like this is because it makes it
easier to tab complete. In all the cases so far that I have tab
completed "git-rebase--i" to open it in an editor or to run some git
command on it, I have wanted "git-rebase--interactive.sh"; I have
never wanted the build result.

It is also nice to have one less file to edit (.gitignore) when you
add a new source file, but that is of course much less important.

Are there any arguments against this part of Nicolas's proposal?

Btw, this is not really related to 1.8.0, is it? It seems to me like
it could be done at any time...


/Martin

[1.8.0] Re: reorganize the mess that the source tree has become

[Nicolas Sebrecht]

The 04/02/11, Drew Northup wrote:

> Having once upon a time (in CVS days) taken over a project that was
> neatly organized into tons of folders I can say that more folders is not
> always better.
> If you are organizing things into modules by folders, and those things
> are mutually exclusive pre-compilation then doing so may make sense. If
> the folders ADD value to the project by adding organization--as opposed
> to hiding disorganization--then they may have value.

This _is_ what we are talking about. Not tons of folders or whathever
you might think.

> destructive things
>                                                       killed that first
> project
>                            project       exploded with a fury resembling
> a religious confrontation.                  see that happen to Git
>                see that project die                  you exasperate
> enough of the core developers

Read again. I'm pretty sure this was not your goal but this almostly
looks like FUD to me. So, I don't think I'll involve to more discussion
in this subthread.

-- 
Nicolas Sebrecht