basics... when reading docs doesn't help

basics... when reading docs doesn't help

[Guennadi Liakhovetski]

Hi all

Since it's a basic question, I couldn't search archives - cannot come up 
with good enough search keywords, but would be greatful for pointers.

I've been using CVS for a few years now, briefly tried a couple of other 
CMS, but only briefly. Now trying to get to grisps with the git... And 
just cannot understand how everybody does it.

The package I'm trying to handle with git is the Linux kernel.

1. Ok, I clone a repository from Linus or some else.

2. as my internet connection is not very fast, although I do have 
flatrate, I prefer keeping a virgin cloned copy somewhere, where I only do 
pulls from the original clone source, no edits.

3. then I do git clone <path to original cloned tree> <new tree>

4. create a new branch in <new tree>

5. go hack / compile in the <new tree>

6. then I decide to build practically the same kernel for another machine, 
i.e., another configuration, maybe a couple of local changes...

Now, that's the first question: suppose I want to build kernels for about 
4 machines. Do you __really__ clone the whole tree 4 times??? And then I 
want try new versions for the same 4 machines without deleting the first 
ones - 4 more clones??? Now, my copy of Linus' tree was ATM 1.5GiB big... 
Slowly it's getting scary. Ok, if I build all that stuff on one 
filesystem, I can use --local to use hard links, right? But is it REALLY 
what everybody does?

The next thing is - I don't need all versions since 2.6.x in every copy I 
use to compile / test / hack - in most cases I only compare to the basis 
version, from which I branched. I might pull further updates into this 
repository, merge, etc., but I don't need all PAST commits! Can I clone 
starting from version x?

Ok, enough for starters:-)

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[J. Bruce Fields]

On Thu, Mar 29, 2007 at 10:50:51PM +0200, Guennadi Liakhovetski wrote:
> Now, my copy of Linus' tree was ATM 1.5GiB big...  Slowly it's getting
> scary.

On my laptop:

[bfields@pad linux]$ du -hs .
1.5G    .
[bfields@pad linux]$ du -hs .git
334M    .git

So it's mostly the checked out working directory and build
stuff.

If you really need a ton of build trees then you might just want to do
cp -al or something.

--b.

Re: basics... when reading docs doesn't help

[Junio C Hamano]

"J. Bruce Fields" <bfields@fieldses.org> writes:

> On Thu, Mar 29, 2007 at 10:50:51PM +0200, Guennadi Liakhovetski wrote:
>> Now, my copy of Linus' tree was ATM 1.5GiB big...  Slowly it's getting
>> scary.
>
> On my laptop:
>
> [bfields@pad linux]$ du -hs .
> 1.5G    .
> [bfields@pad linux]$ du -hs .git
> 334M    .git
>
> So it's mostly the checked out working directory and build
> stuff.
>
> If you really need a ton of build trees then you might just want to do
> cp -al or something.

How about suggesting "clone -l -s"?

Re: basics... when reading docs doesn't help

[J. Bruce Fields]

On Thu, Mar 29, 2007 at 02:26:10PM -0700, Junio C Hamano wrote:
> "J. Bruce Fields" <bfields@fieldses.org> writes:
> 
> > On Thu, Mar 29, 2007 at 10:50:51PM +0200, Guennadi Liakhovetski wrote:
> >> Now, my copy of Linus' tree was ATM 1.5GiB big...  Slowly it's getting
> >> scary.
> >
> > On my laptop:
> >
> > [bfields@pad linux]$ du -hs .
> > 1.5G    .
> > [bfields@pad linux]$ du -hs .git
> > 334M    .git
> >
> > So it's mostly the checked out working directory and build
> > stuff.
> >
> > If you really need a ton of build trees then you might just want to do
> > cp -al or something.
> 
> How about suggesting "clone -l -s"?

If you really want to share as much as possible, then I guess you want
to share the working trees too, since (as evidenced above), they're at
least as large as the compressed history.

Though actually on a second look, clone -l -s produces something that's
only 377M.  I hadn't realized how much space the build output takes up.
So judging from du the 1.5G Guennadi Liakhovetski mentions above seems
to break down into something like:

	330M .git
	380M working tree
	750M build output

--b.

Re: basics... when reading docs doesn't help

[Guennadi Liakhovetski]

On Thu, 29 Mar 2007, J. Bruce Fields wrote:

> On Thu, Mar 29, 2007 at 02:26:10PM -0700, Junio C Hamano wrote:
> > 
> > How about suggesting "clone -l -s"?

Yes, but how do "advanced git users" kernel developers work? Do they just 
do 1 clone and build / clean every time they want to test another 
configuration / arch, or do they clone -l or what? Do they create branches 
for each development thread, then pull / push between trees?...

> If you really want to share as much as possible, then I guess you want
> to share the working trees too, since (as evidenced above), they're at
> least as large as the compressed history.

But I don't want to re-build. Apart from i386 I build for a couple of ARM 
and PPC targets too...

> Though actually on a second look, clone -l -s produces something that's
> only 377M.  I hadn't realized how much space the build output takes up.
> So judging from du the 1.5G Guennadi Liakhovetski mentions above seems
> to break down into something like:
> 
> 	330M .git
> 	380M working tree
> 	750M build output

Strange. Is my git 1.4.0 criminally broken? I have a clone of Linus' tree 
on a USB disk on ext3 without any objects, which I just cloned at some 
point and then did a couple of pulls from the same source. Now

1545084 /mnt/sda2/kernel-git/linux-2.6/
1255084 /mnt/sda2/kernel-git/linux-2.6/.git

Interestingly, both end up with 5084. For comparison:

465044  /mnt/sda2/kernel-git/powerpc
174980  /mnt/sda2/kernel-git/powerpc/.git

But that's a freshly cloned tree, without any pulls. I re-cloned it, 
because the tree I had earlier had the problem with each pull:

Unpacking 12452 objects
 100% (12452/12452) done
* refs/heads/origin: does not fast forward to branch 'master' of 
git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc;
  not updating.

Wonderful and strange git world...

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[Linus Torvalds]

On Fri, 30 Mar 2007, Guennadi Liakhovetski wrote:
> > > 
> > > How about suggesting "clone -l -s"?
> 
> Yes, but how do "advanced git users" kernel developers work? Do they just 
> do 1 clone and build / clean every time they want to test another 
> configuration / arch, or do they clone -l or what? Do they create branches 
> for each development thread, then pull / push between trees?...

I suspect it depends on the developer.

I end up just using different branches and switching between them, but 
then, my branches tend to all be pretty small test-stuff (I only end up 
using one main branch, since 99% of what I do is merge other peoples stuff 
that has already gone through a test-cycle).

> But I don't want to re-build. Apart from i386 I build for a couple of ARM 
> and PPC targets too...

You're probably fine with "git clone -l -s" then.

> Strange. Is my git 1.4.0 criminally broken? I have a clone of Linus' tree 
> on a USB disk on ext3 without any objects, which I just cloned at some 
> point and then did a couple of pulls from the same source. Now
> 
> 1545084 /mnt/sda2/kernel-git/linux-2.6/
> 1255084 /mnt/sda2/kernel-git/linux-2.6/.git

The old git that always exploded all pulls and generated lots of loose 
objects? You can check with "git count-objects".

And to fix it, just do a "git gc" (or with older git versions, the secret 
handshake is just a simple "git repack -a -d").

> But that's a freshly cloned tree, without any pulls. I re-cloned it, 
> because the tree I had earlier had the problem with each pull:
> 
> Unpacking 12452 objects
>  100% (12452/12452) done
> * refs/heads/origin: does not fast forward to branch 'master' of 
> git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc;
>   not updating.

Sounds like either Paul re-based his tree, or you did some work on your 
"origin" branch..

			Linus

Re: basics... when reading docs doesn't help

[Guennadi Liakhovetski]

On Thu, 29 Mar 2007, Linus Torvalds wrote:

> On Fri, 30 Mar 2007, Guennadi Liakhovetski wrote:
> 
> > Strange. Is my git 1.4.0 criminally broken? I have a clone of Linus' tree 
> > on a USB disk on ext3 without any objects, which I just cloned at some 
> > point and then did a couple of pulls from the same source. Now
> > 
> > 1545084 /mnt/sda2/kernel-git/linux-2.6/
> > 1255084 /mnt/sda2/kernel-git/linux-2.6/.git
> 
> The old git that always exploded all pulls and generated lots of loose 
> objects? You can check with "git count-objects".

Installed 1.5.0.6 and its output of "git count-objects" is

180932 objects, 1112656 kilobytes

git gc removed everything (uh?) and then

linux-2.6$ du -ks .git
183040  .git

cool...

> And to fix it, just do a "git gc" (or with older git versions, the secret 
> handshake is just a simple "git repack -a -d").
> 
> > But that's a freshly cloned tree, without any pulls. I re-cloned it, 
> > because the tree I had earlier had the problem with each pull:
> > 
> > Unpacking 12452 objects
> >  100% (12452/12452) done
> > * refs/heads/origin: does not fast forward to branch 'master' of 
> > git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc;
> >   not updating.
> 
> Sounds like either Paul re-based his tree, or you did some work on your 
> "origin" branch..

It must be the former then:-) Did I have a chance to re-synchronize 
locally to be able to pull normally again or was the only way to re-clone?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[Linus Torvalds]

On Fri, 30 Mar 2007, Guennadi Liakhovetski wrote:
> 
> Installed 1.5.0.6 and its output of "git count-objects" is
> 
> 180932 objects, 1112656 kilobytes

Yeah, that is all the loose objects. That's 1.1GB of stuff as loose 
objects, which also likely means that your tree was not only using up too 
much disk-space, it was probably running about ten times slower than than 
it needs to be for many operations.

Using pack-files ends up speeding up a lot of "big" operations a *lot*. If 
all you do is look at individual commits, you'll probably not notice much, 
but even something as simple as running "gitk" with no parameters (which 
will thus traverse the whole history) should be a *lot* faster after it's 
been all re-packed.

> git gc removed everything (uh?) and then

Well, it will generally remove all loose objects, since they get put into 
the pack-files instead. So it didn't remove "everything", but it does 
remove everything that "git count-objects" normally counts (if you give 
count-objects the "-v" flag for verbose output, it will talk about packed 
objects too).

> linux-2.6$ du -ks .git
> 183040  .git
> 
> cool...

That looks about right. Packing generally uses about a tenth of the 
disk-space of loose objects - both from the use of deltas, and from the 
fact that you don't any disk block fragmentation. And because looking at 
objects then doesn't require any system calls any more (just the initial 
"mmap pack and read index" stuff), it ends up being much faster too, 
despite the added overhead of doing the whole delta-chain thing.

> > > Unpacking 12452 objects
> > >  100% (12452/12452) done
> > > * refs/heads/origin: does not fast forward to branch 'master' of 
> > > git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc;
> > >   not updating.
> > 
> > Sounds like either Paul re-based his tree, or you did some work on your 
> > "origin" branch..
> 
> It must be the former then:-) Did I have a chance to re-synchronize 
> locally to be able to pull normally again or was the only way to re-clone?

You need to mark "refs/heads/origin" as always following the remote 
"master", even if it gets re-based.  You do that by adding a "+" before 
the refspec that describes the remote.

These days, with git-1.5.x, "git clone" will do that for you (and make the 
remotes fall under "refs/remotes/origin/*" instead - they're considered 
separate branches from the local branches these days). However, since you 
created the repository with an older git version, it still uses the 
original format (and even though you upgraded your git binary, it will use 
the old-fashioned branch format for remote branches and for 
configuration).

So in your case, the remote is probably described by the 
.git/remotes/origin file, and it looks something like

	URL: git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
	Pull: master:origin

and you should just change that "Pull:" line to have a "+" in front of the 
refspec:

	URL: git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
	Pull: +master:origin

which tells git that the remote "master" branch should now 
*unconditionally* be followed into the local "origin" branch when you 
pull.

(In a more modern setup, you wouldn't have a .git/remotes/origin file at 
all, insead you would have something like this in the .git/config file:

	[remote "origin"]
		url = git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
		fetch = +refs/heads/*:refs/remotes/origin/*

	[branch "master"]
		remote = origin
		merge = refs/heads/master

which means that there is a remote called "origin", and all of the remote 
heads in that remote ("refs/heads/*") will be fetched into the local 
repository under "refs/remotes/origin/*". And the "+" again means that we 
do this even if it's not a fast-forward, ie we trust the remote 
explicitly.

So "git fetch origin" will fetch *all* branches from "origin" into your 
local repository, but to distinguish them from your *own* branches, they 
will be under .git/refs/remotes/ instead of your "real" local branches 
that are in ".git/refs/heads/. So you can now see the difference between 
your *local* version of a branch X and the remote version of that same 
branch by using "X" and "remotes/origin/X" respectively to describe that 
branch.

Then, the second part of the above config file means that when you're on 
the local "master" branch and do a "git pull", it will fetch it from the 
remote "origin", and merge the remote "refs/heads/master" branch from that 
remote (the same one that will be fetched into refs/remotes/origin/master 
by a "git fetch".

Yeah, this is all a bit complex, and it takes a while to wrap your head 
around it, but I have to say, once you do, the git-1.5.x layout really 
*is* very powerful, and it's actually very natural too (but the "very 
natural" part only comes after you have that "Aaahh!" moment!)

			Linus

Re: basics... when reading docs doesn't help

[Guennadi Liakhovetski]

On Fri, 30 Mar 2007, Linus Torvalds wrote:
> 
> You need to mark "refs/heads/origin" as always following the remote 
> "master", even if it gets re-based.  You do that by adding a "+" before 
> the refspec that describes the remote.
> 
> These days, with git-1.5.x, "git clone" will do that for you (and make the 
> remotes fall under "refs/remotes/origin/*" instead - they're considered 
> separate branches from the local branches these days). However, since you 
> created the repository with an older git version, it still uses the 
> original format (and even though you upgraded your git binary, it will use 
> the old-fashioned branch format for remote branches and for 
> configuration).
> 
> So in your case, the remote is probably described by the 
> .git/remotes/origin file, and it looks something like
> 
> 	URL: git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
> 	Pull: master:origin
> 
> and you should just change that "Pull:" line to have a "+" in front of the 
> refspec:
> 
> 	URL: git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
> 	Pull: +master:origin
> 
> which tells git that the remote "master" branch should now 
> *unconditionally* be followed into the local "origin" branch when you 
> pull.
> 
> (In a more modern setup, you wouldn't have a .git/remotes/origin file at 
> all, insead you would have something like this in the .git/config file:
> 
> 	[remote "origin"]
> 		url = git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc
> 		fetch = +refs/heads/*:refs/remotes/origin/*
> 
> 	[branch "master"]
> 		remote = origin
> 		merge = refs/heads/master
> 
> which means that there is a remote called "origin", and all of the remote 
> heads in that remote ("refs/heads/*") will be fetched into the local 
> repository under "refs/remotes/origin/*". And the "+" again means that we 
> do this even if it's not a fast-forward, ie we trust the remote 
> explicitly.
> 
> So "git fetch origin" will fetch *all* branches from "origin" into your 
> local repository, but to distinguish them from your *own* branches, they 
> will be under .git/refs/remotes/ instead of your "real" local branches 
> that are in ".git/refs/heads/. So you can now see the difference between 
> your *local* version of a branch X and the remote version of that same 
> branch by using "X" and "remotes/origin/X" respectively to describe that 
> branch.
> 
> Then, the second part of the above config file means that when you're on 
> the local "master" branch and do a "git pull", it will fetch it from the 
> remote "origin", and merge the remote "refs/heads/master" branch from that 
> remote (the same one that will be fetched into refs/remotes/origin/master 
> by a "git fetch".
> 
> Yeah, this is all a bit complex, and it takes a while to wrap your head 
> around it, but I have to say, once you do, the git-1.5.x layout really 
> *is* very powerful, and it's actually very natural too (but the "very 
> natural" part only comes after you have that "Aaahh!" moment!)

Aha, so, that's how it is then! Why hasn't anybody explained this to me 
strait away?!:-))))

Yeah, hopefully, I'll learn to at least use this thing efficiently enough. 
Someone has to write a book on it though...

And, so, it's a pity I cloned Paul's tree yesterday with the "old" git. 
And from your answer above it seems like some features of the "new" git 
will not be available with this tree, like equally named local and remote 
branches, etc. There isn't a way to convert such a "old style" tree to the 
"new style", is there? Not a big deal, will re-clone at some point, maybe 
when we get local git mirrors...

Many thanks for taking your time to answer, Linus!
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[Linus Torvalds]

On Fri, 30 Mar 2007, Guennadi Liakhovetski wrote:
> 
> And, so, it's a pity I cloned Paul's tree yesterday with the "old" git. 
> And from your answer above it seems like some features of the "new" git 
> will not be available with this tree, like equally named local and remote 
> branches, etc. There isn't a way to convert such a "old style" tree to the 
> "new style", is there? Not a big deal, will re-clone at some point, maybe 
> when we get local git mirrors...

There's a conversion script to help you convert in place if you care.

Look at the git list for an email that looks something like this:

	Date: Wed, 14 Mar 2007 02:16:12 -0400
	From: Shawn O. Pearce <spearce@spearce.org>
	To: git@vger.kernel.org
	Subject: Upgrade to 1.5.0 utility

	Yesterday on #git DrNick wanted a script to update a pre-1.5.0
	Git repository to be like a 1.5.0 (and later) style repository.
	...

which has a script in it to do this (it uses another script that is 
already in git/contrib/ that just moves all the ".git/remotes" entries 
as-is from the remotes files into the .git/config file)

I haven't tested it myself, so caveat emptor. But the config file format 
really isn't *that* complicated - do the conversion with the script, and 
then just go back and look at .git/config and verify that it looks sane, 
or edit it to match your taste.

		Linus

Re: basics... when reading docs doesn't help

[Theodore Tso]

On Fri, Mar 30, 2007 at 09:49:20PM +0200, Guennadi Liakhovetski wrote:
> Aha, so, that's how it is then! Why hasn't anybody explained this to me 
> strait away?!:-))))

A lot of this is in the git 1.5's User Manual:

	http://www.kernel.org/pub/software/scm/git/docs/user-manual.html

See also the man page for git-remote, or here:

	http://www.kernel.org/pub/software/scm/git/docs/git-remote.html


> Yeah, hopefully, I'll learn to at least use this thing efficiently enough. 
> Someone has to write a book on it though...

The user-manual and the git tutorials are pretty good --- just make
sure you look at the ones that come with git 1.5.0, or the ones at
http://www.kernel.org/pub/software/svm/git/docs (which reflects the
Documentation as of the latest development version of git).

The problem is that there are some tutorials on the web which assume
git 1.4.x, or cogito, and that has been listed as a defect by people
who have been confused because those tutorials are out of date with
respect to modern git.[1]

[1] http://changelog.complete.org/posts/594-More-on-Git,-Mercurial,-and-Bzr.html

> And, so, it's a pity I cloned Paul's tree yesterday with the "old" git. 
> And from your answer above it seems like some features of the "new" git 
> will not be available with this tree, like equally named local and remote 
> branches, etc. There isn't a way to convert such a "old style" tree to the 
> "new style", is there? Not a big deal, will re-clone at some point, maybe 
> when we get local git mirrors...

Yeah, we need to really make sure the word gets out that new git users
should really make sure they are using git 1.5, and not git 1.4.x; it
is such an improvement in terms of usability over git 1.4.x that there
really is no comparison.  (Of course, guess what Debian is going to
ship in their soon-to-be-release "stable" distribution?  git 1.4.4.  Sigh...)

It is possible to get the new style naming, by using the git remote
command.  Just do

	git remote add origin <url>

...and that will adjust your git configuration file appropriate.
Check out the git 1.5.0 release notes, and the git-config man page,
and there are some other adjustments you can make as well that will
make git more efficiently, at the cost of breaking some level of
compatibility with git 1.4.x tools which replicate via the "dumb" http
protocol.  But for local development repositories that generally isn't
a concern.

						- Ted

Re: basics... when reading docs doesn't help

[Junio C Hamano]

Guennadi Liakhovetski <g.liakhovetski@gmx.de> writes:

>> Yeah, this is all a bit complex, and it takes a while to wrap your head 
>> around it, but I have to say, once you do, the git-1.5.x layout really 
>> *is* very powerful, and it's actually very natural too (but the "very 
>> natural" part only comes after you have that "Aaahh!" moment!)
>
> Aha, so, that's how it is then! Why hasn't anybody explained this to me 
> strait away?!:-))))

Because I and others have explained that to other people on the
list a few times already, perhaps?  The list archive is your
friend.

> .... There isn't a way to convert such a "old style" tree to the 
> "new style", is there?

That also can be found in the list archive.  I think Shawn
Pearce wrote that script using contrib/remotes2config.sh from
the git.git project source tree.

Re: basics... when reading docs doesn't help

[Guennadi Liakhovetski]

On Fri, 30 Mar 2007, Junio C Hamano wrote:

> Guennadi Liakhovetski <g.liakhovetski@gmx.de> writes:
> 
> >> Yeah, this is all a bit complex, and it takes a while to wrap your head 
> >> around it, but I have to say, once you do, the git-1.5.x layout really 
> >> *is* very powerful, and it's actually very natural too (but the "very 
> >> natural" part only comes after you have that "Aaahh!" moment!)
> >
> > Aha, so, that's how it is then! Why hasn't anybody explained this to me 
> > strait away?!:-))))
> 
> Because I and others have explained that to other people on the
> list a few times already, perhaps?  The list archive is your
> friend.

Emn, sorry, that was supposed to be a joke...

In my original post I said something like "I know, this most probably has 
been asked (many times) before, but since the questions are pretty 
generic, I don't have a very good idea what to search archives for, but 
any pointer to a thread in archive would be appreciated". I am greatful 
somebody took his time to explain a couple of things to me even knowing I 
will only understand a few percent strait away. And I am usually the first 
one to point others to list archives:-) It is easy to search for isp1761, 
but it is not so easy to search for "clone multiple trees reuse disk 
space..."

I am greatful for any help and I can well understand those who decided not 
to repeat what they've already explained a 100 times before.

> > .... There isn't a way to convert such a "old style" tree to the 
> > "new style", is there?
> 
> That also can be found in the list archive.  I think Shawn
> Pearce wrote that script using contrib/remotes2config.sh from
> the git.git project source tree.

Thanks, now this is easy to search!

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[Theodore Tso]

On Fri, Mar 30, 2007 at 12:13:02AM +0200, Guennadi Liakhovetski wrote:
> On Thu, 29 Mar 2007, J. Bruce Fields wrote:
> > Though actually on a second look, clone -l -s produces something that's
> > only 377M.  I hadn't realized how much space the build output takes up.
> > So judging from du the 1.5G Guennadi Liakhovetski mentions above seems
> > to break down into something like:
> > 
> > 	330M .git
> > 	380M working tree
> > 	750M build output

Hmm.... That doesn't look right.  My packed .git directory is 156 megs
(using post git 1.5 and repack.usedeltabaseoffset=true and
core.legacyheaders=false).

My working tree is 287M, and my build output (size of build tree minus
sources) is 1055M.   (This will vary based on .config options, obviously).

The point though is the size of the .git tree is in the noise compared
to the size of the object files.  I do share the .git repository
between trees; in fact I have a standard "base" repository which is
just a mirror of Linus's tree, and my other kernel repositories have
an .git/objects/info/alternates file which points at the base
repository for maximal sharing.

I could try to share the sources where the source files are identical,
but quite frankly, it's just never been worth the effort.  After all,
when you have a 100gig laptop drive, 287 megs isn't that much, and
it's noise compared to the size of my object files.

> Strange. Is my git 1.4.0 criminally broken? I have a clone of Linus' tree 
> on a USB disk on ext3 without any objects, which I just cloned at some 
> point and then did a couple of pulls from the same source. 

You really, really, *REALLY* want to upgrade to at least git 1.5.0.
It's is *so* much better than git 1.5.0, and it's a lot easier to make
sure your repository is kept packed, using "git gc".  

						- Ted

Re: basics... when reading docs doesn't help

[J. Bruce Fields]

On Thu, Mar 29, 2007 at 10:43:27PM -0400, Theodore Tso wrote:
> On Fri, Mar 30, 2007 at 12:13:02AM +0200, Guennadi Liakhovetski wrote:
> > On Thu, 29 Mar 2007, J. Bruce Fields wrote:
> > > Though actually on a second look, clone -l -s produces something that's
> > > only 377M.  I hadn't realized how much space the build output takes up.
> > > So judging from du the 1.5G Guennadi Liakhovetski mentions above seems
> > > to break down into something like:
> > > 
> > > 	330M .git
> > > 	380M working tree
> > > 	750M build output
> 
> Hmm.... That doesn't look right.  My packed .git directory is 156 megs
> (using post git 1.5 and repack.usedeltabaseoffset=true and
> core.legacyheaders=false).

I haven't run more than git-gc in a while, because I have local clones
and didn't want to figure out how to prune.  Hm, now that I've looked it
up:

git prune -- $(cd ../linux-clone/ && echo $(git-rev-parse --all))

just gets me the git-prune usage message.  In fact, contrary to the
prune man page, git-prune doesn't seem to accept any <head> arguments.
Isn't this a bug?  I'm on 1.5.0.3.31.ge47c.

--b.

Re: basics... when reading docs doesn't help

[Andreas Herrmann]

On Fri, Mar 30, 2007 at 12:13:02AM +0200, Guennadi Liakhovetski wrote:
> On Thu, 29 Mar 2007, J. Bruce Fields wrote:
> 
> > On Thu, Mar 29, 2007 at 02:26:10PM -0700, Junio C Hamano wrote:
> > > 
> > > How about suggesting "clone -l -s"?
> 
> Yes, but how do "advanced git users" kernel developers work? Do they just 
> do 1 clone and build / clean every time they want to test another 
> configuration / arch, or do they clone -l or what? Do they create branches 
> for each development thread, then pull / push between trees?...
> 
> > If you really want to share as much as possible, then I guess you want
> > to share the working trees too, since (as evidenced above), they're at
> > least as large as the compressed history.
> 
> But I don't want to re-build. Apart from i386 I build for a couple of ARM 
> and PPC targets too...

Seems to be trivial but:
Why don't you use "make O=/foo/bar/arch<x>-config<y>" to put output
files into separate directories? So you can have one source tree and
put each different kernel config and arch into a separate output
directory.

And if you have different sources for you trees put them into branches.

When switching between branches, atime of files are updated accordingly.
So even make should be happy with that.

Just one drawback:
Switching back and forth between two branches will cause
recompilation of sources that differ between that branches -
although nothing might have changed within a branch in the meantime.

(Not that I have used such an setup, yet.
But I think that should work.)


Regards,

Andreas

Re: basics... when reading docs doesn't help

[Guennadi Liakhovetski]

On Fri, 30 Mar 2007, Andreas Herrmann wrote:

> Seems to be trivial but:
> Why don't you use "make O=/foo/bar/arch<x>-config<y>" to put output
> files into separate directories? So you can have one source tree and
> put each different kernel config and arch into a separate output
> directory.
> 
> And if you have different sources for you trees put them into branches.
> 
> When switching between branches, atime of files are updated accordingly.
> So even make should be happy with that.
> 
> Just one drawback:
> Switching back and forth between two branches will cause
> recompilation of sources that differ between that branches -
> although nothing might have changed within a branch in the meantime.

Exactly, and since I have not only different configs, but also different 
versions, and I don't commit all modifications, so... It would be 
difficult. I think, the setup with "clone -l -s" should be the best.

Thanks to all for ideas
Guennadi
---
Guennadi Liakhovetski

Re: basics... when reading docs doesn't help

[Matthieu Moy]

"J. Bruce Fields" <bfields@fieldses.org> writes:

> If you really need a ton of build trees then you might just want to do
> cp -al or something.

But be careful that this is a potentially dangerous optimization : if
you ever use a command that doesn't break hardlinks (for example
"echo foo >> file"), the modification is applied to both trees.

-- 
Matthieu