Re: kernel.org and GIT tree rebuilding

[Jeff Garzik <jgarzik@pobox.com>]

David S. Miller wrote:
> To get a clean history to push to Linus, I typically blow
> away my trees and make fresh ones to stick patches into
> which I want to merge.
> 
> That mostly works fine here on my local systems, but I know this
> brings the master.org mirroring system to it's knees.  So what is the
> generally condoned way to do stuff like this in a more friendly way?
> 
> Should I:
> 
> 1) Do a git pull from Linus's tree once he takes my changes, then
>    ask GIT to prune the tree?  How do I do that and how does it work?

> It takes sometimes an hour for my tree updates on master.kernel.org
> to propagate to rsync.kernel.org so I can ask Linus to pull.
> That's crazy.

Unfortunately you cannot fix this by changing your actions.  This is the 
cumulative effect of all the git kernel trees on kernel.org.  It now 
takes over an hour for my non-git changes to propagate from master to 
the mirrors, as well.

This is all due to the rsync sweeps, which have to scan metric tons of 
inodes and dentries.  Orders of magnitude over the pre-git days.

ftpadmin@kernel.org folks are supposedly working on an inotify-based 
system, and an improved rsync application.  No ETA or details.

As an aside, cold-cache, git really punishes my disks.  Ted T'so noted 
that it really drains laptop batteries, too.


> 2) Should I use .git/object/ database symlinking?
> 
>    Are there any scripts out there which do this automatically?
>    Something as simple to run as "git-pull-script" and it takes
>    care of using links when possible on a local filesystem.

On both kernel.org and locally, I use 'cp -al' to duplicate the initial 
.git/objects directory, and then rsync (->kernel.org) or git-pull-script 
(<-kernel.org) to update it after that.

That definitely helps.

Maybe somebody needs to script a relink cron job for kernel.org?

	Jeff