Fwd: Re: git checkout -f branch doesn't remove extra files

Fwd: Re: git checkout -f branch doesn't remove extra files

[Dave Jones]

[-- Attachment #1: Type: text/plain, Size: 65 bytes --]

Gar, got caught by the 'reply-to' going to Tony only...

		Dave


[-- Attachment #2: Type: message/rfc822, Size: 1308 bytes --]

From: Dave Jones <davej@redhat.com>
To: "Luck, Tony" <tony.luck@intel.com>
Subject: Re: git checkout -f branch doesn't remove extra files
Date: Sat, 13 Aug 2005 00:16:13 -0400
Message-ID: <20050813041613.GA25236@redhat.com>

On Fri, Aug 12, 2005 at 10:31:17AM -0700, Luck, Tony wrote:
 > I've just got around to noticing some of the new (to
 > me) features in git, and started experimenting with
 > branches.

Seems a good day to learn new features, I was completely
unaware of git-ls-files somehow.

My git snapshot creator that builds the hourly tarballs at
http://www.codemonkey.org.uk/projects/git-snapshots/
currently does an rsync, and then a checkout, and finally
it cleans up by removing all the checked out files.
It currently does this by hand, but on learning about
this 'new' command, I thought, cool, now I can do..

git-ls-files | xargs rm -rf

however, this then leaves a bunch of empty subdirs, as
git-ls-files doesn't list the subdirs by themselves in
the output.  Am I missing some other option ?

		Dave

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Linus Torvalds]

On Sat, 13 Aug 2005, Dave Jones wrote:
>
> My git snapshot creator that builds the hourly tarballs at
> http://www.codemonkey.org.uk/projects/git-snapshots/
> currently does an rsync, and then a checkout, and finally
> it cleans up by removing all the checked out files.
> It currently does this by hand, but on learning about
> this 'new' command, I thought, cool, now I can do..
> 
> git-ls-files | xargs rm -rf
> 
> however, this then leaves a bunch of empty subdirs, as
> git-ls-files doesn't list the subdirs by themselves in
> the output.  Am I missing some other option ?

Nope, you're not missing anything, except that you shouldn't do that.

"git-ls-files" is very nice for things like

	git-ls-files | xargs grep ....

but your example is not one of them. Not without options, and not _with_
strange options.

Your example is an example of just not doing it the right way.

If you really want a temporary tree, what you do is something like

	git-checkout-cache --prefix=tmp-dir/ -f -a

and when you're done, you just do

	rm -rf tmp-dir

and you're done.

NOTE NOTE NOTE! In the above, the order of the parameters is really really 
important! "-a" takes effect when it is seen, so it needs to be last. 
Also, the "--prefix" thing really _really_ needs the slash at the end, 
because it's literally used to prefix the pathname.

HOWEVER, if all you want to do is just a tar-file, then there's a better 
solution. It's called

	snap=git-snapshot-$(date +"%Y%m%d")
	git-tar-tree HEAD $snap | gzip -9 > $snap.tar.gz

which is even easier, and a hell of a lot more efficient.

Git actually has a _lot_ of nifty tools. I didn't realize that people 
didn't know about such basic stuff as "git-tar-tree" and "git-ls-files". 

		Linus

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Dave Jones]

On Fri, Aug 12, 2005 at 10:05:11PM -0700, Linus Torvalds wrote:

 > HOWEVER, if all you want to do is just a tar-file, then there's a better 
 > solution. It's called
 > 
 > 	snap=git-snapshot-$(date +"%Y%m%d")
 > 	git-tar-tree HEAD $snap | gzip -9 > $snap.tar.gz
 > 
 > which is even easier, and a hell of a lot more efficient.

Nice. That takes noticably less time to create the sparse/git snapshots
compared to my crappy old script. And its now less than half the size.
I'm sold.

 > Git actually has a _lot_ of nifty tools. I didn't realize that people 
 > didn't know about such basic stuff as "git-tar-tree" and "git-ls-files". 

Maybe its because things are moving so fast :)  Or maybe I just wasn't
paying attention on that day. (I even read the git changes via RSS,
so I should have no excuse).

		Dave

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Linus Torvalds]

On Sat, 13 Aug 2005, Dave Jones wrote:
> 
>  > Git actually has a _lot_ of nifty tools. I didn't realize that people 
>  > didn't know about such basic stuff as "git-tar-tree" and "git-ls-files". 
> 
> Maybe its because things are moving so fast :)  Or maybe I just wasn't
> paying attention on that day. (I even read the git changes via RSS,
> so I should have no excuse).

Well, git-tar-tree has been there since late April - it's actually one of 
those really early commands. I'm pretty sure the RSS feed came later ;)

I use it all the time in doing releases, it's a lot faster than creating a 
tar tree by reading the filesystem (even if you don't have to check things 
out). A hidden pearl.

This is my crappy "release-script":

	[torvalds@g5 ~]$ cat bin/release-script
	#!/bin/sh
	stable="$1"
	last="$2"
	new="$3"
	echo "# git-tag-script v$new"
	echo "git-tar-tree v$new linux-$new | gzip -9 > ../linux-$new.tar.gz"
	echo "git-diff-tree -p v$stable v$new | gzip -9 > ../patch-$new.gz"
	echo "git-rev-list --pretty v$new ^v$last > ../ChangeLog-$new"
	echo "git-rev-list --pretty=short v$new ^v$last | git-shortlog > ../ShortLog"
	echo "git-diff-tree -p v$last v$new | git-apply --stat > ../diffstat-$new"

and when I want to do a new kernel release I literally first tag it, and 
then do

	release-script 2.6.12 2.6.13-rc6 2.6.13-rc7

and check that things look sane, and then just cut-and-paste the commands.

Yeah, it's stupid.

		Linus

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Linus Torvalds]

On Fri, 12 Aug 2005, Linus Torvalds wrote:
> 
> Git actually has a _lot_ of nifty tools. I didn't realize that people 
> didn't know about such basic stuff as "git-tar-tree" and "git-ls-files". 

Btw, I just checked that git-tar-tree is documented and has a man-page. It 
does. However, that man-page attributes authorship to me, which is wrong. 
git-tar-tree was written by Rene Scharfe <rene.scharfe@lsrfire.ath.cx>.

Don't know how many other man-pages say I wrote something just because 
people copied another man-page around. Worth fixing, though.

		Linus

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Petr Baudis]

Dear diary, on Sat, Aug 13, 2005 at 07:05:11AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> If you really want a temporary tree, what you do is something like
> 
> 	git-checkout-cache --prefix=tmp-dir/ -f -a
> 
> and when you're done, you just do
> 
> 	rm -rf tmp-dir
> 
> and you're done.
> 
> NOTE NOTE NOTE! In the above, the order of the parameters is really really 
> important! "-a" takes effect when it is seen, so it needs to be last. 
> Also, the "--prefix" thing really _really_ needs the slash at the end, 
> because it's literally used to prefix the pathname.
> 
> HOWEVER, if all you want to do is just a tar-file, then there's a better 
> solution. It's called
> 
> 	snap=git-snapshot-$(date +"%Y%m%d")
> 	git-tar-tree HEAD $snap | gzip -9 > $snap.tar.gz
> 
> which is even easier, and a hell of a lot more efficient.

To put my two cents too, Cogito has cg-export for this:

	cg-export tmp-dir

to export to a temporary directory and

	cg-export git-snapshot.tar.gz

to export to a gzipped tarball (it can also do .tar, .tgz and .tar.bz2).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
If you want the holes in your knowledge showing up try teaching
someone.  -- Alan Cox

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Sergey Vlasov]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

On Sat, 13 Aug 2005 13:00:51 +0200 Petr Baudis wrote:

> Dear diary, on Sat, Aug 13, 2005 at 07:05:11AM CEST, I got a letter
> where Linus Torvalds <torvalds@osdl.org> told me that...
[...]
> > 	snap=git-snapshot-$(date +"%Y%m%d")
> > 	git-tar-tree HEAD $snap | gzip -9 > $snap.tar.gz
> > 
> > which is even easier, and a hell of a lot more efficient.
> 
> To put my two cents too, Cogito has cg-export for this:
> 
> 	cg-export tmp-dir
> 
> to export to a temporary directory and
> 
> 	cg-export git-snapshot.tar.gz
> 
> to export to a gzipped tarball (it can also do .tar, .tgz and .tar.bz2).

However, cg-export is buggy - if you use the second argument
(cg-export DESTFILE TREE_ID), the resulting tar file is different
from what git-tar-tree produces for the same TREE_ID (unless that
TREE_ID actually points to a tree - but in practice a commit or even
tag ID is typically used).

The problem is in this line:

id=$(tree-id "$2")

This converts the passed commit or tag to the underlying tree, which
is then passed to git-tar-tree.  However, git-tar-tree can follow
such links itself, and, what's more important, it actually uses some
information from the passed commit (it writes the commit ID to the
tar file as an extended header, and sets timestamp of all archive
members to the time of the commit).  Therefore reducing the ID
passed to git-tar-tree to a plain tree ID is wrong - if a commit ID
is available, it should be used.

-- 
Sergey Vlasov

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Fwd: Re: git checkout -f branch doesn't remove extra files

[Petr Baudis]

Dear diary, on Sat, Aug 13, 2005 at 02:10:53PM CEST, I got a letter
where Sergey Vlasov <vsu@altlinux.ru> told me that...
> However, cg-export is buggy - if you use the second argument
> (cg-export DESTFILE TREE_ID), the resulting tar file is different
> from what git-tar-tree produces for the same TREE_ID (unless that
> TREE_ID actually points to a tree - but in practice a commit or even
> tag ID is typically used).
> 
> The problem is in this line:
> 
> id=$(tree-id "$2")
> 
> This converts the passed commit or tag to the underlying tree, which
> is then passed to git-tar-tree.  However, git-tar-tree can follow
> such links itself, and, what's more important, it actually uses some
> information from the passed commit (it writes the commit ID to the
> tar file as an extended header, and sets timestamp of all archive
> members to the time of the commit).  Therefore reducing the ID
> passed to git-tar-tree to a plain tree ID is wrong - if a commit ID
> is available, it should be used.

Aha! Good catch, fixed, thanks.

BTW, I also changed cg-export usage to be more consistent and flexible,
by changing it from

	cg-export DESTFILE [TREE_ID]

to

	cg-export [-r TREE_ID] DESTFILE

since I think not many people are using cg-export anyway and much less
of them passing it TREE_ID explicitly.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
If you want the holes in your knowledge showing up try teaching
someone.  -- Alan Cox