[PATCH] git-tar-tree: add a test case (resent)

[PATCH] git-tar-tree: add a test case (resent)

[Rene Scharfe]

git-tar-tree: add a simple test case.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>

diff --git a/t/t3200-tar-tree.sh b/t/t3200-tar-tree.sh
new file mode 100755
--- /dev/null
+++ b/t/t3200-tar-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+#
+# Copyright (C) 2005 Rene Scharfe
+#
+
+test_description='git-tar-tree and git-get-tar-commit-id test
+
+This test covers the topics of long paths, file contents, commit date
+handling and commit id embedding:
+
+  Paths longer than 100 characters require the use of a pax extended
+  header to store them.  The test creates files with pathes both longer
+  and shorter than 100 chars, and also checks symlinks with long and
+  short pathes both as their own name and as target path.
+
+  The contents of the repository is compared to the extracted tar
+  archive.  The repository contains simple text files, symlinks and a
+  binary file (/bin/sh).
+
+  git-tar-tree applies the commit date to every file in the archive it
+  creates.  The test sets the commit date to a specific value and checks
+  if the tar archive contains that value.
+
+  When giving git-tar-tree a commit id (in contrast to a tree id) it
+  embeds this commit id into the tar archive as a comment.  The test
+  checks the ability of git-get-tar-commit-id to figure it out from the
+  tar file.
+
+'
+
+. ./test-lib.sh
+
+test_expect_success \
+    'populate workdir' \
+    'mkdir a b c &&
+     p48=1.......10........20........30........40......48 &&
+     p50=1.......10........20........30........40........50 &&
+     p98=${p48}${p50} &&
+     echo simple textfile >a/a &&
+     echo 100 chars in path >a/${p98} &&
+     echo 101 chars in path >a/${p98}x &&
+     echo 102 chars in path >a/${p98}xx &&
+     echo 103 chars in path >a/${p98}xxx &&
+     mkdir a/bin &&
+     cp /bin/sh a/bin/sh &&
+     ln -s a a/l1 &&
+     ln -s ${p98}xx a/l100 &&
+     ln -s ${p98}xxx a/l101 &&
+     ln -s ${p98}xxx a/l${p98} &&
+     (cd a && find .) | sort >a.lst'
+
+test_expect_success \
+    'add files to repository' \
+    'find a -type f | xargs git-update-cache --add &&
+     find a -type l | xargs git-update-cache --add &&
+     treeid=`git-write-tree` &&
+     echo $treeid >treeid &&
+     TZ= GIT_COMMITTER_DATE="2005-05-27 22:00:00" \
+     git-commit-tree $treeid </dev/null >.git/HEAD'
+
+test_expect_success \
+    'git-tar-tree' \
+    'git-tar-tree HEAD >b.tar'
+
+test_expect_success \
+    'validate file modification time' \
+    'tar tvf b.tar a/a | awk \{print\ \$4,\$5\} >b.mtime &&
+     echo "2005-05-27 22:00:00" >expected.mtime &&
+     diff expected.mtime b.mtime'
+
+test_expect_success \
+    'git-get-tar-commit-id' \
+    'git-get-tar-commit-id <b.tar >b.commitid &&
+     diff .git/HEAD b.commitid'
+
+test_expect_success \
+    'extract tar archive' \
+    '(cd b && tar xf -) <b.tar'
+
+test_expect_success \
+    'validate filenames' \
+    '(cd b/a && find .) | sort >b.lst &&
+     diff a.lst b.lst'
+
+test_expect_success \
+    'validate file contents' \
+    'diff -r a b/a'
+
+test_expect_success \
+    'git-tar-tree with prefix' \
+    'git-tar-tree HEAD prefix >c.tar'
+
+test_expect_success \
+    'extract tar archive with prefix' \
+    '(cd c && tar xf -) <c.tar'
+
+test_expect_success \
+    'validate filenames with prefix' \
+    '(cd c/prefix/a && find .) | sort >c.lst &&
+     diff a.lst c.lst'
+
+test_expect_success \
+    'validate file contents with prefix' \
+    'diff -r a c/prefix/a'
+
+test_done

Re: [PATCH] git-tar-tree: add a test case (resent)

[Junio C Hamano]

I like the test but suspect it belongs to t5000 series according
to the numbering scheme Pasky originaly did.

Re: [PATCH] git-tar-tree: add a test case (resent)

[Rene Scharfe]

On Thu, Jun 02, 2005 at 01:05:36PM -0700, Junio C Hamano wrote:
> I like the test but suspect it belongs to t5000 series according
> to the numbering scheme Pasky originaly did.

The README says:
  3 - the other basic commands (e.g. ls-files)
  [...]
  5 - the pull and exporting commands

So I guess you're right.  I shouldn't have stopped reading after point
three..

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>

diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
new file mode 100755
--- /dev/null
+++ b/t/t5000-tar-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+#
+# Copyright (C) 2005 Rene Scharfe
+#
+
+test_description='git-tar-tree and git-get-tar-commit-id test
+
+This test covers the topics of long paths, file contents, commit date
+handling and commit id embedding:
+
+  Paths longer than 100 characters require the use of a pax extended
+  header to store them.  The test creates files with pathes both longer
+  and shorter than 100 chars, and also checks symlinks with long and
+  short pathes both as their own name and as target path.
+
+  The contents of the repository is compared to the extracted tar
+  archive.  The repository contains simple text files, symlinks and a
+  binary file (/bin/sh).
+
+  git-tar-tree applies the commit date to every file in the archive it
+  creates.  The test sets the commit date to a specific value and checks
+  if the tar archive contains that value.
+
+  When giving git-tar-tree a commit id (in contrast to a tree id) it
+  embeds this commit id into the tar archive as a comment.  The test
+  checks the ability of git-get-tar-commit-id to figure it out from the
+  tar file.
+
+'
+
+. ./test-lib.sh
+
+test_expect_success \
+    'populate workdir' \
+    'mkdir a b c &&
+     p48=1.......10........20........30........40......48 &&
+     p50=1.......10........20........30........40........50 &&
+     p98=${p48}${p50} &&
+     echo simple textfile >a/a &&
+     echo 100 chars in path >a/${p98} &&
+     echo 101 chars in path >a/${p98}x &&
+     echo 102 chars in path >a/${p98}xx &&
+     echo 103 chars in path >a/${p98}xxx &&
+     mkdir a/bin &&
+     cp /bin/sh a/bin/sh &&
+     ln -s a a/l1 &&
+     ln -s ${p98}xx a/l100 &&
+     ln -s ${p98}xxx a/l101 &&
+     ln -s ${p98}xxx a/l${p98} &&
+     (cd a && find .) | sort >a.lst'
+
+test_expect_success \
+    'add files to repository' \
+    'find a -type f | xargs git-update-cache --add &&
+     find a -type l | xargs git-update-cache --add &&
+     treeid=`git-write-tree` &&
+     echo $treeid >treeid &&
+     TZ= GIT_COMMITTER_DATE="2005-05-27 22:00:00" \
+     git-commit-tree $treeid </dev/null >.git/HEAD'
+
+test_expect_success \
+    'git-tar-tree' \
+    'git-tar-tree HEAD >b.tar'
+
+test_expect_success \
+    'validate file modification time' \
+    'tar tvf b.tar a/a | awk \{print\ \$4,\$5\} >b.mtime &&
+     echo "2005-05-27 22:00:00" >expected.mtime &&
+     diff expected.mtime b.mtime'
+
+test_expect_success \
+    'git-get-tar-commit-id' \
+    'git-get-tar-commit-id <b.tar >b.commitid &&
+     diff .git/HEAD b.commitid'
+
+test_expect_success \
+    'extract tar archive' \
+    '(cd b && tar xf -) <b.tar'
+
+test_expect_success \
+    'validate filenames' \
+    '(cd b/a && find .) | sort >b.lst &&
+     diff a.lst b.lst'
+
+test_expect_success \
+    'validate file contents' \
+    'diff -r a b/a'
+
+test_expect_success \
+    'git-tar-tree with prefix' \
+    'git-tar-tree HEAD prefix >c.tar'
+
+test_expect_success \
+    'extract tar archive with prefix' \
+    '(cd c && tar xf -) <c.tar'
+
+test_expect_success \
+    'validate filenames with prefix' \
+    '(cd c/prefix/a && find .) | sort >c.lst &&
+     diff a.lst c.lst'
+
+test_expect_success \
+    'validate file contents with prefix' \
+    'diff -r a c/prefix/a'
+
+test_done

Re: [PATCH] git-tar-tree: add a test case (resent)

[Linus Torvalds]

On Thu, 2 Jun 2005, Rene Scharfe wrote:
>
> git-tar-tree: add a simple test case.

I get:

	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
	* FAIL 8: validate file contents diff -r a b/a
	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a

Hmm?

		Linus

Re: [PATCH] git-tar-tree: add a test case (resent)

[Rene Scharfe]

Linus Torvalds schrieb:
> 
> On Thu, 2 Jun 2005, Rene Scharfe wrote:
> 
>>git-tar-tree: add a simple test case.
> 
> 
> I get:
> 
> 	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
> 	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
> 	* FAIL 8: validate file contents diff -r a b/a
> 	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
> 	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a

What version of tar do you use?  Also, can you please send me the output
the following?

   cd t; sh t5000-tar-tree.sh; cd trash; tar tvf b.tar

Thanks,
Rene

Re: [PATCH] git-tar-tree: add a test case (resent)

[Linus Torvalds]

On Fri, 3 Jun 2005, Rene Scharfe wrote:
> 
> What version of tar do you use? 

	torvalds@ppc970:~> tar --version
	tar (GNU tar) 1.13.25
	Copyright © 2001 Free Software Foundation, Inc.
	This program comes with NO WARRANTY, to the extent permitted by law.
	You may redistribute it under the terms of the GNU General Public License;
	see the file named COPYING for details.
	Written by John Gilmore and Jay Fenlason.

> Also, can you please send me the output
> the following?
> 
>    cd t; sh t5000-tar-tree.sh; cd trash; tar tvf b.tar

I get

	?rw------- git/git          52 1969-12-31 16:00:00 pax_global_header unknown file type `g'
	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 a/1.......10........20........30........40......481.......10........20........30........40........50
	?rw------- git/git         111 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.data
	?rw------- git/git         112 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.data
	?rw------- git/git         113 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.paxheader unknown file type `x'
	-rw-r--r-- git/git          18 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.data
	-rw-r--r-- git/git          16 2005-05-27 15:00:00 a/a
	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/bin/
	-rwxr-xr-x git/git      719924 2005-05-27 15:00:00 a/bin/sh
	tar: Unexpected EOF in archive
	tar: Error is not recoverable: exiting now

which doesn't tell me anything, except I assume it means that my tar 
doesn't understand extended headers.

		Linus

Re: [PATCH] git-tar-tree: add a test case (resent)

[Rene Scharfe]

Linus Torvalds schrieb:
> 	torvalds@ppc970:~> tar --version
> 	tar (GNU tar) 1.13.25
> 	Copyright © 2001 Free Software Foundation, Inc.
> 	This program comes with NO WARRANTY, to the extent permitted by law.
> 	You may redistribute it under the terms of the GNU General Public License;
> 	see the file named COPYING for details.
> 	Written by John Gilmore and Jay Fenlason.

[...]

> I get
> 
> 	?rw------- git/git          52 1969-12-31 16:00:00 pax_global_header unknown file type `g'
> 	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 a/1.......10........20........30........40......481.......10........20........30........40........50
> 	?rw------- git/git         111 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 37e07bc886b166ad2525f1cc7a2df9bda0e07a02.data
> 	?rw------- git/git         112 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 f72c8a14fa272506e5de2d779cfe088769158b61.data
> 	?rw------- git/git         113 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.paxheader unknown file type `x'
> 	-rw-r--r-- git/git          18 2005-05-27 15:00:00 b53ac87f1a63ea1a2eec65cc8b5bed9130b8c114.data
> 	-rw-r--r-- git/git          16 2005-05-27 15:00:00 a/a
> 	drwxr-xr-x git/git           0 2005-05-27 15:00:00 a/bin/
> 	-rwxr-xr-x git/git      719924 2005-05-27 15:00:00 a/bin/sh
> 	tar: Unexpected EOF in archive
> 	tar: Error is not recoverable: exiting now
> 
> which doesn't tell me anything, except I assume it means that my tar 
> doesn't understand extended headers.

That's true, but you don't need them if all your pathes' lengths are 100
or shorter.  Of course the test exercising long paths will always fail
with your version of tar.  I should split it out better, so you will at
least get a positive result for short paths.  I just didn't expect such
an old version of tar to be in use anymore. :-P

But your output shows a more serious problem: "Unexpected EOF" means the
archive is missing a piece.  I sent you a patch for that bug in my
previous mail.

Thanks,
Rene

Re: [PATCH] git-tar-tree: add a test case (resent)

[Rene Scharfe]

Linus Torvalds schrieb:
> 
> On Thu, 2 Jun 2005, Rene Scharfe wrote:
> 
>>git-tar-tree: add a simple test case.
> 
> 
> I get:
> 
> 	* FAIL 6: extract tar archive (cd b && tar xf -) <b.tar
> 	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
> 	* FAIL 8: validate file contents diff -r a b/a
> 	* FAIL 11: validate filenames with prefix (cd c/prefix/a && find .) | sort >c.lst &&
> 	* FAIL 12: validate file contents with prefix diff -r a c/prefix/a

Meine Fresse, what a stoopid bug.  Fortunately I introduced it with one
of the other two patches sent together with the test case patch, namely
"[PATCH] git-tar-tree: cleanup write_trailer() (resent)".  Of course I
only wanted to demonstrate how useful those tests really are. ;-)

Btw., I got it right the first time and messed up when resending..

write_trailer() writes the last 10k (a full block) of the tar archive.
write_if_needed() writes out a block *if* it is full and then sets
the offset to 0.  In nine out of ten cases the messed up write_trailer()
function didn't manage to fill the block thus not writing anything at
all, truncating the archive.  I was "lucky" to hit the other case and so
my testing ran OK.

Here's a patch.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>

--- a/tar-tree.c
+++ b/tar-tree.c
@@ -77,7 +77,7 @@
 	write_if_needed();
 	get_record();
 	write_if_needed();
-	if (offset) {
+	while (offset) {
 		get_record();
 		write_if_needed();
 	}

Re: [PATCH] git-tar-tree: add a test case (resent)

[Linus Torvalds]

On Fri, 3 Jun 2005, Rene Scharfe wrote:
> 
> Here's a patch.

This one fixes failure 6 for me, but 7, 8, 11 and 12 still fail.

Doing a

	sh t5000-tar-tree.sh -i -v

ends with

	* expecting success: (cd b/a && find .) | sort >b.lst &&
	     diff a.lst b.lst
	3,5d2
	< ./1.......10........20........30........40......481.......10........20........30........40........50x
	< ./1.......10........20........30........40......481.......10........20........30........40........50xx
	< ./1.......10........20........30........40......481.......10........20........30........40........50xxx
	10d6
	< ./l1.......10........20........30........40......481.......10........20........30........40........50
	12d7
	< ./l101
	* FAIL 7: validate filenames (cd b/a && find .) | sort >b.lst &&
	     diff a.lst b.lst

and you already saw my output from "tar tvf b.tar", so you already know 
that this is probably due to not understanding extended headers.

		Linus

Re: [PATCH] git-tar-tree: add a test case (resent)

[Linus Torvalds]

On Fri, 3 Jun 2005, Linus Torvalds wrote:
> 
> and you already saw my output from "tar tvf b.tar", so you already know 
> that this is probably due to not understanding extended headers.

My other ppc64 box (running Fedora core 3.92) has no problems, so this is 
apparently something that has been changed between tar 1.13.25 and 1.15.1 
(YDL 4.0 vs Fedora Core tar versions).

		Linus