how to git-archive ignore some files?

how to git-archive ignore some files?

[Victor Bogado da Silva Lins]

I want to use git-archive to create source packages, but my git
repository has a .gitignore that I don't want to be pushed into the
archive. So is it possible to keep the .gitignore file in the repository
and create an archive(*) without it? 

*) using git-archive, off course. 

Re: how to git-archive ignore some files?

[Sean Estabrooks]

On Tue, 06 May 2008 15:15:31 -0300
Victor Bogado da Silva Lins <victor@bogado.net> wrote:

> I want to use git-archive to create source packages, but my git
> repository has a .gitignore that I don't want to be pushed into the
> archive. So is it possible to keep the .gitignore file in the repository
> and create an archive(*) without it? 
> 
> *) using git-archive, off course. 
> 

Hi Victor,

You can do something like:

git archive HEAD  | \
	tar --delete .gitignore .mailmap .gitattributes | \
	bzip2 -c > archive.tar.bz2

HTH,
Sean

Re: how to git-archive ignore some files?

[René Scharfe]

Sean Estabrooks schrieb:
> On Tue, 06 May 2008 15:15:31 -0300
> Victor Bogado da Silva Lins <victor@bogado.net> wrote:
> 
>> I want to use git-archive to create source packages, but my git
>> repository has a .gitignore that I don't want to be pushed into the
>> archive. So is it possible to keep the .gitignore file in the repository
>> and create an archive(*) without it? 
>>
>> *) using git-archive, off course. 
>>
> 
> Hi Victor,
> 
> You can do something like:
> 
> git archive HEAD  | \
> 	tar --delete .gitignore .mailmap .gitattributes | \
> 	bzip2 -c > archive.tar.bz2

Or, if there is only a small number of files in your repository, you
could do something like this:

   git archive HEAD -- \
      $(git ls-tree -r --name-only HEAD | grep -v '^\.gitignore$')

To really solve this without external tools, git-archive needed to grow
an --exclude option like git-ls-files, though.

René

Re: how to git-archive ignore some files?

[Junio C Hamano]

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:

> To really solve this without external tools, git-archive needed to grow
> an --exclude option like git-ls-files, though.

Because exclude is never about ignoring what is already tracked, an option
"like" git-ls-files would not help this case, I am afraid.

Re: how to git-archive ignore some files?

[René Scharfe]

Junio C Hamano schrieb:
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> 
>> To really solve this without external tools, git-archive needed to
>> grow an --exclude option like git-ls-files, though.
> 
> Because exclude is never about ignoring what is already tracked, an
> option "like" git-ls-files would not help this case, I am afraid.

True, it should be named differently, e.g. --ignore.

Another idea: would it make sense to have an attribute for that instead
(or in addition to the command line parameter)?  I.e. files marked
export-ignore would not end up in the archive.  I suspect the set of
files to ignore doesn't change from archive to archive one creates.

René

Re: how to git-archive ignore some files?

[Junio C Hamano]

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:

> Another idea: would it make sense to have an attribute for that instead
> (or in addition to the command line parameter)?  I.e. files marked
> export-ignore would not end up in the archive.  I suspect the set of
> files to ignore doesn't change from archive to archive one creates.

That sounds extremely sensible.

Should/does git-archive read .gitattributes from the tree being exported?

Re: how to git-archive ignore some files?

[Jakub Narebski]

Junio C Hamano <gitster@pobox.com> writes:

> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> 
> > Another idea: would it make sense to have an attribute for that instead
> > (or in addition to the command line parameter)?  I.e. files marked
> > export-ignore would not end up in the archive.  I suspect the set of
> > files to ignore doesn't change from archive to archive one creates.
> 
> That sounds extremely sensible.
> 
> Should/does git-archive read .gitattributes from the tree being exported?

If I understand and remember correctly, current implementation of
gitattributes, and current tools (git-check-attr), use and can use
only working directory version of .gitattributes, in part because of
chicken-and-egg problem (attributes effect checkout) IIRC.

It would be nice if git-archive read and use .gitattribues from
a tree being exported; this would for example make `export-subs`
work even for bare repositories.  This would be needed for
`export-ignore`.
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: how to git-archive ignore some files?

[René Scharfe]

Jakub Narebski schrieb:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
>>
>>> Another idea: would it make sense to have an attribute for that instead
>>> (or in addition to the command line parameter)?  I.e. files marked
>>> export-ignore would not end up in the archive.  I suspect the set of
>>> files to ignore doesn't change from archive to archive one creates.
>> That sounds extremely sensible.
>>
>> Should/does git-archive read .gitattributes from the tree being exported?
> 
> If I understand and remember correctly, current implementation of
> gitattributes, and current tools (git-check-attr), use and can use
> only working directory version of .gitattributes, in part because of
> chicken-and-egg problem (attributes effect checkout) IIRC.
> 
> It would be nice if git-archive read and use .gitattribues from
> a tree being exported; this would for example make `export-subs`
> work even for bare repositories.  This would be needed for
> `export-ignore`.

git-archive uses the routines from attr.c, and this is where the ability
to read attributes from git trees should be added.  It's independent from
ignoring files when creating archives.  Maybe it would be a good idea to
add this feature; in the meantime users need to resort to adding their
attributes to $GIT_DIR/info/attributes.

Anyway, here's a patch to add export-ignore support to git-archive.
Files and directories with this attribute are not added to archives.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
---
 Documentation/gitattributes.txt |    6 ++++++
 archive-tar.c                   |    2 ++
 archive-zip.c                   |    2 ++
 archive.c                       |   13 +++++++++++++
 archive.h                       |    1 +
 t/t5000-tar-tree.sh             |    9 +++++++++
 6 files changed, 33 insertions(+), 0 deletions(-)

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 471754e..6e67990 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -502,6 +502,12 @@ frotz	unspecified
 Creating an archive
 ~~~~~~~~~~~~~~~~~~~
 
+`export-ignore`
+^^^^^^^^^^^^^^^
+
+Files and directories with the attribute `export-ignore` won't be added to
+archive files.
+
 `export-subst`
 ^^^^^^^^^^^^^^
 
diff --git a/archive-tar.c b/archive-tar.c
index d7598f9..99db58f 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -247,6 +247,8 @@ static int write_tar_entry(const unsigned char *sha1,
 	strbuf_grow(&path, PATH_MAX);
 	strbuf_add(&path, base, baselen);
 	strbuf_addstr(&path, filename);
+	if (is_archive_path_ignored(path.buf + base_len))
+		return 0;
 	if (S_ISDIR(mode) || S_ISGITLINK(mode)) {
 		strbuf_addch(&path, '/');
 		buffer = NULL;
diff --git a/archive-zip.c b/archive-zip.c
index 18c0f87..5742762 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -176,6 +176,8 @@ static int write_zip_entry(const unsigned char *sha1,
 	crc = crc32(0, NULL, 0);
 
 	path = construct_path(base, baselen, filename, S_ISDIR(mode), &pathlen);
+	if (is_archive_path_ignored(path + base_len))
+		return 0;
 	if (verbose)
 		fprintf(stderr, "%s\n", path);
 	if (pathlen > 0xffff) {
diff --git a/archive.c b/archive.c
index 7a32c19..6502b76 100644
--- a/archive.c
+++ b/archive.c
@@ -82,3 +82,16 @@ void *sha1_file_to_archive(const char *path, const unsigned char *sha1,
 	return buffer;
 }
 
+int is_archive_path_ignored(const char *path)
+{
+	static struct git_attr *attr_export_ignore;
+	struct git_attr_check check[1];
+
+	if (!attr_export_ignore)
+		attr_export_ignore = git_attr("export-ignore", 13);
+
+	check[0].attr = attr_export_ignore;
+	if (git_checkattr(path, ARRAY_SIZE(check), check))
+		return 0;
+	return ATTR_TRUE(check[0].value);
+}
diff --git a/archive.h b/archive.h
index 5791e65..ddf004a 100644
--- a/archive.h
+++ b/archive.h
@@ -44,5 +44,6 @@ extern int write_zip_archive(struct archiver_args *);
 extern void *parse_extra_zip_args(int argc, const char **argv);
 
 extern void *sha1_file_to_archive(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size, const struct commit *commit);
+extern int is_archive_path_ignored(const char *path);
 
 #endif	/* ARCHIVE_H */
diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
index 9b0baac..3f1e25d 100755
--- a/t/t5000-tar-tree.sh
+++ b/t/t5000-tar-tree.sh
@@ -45,6 +45,11 @@ test_expect_success \
      (cd a && find .) | sort >a.lst'
 
 test_expect_success \
+    'add ignored file' \
+    'echo ignore me >a/ignored &&
+     echo ignored export-ignore >.gitattributes'
+
+test_expect_success \
     'add files to repository' \
     'find a -type f | xargs git update-index --add &&
      find a -type l | xargs git update-index --add &&
@@ -54,6 +59,10 @@ test_expect_success \
      git commit-tree $treeid </dev/null)'
 
 test_expect_success \
+    'remove ignored file' \
+    'rm a/ignored'
+
+test_expect_success \
     'git archive' \
     'git archive HEAD >b.tar'
 

Re: how to git-archive ignore some files?

[David Bryson]

[-- Attachment #1: Type: text/plain, Size: 3050 bytes --]

Hi Victor,

On Tue, May 06, 2008 at 03:15:31PM -0300 or thereabouts, Victor Bogado da Silva Lins wrote:
> I want to use git-archive to create source packages, but my git
> repository has a .gitignore that I don't want to be pushed into the
> archive. So is it possible to keep the .gitignore file in the repository
> and create an archive(*) without it? 
> 
> *) using git-archive, off course. 

Others already discussed the --delete method with tar, and I do something
similar except slightly more flexible.  I have for example, a git repo
that checks out directories from CVS, and of course I don't want these
CVS directories in my 'release'.  So put a executable '.release' file in
each repo that tells my release script how to fix up the tarball to be
production ready.

Here is my release script, modeled after some lines Linus put in the git
documentation.  It is meant to be run, reviewed for mistakes, and then
have the commands run via cut-paste.

#!/bin/bash 
stable="$1"
last="$2"
new="$3"
chip="$4"
rel=`pwd | cut -d'/' -f6 | sed -e 's/\.git//'`

if [ -z $chip ]; then
	reldir=$rel
else
	reldir=$rel-$chip
fi

reldir="$reldir-$new"
tarfile="$reldir.tar"

if [ -d $reldir ]; then
	echo "release directory $reldir already exists, please remove" 1>&2
	exit 1
fi

if [ -f "../$reldir.tar" ]; then
	echo "Release file ../$reldir.tar already exists, please remove" 1>&2
	exit 1
fi

if [ -f "../$reldir.tar.gz" ]; then
	echo "Release file ../$reldir.tar.gz already exists, please remove" 1>&2
	exit 1
fi

echo "#git tag v$new"
echo "Tagging new release v$new" 1>&2
echo "git archive --prefix=$reldir/ v$new > ../$tarfile"
echo "mkdir $reldir"
#echo "git diff v$stable v$new | gzip -9 > ../patch-$new.gz"
echo "git log --no-merges v$new ^v$last > $reldir/ChangeLog-$new"
echo "git shortlog --no-merges v$new ^v$last > $reldir/ShortLog"
echo "git diff --stat --summary -M v$last v$new > $reldir/diffstat-$new"
echo "tar -rf ../$tarfile $reldir/*"
./.release $reldir ../$tarfile
if [ $? != 0 ]; then
	echo ".release script failed! removing generated files" 1>&2
	echo "rm -rf $reldir ../$tarfile"
	exit 1
fi
echo "gzip -9 < ../$tarfile > ../$tarfile.gz"
echo "rm -rf $reldir ../$tarfile"
echo "New release at ../$tarfile.gz" 1>&2

And an example .release file

#!/bin/bash
dirlist="make foo_include scripts"
excludes="--exclude CVS --exclude '*.o' --exclude '*.swp' --exclude '*.d' --exclude '*.a'"
reldir=$1
tarfile=$2
#echo "make doc && make -C doc/latex pdf"
echo "cp -a $dirlist $reldir"
echo "cp Makefile.release $reldir/Makefile"
echo "tar $excludes -rf $tarfile $reldir/*"

# write over the makefile
echo "cp Makefile.release $reldir/Makefile"
echo "tar -u -f $tarfile $reldir/Makefile"
echo "tar --delete --file=$tarfile $reldir/Makefile.release"

# delete the doxygen file
echo "cp Doxyfile $reldir"
echo "tar --delete --file=$tarfile $reldir/Doxyfile"

# remove traces of the release file
echo "tar --delete --file=$tarfile $reldir/.release"


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: how to git-archive ignore some files?

[Johannes Schindelin]

Hi,

On Thu, 8 May 2008, David Bryson wrote:

> On Tue, May 06, 2008 at 03:15:31PM -0300 or thereabouts, Victor Bogado da Silva Lins wrote:
> > I want to use git-archive to create source packages, but my git 
> > repository has a .gitignore that I don't want to be pushed into the 
> > archive. So is it possible to keep the .gitignore file in the 
> > repository and create an archive(*) without it?
> > 
> > *) using git-archive, off course. 
> 
> Others already discussed the --delete method with tar, [...]

I wonder if the most natural way with Git would not be to create a 
throw-away tree:

cp .git/index .git/tmp-index &&
GIT_INDEX_FILE=.git/tmp-index git rm --cached .gitignore &&
git archive $(GIT_INDEX_FILE=.git/tmp-index git write-tree) &&
rm .git/tmp-index

WARNING: this has not seen any testing.

Ciao,
Dscho