Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

I was wondering if anybody had any suggestions on a sane way to add a
file to a git-archive when it is being produced.

For  instance build procedures that expect to be run inside of a git
WD wont work if built from an archived version of the tree. Being able
to provide a file of additional data to the archive package would be a
very convenient way to work around this.

Ideally id like to be able to specify a set of additional files to
include in the archive as part of the git-archive command line
interface, but I'd be nearly as a happy with almost any solution other
than the one I came up with, which is to use archive to generate a tar
file, then use tar to append the additional files to the tar, and then
compress it. This process turns out to be quite slow in comparison to
producing a compressed archive directly from git-archive.

Another question is whether anyone has any advice on the best way to
find out the "best" branch an arbitrary commit is on. Where best can
be flexibly definied to handle commits that are reachable from
multiple branches.  I have hacked a solution involving git-log and
grep, but it performs quite poorly. I was wondering if there is a
better solution.

cheers,
Yves



-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[Avery Pennarun]

On Wed, Jul 29, 2009 at 4:15 AM, demerphq<demerphq@gmail.com> wrote:
> Another question is whether anyone has any advice on the best way to
> find out the "best" branch an arbitrary commit is on. Where best can
> be flexibly definied to handle commits that are reachable from
> multiple branches.  I have hacked a solution involving git-log and
> grep, but it performs quite poorly. I was wondering if there is a
> better solution.

'git describe' does this.

Avery

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[Santi Béjar]

2009/7/29 demerphq <demerphq@gmail.com>:
> I was wondering if anybody had any suggestions on a sane way to add a
> file to a git-archive when it is being produced.
>
> For  instance build procedures that expect to be run inside of a git
> WD wont work if built from an archived version of the tree. Being able
> to provide a file of additional data to the archive package would be a
> very convenient way to work around this.
>
> Ideally id like to be able to specify a set of additional files to
> include in the archive as part of the git-archive command line
> interface, but I'd be nearly as a happy with almost any solution other
> than the one I came up with, which is to use archive to generate a tar
> file, then use tar to append the additional files to the tar, and then
> compress it. This process turns out to be quite slow in comparison to
> producing a compressed archive directly from git-archive.

Then I don't know. It is exactly the way it is done in git.git itself,
look the dist target in the makefile.

>
> Another question is whether anyone has any advice on the best way to
> find out the "best" branch an arbitrary commit is on. Where best can
> be flexibly definied to handle commits that are reachable from
> multiple branches.  I have hacked a solution involving git-log and
> grep, but it performs quite poorly. I was wondering if there is a
> better solution.

The "best" tag is easy: git describe commit. For branches I think you
could use "git name-ref --refs=refs/heads/* commit", because git
describe does not have a --branches flag.

HTH,
Santi

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

2009/7/29 Santi Béjar <santi@agolina.net>:
> 2009/7/29 demerphq <demerphq@gmail.com>:
>> I was wondering if anybody had any suggestions on a sane way to add a
>> file to a git-archive when it is being produced.
>>
>> For  instance build procedures that expect to be run inside of a git
>> WD wont work if built from an archived version of the tree. Being able
>> to provide a file of additional data to the archive package would be a
>> very convenient way to work around this.
>>
>> Ideally id like to be able to specify a set of additional files to
>> include in the archive as part of the git-archive command line
>> interface, but I'd be nearly as a happy with almost any solution other
>> than the one I came up with, which is to use archive to generate a tar
>> file, then use tar to append the additional files to the tar, and then
>> compress it. This process turns out to be quite slow in comparison to
>> producing a compressed archive directly from git-archive.
>
> Then I don't know. It is exactly the way it is done in git.git itself,
> look the dist target in the makefile.

So then git also would benefit from support in git-archive for adding
arbitrary files to the archive during generation?

One problem here is that we (the Perl project) are supporting VMS
smoke-testers and need the ability to provide snapshots that are
buildable without using git at all. So we hacked the snapshot support
in gitweb to do this. However we dont want to do this for every
commit, just the ones that people actually fetch. So while the process
we use is fine for rolling a dist, its not so fine when people are
expecting a download to start promptly after clicking on the snapshot
link.

>
>>
>> Another question is whether anyone has any advice on the best way to
>> find out the "best" branch an arbitrary commit is on. Where best can
>> be flexibly definied to handle commits that are reachable from
>> multiple branches.  I have hacked a solution involving git-log and
>> grep, but it performs quite poorly. I was wondering if there is a
>> better solution.
>
> The "best" tag is easy: git describe commit. For branches I think you
> could use "git name-ref --refs=refs/heads/* commit", because git
> describe does not have a --branches flag.

Dang, I guess this is from a newer release than mine. So now i have an
excuse to upgrade.

:-)

Yves




-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[Santi Béjar]

2009/7/29 demerphq <demerphq@gmail.com>:
> 2009/7/29 Santi Béjar <santi@agolina.net>:
>> 2009/7/29 demerphq <demerphq@gmail.com>:
>>> Another question is whether anyone has any advice on the best way to
>>> find out the "best" branch an arbitrary commit is on. Where best can
>>> be flexibly definied to handle commits that are reachable from
>>> multiple branches.  I have hacked a solution involving git-log and
>>> grep, but it performs quite poorly. I was wondering if there is a
>>> better solution.
>>
>> The "best" tag is easy: git describe commit. For branches I think you
>> could use "git name-ref --refs=refs/heads/* commit", because git
>> describe does not have a --branches flag.
>
> Dang, I guess this is from a newer release than mine. So now i have an
> excuse to upgrade.

No, it is quite old (the --refs flag since the v1.5.1). If the problem
is that you don't find the "git name-ref" command is because it is
"git name-rev", oops.

Santi

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

2009/7/29 Santi Béjar <santi@agolina.net>:
> 2009/7/29 demerphq <demerphq@gmail.com>:
>> 2009/7/29 Santi Béjar <santi@agolina.net>:
>>> 2009/7/29 demerphq <demerphq@gmail.com>:
>>>> Another question is whether anyone has any advice on the best way to
>>>> find out the "best" branch an arbitrary commit is on. Where best can
>>>> be flexibly definied to handle commits that are reachable from
>>>> multiple branches.  I have hacked a solution involving git-log and
>>>> grep, but it performs quite poorly. I was wondering if there is a
>>>> better solution.
>>>
>>> The "best" tag is easy: git describe commit. For branches I think you
>>> could use "git name-ref --refs=refs/heads/* commit", because git
>>> describe does not have a --branches flag.
>>
>> Dang, I guess this is from a newer release than mine. So now i have an
>> excuse to upgrade.
>
> No, it is quite old (the --refs flag since the v1.5.1). If the problem
> is that you don't find the "git name-ref" command is because it is
> "git name-rev", oops.

Dang, guess i need a different excuse. :-)

But it doesn't seem to do what i need:

$ git name-rev --refs=refs/heads/* faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7
faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7 undefined

But i think that just because in this case I need remote refs:

git name-rev --refs=refs/remotes/* faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7
faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7 remotes/origin/maint-5.005~25

Which is definitely better. Can I safely strip the ~25 off the end to
get the real branch name?

BTW, this is the shell version of what I'm currently using:

$ for b in $(git branch -r); do git log --pretty='format:%H' $b | grep
faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7 && echo $b; done
faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7
origin/maint-5.005

The main difference with the real code I use (in perl) is that I can
specify the order of the branches to be searched, so that I can ensure
that if its on two it is reported to be "from" the most likely
candidate. And of course the perl version I use stops searching as
soon as it finds a match.

Anyway, if I can reliably strip off the ~25 then this is a big step
forward for me.

Thanks Santi, and of course thanks Johannes (author of name-rev).

cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[Santi Béjar]

2009/7/29 demerphq <demerphq@gmail.com>:
> 2009/7/29 Santi Béjar <santi@agolina.net>:
>> 2009/7/29 demerphq <demerphq@gmail.com>:
>>> 2009/7/29 Santi Béjar <santi@agolina.net>:
>>>> 2009/7/29 demerphq <demerphq@gmail.com>:
>>>>> Another question is whether anyone has any advice on the best way to
>>>>> find out the "best" branch an arbitrary commit is on. Where best can
>>>>> be flexibly definied to handle commits that are reachable from
>>>>> multiple branches.  I have hacked a solution involving git-log and
>>>>> grep, but it performs quite poorly. I was wondering if there is a
>>>>> better solution.
>>>>
>>>> The "best" tag is easy: git describe commit. For branches I think you
>>>> could use "git name-ref --refs=refs/heads/* commit", because git
>>>> describe does not have a --branches flag.
>>>
>>> Dang, I guess this is from a newer release than mine. So now i have an
>>> excuse to upgrade.
>>
>> No, it is quite old (the --refs flag since the v1.5.1). If the problem
>> is that you don't find the "git name-ref" command is because it is
>> "git name-rev", oops.
>
> Dang, guess i need a different excuse. :-)
>
> But it doesn't seem to do what i need:
>
> $ git name-rev --refs=refs/heads/* faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7
> faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7 undefined
>
> But i think that just because in this case I need remote refs:
>
> git name-rev --refs=refs/remotes/* faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7
> faa7dc9f4d3a618b0ad8b3c95edd54e24c6976e7 remotes/origin/maint-5.005~25
>
> Which is definitely better. Can I safely strip the ~25 off the end to
> get the real branch name?

Yes, it describes the relation between both, you can strip everything
after ^ or ~ (and you can use the --name-only to remove the sha1).

HTH,
Santi

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[René Scharfe]

demerphq schrieb:
> So then git also would benefit from support in git-archive for adding
> arbitrary files to the archive during generation?

Yes, and this has come up before.

How about the following?  It's missing documentation and a test case,
but you could try

	$ git archive --add-file extra HEAD >HEAD+extra.tar

or

	$ git archive --prefix=a/ --add-file extra --prefix=b/ HEAD >ba.tar

Only the file name part (after the last slash) of the extra file is used,
together with the prefix, to form the path of the archive entry.

Opening the extra files when parsing the command line arguments and closing
them after they have been written into the archive is a bit iffy, but it's
impractical to report open errors after parts of the archive have already
been created.

René

---
 archive.c |   74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 archive.h |    3 ++
 2 files changed, 71 insertions(+), 6 deletions(-)

diff --git a/archive.c b/archive.c
index 0bca9ca..368e7e8 100644
--- a/archive.c
+++ b/archive.c
@@ -25,6 +25,12 @@ static const struct archiver {
 	{ "zip", write_zip_archive, USES_ZLIB_COMPRESSION },
 };
 
+struct extra_file {
+	char *path;
+	int fd;
+	struct extra_file *next;
+};
+
 static void format_subst(const struct commit *commit,
                          const char *src, size_t len,
                          struct strbuf *buf)
@@ -147,12 +153,29 @@ static int write_archive_entry(const unsigned char *sha1, const char *base,
 	return err;
 }
 
+static int write_extra_file(struct extra_file *extra_file,
+			    struct archiver_args *args,
+			    write_archive_entry_fn_t write_entry)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int err;
+
+	if (strbuf_read(&buf, extra_file->fd, 0) < 0)
+		return error("cannot read extra file: %s", extra_file->path);
+	close(extra_file->fd);
+	err = write_entry(args, null_sha1, extra_file->path,
+			  strlen(extra_file->path), 0100644, buf.buf, buf.len);
+	strbuf_release(&buf);
+	return err;
+}
+
 int write_archive_entries(struct archiver_args *args,
 		write_archive_entry_fn_t write_entry)
 {
 	struct archiver_context context;
 	struct unpack_trees_options opts;
 	struct tree_desc t;
+	struct extra_file *extra_file = args->extra_files;
 	int err;
 
 	if (args->baselen > 0 && args->base[args->baselen - 1] == '/') {
@@ -191,6 +214,12 @@ int write_archive_entries(struct archiver_args *args,
 			args->pathspec, write_archive_entry, &context);
 	if (err == READ_TREE_RECURSIVE)
 		err = 0;
+
+	while (!err && extra_file) {
+		err = write_extra_file(extra_file, args, write_entry);
+		extra_file = extra_file->next;
+	}
+
 	return err;
 }
 
@@ -265,11 +294,41 @@ static void parse_treeish_arg(const char **argv,
 	{ OPTION_SET_INT, (s), NULL, (v), NULL, "", \
 	  PARSE_OPT_NOARG | PARSE_OPT_NONEG | PARSE_OPT_HIDDEN, NULL, (p) }
 
+static int extra_files_cb(const struct option *opt, const char *arg, int unset)
+{
+	struct archiver_args *args = opt->value;
+	struct extra_file *e = xmalloc(sizeof(struct extra_file));
+	struct strbuf path = STRBUF_INIT;
+	const char *last_slash, *prefix = args->base;
+
+	e->fd = open(arg, O_RDONLY);
+	if (e->fd == -1)
+		die_errno("Could not open file '%s'", arg);
+
+	if (prefix)
+		strbuf_addstr(&path, prefix);
+	last_slash = strrchr(arg, '/');
+	strbuf_addstr(&path, last_slash ? last_slash + 1 : arg);
+	e->path = strbuf_detach(&path, NULL);
+
+	e->next = NULL;
+
+	if (args->extra_files) {
+		struct extra_file *prev = args->extra_files;
+		while (prev->next)
+			prev = prev->next;
+		prev->next = e;
+	} else {
+		args->extra_files = e;
+	}
+
+	return 0;
+}
+
 static int parse_archive_args(int argc, const char **argv,
 		const struct archiver **ar, struct archiver_args *args)
 {
 	const char *format = "tar";
-	const char *base = NULL;
 	const char *remote = NULL;
 	const char *exec = NULL;
 	const char *output = NULL;
@@ -281,10 +340,13 @@ static int parse_archive_args(int argc, const char **argv,
 	struct option opts[] = {
 		OPT_GROUP(""),
 		OPT_STRING(0, "format", &format, "fmt", "archive format"),
-		OPT_STRING(0, "prefix", &base, "prefix",
+		OPT_STRING(0, "prefix", &args->base, "prefix",
 			"prepend prefix to each pathname in the archive"),
 		OPT_STRING(0, "output", &output, "file",
 			"write the archive to this file"),
+		{ OPTION_CALLBACK, 0, "add-file", args, "file",
+			"add file to the archive, using prefix as path",
+			PARSE_OPT_NONEG, extra_files_cb },
 		OPT_BOOLEAN(0, "worktree-attributes", &worktree_attributes,
 			"read .gitattributes in working directory"),
 		OPT__VERBOSE(&verbose),
@@ -318,8 +380,8 @@ static int parse_archive_args(int argc, const char **argv,
 	if (output)
 		die("Unexpected option --output");
 
-	if (!base)
-		base = "";
+	if (!args->base)
+		args->base = "";
 
 	if (list) {
 		for (i = 0; i < ARRAY_SIZE(archivers); i++)
@@ -344,8 +406,7 @@ static int parse_archive_args(int argc, const char **argv,
 		}
 	}
 	args->verbose = verbose;
-	args->base = base;
-	args->baselen = strlen(base);
+	args->baselen = strlen(args->base);
 	args->worktree_attributes = worktree_attributes;
 
 	return argc;
@@ -357,6 +418,7 @@ int write_archive(int argc, const char **argv, const char *prefix,
 	const struct archiver *ar = NULL;
 	struct archiver_args args;
 
+	memset(&args, 0, sizeof(struct archiver_args));
 	argc = parse_archive_args(argc, argv, &ar, &args);
 	if (setup_prefix && prefix == NULL)
 		prefix = setup_git_directory();
diff --git a/archive.h b/archive.h
index 038ac35..a88bdbb 100644
--- a/archive.h
+++ b/archive.h
@@ -1,6 +1,8 @@
 #ifndef ARCHIVE_H
 #define ARCHIVE_H
 
+struct extra_file;
+
 struct archiver_args {
 	const char *base;
 	size_t baselen;
@@ -11,6 +13,7 @@ struct archiver_args {
 	const char **pathspec;
 	unsigned int verbose : 1;
 	unsigned int worktree_attributes : 1;
+	struct extra_file *extra_files;
 	int compression_level;
 };
 

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

2009/7/30 René Scharfe <rene.scharfe@lsrfire.ath.cx>:
> demerphq schrieb:
>> So then git also would benefit from support in git-archive for adding
>> arbitrary files to the archive during generation?
>
> Yes, and this has come up before.
>
> How about the following?  It's missing documentation and a test case,
> but you could try
>
>        $ git archive --add-file extra HEAD >HEAD+extra.tar
>
> or
>
>        $ git archive --prefix=a/ --add-file extra --prefix=b/ HEAD >ba.tar
>
> Only the file name part (after the last slash) of the extra file is used,
> together with the prefix, to form the path of the archive entry.
>
> Opening the extra files when parsing the command line arguments and closing
> them after they have been written into the archive is a bit iffy, but it's
> impractical to report open errors after parts of the archive have already
> been created.

I havent managed to try it out yet but this is pretty much exactly
what I was looking for. Only thought I had was that it might be nice
to be able to specify what name the file should be added as so that
you can add files to subpaths deeper than the root/root prefix of the
tar. However for /my/ particular purposes that is unnecessary, it only
occurs to me as a nice to have. But if you were inclined to think
about extending it I was thinking an argument notation like --add-file
foo:bar/baz/bop might be useful.

Anyway, thanks a lot, assuming it works ok, which I'm sure it does,
even if it doesn't get accepted to git it is likely we will build a
git that does use it for the perl gitweb site. Much appreciated.

cheers,
Yves



-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

2009/7/31 Jakub Narebski <jnareb@gmail.com>:
> demerphq <demerphq@gmail.com> writes:
>> 2009/7/30 René Scharfe <rene.scharfe@lsrfire.ath.cx>:
>> > demerphq schrieb:
>
>>>> So then git also would benefit from support in git-archive for adding
>>>> arbitrary files to the archive during generation?
>>>
>>> Yes, and this has come up before.
>>>
>>> How about the following?  It's missing documentation and a test case,
>>> but you could try
>>>
>>>        $ git archive --add-file extra HEAD>HEAD+extra.tar
>>>
>>> or
>>>
>>>        $ git archive --prefix=a/ --add-file extra --prefix=b/ HEAD>ba.tar
>>
>> I havent managed to try it out yet but this is pretty much exactly
>> what I was looking for. Only thought I had was that it might be nice
>> to be able to specify what name the file should be added as so that
>> you can add files to subpaths deeper than the root/root prefix of the
>> tar. However for /my/ particular purposes that is unnecessary, it only
>> occurs to me as a nice to have. But if you were inclined to think
>> about extending it I was thinking an argument notation like --add-file
>> foo:bar/baz/bop might be useful.
>
> Why not use notation used by --graft-points option of mkisofs, i.e.
> "--add-file foo=bar/baz/bop", where 'foo' is name in archive
> (respective to prefix if any, I think), and 'bar/baz/bop' is name in
> filesystem.
>
> The proposed notation "--add-file foo:bar/baz/bop" looks for me (see
> extended sha-1 syntax in git-rev-parse manpage) like adding file
> 'bar/baz/bop' from 'foo' tree-ish.

Whatever is most appropriate. I was just demonstrating what I had in mind.

cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[René Scharfe]

demerphq schrieb:
> 2009/7/30 René Scharfe <rene.scharfe@lsrfire.ath.cx>:
>> demerphq schrieb:
>>> So then git also would benefit from support in git-archive for adding
>>> arbitrary files to the archive during generation?
>> Yes, and this has come up before.
>>
>> How about the following?  It's missing documentation and a test case,
>> but you could try
>>
>>        $ git archive --add-file extra HEAD >HEAD+extra.tar
>>
>> or
>>
>>        $ git archive --prefix=a/ --add-file extra --prefix=b/ HEAD >ba.tar
>>
>> Only the file name part (after the last slash) of the extra file is used,
>> together with the prefix, to form the path of the archive entry.
>>
>> Opening the extra files when parsing the command line arguments and closing
>> them after they have been written into the archive is a bit iffy, but it's
>> impractical to report open errors after parts of the archive have already
>> been created.
> 
> I havent managed to try it out yet but this is pretty much exactly
> what I was looking for. Only thought I had was that it might be nice
> to be able to specify what name the file should be added as so that
> you can add files to subpaths deeper than the root/root prefix of the
> tar. However for /my/ particular purposes that is unnecessary, it only
> occurs to me as a nice to have. But if you were inclined to think
> about extending it I was thinking an argument notation like --add-file
> foo:bar/baz/bop might be useful.

With the patch as-is, you can make a file named bop appear as
bar/baz/bop in the archive (by using a prefix of bar/baz/), but not as
foo.  I wonder how often one needs to rename a file while adding it to
the archive -- or indeed if this is needed at all.  Being lazy, I'd wait
for such a case to appear before supporting it with a double-valued
command line argument.

René

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[demerphq]

2009/8/2 René Scharfe <rene.scharfe@lsrfire.ath.cx>:
> demerphq schrieb:
>> 2009/7/30 René Scharfe <rene.scharfe@lsrfire.ath.cx>:
>>> demerphq schrieb:
>>>> So then git also would benefit from support in git-archive for adding
>>>> arbitrary files to the archive during generation?
>>> Yes, and this has come up before.
>>>
>>> How about the following?  It's missing documentation and a test case,
>>> but you could try
>>>
>>>        $ git archive --add-file extra HEAD >HEAD+extra.tar
>>>
>>> or
>>>
>>>        $ git archive --prefix=a/ --add-file extra --prefix=b/ HEAD >ba.tar
>>>
>>> Only the file name part (after the last slash) of the extra file is used,
>>> together with the prefix, to form the path of the archive entry.
>>>
>>> Opening the extra files when parsing the command line arguments and closing
>>> them after they have been written into the archive is a bit iffy, but it's
>>> impractical to report open errors after parts of the archive have already
>>> been created.
>>
>> I havent managed to try it out yet but this is pretty much exactly
>> what I was looking for. Only thought I had was that it might be nice
>> to be able to specify what name the file should be added as so that
>> you can add files to subpaths deeper than the root/root prefix of the
>> tar. However for /my/ particular purposes that is unnecessary, it only
>> occurs to me as a nice to have. But if you were inclined to think
>> about extending it I was thinking an argument notation like --add-file
>> foo:bar/baz/bop might be useful.
>
> With the patch as-is, you can make a file named bop appear as
> bar/baz/bop in the archive (by using a prefix of bar/baz/), but not as
> foo.  I wonder how often one needs to rename a file while adding it to
> the archive -- or indeed if this is needed at all.  Being lazy, I'd wait
> for such a case to appear before supporting it with a double-valued
> command line argument.

Well I was kinda thinking the case of a snapshot generator on
something like gitweb.

In that case you could have multiple processes trying to create the
same snapshot.

One solution is to do the work in a process specific temporary
directory with the correct name and then add it that way.

A slightly nicer approach is create the file in a common working
directory with a temporary name and then have it added to the archive
with a different name. This is actually the approach I used by
exploiting the the --transform option to tar. So while it might appear
my request is a YAGNI it is actually a IADNI (I Already Did Need It).

Even nicer and cleaner of course would be support something like
"--add-file -=.patch"  with the "-" meaning "read from STDIN", which
then means temporary files can be avoided, assuming one only needs to
add a single file to the archive. This is particularly nice if the
content of  the file can be generated from the output of something
like git-describe or git rev-parse or any generic tool.

So IMO actually this functionality would be very useful, and isnt so
weird when you think of it in context of generating snapshots for a
"git aware" build process.

Of course this is all very easy for me to say as a back-seat driver. :-)

cheers,
Yves




-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[René Scharfe]

demerphq schrieb:
> One solution is to do the work in a process specific temporary
> directory with the correct name and then add it that way.
> 
> A slightly nicer approach is create the file in a common working
> directory with a temporary name and then have it added to the archive
> with a different name. This is actually the approach I used by
> exploiting the the --transform option to tar. So while it might appear
> my request is a YAGNI it is actually a IADNI (I Already Did Need It).
> 
> Even nicer and cleaner of course would be support something like
> "--add-file -=.patch"  with the "-" meaning "read from STDIN", which
> then means temporary files can be avoided, assuming one only needs to
> add a single file to the archive. This is particularly nice if the
> content of  the file can be generated from the output of something
> like git-describe or git rev-parse or any generic tool.

This case (single file in archive, contents from stdin) could be handled
by another option, e.g. named --add-stdin <name_in_archive>, without
resorting to multi-valued option parameters.

Also, if a file can be generated using a --pretty=format: string, then
you can let git-archive expand a checked-in template by setting the
attribute export-subst.  This works for multiple files, but doesn't if
you need git-describe style output (yet).

> So IMO actually this functionality would be very useful, and isnt so
> weird when you think of it in context of generating snapshots for a
> "git aware" build process.

No doubt about it.  The question is: how to expose such functionality?
Perhaps all that is needed for your purposes is a format place-holder
for git-describe output?

(git itself needs more: in addition to the files "version" and
"git.spec", which could benefit from such a place-holder, it adds the
generated "configure" script and "git-gui/version" to the tarball.)

René

Re: Adding files to a git-archive when it is generated, and whats the best way to find out what branch a commit is on?

[Jeff Epler]

When I wanted to add extra files to a tar produced by git-archive, I
used python's tarfile module.  This assumes the tar will fit comfortably
in memory.  The specifics probably don't apply in your case, but maybe
it will be useful anyhow.

You could also use tar's -r or gnu tar's -A modes to add to the
git-archive tar once it's been created:
    git-archive ... > temp.tar
    tar -rf temp.tar additional-file
(unfortunately, -r and -A don't operate on pipes)

def main(args):
    if len(args) == 1:
        version = args[0]
    elif len(args) > 1:
        raise SystemExit, "Usage: %s [version]" % sys.argv[0]
    else:
        status, gitversion = commands.getstatusoutput("git-describe --tags")
        version = highest_version()
        if status != 0 or version != gitversion:
            raise SystemExit, """\
Highest version %r doesn't match description %r.
Specify version number explicitly if this is what you want""" % (
                    version, gitversion)

    version = version.lstrip("v")

    DIRNAME = "%(p)s-%(v)s" % {'p': PACKAGE_NAME, 'v': version}
    TARNAME = DIRNAME + '.tar.gz'

    verstream = StringIO.StringIO("%s\n" % version)
    verinfo = tarfile.TarInfo(DIRNAME + "/VERSION")
    verinfo.mode = 0660
    verinfo.size = len(verstream.getvalue())
    verinfo.mtime = time.time()

    tardata = os.popen("git-archive --prefix=%(p)s/ v%(v)s"
                            % {'p': DIRNAME, 'v': version}).read()
    tarstream = StringIO.StringIO(tardata)

    tar = tarfile.TarFile(mode="a", fileobj=tarstream)
    tar.addfile(verinfo, verstream)
    tar.close()

    out = gzip.open("../" + TARNAME, "wb")
    out.write(tarstream.getvalue())
    out.close()