Passing tar(1) options via git-archive(1)

Passing tar(1) options via git-archive(1)

[Daniel Brockman]

Some background from the git-archive(1) man page:

    git-archive behaves differently when given a tree ID versus when
    given a commit ID or tag ID.  In the first case the current time is
    used as the modification time of each file in the archive.  In the
    latter case the commit time as recorded in the referenced commit
    object is used instead.

Would it make sense to add an --mtime option to git-archive(1) to enable
explicitly setting the mtime for all files in the archive?  It could
just pass through to the tar(1) --mtime option.

My use case is `git archive HEAD | docker build -`, in which the Docker
cache is prevented from working because the mtime keeps getting bumped
on all files.  I would like to have the mtime always be the same.

See, e.g., <https://github.com/deis/deis/issues/1334>.

Otherwise, how about a generic way to pass options to tar(1)?

Thanks,
Daniel

Re: Passing tar(1) options via git-archive(1)

[Thomas Braun]

Am 23.09.2014 um 20:57 schrieb Daniel Brockman:
> Some background from the git-archive(1) man page:
> 
>     git-archive behaves differently when given a tree ID versus when
>     given a commit ID or tag ID.  In the first case the current time is
>     used as the modification time of each file in the archive.  In the
>     latter case the commit time as recorded in the referenced commit
>     object is used instead.
> 
> Would it make sense to add an --mtime option to git-archive(1) to enable
> explicitly setting the mtime for all files in the archive?  It could
> just pass through to the tar(1) --mtime option.
> 
> My use case is `git archive HEAD | docker build -`, in which the Docker
> cache is prevented from working because the mtime keeps getting bumped
> on all files.  I would like to have the mtime always be the same.
> 
> See, e.g., <https://github.com/deis/deis/issues/1334>.
> 
> Otherwise, how about a generic way to pass options to tar(1)?

Actually I wanted to just hint to TAR_OPTIONS
as in
TAR_OPTIONS="--mtime 2014-09-23\ 00:00" git archive -o ausg.tar HEAD
but that does not work here (on windows).

The questions is should it be supported?

Re: Passing tar(1) options via git-archive(1)

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]

On Tue, Sep 23, 2014 at 11:49:24PM +0200, Thomas Braun wrote:
> Am 23.09.2014 um 20:57 schrieb Daniel Brockman:
> > Would it make sense to add an --mtime option to git-archive(1) to enable
> > explicitly setting the mtime for all files in the archive?  It could
> > just pass through to the tar(1) --mtime option.
> > 
> > My use case is `git archive HEAD | docker build -`, in which the Docker
> > cache is prevented from working because the mtime keeps getting bumped
> > on all files.  I would like to have the mtime always be the same.
> > 
> > See, e.g., <https://github.com/deis/deis/issues/1334>.
> > 
> > Otherwise, how about a generic way to pass options to tar(1)?
> 
> Actually I wanted to just hint to TAR_OPTIONS
> as in
> TAR_OPTIONS="--mtime 2014-09-23\ 00:00" git archive -o ausg.tar HEAD
> but that does not work here (on windows).

Git does not invoke tar(1).  It has its own tar (actually, pax)
implementation, so any options would have to be implemented in Git.
We'd probably want to make such a change effective in the zip format as
well.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Passing tar(1) options via git-archive(1)

[Daniel Brockman]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> Git does not invoke tar(1).  It has its own tar (actually, pax)
> implementation, so any options would have to be implemented in Git.
> We'd probably want to make such a change effective in the zip format
> as well.

Ah, I see...

Well, I guess in the meantime I'll just do this:

   git commit -m dummy --allow-empty --date=1970-01-01T00:00:00Z
   git archive HEAD | docker build -
   git reset HEAD~

Or even (slightly more atomically) this:

   git archive HEAD | { tmpdir=`mktemp -d` && tar x -C"$tmpdir" &&
     tar c -C"$tmpdir" --mtime=1970-01-01T00:00:00Z . &&
     rm -r "$tmpdir"; } | docker build -

Re: Passing tar(1) options via git-archive(1)

[Jeff King]

On Wed, Sep 24, 2014 at 03:26:22AM +0200, Daniel Brockman wrote:

> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
> 
> > Git does not invoke tar(1).  It has its own tar (actually, pax)
> > implementation, so any options would have to be implemented in Git.
> > We'd probably want to make such a change effective in the zip format
> > as well.
> 
> Ah, I see...
> 
> Well, I guess in the meantime I'll just do this:
> 
>    git commit -m dummy --allow-empty --date=1970-01-01T00:00:00Z
>    git archive HEAD | docker build -
>    git reset HEAD~

I don't think that will work. The `--date` parameter sets the author
date, but archive uses the committer date. You'd have to set
GIT_COMMITTER_DATE in the environment to override that.

Also, you can avoid writing to the HEAD ref entirely by using the
commit-tree plumbing. Like:

  commit=$(
    GIT_COMMITTER_DATE=1970-01-01T00:00:00Z \
    git commit-tree HEAD^{tree} </dev/null
  )
  git archive $commit | ...

However, the --mtime patch you are asking for is really not that big:

diff --git a/archive.c b/archive.c
index 952a659..9396fca 100644
--- a/archive.c
+++ b/archive.c
@@ -299,7 +299,8 @@ static void parse_treeish_arg(const char **argv,
 	ar_args->tree = tree;
 	ar_args->commit_sha1 = commit_sha1;
 	ar_args->commit = commit;
-	ar_args->time = archive_time;
+	if (!ar_args->time)
+		ar_args->time = archive_time;
 }
 
 #define OPT__COMPR(s, v, h, p) \
@@ -323,6 +324,7 @@ static int parse_archive_args(int argc, const char **argv,
 	int i;
 	int list = 0;
 	int worktree_attributes = 0;
+	unsigned long mtime = 0;
 	struct option opts[] = {
 		OPT_GROUP(""),
 		OPT_STRING(0, "format", &format, N_("fmt"), N_("archive format")),
@@ -332,6 +334,7 @@ static int parse_archive_args(int argc, const char **argv,
 			N_("write the archive to this file")),
 		OPT_BOOL(0, "worktree-attributes", &worktree_attributes,
 			N_("read .gitattributes in working directory")),
+		OPT_DATE(0, "mtime", &mtime, N_("mtime of files in archive")),
 		OPT__VERBOSE(&verbose, N_("report archived files on stderr")),
 		OPT__COMPR('0', &compression_level, N_("store only"), 0),
 		OPT__COMPR('1', &compression_level, N_("compress faster"), 1),
@@ -398,6 +401,7 @@ static int parse_archive_args(int argc, const char **argv,
 	args->base = base;
 	args->baselen = strlen(base);
 	args->worktree_attributes = worktree_attributes;
+	args->time = mtime;
 
 	return argc;
 }

which allows:

  git archive --mtime='yesterday at 3pm' HEAD

For inclusion in git, it would need someone to wrap it up with a commit
message, and add a basic test (see the existing mtime test in
t/t5000-tar-tree) and documentation in Documentation/git-archive.txt.
That someone could be you. :)

-Peff