Passing commit IDs to git-archive

Passing commit IDs to git-archive

[Stephen Bash]

Hi all-

For an upcoming release, I'm attempting to update our build scripts to suck down some content from a remote Git repository and include it en masse in our installer.  My first inclination was to use submodules, but similar to this question on Stack Overflow

  http://stackoverflow.com/questions/6553743/

the content used to be in our development repository and has recently been migrated out, so jumping between branches/back and forth in history is causing problems (unlike the question, I'm not willing to rewrite history to make the submodule approach work).  So after googling around I came to git-archive using the --remote argument, which actually seems nice for my situation because the content is basically support material for the end user that the devs don't manage/edit.

Unfortunately I just attempted

  warp:bar bash$ git archive --remote=file:///Users/bash/Development/foo \
                             --output=test.tgz 3b9e49b \
                             path/to/subdir
  remote: fatal: no such ref: 3b9e49b
  remote: git upload-archive: archiver died with error
  fatal: sent error to the client: git upload-archive: archiver died with error

  warp:bar bash$ git archive --remote=file:///Users/bash/Development/foo \
                             --output=test.tgz 3b9e49b:path/to/subdir
  remote: fatal: no such ref: 3b9e49b
  remote: git upload-archive: archiver died with error
  fatal: sent error to the client: git upload-archive: archiver died with error

  warp:bar bash$ cd /Users/bash/Development/foo
  warp:foo bash$ git rev-parse --short master
  3b9e49b
  warp:foo bash$ 
  warp:foo bash$ git --version
  git version 1.7.9.2
  warp:foo bash$ 

on Mac OS 10.6.8 (obviously this is local testing, the goal is to use ssh remotely).  After parsing the error "no such ref" I attempted the same operation using master as the tree-ish and archive worked as expected (either specifying the path separately or using the colon syntax to reference the tree directly).  Is there a reason git-archive requires a named ref rather than just a commit (or tree) ID?  If not, would it be difficult to patch git-upload-archive to use the IDs?  I could use tags for the ref, but in my case would result in almost every commit being a tag which seems wasteful.

Thoughts?  Thanks in advance!

Stephen

Re: Passing commit IDs to git-archive

[Junio C Hamano]

Stephen Bash <bash@genarts.com> writes:

> Unfortunately I just attempted
>
>   warp:bar bash$ git archive --remote=file:///Users/bash/Development/foo \
>                              --output=test.tgz 3b9e49b \
>                              path/to/subdir
>
> Thoughts?  Thanks in advance!

If you are always fetching from that same location, perhaps doing
this only once

	git clone file:///...

and then every time you want to use the test.tgz, do something like

	git fetch
        git archive --output=test.tgz 3b9e49b path/to/subdir

or something like that?

Re: Passing commit IDs to git-archive

[René Scharfe]

Am 09.03.2012 23:14, schrieb Stephen Bash:
> Unfortunately I just attempted
>
>    warp:bar bash$ git archive --remote=file:///Users/bash/Development/foo \
>                               --output=test.tgz 3b9e49b \
>                               path/to/subdir
>    remote: fatal: no such ref: 3b9e49b
>    remote: git upload-archive: archiver died with error
>    fatal: sent error to the client: git upload-archive: archiver died with error

How about something like this instead?

	$ (
		cd /Users/bash/Development/foo &&
		git archive --format=tgz 3b9e49b path/to/subdir
	) >test.tgz

I.e. instead of using the --remote option, going to the target and 
creating the archive locally.  You could also keep using the --output 
parameter if you specify the target file using an absolute path, of course.

René

Re: Passing commit IDs to git-archive

[Jeff King]

On Fri, Mar 09, 2012 at 05:14:05PM -0500, Stephen Bash wrote:

> on Mac OS 10.6.8 (obviously this is local testing, the goal is to use
> ssh remotely).  After parsing the error "no such ref" I attempted the
> same operation using master as the tree-ish and archive worked as
> expected (either specifying the path separately or using the colon
> syntax to reference the tree directly).  Is there a reason git-archive
> requires a named ref rather than just a commit (or tree) ID?

Yes; generally git repositories do not allow clients to access arbitrary
sha1s. Instead, they require that the requested objects be accessible by
a ref.

git-archive was not properly enforcing this, and was changed recently to
allow only refs by name, as well as sub-trees of refs (e.g.,
HEAD:subdir/). That means we do disallow an arbitrary commit or tree
sha1, even if it is reachable from the advertised refs.

> would it be difficult to patch git-upload-archive to use the IDs?  I
> could use tags for the ref, but in my case would result in almost
> every commit being a tag which seems wasteful.

Doing it right is a bit expensive, because in the general case (somebody
requested a tree sha1), we would need to traverse every tree of every
commit to see if it is reachable.

We could potentially implement a more restricted set of rules, allowing
"<commit>:<subdir>" and checking that <commit> is reachable.  That would
disallow an arbitrary tree sha1, but I suspect it would cover the common
use case (i.e., you want to get the tree, or even a subtree, of a
particular revision).

-Peff

Re: Passing commit IDs to git-archive

[Stephen Bash]

----- Original Message -----
> From: "Jeff King" <peff@peff.net>
> Sent: Monday, March 12, 2012 7:47:42 AM
> Subject: Re: Passing commit IDs to git-archive
> 
> On Fri, Mar 09, 2012 at 05:14:05PM -0500, Stephen Bash wrote:
> 
> > on Mac OS 10.6.8 (obviously this is local testing, the goal is to
> > use ssh remotely).  After parsing the error "no such ref" I
> > attempted the same operation using master as the tree-ish and
> > archive worked as expected (either specifying the path separately or
> > using the colon syntax to reference the tree directly).  Is there a
> > reason git-archive requires a named ref rather than just a commit
> > (or tree) ID?
> 
> Yes; generally git repositories do not allow clients to access
> arbitrary sha1s. Instead, they require that the requested objects be
> accessible by a ref.

Jeff: thanks for the concise summary.  I thought that might be the case, but wanted to confirm.

Junio and René: thanks for the input and ideas. Seems I'll be investigating my options today.

Thanks,
Stephen