`git check-attr' problems & questions

`git check-attr' problems & questions

[Eli Barzilay]

I'm trying to use .gitattributes to use some custom properties that we
have in an svn repository which is migrating to git, and there are
some two big problems with it that I'm running into:

1. Looks like the path should always be the full path from the root,
   even if I'm currently in a subdirectory, so:

     cd $repo/foo/bar
     git check-attr foo -- some-file # doesn't work
     git check-attr foo -- foo/bar/some-file # works

   Worse, if I happen to have an attribute for a toplevel `foo' file,
   the first command produces that attribute.

2. I'm also trying to use an attribute on a hook script on the server
   (with the idea that email notifications are sent according to the
   set of modified files) -- but it looks like `git check-attr'
   doesn't work on a bare repository.  There's a "worse" part here
   too: instead of showing an error, it actually succeeds after
   showing that the attribute is "unspecified".

So my first question is -- am I missing something obvious?  It almost
looks like `check-attr' is intended only for low-level use...

Fixing the first problem seems like it would be easy with a quick
shell script (maybe even with just an alias), but the second one is
more troubling.  Is there any way to do this on a bare repository
without checking it out?

And the last question -- is there anything that does something like
this customized email notifications that I'm trying to do?  (That is:
an attribute that determines recipients, then a `post-receive' hook
that builds a list of modified files, grabs all the corresponding
values of this attributes, and that's makes up the recipient list to
notify.)

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: `git check-attr' problems & questions

[Jonathan Nieder]

Hi,

[quoted text reordered for convenience]

Eli Barzilay wrote:

> I'm trying to use .gitattributes to use some custom properties that we
> have in an svn repository which is migrating to git,
[...]
> So my first question is -- am I missing something obvious?  It almost
> looks like `check-attr' is intended only for low-level use...

Yep.  It’s listed under “purehelpers” (Internal helper commands) in
command-list.txt and hence git(1).  I suspect a good place to put
a user-visible equivalent would be a future ‘git ls’ command [1].

Therefore...

> Fixing the first problem seems like it would be easy with a quick
> shell script (maybe even with just an alias),

If it proves pleasant to use, it would be very nice if you could send
a copy of such a shell script after a while.  Experiments like this
are the foundation of a good UI.

> 2. I'm also trying to use an attribute on a hook script on the server
>    (with the idea that email notifications are sent according to the
>    set of modified files) -- but it looks like `git check-attr'
>    doesn't work on a bare repository.

Current ‘git check-attr’ reads its attributes from actual .gitattribute
files in the work tree first and then examines the versions registered
in the index.  So the secret is to read the .gitattribute files you care
about into a temporary index file --- e.g., something like this.

 : "Usage: $0 commit check-attr-args" &&

 GIT_INDEX=tmp-index git read-tree --reset -i "$1" &&
 shift &&
 GIT_INDEX=tmp-index git check-attr "$@" &&
 rm tmp-index

Ideally, check-attr should learn an option to set a different policy
(e.g., index-only) with git_attr_set_direction(), but that is a
different story.

>    instead of showing an error, it actually succeeds after
>    showing that the attribute is "unspecified".

Probably it should learn to error out when it expects to use a work
tree but there is none.  git_attr_set_direction() does this.

> And the last question -- is there anything that does something like
> this customized email notifications that I'm trying to do?  (That is:
> an attribute that determines recipients, then a `post-receive' hook
> that builds a list of modified files, grabs all the corresponding
> values of this attributes, and that's makes up the recipient list to
> notify.)

Not that I know of.

HTH,
Jonathan

[1] I read the name from here:
http://thread.gmane.org/gmane.comp.version-control.git/136360
That series went in a different direction, though.
http://thread.gmane.org/gmane.comp.version-control.git/141678

Re: `git check-attr' problems & questions

[Eli Barzilay]

On Mar 27, Jonathan Nieder wrote:
> Hi,
> 
> [quoted text reordered for convenience]
> 
> Eli Barzilay wrote:
> 
> > I'm trying to use .gitattributes to use some custom properties that we
> > have in an svn repository which is migrating to git,
> [...]
> > So my first question is -- am I missing something obvious?  It almost
> > looks like `check-attr' is intended only for low-level use...
> 
> Yep.  It’s listed under “purehelpers” (Internal helper commands) in
> command-list.txt and hence git(1).

(Ah, it would be nice to have that in the `git-check-attr' man page.)


> I suspect a good place to put a user-visible equivalent would be a
> future ‘git ls’ command [1].
> 
> Therefore...
> 
> > Fixing the first problem seems like it would be easy with a quick
> > shell script (maybe even with just an alias),
> 
> If it proves pleasant to use, it would be very nice if you could
> send a copy of such a shell script after a while.  Experiments like
> this are the foundation of a good UI.

A script is easy, but even if it's for internal use, I don't see the
point in `check-attr' not doing so...

> > 2. I'm also trying to use an attribute on a hook script on the server
> >    (with the idea that email notifications are sent according to the
> >    set of modified files) -- but it looks like `git check-attr'
> >    doesn't work on a bare repository.
> 
> Current ‘git check-attr’ reads its attributes from actual .gitattribute
> files in the work tree first and then examines the versions registered
> in the index.  So the secret is to read the .gitattribute files you care
> about into a temporary index file --- e.g., something like this.
> 
>  : "Usage: $0 commit check-attr-args" &&
> 
>  GIT_INDEX=tmp-index git read-tree --reset -i "$1" &&
>  shift &&
>  GIT_INDEX=tmp-index git check-attr "$@" &&
>  rm tmp-index

I tried that, but it doesn't work.  (I used GIT_INDEX_FILE.)

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: `git check-attr' problems & questions

[Eli Barzilay]

On Mar 29, Eli Barzilay wrote:
> On Mar 27, Jonathan Nieder wrote:
> >  : "Usage: $0 commit check-attr-args" &&
> > 
> >  GIT_INDEX=tmp-index git read-tree --reset -i "$1" &&
> >  shift &&
> >  GIT_INDEX=tmp-index git check-attr "$@" &&
> >  rm tmp-index
> 
> I tried that, but it doesn't work.  (I used GIT_INDEX_FILE.)

In case I'm doing something wrong, here's a script that shows the
problem that I have:

  #!/bin/sh
  rm -rf /tmp/test
  mkdir /tmp/test
  cd /tmp/test
  mkdir t
  cd t
  git init > /dev/null
  echo "Blah" > foo
  echo "foo x=y" > .gitattributes
  git add foo .gitattributes
  git commit -m "stuff" > /dev/null
  echo -n "check-attr in a working directory: "
  git check-attr x foo
  cd ..
  git clone --bare t > /dev/null
  cd t.git
  git read-tree --reset -i master
  echo -n "check-attr in a bare repository:   "
  git check-attr x foo

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

git check-attr in bare repositories

[Jonathan Nieder]

Eli Barzilay wrote:
> On Mar 29, Eli Barzilay wrote:
>> On Mar 27, Jonathan Nieder wrote:
[in a bare repository]

>>>  GIT_INDEX=tmp-index git check-attr "$@" &&
[...]
>> I tried that, but it doesn't work.  (I used GIT_INDEX_FILE.)

Yes, not sure how I confused myself.

git explicitly guards against that in attr.c.

	/*
	 * Read from parent directories and push them down
	 */
	if (!is_bare_repository() || direction == GIT_ATTR_INDEX) {

That check comes from v1.5.6-rc3~9^2 (Ignore .gitattributes in bare
repositories, 2008-06-08).  This is consistent with how bare
repositories generally work: they are guarded against use with a
populated index, since what filesystem tree would that index track?

To support your use case, it would be nice for check-attr to learn a
--direction option.  Maybe it would be safe to let check-attr read
from the index in bare repositories by default anyway, since the
index is usually missing anyway.

Properly supporting general work in a bare repository would require
a larger effort.  Maybe:

 - Teach code that checks is_bare_repository() to check
   get_git_work_tree() == NULL or similar instead.

 - Make the work_tree and git_dir variables the responsibility of the
   setup code.

 - If the user sets GIT_INDEX_FILE or GIT_WORK_TREE, let the setup
   code respect that wish even if core.bare is set.

Thoughts?
Jonathan

Re: git check-attr in bare repositories

[Eli Barzilay]

On Mar 29, Jonathan Nieder wrote:
> Eli Barzilay wrote:
> > On Mar 29, Eli Barzilay wrote:
> >> On Mar 27, Jonathan Nieder wrote:
> [in a bare repository]
> 
> >>>  GIT_INDEX=tmp-index git check-attr "$@" &&
> [...]
> >> I tried that, but it doesn't work.  (I used GIT_INDEX_FILE.)
> 
> Yes, not sure how I confused myself.
> 
> git explicitly guards against that in attr.c.
> 
> 	/*
> 	 * Read from parent directories and push them down
> 	 */
> 	if (!is_bare_repository() || direction == GIT_ATTR_INDEX) {

Ugh...!


> That check comes from v1.5.6-rc3~9^2 (Ignore .gitattributes in bare
> repositories, 2008-06-08).  This is consistent with how bare
> repositories generally work: they are guarded against use with a
> populated index, since what filesystem tree would that index track?

Well, I've seen more than a few places that say that git attributes
are the right choice to use for svn properties...  (I think that I
also asked a question about this here.)

There are a number of technicalities that make git attributes a poor
substitute for svn properties -- mainly the fact that they limited in
contents (basically just short strings, no quoting, no newlines), and
the fact that they're not tracked with the files (as in svn, where
moving a file somewhere will move its properties with it).  Both of
these are not too bad for what we use them (files don't move often,
and we use short strings only).  But having no access to them on the
server (without a costly checkout) is making it even more problematic,
to the point that coming up with our own home cooked thing is easier
than using git attributes.  Given that they're not intended for
end-user consumption (eg, `check-attr' being an internal helper) it
looks like we're not giving up much anyway.


> To support your use case, it would be nice for check-attr to learn a
> --direction option.  Maybe it would be safe to let check-attr read
> from the index in bare repositories by default anyway, since the
> index is usually missing anyway.

Well, using the index this way seems like a kind of a hack anyway, so
I'm not sure that there is any reason to do this.  If anything, I'd
like it if `check-attr' could just use the repository directly instead
of the index (or a work tree) in a bare repository.  Without that, and
the best solution that I see in the short term, it should throw an
appropriate error instead of just producing an "unspecified".

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: git check-attr in bare repositories

[Jonathan Nieder]

Eli Barzilay wrote:

> Well, using the index this way seems like a kind of a hack anyway, so
> I'm not sure that there is any reason to do this.

Most git commands do write out the tree they are working with to an
(in-memory or on-disk) index, so using the index this way would make a
warped kind of sense.  But I agree that it is ugly.

> If anything, I'd
> like it if `check-attr' could just use the repository directly instead
> of the index (or a work tree) in a bare repository.

I think the right thing to do is to put this functionality in a new
‘git ls’ command.  Maybe something like this:

 $ git ls --format='%p %a(crlf)' master -- '*.txt'
 some/path/foo.txt crlf:input
 some/path/bar.txt crlf
 some/path/other.txt !crlf
 yet/another/path.txt 
 $

I can not promise I will find time before the weekend to work on it.
I wouldn’t be unhappy if someone else gets to it first.

Thanks for the explanations.
Jonathan

Re: git check-attr in bare repositories

[Jakub Narebski]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Eli Barzilay wrote:
> 
> > Well, using the index this way seems like a kind of a hack anyway, so
> > I'm not sure that there is any reason to do this.
> 
> Most git commands do write out the tree they are working with to an
> (in-memory or on-disk) index, so using the index this way would make a
> warped kind of sense.  But I agree that it is ugly.
> 
> > If anything, I'd
> > like it if `check-attr' could just use the repository directly instead
> > of the index (or a work tree) in a bare repository.
> 
> I think the right thing to do is to put this functionality in a new
> ‘git ls’ command.  Maybe something like this:
> 
>  $ git ls --format='%p %a(crlf)' master -- '*.txt'
>  some/path/foo.txt crlf:input
>  some/path/bar.txt crlf
>  some/path/other.txt !crlf
>  yet/another/path.txt 
>  $

Well, that or make `git check-attr` support reading .gitattributes
from repository (from a corresponding tree object).

Unfortunately `git check-attr` doesn't have place to put revision...
well unless as a parameter:

  git check-attr [--cached|--tree <tree-ish>] <attr>... [--] <pathname>...

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: git check-attr in bare repositories

[Eli Barzilay]

On Mar 30, Jonathan Nieder wrote:
> Eli Barzilay wrote:
> > If anything, I'd like it if `check-attr' could just use the
> > repository directly instead of the index (or a work tree) in a
> > bare repository.
> 
> I think the right thing to do is to put this functionality in a new
> ‘git ls’ command.  Maybe something like this:
> 
>  $ git ls --format='%p %a(crlf)' master -- '*.txt'
>  some/path/foo.txt crlf:input
>  some/path/bar.txt crlf
>  some/path/other.txt !crlf
>  yet/another/path.txt 

This looks useful, but it has a problem of being lower level than
`check-attr' in the sense that you need to parse this output.


On Mar 30, Jakub Narebski wrote:
> 
> Well, that or make `git check-attr` support reading .gitattributes
> from repository (from a corresponding tree object).
> 
> Unfortunately `git check-attr` doesn't have place to put revision...
> well unless as a parameter:
> 
>   git check-attr [--cached|--tree <tree-ish>] <attr>... [--] <pathname>...

This looks like a better way to have what I need from it...  It also
looks useful for other uses -- for example, an attribute that is used
by a hook to specify that some file should not be deleted (so the hook
needs to look at the value before the commit).

(But in any case, I'd be happy if there was a way to make it read the
contents of a (temporary) index.)

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Carrying over attributes when moving files

[Jonathan Nieder]

Eli Barzilay wrote:

> There are a number of technicalities that make git attributes a poor
> substitute for svn properties -- mainly
[...]
> and
> the fact that they're not tracked with the files (as in svn, where
> moving a file somewhere will move its properties with it).

It would be great to improve this.

Consider the following directory hierarchy.

	old-files/
		.gitattributes
		some-file.html
		other-file.html
	new-files/
		unrelated-file.html

.gitattributes specifies that old HTML files use a CRLF line ending.

	*.html crlf

What would you expect the following commands to do?

	git mv old-files/some-file.html new-files/
	git commit

How about these?

	mv old-files/some-file.html new-files/
	git add new-files/some-file.html
	git commit -a

I don’t think there’s any fundamental reason this hasn’t been taken
care of; it’s just that nobody has done it yet.

Thoughts welcome.
Jonathan

Re: Carrying over attributes when moving files

[Eli Barzilay]

On Mar 30, Jonathan Nieder wrote:
> Eli Barzilay wrote:
> 
> > There are a number of technicalities that make git attributes a
> > poor substitute for svn properties -- mainly
> [...]
> > and the fact that they're not tracked with the files (as in svn,
> > where moving a file somewhere will move its properties with it).
> 
> It would be great to improve this.
> 
> Consider the following directory hierarchy.
> 
> 	old-files/
> 		.gitattributes
> 		some-file.html
> 		other-file.html
> 	new-files/
> 		unrelated-file.html
> 
> .gitattributes specifies that old HTML files use a CRLF line ending.
> 
> 	*.html crlf
> 
> What would you expect the following commands to do?
> 
> 	git mv old-files/some-file.html new-files/
> 	git commit
> 
> How about these?
> 
> 	mv old-files/some-file.html new-files/
> 	git add new-files/some-file.html
> 	git commit -a
> 
> I don’t think there’s any fundamental reason this hasn’t been taken
> care of; it’s just that nobody has done it yet.

There's of course a whole bunch of similar issues, like having

  a/.gitattribute specifies: b/c foo=bar
  a/b/.gitattribute specifies: c foo=bar

or even worse -- when the two values are different.

In any case, the way that attributes are specified (which is flexible
and convenient, fwiw) is part of what makes them a bad substitution
for subversion properties...  Making attributes tracked with their
files would mean some change to how this specification is done.  For
example, it could be that attributes that are in (tracked)
.gitattribute files nad that have no "/" and no glob characters in the
file names are the only ones that get tracked.  In short -- only
simple names of immediate files and/or directories.  Having such a
convention would also allow some equivalent of `svn propset' since
there's a canonical place to add them.

And this could be taken further -- for example, a command that will
crawl a tree and find "non-canonicalized" attributes, or will turn
them into such.  Another example would be to take something like
"*.html crlf" as meaning that new *.html files should get the crlf
attribute, etc.  But thinking about the kind of work that all of this
would imply, it seems to me that doing the svn thing and making the
attributes part of the content (ie, make them a part of the blob
objects somehow) makes more sense.  But this is a big change too...

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: Carrying over attributes when moving files

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Consider the following directory hierarchy.
>
> 	old-files/
> 		.gitattributes
> 		some-file.html
> 		other-file.html
> 	new-files/
> 		unrelated-file.html
>
> .gitattributes specifies that old HTML files use a CRLF line ending.
>
> 	*.html crlf
>
> What would you expect the following commands to do?
>
> 	git mv old-files/some-file.html new-files/
> 	git commit

I know what you are getting at, but I think it cuts both ways.

I can well imagine that the build structure where "make && make install"
will go to old-files and new-files directories and install the contents
overlayed in the same destination, and .gitattributes may be being used as
a way to "fix" some aspect of "oldness" in old-files/ files until the
sources are fixed.  So an equally valid work sequence might be:

 	git mv old-files/some-file.html new-files/
	edit new-files/some-file.html
        git add new-files/some-file.html
 	git commit

in which case adding a new "some-file.html crlf" entry to .gitattributes
in new-files/ directory would be just _wrong_.

So I am not opposed to a mechanism to _let_ users makes a copy of an entry
that covers (or used to cover) one path to apply to another path that was
not covered by it, but doing it unconditionally would not be such a good
idea.