[PATCH] Do not ignore hidden refs

[PATCH] Do not ignore hidden refs

[Petr Baudis]

Some of the ref manipulation tools (git-for-each-ref and git-show-ref in
particular) would not handle hidden (~ /^\./) refs. This may be an
acceptable or possibly even desirable behaviour for the ref walkers and
repackers, but git-show-ref hiddenrefname must work.

This makes Git not ignore hidden refs at all. I'm not opposed to making
some particular parts of the ref interface to continue to ignore hidden
refs, but the restriction cannot be so deep.

Signed-off-by: Petr Baudis <pasky@suse.cz>
---

 refs.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/refs.c b/refs.c
index f003a0b..06fa5c2 100644
--- a/refs.c
+++ b/refs.c
@@ -141,7 +141,8 @@ static struct ref_list *get_ref_dir(cons
 			int flag;
 			int namelen;
 
-			if (de->d_name[0] == '.')
+			if (de->d_name[0] == '.' && (de->d_name[1] == '\0'
+			    || (de->d_name[1] == '.' && de->d_name[2] == '\0')))
 				continue;
 			namelen = strlen(de->d_name);

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> Some of the ref manipulation tools (git-for-each-ref and git-show-ref in
> particular) would not handle hidden (~ /^\./) refs.

refs.c::check_ref_format() seems to suggest that any ref whose
path component begins with a dot is invalid (since October last
year), so I am a bit surprised you are bringing this up now.  Do
you know of specific examples where this is not enforced?  It
could even be argued that the places in the system that allow
such a ref are buggy.

I do not recall why we decided that this particular restriction
was needed (I do understand the other three restrictions --- see
commit log of 03feddd6), although I do think this is not a
gratuitous change but comes from a list discussion.  I suspect
it was to avoid confusion with ".." operator ("..." was added
much later in July this year), but it is not consistent that we
do not forbid the ones that end with a dot.

Maybe we should have found this inconsistency and added "names
that end with a dot" to the forbidden category when we
introduced the symmetric difference, but apparently nobody
noticed.  Right now foo...bar is seriously ambiguous.  Even
though it cannot be "foo" .. ".bar" (because ".bar" is not a
valid refname), it can be "foo." .. "bar" or "foo" ... "bar",
and worse yet spelling longhand form "refs/heads/foo." does not
help to disambiguate this.

Re: [PATCH] Do not ignore hidden refs

[Petr Baudis]

On Sat, Nov 18, 2006 at 05:39:34AM CET, Junio C Hamano wrote:
> Petr Baudis <pasky@suse.cz> writes:
> 
> > Some of the ref manipulation tools (git-for-each-ref and git-show-ref in
> > particular) would not handle hidden (~ /^\./) refs.
> 
> refs.c::check_ref_format() seems to suggest that any ref whose
> path component begins with a dot is invalid (since October last
> year), so I am a bit surprised you are bringing this up now.

Oops, I must've forgotten that already.

> Do you know of specific examples where this is not enforced?  It
> could even be argued that the places in the system that allow
> such a ref are buggy.

Cogito creates such refs for internal purposes in two scenarios, on the
other hand it could be argued that in one of those cases the file has no
business in refs/ at all (temporary fetching refs, but they may be
actually symrefs) and in the other case it has no business in refs/heads/
at all (pointers to shelved changes in a branch).

However, I in fact *did* intend to make leading-dot refnames a public
interface. The thing is, I need a way to mark some tags as private to
your repository if Cogito is to support autopushing tags, and I still
think the most elegant way to do that is to make them like hidden files.
Alternative suggestions welcome.

I don't *need* but it might be nice to have also private heads, for
possible setups when you declare heads namespace of two repositories
matching 1:1 but would like to temporarily make a short-lived head in
one of them or so.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
The meaning of Stonehenge in Traflamadorian, when viewed from above, is:
"Replacement part being rushed with all possible speed."

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> I don't *need* but it might be nice to have also private heads, for
> possible setups when you declare heads namespace of two repositories
> matching 1:1 but would like to temporarily make a short-lived head in
> one of them or so.

I agree that different classes of heads and tags (not just
"public" vs "private") may come in handy in different workflows.

I think, however, if we (collectively as all the Porcelain
writers although I am not really one of them) are to support it,
they should not make distinction to the core, and it should be
handled with the agreed-upon convention.  In other words, if
".bar" were a valid refname to the core then I would agree that
not packing them had definitely been a bug.

And we already have one such convention of using refs/remotes
for tracking branches, which is promoted to become the default
(thus an official BCP).

Personally I established a convention to treat heads/??/* as
"private namespace" while using heads/* as public refs for my
own work (I do that for git.git as people know, and I do that
for my day job project as well).  I do not think it is a great
enough convention to be promoted as the official BCP, but it has
been good enough for me, especially commands like "show-branch"
and "tag -l" understands the shell-style filegrobs (e.g
"show-branch master heads/??/*" would show what's yet to be
polished and merged).

Re: [PATCH] Do not ignore hidden refs

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Personally I established a convention to treat heads/??/* as
> "private namespace" while using heads/* as public refs for my
> own work (I do that for git.git as people know, and I do that
> for my day job project as well).  I do not think it is a great
> enough convention to be promoted as the official BCP, but it has
> been good enough for me, especially commands like "show-branch"
> and "tag -l" understands the shell-style filegrobs (e.g
> "show-branch master heads/??/*" would show what's yet to be
> polished and merged).

We use refs/heads/??/* as "developer specific" namespaces.

That is we use an update hook in our central repository to control
who can push into refs/heads/??/*.  If ?? matches the unix account
holder's initials then they can push into that name, otherwise
they can't.  This prevents any sort of naming collisions between
developers.

To make things slightly easier I've also suggested that people name
their local branches with the same ??/ prefix as they need to use
on the central repository, thereby making the branch name the same
everywhere.  This is actually one reason why git-completion.bash
suggests "??/foo:??/foo" when completing a branch name to git-push.
:)

So I don't think its a great idea to assume by default that
refs/heads/??/* is private to this repository.


Though on second thought maybe we should be using developer private
repositories with object alternates pointing at a central repository.
But that means users have to manipulate URLs to fetch another
developer's branch vs. just using an existing remote.<nick>.url,
which is the main reason we put everything into one repository.

-- 

Re: [PATCH] Do not ignore hidden refs

[Petr Baudis]

On Sat, Nov 18, 2006 at 08:27:09AM CET, Junio C Hamano wrote:
> I think, however, if we (collectively as all the Porcelain
> writers although I am not really one of them) are to support it,
> they should not make distinction to the core, and it should be
> handled with the agreed-upon convention.

I guess I agree.

> Personally I established a convention to treat heads/??/* as
> "private namespace" while using heads/* as public refs for my
> own work (I do that for git.git as people know, and I do that
> for my day job project as well).  I do not think it is a great
> enough convention to be promoted as the official BCP, but it has
> been good enough for me, especially commands like "show-branch"
> and "tag -l" understands the shell-style filegrobs (e.g
> "show-branch master heads/??/*" would show what's yet to be
> polished and merged).

That's way too arbitrary for my taste, I think I needn't explain why.
:-)


What about leading underscore?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
The meaning of Stonehenge in Traflamadorian, when viewed from above, is:
"Replacement part being rushed with all possible speed."

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> On Sat, Nov 18, 2006 at 08:27:09AM CET, Junio C Hamano wrote:
>> I think, however, if we (collectively as all the Porcelain
>> writers although I am not really one of them) are to support it,
>> they should not make distinction to the core, and it should be
>> handled with the agreed-upon convention.
>
> I guess I agree.
>
>> ...  I do not think it is a great
>> enough convention to be promoted as the official BCP, but it has
>> been good enough for me, ...
>
> That's way too arbitrary for my taste, I think I needn't explain why.

Because _I_ explained why already ;-).

> What about leading underscore?

I would rather prefer to do refs/{heads,private}/ and allow
checkout to treat either of them as branches.  We are talking
about allowing checkout to go to a non-branch by storing a raw
commit object name in HEAD instead of leaving it as a symref, so
we know we are going to touch that area already.

Re: [PATCH] Do not ignore hidden refs

[Petr Baudis]

On Sat, Nov 18, 2006 at 08:50:23PM CET, Junio C Hamano wrote:
> Petr Baudis <pasky@suse.cz> writes:
> 
> > On Sat, Nov 18, 2006 at 08:27:09AM CET, Junio C Hamano wrote:
> >> ...  I do not think it is a great
> >> enough convention to be promoted as the official BCP, but it has
> >> been good enough for me, ...
> >
> > That's way too arbitrary for my taste, I think I needn't explain why.
> 
> Because _I_ explained why already ;-).

Oops. ;-) Me too sloppy today, sorry.

> > What about leading underscore?
> 
> I would rather prefer to do refs/{heads,private}/ and allow
> checkout to treat either of them as branches.  We are talking
> about allowing checkout to go to a non-branch by storing a raw
> commit object name in HEAD instead of leaving it as a symref, so
> we know we are going to touch that area already.

Cogito _heavily_ assumes on a lot of places that heads live in
refs/heads/ and tags live in refs/tags/. Besides, I think private tags
are much more useful to support than private heads (not that you would
necessarily make more private tags than private heads, but you are more
likely to want tags autopushed than heads; or perhaps that just in my
head?). So what about refs/{heads,tags}/private/? :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
The meaning of Stonehenge in Traflamadorian, when viewed from above, is:
"Replacement part being rushed with all possible speed."

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

>> What about leading underscore?
>
> I would rather prefer to do refs/{heads,private}/ and allow
> checkout to treat either of them as branches.  We are talking
> about allowing checkout to go to a non-branch by storing a raw
> commit object name in HEAD instead of leaving it as a symref, so
> we know we are going to touch that area already.

Oops.  Consider this rescinded.  I myself already talked about
"not necessarily just public vs private".  Silly me.

I think this is related to the common gripe of "why can't Junio
mark pu to be rewinded in his public repository".  We should be
able to leave the branch grouping to users.

Not just public vs private, but I can see an organization that
uses something like this:

    heads/1.5/{maint,master,next,pu} are to maintain and advance 1.5 series
    heads/1.5/topics/{foo,bar} are topics applicable to 1.5
    heads/2.0/{maint,master,next,pu} are to maintain and advance 2.0 series
    heads/2.0/topics/{foo,bar} are topics applicable only to 2.0
    heads/topics/{frotz,nitfol} are topics applicable to both

and wanting to view all the topics, only things related to 1.5,
etc.  So Porcelains _can_ enforce their own policies to arrange
things differently but we should also be able to let the users
(and project branch naming policy) to do what we've been talking
about by saying:

	use heads/private/ for your own stuff.  And have
	configuration that says "heads/private/" are private
	branches that are not subject to default
	pushing/pulling.

The real instruction from the project would say what the syntax
for telling that to git but I think you got the idea...

Re: [PATCH] Do not ignore hidden refs

[Petr Baudis]

On Sat, Nov 18, 2006 at 09:05:12PM CET, Junio C Hamano wrote:
> 	use heads/private/ for your own stuff.  And have
> 	configuration that says "heads/private/" are private
> 	branches that are not subject to default
> 	pushing/pulling.
> 
> The real instruction from the project would say what the syntax
> for telling that to git but I think you got the idea...

Yes, I fully agree that being able to have this configurable is cool,
but I'm still interested in providing a sensible out-of-the-box default
configuration for Cogito to use. But if we can agree that heads/private/
and tags/private/ are good BCP candidates, that's great. (The only
possible problem is a lot of typing incurred. Perhaps the default refs
search order should become configurable first.)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
The meaning of Stonehenge in Traflamadorian, when viewed from above, is:
"Replacement part being rushed with all possible speed."

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> On Sat, Nov 18, 2006 at 09:05:12PM CET, Junio C Hamano wrote:
>> 	use heads/private/ for your own stuff.  And have
>> 	configuration that says "heads/private/" are private
>> 	branches that are not subject to default
>> 	pushing/pulling.
>> 
>> The real instruction from the project would say what the syntax
>> for telling that to git but I think you got the idea...
>
> Yes, I fully agree that being able to have this configurable is cool,
> but I'm still interested in providing a sensible out-of-the-box default
> configuration for Cogito to use. But if we can agree that heads/private/
> and tags/private/ are good BCP candidates, that's great. (The only
> possible problem is a lot of typing incurred. Perhaps the default refs
> search order should become configurable first.)

It is not clear from the discussion so far why it is necessary
(or even is a good change), and I suspect a confusion to mix up
two different things has slipped in somewhere in the discussion.

My understanding of your original plan was to use ".foo" as
a private thing for Cogito to use to implement some cleverness
when the user talks about the branch "foo" (just like StGIT
uses "refs/bases/foo" to keep track of its private stuff related
to user branch "foo").  When the user says "my 'foo' branch",
you were going to munge that name into ".foo" and use both to
implement that cleverness (just like StGIT uses "refs/bases/foo"
in addition to "refs/heads/foo" when the user is talking about
his branch "foo").

I would rather think that it would actively be a bad thing to
make the core level to consider heads/private/foo and heads/foo
ambiguous.  When the user says "my 'foo' branch" that should
mean the "foo" branch not the "private/foo" branch.

Which certainly suggests that heads/private/, as a user visible
convention to keep developer-repository private stuff for local
use, is too verbose.

StGIT's use of refs/bases (i.e. outside refs/heads) is probably
sensible because it is not something the end user should
directly muck with nor check out.  If Porcelains want some
"special branch" for their own use to do their magic, however,
the ref cannot be outside refs/heads for it to be pointed at by
HEAD to become the "current branch" ("bisect" command comes to
mind, and I suspect "cg-seek" would have similar issues).  But
that kind of use is all under controll of the Porcelain, and I
would imagine "too long to type" objection would not apply.

Re: [PATCH] Do not ignore hidden refs

[Petr Baudis]

On Sun, Nov 19, 2006 at 01:29:27AM CET, Junio C Hamano wrote:
> My understanding of your original plan was to use ".foo" as
> a private thing for Cogito to use to implement some cleverness
> when the user talks about the branch "foo" (just like StGIT
> uses "refs/bases/foo" to keep track of its private stuff related
> to user branch "foo").  When the user says "my 'foo' branch",
> you were going to munge that name into ".foo" and use both to
> implement that cleverness (just like StGIT uses "refs/bases/foo"
> in addition to "refs/heads/foo" when the user is talking about
> his branch "foo").

(Not a plan, that's what I do now. ;-)

Yes, that's correct, but it's not what I'm talking about anymore - sorry
for the confusion. The proper solution for this is of course to move
this around in the refs/ hierarchy out of heads to a more suitable
place. I'm going to do that in few moments.

What I'm talking about now are _real_ user-visible and user-created
refs, which should however stay local to the repository the user created
them in.

> I would rather think that it would actively be a bad thing to
> make the core level to consider heads/private/foo and heads/foo
> ambiguous.  When the user says "my 'foo' branch" that should
> mean the "foo" branch not the "private/foo" branch.
> 
> Which certainly suggests that heads/private/, as a user visible
> convention to keep developer-repository private stuff for local
> use, is too verbose.

I don't think the automatic lookup of the branch with private/ prepended
is too bad a thing, but if you have a better solution...

> StGIT's use of refs/bases (i.e. outside refs/heads) is probably
> sensible because it is not something the end user should
> directly muck with nor check out.  If Porcelains want some
> "special branch" for their own use to do their magic, however,
> the ref cannot be outside refs/heads for it to be pointed at by
> HEAD to become the "current branch" ("bisect" command comes to
> mind, and I suspect "cg-seek" would have similar issues).  But
> that kind of use is all under controll of the Porcelain, and I
> would imagine "too long to type" objection would not apply.

Yes, but what I have on mind are branches the _user_ wants to declare as
local.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
The meaning of Stonehenge in Traflamadorian, when viewed from above, is:
"Replacement part being rushed with all possible speed."

Re: [PATCH] Do not ignore hidden refs

[Linus Torvalds]

On Sat, 18 Nov 2006, Petr Baudis wrote:
>
> Some of the ref manipulation tools (git-for-each-ref and git-show-ref in
> particular) would not handle hidden (~ /^\./) refs. This may be an
> acceptable or possibly even desirable behaviour for the ref walkers and
> repackers, but git-show-ref hiddenrefname must work.

No.

Refnames MUST NOT start with a ".". 

It's not even about the traditional unix "hidden file" thing. It's simply 
a syntactic issue. A ref cannot start with a ".", because we use things 
like ".." and "..." to separate them.

For the same reason, a ref must not contain - anywhere in its name - a "^" 
or a ".." or the other magic characters.

If you want to hide refs, you should do so some other way.

Re: [PATCH] Do not ignore hidden refs

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> On Sat, 18 Nov 2006, Petr Baudis wrote:
>>
>> Some of the ref manipulation tools (git-for-each-ref and git-show-ref in
>> particular) would not handle hidden (~ /^\./) refs. This may be an
>> acceptable or possibly even desirable behaviour for the ref walkers and
>> repackers, but git-show-ref hiddenrefname must work.
>
> No.
>
> Refnames MUST NOT start with a ".". 
>
> It's not even about the traditional unix "hidden file" thing. It's simply 
> a syntactic issue. A ref cannot start with a ".", because we use things 
> like ".." and "..." to separate them.

We do not forbid a refname that ends with a ".", but I think it
was a mistake.  We _can_ disambiguate the ones that begin with a
"." by saying "whatever..heads/.I-begin-with-dot", but not the
ones that end with "heads/I-end-with-dot...end-of-range".