refspecs with '*' as part of pattern

refspecs with '*' as part of pattern

[Jacob Keller]

Hi,

I've been looking at the refspecs for git fetch, and noticed that
globs are partially supported. I wanted to use something like:

refs/tags/some-prefix-*:refs/tags/some-prefix-*

as a refspec, so that I can fetch only tags which have a specific
prefix. I know that I could use namespaces to separate tags, but
unfortunately, I am unable to fix the tag format. The specific
repository in question is also generating several tags which are not
relevant to me, in formats that are not really useful for human
consumption. I am also not able to fix this less than useful practice.

However, I noticed that refspecs only support * as a single component.
The match algorithm works perfectly fine, as documented in
abd2bde78bd9 ("Support '*' in the middle of a refspec")

What is the reason for not allowing slightly more arbitrary
expressions? Obviously no more than one *...

It seems like the "strict" requirements, as in the above commit, but i
an unable to find what these requirements are, and why they can't
allow more expressions.

It's possible that users might not expect it to work, but maybe it
could be configured behind an extra option to prevent accidental use?
I think it's quite intuitive though.

Maybe because it allows some shenanagins to rename tags, but... that's
really the user's fault... Another reason for putting this behind an
option possibly?

Personally I would like to be able to use this as it allows much more
fine grained control over what I fetch, and lets me stick with the
un-namespaced tags which are something I can't change.

If this isn't something that can be done I would appreciate a good
explanation of what it might break and why it's a bad idea...

Regards,
Jake

Re: refspecs with '*' as part of pattern

[Junio C Hamano]

Jacob Keller <jacob.keller@gmail.com> writes:

> I've been looking at the refspecs for git fetch, and noticed that
> globs are partially supported. I wanted to use something like:
>
> refs/tags/some-prefix-*:refs/tags/some-prefix-*
>
> as a refspec, so that I can fetch only tags which have a specific
> prefix. I know that I could use namespaces to separate tags, but
> unfortunately, I am unable to fix the tag format. The specific
> repository in question is also generating several tags which are not
> relevant to me, in formats that are not really useful for human
> consumption. I am also not able to fix this less than useful practice.
>
> However, I noticed that refspecs only support * as a single component.
> The match algorithm works perfectly fine, as documented in
> abd2bde78bd9 ("Support '*' in the middle of a refspec")
>
> What is the reason for not allowing slightly more arbitrary
> expressions? Obviously no more than one *...

I cannot seem to be able to find related discussions around that
patch, so this is only my guess, but I suspect that this is to
discourage people from doing something like:

	refs/tags/*:refs/tags/foo-*

which would open can of worms (e.g. imagine you fetch with that
pathspec and then push with refs/tags/*:refs/tags/* back there;
would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
tag?) we'd prefer not having to worry about.

Re: refspecs with '*' as part of pattern

[Jacob Keller]

On Mon, Jul 6, 2015 at 4:01 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jacob Keller <jacob.keller@gmail.com> writes:
>
>> What is the reason for not allowing slightly more arbitrary
>> expressions? Obviously no more than one *...
>
> I cannot seem to be able to find related discussions around that
> patch, so this is only my guess, but I suspect that this is to
> discourage people from doing something like:
>
>         refs/tags/*:refs/tags/foo-*
>
> which would open can of worms (e.g. imagine you fetch with that
> pathspec and then push with refs/tags/*:refs/tags/* back there;
> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
> tag?) we'd prefer not having to worry about.
>
>
>

That is what I assumed. Would some sort of fetch option you could set
in gitconfig be acceptable? Or possibly extending code to verify that
it doesn't do something too silly? Like extend it to verify that the
per-section path is actually the same?

Today you can get a similar issue with

refs/tags/*:refs/tags/foo/*

It just is easier to fix since they're all name-spaced into foo instead.

The issue I have is that I only want to download specific tag format,
instead of all tags.. Maybe there is a better way to handle this?

Basically, the automated build software that my team uses (I don't
have a choice here) generates tags like

scm_063015_050528

and then I re-generate human readable tags based on information so
they look like

driver-name-0-16-3

I don't want to ever pull scm_* tags because these are useless. I have
tried very much to change the behavior of the team running the build
software, but they refuse to budge (as this software works against
CVS/SVN etc, and supports some older work, and they don't want to
special case it if they can avoid it).

Basically, all I see in my git tags are useless scm tags, and I want
to not download them ever. (so that they don't show up as possible
refs at all)

Is there a possible better solution to this?

Regards,
Jake

Re: refspecs with '*' as part of pattern

[Daniel Barkalow]

On Mon, 6 Jul 2015, Junio C Hamano wrote:

> Jacob Keller <jacob.keller@gmail.com> writes:
> 
> > I've been looking at the refspecs for git fetch, and noticed that
> > globs are partially supported. I wanted to use something like:
> >
> > refs/tags/some-prefix-*:refs/tags/some-prefix-*
> >
> > as a refspec, so that I can fetch only tags which have a specific
> > prefix. I know that I could use namespaces to separate tags, but
> > unfortunately, I am unable to fix the tag format. The specific
> > repository in question is also generating several tags which are not
> > relevant to me, in formats that are not really useful for human
> > consumption. I am also not able to fix this less than useful practice.
> >
> > However, I noticed that refspecs only support * as a single component.
> > The match algorithm works perfectly fine, as documented in
> > abd2bde78bd9 ("Support '*' in the middle of a refspec")
> >
> > What is the reason for not allowing slightly more arbitrary
> > expressions? Obviously no more than one *...
> 
> I cannot seem to be able to find related discussions around that
> patch, so this is only my guess, but I suspect that this is to
> discourage people from doing something like:
> 
> 	refs/tags/*:refs/tags/foo-*
> 
> which would open can of worms (e.g. imagine you fetch with that
> pathspec and then push with refs/tags/*:refs/tags/* back there;
> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
> tag?) we'd prefer not having to worry about.

That wouldn't be it, since refs/tags/*:refs/tags/foo/* would have the same 
problem, assuming you didn't set up the push refspec carefully.

I think it was mostly that it would be too easy to accidentally do 
something you don't want by having some other character instead of a 
slash, like refs/heads/*:refs/heads-*.

Aside from the increased risk of hard-to-spot typos leading to very weird 
behavior, nothing actually goes wrong; in fact, I've been using git with 
that check removed for ages because I wanted a refspec like 
refs/heads/something-*:refs/heads/*. And it works fine as a local patch, 
since you don't need your refspec handling to interoperate with other 
repositories.

	-Daniel
*This .sig left intentionally blank*

Re: refspecs with '*' as part of pattern

[Jacob Keller]

On Mon, Jul 6, 2015 at 7:20 PM, Daniel Barkalow
<barkalow@iabervon.iabervon.org> wrote:
> On Mon, 6 Jul 2015, Junio C Hamano wrote:
>
>> Jacob Keller <jacob.keller@gmail.com> writes:
>>
>> > I've been looking at the refspecs for git fetch, and noticed that
>> > globs are partially supported. I wanted to use something like:
>> >
>> > refs/tags/some-prefix-*:refs/tags/some-prefix-*
>> >
>> > as a refspec, so that I can fetch only tags which have a specific
>> > prefix. I know that I could use namespaces to separate tags, but
>> > unfortunately, I am unable to fix the tag format. The specific
>> > repository in question is also generating several tags which are not
>> > relevant to me, in formats that are not really useful for human
>> > consumption. I am also not able to fix this less than useful practice.
>> >
>> > However, I noticed that refspecs only support * as a single component.
>> > The match algorithm works perfectly fine, as documented in
>> > abd2bde78bd9 ("Support '*' in the middle of a refspec")
>> >
>> > What is the reason for not allowing slightly more arbitrary
>> > expressions? Obviously no more than one *...
>>
>> I cannot seem to be able to find related discussions around that
>> patch, so this is only my guess, but I suspect that this is to
>> discourage people from doing something like:
>>
>>       refs/tags/*:refs/tags/foo-*
>>
>> which would open can of worms (e.g. imagine you fetch with that
>> pathspec and then push with refs/tags/*:refs/tags/* back there;
>> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
>> tag?) we'd prefer not having to worry about.
>
> That wouldn't be it, since refs/tags/*:refs/tags/foo/* would have the same
> problem, assuming you didn't set up the push refspec carefully.
>
> I think it was mostly that it would be too easy to accidentally do
> something you don't want by having some other character instead of a
> slash, like refs/heads/*:refs/heads-*.
>
> Aside from the increased risk of hard-to-spot typos leading to very weird
> behavior, nothing actually goes wrong; in fact, I've been using git with
> that check removed for ages because I wanted a refspec like
> refs/heads/something-*:refs/heads/*. And it works fine as a local patch,
> since you don't need your refspec handling to interoperate with other
> repositories.
>
>         -Daniel
> *This .sig left intentionally blank*


Which is why I suggested a patch that adds this behavior conditionally
and only for fetch. This way you don't have to use a local patch, and
it wouldn't hit normal users. Ideally we can add a flag that only
enables it for refspecs that don't interoperate.

Does this seem ok? If so I will go ahead and try to work up a patch

Regards,
Jake

Re: refspecs with '*' as part of pattern

[Jacob Keller]

On Mon, Jul 6, 2015 at 7:33 PM, Jacob Keller <jacob.keller@gmail.com> wrote:
> On Mon, Jul 6, 2015 at 7:20 PM, Daniel Barkalow
> <barkalow@iabervon.iabervon.org> wrote:
>> On Mon, 6 Jul 2015, Junio C Hamano wrote:
>>
>>> Jacob Keller <jacob.keller@gmail.com> writes:
>>>
>>> > I've been looking at the refspecs for git fetch, and noticed that
>>> > globs are partially supported. I wanted to use something like:
>>> >
>>> > refs/tags/some-prefix-*:refs/tags/some-prefix-*
>>> >
>>> > as a refspec, so that I can fetch only tags which have a specific
>>> > prefix. I know that I could use namespaces to separate tags, but
>>> > unfortunately, I am unable to fix the tag format. The specific
>>> > repository in question is also generating several tags which are not
>>> > relevant to me, in formats that are not really useful for human
>>> > consumption. I am also not able to fix this less than useful practice.
>>> >
>>> > However, I noticed that refspecs only support * as a single component.
>>> > The match algorithm works perfectly fine, as documented in
>>> > abd2bde78bd9 ("Support '*' in the middle of a refspec")
>>> >
>>> > What is the reason for not allowing slightly more arbitrary
>>> > expressions? Obviously no more than one *...
>>>
>>> I cannot seem to be able to find related discussions around that
>>> patch, so this is only my guess, but I suspect that this is to
>>> discourage people from doing something like:
>>>
>>>       refs/tags/*:refs/tags/foo-*
>>>
>>> which would open can of worms (e.g. imagine you fetch with that
>>> pathspec and then push with refs/tags/*:refs/tags/* back there;
>>> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
>>> tag?) we'd prefer not having to worry about.
>>
>> That wouldn't be it, since refs/tags/*:refs/tags/foo/* would have the same
>> problem, assuming you didn't set up the push refspec carefully.
>>
>> I think it was mostly that it would be too easy to accidentally do
>> something you don't want by having some other character instead of a
>> slash, like refs/heads/*:refs/heads-*.
>>
>> Aside from the increased risk of hard-to-spot typos leading to very weird
>> behavior, nothing actually goes wrong; in fact, I've been using git with
>> that check removed for ages because I wanted a refspec like
>> refs/heads/something-*:refs/heads/*. And it works fine as a local patch,
>> since you don't need your refspec handling to interoperate with other
>> repositories.
>>
>>         -Daniel
>> *This .sig left intentionally blank*
>
>
> Which is why I suggested a patch that adds this behavior conditionally
> and only for fetch. This way you don't have to use a local patch, and
> it wouldn't hit normal users. Ideally we can add a flag that only
> enables it for refspecs that don't interoperate.
>
> Does this seem ok? If so I will go ahead and try to work up a patch
>
> Regards,
> Jake

I am working up a patch to try and get this configurable. I'll
hopefully send it out tomorrow morning sometime.

Regards,
Jake

Re: refspecs with '*' as part of pattern

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.iabervon.org> writes:

> On Mon, 6 Jul 2015, Junio C Hamano wrote:
>
>> I cannot seem to be able to find related discussions around that
>> patch, so this is only my guess, but I suspect that this is to
>> discourage people from doing something like:
>> 
>> 	refs/tags/*:refs/tags/foo-*
>> 
>> which would open can of worms (e.g. imagine you fetch with that
>> pathspec and then push with refs/tags/*:refs/tags/* back there;
>> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
>> tag?) we'd prefer not having to worry about.
>
> That wouldn't be it, since refs/tags/*:refs/tags/foo/* would have the same 
> problem, assuming you didn't set up the push refspec carefully.

Thanks, I was wondering where you were ;-)  Nice to hear from you
from time to time.

> I think it was mostly that it would be too easy to accidentally do 
> something you don't want by having some other character instead of a 
> slash, like refs/heads/*:refs/heads-*.

Hmm, interesting thought.

But refs/heads/*:refs/heade/* would not be saved, so I do not think
that is it, either.

Re: refspecs with '*' as part of pattern

[Jacob Keller]

On Tue, Jul 7, 2015 at 9:28 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Daniel Barkalow <barkalow@iabervon.iabervon.org> writes:
>
>> On Mon, 6 Jul 2015, Junio C Hamano wrote:
>>
>>> I cannot seem to be able to find related discussions around that
>>> patch, so this is only my guess, but I suspect that this is to
>>> discourage people from doing something like:
>>>
>>>      refs/tags/*:refs/tags/foo-*
>>>
>>> which would open can of worms (e.g. imagine you fetch with that
>>> pathspec and then push with refs/tags/*:refs/tags/* back there;
>>> would you now get foo-v1.0.0 and foo-foo-v1.0.0 for their v1.0.0
>>> tag?) we'd prefer not having to worry about.
>>
>> That wouldn't be it, since refs/tags/*:refs/tags/foo/* would have the same
>> problem, assuming you didn't set up the push refspec carefully.
>
> Thanks, I was wondering where you were ;-)  Nice to hear from you
> from time to time.
>
>> I think it was mostly that it would be too easy to accidentally do
>> something you don't want by having some other character instead of a
>> slash, like refs/heads/*:refs/heads-*.
>
> Hmm, interesting thought.
>
> But refs/heads/*:refs/heade/* would not be saved, so I do not think
> that is it, either.

In this case, I'm more in favor of just allowing these refs because
the user already has to manually change the refspecs, which is
something many users will never do. I also think that given the above
comments, we're not really protecting the user from anything extra...
aside from adding more pain because the globs don't work as expected.

I did submit a patch (from my @intel.com address since I can't seem to
get git-for-windows to send from my home computer) but I am willing to
re-work to drop the setting if everyone is ok with that...

Thoughts?

Regards,
Jake