[PATCH 0/5] Extend pattern refspecs

[PATCH 0/5] Extend pattern refspecs

[Daniel Barkalow]

This series only supports the narrowest case of having the * in the middle 
of a side of a refspec: having it as a full path component on each side.

Patches 1-3 centralize all of the parsing and matching rules to a pair of 
functions; patch 4 makes the stored representation more convenient (and 
serves as a distinguished bisection outcome for anything I missed that was 
relying on the contents of struct refspec for patterns); and patch 5 
extends the matching implementation and loosens the ref format 
requirements to allow the * to be in the middle.

An easy followup would relax the restrictions further without requiring 
any particularly tricky further changes.

Daniel Barkalow (5):
  Make clone parse the default fetch refspec with the regular code
  Use a single function to match names against patterns
  Use the matching function to generate the match results
  Keep '*' in pattern refspecs
  Support '*' in the middle of a refspec

 builtin-clone.c    |   25 ++++++++--------
 refs.c             |   15 +++++----
 remote.c           |   78 +++++++++++++++++++++++++++++----------------------
 t/t5511-refspec.sh |   12 ++++++++
 4 files changed, 77 insertions(+), 53 deletions(-)

Re: [PATCH 0/5] Extend pattern refspecs

[Jay Soffian]

On Thu, Mar 5, 2009 at 11:56 PM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> This series only supports the narrowest case of having the * in the middle
> of a side of a refspec: having it as a full path component on each side.
>
> Patches 1-3 centralize all of the parsing and matching rules to a pair of
> functions; patch 4 makes the stored representation more convenient (and
> serves as a distinguished bisection outcome for anything I missed that was
> relying on the contents of struct refspec for patterns); and patch 5
> extends the matching implementation and loosens the ref format
> requirements to allow the * to be in the middle.
>
> An easy followup would relax the restrictions further without requiring
> any particularly tricky further changes.

This series and js/remote-improvements (e5dcbfd) in pu may not get
along completely. "git remote show" tries to show how the refspecs
expand out. And actually, that should be fine since builtin-remote now
uses the same code as fetch/push to expand the refs.

However, "git remote show -n" (-n means don't query the remote) makes
use of a new function, get_push_ref_states_noquery(), which more or
less tries to reverse the parsed refspec back into the original
string. That function relies on the current (before your patch)
refspec semantics and assumes if refspec.pattern is set, then the '*'
is at the end. So it will need tweaking.

j.

Re: [PATCH 0/5] Extend pattern refspecs

[Daniel Barkalow]

On Fri, 6 Mar 2009, Jay Soffian wrote:

> On Thu, Mar 5, 2009 at 11:56 PM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> > This series only supports the narrowest case of having the * in the middle
> > of a side of a refspec: having it as a full path component on each side.
> >
> > Patches 1-3 centralize all of the parsing and matching rules to a pair of
> > functions; patch 4 makes the stored representation more convenient (and
> > serves as a distinguished bisection outcome for anything I missed that was
> > relying on the contents of struct refspec for patterns); and patch 5
> > extends the matching implementation and loosens the ref format
> > requirements to allow the * to be in the middle.
> >
> > An easy followup would relax the restrictions further without requiring
> > any particularly tricky further changes.
> 
> This series and js/remote-improvements (e5dcbfd) in pu may not get
> along completely. "git remote show" tries to show how the refspecs
> expand out. And actually, that should be fine since builtin-remote now
> uses the same code as fetch/push to expand the refs.
> 
> However, "git remote show -n" (-n means don't query the remote) makes
> use of a new function, get_push_ref_states_noquery(), which more or
> less tries to reverse the parsed refspec back into the original
> string. That function relies on the current (before your patch)
> refspec semantics and assumes if refspec.pattern is set, then the '*'
> is at the end. So it will need tweaking.

Actually, you should be able to just drop your "buf" and use spec->src and 
spec->dst, since it just stores the original strings. So that should be 
easy enough, although it might be good to go through a remote.c function 
just in case it becomes more complicated later. On the other hand, 
get_head_names() should probably get a patch like my 1/5 to have it use 
the remote.c parser, or should use a constant "head mirror" refspec like 
that tag_refspec already in remote.c

Do you have tests for "git remote show -n"? Merging my series (on top of 
origin/master) and e5dcbfd and adding a final '*' to the string in 
get_head_names() made everything pass for me, without doing anything about 
the extra *s, but the output is clearly not quite right.

I'm not seeing anything that makes assumptions about the matching 
semantics of pattern refspecs, just stuff about how the stored form 
relates to the config-file form.

	-Daniel
*This .sig left intentionally blank*

Re: [PATCH 0/5] Extend pattern refspecs

[Jay Soffian]

On Fri, Mar 6, 2009 at 1:07 AM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> On Fri, 6 Mar 2009, Jay Soffian wrote:
>
> Actually, you should be able to just drop your "buf" and use spec->src and
> spec->dst, since it just stores the original strings. So that should be
> easy enough, although it might be good to go through a remote.c function
> just in case it becomes more complicated later. On the other hand,
> get_head_names() should probably get a patch like my 1/5 to have it use
> the remote.c parser, or should use a constant "head mirror" refspec like
> that tag_refspec already in remote.c

Okay.

> Do you have tests for "git remote show -n"?

Yes. Apparently not enough of them though if nothing is failing.

> Merging my series (on top of
> origin/master) and e5dcbfd and adding a final '*' to the string in
> get_head_names() made everything pass for me, without doing anything about
> the extra *s, but the output is clearly not quite right.

Hmm, alright.

> I'm not seeing anything that makes assumptions about the matching
> semantics of pattern refspecs, just stuff about how the stored form
> relates to the config-file form.

Okay, that sounds right.

I assume your series will end up in pu soon enough, and I think my
series is about to hop to next. What's the right way to to have them
be happy together?

j.

Re: [PATCH 0/5] Extend pattern refspecs

[Daniel Barkalow]

On Fri, 6 Mar 2009, Jay Soffian wrote:

> On Fri, Mar 6, 2009 at 1:07 AM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> > On Fri, 6 Mar 2009, Jay Soffian wrote:
> >
> > Actually, you should be able to just drop your "buf" and use spec->src and
> > spec->dst, since it just stores the original strings. So that should be
> > easy enough, although it might be good to go through a remote.c function
> > just in case it becomes more complicated later. On the other hand,
> > get_head_names() should probably get a patch like my 1/5 to have it use
> > the remote.c parser, or should use a constant "head mirror" refspec like
> > that tag_refspec already in remote.c
> 
> Okay.
> 
> > Do you have tests for "git remote show -n"?
> 
> Yes. Apparently not enough of them though if nothing is failing.

It only seems to be off by saying:

  Local ref configured for 'git push' (status not queried):
    refs/heads/** forces to refs/heads/**

so you didn't necessarily miss much, just the one thing I seem to have 
broken.

> > Merging my series (on top of
> > origin/master) and e5dcbfd and adding a final '*' to the string in
> > get_head_names() made everything pass for me, without doing anything about
> > the extra *s, but the output is clearly not quite right.
> 
> Hmm, alright.
> 
> > I'm not seeing anything that makes assumptions about the matching
> > semantics of pattern refspecs, just stuff about how the stored form
> > relates to the config-file form.
> 
> Okay, that sounds right.
> 
> I assume your series will end up in pu soon enough, and I think my
> series is about to hop to next. What's the right way to to have them
> be happy together?

The only "correctness of outcome" issue is the open-coded refspec 
initialization, I think, which is probably actually cleaner to have as a 
constant in remote.c anyway (unlike in builtin-clone, there's no 
variability at all, so it might as well be a constant.

I can amend my series to avoid adding the extra * in the message when your 
series graduates, and it should be clean enough to deal with if your 
series wound up getting dropped later; it'll be the only change in that 
file for my series, so I'd be able to drop it easily.

It'd be useful to have that message tested by your series, though, so I 
can verify my series reliably without worrying about whether I 
accidentally dropped both the fix and the test.

	-Daniel
*This .sig left intentionally blank*

Re: [PATCH 0/5] Extend pattern refspecs

[Jay Soffian]

On Fri, Mar 6, 2009 at 2:03 AM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> It'd be useful to have that message tested by your series, though, so I
> can verify my series reliably without worrying about whether I
> accidentally dropped both the fix and the test.

I'll send a patch for it in the next day or two.

j.

Re: [PATCH 0/5] Extend pattern refspecs

[Junio C Hamano]

Jay Soffian <jaysoffian@gmail.com> writes:

> This series and js/remote-improvements (e5dcbfd) in pu may not get
> along completely. "git remote show" tries to show how the refspecs
> expand out.

I've created an "early semantic conflict resolution" topic branch that is
a cross between this series and js/remote-improvements, like so:

    $ git checkout -b xx/db-refspec-vs-js-remote db/refspec-wildcard-in-the-middle
    $ git merge js/remote-improvements
    $ git apply evil-fixup.diff
    $ git commit --amend -a -m "Evil merge."

with the following "fixup-as-an-evil-merge patch", which I'd appreciate if
you two can sanity check.

The result will be queued at the tip of 'pu', and hopefully I can merge it
to 'next' when this series goes to 'next'.  Similarly, whichever one goes
first to 'master' can be merged straight, but the merge of the other one
needs to merge this branch as well.

diff --git a/builtin-remote.c b/builtin-remote.c
index 7e82a52..3e4a41b 100644
--- a/builtin-remote.c
+++ b/builtin-remote.c
@@ -359,14 +358,9 @@ static int get_push_ref_states_noquery(struct ref_states *states)
 	}
 	for (i = 0; i < remote->push_refspec_nr; i++) {
 		struct refspec *spec = remote->push + i;
-		char buf[PATH_MAX];
 		if (spec->matching)
 			item = string_list_append("(matching)", &states->push);
-		else if (spec->pattern) {
-			snprintf(buf, (sizeof(buf)), "%s*", spec->src);
-			item = string_list_append(buf, &states->push);
-			snprintf(buf, (sizeof(buf)), "%s*", spec->dst);
-		} else if (strlen(spec->src))
+		else if (strlen(spec->src))
 			item = string_list_append(spec->src, &states->push);
 		else
 			item = string_list_append("(delete)", &states->push);
@@ -374,10 +368,7 @@ static int get_push_ref_states_noquery(struct ref_states *states)
 		info = item->util = xcalloc(sizeof(struct push_info), 1);
 		info->forced = spec->force;
 		info->status = PUSH_STATUS_NOTQUERIED;
-		if (spec->pattern)
-			info->dest = xstrdup(buf);
-		else
-			info->dest = xstrdup(spec->dst ? spec->dst : item->string);
+		info->dest = xstrdup(spec->dst ? spec->dst : item->string);
 	}
 	return 0;
 }
@@ -390,7 +381,7 @@ static int get_head_names(const struct ref *remote_refs, struct ref_states *stat
 
 	refspec.force = 0;
 	refspec.pattern = 1;
-	refspec.src = refspec.dst = "refs/heads/";
+	refspec.src = refspec.dst = "refs/heads/*";
 	states->heads.strdup_strings = 1;
 	get_fetch_map(remote_refs, &refspec, &fetch_map_tail, 0);
 	matches = guess_remote_head(find_ref_by_name(remote_refs, "HEAD"),

Re: [PATCH 0/5] Extend pattern refspecs

[Daniel Barkalow]

On Sun, 8 Mar 2009, Junio C Hamano wrote:

> Jay Soffian <jaysoffian@gmail.com> writes:
> 
> > This series and js/remote-improvements (e5dcbfd) in pu may not get
> > along completely. "git remote show" tries to show how the refspecs
> > expand out.
> 
> I've created an "early semantic conflict resolution" topic branch that is
> a cross between this series and js/remote-improvements, like so:
> 
>     $ git checkout -b xx/db-refspec-vs-js-remote db/refspec-wildcard-in-the-middle
>     $ git merge js/remote-improvements
>     $ git apply evil-fixup.diff
>     $ git commit --amend -a -m "Evil merge."
> 
> with the following "fixup-as-an-evil-merge patch", which I'd appreciate if
> you two can sanity check.

That looks like what I'd come up with as a resolution, too, so I think 
it's sane unless Jay knows of another way to get remote to care about 
patterns.

> The result will be queued at the tip of 'pu', and hopefully I can merge it
> to 'next' when this series goes to 'next'.  Similarly, whichever one goes
> first to 'master' can be merged straight, but the merge of the other one
> needs to merge this branch as well.
> 
> diff --git a/builtin-remote.c b/builtin-remote.c
> index 7e82a52..3e4a41b 100644
> --- a/builtin-remote.c
> +++ b/builtin-remote.c
> @@ -359,14 +358,9 @@ static int get_push_ref_states_noquery(struct ref_states *states)
>  	}
>  	for (i = 0; i < remote->push_refspec_nr; i++) {
>  		struct refspec *spec = remote->push + i;
> -		char buf[PATH_MAX];
>  		if (spec->matching)
>  			item = string_list_append("(matching)", &states->push);
> -		else if (spec->pattern) {
> -			snprintf(buf, (sizeof(buf)), "%s*", spec->src);
> -			item = string_list_append(buf, &states->push);
> -			snprintf(buf, (sizeof(buf)), "%s*", spec->dst);
> -		} else if (strlen(spec->src))
> +		else if (strlen(spec->src))
>  			item = string_list_append(spec->src, &states->push);
>  		else
>  			item = string_list_append("(delete)", &states->push);
> @@ -374,10 +368,7 @@ static int get_push_ref_states_noquery(struct ref_states *states)
>  		info = item->util = xcalloc(sizeof(struct push_info), 1);
>  		info->forced = spec->force;
>  		info->status = PUSH_STATUS_NOTQUERIED;
> -		if (spec->pattern)
> -			info->dest = xstrdup(buf);
> -		else
> -			info->dest = xstrdup(spec->dst ? spec->dst : item->string);
> +		info->dest = xstrdup(spec->dst ? spec->dst : item->string);
>  	}
>  	return 0;
>  }
> @@ -390,7 +381,7 @@ static int get_head_names(const struct ref *remote_refs, struct ref_states *stat
>  
>  	refspec.force = 0;
>  	refspec.pattern = 1;
> -	refspec.src = refspec.dst = "refs/heads/";
> +	refspec.src = refspec.dst = "refs/heads/*";
>  	states->heads.strdup_strings = 1;
>  	get_fetch_map(remote_refs, &refspec, &fetch_map_tail, 0);
>  	matches = guess_remote_head(find_ref_by_name(remote_refs, "HEAD"),
> 

Re: [PATCH 0/5] Extend pattern refspecs

[Jay Soffian]

On Sun, Mar 8, 2009 at 4:49 AM, Daniel Barkalow <barkalow@iabervon.org> wrote:
> On Sun, 8 Mar 2009, Junio C Hamano wrote:
>
>> Jay Soffian <jaysoffian@gmail.com> writes:
>>
>> > This series and js/remote-improvements (e5dcbfd) in pu may not get
>> > along completely. "git remote show" tries to show how the refspecs
>> > expand out.
>>
>> I've created an "early semantic conflict resolution" topic branch that is
>> a cross between this series and js/remote-improvements, like so:
>>
>>     $ git checkout -b xx/db-refspec-vs-js-remote db/refspec-wildcard-in-the-middle
>>     $ git merge js/remote-improvements
>>     $ git apply evil-fixup.diff
>>     $ git commit --amend -a -m "Evil merge."
>>
>> with the following "fixup-as-an-evil-merge patch", which I'd appreciate if
>> you two can sanity check.
>
> That looks like what I'd come up with as a resolution, too, so I think
> it's sane unless Jay knows of another way to get remote to care about
> patterns.

Looks good to me too.

j.