git remote set-head automatically

git remote set-head automatically

[Caleb Cushing]

Context: recently I've been trying to develop a feature for my Gradle
plugin that derives a semantic version from git describe.  In order to
achieve my next level of feature I need the HEAD Branch. Now, I can
get this branch using git remote show <remote> and parsing the output.
I'm a firm believer that it should be possible for me to write code,
long term even, without the internet; I did this once with my job when
I needed to go home to my parents who were very rural and didn't have
internet; I was able to keep working without access. I want that for
anyone that uses this tool.

Problem: I don't want to have to do a network request every time I do
a build, in fact I'd rather almost never have to do a network request.
Gradle makes reducing the network request to less than once per build
something ... unsupported. jgit doesn't expose support for fetching
this information. Then I found out that you could do `git remote
set-head <remote> <branch>` which appears to behave exactly how I'd
want and expect. It doesn't easily solve my problem though because I
want my solution to be generally applicable.

Ideal Long-Term Solution: git remote set-head happens automatically on
lone, and even fetch if it's not set. Explicit setting would override
any automatic setting; and it might be a good idea to automatically
unset if the HEAD branch disappears from the remote.

Could this change be made? is there a reason not to?

Please cc, I'm not on the list.

-- 
Caleb Cushing

Appointments https://calendly.com/caleb-cushing
https://xenoterracide.com

Re: git remote set-head automatically

[Jeff King]

On Fri, Nov 15, 2024 at 09:34:28AM -0500, Caleb Cushing wrote:

> Context: recently I've been trying to develop a feature for my Gradle
> plugin that derives a semantic version from git describe.  In order to
> achieve my next level of feature I need the HEAD Branch. Now, I can
> get this branch using git remote show <remote> and parsing the output.
> I'm a firm believer that it should be possible for me to write code,
> long term even, without the internet; I did this once with my job when
> I needed to go home to my parents who were very rural and didn't have
> internet; I was able to keep working without access. I want that for
> anyone that uses this tool.

You should use the refs/remotes/<remote>/HEAD symref instead (or its
alias, just "<remote>"), which is more convenient and doesn't hit the
network. Which is exactly what...

> Problem: I don't want to have to do a network request every time I do
> a build, in fact I'd rather almost never have to do a network request.
> Gradle makes reducing the network request to less than once per build
> something ... unsupported. jgit doesn't expose support for fetching
> this information. Then I found out that you could do `git remote
> set-head <remote> <branch>` which appears to behave exactly how I'd
> want and expect. It doesn't easily solve my problem though because I
> want my solution to be generally applicable.

...set-head will set. So OK, but...

> Ideal Long-Term Solution: git remote set-head happens automatically on
> lone, and even fetch if it's not set. Explicit setting would override
> any automatic setting; and it might be a good idea to automatically
> unset if the HEAD branch disappears from the remote.

We already do the equivalent of set-head on clone. If you do:

  git clone https://github.com/git/git
  cd git
  git symbolic-ref refs/remotes/origin/HEAD

you should see it pointing to "refs/remotes/origin/master" (and like you
can refer to "origin/HEAD" or "origin" from git-log, etc). Are you not
seeing that?

We don't update it on fetch. That has been discussed, but there are some
questions about when it should overwrite what's available locally. E.g.
this thread:

  https://lore.kernel.org/git/D3HBD7C1FR14.74FL1Q1S9UCB@ferdinandy.com/

There have been some patches in that direction but I have not kept up
with the current state:

  https://lore.kernel.org/git/20241023153736.257733-2-bence@ferdinandy.com/

-Peff

Re: git remote set-head automatically

[Bence Ferdinandy]

On Sat Nov 16, 2024 at 04:36, Jeff King <peff@peff.net> wrote:
> On Fri, Nov 15, 2024 at 09:34:28AM -0500, Caleb Cushing wrote:
>
>> Context: recently I've been trying to develop a feature for my Gradle
>> plugin that derives a semantic version from git describe.  In order to
>> achieve my next level of feature I need the HEAD Branch. Now, I can
>> get this branch using git remote show <remote> and parsing the output.
>> I'm a firm believer that it should be possible for me to write code,
>> long term even, without the internet; I did this once with my job when
>> I needed to go home to my parents who were very rural and didn't have
>> internet; I was able to keep working without access. I want that for
>> anyone that uses this tool.
>
> You should use the refs/remotes/<remote>/HEAD symref instead (or its
> alias, just "<remote>"), which is more convenient and doesn't hit the
> network. Which is exactly what...
>
>> Problem: I don't want to have to do a network request every time I do
>> a build, in fact I'd rather almost never have to do a network request.
>> Gradle makes reducing the network request to less than once per build
>> something ... unsupported. jgit doesn't expose support for fetching
>> this information. Then I found out that you could do `git remote
>> set-head <remote> <branch>` which appears to behave exactly how I'd
>> want and expect. It doesn't easily solve my problem though because I
>> want my solution to be generally applicable.
>
> ...set-head will set. So OK, but...
>
>> Ideal Long-Term Solution: git remote set-head happens automatically on
>> lone, and even fetch if it's not set. Explicit setting would override
>> any automatic setting; and it might be a good idea to automatically
>> unset if the HEAD branch disappears from the remote.
>
> We already do the equivalent of set-head on clone. If you do:
>
>   git clone https://github.com/git/git
>   cd git
>   git symbolic-ref refs/remotes/origin/HEAD
>
> you should see it pointing to "refs/remotes/origin/master" (and like you
> can refer to "origin/HEAD" or "origin" from git-log, etc). Are you not
> seeing that?
>
> We don't update it on fetch. That has been discussed, but there are some
> questions about when it should overwrite what's available locally. E.g.
> this thread:
>
>   https://lore.kernel.org/git/D3HBD7C1FR14.74FL1Q1S9UCB@ferdinandy.com/

That part we already have pinned down: fetch will only update remote/HEAD if it
doesn't already exist. So running "init && remote add && fetch" should end you
up in the same place as "clone" (although fetch will report if the remote has
changed HEAD compared to what you have for transparency). If you _always_ want
to have the latest head you'll need to run "fetch && remote set-head --auto
[remote]" every time.

Something like
	git fetch --all && git remote | xargs -i git remote set-head -a {}
covers everything.

>
> There have been some patches in that direction but I have not kept up
> with the current state:
>
>   https://lore.kernel.org/git/20241023153736.257733-2-bence@ferdinandy.com/

There's definitely going to be another round, but I think (hope :)) we're not
very far off from finishing.

Best,
Bence

Re: git remote set-head automatically

[Caleb Cushing]

sounds great. I think I realized why I didn't have it. It's not done
by `git remote add <origin> https://...`  my experiment was `git
remote rm origin` and then `git remote add origin ... ; git fetch
--all --prune` I think I also tried without the prune option. git
version 2.46.1

What I want mostly is for that HEAD ref to always exist. As far as
there being ways to configure it, that's all good but I don't want to
explain doing that to consumers of my code. I'd rather it just work
for them, on clones, and add remotes or fetch if it's missing.

I'm not super worried about it being updated as I feel like if that
ever happens it's something loudly communicated, and I'm more willing
to find it ok to make that an FAQ, if this breaks because that changed
you can manually update that. Mostly I'm of the opinion that what I'm
doing needs to work in various CI environments out of the box.

Thanks for the info, hopefully soon (tm).

On Sat, Nov 16, 2024 at 10:02 AM Bence Ferdinandy <bence@ferdinandy.com> wrote:
>
>
> On Sat Nov 16, 2024 at 04:36, Jeff King <peff@peff.net> wrote:
> > On Fri, Nov 15, 2024 at 09:34:28AM -0500, Caleb Cushing wrote:
> >
> >> Context: recently I've been trying to develop a feature for my Gradle
> >> plugin that derives a semantic version from git describe.  In order to
> >> achieve my next level of feature I need the HEAD Branch. Now, I can
> >> get this branch using git remote show <remote> and parsing the output.
> >> I'm a firm believer that it should be possible for me to write code,
> >> long term even, without the internet; I did this once with my job when
> >> I needed to go home to my parents who were very rural and didn't have
> >> internet; I was able to keep working without access. I want that for
> >> anyone that uses this tool.
> >
> > You should use the refs/remotes/<remote>/HEAD symref instead (or its
> > alias, just "<remote>"), which is more convenient and doesn't hit the
> > network. Which is exactly what...
> >
> >> Problem: I don't want to have to do a network request every time I do
> >> a build, in fact I'd rather almost never have to do a network request.
> >> Gradle makes reducing the network request to less than once per build
> >> something ... unsupported. jgit doesn't expose support for fetching
> >> this information. Then I found out that you could do `git remote
> >> set-head <remote> <branch>` which appears to behave exactly how I'd
> >> want and expect. It doesn't easily solve my problem though because I
> >> want my solution to be generally applicable.
> >
> > ...set-head will set. So OK, but...
> >
> >> Ideal Long-Term Solution: git remote set-head happens automatically on
> >> lone, and even fetch if it's not set. Explicit setting would override
> >> any automatic setting; and it might be a good idea to automatically
> >> unset if the HEAD branch disappears from the remote.
> >
> > We already do the equivalent of set-head on clone. If you do:
> >
> >   git clone https://github.com/git/git
> >   cd git
> >   git symbolic-ref refs/remotes/origin/HEAD
> >
> > you should see it pointing to "refs/remotes/origin/master" (and like you
> > can refer to "origin/HEAD" or "origin" from git-log, etc). Are you not
> > seeing that?
> >
> > We don't update it on fetch. That has been discussed, but there are some
> > questions about when it should overwrite what's available locally. E.g.
> > this thread:
> >
> >   https://lore.kernel.org/git/D3HBD7C1FR14.74FL1Q1S9UCB@ferdinandy.com/
>
> That part we already have pinned down: fetch will only update remote/HEAD if it
> doesn't already exist. So running "init && remote add && fetch" should end you
> up in the same place as "clone" (although fetch will report if the remote has
> changed HEAD compared to what you have for transparency). If you _always_ want
> to have the latest head you'll need to run "fetch && remote set-head --auto
> [remote]" every time.
>
> Something like
>         git fetch --all && git remote | xargs -i git remote set-head -a {}
> covers everything.
>
> >
> > There have been some patches in that direction but I have not kept up
> > with the current state:
> >
> >   https://lore.kernel.org/git/20241023153736.257733-2-bence@ferdinandy.com/
>
> There's definitely going to be another round, but I think (hope :)) we're not
> very far off from finishing.
>
> Best,
> Bence



-- 
Caleb Cushing

Appointments https://calendly.com/caleb-cushing
https://xenoterracide.com

Re: git remote set-head automatically

[Caleb Cushing]

A little more progress on what I'm going to have to do, github action,
without set-head the ref/HEAD isn't present

- uses: actions/checkout@v4
  with:
    ref: ${{ github.event.workflow_run.head_branch}}
    filter: "blob:none"
    fetch-depth: 0
- run: git remote set-head origin --auto # fixes otherwise the cat
will not find a file
- run: cat .git/refs/remotes/origin/HEAD

https://github.com/xenoterracide/gradle-convention/actions/runs/11917790368/job/33213687888

My problem of course isn't what *I* have to do, it's explaining to
other people what to do and why; as it feels like this could set-head
implicitly.

/usr/bin/git -c protocol.version=2 fetch --prune
--no-recurse-submodules --filter=blob:none origin
+refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
+e1f39782ea2c90f5a9657ae88369599518b22a51:refs/remotes/pull/57/merge

On Tue, Nov 19, 2024 at 10:40 AM Caleb Cushing <xenoterracide@gmail.com> wrote:
>
> sounds great. I think I realized why I didn't have it. It's not done
> by `git remote add <origin> https://...`  my experiment was `git
> remote rm origin` and then `git remote add origin ... ; git fetch
> --all --prune` I think I also tried without the prune option. git
> version 2.46.1
>
> What I want mostly is for that HEAD ref to always exist. As far as
> there being ways to configure it, that's all good but I don't want to
> explain doing that to consumers of my code. I'd rather it just work
> for them, on clones, and add remotes or fetch if it's missing.
>
> I'm not super worried about it being updated as I feel like if that
> ever happens it's something loudly communicated, and I'm more willing
> to find it ok to make that an FAQ, if this breaks because that changed
> you can manually update that. Mostly I'm of the opinion that what I'm
> doing needs to work in various CI environments out of the box.
>
> Thanks for the info, hopefully soon (tm).
>
> On Sat, Nov 16, 2024 at 10:02 AM Bence Ferdinandy <bence@ferdinandy.com> wrote:
> >
> >
> > On Sat Nov 16, 2024 at 04:36, Jeff King <peff@peff.net> wrote:
> > > On Fri, Nov 15, 2024 at 09:34:28AM -0500, Caleb Cushing wrote:
> > >
> > >> Context: recently I've been trying to develop a feature for my Gradle
> > >> plugin that derives a semantic version from git describe.  In order to
> > >> achieve my next level of feature I need the HEAD Branch. Now, I can
> > >> get this branch using git remote show <remote> and parsing the output.
> > >> I'm a firm believer that it should be possible for me to write code,
> > >> long term even, without the internet; I did this once with my job when
> > >> I needed to go home to my parents who were very rural and didn't have
> > >> internet; I was able to keep working without access. I want that for
> > >> anyone that uses this tool.
> > >
> > > You should use the refs/remotes/<remote>/HEAD symref instead (or its
> > > alias, just "<remote>"), which is more convenient and doesn't hit the
> > > network. Which is exactly what...
> > >
> > >> Problem: I don't want to have to do a network request every time I do
> > >> a build, in fact I'd rather almost never have to do a network request.
> > >> Gradle makes reducing the network request to less than once per build
> > >> something ... unsupported. jgit doesn't expose support for fetching
> > >> this information. Then I found out that you could do `git remote
> > >> set-head <remote> <branch>` which appears to behave exactly how I'd
> > >> want and expect. It doesn't easily solve my problem though because I
> > >> want my solution to be generally applicable.
> > >
> > > ...set-head will set. So OK, but...
> > >
> > >> Ideal Long-Term Solution: git remote set-head happens automatically on
> > >> lone, and even fetch if it's not set. Explicit setting would override
> > >> any automatic setting; and it might be a good idea to automatically
> > >> unset if the HEAD branch disappears from the remote.
> > >
> > > We already do the equivalent of set-head on clone. If you do:
> > >
> > >   git clone https://github.com/git/git
> > >   cd git
> > >   git symbolic-ref refs/remotes/origin/HEAD
> > >
> > > you should see it pointing to "refs/remotes/origin/master" (and like you
> > > can refer to "origin/HEAD" or "origin" from git-log, etc). Are you not
> > > seeing that?
> > >
> > > We don't update it on fetch. That has been discussed, but there are some
> > > questions about when it should overwrite what's available locally. E.g.
> > > this thread:
> > >
> > >   https://lore.kernel.org/git/D3HBD7C1FR14.74FL1Q1S9UCB@ferdinandy.com/
> >
> > That part we already have pinned down: fetch will only update remote/HEAD if it
> > doesn't already exist. So running "init && remote add && fetch" should end you
> > up in the same place as "clone" (although fetch will report if the remote has
> > changed HEAD compared to what you have for transparency). If you _always_ want
> > to have the latest head you'll need to run "fetch && remote set-head --auto
> > [remote]" every time.
> >
> > Something like
> >         git fetch --all && git remote | xargs -i git remote set-head -a {}
> > covers everything.
> >
> > >
> > > There have been some patches in that direction but I have not kept up
> > > with the current state:
> > >
> > >   https://lore.kernel.org/git/20241023153736.257733-2-bence@ferdinandy.com/
> >
> > There's definitely going to be another round, but I think (hope :)) we're not
> > very far off from finishing.
> >
> > Best,
> > Bence
>
>
>
> --
> Caleb Cushing
>
> Appointments https://calendly.com/caleb-cushing
> https://xenoterracide.com



-- 
Caleb Cushing

Appointments https://calendly.com/caleb-cushing
https://xenoterracide.com

Re: git remote set-head automatically

[Jeff King]

On Tue, Nov 19, 2024 at 12:06:21PM -0500, Caleb Cushing wrote:

> A little more progress on what I'm going to have to do, github action,
> without set-head the ref/HEAD isn't present
> 
> - uses: actions/checkout@v4
>   with:
>     ref: ${{ github.event.workflow_run.head_branch}}
>     filter: "blob:none"
>     fetch-depth: 0
> - run: git remote set-head origin --auto # fixes otherwise the cat
> will not find a file

We do create the HEAD symref in clone by default, but IIRC
actions/checkout does a more limited clone with --depth and
--single-branch.

The separate set-head should work, but it is a shame that it will have
to hit the server for a second request.

You might look for options in actions/checkout to change the way it
invokes Git. Though I suspect there might not be a way to trigger
git-clone to do a single-branch clone _and_ create the HEAD symref. So
you may be stuck with the two calls.

> - run: cat .git/refs/remotes/origin/HEAD

Don't access the file directly; it will break if the ref storage format
changes. Use "git symbolic-ref refs/remotes/origin/HEAD" instead.

(Not sure if this was just illustrative or what you are planning to do
in a real CI job).

-Peff

Re: git remote set-head automatically

[Junio C Hamano]

Re: git remote set-head automatically

[Junio C Hamano]

Caleb Cushing <xenoterracide@gmail.com> writes:

> sounds great. I think I realized why I didn't have it. It's not done
> by `git remote add <origin> https://...`  my experiment was `git
> remote rm origin` and then `git remote add origin ... ; git fetch
> --all --prune` I think I also tried without the prune option. git
> version 2.46.1

Yes, "git fetch" does not notice a missing remotes/$name/HEAD and
does not automatically create it.

This is being worked on in a separate thread.

Doing it unconditionally may hurt some existing users (including me)
who see more than one primarily interesting branches in a single
remote and want to force themselves to be more explicit, though.
For us, leaving remotes/$name/HEAD missing (e.g. by "git clone"
followed by "git update-ref --no-deref -d refs/remotes/origin/HEAD")
is a way to allow ourselves to say things like

    $ git checkout -b mytopic origin/maint
    $ git rebase origin/master mytopic

but not

    $ git checkout --detach origin

because the last one is ambiguous between the two branches of
primary interest.

But hopefully they have trained their fingers not to say "origin" by
now ;-) So changing "git fetch" to auto-fill remotes/origin/HEAD to
whatever branch the remote is pointing at at the time of running
would be good enough for an initial enhanced version, even though we
might need to further improve on by allowing folks to opt out of the
feature.

Re: git remote set-head automatically

[Bence Ferdinandy]

On Wed Nov 20, 2024 at 02:17, Junio C Hamano <gitster@pobox.com> wrote:
> Caleb Cushing <xenoterracide@gmail.com> writes:
>
>> sounds great. I think I realized why I didn't have it. It's not done
>> by `git remote add <origin> https://...`  my experiment was `git
>> remote rm origin` and then `git remote add origin ... ; git fetch
>> --all --prune` I think I also tried without the prune option. git
>> version 2.46.1
>
> Yes, "git fetch" does not notice a missing remotes/$name/HEAD and
> does not automatically create it.
>
> This is being worked on in a separate thread.
>
> Doing it unconditionally may hurt some existing users (including me)
> who see more than one primarily interesting branches in a single
> remote and want to force themselves to be more explicit, though.
> For us, leaving remotes/$name/HEAD missing (e.g. by "git clone"
> followed by "git update-ref --no-deref -d refs/remotes/origin/HEAD")
> is a way to allow ourselves to say things like
>
>     $ git checkout -b mytopic origin/maint
>     $ git rebase origin/master mytopic
>
> but not
>
>     $ git checkout --detach origin
>
> because the last one is ambiguous between the two branches of
> primary interest.

You learn every day :) I had no idea that the remote's name works this way.

Thanks!

>
> But hopefully they have trained their fingers not to say "origin" by
> now ;-) So changing "git fetch" to auto-fill remotes/origin/HEAD to
> whatever branch the remote is pointing at at the time of running
> would be good enough for an initial enhanced version, even though we
> might need to further improve on by allowing folks to opt out of the
> feature.

Re: git remote set-head automatically

[Bence Ferdinandy]

On Tue Nov 19, 2024 at 16:40, Caleb Cushing <xenoterracide@gmail.com> wrote:
> sounds great. I think I realized why I didn't have it. It's not done
> by `git remote add <origin> https://...`  my experiment was `git
> remote rm origin` and then `git remote add origin ... ; git fetch
> --all --prune` I think I also tried without the prune option. git
> version 2.46.1
>
> What I want mostly is for that HEAD ref to always exist. As far as
> there being ways to configure it, that's all good but I don't want to
> explain doing that to consumers of my code. I'd rather it just work
> for them, on clones, and add remotes or fetch if it's missing.
>
> I'm not super worried about it being updated as I feel like if that
> ever happens it's something loudly communicated, and I'm more willing
> to find it ok to make that an FAQ, if this breaks because that changed
> you can manually update that. Mostly I'm of the opinion that what I'm
> doing needs to work in various CI environments out of the box.
>
> Thanks for the info, hopefully soon (tm).

It's in the new release now, it also got some configuration options, but the
default should be what you want

https://git-scm.com/docs/git-config#Documentation/git-config.txt-remoteltnamegtfollowRemoteHEAD

Best,
Bence