Fetch remote only if remote hasn't been fetched recently

Fetch remote only if remote hasn't been fetched recently

[M Hickford]

Hi. I'd like to fetch from a particular remote, but only if that
remote hasn't been fetched in the last hour. How could I achieve this?
Is there a relevant option for `git fetch`?

https://git-scm.com/docs/git-fetch

Re: Fetch remote only if remote hasn't been fetched recently

[Konstantin Ryabitsev]

On Thu, Jun 05, 2025 at 12:16:26PM +0100, M Hickford wrote:
> Hi. I'd like to fetch from a particular remote, but only if that
> remote hasn't been fetched in the last hour. How could I achieve this?
> Is there a relevant option for `git fetch`?

Not to my knowledge, but you can either look at the timestamp of FETCH_HEAD
(if that's the only remote you're fetching), or wrap git-fetch in a small
script that touches .git/last-fetched-foo file and you can then check that
file and exit if it's newer than 1 hour ago.

-K

Re: Fetch remote only if remote hasn't been fetched recently

[Junio C Hamano]

M Hickford <mirth.hickford@gmail.com> writes:

> Hi. I'd like to fetch from a particular remote, but only if that
> remote hasn't been fetched in the last hour. How could I achieve this?
> Is there a relevant option for `git fetch`?
>
> https://git-scm.com/docs/git-fetch

"git fetch" is "I want to fetch now".  If you want to pace your
fetches, you have to keep a record of when you fetched in the past
and work from there.

I sense there perhaps is an XY problem?

If "git fetch" is done outside end-user's supervision and explicit
intent, the remote-tracking branches will become much less useful to
the human users.  A good solusion that avoids this issue already
exists as the "prefetch" task of the "git maintenance" suite.

Re: Fetch remote only if remote hasn't been fetched recently

[M Hickford]

On Thu, 5 Jun 2025 at 17:55, Junio C Hamano <gitster@pobox.com> wrote:
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > Hi. I'd like to fetch from a particular remote, but only if that
> > remote hasn't been fetched in the last hour. How could I achieve this?
> > Is there a relevant option for `git fetch`?
> >
> > https://git-scm.com/docs/git-fetch
>
> "git fetch" is "I want to fetch now".  If you want to pace your
> fetches, you have to keep a record of when you fetched in the past
> and work from there.
>
> I sense there perhaps is an XY problem?
>
> If "git fetch" is done outside end-user's supervision and explicit
> intent, the remote-tracking branches will become much less useful to
> the human users.  A good solusion that avoids this issue already
> exists as the "prefetch" task of the "git maintenance" suite.

Interesting, thank you.

Suppose my network connection is intermittent. While the machine is
online, `git fetch --prefetch` runs successfully. Later I am offline.
How can I complete the fetch?

Re: Fetch remote only if remote hasn't been fetched recently

[Junio C Hamano]

M Hickford <mirth.hickford@gmail.com> writes:

> Interesting, thank you.
>
> Suppose my network connection is intermittent. While the machine is
> online, `git fetch --prefetch` runs successfully. Later I am offline.
> How can I complete the fetch?

I _think_ that it uses "git fetch --prefetch" to store what is
usually stored at refs/$R to refs/prefetch/$R (so your
remote-tracking branch refs/remotes/origin/frotz is stored at
refs/prefetch/remotes/origin/frotz instead), so I would presume that
it should be the matter of doing something silly like this:

    $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'

Re: Fetch remote only if remote hasn't been fetched recently

[M Hickford]

On Mon, 9 Jun 2025 at 15:08, Junio C Hamano <gitster@pobox.com> wrote:
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > Interesting, thank you.
> >
> > Suppose my network connection is intermittent. While the machine is
> > online, `git fetch --prefetch` runs successfully. Later I am offline.
> > How can I complete the fetch?
>
> I _think_ that it uses "git fetch --prefetch" to store what is
> usually stored at refs/$R to refs/prefetch/$R (so your
> remote-tracking branch refs/remotes/origin/frotz is stored at
> refs/prefetch/remotes/origin/frotz instead), so I would presume that
> it should be the matter of doing something silly like this:
>
>     $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'
>

Thanks, that worked.

To support Git users with sporadic connectivity, might it be useful to
add an option to git fetch? Perhaps  `git fetch --offline` or `git
fetch --complete-prefetch`?

https://git-scm.com/docs/git-fetch

Re: Fetch remote only if remote hasn't been fetched recently

[Junio C Hamano]

M Hickford <mirth.hickford@gmail.com> writes:

>>     $ git fetch . 'refs/prefetch/remotes/origin/*:refs/remotes/origin/*'
>>
>
> Thanks, that worked.
>
> To support Git users with sporadic connectivity, might it be useful to
> add an option to git fetch? Perhaps  `git fetch --offline` or `git
> fetch --complete-prefetch`?

"offline catch-up" is probably the phrase I would prefer to see in
the documentation page to explain the concept of the operation than
"complete prefetch".

"complete prefetch" sounds like an oxymoron, in that the prefetch
has already been completed long ago, and the operation being
proposed is more about using the result of that operation completed
long ago to get yourself closer to the present state of the remote
without any guarantees that you would end up being close enough.

In any case, to present it as a first-class option to end-users,
there needs a lot more thinking and a bit of work need to go into
the way "prefetch" works.  For example, the longhand I gave would
work as a good solution only when the user _knows_ that the prefetch
is more recent than their actual remote-tracking branches.  If
refs/prefetch/remotes/R/* is older, then you wouldn't be bringing
yourself closer to the present state of the remote at all with such
a fetch, but as far as I know, a normal "git fetch R" from the
remote R would not clear refs/prefetch/remotes/R/* when it
completes.  There is no "last time we fetched from there" record
kept anywhere in the repository either.  So offhand, you'd at least
need to do these:

 - teach "git fetch" that updates remote-tracking branches to remove
   the corresponding ref in the refs/prefetch/ hierarchy;

 - decide on what option name to use for this new operation and
   document it;

 - implement the option "git fetch --that-option nickname" to

   - figure out the remote-tracking hierarchy for nickname and call
     it R

   - when refs/prefetch/remotes/R/* exists, do an

     $ git fetch . "+refs/prefetch/remotes/R/*:refs/remotes/R/*"

Re: Fetch remote only if remote hasn't been fetched recently

[M Hickford]

On Wed, 11 Jun 2025 at 16:23, Junio C Hamano <gitster@pobox.com> wrote:
>
> M Hickford <mirth.hickford@gmail.com> writes:
>
> > To support Git users with sporadic connectivity, might it be useful to
> > add an option to git fetch? Perhaps  `git fetch --offline` or `git
> > fetch --complete-prefetch`?
>
> "offline catch-up" is probably the phrase I would prefer to see in
> the documentation page to explain the concept of the operation than
> "complete prefetch".

That makes sense to me.

> In any case, to present it as a first-class option to end-users,
> there needs a lot more thinking and a bit of work need to go into
> the way "prefetch" works.  For example, the longhand I gave would
> work as a good solution only when the user _knows_ that the prefetch
> is more recent than their actual remote-tracking branches.  If
> refs/prefetch/remotes/R/* is older, then you wouldn't be bringing
> yourself closer to the present state of the remote at all with such
> a fetch, but as far as I know, a normal "git fetch R" from the
> remote R would not clear refs/prefetch/remotes/R/* when it
> completes.  There is no "last time we fetched from there" record
> kept anywhere in the repository either.  So offhand, you'd at least
> need to do these:
>
>  - teach "git fetch" that updates remote-tracking branches to remove
>    the corresponding ref in the refs/prefetch/ hierarchy;
>
>  - decide on what option name to use for this new operation and
>    document it;
>
>  - implement the option "git fetch --that-option nickname" to
>
>    - figure out the remote-tracking hierarchy for nickname and call
>      it R
>
>    - when refs/prefetch/remotes/R/* exists, do an
>
>      $ git fetch . "+refs/prefetch/remotes/R/*:refs/remotes/R/*"
>

Thanks Junio for your insight.

I shall leave this as a feature request, perhaps suitable for a
microproject #leftoverbits