Migration to CTI services should be an opportunity to reduce dependency on git hooks.

Migration to CTI services should be an opportunity to reduce dependency on git hooks.

[Carlos O'Donell]

The gcc community was locked out of committing because of
issues with the AdaCore hooks:

https://inbox.sourceware.org/gcc/ZmmK9V1j8F89DLud@tucnak/

This happened again today:

https://inbox.sourceware.org/overseers/3165b74f-4357-4a98-9c25-9d88f253d5a8@foss.st.com/

My strong opinion, particularly for glibc is that we have as
few hooks as possible, and timeout those remote hooks in a
reasonable amount of time.

As we transition the services we should be looking to architect
a safe and robust system that can handle the hooks timing out,
and degrade to a point where information in bugzilla
(which is not critical IMO) can be dropped or updated later
by hand.

-- 
Cheers,
Carlos.

Re: Migration to CTI services should be an opportunity to reduce dependency on git hooks.

[Frank Ch. Eigler]

Hi -

> The gcc community was locked out of committing because of
> issues with the AdaCore hooks: [...]
> My strong opinion, particularly for glibc is that we have as
> few hooks as possible, and timeout those remote hooks in a
> reasonable amount of time.

Note that these hooks exist only at the pleasure and control of each
individual project.

> As we transition the services we should be looking to architect
> a safe and robust system that can handle the hooks timing out [...]

You don't have to wait - such a thing can be prototyped today.
Is the issue mainly the lack of anyone's time to do that?


- FChE

Re: Migration to CTI services should be an opportunity to reduce dependency on git hooks.

[Carlos O'Donell]

On 6/21/24 9:30 AM, Frank Ch. Eigler wrote:
>> The gcc community was locked out of committing because of
>> issues with the AdaCore hooks: [...]
>> My strong opinion, particularly for glibc is that we have as
>> few hooks as possible, and timeout those remote hooks in a
>> reasonable amount of time.
> 
> Note that these hooks exist only at the pleasure and control of each
> individual project.

To the degree that these hooks don't impact other services though.

The hooks themselves use resources, so we need to have a design discussion
with the community when that community needs custom hooks:

 * what is required
 * what is desirable
 * what can we do to improve security and isolation
 * how can we design a system that degrades gracefully
 
etc. 
 
>> As we transition the services we should be looking to architect
>> a safe and robust system that can handle the hooks timing out [...]
> > You don't have to wait - such a thing can be prototyped today.

We're tackling glibc today and doing just that as we transition to CTI services.

> Is the issue mainly the lack of anyone's time to do that?

We will get to it as part of CTI services since it's in the SOW scope?

For gcc, binutils and gdb we could engage to gather more exact requirements
from the developers doing the day-to-day work e.g. what features did we really
need from the AdaCore hooks.

Then scope out the solution and what it looks like, and a prototype would
certainly go a long way to giving a good budget estimate.

Then figure out deployment and cost e.g. small vm to run the isolated hook
that returns a result, average timeout, and degraded result e.g. no bugzilla
data.

Then the usual TAC review, vote, and approve the spend (unless already covered
under the budget).

Would you be interested in working on this for the gcc hooks?

-- 
Cheers,
Carlos.

Re: Migration to CTI services should be an opportunity to reduce dependency on git hooks.

[Joseph Myers]

On Fri, 21 Jun 2024, Carlos O'Donell wrote:

> Then figure out deployment and cost e.g. small vm to run the isolated hook
> that returns a result, average timeout, and degraded result e.g. no bugzilla
> data.

There is a significant difference between pre-commit and post-commit 
actions from hooks.

Post-commit actions can be run in some completely different context (that 
receives some kind of append-only feed of ref changes in the repository), 
and a degraded result might be e.g. no commit mail or Bugzilla update.

Pre-commit actions ensure that some invariant of the commit history is 
maintained.  For example, in the GCC context, they ensure that the nightly 
ChangeLog updates won't fall over by verifying various commit properties 
(on master and release branches) required by those updates (imperfectly, 
because some things pass the checks at commit time but the updates still 
fail), and that new commits don't masquerade as ones converted from SVN so 
that tools expecting a single commit with a From-SVN: line referencing a 
given SVN commit, in order to map from SVN commits to git commits, don't 
fall over from a new commit quoting an old commit message in a ways that 
could confuse the commits.  (They also enforce rules on e.g. which refs 
are allowed non-fast-forward updates.)

The safe form of degraded result from a pre-commit check is to block the 
commit within a limited time if the check doesn't complete quickly (rather 
than running for hours).  One long-term way to avoid hooks for such 
pre-commit checks would be a system based on merge-requests where pushes 
to the main repository, or at least to branches on that repository needing 
greater levels of checks, are normally done only by a CI system that 
verifies the desired invariants before pushing, rather than directly by 
individual developers.  Although that sort of thing is more commonly done 
to ensure invariants of the state of the tree (i.e. that it builds and 
passes tests in some set of configurations), it certainly could be done to 
ensure invariants of the commit history instead.

-- 
Joseph S. Myers
josmyers@redhat.com