Re: [PATCH 0/9] git archive: use gzip again by default, document output stabilty

["Theodore Ts'o" <tytso@mit.edu>]

On Thu, Feb 02, 2023 at 04:17:09PM +0000, Phillip Wood wrote:
> Playing devil's advocate for a moment as we're not going to promise that the
> compressed output of "git archive" will be stable in the future perhaps we
> should use this breakage as an opportunity to highlight that to users and to
> advertize the config setting that allows them to use gzip for compressing
> archives. Reverting the change gives the misleading impression that we're
> making a commitment to keeping the output stable. The focus of this thread
> seems to be the problems relating to github which they have already
> addressed.
> 
> I think there is general agreement that it is not practical to promise that
> the compressed output of "git archive" is stable so maybe it is better to
> make that clear now while users can work around it in the short term with a
> config setting rather than waiting until we're faced with some security or
> other issue that forces a change to the output which users cannot work
> around so easily.

I would be in favor of adding a config option that allows using the
internal gzip option, although leave the default to be keep things
compatible.

The reason for that it should be easy for a forge provider such as
GitHub to break things, deliberately.  Sound insane?  Hear me out.

At $WORK, we have a highly reliable system, Paxos.  It is a highly
fault-tolerant system, so it rarely fails.  But "rarely fails" is not
the same as "never fails".  And hopefully, things should degrade
gracefully if there is a Paxos outage.  But as the Google SRE's are
fond of saying, "Hope is not a strategy".

So periodically, the people who run the Paxos service will
deliberately force downtime for a short amount of time.  The fact that
they will do this is well advertised, and scheduled ahead of time ---
and teams responsible for user-facing services are supposed to make
sure that end-users don't notice when this happens.  Maybe they won't
be able to update configurations as easily while Paxos is down, but it
shouldn't cause a user-visible outage.

So what I would recommend to the GitHub product manager, is that once
a quarter, on a well-advertised date, that they flip the switch and
break the git archive checksums for say, an hour.  Then next quarter,
they advertise that the switch will be thrown for 2 hours, doubling
each time, until it is ramped up to 16 hours.

This will provide the necessary nudge so that all of these badly
designed systems that depend on downloaded archives of arbitrary git
hubs to be stable will rethink their position, while minimizing the
end-user customer impact.  Otherwise, I predict that Bazel, homebrew,
etc will consider to rely on this ill-considered assumption, and at
some point in the future, when we *do* have a much better reason to
want to make a change to the tar or compression algorithm, all of
these end users will once again scream bloody murder.

Of course, this is going to be up to each forge provider to decide
whether they want to do this.  But we can make it easy for them to do
this thing, and I'd argue it is in our interest to make it easy for
them to do this.  Otherwise we'll get constrained in the future by the
fear of massive user blowback, no metter what we say in our
documentation regarding "no promises --- and next time, we really
mean it!"

	      	       	       	    	  - Ted