Re: [PATCH v3 02/15] xdiff: introduce rust

[Patrick Steinhardt <ps@pks.im>]

On Thu, Sep 04, 2025 at 08:54:19PM -0700, Elijah Newren wrote:
> On Thu, Sep 4, 2025 at 4:40 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:
> > > On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:
> > > Also, the approach of making it an optional component directly
> > > contradicts the proposed policy I wrote up.  That's a recipe for
> > > additional burdensome work maintaining two implementations, when we
> > > actually want to make it easier for people to contribute functionality.
> > > It also doesn't provide any of the memory safety benefits or address any
> > > of the concerns from governments, security professionals, and other
> > > parties about the real and substantial risks of continuing to develop in
> > > C.
> >
> > The only reason why we want to have it as an optional component is to
> > make the transitioning period easier for downstream distributors. And
> > the intent is not to convert major components -- it should be trivial
> > components that we can use as test balloons, similar to how we did it
> > for all of our C99 test balloons.
> >
> > We cannot just pull the rug away under their feet without advance notice
> > that this is going to happen.
> 
> I find this statement a bit problematic for four reasons:
> 
> (1) "without advance notice" was already pointed out to be inaccurate
> in this thread, including in the exact email you are responding to;
> you could argue that there hasn't been _sufficient_ advance notice,
> but then there should be more details about what is and isn't
> sufficient.  Merely repeating this claim which brian just barely
> pointed out to you as false almost feels dishonest.

I think there is a difference between communication that happens on the
mailing list/contributors summit and communication that is intended for
the broader ecosystem:

  - The former is basically us developers discussing potential futures
    and reviewing patches. It would be _nice_ if distro maintainers of
    Git were to read these, but given the large volume of traffic in
    general I think it unlikely that majority of maintainers is keeping
    up with that traffic.

  - The latter is in the form of e.g. our release notes as well as our
    BreakingChanges document. These _are_ intended to be reviewed by
    maintainers, and the blame is on them if they don't do so.

We have never communicated either via release notes or via any kind of
committed document that Rust is going to become mandatory. There have
been lots of large threads discussing it, true. But navigating these
threads and estimating consensus isn't easy even for us developers, so
it's going to be even harder for outsiders to the community.

> (2) "pull the rug away" seems hyperbolic.  I would have liked some
> explanation as to how a transition period is expected to help, and how
> the existing transition period has been insufficient.  You do hint a
> little at the former, which I'll discuss more in point 4, but you
> neglect the latter to the point of pretending it didn't exist.   In
> short, why is a further transition period needed, and how will it
> differ from the existing one we've already had?  It's not clear to me
> why distributors must immediately update to the latest git version.
> Taylor discussed this aspect in detail in this thread; you even
> responded briefly (and tangentially?), but still as far as I can tell
> presume the latest and greatest is mandatory for them to adopt without
> stating why.  Maybe they do need to adopt the latest and greatest, but
> I haven't seen folks state why that's the case.  Did I miss it?

The problem here is that we don't have a story to tell yet. I agree that
not everyone always needs the latest and greatest, which is also why I
mentioned that I think it's fine for _new_ features to be developed in
Rust right away.

But the story is altogether different for bug and security fixes.

  - We of course backport security fixes, but would that also be the
    case if we had ported the subsystem to Rust already and now had to
    implement the security fix twice?

  - What happens if only the old C version has a security bug? Do we
    still fix it?

  - Likewise, what happens with important bug fixes? We tend to backport
    those that are easy-ish to backport, but if people are potentially
    stuck with an older Git version for years it will become harder for
    us to do so.

I think without us having a proper answer to these questions we _are_
pulling the rug away. Distros may be stuck with an old version of Git
for a significant time, and from my point of view we have to do a couple
of compromises there.

> It also feels like Rust support is being lumped in with "breaking
> changes", which to me feels misleading.  Historically, we have talked
> about breaking changes and deprecation periods and such so that users
> could adjust scripts or their command lines such that they would work
> across multiple versions of Git.  The Rust case is somewhat different
> in that we're not discussing behavioral changes of git, merely
> implementation differences.  If someone has both a C-only version of
> git and a newer version of git that was built with both Rust and C,
> any commands they run should behave the same as far as the C-vs-Rust
> goes (unless we have our normal discussions about specific behavior
> and any deprecations we want to do related to it, of course).
> 
> I do agree that reduced platform support is a negative change (though
> Rust brings other advantages that may offset this downside depending
> on your viewpoint), but I don't see why it's a breaking change and
> especially not a "pull the rug away under their feet" change.

I honestly don't quite understand this perspective. How isn't it
breaking that you cannot use that Git version at all anymore?

> (3) the use of "cannot" presupposes the policy stance which we are
> having a discussion about, which, whether intended or not, feels like
> an unfair way to attempt to shut down the conversation.

Sorry, that's not my intent.

> (4) you suggest that adding Rust as an optional component should avoid
> the problem, yet we've already had Rust as an optional component for
> the last three releases, going back to 2.49.0.  (libgit-rs and
> libgit-sys).

I don't really think that either libgit-rs or libgit-sys help in any
way. These are part of "contrib/", not built by default, and neither are
they consumed by anyone out there. So there is no reason for anyone to
build that library to the best of my knowledge.

> In this case, you helpfully provided some details distinguishing the
> type of optional component you want -- the reference to a test balloon
> suggests you want an optional component that is turned on by default
> (but which users can easily turn off). Am I correct that this is your
> intention?  If that's the case, then that's a useful distinction, but
> I think that distinction needs to be made a bit more clearly (and as a
> side effect, acknowledge that Rust has already been optionally shipped
> in some form, and was even specifically highlighted by GitHub's and
> GitLab's blog posts about the v2.49.0 release, among other places)

Yes. I think we need to have a test balloon that allows us to iterate on
the build infrastructure and allows distributors to test with them. I
think that test balloon needs to be integrated into core Git so that it
is part of the normal build process, because otherwise it wouldn't have
any exposure at all and thus not serve its purpose.

> > > For example, there is zero chance I will implement any of the
> > > SHA-1/SHA-256 compatibility code twice.  I'm already doing that in my
> > > free time without any compensation at all and it's unreasonable to
> > > expect me to do it twice or even to #ifdef out all the places it would
> > > need to go.  I am happy to let someone else take responsibility for the
> > > project instead, however, if they would like to do those things.
> >
> > And that's totally fair. From my point of view, this compatibility code
> > is a _new_ feature that we are adding to Git. And as I mentioned, I
> > think it is reasonable to say that new features may be implemented in
> > Rust now already, as platforms that aren't yet ready wouldn't lose any
> > existing functionality.
> 
> Am I correct to understand that you're suggesting a policy where brian
> cannot modify any existing code to be written in Rust, and can only
> add new Rust code?  Perhaps the SHA-1/SHA-256 compatibility code is
> just new code, or can be done with minimal changes to existing C code
> while adding new code.  If so, maybe this is a workable solution for
> him.

Yeah, that's my hope, as well. There's probably nouances to this though,
and we'll have to figure it out once the series hits the mailing list.
So...

> But if it can't be done with minimal changes to existing C code and
> this policy would impair brian's ability to deliver the compatibility
> code, then I think this policy would be unworkable.  I really don't
> want to hamstring brian's ability to implement the compatibility code.
> It has sat dormant for years with no one else stepping up to the
> plate, it's a really important project, and brian has time and energy
> now.  I don't want any chicken-and-egg problems introduced for him
> with the 3.0 release.  Even though I've been working with Ezekiel on
> xdiff, and I'm obviously a bit biased in that area, I find the
> sha1-sha256 compatibility work to be more critical and something we
> should do everything possible to facilitate.

... I guess we'll have to see how this looks like in the end. If the
series rewrites a bunch of subsystems in Rust I think we should figure
out whether we can do without that. Or, in the worst case, whether it is
feasible to conditionally compile some of the code with either C or
Rust, even though nobody likes that.

Patrick