[RFC/PROPOSAL] Shifting the x.y.z Stable Tree to a Continuous, Signed Patch-Stream Model

[RFC/PROPOSAL] Shifting the x.y.z Stable Tree to a Continuous, Signed Patch-Stream Model

[Artem S. Tashkinov]

Hi all,

The relentless cadence of critical vulnerability disclosures and public 
exploits over the past month—including Copy Fail (CVE-2026-31431), Dirty 
Frag (CVE-2026-43284/500), Fragnesia (CVE-2026-46300), and the ptrace 
exit race (CVE-2026-46333)—has highlighted a severe structural 
bottleneck in how we package and distribute stable backports.

When fatal logic flaws or memory corruptions strike core subsystems, our 
current point-release model fractures. Spinning up whole new point 
releases (7.0.4, 7.0.5, 7.0.7) in a matter of days just to address 
incomplete fixes, subsystem regressions, or independent public 
disclosures (such as the recent GRO managed-frag UAF exploit dropped 
directly to GitHub gists by researchers) creates massive administrative 
fatigue for maintainers and downstream teams alike.

Upstream has long maintained that the stable tree is effectively a 
continuous stream of fixes, and that users should track the tip of the 
stable branch rather than cherry-picking. It is time our release 
infrastructure matches this reality.

### The Proposal

I propose transitioning the stable tree (`linux-x.y.y`) away from 
manual,discrete point-release tarballs (`x.y.z`). Instead, we should 
treat the stable sub-version purely as an append-only, continuous, 
git-native patch stream.

Major releases (e.g., 7.0, 7.1) remain the foundational code boundaries, 
but sub-versions are eliminated as monolithic manual artifacts.

### The Implementation: How It Works

To ensure downstream distributions, enterprise compliance engines, and 
automated testing rings can still securely ingest code, we can replace 
the manual tarball with a decoupled, automated asset pipeline:

1. **The Git-First Stream:** The stable branch (`linux-7.0.y`) remains 
the single source of truth. Commits are pushed as soon as they pass 
stable criteria and automated sanity testing.

2. **The Signed Patch-Stream Archive:** Instead of packaging the entire 
30M+ line source code tree into a new tarball for every quick fix, 
upstream infrastructure maintains a rolling, cumulative patch sequence 
for the major cycle:

linux-7.0-stable.series = \sum (patch_1 + patch_2 + ... + patch_n)

Every time a fix is merged to the stable branch, the patch is appended 
to a publicly accessible, cryptographically signed manifest file
(`linux-7.0-stable-patches.tar.bz2` or a standard `series` file) 
alongside a detached signature.

3. **Automated Snapshot Tags:** If the industry strictly requires an 
immutable archive for compliance, point-release numbers can be replaced 
by automated, time-stamped git tags and machine-generated source 
snapshots cut on a strict, automated interval (e.g., every 48 hours), 
removing human maintainers entirely from the release timing.

### Why This Benefits the Ecosystem

* **Eliminates Churn and Latency:**

When a patch introduces an edge-case regression or requires an immediate 
follow-up (a common reason for rapid point-release sequences), 
maintainers do not need to coordinate a whole new release event. The 
follow-up fix is simply patch $n+1$. Downstream CI pipelines ingest it 
natively via standard git fetches.

* **Maintains Git-Native Debugging:**

Debugging stable regressions via `git bisect` has always been 
patch-based, not release-based. Since point releases are meant strictly 
for backported bug fixes, removing the arbitrary `x.y.z` release tags 
changes nothing about a developer's ability to isolate a regression. If 
anything, it prevents downstream vendors from pulling out-of-order 
patches that complicate bisection across distros.

* **Eases Downstream Automation:**

Modern tracking distributions (Arch, Fedora snapshotting, etc.) can 
switch to trunk-based intake, automatically building from the signed tip.

For enterprise distributions (RHEL, Ubuntu LTS) where constant kernel 
packaging and reboots are untenable, a fluid patch stream allows vendor 
security teams to more rapidly feed live-patching infrastructure 
(`kpatch`, `kgraft`), applying critical CVE fixes directly to runtime 
memory without changing the base package version.

* **Bridges the Compliance Gap:**

Embedded, automotive, or medical compliance pipelines
that legally require a static, verifiable code artifact can validate 
their software against the base major release tarball ($7.0.0$) plus the 
cryptographically signed, append-only stable patch series manifest.

The manual compilation, testing, and cutting of sub-version tarballs is 
an administrative artifact of the late 1990s. Shifting to an explicit, 
signed patch-stream architecture acknowledges the velocity of modern 
vulnerability research, strips away artificial latency, and frees our 
stable maintainers to focus on code quality rather than release 
management overhead.

I would love to hear thoughts, architectural blockers, or feedback from 
the stable maintainers and distribution teams on the feasibility of this 
transition.

Best regards,
Artem S. Tashkinov

Re: [RFC/PROPOSAL] Shifting the x.y.z Stable Tree to a Continuous, Signed Patch-Stream Model

[Greg Kroah-Hartman]

On Sun, May 24, 2026 at 01:38:55PM +0400, Artem S. Tashkinov wrote:
> Hi all,
> 
> The relentless cadence of critical vulnerability disclosures and public
> exploits over the past month—including Copy Fail (CVE-2026-31431), Dirty
> Frag (CVE-2026-43284/500), Fragnesia (CVE-2026-46300), and the ptrace exit
> race (CVE-2026-46333)—has highlighted a severe structural bottleneck in how
> we package and distribute stable backports.

Who is "we"?

And there's nothing really "new" here, these issues are all normal,
remember, we resolve, on average, 13 CVEs a day, most much more severe
than the ones that happened to get marketing names that you list here
(and how many systems have untrusted users?)

> When fatal logic flaws or memory corruptions strike core subsystems, our
> current point-release model fractures. Spinning up whole new point releases
> (7.0.4, 7.0.5, 7.0.7) in a matter of days just to address incomplete fixes,
> subsystem regressions, or independent public disclosures (such as the recent
> GRO managed-frag UAF exploit dropped directly to GitHub gists by
> researchers) creates massive administrative fatigue for maintainers and
> downstream teams alike.

it takes just a minute to "spin up" a point release, what is difficult
about that?  If needed, just let us know and we can easily do so.

> Upstream has long maintained that the stable tree is effectively a
> continuous stream of fixes, and that users should track the tip of the
> stable branch rather than cherry-picking. It is time our release
> infrastructure matches this reality.
> 
> ### The Proposal
> 
> I propose transitioning the stable tree (`linux-x.y.y`) away from
> manual,discrete point-release tarballs (`x.y.z`). Instead, we should treat
> the stable sub-version purely as an append-only, continuous, git-native
> patch stream.

That's what we do today, we just happen go "jump" on a weekly basis.

> Major releases (e.g., 7.0, 7.1) remain the foundational code boundaries, but
> sub-versions are eliminated as monolithic manual artifacts.
> 
> ### The Implementation: How It Works
> 
> To ensure downstream distributions, enterprise compliance engines, and
> automated testing rings can still securely ingest code, we can replace the
> manual tarball with a decoupled, automated asset pipeline:
> 
> 1. **The Git-First Stream:** The stable branch (`linux-7.0.y`) remains the
> single source of truth. Commits are pushed as soon as they pass stable
> criteria and automated sanity testing.

Again, that's what we do today.

> 2. **The Signed Patch-Stream Archive:** Instead of packaging the entire 30M+
> line source code tree into a new tarball for every quick fix, upstream
> infrastructure maintains a rolling, cumulative patch sequence for the major
> cycle:
> 
> linux-7.0-stable.series = \sum (patch_1 + patch_2 + ... + patch_n)
> 
> Every time a fix is merged to the stable branch, the patch is appended to a
> publicly accessible, cryptographically signed manifest file
> (`linux-7.0-stable-patches.tar.bz2` or a standard `series` file) alongside a
> detached signature.

Who would use/need such a thing?  What's wrong with the 2 systems we
have today that this would somehow help out with?

> 3. **Automated Snapshot Tags:** If the industry strictly requires an
> immutable archive for compliance,

What "compliance"?

> point-release numbers can be replaced by
> automated, time-stamped git tags and machine-generated source snapshots cut
> on a strict, automated interval (e.g., every 48 hours), removing human
> maintainers entirely from the release timing.

That's probably not a good idea anyway.  Are you doing continous testing
of the stable queue?  If so, great, just take from there today.
Everyone adds patches on top of releases anyway, what's a few more if it
happens to resolve specific issues for a day or so before a .y release
can be cut?

> ### Why This Benefits the Ecosystem
> 
> * **Eliminates Churn and Latency:**
> 
> When a patch introduces an edge-case regression or requires an immediate
> follow-up (a common reason for rapid point-release sequences), maintainers
> do not need to coordinate a whole new release event.

No real "coordination" happens here.

> The follow-up fix is simply patch $n+1$. Downstream CI pipelines
> ingest it natively via standard git fetches.

Again, we do that today.

> * **Maintains Git-Native Debugging:**
> 
> Debugging stable regressions via `git bisect` has always been patch-based,
> not release-based. Since point releases are meant strictly for backported
> bug fixes, removing the arbitrary `x.y.z` release tags changes nothing about
> a developer's ability to isolate a regression. If anything, it prevents
> downstream vendors from pulling out-of-order patches that complicate
> bisection across distros.

Who bisects across distros?

> * **Eases Downstream Automation:**
> 
> Modern tracking distributions (Arch, Fedora snapshotting, etc.) can switch
> to trunk-based intake, automatically building from the signed tip.

Have you asked them if they need/want this?

> For enterprise distributions (RHEL, Ubuntu LTS) where constant kernel
> packaging and reboots are untenable,

Why are reboots for these systems untenable?  Why not fix that root
problem instead?

> a fluid patch stream allows vendor
> security teams to more rapidly feed live-patching infrastructure (`kpatch`,
> `kgraft`), applying critical CVE fixes directly to runtime memory without
> changing the base package version.

They can do that today, and do do that today.  So again, what distro
needs this?

> * **Bridges the Compliance Gap:**
> 
> Embedded, automotive, or medical compliance pipelines
> that legally require a static, verifiable code artifact can validate their
> software against the base major release tarball ($7.0.0$) plus the
> cryptographically signed, append-only stable patch series manifest.

Do they really need that?  Again, they can have that today, nothing new
here.

> The manual compilation, testing, and cutting of sub-version tarballs is an
> administrative artifact of the late 1990s.

Weekly releases is not an artivact of the 1990s :)

> Shifting to an explicit, signed
> patch-stream architecture acknowledges the velocity of modern vulnerability
> research, strips away artificial latency, and frees our stable maintainers
> to focus on code quality rather than release management overhead.

Again, we have that today, on a weekly basis.

greg k-h

Re: [RFC/PROPOSAL] Shifting the x.y.z Stable Tree to a Continuous, Signed Patch-Stream Model

[Theodore Tso]

On Sun, May 24, 2026 at 12:56:54PM +0200, Greg Kroah-Hartman wrote:
> > point-release numbers can be replaced by
> > automated, time-stamped git tags and machine-generated source snapshots cut
> > on a strict, automated interval (e.g., every 48 hours), removing human
> > maintainers entirely from the release timing.
> 
> That's probably not a good idea anyway.  Are you doing continous testing
> of the stable queue?  If so, great, just take from there today.
> Everyone adds patches on top of releases anyway, what's a few more if it
> happens to resolve specific issues for a day or so before a .y release
> can be cut?

It's already the case that not all maintainers have the time to test
the stable queue, and it's not clear that current testing of the
weekly release is all that great.  I have seen stabilty regressions
where an xfstest running against ext4 will cause the kernel to crash
with the 6.1 and 6.6 LTS kernels.  It took me several days to figure
out the 6.1 regression, and I still haven't had time to look into the
6.6 regression, because my day job (which is not ext4, but herding
cats for an AI infrastructure project --- it's amazing how many fellow
developers I met at LSF/MM are actually doing AI infrastructure
projects for $WORK, and not kernel development as their primary job
responsibilities.)

So even the weekly cadence is starting to creek a bit from a quality
perspective.  I can't even *imagine* what a continuous, automated, "it
builds, ship it!" would do to the quality of the stable kernel series.

		     	  	      	  - Ted

P.S.  If someone is interesting in helping to test ext4 and xfs stable
kernel patches, talk to me.  There is partial automation to test
updates to the stable-rc trees, but I've never had time to automate
the rest of the test regression analysis combined with the automated
"which patches need to be backed out to avoid the regression / kernel
crash".  There had been a few companies contributing fractions of
engineers to do XFS stable maintainenace, all of those resources have
been withdrawn by their respective companies in the past year.




> 
> > ### Why This Benefits the Ecosystem
> > 
> > * **Eliminates Churn and Latency:**
> > 
> > When a patch introduces an edge-case regression or requires an immediate
> > follow-up (a common reason for rapid point-release sequences), maintainers
> > do not need to coordinate a whole new release event.
> 
> No real "coordination" happens here.
> 
> > The follow-up fix is simply patch $n+1$. Downstream CI pipelines
> > ingest it natively via standard git fetches.
> 
> Again, we do that today.
> 
> > * **Maintains Git-Native Debugging:**
> > 
> > Debugging stable regressions via `git bisect` has always been patch-based,
> > not release-based. Since point releases are meant strictly for backported
> > bug fixes, removing the arbitrary `x.y.z` release tags changes nothing about
> > a developer's ability to isolate a regression. If anything, it prevents
> > downstream vendors from pulling out-of-order patches that complicate
> > bisection across distros.
> 
> Who bisects across distros?
> 
> > * **Eases Downstream Automation:**
> > 
> > Modern tracking distributions (Arch, Fedora snapshotting, etc.) can switch
> > to trunk-based intake, automatically building from the signed tip.
> 
> Have you asked them if they need/want this?
> 
> > For enterprise distributions (RHEL, Ubuntu LTS) where constant kernel
> > packaging and reboots are untenable,
> 
> Why are reboots for these systems untenable?  Why not fix that root
> problem instead?
> 
> > a fluid patch stream allows vendor
> > security teams to more rapidly feed live-patching infrastructure (`kpatch`,
> > `kgraft`), applying critical CVE fixes directly to runtime memory without
> > changing the base package version.
> 
> They can do that today, and do do that today.  So again, what distro
> needs this?
> 
> > * **Bridges the Compliance Gap:**
> > 
> > Embedded, automotive, or medical compliance pipelines
> > that legally require a static, verifiable code artifact can validate their
> > software against the base major release tarball ($7.0.0$) plus the
> > cryptographically signed, append-only stable patch series manifest.
> 
> Do they really need that?  Again, they can have that today, nothing new
> here.
> 
> > The manual compilation, testing, and cutting of sub-version tarballs is an
> > administrative artifact of the late 1990s.
> 
> Weekly releases is not an artivact of the 1990s :)
> 
> > Shifting to an explicit, signed
> > patch-stream architecture acknowledges the velocity of modern vulnerability
> > research, strips away artificial latency, and frees our stable maintainers
> > to focus on code quality rather than release management overhead.
> 
> Again, we have that today, on a weekly basis.
> 
> greg k-h
>