[PATCH v3 0/7] agents: add skills for Fixes tags, stable tags, similar bugs, and commit messages

[PATCH v3 0/7] agents: add skills for Fixes tags, stable tags, similar bugs, and commit messages

[Sasha Levin]

At the Maintainers Summit, Ted asked whether the stable trees have a way
to deal with incorrect Fixes tags. Hannes estimated that 15-20% of them
are wrong, often because the author ran git blame and took the first
commit that touched the line [1]. We use LLM workflows to find the
commit that actually introduced a bug, and others asked us to share
them.

This series adds them as agent skills under agents/skills/, together
with a skill for writing commit messages. Each skill is a short SKILL.md
that points the agent at the process documents for its task and adds
only the checks agents tend to get wrong:

 - find-fixes determines the commit for a Fixes tag, following the
   history past code moves and refactors.

 - stable-tag decides whether a fix should go to stable and generates the
   tag. It finds the first release with the bug and checks that the fix
   applies there.

 - find-siblings looks for the same bug elsewhere in the tree, with a
   Coccinelle semantic patch when Coccinelle is available, and verifies
   each candidate.

 - commit-message writes, rewrites, or reviews a commit message or cover
   letter, and writes the text of a pull request. It follows the P:
   profiles for the changed files and keeps the text short and in the
   imperative voice, as Linus asked [2].

Rules that people need as much as agents go into the existing documents
instead: submitting-patches.rst now says which commit a Fixes tag names.
stable-kernel-rules.rst drops the 100 line limit, the rule against
theoretical issues, and the requirement that a fix be obviously correct,
none of which the stable trees apply.

Most of the skills only point at Documentation/. The exception is
stable-tag, which also lists the kinds of changes the stable trees take
and leave out. The lists describe how the stable rules are applied, not
new rules, and they save smaller models from working that out on every
fix. To keep agents/ consistent with Documentation/, I'll check one
against the other every release and trim anything in agents/ that
duplicates or contradicts it, as Kees suggested [4].

The skills only inspect and report. The calling workflow writes the
commit, creates the tag, and sends the mail. coding-assistants.rst tells
agents when to use each skill. Tools do not load agents/skills/ on their
own; to have a tool load them automatically, link the skill directories
into its skills directory.

Changes since v2:
 - Keep the skills in agents/. They are now short pointers into
   Documentation/ rather than documentation of their own; see the
   discussion on v1 [3].
 - Cut each skill to a single short SKILL.md that points at the existing
   documentation, and drop the git how-to (Kees). The skill text went
   from about 960 lines to 250.
 - find-siblings: describe how to search with Coccinelle in a separate
   file that agents read only when Coccinelle is available.
 - Say in submitting-patches.rst which commit to name when a later commit
   exposed the bug or more than one commit introduced it, and which trees
   need the fix (Michael, Thorsten, Dan).
 - Drop the 100 line limit and the exclusion of theoretical issues from
   stable-kernel-rules.rst, and require a fix to be correct rather than
   obviously correct.
 - State once, in coding-assistants.rst, that skills only inspect and
   report.

v2: https://lore.kernel.org/r/20261012213452.1863248-1-sashal@kernel.org
v1: https://lore.kernel.org/r/cover.1791499378.git.sashal@kernel.org
[1] https://lore.kernel.org/r/askTmFZC6RJKAE_p@mit.edu
[2] https://lore.kernel.org/r/CAHk-=wgXU50=KgZNZc1JDxsSxg9Z+DjG1_2yU-fNXpZtUe4XDw@mail.gmail.com
[3] https://lore.kernel.org/r/87y0bzq8oe.fsf@trenco.lwn.net
[4] https://lore.kernel.org/r/202610150832.58E80C02A@keescook

Sasha Levin (7):
  agents: add infrastructure for agent resources
  docs: submitting-patches: say which commit a Fixes tag names
  docs: stable-kernel-rules: drop rules the stable trees do not apply
  agents: add a skill to determine Fixes tags
  agents: add a skill to determine stable tags
  agents: add a skill to find other instances of a fixed bug
  agents: add a skill to write commit messages

 Documentation/process/coding-assistants.rst   | 35 +++++++--
 Documentation/process/stable-kernel-rules.rst |  6 +-
 Documentation/process/submitting-patches.rst  |  9 +++
 MAINTAINERS                                   |  6 ++
 agents/skills/commit-message/SKILL.md         | 42 +++++++++++
 agents/skills/find-fixes/SKILL.md             | 39 ++++++++++
 agents/skills/find-siblings/SKILL.md          | 31 ++++++++
 .../find-siblings/references/coccinelle.md    | 66 +++++++++++++++++
 agents/skills/stable-tag/SKILL.md             | 72 +++++++++++++++++++
 9 files changed, 297 insertions(+), 9 deletions(-)
 create mode 100644 agents/skills/commit-message/SKILL.md
 create mode 100644 agents/skills/find-fixes/SKILL.md
 create mode 100644 agents/skills/find-siblings/SKILL.md
 create mode 100644 agents/skills/find-siblings/references/coccinelle.md
 create mode 100644 agents/skills/stable-tag/SKILL.md


base-commit: 3857c2fe5449541e24afc5efdb0f81a8a8f9a3a0
-- 
2.53.0

[PATCH v3 3/7] docs: stable-kernel-rules: drop rules the stable trees do not apply

[Sasha Levin]

The stable trees do not apply the 100 line limit or the exclusion of
theoretical issues, and they require a fix to be correct and tested
rather than obviously correct. Drop those rules.

Assisted-by: LLM
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 Documentation/process/stable-kernel-rules.rst | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst
index edf90bbe30f43..625c1b5b0e16b 100644
--- a/Documentation/process/stable-kernel-rules.rst
+++ b/Documentation/process/stable-kernel-rules.rst
@@ -7,8 +7,7 @@ Rules on what kind of patches are accepted, and which ones are not, into the
 "-stable" tree:
 
 - It or an equivalent fix must already exist in Linux mainline (upstream).
-- It must be obviously correct and tested.
-- It cannot be bigger than 100 lines, with context.
+- It must be correct and tested.
 - It must follow the
   :ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
   rules.
@@ -24,9 +23,6 @@ Rules on what kind of patches are accepted, and which ones are not, into the
     regression they should only be submitted by a distribution kernel
     maintainer and include an addendum linking to a bugzilla entry if it
     exists and additional information on the user-visible impact.
-  - No "This could be a problem..." type of things like a "theoretical race
-    condition", unless an explanation of how the bug can be exploited is also
-    provided.
   - No "trivial" fixes without benefit for users (spelling changes, whitespace
     cleanups, etc).
 
-- 
2.53.0

Re: [PATCH v3 3/7] docs: stable-kernel-rules: drop rules the stable trees do not apply

[Greg Kroah-Hartman]

On Fri, Oct 16, 2026 at 11:28:23AM -0400, Sasha Levin wrote:
> The stable trees do not apply the 100 line limit or the exclusion of
> theoretical issues, and they require a fix to be correct and tested
> rather than obviously correct. Drop those rules.
> 
> Assisted-by: LLM
> Signed-off-by: Sasha Levin <sashal@kernel.org>
> ---
>  Documentation/process/stable-kernel-rules.rst | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [PATCH v3 3/7] docs: stable-kernel-rules: drop rules the stable trees do not apply

[Bradley Morgan]

On 16 October 2026 16:28:23 BST, Sasha Levin <sashal@kernel.org> wrote:
>The stable trees do not apply the 100 line limit or the exclusion of
>theoretical issues, and they require a fix to be correct and tested
>rather than obviously correct. Drop those rules.
>
>Assisted-by: LLM
>Signed-off-by: Sasha Levin <sashal@kernel.org>
>---
> Documentation/process/stable-kernel-rules.rst | 6 +-----
> 1 file changed, 1 insertion(+), 5 deletions(-)
>
>diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst
>index edf90bbe30f43..625c1b5b0e16b 100644
>--- a/Documentation/process/stable-kernel-rules.rst
>+++ b/Documentation/process/stable-kernel-rules.rst
>@@ -7,8 +7,7 @@ Rules on what kind of patches are accepted, and which ones are not, into the
> "-stable" tree:
> 
> - It or an equivalent fix must already exist in Linux mainline (upstream).
>-- It must be obviously correct and tested.
>-- It cannot be bigger than 100 lines, with context.
>+- It must be correct and tested.

s/tested/fully tested ?

'Tested" to agents would be just a basic compile test, maybe we could
define here what you want the agent to do?

E.g: a W1? A QEMU repro? And others.

> - It must follow the
>   :ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
>   rules.
>@@ -24,9 +23,6 @@ Rules on what kind of patches are accepted, and which ones are not, into the
>     regression they should only be submitted by a distribution kernel
>     maintainer and include an addendum linking to a bugzilla entry if it
>     exists and additional information on the user-visible impact.
>-  - No "This could be a problem..." type of things like a "theoretical race
>-    condition", unless an explanation of how the bug can be exploited is also
>-    provided.

To help agents filter bugs see below:

>   - No "trivial" fixes without benefit for users (spelling changes, whitespace
>     cleanups, etc).

What I tend to do when I use AI on my own projects to find bugs are to get
it to "verify" what it finds to be a good enough bug. Could you refer to
the security document here? That could help very well for AI agents to
find proper bugs.


Id also suggest that perhaps you *could* make a new patch saying that 

"When finding bugs, you need to verify that this bug is severe enough to
need stable backporting or a CVE, if you reckon it is not severe enough
for them, if it's not reproducible, eliminate the bug from the results"

What do you reckon Sasha?

> 
>

--- Thanks!
"I'm not a very positive person" - Linus torvalds

Re: [PATCH v3 3/7] docs: stable-kernel-rules: drop rules the stable trees do not apply

[Sasha Levin]

On Fri, Oct 16, 2026 at 05:01:54PM +0100, Bradley Morgan wrote:
>On 16 October 2026 16:28:23 BST, Sasha Levin <sashal@kernel.org> wrote:
>>-- It must be obviously correct and tested.
>>-- It cannot be bigger than 100 lines, with context.
>>+- It must be correct and tested.
>
>s/tested/fully tested ?
>
>'Tested" to agents would be just a basic compile test, maybe we could
>define here what you want the agent to do?
>
>E.g: a W1? A QEMU repro? And others.

This rule applies to every stable patch, and the right testing differs a
lot between, say, a driver fix and an mm change, so I'd rather not pin it
down here. For agents, coding-assistants.rst already covers it: the fix
has to be checked with the reproducer or a full re-analysis, and the
agent has to say explicitly when it could not be built or tested.

>>   - No "trivial" fixes without benefit for users (spelling changes, whitespace
>>     cleanups, etc).
>
>What I tend to do when I use AI on my own projects to find bugs are to get
>it to "verify" what it finds to be a good enough bug. Could you refer to
>the security document here? That could help very well for AI agents to
>find proper bugs.

stable-kernel-rules.rst already points security fixes at
security-bugs.rst, and coding-assistants.rst has agents read
threat-model.rst to decide whether a bug is a vulnerability.

>Id also suggest that perhaps you *could* make a new patch saying that
>
>"When finding bugs, you need to verify that this bug is severe enough to
>need stable backporting or a CVE, if you reckon it is not severe enough
>for them, if it's not reproducible, eliminate the bug from the results"

I'd rather not. A bug that doesn't qualify for stable still needs a fix
in mainline, and many real bugs, races in particular, are hard to
reproduce; that is why this patch drops the "theoretical race" rule. CVEs
are assigned by the CVE team once a fix is in a stable tree, so the
finder doesn't need to decide that.

Confirming that a bug is real before reporting it is the part that
matters, and coding-assistants.rst already asks for a reproducer.

-- 
Thanks,
Sasha

Re: [PATCH v3 3/7] docs: stable-kernel-rules: drop rules the stable trees do not apply

[SJ Park]

On Fri, 16 Oct 2026 11:28:23 -0400 Sasha Levin <sashal@kernel.org> wrote:

> The stable trees do not apply the 100 line limit or the exclusion of
> theoretical issues, and they require a fix to be correct and tested
> rather than obviously correct. Drop those rules.
> 
> Assisted-by: LLM
> Signed-off-by: Sasha Levin <sashal@kernel.org>

Acked-by: SJ Park <sj@kernel.org>

[...]
> @@ -24,9 +23,6 @@ Rules on what kind of patches are accepted, and which ones are not, into the
>      regression they should only be submitted by a distribution kernel
>      maintainer and include an addendum linking to a bugzilla entry if it
>      exists and additional information on the user-visible impact.
> -  - No "This could be a problem..." type of things like a "theoretical race
> -    condition", unless an explanation of how the bug can be exploited is also
> -    provided.

I initially thought requestion the explanation of how it can be exploited is
still good.  But, that's maybe what need to be requested when the commit is
landed on the mainline.  So this looks good to me.


Thanks,
SJ

[...]

Re: [PATCH v3 0/7] agents: add skills for Fixes tags, stable tags, similar bugs, and commit messages

[Miguel Ojeda]

On Fri, 16 Oct 2026 11:28:20 -0400 Sasha Levin <sashal@kernel.org> wrote:
>
> Changes since v2:
>  - Keep the skills in agents/. They are now short pointers into
>    Documentation/ rather than documentation of their own; see the
>    discussion on v1 [3].
>  - Cut each skill to a single short SKILL.md that points at the existing
>    documentation, and drop the git how-to (Kees). The skill text went
>    from about 960 lines to 250.

Thanks for both of these -- it does seem simpler to me this way, which
is what I expected.

>  - Drop the 100 line limit and the exclusion of theoretical issues from
>    stable-kernel-rules.rst, and require a fix to be correct rather than
>    obviously correct.

If we are going to have a skill for the stable bit and you are updating
the list, then may we take the chance to include Rust soundness issue?
Please see below.

Thanks!

Cheers,
Miguel

From c2459728a744144a592b45b5b8b807c0f15b8608 Mon Sep 17 00:00:00 2001
From: Miguel Ojeda <ojeda@kernel.org>
Date: Fri, 16 Oct 2026 17:37:25 +0200
Subject: [PATCH] docs: stable-kernel-rules: mention Rust soundness issues

Back in 2023 I asked the stable kernel team whether we could consider
Rust soundness issues for backporting, since we wanted to be able to
protect (as much as possible) downstream users of stable kernels.

After the Rust experiment ended, I documented it in the "Submit checklist
addendum" in our "Subsystem Profile document" [1] (i.e. `P:` entry in
`MAINTAINERS`):

    #### Soundness issues and backporting

    In general, fixes for Rust soundness issues should be marked for
    backport, unless there is a reason not to (such as situations where
    the backport may require a lot of effort but the risk of users
    hitting UB is low).

    For context, while the stable kernel rules mention "theoretical
    issues" should not be backported (which one could argue Rust soundness
    issues are if no existing users hit UB), we discussed the topic of
    Rust soundness issues with the stable team and decided that we would
    do our best to backport them.

    The goal is to help to prevent actual issues for downstream users,
    distributions and vendors that patch the kernel further or even
    add completely new code on top, since those modifications could
    hit UB. In other words, the idea is to maintain the "extra layer"
    of protection in stable kernels too, as much as reasonably possible.

Thus make it clearer by briefly mentioning it as well in the stable
kernel rules.

Link: https://rust-for-linux.com/contributing#soundness-issues-and-backporting [1]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
---
 Documentation/process/stable-kernel-rules.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst
index edf90bbe30f4..ffa7905543a6 100644
--- a/Documentation/process/stable-kernel-rules.rst
+++ b/Documentation/process/stable-kernel-rules.rst
@@ -17,7 +17,7 @@ Rules on what kind of patches are accepted, and which ones are not, into the

   - It fixes a problem like an oops, a hang, data corruption, a real security
     issue, a hardware quirk, a build error (but not for things marked
-    CONFIG_BROKEN), or some "oh, that's not good" issue.
+    CONFIG_BROKEN), a Rust soundness issue, or some "oh, that's not good" issue.
   - Serious issues as reported by a user of a distribution kernel may also
     be considered if they fix a notable performance or interactivity issue.
     As these fixes are not as obvious and have a higher risk of a subtle

base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
--
2.56.0

Re: [PATCH v3 0/7] agents: add skills for Fixes tags, stable tags, similar bugs, and commit messages

[Sasha Levin]

On Fri, Oct 16, 2026 at 06:00:30PM +0200, Miguel Ojeda wrote:
>If we are going to have a skill for the stable bit and you are updating
>the list, then may we take the chance to include Rust soundness issue?
>Please see below.

Could you send it as a separate patch to Greg and stable@? It doesn't really
depend on this series.

3/7 only drops rules that LLMs follow more strictly than the stable trees
actually apply them. Adding a new kind of fix to the list is a policy
change, and it should get its own review rather than ride along here.

-- 
Thanks,
Sasha

Re: [PATCH v3 0/7] agents: add skills for Fixes tags, stable tags, similar bugs, and commit messages

[Miguel Ojeda]

On Tue, Oct 20, 2026 at 3:31 PM Sasha Levin <sashal@kernel.org> wrote:
>
> Could you send it as a separate patch to Greg and stable@? It doesn't really
> depend on this series.
>
> 3/7 only drops rules that LLMs follow more strictly than the stable trees
> actually apply them. Adding a new kind of fix to the list is a policy
> change, and it should get its own review rather than ride along here.

Sounds good, I saw you split it in v4.

(To clarify, it is already the policy in practice and it is documented
on our side too; but I agree it is a good idea to discuss it
separately).

Cheers,
Miguel