[PATCH 0/5] docs: improve guidance for AI-assisted bug reports

[PATCH 0/5] docs: improve guidance for AI-assisted bug reports

[Willy Tarreau]

While vulnerability reporters have now started CCing maintainers,
showing they read the docs, the security team still spends a lot of
time repeating the same comments about tested version, incomplete
fixes, poor email client setup causing formatting issues making
patches unusable, unverified reports and missing Assisted-By tags,
each time for AI-assisted reports.

This series adds small updates to security-bugs.rst, threat-model.rst
and coding-assistant.rst to better deal with this and provide minimal
instructions helping the LLM follow our expectations.

The updates were iteratively and carefully tested with 3 models,
Opus-5, Qwen3.6-27B-Architect-Polaris2-Fable-B-F451, and Gemini,
until all of them strictly followed the rules.

It is expected to further improve the situation.

Willy Tarreau (5):
  docs: threat-model: clarify "security bug" vs "vulnerability"
  docs: threat-model: move fake devices out of "non production use"
  docs: security-bugs: clarify what counts as a valid version
  docs: coding-assistant: explain important steps when looking for bugs
  docs: security-bugs: clarify some mandatory steps for AI reports

 Documentation/process/coding-assistants.rst | 37 +++++++++++++++++++
 Documentation/process/security-bugs.rst     | 26 ++++++++++++++
 Documentation/process/threat-model.rst      | 39 ++++++++++++---------
 3 files changed, 85 insertions(+), 17 deletions(-)

-- 
2.52.0

[PATCH 1/5] docs: threat-model: clarify "security bug" vs "vulnerability"

[Willy Tarreau]

Throwing an LLM (Opus 5) at a file looking for random bugs after having
read the threat model made it ignore certain bugs it found because "the
threat model said they must be ignored". When asked why, the LLM
rightfully reported the ambiguous wording used at a few places:
"is not a security bug", which can also be read as "is not a bug",
despite the rest of the document. That is particularly true when the
LLM decides to look for info using grep.

This change replaces "security bug" with "vulnerability" at a few places
in order to remove this identified ambiguity, and uses "bugs" instead of
issues in two such sentences to insist that what is described remains a
bug.

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 Documentation/process/threat-model.rst | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/Documentation/process/threat-model.rst b/Documentation/process/threat-model.rst
index 9dd8011dde828..7990673072c4d 100644
--- a/Documentation/process/threat-model.rst
+++ b/Documentation/process/threat-model.rst
@@ -98,11 +98,11 @@ measures whose purpose is to avoid crossing a security boundary when certain
 classes of bugs are found, but a failure of these extra protections do not
 constitute a vulnerability alone.
 
-What does not constitute a security bug
----------------------------------------
+What classes of problems are not considered vulnerabilities
+-----------------------------------------------------------
 
 In the Linux kernel's threat model, the following classes of problems are
-**NOT** considered as Linux Kernel security bugs. However, when it is believed
+**NOT** considered Linux Kernel vulnerabilities. However, when it is believed
 that the kernel could do better, they should be reported, so that they can be
 reviewed and fixed where reasonably possible, but they will be handled as any
 regular bug:
@@ -111,8 +111,8 @@ regular bug:
 
   * outdated kernels and particularly end-of-life branches are out of the scope
     of the kernel's threat model: administrators are responsible for keeping
-    their system up to date. For a bug to qualify as a security bug, it must be
-    demonstrated that it affects actively maintained versions.
+    their system up to date. For a bug to qualify as a vulnerability, it must
+    be demonstrated that it affects actively maintained versions.
 
   * build-level: changes to the kernel configuration that are explicitly
     documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or
@@ -178,7 +178,7 @@ regular bug:
     involving tens of millions of threads, tens of thousands of CPUs,
     unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds).
 
-  * issues whose reproduction requires hardware modification or emulation,
+  * bugs whose reproduction requires hardware modification or emulation,
     including fake USB devices that pretend to be another one.
 
   * as well as issues that can be triggered at a cost that is orders of
@@ -208,17 +208,17 @@ regular bug:
     messages.
 
   * Leaks of kernel memory addresses/pointers do not constitute an immediately
-    exploitable vector and are not security bugs, though they must be reported
-    and fixed.
+    exploitable vector and are not vulnerabilities, though they must be
+    reported and fixed.
 
 * **Crafted file system images**:
 
   * bugs triggered by mounting a corrupted or maliciously crafted file system
-    image are generally not security bugs, as the kernel assumes the underlying
+    image are generally not vulnerabilities, as the kernel assumes the underlying
     storage media is under the administrator's control, unless the filesystem
     driver is specifically documented as being hardened against untrusted media.
 
-  * issues that are resolved, mitigated, or detected by running a filesystem
+  * bugs that are resolved, mitigated, or detected by running a filesystem
     consistency check (fsck) on the image prior to mounting.
 
 * **Physical access**:
@@ -232,4 +232,4 @@ regular bug:
 * **Functional and performance regressions**:
 
   Any issue that can be mitigated by setting proper permissions and limits
-  doesn't qualify as a security bug.
+  doesn't qualify as a vulnerability.
-- 
2.52.0

[PATCH 2/5] docs: threat-model: move fake devices out of "non production use"

[Willy Tarreau]

When originally writing the "fake USB device" entry, it was difficult
to find a suitable section for it and it ended up in "non production
use" but that doesn't fit particularly well. Actually it's very similar
to crafted filesystems, it's a matter of spec violation. Both drivers
and FS are written against a spec, and what the threat model doesn't
cover is out-of-spec use. So let's move the entry there and rename the
crafted FS entry to "Non-conforming devices and media" instead.

Overall it looks more consistent.

The spec was tested agains Qwen3.6-27B-Architect-Polaris2-Fable-B-F451,
Opus-5 and Gemini by reading the threat-model file, then reading a tens
of FS and driver fixes, and they were now all properly classified as
regular bugs, except two that Gemini and Opus rightfully classified as
vulns (Qwen didn't spot the security potential but that's out of our
scope).

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 Documentation/process/threat-model.rst | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/Documentation/process/threat-model.rst b/Documentation/process/threat-model.rst
index 7990673072c4d..a68be888ce8e5 100644
--- a/Documentation/process/threat-model.rst
+++ b/Documentation/process/threat-model.rst
@@ -178,9 +178,6 @@ regular bug:
     involving tens of millions of threads, tens of thousands of CPUs,
     unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds).
 
-  * bugs whose reproduction requires hardware modification or emulation,
-    including fake USB devices that pretend to be another one.
-
   * as well as issues that can be triggered at a cost that is orders of
     magnitude higher than the expected benefits (e.g. fully functional keyboard
     emulator only to retrieve 7 uninitialized bytes in a structure, or
@@ -211,16 +208,24 @@ regular bug:
     exploitable vector and are not vulnerabilities, though they must be
     reported and fixed.
 
-* **Crafted file system images**:
+* **Non-conforming devices and media**:
 
-  * bugs triggered by mounting a corrupted or maliciously crafted file system
-    image are generally not vulnerabilities, as the kernel assumes the underlying
-    storage media is under the administrator's control, unless the filesystem
-    driver is specifically documented as being hardened against untrusted media.
+  Drivers are implemented against a specification. When a device or a storage
+  medium violates the specification its driver was written against, the
+  resulting misbehaviour is a regular bug to be fixed, not a vulnerability,
+  unless the driver is specifically documented as being hardened against
+  hostile inputs. The following are therefore not considered vulnerabilities:
 
-  * bugs that are resolved, mitigated, or detected by running a filesystem
+  * bugs triggered by mounting a corrupted or maliciously crafted file system
+    image: mounting a block device is a privileged operation (see above), and
+    the administrator is responsible for the media they mount. This includes
+    issues that are resolved, mitigated, or detected by running a filesystem
     consistency check (fsck) on the image prior to mounting.
 
+  * bugs whose reproduction requires hardware modification or emulation,
+    including fake USB devices that pretend to be another one, or devices
+    reporting values outside their documented ranges.
+
 * **Physical access**:
 
   Issues that require physical access to the machine, hardware modification, or
-- 
2.52.0

[PATCH 3/5] docs: security-bugs: clarify what counts as a valid version

[Willy Tarreau]

Too often we're getting reports saying "still valid in latest mainline"
with no indication of when this was verified, making this indication
pointless. Let's clarify it and insist on having a version or commit ID,
and that the version must necessarily be for a kernel.org kernel and not
a distro one.

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 Documentation/process/security-bugs.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Documentation/process/security-bugs.rst b/Documentation/process/security-bugs.rst
index 3c51ddde31dd9..6f7c21515e9ea 100644
--- a/Documentation/process/security-bugs.rst
+++ b/Documentation/process/security-bugs.rst
@@ -18,6 +18,10 @@ information is helpful.  The following information are absolutely necessary in
 **any** security bug report:
 
   * **affected kernel version range**: with no version indication, your report
+    will not be processed.  Note that a time-dependent version such as "latest
+    mainline" is not acceptable.  A stable identifier such as a commit ID or an
+    exact version is required.  Versions designating kernels not coming from
+    kernel.org (such as distro kernels) are meaningless to maintainers and
     will not be processed.  A significant part of reports are for bugs that
     have already been fixed, so it is extremely important that vulnerabilities
     are verified on recent versions (development tree or latest stable
-- 
2.52.0

[PATCH 4/5] docs: coding-assistant: explain important steps when looking for bugs

[Willy Tarreau]

Due to the increasing capabilities of available AI models, it's becoming
common to see them used to find bugs anywhere. Unfortunately the quality
of reports (especially when they're believed to be security relevant) is
still lacking a lot.

Let's add a section dedicated to bug finding, explaining the few
mandatory steps (noting commit ID, writing the fix from the session that
found the bug, building and testing, etc). This was tested both against
Qwen3.6-27B-Architect-Polaris2-Fable-B-F451 running under Hermes, and
Opus-5, and both followed the instructions to the letter, verifying
their results and checking threat-model.rst to decline the vulnerability
aspect. At least in the current form it's expected to improve the
situation a little bit.

Cc: Greg KH <greg@kroah.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 Documentation/process/coding-assistants.rst | 37 +++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/Documentation/process/coding-assistants.rst b/Documentation/process/coding-assistants.rst
index 899f4459c52d2..e71df7d28467e 100644
--- a/Documentation/process/coding-assistants.rst
+++ b/Documentation/process/coding-assistants.rst
@@ -57,3 +57,40 @@ Basic development tools (git, gcc, make, editors) should not be listed.
 Example::
 
   Assisted-by: Claude:claude-3-opus coccinelle sparse
+
+Procedure for finding and fixing bugs
+=====================================
+
+When an AI assistant is used to find and fix bugs, it **MUST** follow at least
+these steps:
+
+1. Before starting, read the whole process documentation listed above, as well
+   as any other document mentioned in the request. Do not rely on isolated
+   parts found by keyword search.
+2. Note the commit ID and Locate a bug as instructed.
+3. For any bug found that is not trivial, verify that it looks real by
+   attempting to create a reproducer to demonstrate it. Lacking it may cause
+   the report to be ignored, as many unverified bug reports sent to maintainers
+   happen to be invalid. Stop here if it finally looks wrong.
+4. Write a fix for the bug. This part is not optional: except in a few very
+   rare cases, an AI assistant able to find a bug is able to fix it. Note that
+   fixes written in the same session as used to find the bug will generally
+   lead to better and more accurate fixes as the LLM's reasoning context
+   remains present.
+5. Build and verify that the fix works either using the reproducer or by
+   re-running a complete analysis; drop any fix that doesn't work and try
+   another one. The fix must not add build warnings and must pass the
+   checkpatch.pl checks (see submitting-patches.rst).
+6. Commit the working fix with a detailed message describing the problem, the
+   solution and a Fixes tag. Do not add a Signed-off-by tag, and add an
+   Assisted-by tag, as described above.
+7. Identify the maintainers and lists using scripts/get_maintainer.pl.
+   Documentation/process/security-bugs.rst shows how to do that.
+8. Indicate what could not be done. If the fix could not be built or tested, or
+   if no reproducer could be produced, say so explicitly: maintainers currently
+   waste too much time analyzing unverified reports and untested fixes.
+9. Read Documentation/process/threat-model.rst to determine whether the bug is
+   a vulnerability or a regular bug, and leave the result to the reporter for
+   review (the assistant must never send anything itself). Regular bugs are
+   submitted as described in Documentation/process/submitting-patches.rst,
+   vulnerabilities as described in Documentation/process/security-bugs.rst.
-- 
2.52.0

[PATCH 5/5] docs: security-bugs: clarify some mandatory steps for AI reports

[Willy Tarreau]

The security team is still seeing a lot of reports lacking a full patch
and showing missing contents and formatting issues. Since AI assistants
tend to be better than humans at following instructions, let's defer to
coding-assistants.rst to follow mandatory steps, and insist on the plain
text format, as well as asking for recipient addresses and an e-mail
client setup hint to be mentioned early in the report for the reporter.

Also add a link to https://github.com/masoncl/kres.git which contains
way more advanced and detailed steps for those willing to go further.

Tested with Opus-5 and Qwen3.6-27B-Architect-Polaris2-Fable-B-F451, both
of which proceeded according to instructions.

Cc: Greg KH <greg@kroah.com>
Cc: Chris Mason <clm@meta.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 Documentation/process/security-bugs.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/Documentation/process/security-bugs.rst b/Documentation/process/security-bugs.rst
index 6f7c21515e9ea..c7dfecc87764c 100644
--- a/Documentation/process/security-bugs.rst
+++ b/Documentation/process/security-bugs.rst
@@ -229,6 +229,28 @@ there is no need to consume a maintainer's time with an unimportant report. If
 the issue is clearly trivial and publicly discoverable, you should report it
 directly to the public mailing lists.
 
+At the very least, when using an AI assistant to find and report bugs, make the
+assistant read this file and threat-model.rst before proceeding, and have it
+follow the procedure described in coding-assistants.rst: work on an up-to-date
+mainline tree and note the commit ID, verify the bug is real, write a fix,
+build it warning-free and checkpatch-clean, commit it with a Fixes tag, and
+identify the maintainers with get_maintainer.pl.
+
+On top of that procedure, the AI assistant **MUST**:
+
+1. Prepare a plain-text report explaining the problem. It must contain the
+   four items listed at the top of this file as absolutely necessary: the
+   affected version or commit ID noted while following the procedure above,
+   the description of the problem, the reproducer or its status, and the
+   triggering conditions.
+2. Start the report with a temporary section listing the recipients' addresses
+   (maintainers+list for the patch, maintainers only for the report and
+   reproducer), and with instructions reminding the reporter to check that
+   their email client is properly setup (see email-clients.rst), and leave it
+   to the reporter to remove that temporary section.
+
+A more detailed process is covered at https://github.com/masoncl/kres.git.
+
 Sending the report
 ------------------
 
-- 
2.52.0