[PATCH 0/2] Documentation: fixups for L1D flushing

[PATCH 0/2] Documentation: fixups for L1D flushing

[Brendan Jackman]

Signed-off-by: Brendan Jackman <jackmanb@google.com>
---
Brendan Jackman (2):
      Documentation: clarify PR_SPEC_L1D_FLUSH
      Documentation: fix reference to PR_SPEC_L1D_FLUSH

 Documentation/admin-guide/hw-vuln/l1d_flush.rst | 2 +-
 Documentation/userspace-api/spec_ctrl.rst       | 6 +++++-
 2 files changed, 6 insertions(+), 2 deletions(-)
---
base-commit: 0292ef418ce08aad597fc0bba65b6dbb841808ba
change-id: 20251015-l1d-flush-doc-029f64d2b0d3

Best regards,
-- 
Brendan Jackman <jackmanb@google.com>

[PATCH 1/2] Documentation: clarify PR_SPEC_L1D_FLUSH

[Brendan Jackman]

For PR_SPEC_STORE_BYPASS and PR_SPEC_INDIRECT_BRANCH, PR_SPEC_DISABLE
means "disable the speculation bug" i.e. "enable the mitigation".

For PR_SPEC_L1D_FLUSH, PR_SPEC_DISABLE means "disable the mitigation".
This is not obvious, so document it.

Signed-off-by: Brendan Jackman <jackmanb@google.com>
---
 Documentation/userspace-api/spec_ctrl.rst | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/spec_ctrl.rst b/Documentation/userspace-api/spec_ctrl.rst
index 5e8ed9eef9aa84f12fa255af7b15c163ff01aab8..ca89151fc0a8e7205e0a0062134d63b213b9ef11 100644
--- a/Documentation/userspace-api/spec_ctrl.rst
+++ b/Documentation/userspace-api/spec_ctrl.rst
@@ -26,7 +26,8 @@ PR_GET_SPECULATION_CTRL
 
 PR_GET_SPECULATION_CTRL returns the state of the speculation misfeature
 which is selected with arg2 of prctl(2). The return value uses bits 0-3 with
-the following meaning:
+the following meaning (with the caveat that PR_SPEC_L1D_FLUSH has less obvious
+semantics, see documentation for that specific control below):
 
 ==== ====================== ==================================================
 Bit  Define                 Description
@@ -110,6 +111,9 @@ Speculation misfeature controls
 - PR_SPEC_L1D_FLUSH: Flush L1D Cache on context switch out of the task
                         (works only when tasks run on non SMT cores)
 
+For this control, PR_SPEC_ENABLE means that the **mitigation** is enabled (L1D
+is flushed), PR_SPEC_DISABLE means it is disabled.
+
   Invocations:
    * prctl(PR_GET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, 0, 0, 0);
    * prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, PR_SPEC_ENABLE, 0, 0);

-- 
2.50.1

Re: [PATCH 1/2] Documentation: clarify PR_SPEC_L1D_FLUSH

[Kees Cook]

On Wed, Oct 15, 2025 at 05:02:05PM +0000, Brendan Jackman wrote:
> For PR_SPEC_STORE_BYPASS and PR_SPEC_INDIRECT_BRANCH, PR_SPEC_DISABLE
> means "disable the speculation bug" i.e. "enable the mitigation".
> 
> For PR_SPEC_L1D_FLUSH, PR_SPEC_DISABLE means "disable the mitigation".
> This is not obvious, so document it.

The only thing I can find in Debian Code Search that actually uses
PR_SPEC_L1D_FLUSH is stress-ng, and it literally just toggles it before
restoring it:
https://sources.debian.org/src/stress-ng/0.19.05-1/stress-prctl.c?hl=893#L893

I wonder if we should just fix the prctl to match the existing
behaviors?

-Kees

> 
> Signed-off-by: Brendan Jackman <jackmanb@google.com>
> ---
>  Documentation/userspace-api/spec_ctrl.rst | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/userspace-api/spec_ctrl.rst b/Documentation/userspace-api/spec_ctrl.rst
> index 5e8ed9eef9aa84f12fa255af7b15c163ff01aab8..ca89151fc0a8e7205e0a0062134d63b213b9ef11 100644
> --- a/Documentation/userspace-api/spec_ctrl.rst
> +++ b/Documentation/userspace-api/spec_ctrl.rst
> @@ -26,7 +26,8 @@ PR_GET_SPECULATION_CTRL
>  
>  PR_GET_SPECULATION_CTRL returns the state of the speculation misfeature
>  which is selected with arg2 of prctl(2). The return value uses bits 0-3 with
> -the following meaning:
> +the following meaning (with the caveat that PR_SPEC_L1D_FLUSH has less obvious
> +semantics, see documentation for that specific control below):
>  
>  ==== ====================== ==================================================
>  Bit  Define                 Description
> @@ -110,6 +111,9 @@ Speculation misfeature controls
>  - PR_SPEC_L1D_FLUSH: Flush L1D Cache on context switch out of the task
>                          (works only when tasks run on non SMT cores)
>  
> +For this control, PR_SPEC_ENABLE means that the **mitigation** is enabled (L1D
> +is flushed), PR_SPEC_DISABLE means it is disabled.
> +
>    Invocations:
>     * prctl(PR_GET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, 0, 0, 0);
>     * prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, PR_SPEC_ENABLE, 0, 0);
> 
> -- 
> 2.50.1
> 

-- 
Kees Cook

Re: [PATCH 1/2] Documentation: clarify PR_SPEC_L1D_FLUSH

[Josh Poimboeuf]

On Wed, Oct 15, 2025 at 02:41:18PM -0700, Kees Cook wrote:
> On Wed, Oct 15, 2025 at 05:02:05PM +0000, Brendan Jackman wrote:
> > For PR_SPEC_STORE_BYPASS and PR_SPEC_INDIRECT_BRANCH, PR_SPEC_DISABLE
> > means "disable the speculation bug" i.e. "enable the mitigation".
> > 
> > For PR_SPEC_L1D_FLUSH, PR_SPEC_DISABLE means "disable the mitigation".
> > This is not obvious, so document it.
> 
> The only thing I can find in Debian Code Search that actually uses
> PR_SPEC_L1D_FLUSH is stress-ng, and it literally just toggles it before
> restoring it:
> https://sources.debian.org/src/stress-ng/0.19.05-1/stress-prctl.c?hl=893#L893
> 
> I wonder if we should just fix the prctl to match the existing
> behaviors?

This feature has existed for almost 5 years, I don't think we can just
reverse the functionality like that?  There could be proprietary users
out there (e.g., cloud companies).

-- 
Josh

Re: [PATCH 1/2] Documentation: clarify PR_SPEC_L1D_FLUSH

[Brendan Jackman]

On Wed Oct 15, 2025 at 11:42 PM UTC, Josh Poimboeuf wrote:
> On Wed, Oct 15, 2025 at 02:41:18PM -0700, Kees Cook wrote:
>> On Wed, Oct 15, 2025 at 05:02:05PM +0000, Brendan Jackman wrote:
>> > For PR_SPEC_STORE_BYPASS and PR_SPEC_INDIRECT_BRANCH, PR_SPEC_DISABLE
>> > means "disable the speculation bug" i.e. "enable the mitigation".
>> > 
>> > For PR_SPEC_L1D_FLUSH, PR_SPEC_DISABLE means "disable the mitigation".
>> > This is not obvious, so document it.
>> 
>> The only thing I can find in Debian Code Search that actually uses
>> PR_SPEC_L1D_FLUSH is stress-ng, and it literally just toggles it before
>> restoring it:
>> https://sources.debian.org/src/stress-ng/0.19.05-1/stress-prctl.c?hl=893#L893
>> 
>> I wonder if we should just fix the prctl to match the existing
>> behaviors?
>
> This feature has existed for almost 5 years, I don't think we can just
> reverse the functionality like that?  There could be proprietary users
> out there (e.g., cloud companies).

Yeah I'm with Josh on this one. I would guess these 3 situations are
about equally likely:

1. Nobody uses this flag for security.

2. People use it, and they are not getting the security properties they
   expect.

3. People do, but they know the meaning of the flag because either they
   read the kernel code (in my experience at Google it's pretty typical
   for security analysts to do this, they are smart and rigorous) or got
   suspicious performance numbers (I've also seen this at Google, IIRC
   this was how we found out about a bug in the Retbleed mitigations).
   Or, it could be the users are working on the same principle as the
   author/reviewers of the patch and they expected the current behaviour
   in the first place.

I think we have to privilege the third case here. Changing the behaviour
is gonna be extremely inconvenient for anyone depending on the current
one (if they even find out about it) so I think we'd have to introduce
a new variant of the API or something. That doesn't seem worth it, I
think the chance of anyone actually adopting the new API is pretty low.

[PATCH 2/2] Documentation: fix reference to PR_SPEC_L1D_FLUSH

[Brendan Jackman]

PR_SET_L1D_FLUSH does not exist.

Signed-off-by: Brendan Jackman <jackmanb@google.com>
---
 Documentation/admin-guide/hw-vuln/l1d_flush.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/hw-vuln/l1d_flush.rst b/Documentation/admin-guide/hw-vuln/l1d_flush.rst
index 210020bc3f5684a4510792aad5d0fcec659ad7d2..35dc25159b28a280793ad147d25d582f31a77999 100644
--- a/Documentation/admin-guide/hw-vuln/l1d_flush.rst
+++ b/Documentation/admin-guide/hw-vuln/l1d_flush.rst
@@ -31,7 +31,7 @@ specifically opt into the feature to enable it.
 Mitigation
 ----------
 
-When PR_SET_L1D_FLUSH is enabled for a task a flush of the L1D cache is
+When PR_SPEC_L1D_FLUSH is enabled for a task a flush of the L1D cache is
 performed when the task is scheduled out and the incoming task belongs to a
 different process and therefore to a different address space.
 

-- 
2.50.1

Re: [PATCH 0/2] Documentation: fixups for L1D flushing

[Brendan Jackman]

On Wed Oct 15, 2025 at 5:02 PM UTC, Brendan Jackman wrote:
> Signed-off-by: Brendan Jackman <jackmanb@google.com>
> ---
> Brendan Jackman (2):
>       Documentation: clarify PR_SPEC_L1D_FLUSH
>       Documentation: fix reference to PR_SPEC_L1D_FLUSH
>
>  Documentation/admin-guide/hw-vuln/l1d_flush.rst | 2 +-
>  Documentation/userspace-api/spec_ctrl.rst       | 6 +++++-
>  2 files changed, 6 insertions(+), 2 deletions(-)
> ---
> base-commit: 0292ef418ce08aad597fc0bba65b6dbb841808ba
> change-id: 20251015-l1d-flush-doc-029f64d2b0d3
>
> Best regards,

I just noticed another issue - the docs say you get -ENXIO if control
isn't possible, but for L1D_FLUSH and INDIR_BRANCH you get -EPERM.

TBH I think this is a bug but it seems like it's still better to just
document it than change the behaviour.

Re: [PATCH 0/2] Documentation: fixups for L1D flushing

[Brendan Jackman]

On Thu Oct 16, 2025 at 2:54 PM UTC, Brendan Jackman wrote:
> On Wed Oct 15, 2025 at 5:02 PM UTC, Brendan Jackman wrote:
>> Signed-off-by: Brendan Jackman <jackmanb@google.com>
>> ---
>> Brendan Jackman (2):
>>       Documentation: clarify PR_SPEC_L1D_FLUSH
>>       Documentation: fix reference to PR_SPEC_L1D_FLUSH
>>
>>  Documentation/admin-guide/hw-vuln/l1d_flush.rst | 2 +-
>>  Documentation/userspace-api/spec_ctrl.rst       | 6 +++++-
>>  2 files changed, 6 insertions(+), 2 deletions(-)
>> ---
>> base-commit: 0292ef418ce08aad597fc0bba65b6dbb841808ba
>> change-id: 20251015-l1d-flush-doc-029f64d2b0d3
>>
>> Best regards,
>
> I just noticed another issue - the docs say you get -ENXIO if control
> isn't possible, but for L1D_FLUSH and INDIR_BRANCH you get -EPERM.
>
> TBH I think this is a bug but it seems like it's still better to just
> document it than change the behaviour.

Since I didn't have to respin this series I sent a separate one for this
issue:

https://lore.kernel.org/all/DDJU0415JEBQ.H2SD942NMDWX@google.com/T/#t

Re: [PATCH 0/2] Documentation: fixups for L1D flushing

[Jonathan Corbet]

Brendan Jackman <jackmanb@google.com> writes:

> Signed-off-by: Brendan Jackman <jackmanb@google.com>
> ---
> Brendan Jackman (2):
>       Documentation: clarify PR_SPEC_L1D_FLUSH
>       Documentation: fix reference to PR_SPEC_L1D_FLUSH
>
>  Documentation/admin-guide/hw-vuln/l1d_flush.rst | 2 +-
>  Documentation/userspace-api/spec_ctrl.rst       | 6 +++++-
>  2 files changed, 6 insertions(+), 2 deletions(-)

I've gone ahead and applied these two, thanks.

jon