Re: [RFC PATCH 00/56] Dynamic mitigations

Pawan Gupta
2025-10-24T05:01:19Z

On Mon, Oct 13, 2025 at 09:33:48AM -0500, David Kaplan wrote:
> Dynamic mitigations enables changing the kernel CPU security mitigations at
> runtime without a reboot/kexec.
>
> Previously, mitigation choices had to be made on the kernel cmdline. With
> this feature an administrator can select new mitigation choices by writing
> a sysfs file, after which the kernel will re-patch itself based on the new
> mitigations.
>
> As the performance cost of CPU mitigations can be significant, selecting
> the right set of mitigations is important to achieve the correct balance of
> performance/security.
>
> Use
> ---
> As described in the supplied documentation file, new mitigations are
> selected by writing cmdline options to a new sysfs file. Only cmdline
> options related to mitigations are recognized via this interface. All
> previous mitigation-related cmdline options are ignored and selections are
> done based on the new options.
>
> Examples:
> echo "mitigations=off" > /sys/devices/system/cpu/mitigations
> echo "spectre_v2=retpoline tsa=off" > /sys/devices/system/cpu/mitigations
>
>
> There are several use cases that will benefit from dynamic mitigations:
>
> Use Cases
> ---------
> 1. Runtime Policy
>
> Some workflows rely on booting a generic kernel before customizing the system.
> cloud-init is a popular example of this where a VM is started typically with
> default settings and then is customized based on a customer-provided
> configuration file.
>
> As flows like this rely on configuring the system after boot, they currently
> cannot customize the mitigation policy. With dynamic mitigations, this
> configuration information can be augmented to include security policy
> information.
>
> For example, a cloud VM which runs only trusted workloads likely does not
> need any CPU security mitigations applied. But as this policy information
> is not known at boot time, the kernel will be booted with unnecessary
> mitigations enabled. With dynamic mitigations, these mitigations can be
> disabled during boot after policy information is retrieved, improving
> performance.
>
> 2. Mitigation Changes
>
> Sometimes there are needs to change the mitigation settings in light of new
> security findings. For example, AMD-SB-1036 advised of a security issue
> with a spectre v2 mitigation and advised using a different one instead.
>
> With dynamic mitigations, such changes can be made without a reboot/kexec
> which minimizes disruption in environments which cannot easily tolerate
> such an event.
>
> 3. Mitigation Testing
>
> Being able to quickly change between different mitigation settings without
> having to restart applications is beneficial when conducting mitigation
> development and testing.
>
> Note that some bugs have multiple mitigation options, which may have
> varying performance impacts. Being able to quickly switch between them
> makes evaluating such options easier.
>
>
> Implementation Details
> ----------------------
> Re-patching the kernel is expected to be a very rare operation and is done
> under very big hammers. All tasks are put into the freezer and the
> re-patching is then done under the (new) stop_machine_nmi() routine.
>
> To re-patch the kernel, it is first reverted back to its compile-time
> state. The original bytes from alternatives, retpolines, etc. are saved
> during boot so they can later be used to restore the original kernel image.
> After that, the kernel is patched based on the new feature flags.
>
> This simplifies the re-patch process as restoring the original kernel image
> is relatively straightforward. In other words, instead of having to
> re-patch from mitigation A to mitigation B directly, we first restore the
> original image and then patch from that to mitigation B, similar to if the
> system had booted with mitigation B selected originally.
>
>
> Performance
> -----------
> Testing so far has demonstrated that re-patching takes ~50ms on an AMD EPYC
> 7713 running a typical Ubuntu kernel with around 100 modules loaded.
>
> Guide to Patch Series
> ---------------------
> As this series is rather lengthy, this may help with understanding it:
>
> Patches 3-18 focus on "resetting" mitigations. Every bug that may set feature
> flags, MSRs, static branches, etc. now has matching "reset" functions that will
> undo all these changes. This is used at the beginning of the re-patch flow.
>
> Patches 20-22 move various functions and values out of the .init section. Most
> of the existing mitigation logic was marked as __init and the mitigation
> settings as __ro_after_init but now these can be changed at runtime. The
> __ro_after_init marking functioned as a defense-in-depth measure but is arguably
> of limited meaningful security value as an attacker who can modify kernel data
> can do a lot worse than change some speculation settings. As re-patching
> requires being able to modify these settings, it was simplest to remove them
> from that section.
>
> Patches 23-27 involve linker and related modifications to keep alternative
> information around at runtime instead of free'ing it after boot. This does
> result in slightly higher runtime memory consumption which is one reason why
> this feature is behind a Kconfig option. On a typical kernel, this was measured
> at around 2MB of extra kernel memory usage.
>
> Patches 28-30 focus on the new stop_machine_nmi() which behaves like
> stop_machine() but runs the handler in NMI context, thus ensuring that even NMIs
> cannot interrupt the handler. As dynamic mitigations involves re-patching
> functions used by NMI entry code, this is required for safety.
>
> Patches 31-40 focus on support for restoring the kernel text at runtime. This
> involves saving the original kernel bytes when patched the first time and adding
> support to then restore those later.
>
> Patches 41-44 start building support for updating code, in particular module
> code at runtime.
>
> Patches 45-47 focus on support for the Indirect Target Selection mitigation
> which is particularly challenging because it requires runtime memory allocations
> and permission changes which are not possible in NMI context. As a result, ITS
> memory is pre-allocated before entering NMI context.
>
> Patch 50 adds the complete function for resetting and re-patching the kernel.
>
> Patches 51-53 build the sysfs interface for re-patching and support for parsing
> the new options provided.
>
> Patches 54-56 add debugfs interfaces to values which are important for
> mitigations. These are useful for userspace test utilities to be able to force
> a CPU to appear to be vulnerable or immune to certain bugs as well as being able
> to help verify if the kernel is correctly mitigating various vulnerabilities.

Although it adds some complexity, this adds a very useful feature. Thanks
for doing this series.

Just curious, for patching indirect branches, was replacing alternatives
with static_calls considered? I haven't looked at the feasibility, but
static_calls seems to be more suited for post-boot patching.

Thinking out loud, patching in something similar to suspend-to-RAM flow may
reduce some corner cases. Barring the BSP, the APs gets reinitialized in
that case.