Re: RFC: CXL Isolation Support

["Cheatham, Benjamin" <benjamin.cheatham@amd.com>]

On 2/2/2026 10:50 AM, Gregory Price wrote:
> On Mon, Feb 02, 2026 at 03:59:05PM +0000, Jonathan Cameron wrote:
>> On Fri, 30 Jan 2026 16:30:51 -0500
>> Gregory Price <gourry@gourry.net> wrote:
>>
>>> But even then, you have bigger issues for shared file-backed VMAs.
>>>
>>> e.g. imagine libc gets demoted and comes back poisoned: kabloooey
>>>
>>> But, you can at least get data from the system that the link went
>>> down and even have a chance to investigate before nicely blowing up.
>>>
>>> Which is at least much more helpful.
>>
>> Let's pretend it's tagged (just for ease of thinking about it).
>> So shareable that happens not be shared ever.
>>
>> Application specific memory isolated to one application - using
>> famfs or similar.
>>
>> Then it's safe enough, but maybe not that useful.  It's a possible
>> path to get to a world in which type 3 mem an be isolated.
>>
> 
> Absolutely.  I agree any kind of explicit-use isolation is feasible to
> implement large-scale recoverability.
> 
> But I suppose to clarify my concerns - I think isolation guarantees
> require much more clarity on how involved BIOS can be.
> 
> Auto-regions online memory as nodes by default.  If CXL memory is online
> as a node (in the current kernel) - then isolation is broken, even in
> ZONE_MOVABLE.  No amount of desire for recoverability is feasible.
> 
> This obviously changes if the exposure is limited via some explicit
> mechanism (FAMFS, N_MEMORY_PRIVATE, etc).  But this is already true
> of those paths - users of FAMFS will get SIGBUS'd instead of MCE'd on
> poison, for example.

I should add that, at least on AMD platforms, if the CXL link goes down
the system will immediately reset (AFAIK). So this use case would require
enabling isolation just so the hardware doesn't rug-pull you.
> 
> So if isolation is desired, then the default opinion should be that all
> management of the CXL bus (endpoints, decoders, etc) should be deferred
> to the driver and not programmed by the BIOS.

I think we're slowly moving this way, but it isn't feasible for current
AMD platforms. However, I don't think this is too much of an issue for
our platforms since recovery isn't supported anyway.
> 
> Auto-regions are basically incompatible with this feature.
>    (for more useless information - see: $REASONS)
> 
> ~Gregory
> 
> --- $REASONS
> 
>    This is partially informed by the fact that auto-regions are defined
>    as BIOS-programmed decoders - which the current driver auto-plugs
>    into the dax_kmem driver, and we're stuck with that unfortunate
>    backwards compatibility story for quite some time.
> 
>    And the platform which use auto-regions may not adhere to expected
>    programming patterns due to some subtle deviations from the spec.
> 
>    So tearing complexes down and reprogramming them may not even be
>    feasible.  ( cough Zen5 :[ )
> 
>    But more generally, proposing additional features for auto-regions
>    creates an incentive to push ever-increasingly-complex policy down
>    into the BIOS, which will just lead to sadness and heartache.

I agree :)

Thanks,
Ben