Re: [PATCH v23 12/22] cxl: Define a driver interface for HPA free space enumeration

["Cheatham, Benjamin" <benjamin.cheatham@amd.com>]

On 2/19/2026 3:58 AM, Alejandro Lucero Palau wrote:
> 
> On 2/11/26 22:10, Cheatham, Benjamin wrote:
>> On 2/1/2026 9:54 AM, alejandro.lucero-palau@amd.com wrote:
>>> From: Alejandro Lucero <alucerop@amd.com>
>>>
>>> CXL region creation involves allocating capacity from Device Physical
>>> Address (DPA) and assigning it to decode a given Host Physical Address
>>> (HPA). Before determining how much DPA to allocate the amount of available
>>> HPA must be determined. Also, not all HPA is created equal, some HPA
>>> targets RAM, some targets PMEM, some is prepared for device-memory flows
>>> like HDM-D and HDM-DB, and some is HDM-H (host-only).
>>>
>>> In order to support Type2 CXL devices, wrap all of those concerns into
>>> an API that retrieves a root decoder (platform CXL window) that fits the
>>> specified constraints and the capacity available for a new region.
>>>
>>> Add a complementary function for releasing the reference to such root
>>> decoder.
>>>
>>> Based on https://lore.kernel.org/linux-cxl/168592159290.1948938.13522227102445462976.stgit@dwillia2-xfh.jf.intel.com/
>>>
>>> Signed-off-by: Alejandro Lucero <alucerop@amd.com>
>>> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>>> ---
>>>   drivers/cxl/core/region.c | 164 ++++++++++++++++++++++++++++++++++++++
>>>   drivers/cxl/cxl.h         |   3 +
>>>   include/cxl/cxl.h         |   6 ++
>>>   3 files changed, 173 insertions(+)
>>>
>>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>> index 954b8fcdbac6..bdefd088f5f1 100644
>>> --- a/drivers/cxl/core/region.c
>>> +++ b/drivers/cxl/core/region.c
>>> @@ -705,6 +705,170 @@ static int free_hpa(struct cxl_region *cxlr)
>>>       return 0;
>>>   }
>>>   +struct cxlrd_max_context {
>>> +    struct device * const *host_bridges;
>>> +    int interleave_ways;
>>> +    unsigned long flags;
>>> +    resource_size_t max_hpa;
>>> +    struct cxl_root_decoder *cxlrd;
>>> +};
>>> +
>>> +static int find_max_hpa(struct device *dev, void *data)
>>> +{
>>> +    struct cxlrd_max_context *ctx = data;
>>> +    struct cxl_switch_decoder *cxlsd;
>>> +    struct cxl_root_decoder *cxlrd;
>>> +    struct resource *res, *prev;
>>> +    struct cxl_decoder *cxld;
>>> +    resource_size_t free = 0;
>>> +    resource_size_t max;
>>> +    int found = 0;
>>> +
>>> +    if (!is_root_decoder(dev))
>>> +        return 0;
>>> +
>>> +    cxlrd = to_cxl_root_decoder(dev);
>>> +    cxlsd = &cxlrd->cxlsd;
>>> +    cxld = &cxlsd->cxld;
>>> +
>>> +    if ((cxld->flags & ctx->flags) != ctx->flags) {
>>> +        dev_dbg(dev, "flags not matching: %08lx vs %08lx\n",
>>> +            cxld->flags, ctx->flags);
>>> +        return 0;
>>> +    }
>>> +
>>> +    for (int i = 0; i < ctx->interleave_ways; i++) {
>>> +        for (int j = 0; j < ctx->interleave_ways; j++) {
>>> +            if (ctx->host_bridges[i] == cxlsd->target[j]->dport_dev) {
>>> +                found++;
>>> +                break;
>>> +            }
>>> +        }
>>> +    }
>> This may be over complicated. I'm not quite sure how it works (I'm just slow today I guess), but I understand
>> what the intention is based on the debug print below. My issue is that ctx->host_bridges is only set to 1 host
>> bridge (endpoint->host_bridge) in cxl_get_hpa_freespace(), which is the only caller of this function. At that
>> point, why have the outer loop at all? At that point, you could also simplify ctx->host_bridges to only
>> be a struct device * const.
>>
>> Maybe this gets called elsewhere later on in the series? I haven't looked at the rest yet. If I'm wrong, then
>> I'd probably add a comment saying what the cxlsd->target[] entries are supposed to be pointing at.
> 
> 
> Hi Ben,
> 
> 
> I do remember this one.
> 
> 
> Dan's original patches had this support for interleaving, then I removed it as the case for Type2 and interleaving is quite unlikely, at least right now and likely in the near future. But I was told why do not support it as it was trivial to do so. FWIW, If I think only about the use case coming with the patchset, I agree with you, but because those previous discussions, I think I have to leave it.
> 

I'm fine with that, but I would at least do the fix with the decoder position in 19/22 and make a note that the
interleave_ways parameter in cxl_get_hpa_freespace() below is currently unused (unless I'm misunderstanding
the endpoint->host_bridge member).

That way, the support is mostly there and just requires a small, previously noted, addition to enable. If you're
fine with that then feel free to add my Reviewed-by after implementing in v24.

Thanks,
Ben

> 
> Thank you
> 
> 
>>> +
>>> +    if (found != ctx->interleave_ways) {
>>> +        dev_dbg(dev,
>>> +            "Not enough host bridges. Found %d for %d interleave ways requested\n",
>>> +            found, ctx->interleave_ways);
>>> +        return 0;
>>> +    }
>>> +
>>> +    /*
>>> +     * Walk the root decoder resource range relying on cxl_rwsem.region to
>>> +     * preclude sibling arrival/departure and find the largest free space
>>> +     * gap.
>>> +     */
>>> +    lockdep_assert_held_read(&cxl_rwsem.region);
>>> +    res = cxlrd->res->child;
>>> +
>>> +    /* With no resource child the whole parent resource is available */
>>> +    if (!res)
>>> +        max = resource_size(cxlrd->res);
>>> +    else
>>> +        max = 0;
>>> +
>>> +    for (prev = NULL; res; prev = res, res = res->sibling) {
>>> +        if (!prev && res->start == cxlrd->res->start &&
>>> +            res->end == cxlrd->res->end) {
>>> +            max = resource_size(cxlrd->res);
>>> +            break;
>>> +        }
>>> +        /*
>>> +         * Sanity check for preventing arithmetic problems below as a
>>> +         * resource with size 0 could imply using the end field below
>>> +         * when set to unsigned zero - 1 or all f in hex.
>>> +         */
>>> +        if (prev && !resource_size(prev))
>>> +            continue;
>>> +
>>> +        if (!prev && res->start > cxlrd->res->start) {
>>> +            free = res->start - cxlrd->res->start;
>>> +            max = max(free, max);
>>> +        }
>>> +        if (prev && res->start > prev->end + 1) {
>>> +            free = res->start - prev->end + 1;
>>> +            max = max(free, max);
>>> +        }
>>> +    }
>>> +
>>> +    if (prev && prev->end + 1 < cxlrd->res->end + 1) {
>>> +        free = cxlrd->res->end + 1 - prev->end + 1;
>>> +        max = max(free, max);
>>> +    }
>>> +
>>> +    dev_dbg(cxlrd_dev(cxlrd), "found %pa bytes of free space\n", &max);
>>> +    if (max > ctx->max_hpa) {
>>> +        if (ctx->cxlrd)
>>> +            put_device(cxlrd_dev(ctx->cxlrd));
>>> +        get_device(cxlrd_dev(cxlrd));
>>> +        ctx->cxlrd = cxlrd;
>>> +        ctx->max_hpa = max;
>>> +    }
>>> +    return 0;
>>> +}
>>> +
>>> +/**
>>> + * cxl_get_hpa_freespace - find a root decoder with free capacity per constraints
>>> + * @cxlmd: the mem device requiring the HPA
>>> + * @interleave_ways: number of entries in @host_bridges
>>> + * @flags: CXL_DECODER_F flags for selecting RAM vs PMEM, and Type2 device
>>> + * @max_avail_contig: output parameter of max contiguous bytes available in the
>>> + *              returned decoder
>>> + *
>>> + * Returns a pointer to a struct cxl_root_decoder
>>> + *
>>> + * The return tuple of a 'struct cxl_root_decoder' and 'bytes available given
>>> + * in (@max_avail_contig))' is a point in time snapshot. If by the time the
>>> + * caller goes to use this decoder and its capacity is reduced then caller needs
>>> + * to loop and retry.
>>> + *
>>> + * The returned root decoder has an elevated reference count that needs to be
>>> + * put with cxl_put_root_decoder(cxlrd).
>>> + */
>>> +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_memdev *cxlmd,
>>> +                           int interleave_ways,
>>> +                           unsigned long flags,
>>> +                           resource_size_t *max_avail_contig)
>>> +{
>>> +    struct cxlrd_max_context ctx = {
>>> +        .flags = flags,
>>> +        .interleave_ways = interleave_ways,
>>> +    };
>>> +    struct cxl_port *root_port;
>>> +    struct cxl_port *endpoint;
>>> +
>>> +    endpoint = cxlmd->endpoint;
>>> +    if (!endpoint) {
>>> +        dev_dbg(&cxlmd->dev, "endpoint not linked to memdev\n");
>>> +        return ERR_PTR(-ENXIO);
>>> +    }
>>> +
>>> +    ctx.host_bridges = &endpoint->host_bridge;
>> Mentioned earlier, interleave_ways is effectively hardcoded to 1 (unless I'm misunderstanding
>> something). I think what you want here is to go to the CXL root and pass in the children (i.e. host bridges)?
>> I'm not sure of what the fix is to get the intended behavior.
>>
>> It may be worth getting rid of the interleave_ways portion of this function and
>> add it later when someone needs it. You could also explain it's hard coded to 1/unused
>> in the doc comment if you know of an immediate need for it.
>>
>>> +
>>> +    struct cxl_root *root __free(put_cxl_root) = find_cxl_root(endpoint);
>>> +    if (!root) {
>>> +        dev_dbg(&endpoint->dev, "endpoint is not related to a root port\n");
>>> +        return ERR_PTR(-ENXIO);
>>> +    }
>>> +
>>> +    root_port = &root->port;
>>> +    scoped_guard(rwsem_read, &cxl_rwsem.region)
>>> +        device_for_each_child(&root_port->dev, &ctx, find_max_hpa);
>> Can just use a guard() here.
>>
>>> +
>>> +    if (!ctx.cxlrd)
>>> +        return ERR_PTR(-ENOMEM);
>>> +
>>> +    *max_avail_contig = ctx.max_hpa;
>>> +    return ctx.cxlrd;
>>> +}
>>> +EXPORT_SYMBOL_NS_GPL(cxl_get_hpa_freespace, "CXL");
>>> +
>>> +void cxl_put_root_decoder(struct cxl_root_decoder *cxlrd)
>>> +{
>>> +    put_device(cxlrd_dev(cxlrd));
>>> +}
>>> +EXPORT_SYMBOL_NS_GPL(cxl_put_root_decoder, "CXL");
>>> +
>>>   static ssize_t size_store(struct device *dev, struct device_attribute *attr,
>>>                 const char *buf, size_t len)
>>>   {
>>> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
>>> index 944c5d1ccceb..c7d9b2c2908f 100644
>>> --- a/drivers/cxl/cxl.h
>>> +++ b/drivers/cxl/cxl.h
>>> @@ -706,6 +706,9 @@ struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev);
>>>   struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev);
>>>   struct cxl_endpoint_decoder *to_cxl_endpoint_decoder(struct device *dev);
>>>   bool is_root_decoder(struct device *dev);
>>> +
>>> +#define cxlrd_dev(cxlrd) (&(cxlrd)->cxlsd.cxld.dev)
>>> +
>>>   bool is_switch_decoder(struct device *dev);
>>>   bool is_endpoint_decoder(struct device *dev);
>>>   struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port,
>>> diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
>>> index 92880c26b2d5..834dc7e78934 100644
>>> --- a/include/cxl/cxl.h
>>> +++ b/include/cxl/cxl.h
>>> @@ -255,4 +255,10 @@ struct cxl_endpoint_decoder *cxl_get_committed_decoder(struct cxl_memdev *cxlmd,
>>>   struct range;
>>>   int cxl_get_region_range(struct cxl_region *region, struct range *range);
>>>   void cxl_unregister_region(struct cxl_region *cxlr);
>>> +struct cxl_port;
>>> +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_memdev *cxlmd,
>>> +                           int interleave_ways,
>>> +                           unsigned long flags,
>>> +                           resource_size_t *max);
>>> +void cxl_put_root_decoder(struct cxl_root_decoder *cxlrd);
>>>   #endif /* __CXL_CXL_H__ */