From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39A1174418; Fri, 13 Sep 2024 17:52:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726249959; cv=none; b=leJVbPkJmpK9FDPVrhKqk6SZnQKJSsJG2jlEtM9RuDNdlie0tvCO89YTvF9cQSqxytTdQfgJskwPsghqxcj/mnseS4ffW4gNbBMe9uU+WhzWxYoeKGEYHvdaTkG5BSY7O57m0aVBOSfgyNERUJnaTxSSh8/QXM8LW+M357VK1Ls= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726249959; c=relaxed/simple; bh=2bUBFh7Buj7Z7hMFpP2B8beee3EQ7zqHKLTobnQ01qM=; h=Date:From:To:CC:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=T/wk6Q+n+todOAmrk4YdOj08bbcwWuzZ+Ntil26PWatpjeYl6sAeALph6I20M8XkbpTGHbgww2U8z/HUP1zuW81HbVNsL04017AF6Y5En6HJfTwIDWIaN4Dl4xY0QdI/Kx2uw7DJRgdPAalbePEcaVgXMCUQoLJfQWhucTifh4s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=Huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=Huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.216]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X51yG2DbXz6L6s2; Sat, 14 Sep 2024 01:48:54 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id A6F9B140A77; Sat, 14 Sep 2024 01:52:34 +0800 (CST) Received: from localhost (10.48.150.243) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 13 Sep 2024 19:52:33 +0200 Date: Fri, 13 Sep 2024 18:52:32 +0100 From: Jonathan Cameron To: CC: , , , , , , , , , Alejandro Lucero Subject: Re: [PATCH v3 11/20] cxl: define a driver interface for HPA free space enumaration Message-ID: <20240913185232.000000c9@Huawei.com> In-Reply-To: <20240907081836.5801-12-alejandro.lucero-palau@amd.com> References: <20240907081836.5801-1-alejandro.lucero-palau@amd.com> <20240907081836.5801-12-alejandro.lucero-palau@amd.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: lhrpeml100006.china.huawei.com (7.191.160.224) To frapeml500008.china.huawei.com (7.182.85.71) On Sat, 7 Sep 2024 09:18:27 +0100 alejandro.lucero-palau@amd.com wrote: > From: Alejandro Lucero > > CXL region creation involves allocating capacity from device DPA > (device-physical-address space) and assigning it to decode a given HPA > (host-physical-address space). Before determining how much DPA to > allocate the amount of available HPA must be determined. Also, not all > HPA is create equal, some specifically targets RAM, some target PMEM, > some is prepared for device-memory flows like HDM-D and HDM-DB, and some > is host-only (HDM-H). > > Wrap all of those concerns into an API that retrieves a root decoder > (platform CXL window) that fits the specified constraints and the > capacity available for a new region. > > Based on https://lore.kernel.org/linux-cxl/168592159290.1948938.13522227102445462976.stgit@dwillia2-xfh.jf.intel.com/ > > Signed-off-by: Alejandro Lucero > Co-developed-by: Dan Williams Trivial comment inline. J > --- > drivers/cxl/core/region.c | 141 ++++++++++++++++++++++++++++++++++++++ > drivers/cxl/cxl.h | 3 + > drivers/cxl/cxlmem.h | 3 + > include/linux/cxl/cxl.h | 8 +++ > 4 files changed, 155 insertions(+) > > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c > index 21ad5f242875..bb227bf894c4 100644 > --- a/drivers/cxl/core/region.c > +++ b/drivers/cxl/core/region.c > @@ -703,6 +703,147 @@ static int free_hpa(struct cxl_region *cxlr) > return 0; > } > > +struct cxlrd_max_context { > + struct device *host_bridge; > + unsigned long flags; > + resource_size_t max_hpa; > + struct cxl_root_decoder *cxlrd; > +}; > + > +static int find_max_hpa(struct device *dev, void *data) > +{ > + struct cxlrd_max_context *ctx = data; > + struct cxl_switch_decoder *cxlsd; > + struct cxl_root_decoder *cxlrd; > + struct resource *res, *prev; > + struct cxl_decoder *cxld; > + resource_size_t max; > + > + if (!is_root_decoder(dev)) > + return 0; > + > + cxlrd = to_cxl_root_decoder(dev); > + cxld = &cxlrd->cxlsd.cxld; > + if ((cxld->flags & ctx->flags) != ctx->flags) { > + dev_dbg(dev, "%s, flags not matching: %08lx vs %08lx\n", > + __func__, cxld->flags, ctx->flags); > + return 0; > + } > + > + /* An accelerator can not be part of an interleaved HPA range. */ > + if (cxld->interleave_ways != 1) { > + dev_dbg(dev, "%s, interleave_ways not matching\n", __func__); > + return 0; > + } > + > + cxlsd = &cxlrd->cxlsd; Perhaps move this before the cxld = and use it there as well? > + > + guard(rwsem_read)(&cxl_region_rwsem); > + if (ctx->host_bridge != cxlsd->target[0]->dport_dev) { > + dev_dbg(dev, "%s, HOST BRIDGE DOES NOT MATCH\n", __func__); Capitals seem a bit ott. > + return 0; > + } > + > + /* > + * Walk the root decoder resource range relying on cxl_region_rwsem to > + * preclude sibling arrival/departure and find the largest free space > + * gap. > + */ > + lockdep_assert_held_read(&cxl_region_rwsem); > + max = 0; > + res = cxlrd->res->child; > + if (!res) > + max = resource_size(cxlrd->res); > + else > + max = 0; > + > + for (prev = NULL; res; prev = res, res = res->sibling) { > + struct resource *next = res->sibling; > + resource_size_t free = 0; > + > + if (!prev && res->start > cxlrd->res->start) { > + free = res->start - cxlrd->res->start; > + max = max(free, max); > + } > + if (prev && res->start > prev->end + 1) { > + free = res->start - prev->end + 1; > + max = max(free, max); > + } > + if (next && res->end + 1 < next->start) { > + free = next->start - res->end + 1; > + max = max(free, max); > + } > + if (!next && res->end + 1 < cxlrd->res->end + 1) { > + free = cxlrd->res->end + 1 - res->end + 1; > + max = max(free, max); > + } > + } > + > + dev_dbg(CXLRD_DEV(cxlrd), "%s, found %pa bytes of free space\n", > + __func__, &max); > + if (max > ctx->max_hpa) { > + if (ctx->cxlrd) > + put_device(CXLRD_DEV(ctx->cxlrd)); > + get_device(CXLRD_DEV(cxlrd)); > + ctx->cxlrd = cxlrd; > + ctx->max_hpa = max; > + dev_dbg(CXLRD_DEV(cxlrd), "%s, found %pa bytes of free space\n", > + __func__, &max); > + } > + return 0; > +}