[PATCH 0/3] cxl: Preparation of type2 accelerators support

[PATCH 0/3] cxl: Preparation of type2 accelerators support

[Huang Ying]

There have been 2 series to add type 2 accelerators support in [1] and [2].

[1] https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/
[2] https://lore.kernel.org/linux-cxl/20240516081202.27023-1-alucerop@amd.com/

Both provide relative complete support, but both are long too.  To
make it easier to review, some preparation of type2 accelerators
support is implemented in this series.  More complete support can be
implemented based on it.

This series has been tested with cxl_test via mocking a type2
accelerator as in [1] above.  Because more type2 accelerators support
than that provided by this series is needed to simulate the device,
the cxl_test patch isn't included in the series.

Huang Ying (3):
  cxl: Set target type of root decoder based on CFMWS restrictions
  cxl: Set target type of region with that of root decoder
  cxl: Avoid to create dax regions for type2 accelerators

 drivers/cxl/acpi.c        |  5 ++++-
 drivers/cxl/core/region.c | 11 ++++++++++-
 2 files changed, 14 insertions(+), 2 deletions(-)

Best Regards,
Huang, Ying

[PATCH 1/3] cxl: Set target type of root decoder based on CFMWS restrictions

[Huang Ying]

Now, the target type of root decoder is hard-coded to HOSTONLYMEM,
because only type3 expanders are supported.  To support type2
accelerators, set the target type of root decoder based on the
window restrictions field of CFMWS entry.

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Suggested-by: Dan Williams <dan.j.williams@intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alejandro Lucero <alucerop@amd.com>
---
 drivers/cxl/acpi.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
index 82b78e331d8e..40c92ad29122 100644
--- a/drivers/cxl/acpi.c
+++ b/drivers/cxl/acpi.c
@@ -382,7 +382,10 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,
 
 	cxld = &cxlrd->cxlsd.cxld;
 	cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions);
-	cxld->target_type = CXL_DECODER_HOSTONLYMEM;
+	if (cxld->flags & CXL_DECODER_F_TYPE2)
+		cxld->target_type = CXL_DECODER_DEVMEM;
+	else
+		cxld->target_type = CXL_DECODER_HOSTONLYMEM;
 	cxld->hpa_range = (struct range) {
 		.start = cfmws->base_hpa,
 		.end = cfmws->base_hpa + cfmws->window_size - 1,
-- 
2.39.2

[PATCH 2/3] cxl: Set target type of region with that of root decoder

[Huang Ying]

Now, the target type of region is hard-coded to HOSTONLYMEM, because
only type3 expanders are supported.  To support type2 accelerators,
set the target type of region root decoder with that of the root
decoder.

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Suggested-by: Dan Williams <dan.j.williams@intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alejandro Lucero <alucerop@amd.com>
---
 drivers/cxl/core/region.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 21ad5f242875..9a483c8a32fd 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -2545,7 +2545,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
 		return ERR_PTR(-EBUSY);
 	}
 
-	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
+	return devm_cxl_add_region(cxlrd, id, mode,
+				   cxlrd->cxlsd.cxld.target_type);
 }
 
 static ssize_t create_pmem_region_store(struct device *dev,
-- 
2.39.2

[PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Huang Ying]

The memory range of a type2 accelerator should be managed by the type2
accelerator specific driver instead of the common dax region drivers,
as discussed in [1].

[1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/

So, in this patch, we skip dax regions creation for type2 accelerator
device memory regions.

Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Co-developed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alejandro Lucero <alucerop@amd.com>
---
 drivers/cxl/core/region.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 9a483c8a32fd..b37e12bb4a35 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
 					p->res->start, p->res->end, cxlr,
 					is_system_ram) > 0)
 			return 0;
+		/*
+		 * HDM-D[B] (device-memory) regions have accelerator
+		 * specific usage, skip device-dax registration.
+		 */
+		if (cxlr->type == CXL_DECODER_DEVMEM)
+			return 0;
+
+		/* HDM-H routes to device-dax */
 		return devm_cxl_add_dax_region(cxlr);
 	default:
 		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",
-- 
2.39.2

Re: [PATCH 0/3] cxl: Preparation of type2 accelerators support

[Alejandro Lucero Palau]

Hi,


I'm a bit "surprised" by this patchset. As you rightly say in this cover 
letter, there is a patchset under review, and I have not seen you 
commenting there on the concerns stated in this cover letter.


I would say that is the first thing you should do, at least, to comment 
there, suggest to do other way, pointing to other needed changes, and if 
things do not go well for reaching an agreement, then a patchset like 
this could make sense exposing another way.


Anyway, I think a CXL maintainer should say something about it, but 
after 24hours, I had to say something.


About testing these changes, I wonder how did you proceed. If you have 
used an emulated Type2 device, as I did with the first patchset version, 
you should trigger some of the problems I found, what makes any Type2 
initialization for getting a memdev and finally a cxl region to fail. In 
other words, testing these changes can not be a partly initialised Type2 
but a complete one. This does not mean a patch fixing a known and 
obvious issue for supporting Type2 should not be approved if not tested, 
since the Type2 support is not yet there, but mentioning testing makes 
things confusing, at least for me.


On 7/29/24 09:46, Huang Ying wrote:
> There have been 2 series to add type 2 accelerators support in [1] and [2].
>
> [1] https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/
> [2] https://lore.kernel.org/linux-cxl/20240516081202.27023-1-alucerop@amd.com/
>
> Both provide relative complete support, but both are long too.  To
> make it easier to review, some preparation of type2 accelerators
> support is implemented in this series.  More complete support can be
> implemented based on it.
>
> This series has been tested with cxl_test via mocking a type2
> accelerator as in [1] above.  Because more type2 accelerators support
> than that provided by this series is needed to simulate the device,
> the cxl_test patch isn't included in the series.
>
> Huang Ying (3):
>    cxl: Set target type of root decoder based on CFMWS restrictions
>    cxl: Set target type of region with that of root decoder
>    cxl: Avoid to create dax regions for type2 accelerators
>
>   drivers/cxl/acpi.c        |  5 ++++-
>   drivers/cxl/core/region.c | 11 ++++++++++-
>   2 files changed, 14 insertions(+), 2 deletions(-)
>
> Best Regards,
> Huang, Ying

Re: [PATCH 0/3] cxl: Preparation of type2 accelerators support

[Huang, Ying]

Hi, Alejandro,

Alejandro Lucero Palau <alucerop@amd.com> writes:

> Hi,
>
>
> I'm a bit "surprised" by this patchset. As you rightly say in this
> cover letter, there is a patchset under review, and I have not seen
> you commenting there on the concerns stated in this cover letter.
>
>
> I would say that is the first thing you should do, at least, to
> comment there, suggest to do other way, pointing to other needed
> changes, and if things do not go well for reaching an agreement, then
> a patchset like this could make sense exposing another way.

Sorry for confusing.  The patchset is just some preparation work for
type2 accelerator support.  Your patchset is more complete.  I just want
to relief your overhead a bit.

> Anyway, I think a CXL maintainer should say something about it, but
> after 24hours, I had to say something.
>
>
> About testing these changes, I wonder how did you proceed. If you have
> used an emulated Type2 device, as I did with the first patchset
> version, you should trigger some of the problems I found, what makes
> any Type2 initialization for getting a memdev and finally a cxl region
> to fail. In other words, testing these changes can not be a partly
> initialised Type2 but a complete one. This does not mean a patch
> fixing a known and obvious issue for supporting Type2 should not be
> approved if not tested, since the Type2 support is not yet there, but
> mentioning testing makes things confusing, at least for me.

Yes.  To test this patchset, I need more complete type2 support.  I used
some patches from your and Dan's patchset on top of this patchset to do
the test.

>
> On 7/29/24 09:46, Huang Ying wrote:
>> There have been 2 series to add type 2 accelerators support in [1] and [2].
>>
>> [1] https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/
>> [2] https://lore.kernel.org/linux-cxl/20240516081202.27023-1-alucerop@amd.com/
>>
>> Both provide relative complete support, but both are long too.  To
>> make it easier to review, some preparation of type2 accelerators
>> support is implemented in this series.  More complete support can be
>> implemented based on it.
>>
>> This series has been tested with cxl_test via mocking a type2
>> accelerator as in [1] above.  Because more type2 accelerators support
>> than that provided by this series is needed to simulate the device,
>> the cxl_test patch isn't included in the series.
>>
>> Huang Ying (3):
>>    cxl: Set target type of root decoder based on CFMWS restrictions
>>    cxl: Set target type of region with that of root decoder
>>    cxl: Avoid to create dax regions for type2 accelerators
>>
>>   drivers/cxl/acpi.c        |  5 ++++-
>>   drivers/cxl/core/region.c | 11 ++++++++++-
>>   2 files changed, 14 insertions(+), 2 deletions(-)

--
Best Regards,
Huang, Ying

Re: [PATCH 1/3] cxl: Set target type of root decoder based on CFMWS restrictions

[Alison Schofield]

On Mon, Jul 29, 2024 at 04:46:09PM +0800, Ying Huang wrote:
> Now, the target type of root decoder is hard-coded to HOSTONLYMEM,
> because only type3 expanders are supported.  To support type2
> accelerators, set the target type of root decoder based on the
> window restrictions field of CFMWS entry.
> 

Reviewed-by: Alison Schofield <alison.schofield@intel.com>

> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---
>  drivers/cxl/acpi.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index 82b78e331d8e..40c92ad29122 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -382,7 +382,10 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,
>  
>  	cxld = &cxlrd->cxlsd.cxld;
>  	cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions);
> -	cxld->target_type = CXL_DECODER_HOSTONLYMEM;
> +	if (cxld->flags & CXL_DECODER_F_TYPE2)
> +		cxld->target_type = CXL_DECODER_DEVMEM;
> +	else
> +		cxld->target_type = CXL_DECODER_HOSTONLYMEM;
>  	cxld->hpa_range = (struct range) {
>  		.start = cfmws->base_hpa,
>  		.end = cfmws->base_hpa + cfmws->window_size - 1,
> -- 
> 2.39.2
> 

Re: [PATCH 2/3] cxl: Set target type of region with that of root decoder

[Alison Schofield]

On Mon, Jul 29, 2024 at 04:46:10PM +0800, Ying Huang wrote:
> Now, the target type of region is hard-coded to HOSTONLYMEM, because
> only type3 expanders are supported.  To support type2 accelerators,
> set the target type of region root decoder with that of the root
> decoder.

Hi Ying,

If the target type of a region is always the same as it's root decoder,
(is it?) why do we store it as an attribute of the region. Can we look
it up when needed?

A bit more below -

> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---
>  drivers/cxl/core/region.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 21ad5f242875..9a483c8a32fd 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -2545,7 +2545,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
>  		return ERR_PTR(-EBUSY);
>  	}
>  
> -	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
> +	return devm_cxl_add_region(cxlrd, id, mode,
> +				   cxlrd->cxlsd.cxld.target_type);
>  }

Passing the 'cxlrd' and then a piece of the cxlrd (.target_type) looks
redundant.


-- Alison

>  
>  static ssize_t create_pmem_region_store(struct device *dev,
> -- 
> 2.39.2
> 

Re: [PATCH 2/3] cxl: Set target type of region with that of root decoder

[Huang, Ying]

Alison Schofield <alison.schofield@intel.com> writes:

> On Mon, Jul 29, 2024 at 04:46:10PM +0800, Ying Huang wrote:
>> Now, the target type of region is hard-coded to HOSTONLYMEM, because
>> only type3 expanders are supported.  To support type2 accelerators,
>> set the target type of region root decoder with that of the root
>> decoder.
>
> Hi Ying,
>
> If the target type of a region is always the same as it's root decoder,
> (is it?)

IIUC, it is.  Do you know when they may be different?

> why do we store it as an attribute of the region. Can we look
> it up when needed?

Yes.  This is possible via to_cxl_root_decoder().  It's just
a little inconvenient.

> A bit more below -
>
>> 
>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>> Suggested-by: Dan Williams <dan.j.williams@intel.com>
>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>> Cc: Dave Jiang <dave.jiang@intel.com>
>> Cc: Alison Schofield <alison.schofield@intel.com>
>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>> Cc: Ira Weiny <ira.weiny@intel.com>
>> Cc: Alejandro Lucero <alucerop@amd.com>
>> ---
>>  drivers/cxl/core/region.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>> index 21ad5f242875..9a483c8a32fd 100644
>> --- a/drivers/cxl/core/region.c
>> +++ b/drivers/cxl/core/region.c
>> @@ -2545,7 +2545,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
>>  		return ERR_PTR(-EBUSY);
>>  	}
>>  
>> -	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
>> +	return devm_cxl_add_region(cxlrd, id, mode,
>> +				   cxlrd->cxlsd.cxld.target_type);
>>  }
>
> Passing the 'cxlrd' and then a piece of the cxlrd (.target_type) looks
> redundant.

Yes.  We can remove the parameter.  Will change this if we still need
cxlr->type.  Thanks!

--
Best Regards,
Huang, Ying

>
> -- Alison
>
>>  
>>  static ssize_t create_pmem_region_store(struct device *dev,
>> -- 
>> 2.39.2
>> 

Re: [PATCH 1/3] cxl: Set target type of root decoder based on CFMWS restrictions

[Jonathan Cameron]

On Mon, 29 Jul 2024 16:46:09 +0800
Huang Ying <ying.huang@intel.com> wrote:

> Now, the target type of root decoder is hard-coded to HOSTONLYMEM,
> because only type3 expanders are supported.  To support type2
> accelerators, set the target type of root decoder based on the
> window restrictions field of CFMWS entry.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---
>  drivers/cxl/acpi.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index 82b78e331d8e..40c92ad29122 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -382,7 +382,10 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,
>  
>  	cxld = &cxlrd->cxlsd.cxld;
>  	cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions);
> -	cxld->target_type = CXL_DECODER_HOSTONLYMEM;
> +	if (cxld->flags & CXL_DECODER_F_TYPE2)

These flags need updating or we are going to run into problems
long term.

As of more recent specs, the distinction is messier than it was and
it's device coherent HDM-D / HDM-DB (second one being type2 or type3 with
BI support) and/or Host only coherent HDM-H.

I'm curious on whether anyone is support both on same CFWMS?
I believe it is possible and the spec doesn't rule it out.

Jonathan


> +		cxld->target_type = CXL_DECODER_DEVMEM;
> +	else
> +		cxld->target_type = CXL_DECODER_HOSTONLYMEM;
>  	cxld->hpa_range = (struct range) {
>  		.start = cfmws->base_hpa,
>  		.end = cfmws->base_hpa + cfmws->window_size - 1,

Re: [PATCH 2/3] cxl: Set target type of region with that of root decoder

[Jonathan Cameron]

On Thu, 01 Aug 2024 14:28:55 +0800
"Huang, Ying" <ying.huang@intel.com> wrote:

> Alison Schofield <alison.schofield@intel.com> writes:
> 
> > On Mon, Jul 29, 2024 at 04:46:10PM +0800, Ying Huang wrote:  
> >> Now, the target type of region is hard-coded to HOSTONLYMEM, because
> >> only type3 expanders are supported.  To support type2 accelerators,
> >> set the target type of region root decoder with that of the root
> >> decoder.  
> >
> > Hi Ying,
> >
> > If the target type of a region is always the same as it's root decoder,
> > (is it?)  
> 
> IIUC, it is.  Do you know when they may be different?

Root decoder (CFMW I think) allows both and target device only one.

More likely when it's HDM-DB / HDM-H though I think than
HDM-D / HDM-H.  A host would do this because it has simple
address routing (maybe a single root complex) and doesn't
want to pay the HPA space cost of providing separate regions,
so decisions on protocol etc derived from RC HDM decoder
programming, not the fixed bit in in the host.

Note for those not so buried in CXL terms, HDM-DB includes back
invalidate memory devices (for P2P or hardware coherent sharing),
HDM-H is simpler memory devices that don't support back invalidate.

Jonathan


> 
> > why do we store it as an attribute of the region. Can we look
> > it up when needed?  
> 
> Yes.  This is possible via to_cxl_root_decoder().  It's just
> a little inconvenient.
> 
> > A bit more below -
> >  
> >> 
> >> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> >> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> >> Cc: Davidlohr Bueso <dave@stgolabs.net>
> >> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> >> Cc: Dave Jiang <dave.jiang@intel.com>
> >> Cc: Alison Schofield <alison.schofield@intel.com>
> >> Cc: Vishal Verma <vishal.l.verma@intel.com>
> >> Cc: Ira Weiny <ira.weiny@intel.com>
> >> Cc: Alejandro Lucero <alucerop@amd.com>
> >> ---
> >>  drivers/cxl/core/region.c | 3 ++-
> >>  1 file changed, 2 insertions(+), 1 deletion(-)
> >> 
> >> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> >> index 21ad5f242875..9a483c8a32fd 100644
> >> --- a/drivers/cxl/core/region.c
> >> +++ b/drivers/cxl/core/region.c
> >> @@ -2545,7 +2545,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
> >>  		return ERR_PTR(-EBUSY);
> >>  	}
> >>  
> >> -	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
> >> +	return devm_cxl_add_region(cxlrd, id, mode,
> >> +				   cxlrd->cxlsd.cxld.target_type);
> >>  }  
> >
> > Passing the 'cxlrd' and then a piece of the cxlrd (.target_type) looks
> > redundant.  
> 
> Yes.  We can remove the parameter.  Will change this if we still need
> cxlr->type.  Thanks!
> 
> --
> Best Regards,
> Huang, Ying
> 
> >
> > -- Alison
> >  
> >>  
> >>  static ssize_t create_pmem_region_store(struct device *dev,
> >> -- 
> >> 2.39.2
> >>   
> 
> 

Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Jonathan Cameron]

On Mon, 29 Jul 2024 16:46:11 +0800
Huang Ying <ying.huang@intel.com> wrote:

> The memory range of a type2 accelerator should be managed by the type2
> accelerator specific driver instead of the common dax region drivers,
> as discussed in [1].
> 
> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
> 
> So, in this patch, we skip dax regions creation for type2 accelerator
> device memory regions.
> 
> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---
>  drivers/cxl/core/region.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 9a483c8a32fd..b37e12bb4a35 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>  					p->res->start, p->res->end, cxlr,
>  					is_system_ram) > 0)
>  			return 0;
> +		/*
> +		 * HDM-D[B] (device-memory) regions have accelerator
> +		 * specific usage, skip device-dax registration.
> +		 */
> +		if (cxlr->type == CXL_DECODER_DEVMEM)
> +			return 0;

As in previous need to be careful as that may not mean it's
an accelerator.

However, we do need to deal with BI setup for HDM-DB type 3 devices
etc and to check the HDM Decoder capability registers to make sure
Supported Coherence model is appropriate. (e.g. 11 for host only or
device coherency - HDM-H/HDM-DB)

> +
> +		/* HDM-H routes to device-dax */
>  		return devm_cxl_add_dax_region(cxlr);
>  	default:
>  		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",

Re: [PATCH 1/3] cxl: Set target type of root decoder based on CFMWS restrictions

[Huang, Ying]

Hi, Jonathan,

Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:

> On Mon, 29 Jul 2024 16:46:09 +0800
> Huang Ying <ying.huang@intel.com> wrote:
>
>> Now, the target type of root decoder is hard-coded to HOSTONLYMEM,
>> because only type3 expanders are supported.  To support type2
>> accelerators, set the target type of root decoder based on the
>> window restrictions field of CFMWS entry.
>> 
>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>> Suggested-by: Dan Williams <dan.j.williams@intel.com>
>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>> Cc: Dave Jiang <dave.jiang@intel.com>
>> Cc: Alison Schofield <alison.schofield@intel.com>
>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>> Cc: Ira Weiny <ira.weiny@intel.com>
>> Cc: Alejandro Lucero <alucerop@amd.com>
>> ---
>>  drivers/cxl/acpi.c | 5 ++++-
>>  1 file changed, 4 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
>> index 82b78e331d8e..40c92ad29122 100644
>> --- a/drivers/cxl/acpi.c
>> +++ b/drivers/cxl/acpi.c
>> @@ -382,7 +382,10 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,
>>  
>>  	cxld = &cxlrd->cxlsd.cxld;
>>  	cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions);
>> -	cxld->target_type = CXL_DECODER_HOSTONLYMEM;
>> +	if (cxld->flags & CXL_DECODER_F_TYPE2)
>
> These flags need updating or we are going to run into problems
> long term.
>
> As of more recent specs, the distinction is messier than it was and
> it's device coherent HDM-D / HDM-DB (second one being type2 or type3 with
> BI support) and/or Host only coherent HDM-H.

I got your idea.  Previously, Device Coherent (HDM-D/DB) means type2
devices, while Host-only Coherent (HDM-H) means type3 devices.  But in
recent specs, type3 devices could be HDM-DB too.  So, we should rename
ACPI_CEDT_CFMWS_RESTRICT_TYPEX and CXL_DECODER_F_TYPEX.  What's your
suggestion for the new name? _DEVMEM and _HOSTONLYMEM?

> I'm curious on whether anyone is support both on same CFWMS?
> I believe it is possible and the spec doesn't rule it out.

This sounds possible.

> Jonathan
>
>
>> +		cxld->target_type = CXL_DECODER_DEVMEM;
>> +	else
>> +		cxld->target_type = CXL_DECODER_HOSTONLYMEM;
>>  	cxld->hpa_range = (struct range) {
>>  		.start = cfmws->base_hpa,
>>  		.end = cfmws->base_hpa + cfmws->window_size - 1,

--
Best Regards,
Huang, Ying

Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Huang, Ying]

Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:

> On Mon, 29 Jul 2024 16:46:11 +0800
> Huang Ying <ying.huang@intel.com> wrote:
>
>> The memory range of a type2 accelerator should be managed by the type2
>> accelerator specific driver instead of the common dax region drivers,
>> as discussed in [1].
>> 
>> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>> 
>> So, in this patch, we skip dax regions creation for type2 accelerator
>> device memory regions.
>> 
>> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>> 
>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>> Cc: Dave Jiang <dave.jiang@intel.com>
>> Cc: Alison Schofield <alison.schofield@intel.com>
>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>> Cc: Ira Weiny <ira.weiny@intel.com>
>> Cc: Alejandro Lucero <alucerop@amd.com>
>> ---
>>  drivers/cxl/core/region.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>> 
>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>> index 9a483c8a32fd..b37e12bb4a35 100644
>> --- a/drivers/cxl/core/region.c
>> +++ b/drivers/cxl/core/region.c
>> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>>  					p->res->start, p->res->end, cxlr,
>>  					is_system_ram) > 0)
>>  			return 0;
>> +		/*
>> +		 * HDM-D[B] (device-memory) regions have accelerator
>> +		 * specific usage, skip device-dax registration.
>> +		 */
>> +		if (cxlr->type == CXL_DECODER_DEVMEM)
>> +			return 0;
>
> As in previous need to be careful as that may not mean it's
> an accelerator.

Yes.  We need some other way to identify type2 devices.

> However, we do need to deal with BI setup for HDM-DB type 3 devices
> etc and to check the HDM Decoder capability registers to make sure
> Supported Coherence model is appropriate. (e.g. 11 for host only or
> device coherency - HDM-H/HDM-DB)

Yes.  We need to check BI configuration too.

>> +
>> +		/* HDM-H routes to device-dax */
>>  		return devm_cxl_add_dax_region(cxlr);
>>  	default:
>>  		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",

--
Best Regards,
Huang, Ying

Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Alejandro Lucero Palau]

I do not understand why you took this change from my patchset.

Maybe the other two patches have a pass, but this should not be removed 
from my patchset, not at least after discussing it publicly.

The reason you mentioned for doing it was for making things easier for 
the other changes in my larger patchset, but again, you should have 
discussed this first publicly, and second, I do not remember other large 
patchsets, far larger than mine, being partially picked up  by someone 
else but the patchset submitter, and sending those picked changes in 
another patchset.


On 8/6/24 06:52, Huang, Ying wrote:
> Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:
>
>> On Mon, 29 Jul 2024 16:46:11 +0800
>> Huang Ying <ying.huang@intel.com> wrote:
>>
>>> The memory range of a type2 accelerator should be managed by the type2
>>> accelerator specific driver instead of the common dax region drivers,
>>> as discussed in [1].
>>>
>>> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>>>
>>> So, in this patch, we skip dax regions creation for type2 accelerator
>>> device memory regions.
>>>
>>> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>>>
>>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>>> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
>>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>>> Cc: Dave Jiang <dave.jiang@intel.com>
>>> Cc: Alison Schofield <alison.schofield@intel.com>
>>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>>> Cc: Ira Weiny <ira.weiny@intel.com>
>>> Cc: Alejandro Lucero <alucerop@amd.com>
>>> ---
>>>   drivers/cxl/core/region.c | 8 ++++++++
>>>   1 file changed, 8 insertions(+)
>>>
>>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>> index 9a483c8a32fd..b37e12bb4a35 100644
>>> --- a/drivers/cxl/core/region.c
>>> +++ b/drivers/cxl/core/region.c
>>> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>>>   					p->res->start, p->res->end, cxlr,
>>>   					is_system_ram) > 0)
>>>   			return 0;
>>> +		/*
>>> +		 * HDM-D[B] (device-memory) regions have accelerator
>>> +		 * specific usage, skip device-dax registration.
>>> +		 */
>>> +		if (cxlr->type == CXL_DECODER_DEVMEM)
>>> +			return 0;
>> As in previous need to be careful as that may not mean it's
>> an accelerator.
> Yes.  We need some other way to identify type2 devices.
>
>> However, we do need to deal with BI setup for HDM-DB type 3 devices
>> etc and to check the HDM Decoder capability registers to make sure
>> Supported Coherence model is appropriate. (e.g. 11 for host only or
>> device coherency - HDM-H/HDM-DB)
> Yes.  We need to check BI configuration too.
>
>>> +
>>> +		/* HDM-H routes to device-dax */
>>>   		return devm_cxl_add_dax_region(cxlr);
>>>   	default:
>>>   		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",
> --
> Best Regards,
> Huang, Ying

Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Alejandro Lucero Palau]

On 8/6/24 06:52, Huang, Ying wrote:
> Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:
>
>> On Mon, 29 Jul 2024 16:46:11 +0800
>> Huang Ying <ying.huang@intel.com> wrote:
>>
>>> The memory range of a type2 accelerator should be managed by the type2
>>> accelerator specific driver instead of the common dax region drivers,
>>> as discussed in [1].
>>>
>>> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>>>
>>> So, in this patch, we skip dax regions creation for type2 accelerator
>>> device memory regions.
>>>
>>> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>>>
>>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>>> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
>>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>>> Cc: Dave Jiang <dave.jiang@intel.com>
>>> Cc: Alison Schofield <alison.schofield@intel.com>
>>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>>> Cc: Ira Weiny <ira.weiny@intel.com>
>>> Cc: Alejandro Lucero <alucerop@amd.com>
>>> ---
>>>   drivers/cxl/core/region.c | 8 ++++++++
>>>   1 file changed, 8 insertions(+)
>>>
>>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>> index 9a483c8a32fd..b37e12bb4a35 100644
>>> --- a/drivers/cxl/core/region.c
>>> +++ b/drivers/cxl/core/region.c
>>> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>>>   					p->res->start, p->res->end, cxlr,
>>>   					is_system_ram) > 0)
>>>   			return 0;
>>> +		/*
>>> +		 * HDM-D[B] (device-memory) regions have accelerator
>>> +		 * specific usage, skip device-dax registration.
>>> +		 */
>>> +		if (cxlr->type == CXL_DECODER_DEVMEM)
>>> +			return 0;
>> As in previous need to be careful as that may not mean it's
>> an accelerator.
> Yes.  We need some other way to identify type2 devices.


Maybe the easier option is the accel driver specifying if DAX should be 
used.

We are adding mailbox and hdm as optional for accel/type2, with the 
driver specifying what is supported. Another optional param could be 
this DAX usage.


>> However, we do need to deal with BI setup for HDM-DB type 3 devices
>> etc and to check the HDM Decoder capability registers to make sure
>> Supported Coherence model is appropriate. (e.g. 11 for host only or
>> device coherency - HDM-H/HDM-DB)
> Yes.  We need to check BI configuration too.
>
>>> +
>>> +		/* HDM-H routes to device-dax */
>>>   		return devm_cxl_add_dax_region(cxlr);
>>>   	default:
>>>   		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",
> --
> Best Regards,
> Huang, Ying

Re: [PATCH 1/3] cxl: Set target type of root decoder based on CFMWS restrictions

[Fan Ni]

On Mon, Jul 29, 2024 at 04:46:09PM +0800, Huang Ying wrote:
> Now, the target type of root decoder is hard-coded to HOSTONLYMEM,
> because only type3 expanders are supported.  To support type2
> accelerators, set the target type of root decoder based on the
> window restrictions field of CFMWS entry.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---

Reviewed-by: Fan Ni <fan.ni@samsung.com>

>  drivers/cxl/acpi.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index 82b78e331d8e..40c92ad29122 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -382,7 +382,10 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,
>  
>  	cxld = &cxlrd->cxlsd.cxld;
>  	cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions);
> -	cxld->target_type = CXL_DECODER_HOSTONLYMEM;
> +	if (cxld->flags & CXL_DECODER_F_TYPE2)
> +		cxld->target_type = CXL_DECODER_DEVMEM;
> +	else
> +		cxld->target_type = CXL_DECODER_HOSTONLYMEM;
>  	cxld->hpa_range = (struct range) {
>  		.start = cfmws->base_hpa,
>  		.end = cfmws->base_hpa + cfmws->window_size - 1,
> -- 
> 2.39.2
> 

Re: [PATCH 2/3] cxl: Set target type of region with that of root decoder

[Fan Ni]

On Mon, Jul 29, 2024 at 04:46:10PM +0800, Huang Ying wrote:
> Now, the target type of region is hard-coded to HOSTONLYMEM, because
> only type3 expanders are supported.  To support type2 accelerators,
> set the target type of region root decoder with that of the root
> decoder.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alejandro Lucero <alucerop@amd.com>
> ---

Reviewed-by: Fan Ni <fan.ni@samsung.com>

>  drivers/cxl/core/region.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 21ad5f242875..9a483c8a32fd 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -2545,7 +2545,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
>  		return ERR_PTR(-EBUSY);
>  	}
>  
> -	return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM);
> +	return devm_cxl_add_region(cxlrd, id, mode,
> +				   cxlrd->cxlsd.cxld.target_type);
>  }
>  
>  static ssize_t create_pmem_region_store(struct device *dev,
> -- 
> 2.39.2
> 

Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

[Huang, Ying]

Alejandro Lucero Palau <alucerop@amd.com> writes:

> On 8/6/24 06:52, Huang, Ying wrote:
>> Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:
>>
>>> On Mon, 29 Jul 2024 16:46:11 +0800
>>> Huang Ying <ying.huang@intel.com> wrote:
>>>
>>>> The memory range of a type2 accelerator should be managed by the type2
>>>> accelerator specific driver instead of the common dax region drivers,
>>>> as discussed in [1].
>>>>
>>>> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>>>>
>>>> So, in this patch, we skip dax regions creation for type2 accelerator
>>>> device memory regions.
>>>>
>>>> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>>>>
>>>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>>>> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
>>>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>>>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>>>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>>>> Cc: Dave Jiang <dave.jiang@intel.com>
>>>> Cc: Alison Schofield <alison.schofield@intel.com>
>>>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>>>> Cc: Ira Weiny <ira.weiny@intel.com>
>>>> Cc: Alejandro Lucero <alucerop@amd.com>
>>>> ---
>>>>   drivers/cxl/core/region.c | 8 ++++++++
>>>>   1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>>> index 9a483c8a32fd..b37e12bb4a35 100644
>>>> --- a/drivers/cxl/core/region.c
>>>> +++ b/drivers/cxl/core/region.c
>>>> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>>>>   					p->res->start, p->res->end, cxlr,
>>>>   					is_system_ram) > 0)
>>>>   			return 0;
>>>> +		/*
>>>> +		 * HDM-D[B] (device-memory) regions have accelerator
>>>> +		 * specific usage, skip device-dax registration.
>>>> +		 */
>>>> +		if (cxlr->type == CXL_DECODER_DEVMEM)
>>>> +			return 0;
>>> As in previous need to be careful as that may not mean it's
>>> an accelerator.
>> Yes.  We need some other way to identify type2 devices.
>
>
> Maybe the easier option is the accel driver specifying if DAX should
> be used.
>
> We are adding mailbox and hdm as optional for accel/type2, with the
> driver specifying what is supported. Another optional param could be
> this DAX usage.

Another way is let accel/cxl_pci driver specify the device type (type2
vs. type3).  cxl_pci_probe()->cxl_memdev_state_create() will set
cxl_dev_state.type = CXL_DEVTYPE_CLASSMEM. And
accel_X_probe()->cxl_accel_state_create() can set cxl_dev_state.type =
CXL_DEVTYPE_DEVMEM.  This can be passed to cxl_region.  Then we can
create cxl_dax_region for type3 devices only by default.

>
>>> However, we do need to deal with BI setup for HDM-DB type 3 devices
>>> etc and to check the HDM Decoder capability registers to make sure
>>> Supported Coherence model is appropriate. (e.g. 11 for host only or
>>> device coherency - HDM-H/HDM-DB)
>> Yes.  We need to check BI configuration too.
>>
>>>> +
>>>> +		/* HDM-H routes to device-dax */
>>>>   		return devm_cxl_add_dax_region(cxlr);
>>>>   	default:
>>>>   		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",

--
Best Regards,
Huang, Ying