[PATCH] hw/cxl/device: Add Flex Bus Port DVSEC

[PATCH] hw/cxl/device: Add Flex Bus Port DVSEC

[Ira Weiny]

The Flex Bus Port DVSEC was missing on type 3 devices which was blocking
RAS checks.[1]

Add the Flex Bus Port DVSEC to type 3 devices as per CXL 3.0 8.2.1.3.

[1] https://lore.kernel.org/linux-cxl/167096738875.2861540.11815053323626849940.stgit@djiang5-desk3.ch.intel.com/

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Ben Widawsky <bwidawsk@kernel.org>
Cc: qemu-devel@nongnu.org
Cc: linux-cxl@vger.kernel.org
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
 hw/mem/cxl_type3.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 0317bd96a6fb..27f6ac0cb3c1 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -416,6 +416,17 @@ static void build_dvsecs(CXLType3Dev *ct3d)
     cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
                                GPF_DEVICE_DVSEC_LENGTH, GPF_DEVICE_DVSEC,
                                GPF_DEVICE_DVSEC_REVID, dvsec);
+
+    dvsec = (uint8_t *)&(CXLDVSECPortFlexBus){
+        .cap                     = 0x27, /* Cache, IO, Mem, non-MLD */
+        .ctrl                    = 0x02, /* IO always enabled */
+        .status                  = 0x27, /* same as capabilities */
+        .rcvd_mod_ts_data_phase1 = 0xef, /* WTF? */
+    };
+    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
+                               PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0,
+                               PCIE_FLEXBUS_PORT_DVSEC,
+                               PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0, dvsec);
 }
 
 static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)

---
base-commit: e11b57108b0cb746bb9f3887054f34a2f818ed79
change-id: 20221213-ira-flexbus-port-ce526de8111d

Best regards,
-- 
Ira Weiny <ira.weiny@intel.com>

Re: [PATCH] hw/cxl/device: Add Flex Bus Port DVSEC

[Jonathan Cameron via]

On Tue, 13 Dec 2022 16:34:52 -0800
Ira Weiny <ira.weiny@intel.com> wrote:

> The Flex Bus Port DVSEC was missing on type 3 devices which was blocking
> RAS checks.[1]
> 
> Add the Flex Bus Port DVSEC to type 3 devices as per CXL 3.0 8.2.1.3.
> 
> [1] https://lore.kernel.org/linux-cxl/167096738875.2861540.11815053323626849940.stgit@djiang5-desk3.ch.intel.com/
> 
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Cc: Ben Widawsky <bwidawsk@kernel.org>
> Cc: qemu-devel@nongnu.org
> Cc: linux-cxl@vger.kernel.org
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>

Not sure how we missed this one given the whole fun a while back with Mem Enable not
being set by the kernel code.  Ah well - I'm not going to go digging in the history
for that.

Minor thing inline - unlike root ports, switches etc, type 3 device should not
report CXL.cache support.

Jonathan


> ---
>  hw/mem/cxl_type3.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 0317bd96a6fb..27f6ac0cb3c1 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -416,6 +416,17 @@ static void build_dvsecs(CXLType3Dev *ct3d)
>      cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
>                                 GPF_DEVICE_DVSEC_LENGTH, GPF_DEVICE_DVSEC,
>                                 GPF_DEVICE_DVSEC_REVID, dvsec);
> +
> +    dvsec = (uint8_t *)&(CXLDVSECPortFlexBus){
> +        .cap                     = 0x27, /* Cache, IO, Mem, non-MLD */

Type 3 typically wouldn't support cache.  Probably want the comment to include
bit 5 (68B Flit and VH capable) That should probably true in the other instances
of this as well.


> +        .ctrl                    = 0x02, /* IO always enabled */
> +        .status                  = 0x27, /* same as capabilities */
Again, not cache.

> +        .rcvd_mod_ts_data_phase1 = 0xef, /* WTF? */
> +    };
> +    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
> +                               PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0,
> +                               PCIE_FLEXBUS_PORT_DVSEC,
> +                               PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0, dvsec);
>  }
>  
>  static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
> 
> ---
> base-commit: e11b57108b0cb746bb9f3887054f34a2f818ed79
> change-id: 20221213-ira-flexbus-port-ce526de8111d
> 
> Best regards,

Re: [PATCH] hw/cxl/device: Add Flex Bus Port DVSEC

[Jonathan Cameron via]

On Wed, 14 Dec 2022 12:39:45 +0000
Jonathan Cameron via <qemu-devel@nongnu.org> wrote:

> On Tue, 13 Dec 2022 16:34:52 -0800
> Ira Weiny <ira.weiny@intel.com> wrote:
> 
> > The Flex Bus Port DVSEC was missing on type 3 devices which was blocking
> > RAS checks.[1]
> > 
> > Add the Flex Bus Port DVSEC to type 3 devices as per CXL 3.0 8.2.1.3.
> > 
> > [1] https://lore.kernel.org/linux-cxl/167096738875.2861540.11815053323626849940.stgit@djiang5-desk3.ch.intel.com/
> > 
> > Cc: Dave Jiang <dave.jiang@intel.com>
> > Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Cc: Ben Widawsky <bwidawsk@kernel.org>
> > Cc: qemu-devel@nongnu.org
> > Cc: linux-cxl@vger.kernel.org
> > Signed-off-by: Ira Weiny <ira.weiny@intel.com>  
> 
> Not sure how we missed this one given the whole fun a while back with Mem Enable not
> being set by the kernel code.  Ah well - I'm not going to go digging in the history
> for that.
Ah. I forgot the difference between the 'control' in here which isn't really a control
in this case and the one in PCIe DVSEC for CXL devices which is the one were we enable it.

I guess this calls for a more thorough check on what else we are missing
in the way of DVSEC - including this stuff which is all about link training but does
provide some useful info the stuff that results from the training.

> 
> Minor thing inline - unlike root ports, switches etc, type 3 device should not
> report CXL.cache support.
> 
> Jonathan
> 
> 
> > ---
> >  hw/mem/cxl_type3.c | 11 +++++++++++
> >  1 file changed, 11 insertions(+)
> > 
> > diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> > index 0317bd96a6fb..27f6ac0cb3c1 100644
> > --- a/hw/mem/cxl_type3.c
> > +++ b/hw/mem/cxl_type3.c
> > @@ -416,6 +416,17 @@ static void build_dvsecs(CXLType3Dev *ct3d)
> >      cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
> >                                 GPF_DEVICE_DVSEC_LENGTH, GPF_DEVICE_DVSEC,
> >                                 GPF_DEVICE_DVSEC_REVID, dvsec);
> > +
> > +    dvsec = (uint8_t *)&(CXLDVSECPortFlexBus){
> > +        .cap                     = 0x27, /* Cache, IO, Mem, non-MLD */  
> 
> Type 3 typically wouldn't support cache.  Probably want the comment to include
> bit 5 (68B Flit and VH capable) That should probably true in the other instances
> of this as well.
> 
> 
> > +        .ctrl                    = 0x02, /* IO always enabled */
> > +        .status                  = 0x27, /* same as capabilities */  
> Again, not cache.
> 
> > +        .rcvd_mod_ts_data_phase1 = 0xef, /* WTF? */
> > +    };
> > +    cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE,
> > +                               PCIE_FLEXBUS_PORT_DVSEC_LENGTH_2_0,
> > +                               PCIE_FLEXBUS_PORT_DVSEC,
> > +                               PCIE_FLEXBUS_PORT_DVSEC_REVID_2_0, dvsec);
> >  }
> >  
> >  static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
> > 
> > ---
> > base-commit: e11b57108b0cb746bb9f3887054f34a2f818ed79
> > change-id: 20221213-ira-flexbus-port-ce526de8111d
> > 
> > Best regards,  
> 
>