NTB driver support in haswell platform?

NTB driver support in haswell platform?

[Yijing Wang]

Hi Jon,
   I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id
is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support
the ntb device in Haswell platform ?

Haswell NTB device id:

>From Haswell EDS 7.4.2

did
Bus: 0 Device: 3 Function: 0 Offset: 2
Bit Attr Default Description
15:0 RO-V 2F08h Device_Identification_Number — Device ID values vary from function to function.
Bits 15:8 are equal to 0x2F. The following list is a breakdown of the function groups.
  0x2F00 - 0x2F1F : PCI Express and DMI2
  0x2F20 - 0x2F3F : Integrated I/O Features
  0x2F40 - 0x2F5F : Performance Monitors
  0x2F80 - 0x2F9F : Intel QPI
  0x2FA0 - 0x2FBF : Home Agent/Memory Controller
  0x2FC0 - 0x2FDF : Power Management
  0x2FE0 - 0x2FFF : Cbo/Ring
Default value may vary based on bus, device, and function of this CSR location.


Current ntb_pci_tbl[] in Linux:

#define PCI_DEVICE_ID_INTEL_NTB_B2B_JSF		0x3725
#define PCI_DEVICE_ID_INTEL_NTB_PS_JSF		0x3726
#define PCI_DEVICE_ID_INTEL_NTB_SS_JSF		0x3727
#define PCI_DEVICE_ID_INTEL_NTB_B2B_SNB		0x3C0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_SNB		0x3C0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_SNB		0x3C0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_IVT		0x0E0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_IVT		0x0E0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_IVT		0x0E0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_HSX		0x2F0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_HSX		0x2F0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_HSX		0x2F0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_BWD		0x0C4E

So we should modify the default device id to 0x2F0D, 0x2F0E or 0x2F0F ?
What's the difference between them?

Thanks!
Yijing.




-- 
Thanks!
Yijing

Re: NTB driver support in haswell platform?

[Jon Mason]

On Wed, Jun 11, 2014 at 05:03:38PM +0800, Yijing Wang wrote:
> Hi Jon,
>    I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
> I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id
> is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support
> the ntb device in Haswell platform ?

Yes, it does support Haswell and the Device IDs are in there.
PCI_DEVICE_ID_INTEL_NTB_B2B_HSX, PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and
PCI_DEVICE_ID_INTEL_NTB_SS_HSX are the relevant dev ids for Haswell.

> Haswell NTB device id:
> 
> From Haswell EDS 7.4.2
> 
> did
> Bus: 0 Device: 3 Function: 0 Offset: 2
> Bit Attr Default Description
> 15:0 RO-V 2F08h Device_Identification_Number — Device ID values vary from function to function.
> Bits 15:8 are equal to 0x2F. The following list is a breakdown of the function groups.
>   0x2F00 - 0x2F1F : PCI Express and DMI2
>   0x2F20 - 0x2F3F : Integrated I/O Features
>   0x2F40 - 0x2F5F : Performance Monitors
>   0x2F80 - 0x2F9F : Intel QPI
>   0x2FA0 - 0x2FBF : Home Agent/Memory Controller
>   0x2FC0 - 0x2FDF : Power Management
>   0x2FE0 - 0x2FFF : Cbo/Ring
> Default value may vary based on bus, device, and function of this CSR location.
> 
> 
> Current ntb_pci_tbl[] in Linux:
> 
> #define PCI_DEVICE_ID_INTEL_NTB_B2B_JSF		0x3725
> #define PCI_DEVICE_ID_INTEL_NTB_PS_JSF		0x3726
> #define PCI_DEVICE_ID_INTEL_NTB_SS_JSF		0x3727
> #define PCI_DEVICE_ID_INTEL_NTB_B2B_SNB		0x3C0D
> #define PCI_DEVICE_ID_INTEL_NTB_PS_SNB		0x3C0E
> #define PCI_DEVICE_ID_INTEL_NTB_SS_SNB		0x3C0F
> #define PCI_DEVICE_ID_INTEL_NTB_B2B_IVT		0x0E0D
> #define PCI_DEVICE_ID_INTEL_NTB_PS_IVT		0x0E0E
> #define PCI_DEVICE_ID_INTEL_NTB_SS_IVT		0x0E0F
> #define PCI_DEVICE_ID_INTEL_NTB_B2B_HSX		0x2F0D
> #define PCI_DEVICE_ID_INTEL_NTB_PS_HSX		0x2F0E
> #define PCI_DEVICE_ID_INTEL_NTB_SS_HSX		0x2F0F
> #define PCI_DEVICE_ID_INTEL_NTB_B2B_BWD		0x0C4E
> 
> So we should modify the default device id to 0x2F0D, 0x2F0E or 0x2F0F ?

The device IDs are present above: PCI_DEVICE_ID_INTEL_NTB_B2B_HSX,
PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and PCI_DEVICE_ID_INTEL_NTB_SS_HSX.

> What's the difference between them?

The last 3 letters are the name of the CPU where NTB is found.  HSX is
Haswell Xeon.  The 2-3 letters before that are the configuration type
of the NTB device.  B2B is for "Back-to-back" configurations, aka
"NTB-NTB". 

B2B
[CPU]---[NTB]===[NTB]---[CPU]

PS/SS is for NTB-RP configurations.  PS is "Primary Side" and SS is
"Secondary Side".

[CPU]---[SS|PS]---[CPU]

I have an NTB wiki on my github account
(https://github.com/jonmason/ntb/wiki) describing the configuration,
etc.  Also on the wiki is a link to a doc (not written by me, and
contains references to a driver that was not made public) that has
some graphics that might be useful.  Specifically, pages 10 and 17.
To save time, the URL is
http://download.intel.com/design/intarch/papers/323328.pdf

Let me know if you have any questions or issues, and I'll be happy to
walk you through it.

Thanks,
Jon

> 
> Thanks!
> Yijing.
> 
> 
> 
> 
> -- 
> Thanks!
> Yijing
> 

Re: NTB driver support in haswell platform?

[Yijing Wang]

Hi Jon, Thanks for your detailed explanation. Now I have a clearer understanding of it.

Thanks! :)
Yijing.

On 2014/6/12 1:18, Jon Mason wrote:
> On Wed, Jun 11, 2014 at 05:03:38PM +0800, Yijing Wang wrote:
>> Hi Jon,
>>    I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
>> I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id
>> is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support
>> the ntb device in Haswell platform ?
> 
> Yes, it does support Haswell and the Device IDs are in there.
> PCI_DEVICE_ID_INTEL_NTB_B2B_HSX, PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and
> PCI_DEVICE_ID_INTEL_NTB_SS_HSX are the relevant dev ids for Haswell.
> 
>> Haswell NTB device id:
>>
>> From Haswell EDS 7.4.2
>>
>> did
>> Bus: 0 Device: 3 Function: 0 Offset: 2
>> Bit Attr Default Description
>> 15:0 RO-V 2F08h Device_Identification_Number — Device ID values vary from function to function.
>> Bits 15:8 are equal to 0x2F. The following list is a breakdown of the function groups.
>>   0x2F00 - 0x2F1F : PCI Express and DMI2
>>   0x2F20 - 0x2F3F : Integrated I/O Features
>>   0x2F40 - 0x2F5F : Performance Monitors
>>   0x2F80 - 0x2F9F : Intel QPI
>>   0x2FA0 - 0x2FBF : Home Agent/Memory Controller
>>   0x2FC0 - 0x2FDF : Power Management
>>   0x2FE0 - 0x2FFF : Cbo/Ring
>> Default value may vary based on bus, device, and function of this CSR location.
>>
>>
>> Current ntb_pci_tbl[] in Linux:
>>
>> #define PCI_DEVICE_ID_INTEL_NTB_B2B_JSF		0x3725
>> #define PCI_DEVICE_ID_INTEL_NTB_PS_JSF		0x3726
>> #define PCI_DEVICE_ID_INTEL_NTB_SS_JSF		0x3727
>> #define PCI_DEVICE_ID_INTEL_NTB_B2B_SNB		0x3C0D
>> #define PCI_DEVICE_ID_INTEL_NTB_PS_SNB		0x3C0E
>> #define PCI_DEVICE_ID_INTEL_NTB_SS_SNB		0x3C0F
>> #define PCI_DEVICE_ID_INTEL_NTB_B2B_IVT		0x0E0D
>> #define PCI_DEVICE_ID_INTEL_NTB_PS_IVT		0x0E0E
>> #define PCI_DEVICE_ID_INTEL_NTB_SS_IVT		0x0E0F
>> #define PCI_DEVICE_ID_INTEL_NTB_B2B_HSX		0x2F0D
>> #define PCI_DEVICE_ID_INTEL_NTB_PS_HSX		0x2F0E
>> #define PCI_DEVICE_ID_INTEL_NTB_SS_HSX		0x2F0F
>> #define PCI_DEVICE_ID_INTEL_NTB_B2B_BWD		0x0C4E
>>
>> So we should modify the default device id to 0x2F0D, 0x2F0E or 0x2F0F ?
> 
> The device IDs are present above: PCI_DEVICE_ID_INTEL_NTB_B2B_HSX,
> PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and PCI_DEVICE_ID_INTEL_NTB_SS_HSX.
> 
>> What's the difference between them?
> 
> The last 3 letters are the name of the CPU where NTB is found.  HSX is
> Haswell Xeon.  The 2-3 letters before that are the configuration type
> of the NTB device.  B2B is for "Back-to-back" configurations, aka
> "NTB-NTB". 
> 
> B2B
> [CPU]---[NTB]===[NTB]---[CPU]
> 
> PS/SS is for NTB-RP configurations.  PS is "Primary Side" and SS is
> "Secondary Side".
> 
> [CPU]---[SS|PS]---[CPU]
> 
> I have an NTB wiki on my github account
> (https://github.com/jonmason/ntb/wiki) describing the configuration,
> etc.  Also on the wiki is a link to a doc (not written by me, and
> contains references to a driver that was not made public) that has
> some graphics that might be useful.  Specifically, pages 10 and 17.
> To save time, the URL is
> http://download.intel.com/design/intarch/papers/323328.pdf
> 
> Let me know if you have any questions or issues, and I'll be happy to
> walk you through it.
> 
> Thanks,
> Jon
> 
>>
>> Thanks!
>> Yijing.
>>
>>
>>
>>
>> -- 
>> Thanks!
>> Yijing
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> .
> 


-- 
Thanks!
Yijing