[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

Update version after incorporating David Miller's comment from earlier
posting [1]. I would like to get these merged for upcoming 3.18 merge
window if there are no concerns on this version.

The network coprocessor (NetCP) is a hardware accelerator that processes
Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
switch sub-module to send and receive packets. NetCP also includes a packet
accelerator (PA) module to perform packet classification operations such as
header matching, and packet modification operations such as checksum
generation. NetCP can also optionally include a Security Accelerator(SA)
capable of performing IPSec operations on ingress/egress packets.
    
Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
1Gb/s rates per Ethernet port.
    
NetCP driver has a plug-in module architecture where each of the NetCP
sub-modules exist as a loadable kernel module which plug in to the netcp
core. These sub-modules are represented as "netcp-devices" in the dts
bindings. It is mandatory to have the ethernet switch sub-module for
the ethernet interface to be operational. Any other sub-module like the
PA is optional.

Both GBE and XGBE network processors supported using common driver. It
is also designed to handle future variants of NetCP.

Cc: David Miller <davem@davemloft.net>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Sandeep Nair <sandeep_n@ti.com>

Sandeep Nair (3):
  Documentation: dt: net: Add binding doc for Keystone NetCP ethernet
    driver
  net: Add Keystone NetCP ethernet driver
  MAINTAINER: net: Add TI NETCP Ethernet driver entry

 .../devicetree/bindings/net/keystone-netcp.txt     |  197 ++
 MAINTAINERS                                        |    6 +
 drivers/net/ethernet/ti/Kconfig                    |   15 +-
 drivers/net/ethernet/ti/Makefile                   |    4 +
 drivers/net/ethernet/ti/netcp.h                    |  227 ++
 drivers/net/ethernet/ti/netcp_core.c               | 2276 ++++++++++++++++++++
 drivers/net/ethernet/ti/netcp_ethss.c              | 2178 +++++++++++++++++++
 drivers/net/ethernet/ti/netcp_sgmii.c              |  130 ++
 drivers/net/ethernet/ti/netcp_xgbepcsr.c           |  502 +++++
 9 files changed, 5532 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/net/keystone-netcp.txt
 create mode 100644 drivers/net/ethernet/ti/netcp.h
 create mode 100644 drivers/net/ethernet/ti/netcp_core.c
 create mode 100644 drivers/net/ethernet/ti/netcp_ethss.c
 create mode 100644 drivers/net/ethernet/ti/netcp_sgmii.c
 create mode 100644 drivers/net/ethernet/ti/netcp_xgbepcsr.c

regards,
Santosh

[1] https://lkml.org/lkml/2014/4/22/805

-- 
1.7.9.5

[PATCH v2 1/3] Documentation: dt: net: Add binding doc for Keystone NetCP ethernet driver

[Santosh Shilimkar]

From: Sandeep Nair <sandeep_n@ti.com>

The network coprocessor (NetCP) is a hardware accelerator that processes
Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
switch sub-module to send and receive packets. NetCP also includes a packet
accelerator (PA) module to perform packet classification operations such as
header matching, and packet modification operations such as checksum
generation. NetCP can also optionally include a Security Accelerator(SA)
capable of performing IPSec operations on ingress/egress packets.

Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
1Gb/s rates per Ethernet port.

NetCP Subsystem device tree layout looks something like below:

-----------------------------
  NetCP subsystem(10G or 1G)
-----------------------------
	|
	|-> NetCP Devices ->	|
	|			|-> GBE/XGBE Switch
	|			|
	|			|-> Packet Accelerator
	|			|
	|			|-> Security Accelerator
	|
	|
	|
	|-> NetCP Interfaces ->	|
				|-> Ethernet Port 0
				|
				|-> Ethernet Port 1
				|
				|-> Ethernet Port 2
				|
				|-> Ethernet Port 3

Common driver supports GBE as well XGBE network processors.

Cc: Rob Herring <robh+dt@kernel.org>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Pawel Moll <pawel.moll@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ian Campbell <ijc+devicetree@hellion.org.uk>
Cc: Kumar Gala <galak@codeaurora.org>
Cc: David Miller <davem@davemloft.net>

Signed-off-by: Sandeep Nair <sandeep_n@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 .../devicetree/bindings/net/keystone-netcp.txt     |  197 ++++++++++++++++++++
 1 file changed, 197 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/net/keystone-netcp.txt

diff --git a/Documentation/devicetree/bindings/net/keystone-netcp.txt b/Documentation/devicetree/bindings/net/keystone-netcp.txt
new file mode 100644
index 0000000..a7d061b
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/keystone-netcp.txt
@@ -0,0 +1,197 @@
+This document describes the device tree bindings associated with the
+keystone network coprocessor(NetCP) driver support.
+
+The network coprocessor (NetCP) is a hardware accelerator that processes
+Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsytem with a ethernet
+switch sub-module to send and receive packets. NetCP also includes a packet
+accelerator (PA) module to perform packet classification operations such as
+header matching, and packet modification operations such as checksum
+generation. NetCP can also optionally include a Security Accelerator (SA)
+capable of performing IPSec operations on ingress/egress packets.
+
+Keystone II SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
+includes a 3-port Ethernet switch sub-module capable of 10Gb/s and 1Gb/s rates
+per Ethernet port.
+
+Keystone NetCP driver has a plug-in module architecture where each of the NetCP
+sub-modules exist as a loadable kernel module which plug in to the netcp core.
+These sub-modules are represented as "netcp-devices" in the dts bindings. It is
+mandatory to have the ethernet switch sub-module for the ethernet interface to
+be operational. Any other sub-module like the PA is optional.
+
+NetCP Ethernet SubSystem Layout:
+
+-----------------------------
+  NetCP subsystem(10G or 1G)
+-----------------------------
+	|
+	|-> NetCP Devices ->	|
+	|			|-> GBE/XGBE Switch
+	|			|
+	|			|-> Packet Accelerator
+	|			|
+	|			|-> Security Accelerator
+	|
+	|
+	|
+	|-> NetCP Interfaces ->	|
+				|-> Ethernet Port 0
+				|
+				|-> Ethernet Port 1
+				|
+				|-> Ethernet Port 2
+				|
+				|-> Ethernet Port 3
+
+
+NetCP subsystem properties:
+Required properties:
+- compatible:	Should be "ti,netcp-1.0"
+- clocks:	phandle to the reference clocks for the subsystem.
+- dma-id:	Navigator packet dma instance id.
+
+Optional properties:
+- reg:		register location and the size for the following register
+		regions in the specified order.
+		- Efuse MAC address register
+- dma-coherent:	Present if dma operations are coherent
+- big-endian:	Keystone devices can be operated in a mode where the DSP is in
+		the big endian mode. In such cases enable this option. This
+		option should also be enabled if the ARM is operated in
+		big endian mode with the DSP in little endian.
+
+NetCP device properties: Device specification for NetCP sub-modules.
+1Gb/10Gb (gbe/xgbe) ethernet switch sub-module specifications.
+Required properties:
+- label:	Must be "netcp-gbe" for 1Gb & "netcp-xgbe" for 10Gb.
+- reg:		register location and the size for the following register
+		regions in the specified order.
+		- subsystem registers
+		- serdes registers
+- tx-channel:	the navigator packet dma channel number for tx.
+- tx-queue:	the navigator queue number associated with the tx dma channel.
+- interfaces:	specification for each of the switch port to be registered as a
+		network interface in the stack.
+-- slave-port:	Switch port number, 0 based numbering.
+-- link-interface:	type of link interface, supported options are
+			- mac<->mac auto negotiate mode: 0
+			- mac<->phy mode: 1
+			- mac<->mac forced mode: 2
+			- mac<->fiber mode: 3
+			- mac<->phy mode with no mdio: 4
+			- 10Gb mac<->phy mode : 10
+			- 10Gb mac<->mac forced mode : 11
+----phy-handle:	phandle to PHY device
+
+Optional properties:
+- enable-ale:	NetCP driver keeps the address learning feature in the ethernet
+		switch module disabled. This attribute is to enable the address
+		learning.
+- secondary-slave-ports:	specification for each of the switch port not be
+				registered as a network interface. NetCP driver
+				will only initialize these ports and attach PHY
+				driver to them if needed.
+
+NetCP interface properties: Interface specification for NetCP sub-modules.
+Required properties:
+- rx-channel:	the navigator packet dma channel number for rx.
+- rx-queue:	the navigator queue number associated with rx dma channel.
+- rx-pool:	specifies the number of descriptors to be used & the region-id
+		for creating the rx descriptor pool.
+- tx-pool:	specifies the number of descriptors to be used & the region-id
+		for creating the tx descriptor pool.
+- rx-queue-depth:	number of descriptors in each of the free descriptor
+			queue (FDQ) for the pktdma Rx flow. There can be at
+			present a maximum of 4 queues per Rx flow.
+- rx-buffer-size:	the buffer size for each of the Rx flow FDQ.
+- tx-completion-queue:	the navigator queue number where the descriptors are
+			recycled after Tx DMA completion.
+
+Optional properties:
+- efuse-mac:	If this is 1, then the MAC address for the interface is
+		obtained from the device efuse mac address register
+- local-mac-address:	the driver is designed to use the of_get_mac_address api
+			only if efuse-mac is 0. When efuse-mac is 0, the MAC
+			address is obtained from local-mac-address. If this
+			attribute is not present, then the driver will use a
+			random MAC address.
+- "netcp-device label":	phandle to the device specification for each of NetCP
+			sub-module attached to this interface.
+
+Example binding:
+
+netcp: netcp at 2090000 {
+	reg = <0x2620110 0x8>;
+	reg-names = "efuse";
+	compatible = "ti,netcp-1.0";
+	#address-cells = <1>;
+	#size-cells = <1>;
+	ranges;
+
+	clocks = <&papllclk>, <&clkcpgmac>, <&chipclk12>;
+	dma-coherent;
+	/* big-endian; */
+	dma-id = <0>;
+
+	netcp-devices {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+		gbe at 0x2090000 {
+			label = "netcp-gbe";
+			reg = <0x2090000 0xf00>;
+			/* enable-ale; */
+			tx-queue = <648>;
+			tx-channel = <8>;
+
+			interfaces {
+				gbe0: interface-0 {
+					slave-port = <0>;
+					link-interface	= <4>;
+				};
+				gbe1: interface-1 {
+					slave-port = <1>;
+					link-interface	= <4>;
+				};
+			};
+
+			secondary-slave-ports {
+				port-2 {
+					slave-port = <2>;
+					link-interface	= <2>;
+				};
+				port-3 {
+					slave-port = <3>;
+					link-interface	= <2>;
+				};
+			};
+		};
+	};
+
+	netcp-interfaces {
+		interface-0 {
+			rx-channel = <22>;
+			rx-pool = <1024 12>;
+			tx-pool = <1024 12>;
+			rx-queue-depth = <128 128 0 0>;
+			rx-buffer-size = <1518 4096 0 0>;
+			rx-queue = <8704>;
+			tx-completion-queue = <8706>;
+			efuse-mac = <1>;
+			netcp-gbe = <&gbe0>;
+
+		};
+		interface-1 {
+			rx-channel = <23>;
+			rx-pool = <1024 12>;
+			tx-pool = <1024 12>;
+			rx-queue-depth = <128 128 0 0>;
+			rx-buffer-size = <1518 4096 0 0>;
+			rx-queue = <8705>;
+			tx-completion-queue = <8707>;
+			efuse-mac = <0>;
+			local-mac-address = [02 18 31 7e 3e 6f];
+			netcp-gbe = <&gbe1>;
+		};
+	};
+};
-- 
1.7.9.5

[PATCH v2 3/3] MAINTAINER: net: Add TI NETCP Ethernet driver entry

[Santosh Shilimkar]

From: Sandeep Nair <sandeep_n@ti.com>

Signed-off-by: Sandeep Nair <sandeep_n@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
---
 MAINTAINERS |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 134483f..7b1c41d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9038,6 +9038,12 @@ F:	drivers/power/lp8788-charger.c
 F:	drivers/regulator/lp8788-*.c
 F:	include/linux/mfd/lp8788*.h
 
+TI NETCP ETHERNET DRIVER
+M:	Sandeep Nair <sandeep_n@ti.com>
+L:	netdev at vger.kernel.org
+S:	Maintained
+F:	drivers/net/ethernet/ti/netcp*
+
 TI TWL4030 SERIES SOC CODEC DRIVER
 M:	Peter Ujfalusi <peter.ujfalusi@ti.com>
 L:	alsa-devel at alsa-project.org (moderated for non-subscribers)
-- 
1.7.9.5

[PATCH v2 2/3] net: Add Keystone NetCP ethernet driver

[Jamal Hadi Salim]

On 08/15/14 11:12, Santosh Shilimkar wrote:


I am curious about these two calls below(netcp_process_one_rx_packet
and netcp_tx_submit_skb):
On tx you seem to be broadcasting to all "sub-modules" and on receive
you seem to be invoking from all as well.
I couldnt find the code for any of the sub-modules but i suspect
things like switching/bridging or ipsec etc could be sub-modules.
If yes, have you thought about integrating these features into
Linux proper instead?

cheers,
jamal

> +
> +static inline int netcp_process_one_rx_packet(struct netcp_intf *netcp)
> +{


> +
> +	/* Call each of the RX hooks */
> +	p_info.skb = skb;
> +	p_info.rxtstamp_complete = false;
> +	list_for_each_entry(rx_hook, &netcp->rxhook_list_head, list) {
> +		int ret;
> +
> +		ret = rx_hook->hook_rtn(rx_hook->order, rx_hook->hook_data,
> +					&p_info);
> +		if (unlikely(ret)) {
> +			dev_err(netcp->ndev_dev, "RX hook %d failed: %d\n",
> +				rx_hook->order, ret);
> +			netcp->ndev->stats.rx_errors++;
> +			dev_kfree_skb(skb);
> +			return 0;
> +		}
> +	}
..
....
> +	return 0;
> +}

[..]

> +static inline int netcp_tx_submit_skb(struct netcp_intf *netcp,
> +				      struct sk_buff *skb,
> +				      struct knav_dma_desc *desc)
> +{
> +	struct netcp_tx_pipe *tx_pipe = NULL;

> +
> +	p_info.netcp = netcp;
> +	p_info.skb = skb;
> +	p_info.tx_pipe = NULL;
> +	p_info.psdata_len = 0;
> +	p_info.ts_context = NULL;
> +	p_info.txtstamp_complete = NULL;
> +	p_info.epib = desc->epib;
> +	p_info.psdata = desc->psdata;
> +	memset(p_info.epib, 0, KNAV_DMA_NUM_EPIB_WORDS * sizeof(u32));
> +
> +	/* Find out where to inject the packet for transmission */
> +	list_for_each_entry(tx_hook, &netcp->txhook_list_head, list) {
> +		ret = tx_hook->hook_rtn(tx_hook->order, tx_hook->hook_data,
> +					&p_info);
> +		if (unlikely(ret != 0)) {
> +			dev_err(netcp->ndev_dev, "TX hook %d rejected the packet with reason(%d)\n",
> +				tx_hook->order, ret);
> +			ret = (ret < 0) ? ret : NETDEV_TX_OK;
> +			goto out;
> +		}
> +	}
> +

[PATCH v2 2/3] net: Add Keystone NetCP ethernet driver

[Stephen Hemminger]

On Fri, 15 Aug 2014 11:12:41 -0400
Santosh Shilimkar <santosh.shilimkar@ti.com> wrote:

> NetCP driver has a plug-in module architecture where each of the NetCP
> sub-modules exist as a loadable kernel module which plug in to the netcp
> core. These sub-modules are represented as "netcp-devices" in the dts
> bindings. It is mandatory to have the ethernet switch sub-module for
> the ethernet interface to be operational. Any other sub-module like the
> PA is optional.

What are you doing to prevent/discourage proprietary binary only
sub-modules?

[PATCH v2 2/3] net: Add Keystone NetCP ethernet driver

[Santosh Shilimkar]

On Thursday 21 August 2014 10:48 PM, Stephen Hemminger wrote:
> On Fri, 15 Aug 2014 11:12:41 -0400
> Santosh Shilimkar <santosh.shilimkar@ti.com> wrote:
> 
>> NetCP driver has a plug-in module architecture where each of the NetCP
>> sub-modules exist as a loadable kernel module which plug in to the netcp
>> core. These sub-modules are represented as "netcp-devices" in the dts
>> bindings. It is mandatory to have the ethernet switch sub-module for
>> the ethernet interface to be operational. Any other sub-module like the
>> PA is optional.
> 
> What are you doing to prevent/discourage proprietary binary only
> sub-modules?
> 
Not sure if you mistook this infrastructure as a replacement for the
kernel module. The sub modules are traditional kernel modules that
just registers into the NetCP framework. Kernel module infrastructure
already discourages the proprietary binary only sub-modules.

Regards,
Santosh

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[David Miller]

From: Santosh Shilimkar <santosh.shilimkar@ti.com>
Date: Fri, 15 Aug 2014 11:12:39 -0400

> Update version after incorporating David Miller's comment from earlier
> posting [1]. I would like to get these merged for upcoming 3.18 merge
> window if there are no concerns on this version.
> 
> The network coprocessor (NetCP) is a hardware accelerator that processes
> Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
> switch sub-module to send and receive packets. NetCP also includes a packet
> accelerator (PA) module to perform packet classification operations such as
> header matching, and packet modification operations such as checksum
> generation. NetCP can also optionally include a Security Accelerator(SA)
> capable of performing IPSec operations on ingress/egress packets.
>     
> Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
> includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
> 1Gb/s rates per Ethernet port.
>     
> NetCP driver has a plug-in module architecture where each of the NetCP
> sub-modules exist as a loadable kernel module which plug in to the netcp
> core. These sub-modules are represented as "netcp-devices" in the dts
> bindings. It is mandatory to have the ethernet switch sub-module for
> the ethernet interface to be operational. Any other sub-module like the
> PA is optional.
> 
> Both GBE and XGBE network processors supported using common driver. It
> is also designed to handle future variants of NetCP.

I don't want to see an offload driver that doesn't plug into the existing
generic frameworks for configuration et al.

If no existing facility exists to support what you need, you must work
with the upstream maintainers to design and create one.

It is absolutely no reasonable for every "switch on a chip" driver to
export it's own configuration knob, we need a standard interface all
such drivers will plug into and provide.

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

Hi David,

On Thursday 21 August 2014 07:36 PM, David Miller wrote:
> From: Santosh Shilimkar <santosh.shilimkar@ti.com>
> Date: Fri, 15 Aug 2014 11:12:39 -0400
> 
>> Update version after incorporating David Miller's comment from earlier
>> posting [1]. I would like to get these merged for upcoming 3.18 merge
>> window if there are no concerns on this version.
>>
>> The network coprocessor (NetCP) is a hardware accelerator that processes
>> Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
>> switch sub-module to send and receive packets. NetCP also includes a packet
>> accelerator (PA) module to perform packet classification operations such as
>> header matching, and packet modification operations such as checksum
>> generation. NetCP can also optionally include a Security Accelerator(SA)
>> capable of performing IPSec operations on ingress/egress packets.
>>     
>> Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
>> includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
>> 1Gb/s rates per Ethernet port.
>>     
>> NetCP driver has a plug-in module architecture where each of the NetCP
>> sub-modules exist as a loadable kernel module which plug in to the netcp
>> core. These sub-modules are represented as "netcp-devices" in the dts
>> bindings. It is mandatory to have the ethernet switch sub-module for
>> the ethernet interface to be operational. Any other sub-module like the
>> PA is optional.
>>
>> Both GBE and XGBE network processors supported using common driver. It
>> is also designed to handle future variants of NetCP.
> 
> I don't want to see an offload driver that doesn't plug into the existing
> generic frameworks for configuration et al.
> 
> If no existing facility exists to support what you need, you must work
> with the upstream maintainers to design and create one.
> 
> It is absolutely no reasonable for every "switch on a chip" driver to
> export it's own configuration knob, we need a standard interface all
> such drivers will plug into and provide.
> 
The NetCP plugin module infrastructure use all the standard kernel
infrastructure and its very tiny. To best represent the Network processor
and its sub module hardware which have inter dependency and ordering
needs, we needed such infrastructure. This lets us handle all the
hardware needs without any code duplication per module.

To elaborate more, there are 4 variants of network switch modules and
then few accelerator modules like Packet accelerator, QOS and Security
accelerator. There can be multiple instances of switches on same SOC.
Example 1 Gbe and 10 Gbe switches. Then additional accelerator modules
are inter connected with switch, streaming fabric and packet DMA.
Packet routing changes based on the various offload modules presence and hence
needs hooks for tx/rx to be called in particular order with special
handling. This scheme is very hardware specific and doesn't have ways
to isolate the modules from each other.

On the other hand, we definitely wanted to have minimal code
instead of duplicating ndo operations and core packet processing logic
in multiple drivers or layers. The module approach helps
to isolate the code based on the customer choice who can choose
say not to build 10 Gbe hardware or say don't need QOS or Security
accelerators. That way we keep the packet processing hot path as
what we need without any overhead.

As you can see, the tiny module handling was added more to represent
the hardware, keep the modularity and avoid code duplication. The
infrastructure is very minimal and NETCP specific. With this small
infrastructure we are able to re-use code for NetCP1.0, NetCP1.5,
10 GBe and upcoming NetCP variants from just *one* driver.

Hope this gives you a better idea and rationale behind the design.

Regards,
Santosh

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

Hi Dave,

On 8/22/14 3:45 PM, Santosh Shilimkar wrote:
> Hi David,
>
> On Thursday 21 August 2014 07:36 PM, David Miller wrote:
>> From: Santosh Shilimkar <santosh.shilimkar@ti.com>
>> Date: Fri, 15 Aug 2014 11:12:39 -0400
>>
>>> Update version after incorporating David Miller's comment from earlier
>>> posting [1]. I would like to get these merged for upcoming 3.18 merge
>>> window if there are no concerns on this version.
>>>
>>> The network coprocessor (NetCP) is a hardware accelerator that processes
>>> Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
>>> switch sub-module to send and receive packets. NetCP also includes a packet
>>> accelerator (PA) module to perform packet classification operations such as
>>> header matching, and packet modification operations such as checksum
>>> generation. NetCP can also optionally include a Security Accelerator(SA)
>>> capable of performing IPSec operations on ingress/egress packets.
>>>
>>> Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
>>> includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
>>> 1Gb/s rates per Ethernet port.
>>>
>>> NetCP driver has a plug-in module architecture where each of the NetCP
>>> sub-modules exist as a loadable kernel module which plug in to the netcp
>>> core. These sub-modules are represented as "netcp-devices" in the dts
>>> bindings. It is mandatory to have the ethernet switch sub-module for
>>> the ethernet interface to be operational. Any other sub-module like the
>>> PA is optional.
>>>
>>> Both GBE and XGBE network processors supported using common driver. It
>>> is also designed to handle future variants of NetCP.
>>
>> I don't want to see an offload driver that doesn't plug into the existing
>> generic frameworks for configuration et al.
>>
>> If no existing facility exists to support what you need, you must work
>> with the upstream maintainers to design and create one.
>>
>> It is absolutely no reasonable for every "switch on a chip" driver to
>> export it's own configuration knob, we need a standard interface all
>> such drivers will plug into and provide.
>>
> The NetCP plugin module infrastructure use all the standard kernel
> infrastructure and its very tiny. To best represent the Network processor
> and its sub module hardware which have inter dependency and ordering
> needs, we needed such infrastructure. This lets us handle all the
> hardware needs without any code duplication per module.
>
> To elaborate more, there are 4 variants of network switch modules and
> then few accelerator modules like Packet accelerator, QOS and Security
> accelerator. There can be multiple instances of switches on same SOC.
> Example 1 Gbe and 10 Gbe switches. Then additional accelerator modules
> are inter connected with switch, streaming fabric and packet DMA.
> Packet routing changes based on the various offload modules presence and hence
> needs hooks for tx/rx to be called in particular order with special
> handling. This scheme is very hardware specific and doesn't have ways
> to isolate the modules from each other.
>
> On the other hand, we definitely wanted to have minimal code
> instead of duplicating ndo operations and core packet processing logic
> in multiple drivers or layers. The module approach helps
> to isolate the code based on the customer choice who can choose
> say not to build 10 Gbe hardware or say don't need QOS or Security
> accelerators. That way we keep the packet processing hot path as
> what we need without any overhead.
>
> As you can see, the tiny module handling was added more to represent
> the hardware, keep the modularity and avoid code duplication. The
> infrastructure is very minimal and NETCP specific. With this small
> infrastructure we are able to re-use code for NetCP1.0, NetCP1.5,
> 10 GBe and upcoming NetCP variants from just *one* driver.
>
> Hope this gives you a better idea and rationale behind the design.
>
Did you happen to see the reply ?
I am hoping to get this driver in for upcoming merge window.

Regards,
Santosh

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Jamal Hadi Salim]

On 09/08/14 10:41, Santosh Shilimkar wrote:

>> The NetCP plugin module infrastructure use all the standard kernel
>> infrastructure and its very tiny.

So i found this manual here:
http://www.silica.com/fileadmin/02_Products/Productdetails/Texas_Instruments/SILICA_TI_66AK2E05-ds.pdf

Glad there is an open document!
There are a couple of ethernet switch chips I can spot there.

Can i control those with "bridge" or say "brctl" utilities?

I can see the bridge ports are exposed and i should be able to
control them via ifconfig or ip link. Thats what "standard
kernel infrastructure" means. Magic hidden in a driver is
not.

Take a look at recent netconf discussion (as well as earlier
referenced discussions):
http://vger.kernel.org/netconf-nf-offload.pdf

Maybe we can help providing you some direction?
The problem is it doesnt seem that the offload specs for
those other pieces are open? e.g how do i add an entry
to the L2 switch?

cheers,
jamal

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

On Tuesday 09 September 2014 07:44 AM, Jamal Hadi Salim wrote:
> On 09/08/14 10:41, Santosh Shilimkar wrote:
> 
>>> The NetCP plugin module infrastructure use all the standard kernel
>>> infrastructure and its very tiny.
> 
> So i found this manual here:
> http://www.silica.com/fileadmin/02_Products/Productdetails/Texas_Instruments/SILICA_TI_66AK2E05-ds.pdf
> 
> Glad there is an open document!
> There are a couple of ethernet switch chips I can spot there.
> 
All the documentation is open including packet accelerator offload
in ti.com.

> Can i control those with "bridge" or say "brctl" utilities?
> 
> I can see the bridge ports are exposed and i should be able to
> control them via ifconfig or ip link. Thats what "standard
> kernel infrastructure" means. Magic hidden in a driver is
> not.
> 
There is nothing magic hidden in the driver. The bridge ports
are exposed as standard network interfaces. Currently the
drivers don't support bridge offload functionality and
the bridging is disabled in the switch by default. 

> Take a look at recent netconf discussion (as well as earlier
> referenced discussions):
> http://vger.kernel.org/netconf-nf-offload.pdf
> 
> Maybe we can help providing you some direction?
> The problem is it doesnt seem that the offload specs for
> those other pieces are open? e.g how do i add an entry
> to the L2 switch?
> 
We got such requests from customers but couldn't
support it for Linux. We are also looking for such
support and any direction are welcome. Your slide
deck seems to capture the key topics like L2/IPSEC
offload which we are also interested to hear.

Just to be clear, your point was about L2 switch offload
which the driver don't support at the moment. It might confuse
others. The driver doesn't implements anything non-standard.

Regards,
Santosh

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Jamal Hadi Salim]

On 09/09/14 11:19, Santosh Shilimkar wrote:

> All the documentation is open including packet accelerator offload
> in ti.com.

Very nice.
Would you do me a kindness and point to the switch interface
documentation (and other ones on that soc)?

> We got such requests from customers but couldn't
> support it for Linux.

It has been difficult because every chip vendor is trying
to do their own thing. Some have huge (fugly) SDKs in user space
which make it worse. Thats the struggle we are trying to
deal with. Of course none of those vendors want to open
up their specs. You present a nice opportunity to not follow
that path.

> We are also looking for such
> support and any direction are welcome. Your slide
> deck seems to capture the key topics like L2/IPSEC
> offload which we are also interested to hear.
>

The slides list the most popular offloads. But not necessarily
all known offloads.

> Just to be clear, your point was about L2 switch offload
> which the driver don't support at the moment. It might confuse
> others. The driver doesn't implements anything non-standard.
>

If i understood you correctly:
Your initial patches dont intend to expose any offloads - you are just
abstracting this as a NIC. I think that is a legit reason.
However, the problem is you are also exposing the packet processors
and switch offloading in a proprietary way.
For a sample of how L2 basic functions like FDB tables are controlled
within a NIC - take a look at the Intel NICs.
Either that or you hide all the offload interfaces and over time add
them (starting with L2 - NICs with L2 are common).

cheers,
jamal

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

On Wednesday 10 September 2014 07:33 AM, Jamal Hadi Salim wrote:
> On 09/09/14 11:19, Santosh Shilimkar wrote:
> 
>> All the documentation is open including packet accelerator offload
>> in ti.com.
> 
> Very nice.
> Would you do me a kindness and point to the switch interface
> documentation (and other ones on that soc)?
>
You can find it here [1], [2], [3]

>> We got such requests from customers but couldn't
>> support it for Linux.
> 
> It has been difficult because every chip vendor is trying
> to do their own thing. Some have huge (fugly) SDKs in user space
> which make it worse. Thats the struggle we are trying to
> deal with. Of course none of those vendors want to open
> up their specs. You present a nice opportunity to not follow
> that path.
> 
>> We are also looking for such
>> support and any direction are welcome. Your slide
>> deck seems to capture the key topics like L2/IPSEC
>> offload which we are also interested to hear.
>>
> 
> The slides list the most popular offloads. But not necessarily
> all known offloads.
> 
>> Just to be clear, your point was about L2 switch offload
>> which the driver don't support at the moment. It might confuse
>> others. The driver doesn't implements anything non-standard.
>>
> 
> If i understood you correctly:
> Your initial patches dont intend to expose any offloads - you are just
> abstracting this as a NIC. I think that is a legit reason.
Yes. The NetCP hardware is abstracted as a regular NIC.

> However, the problem is you are also exposing the packet processors
> and switch offloading in a proprietary way.
> For a sample of how L2 basic functions like FDB tables are controlled
> within a NIC - take a look at the Intel NICs.
> Either that or you hide all the offload interfaces and over time add
> them (starting with L2 - NICs with L2 are common).
>
Switch offload isn't supported but we do agree that for packet
accelerator, we are using custom hooks because of lack of other
mechanism.

We will definitely use the new ndo based fdb offload scheme when
we get to it. We understand that the forward direction is to
have ndo operation based offloads and its the right way probably.

We will update the patch and drop all the custom exports. Anyway
the current driver doesn't support any offloads now. We can add
support for it as the frameworks evolves.

Thanks a lot for informative discussion and those links.

Regards,
Santosh
[1] http://www.ti.com/lit/pdf/sprugv9
[2] http://www.ti.com/lit/pdf/spruhj5
[3] http://www.ti.com/lit/pdf/sprugs4

[PATCH v2 0/3] net: Add Keystone NetCP ethernet driver support

[Santosh Shilimkar]

Dave,

On Monday 08 September 2014 10:41 AM, Santosh Shilimkar wrote:
> Hi Dave,
> 
> On 8/22/14 3:45 PM, Santosh Shilimkar wrote:
>> Hi David,
>>
>> On Thursday 21 August 2014 07:36 PM, David Miller wrote:
>>> From: Santosh Shilimkar <santosh.shilimkar@ti.com>
>>> Date: Fri, 15 Aug 2014 11:12:39 -0400
>>>
>>>> Update version after incorporating David Miller's comment from earlier
>>>> posting [1]. I would like to get these merged for upcoming 3.18 merge
>>>> window if there are no concerns on this version.
>>>>
>>>> The network coprocessor (NetCP) is a hardware accelerator that processes
>>>> Ethernet packets. NetCP has a gigabit Ethernet (GbE) subsystem with a ethernet
>>>> switch sub-module to send and receive packets. NetCP also includes a packet
>>>> accelerator (PA) module to perform packet classification operations such as
>>>> header matching, and packet modification operations such as checksum
>>>> generation. NetCP can also optionally include a Security Accelerator(SA)
>>>> capable of performing IPSec operations on ingress/egress packets.
>>>>
>>>> Keystone SoC's also have a 10 Gigabit Ethernet Subsystem (XGbE) which
>>>> includes a 3-port Ethernet switch sub-module capable of 10Gb/s and
>>>> 1Gb/s rates per Ethernet port.
>>>>
>>>> NetCP driver has a plug-in module architecture where each of the NetCP
>>>> sub-modules exist as a loadable kernel module which plug in to the netcp
>>>> core. These sub-modules are represented as "netcp-devices" in the dts
>>>> bindings. It is mandatory to have the ethernet switch sub-module for
>>>> the ethernet interface to be operational. Any other sub-module like the
>>>> PA is optional.
>>>>
>>>> Both GBE and XGBE network processors supported using common driver. It
>>>> is also designed to handle future variants of NetCP.
>>>
>>> I don't want to see an offload driver that doesn't plug into the existing
>>> generic frameworks for configuration et al.
>>>
>>> If no existing facility exists to support what you need, you must work
>>> with the upstream maintainers to design and create one.
>>>
>>> It is absolutely no reasonable for every "switch on a chip" driver to
>>> export it's own configuration knob, we need a standard interface all
>>> such drivers will plug into and provide.
>>>
As discussed on other thread, we are dropping the custom exports. I
will be spinning updated version with the exports removed.

And for the future offload support additions, we will plug into
generic frameworks as when they are available.

Regards,
Santosh