[PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Sivakumar Subramani]

- Default to IntA interrupt type when there are less than 4 CPUs in the system.

Signed-off-by: Sivakumar Subramani <sivakumar.subramani@neterion.com>
Signed-off-by: Ramkrishna Vepa <ram.vepa@neterion.com>
---
diff -Nurp 2.0.26.2/drivers/net/s2io.c 2.0.26.3/drivers/net/s2io.c
--- 2.0.26.2/drivers/net/s2io.c	2007-08-07 13:39:45.000000000 -0700
+++ 2.0.26.3/drivers/net/s2io.c	2007-08-07 15:51:45.000000000 -0700
@@ -84,7 +84,7 @@
 #include "s2io.h"
 #include "s2io-regs.h"
 
-#define DRV_VERSION "2.0.26.2"
+#define DRV_VERSION "2.0.26.3"
 
 /* S2io Driver name & version. */
 static char s2io_driver_name[] = "Neterion";
@@ -452,7 +452,7 @@ S2IO_PARM_INT(l3l4hdr_size, 128);
 /* Frequency of Rx desc syncs expressed as power of 2 */
 S2IO_PARM_INT(rxsync_frequency, 3);
 /* Interrupt type. Values can be 0(INTA), 2(MSI_X) */
-S2IO_PARM_INT(intr_type, 2);
+S2IO_PARM_INT(intr_type, DEF_MSI_X);
 /* Large receive offload feature */
 S2IO_PARM_INT(lro, 0);
 /* Max pkts to be aggregated by LRO at one time. If not specified,
@@ -7671,10 +7671,11 @@ static int s2io_verify_parm(struct pci_d
 	if (*dev_intr_type != INTA)
 		napi = 0;
 
-	if ((*dev_intr_type != INTA) && (*dev_intr_type != MSI_X)) {
+	if ((intr_type != INTA) && (intr_type != MSI_X)) {
 		DBG_PRINT(ERR_DBG, "s2io: Wrong intr_type requested. "
 			  "Defaulting to INTA\n");
 		*dev_intr_type = INTA;
+		intr_type = INTA;
 	}
 
 	if ((*dev_intr_type == MSI_X) &&
@@ -7752,8 +7753,19 @@ s2io_init_nic(struct pci_dev *pdev, cons
 	struct mac_info *mac_control;
 	struct config_param *config;
 	int mode;
-	u8 dev_intr_type = intr_type;
+	u8 dev_intr_type;
+	int no_cpus = 1;
 
+	no_cpus = num_online_cpus();
+	if(DEF_MSI_X == intr_type) {
+		/* Don't enable MSI-X if there are less than 4 CPUs */
+		if(no_cpus < 4)
+			intr_type = INTA;
+		else
+			intr_type = MSI_X;
+	}
+
+	dev_intr_type = intr_type;
 	if ((ret = s2io_verify_parm(pdev, &dev_intr_type)))
 		return ret;
 
diff -Nurp 2.0.26.2/drivers/net/s2io.h 2.0.26.3/drivers/net/s2io.h
--- 2.0.26.2/drivers/net/s2io.h	2007-08-07 13:39:45.000000000 -0700
+++ 2.0.26.3/drivers/net/s2io.h	2007-08-07 13:40:18.000000000 -0700
@@ -433,6 +433,7 @@ struct config_param {
 	u64 tx_intr_type;
 #define INTA	0
 #define MSI_X	2
+#define	DEF_MSI_X	99
 	u8 intr_type;
 	u8 napi;
 

Re: [PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Rick Jones]

Sivakumar Subramani wrote:
> - Default to IntA interrupt type when there are less than 4 CPUs in the system.

It might be good to include _why_ in the comment(s).  I certainly am 
curious to know the reason, and it would be good to have in there for 
"posterity" should the underlying conditions change.

rick jones

Re: [PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Jeff Garzik]

Rick Jones wrote:
> Sivakumar Subramani wrote:
>> - Default to IntA interrupt type when there are less than 4 CPUs in 
>> the system.
> 
> It might be good to include _why_ in the comment(s).  I certainly am 
> curious to know the reason, and it would be good to have in there for 
> "posterity" should the underlying conditions change.

Indeed.  I really want to know 'why'?

	Jeff

RE: [PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Ramkrishna Vepa]

We had found during performance measurement/analysis that with 2 cpus
and napi disabled, and the system transmitting (tcp), the cpus get too
busy due to receive interrupt handling, reducing performance by around
10%.
 
But then again, when the number of cpus increase, the interrupt scaling
is much better and msi-x gives better numbers.

We are modifying the driver to enable napi with msi-x, but this work is
in progress and may get released in a few months. 

The Xframe hardware has 8 independent dma channels for transmit and
receive, with doorbells for each channel. Multiple channels are in use
for a single netdev. In one of the variations of this driver that is not
released to netdev, the received packets are steered to a channel based
on hashing on a preconfigured criteria such as sockets on tcp_ipv4,
udp_ipv4, tcp_ipv6, udp_ipv6 or addresses in ipv4/6. The transmits are
handled in a similar manner. 

The current napi implementation looks like  -
driver_poll(struct net_device *dev, int *budget)

where as, something like -
driver_poll(struct context *context, int *budget)

would be ideal for hardware like ours. Each channel could schedule a
respective poller, rather than the current method which requires
synchronization between the poller and the channels.

Ram
> -----Original Message-----
> From: Jeff Garzik [mailto:jeff@garzik.org]
> Sent: Friday, August 10, 2007 10:39 AM
> To: Rick Jones
> Cc: Sivakumar Subramani; netdev@vger.kernel.org; support
> Subject: Re: [PATCH 2.6.24]S2io: Default to IntA interrupt type when
there
> are less than 4 CPUs in the system.
> 
> Rick Jones wrote:
> > Sivakumar Subramani wrote:
> >> - Default to IntA interrupt type when there are less than 4 CPUs in
> >> the system.
> >
> > It might be good to include _why_ in the comment(s).  I certainly am
> > curious to know the reason, and it would be good to have in there
for
> > "posterity" should the underlying conditions change.
> 
> Indeed.  I really want to know 'why'?
> 
> 	Jeff
> 
> 

Re: [PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Andi Kleen]

"Ramkrishna Vepa" <Ramkrishna.Vepa@neterion.com> writes:

> In one of the variations of this driver that is not
> released to netdev, the received packets are steered to a channel based
> on hashing on a preconfigured criteria such as sockets on tcp_ipv4,
> udp_ipv4, tcp_ipv6, udp_ipv6 or addresses in ipv4/6.

Why is it not released for mainline? It sounds interesting.

-Andi

RE: [PATCH 2.6.24]S2io: Default to IntA interrupt type when there are less than 4 CPUs in the system.

[Ramkrishna Vepa]

We will be releasing it to the mainline soon. The patches are getting
queued at our end for test and release. It should be out in a couple of
weeks.

Ram
> -----Original Message-----
> From: ak@suse.de [mailto:ak@suse.de] On Behalf Of Andi Kleen
> Sent: Sunday, August 12, 2007 10:16 AM
> To: Ramkrishna Vepa
> Cc: Jeff Garzik; Rick Jones; Sivakumar Subramani;
netdev@vger.kernel.org;
> support
> Subject: Re: [PATCH 2.6.24]S2io: Default to IntA interrupt type when
there
> are less than 4 CPUs in the system.
> 
> "Ramkrishna Vepa" <Ramkrishna.Vepa@neterion.com> writes:
> 
> > In one of the variations of this driver that is not
> > released to netdev, the received packets are steered to a channel
based
> > on hashing on a preconfigured criteria such as sockets on tcp_ipv4,
> > udp_ipv4, tcp_ipv6, udp_ipv6 or addresses in ipv4/6.
> 
> Why is it not released for mainline? It sounds interesting.
> 
> -Andi