* Re: 2.6.12-rc5-mm2: "bic unavailable using TCP reno" messages
From: Adrian Bunk @ 2005-06-02 20:38 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Baruch Even, Andrew Morton, linux-kernel, netdev
In-Reply-To: <20050602103805.6beb4f4e@dxpl.pdx.osdl.net>
On Thu, Jun 02, 2005 at 10:38:05AM -0700, Stephen Hemminger wrote:
> On Thu, 02 Jun 2005 14:58:17 +0100
> Baruch Even <baruch@ev-en.org> wrote:
>
> >...
>
> Your right, the sysctl handler should be smarter, but that is not the problem here.
> The problem is that the default value is set to be BIC to be compatible with earlier kernels.
> Since 75% of the world isn't smart enough to figure out how to use sysctl, there is a
> question of what the default should be, and what to do if that is missing.
>
> One version had a messy ifdef chain to try and avoid the warning:
>
> char sysctl_tcp_congestion_control[] =
> #if defined(CONFIG_TCP_BIC)
> "bic"
> #elif defined(CONFIG_TCP_HTCP)
> "htcp"
> #else
> "reno"
> #endif
> ;
>
> but that was ugly.
>
> Another possibility is putting it in as yet another config value at kernel build time.
>...
One thing that currently makes all solutions harder (and the #ifdef
example above not ugly but simply wrong) is that you allow modular
congestion control options for the always static net support.
Is this really required?
The IO schedulers have a similar problem, and they are using the #ifdef
approach for selecting the default.
One approach is to actually choose the default using #ifdef's.
You could also do any kind of runtime selection, but please don't print
the warning more than once.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply
* RE: RFC: NAPI packet weighting patch
From: Ronciak, John @ 2005-06-02 21:19 UTC (permalink / raw)
To: hadi, Jon Mason
Cc: David S. Miller, Williams, Mitch A, shemminger, netdev,
Robert.Olsson, Venkatesan, Ganesh, Brandeburg, Jesse
The DRR algorithm assumes a perfect world, where hardware resources are
infinite, packets arrive continuously (or separated by very long
delays), there are no bus latencies, and CPU speed is infinite.
The real world is much messier: hardware starves for resources if it's
not serviced quickly enough, packets arrive at inconvenient intervals
(especially at 10 and 100 Mbps speeds), and buses and CPUs are slow.
Thus, the driver should have the intelligence built into it to make an
"intelligent" choice on what the weight should be for that
driver/hardware. The calculation in the driver should take into account
all the factors that the driver has access to. These include link
speed, bus type and speed, processor speed and some amount of actual
device FIFO size and latency smarts. The driver would use all of the
factors to come up with a weight to prevent it from dropping frames and
not to starve out other devices in the system or hinder performance. It
seems to us that the driver is the one that know best and should try to
come up with a reasonable value for weight based on its own knowledge of
the hardware.
This has been showing up in our NAPI test data which Mitch is currently
scrubbing for release. It shows that there is a need for either better
default static weight numbers or for them to be calculated based on some
system dynamic variables. We would like to see the latter tried but the
only problem is that each driver would have to make its own
calculations, and it may not have access to all of the system-wide data
it would need to make a proper calculation.
Even with a more intelligent driver, we still would like to see some
mechanism for the weight to be changed at runtime, such as with
Stephen's sysfs patch. This would allow a sysadmin (or user-space app)
to tune the system based on statistical data that isn't available to the
individual driver.
Cheers,
John
> -----Original Message-----
> From: jamal [mailto:hadi@cyberus.ca]
> Sent: Thursday, June 02, 2005 5:27 AM
> To: Jon Mason
> Cc: David S. Miller; Williams, Mitch A; shemminger@osdl.org;
> netdev@oss.sgi.com; Robert.Olsson@data.slu.se; Ronciak, John;
> Venkatesan, Ganesh; Brandeburg, Jesse
> Subject: Re: RFC: NAPI packet weighting patch
>
>
> On Tue, 2005-31-05 at 18:28 -0500, Jon Mason wrote:
> > On Tuesday 31 May 2005 05:14 pm, David S. Miller wrote:
> > > From: Jon Mason <jdmason@us.ibm.com>
> > > Date: Tue, 31 May 2005 17:07:54 -0500
> > >
> > > > Of course some performace analysis would have to be
> done to determine the
> > > > optimal numbers for each speed/duplexity setting per driver.
> > >
> > > per cpu speed, per memory bus speed, per I/O bus speed,
> and add in other
> > > complications such as NUMA
> > >
> > > My point is that whatever experimental number you come up
> with will be
> > > good for that driver on your systems, not necessarily for others.
> > >
> > > Even within a system, whatever number you select will be the wrong
> > > thing to use if one starts a continuous I/O stream to the SATA
> > > controller in the next PCI slot, for example.
> > >
> > > We keep getting bitten by this, as the Altix perf data
> continually shows,
> > > and we need to absolutely stop thinking this way.
> > >
> > > The way to go is to make selections based upon observed events and
> > > mesaurements.
> >
> > I'm not arguing against a /proc entry to tune dev->weight
> for those sysadmins
> > advanced enough to do that. I am arguing that we can make
> the driver smarter
> > (at little/no cost) for "out of the box" users.
> >
>
> What is the point of making the driver "smarter"?
> Recall, the algorithm used to schedule the netdevices is based on an
> extension of Weighted Round Robin from Varghese et al known
> as DRR (ask
> gooogle for details).
> The idea is to provide fairness amongst many drivers. As an
> example, if
> you have a gige driver it shouldnt be taking all the resources at the
> expense of starving the fastether driver.
> If the admin wants one driver to be more "important" than the other,
> s/he will make sure it has a higher weight.
>
> cheers,
> jamal
>
>
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Stephen Hemminger @ 2005-06-02 21:31 UTC (permalink / raw)
To: Ronciak, John
Cc: hadi, Jon Mason, David S. Miller, Williams, Mitch A, netdev,
Robert.Olsson, Venkatesan, Ganesh, Brandeburg, Jesse
In-Reply-To: <468F3FDA28AA87429AD807992E22D07E0450BFD0@orsmsx408>
On Thu, 2 Jun 2005 14:19:55 -0700
"Ronciak, John" <john.ronciak@intel.com> wrote:
> The DRR algorithm assumes a perfect world, where hardware resources are
> infinite, packets arrive continuously (or separated by very long
> delays), there are no bus latencies, and CPU speed is infinite.
>
> The real world is much messier: hardware starves for resources if it's
> not serviced quickly enough, packets arrive at inconvenient intervals
> (especially at 10 and 100 Mbps speeds), and buses and CPUs are slow.
>
> Thus, the driver should have the intelligence built into it to make an
> "intelligent" choice on what the weight should be for that
> driver/hardware. The calculation in the driver should take into account
> all the factors that the driver has access to. These include link
> speed, bus type and speed, processor speed and some amount of actual
> device FIFO size and latency smarts. The driver would use all of the
> factors to come up with a weight to prevent it from dropping frames and
> not to starve out other devices in the system or hinder performance. It
> seems to us that the driver is the one that know best and should try to
> come up with a reasonable value for weight based on its own knowledge of
> the hardware.
This is like saying each CPU vendor should write their own process scheduler
for Linux. Now with NUMA and HT, it is getting almost that bad but we still
try and keep it CPU neutral.
For networking the problem is worse, the "right" choice depends on workload
and relationship between components in the system. I can't see how you could
ever expect a driver specific solution.
> This has been showing up in our NAPI test data which Mitch is currently
> scrubbing for release. It shows that there is a need for either better
> default static weight numbers or for them to be calculated based on some
> system dynamic variables. We would like to see the latter tried but the
> only problem is that each driver would have to make its own
> calculations, and it may not have access to all of the system-wide data
> it would need to make a proper calculation.
And for other workloads, and other systems (think about the Altix with
long access latencies), your numbers will be wrong. Perhaps we need
to quit trying for a perfect solution and just get a "good enough" one
that works.
Let's keep the intelligence out of the driver. Most of the existing
smart drivers end up looking like crap and don't work that well.
> Even with a more intelligent driver, we still would like to see some
> mechanism for the weight to be changed at runtime, such as with
> Stephen's sysfs patch. This would allow a sysadmin (or user-space app)
> to tune the system based on statistical data that isn't available to the
> individual driver.
>
It will be yet another knob that all except the benchmark tweakers can
ignore (hopefully).
^ permalink raw reply
* [PATCH][5/5] RapidIO support: net driver over messaging
From: Matt Porter @ 2005-06-02 21:34 UTC (permalink / raw)
To: torvalds, akpm, jgarzik; +Cc: netdev, linux-kernel, linuxppc-embedded
In-Reply-To: <20050602142509.E24818@cox.net>
Adds an "Ethernet" driver which sends Ethernet packets over the
standard RapidIO messaging. This depends on the core RIO
patch for mailbox/doorbell access.
Signed-off-by: Matt Porter <mporter@kernel.crashing.org>
Index: drivers/net/Kconfig
===================================================================
--- 711ec47634f5d5ded866eaa965a0f7dadcbc65f4/drivers/net/Kconfig (mode:100644)
+++ 8bdd37ff79724c95795ed39c28588a94e1f13e60/drivers/net/Kconfig (mode:100644)
@@ -2185,6 +2185,20 @@
tristate "iSeries Virtual Ethernet driver support"
depends on NETDEVICES && PPC_ISERIES
+config RIONET
+ tristate "RapidIO Ethernet over messaging driver support"
+ depends on NETDEVICES && RAPIDIO
+
+config RIONET_TX_SIZE
+ int "Number of outbound queue entries"
+ depends on RIONET
+ default "128"
+
+config RIONET_RX_SIZE
+ int "Number of inbound queue entries"
+ depends on RIONET
+ default "128"
+
config FDDI
bool "FDDI driver support"
depends on NETDEVICES && (PCI || EISA)
Index: drivers/net/Makefile
===================================================================
--- 711ec47634f5d5ded866eaa965a0f7dadcbc65f4/drivers/net/Makefile (mode:100644)
+++ 8bdd37ff79724c95795ed39c28588a94e1f13e60/drivers/net/Makefile (mode:100644)
@@ -58,6 +58,7 @@
obj-$(CONFIG_VIA_RHINE) += via-rhine.o
obj-$(CONFIG_VIA_VELOCITY) += via-velocity.o
obj-$(CONFIG_ADAPTEC_STARFIRE) += starfire.o
+obj-$(CONFIG_RIONET) += rionet.o
#
# end link order section
Index: drivers/net/rionet.c
===================================================================
--- /dev/null (tree:711ec47634f5d5ded866eaa965a0f7dadcbc65f4)
+++ 8bdd37ff79724c95795ed39c28588a94e1f13e60/drivers/net/rionet.c (mode:100644)
@@ -0,0 +1,622 @@
+/*
+ * rionet - Ethernet driver over RapidIO messaging services
+ *
+ * Copyright 2005 MontaVista Software, Inc.
+ * Matt Porter <mporter@kernel.crashing.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/dma-mapping.h>
+#include <linux/delay.h>
+#include <linux/rio.h>
+#include <linux/rio_drv.h>
+#include <linux/rio_ids.h>
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/skbuff.h>
+#include <linux/crc32.h>
+#include <linux/ethtool.h>
+
+#define DRV_NAME "rionet"
+#define DRV_VERSION "0.1"
+#define DRV_AUTHOR "Matt Porter <mporter@kernel.crashing.org>"
+#define DRV_DESC "Ethernet over RapidIO"
+
+MODULE_AUTHOR(DRV_AUTHOR);
+MODULE_DESCRIPTION(DRV_DESC);
+MODULE_LICENSE("GPL");
+
+#define RIONET_DEFAULT_MSGLEVEL 0
+#define RIONET_DOORBELL_JOIN 0x1000
+#define RIONET_DOORBELL_LEAVE 0x1001
+
+#define RIONET_MAILBOX 0
+
+#define RIONET_TX_RING_SIZE CONFIG_RIONET_TX_SIZE
+#define RIONET_RX_RING_SIZE CONFIG_RIONET_RX_SIZE
+
+LIST_HEAD(rionet_peers);
+
+struct rionet_private {
+ struct rio_mport *mport;
+ struct sk_buff *rx_skb[RIONET_RX_RING_SIZE];
+ struct sk_buff *tx_skb[RIONET_TX_RING_SIZE];
+ struct net_device_stats stats;
+ int rx_slot;
+ int tx_slot;
+ int tx_cnt;
+ int ack_slot;
+ spinlock_t lock;
+ u32 msg_enable;
+};
+
+struct rionet_peer {
+ struct list_head node;
+ struct rio_dev *rdev;
+ struct resource *res;
+};
+
+static int rionet_check = 0;
+static int rionet_capable = 1;
+static struct net_device *sndev = NULL;
+
+/*
+ * This is a fast lookup table for for translating TX
+ * Ethernet packets into a destination RIO device. It
+ * could be made into a hash table to save memory depending
+ * on system trade-offs.
+ */
+static struct rio_dev *rionet_active[RIO_MAX_ROUTE_ENTRIES];
+
+#define is_rionet_capable(pef, src_ops, dst_ops) \
+ ((pef & RIO_PEF_INB_MBOX) && \
+ (pef & RIO_PEF_INB_DOORBELL) && \
+ (src_ops & RIO_SRC_OPS_DOORBELL) && \
+ (dst_ops & RIO_DST_OPS_DOORBELL))
+#define dev_rionet_capable(dev) \
+ is_rionet_capable(dev->pef, dev->src_ops, dev->dst_ops)
+
+#define RIONET_MAC_MATCH(x) (*(u32 *)x == 0x00010001)
+#define RIONET_GET_DESTID(x) (*(u16 *)(x + 4))
+
+static struct net_device_stats *rionet_stats(struct net_device *ndev)
+{
+ struct rionet_private *rnet = ndev->priv;
+ return &rnet->stats;
+}
+
+static int rionet_rx_clean(struct net_device *ndev)
+{
+ int i;
+ int error = 0;
+ struct rionet_private *rnet = ndev->priv;
+ void *data;
+
+ i = rnet->rx_slot;
+
+ do {
+ if (!rnet->rx_skb[i]) {
+ rnet->stats.rx_dropped++;
+ continue;
+ }
+
+ if (!(data = rio_get_inb_message(rnet->mport, RIONET_MAILBOX)))
+ break;
+
+ rnet->rx_skb[i]->data = data;
+ skb_put(rnet->rx_skb[i], RIO_MAX_MSG_SIZE);
+ rnet->rx_skb[i]->dev = sndev;
+ rnet->rx_skb[i]->protocol =
+ eth_type_trans(rnet->rx_skb[i], sndev);
+ error = netif_rx(rnet->rx_skb[i]);
+
+ if (error == NET_RX_DROP) {
+ rnet->stats.rx_dropped++;
+ } else if (error == NET_RX_BAD) {
+ if (netif_msg_rx_err(rnet))
+ printk(KERN_WARNING "%s: bad rx packet\n",
+ DRV_NAME);
+ rnet->stats.rx_errors++;
+ } else {
+ rnet->stats.rx_packets++;
+ rnet->stats.rx_bytes += RIO_MAX_MSG_SIZE;
+ }
+
+ } while ((i = (i + 1) % RIONET_RX_RING_SIZE) != rnet->rx_slot);
+
+ return i;
+}
+
+static void rionet_rx_fill(struct net_device *ndev, int end)
+{
+ int i;
+ struct rionet_private *rnet = ndev->priv;
+
+ i = rnet->rx_slot;
+ do {
+ rnet->rx_skb[i] = dev_alloc_skb(RIO_MAX_MSG_SIZE);
+
+ if (!rnet->rx_skb[i])
+ break;
+
+ rio_add_inb_buffer(rnet->mport, RIONET_MAILBOX,
+ rnet->rx_skb[i]->data);
+ } while ((i = (i + 1) % RIONET_RX_RING_SIZE) != end);
+
+ rnet->rx_slot = i;
+}
+
+static int rionet_queue_tx_msg(struct sk_buff *skb, struct net_device *ndev,
+ struct rio_dev *rdev)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ rio_add_outb_message(rnet->mport, rdev, 0, skb->data, skb->len);
+ rnet->tx_skb[rnet->tx_slot] = skb;
+
+ rnet->stats.tx_packets++;
+ rnet->stats.tx_bytes += skb->len;
+
+ if (++rnet->tx_cnt == RIONET_TX_RING_SIZE)
+ netif_stop_queue(ndev);
+
+ if (++rnet->tx_slot == RIONET_TX_RING_SIZE)
+ rnet->tx_slot = 0;
+
+ if (netif_msg_tx_queued(rnet))
+ printk(KERN_INFO "%s: queued skb %8.8x len %8.8x\n", DRV_NAME,
+ (u32) skb, skb->len);
+
+ return 0;
+}
+
+static int rionet_start_xmit(struct sk_buff *skb, struct net_device *ndev)
+{
+ int i;
+ struct rionet_private *rnet = ndev->priv;
+ struct ethhdr *eth = (struct ethhdr *)skb->data;
+ u16 destid;
+
+ spin_lock_irq(&rnet->lock);
+
+ if ((rnet->tx_cnt + 1) > RIONET_TX_RING_SIZE) {
+ netif_stop_queue(ndev);
+ spin_unlock_irq(&rnet->lock);
+ return -EBUSY;
+ }
+
+ if (eth->h_dest[0] & 0x01) {
+ /*
+ * XXX Need to delay queuing if ring max is reached,
+ * flush additional packets in tx_event() before
+ * awakening the queue. We can easily exceed ring
+ * size with a large number of nodes or even a
+ * small number where the ring is relatively full
+ * on entrance to hard_start_xmit.
+ */
+ for (i = 0; i < RIO_MAX_ROUTE_ENTRIES; i++)
+ if (rionet_active[i])
+ rionet_queue_tx_msg(skb, ndev,
+ rionet_active[i]);
+ } else if (RIONET_MAC_MATCH(eth->h_dest)) {
+ destid = RIONET_GET_DESTID(eth->h_dest);
+ if (rionet_active[destid])
+ rionet_queue_tx_msg(skb, ndev, rionet_active[destid]);
+ }
+
+ spin_unlock_irq(&rnet->lock);
+
+ return 0;
+}
+
+static int rionet_set_mac_address(struct net_device *ndev, void *p)
+{
+ struct sockaddr *addr = p;
+
+ if (!is_valid_ether_addr(addr->sa_data))
+ return -EADDRNOTAVAIL;
+
+ memcpy(ndev->dev_addr, addr->sa_data, ndev->addr_len);
+
+ return 0;
+}
+
+static int rionet_change_mtu(struct net_device *ndev, int new_mtu)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ if (netif_msg_drv(rnet))
+ printk(KERN_WARNING
+ "%s: rionet_change_mtu(): not implemented\n", DRV_NAME);
+
+ return 0;
+}
+
+static void rionet_set_multicast_list(struct net_device *ndev)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ if (netif_msg_drv(rnet))
+ printk(KERN_WARNING
+ "%s: rionet_set_multicast_list(): not implemented\n",
+ DRV_NAME);
+}
+
+static void rionet_dbell_event(struct rio_mport *mport, u16 sid, u16 tid,
+ u16 info)
+{
+ struct net_device *ndev = sndev;
+ struct rionet_private *rnet = ndev->priv;
+ struct rionet_peer *peer;
+
+ if (netif_msg_intr(rnet))
+ printk(KERN_INFO "%s: doorbell sid %4.4x tid %4.4x info %4.4x",
+ DRV_NAME, sid, tid, info);
+ if (info == RIONET_DOORBELL_JOIN) {
+ if (!rionet_active[sid]) {
+ list_for_each_entry(peer, &rionet_peers, node) {
+ if (peer->rdev->destid == sid)
+ rionet_active[sid] = peer->rdev;
+ }
+ rio_mport_send_doorbell(mport, sid,
+ RIONET_DOORBELL_JOIN);
+ }
+ } else if (info == RIONET_DOORBELL_LEAVE) {
+ rionet_active[sid] = NULL;
+ } else {
+ if (netif_msg_intr(rnet))
+ printk(KERN_WARNING "%s: unhandled doorbell\n",
+ DRV_NAME);
+ }
+}
+
+static void rionet_inb_msg_event(struct rio_mport *mport, int mbox, int slot)
+{
+ int n;
+ struct net_device *ndev = sndev;
+ struct rionet_private *rnet = (struct rionet_private *)ndev->priv;
+
+ if (netif_msg_intr(rnet))
+ printk(KERN_INFO "%s: inbound message event, mbox %d slot %d\n",
+ DRV_NAME, mbox, slot);
+
+ spin_lock(&rnet->lock);
+ if ((n = rionet_rx_clean(ndev)) != rnet->rx_slot)
+ rionet_rx_fill(ndev, n);
+ spin_unlock(&rnet->lock);
+}
+
+static void rionet_outb_msg_event(struct rio_mport *mport, int mbox, int slot)
+{
+ struct net_device *ndev = sndev;
+ struct rionet_private *rnet = ndev->priv;
+
+ spin_lock(&rnet->lock);
+
+ if (netif_msg_intr(rnet))
+ printk(KERN_INFO
+ "%s: outbound message event, mbox %d slot %d\n",
+ DRV_NAME, mbox, slot);
+
+ while (rnet->tx_cnt && (rnet->ack_slot != slot)) {
+ /* dma unmap single */
+ dev_kfree_skb_irq(rnet->tx_skb[rnet->ack_slot]);
+ rnet->tx_skb[rnet->ack_slot] = NULL;
+ if (++rnet->ack_slot == RIONET_TX_RING_SIZE)
+ rnet->ack_slot = 0;
+ rnet->tx_cnt--;
+ }
+
+ if (rnet->tx_cnt < RIONET_TX_RING_SIZE)
+ netif_wake_queue(ndev);
+
+ spin_unlock(&rnet->lock);
+}
+
+static int rionet_open(struct net_device *ndev)
+{
+ int i, rc = 0;
+ struct rionet_peer *peer, *tmp;
+ u32 pwdcsr;
+ struct rionet_private *rnet = ndev->priv;
+
+ if (netif_msg_ifup(rnet))
+ printk(KERN_INFO "%s: open\n", DRV_NAME);
+
+ if ((rc = rio_request_inb_dbell(rnet->mport,
+ RIONET_DOORBELL_JOIN,
+ RIONET_DOORBELL_LEAVE,
+ rionet_dbell_event)) < 0)
+ goto out;
+
+ if ((rc = rio_request_inb_mbox(rnet->mport,
+ RIONET_MAILBOX,
+ RIONET_RX_RING_SIZE,
+ rionet_inb_msg_event)) < 0)
+ goto out;
+
+ if ((rc = rio_request_outb_mbox(rnet->mport,
+ RIONET_MAILBOX,
+ RIONET_TX_RING_SIZE,
+ rionet_outb_msg_event)) < 0)
+ goto out;
+
+ /* Initialize inbound message ring */
+ for (i = 0; i < RIONET_RX_RING_SIZE; i++)
+ rnet->rx_skb[i] = NULL;
+ rnet->rx_slot = 0;
+ rionet_rx_fill(ndev, 0);
+
+ rnet->tx_slot = 0;
+ rnet->tx_cnt = 0;
+ rnet->ack_slot = 0;
+
+ spin_lock_init(&rnet->lock);
+
+ rnet->msg_enable = RIONET_DEFAULT_MSGLEVEL;
+
+ netif_carrier_on(ndev);
+ netif_start_queue(ndev);
+
+ list_for_each_entry_safe(peer, tmp, &rionet_peers, node) {
+ if (!(peer->res = rio_request_outb_dbell(peer->rdev,
+ RIONET_DOORBELL_JOIN,
+ RIONET_DOORBELL_LEAVE)))
+ {
+ printk(KERN_ERR "%s: error requesting doorbells\n",
+ DRV_NAME);
+ continue;
+ }
+
+ /*
+ * If device has initialized inbound doorbells,
+ * send a join message
+ */
+ rio_read_config_32(peer->rdev, RIO_WRITE_PORT_CSR, &pwdcsr);
+ if (pwdcsr & RIO_DOORBELL_AVAIL)
+ rio_send_doorbell(peer->rdev, RIONET_DOORBELL_JOIN);
+ }
+
+ out:
+ return rc;
+}
+
+static int rionet_close(struct net_device *ndev)
+{
+ struct rionet_private *rnet = (struct rionet_private *)ndev->priv;
+ struct rionet_peer *peer, *tmp;
+ int i;
+
+ if (netif_msg_ifup(rnet))
+ printk(KERN_INFO "%s: close\n", DRV_NAME);
+
+ netif_stop_queue(ndev);
+ netif_carrier_off(ndev);
+
+ for (i = 0; i < RIONET_RX_RING_SIZE; i++)
+ if (rnet->rx_skb[i])
+ kfree_skb(rnet->rx_skb[i]);
+
+ list_for_each_entry_safe(peer, tmp, &rionet_peers, node) {
+ if (rionet_active[peer->rdev->destid]) {
+ rio_send_doorbell(peer->rdev, RIONET_DOORBELL_LEAVE);
+ rionet_active[peer->rdev->destid] = NULL;
+ }
+ rio_release_outb_dbell(peer->rdev, peer->res);
+ }
+
+ rio_release_inb_dbell(rnet->mport, RIONET_DOORBELL_JOIN,
+ RIONET_DOORBELL_LEAVE);
+ rio_release_inb_mbox(rnet->mport, RIONET_MAILBOX);
+ rio_release_outb_mbox(rnet->mport, RIONET_MAILBOX);
+
+ return 0;
+}
+
+static void rionet_remove(struct rio_dev *rdev)
+{
+ struct net_device *ndev = NULL;
+ struct rionet_peer *peer, *tmp;
+
+ unregister_netdev(ndev);
+ kfree(ndev);
+
+ list_for_each_entry_safe(peer, tmp, &rionet_peers, node) {
+ list_del(&peer->node);
+ kfree(peer);
+ }
+}
+
+static int rionet_ioctl(struct net_device *ndev, struct ifreq *rq, int cmd)
+{
+ return -EOPNOTSUPP;
+}
+
+static void rionet_get_drvinfo(struct net_device *ndev,
+ struct ethtool_drvinfo *info)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ strcpy(info->driver, DRV_NAME);
+ strcpy(info->version, DRV_VERSION);
+ strcpy(info->fw_version, "n/a");
+ sprintf(info->bus_info, "RIO master port %d", rnet->mport->id);
+}
+
+static u32 rionet_get_msglevel(struct net_device *ndev)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ return rnet->msg_enable;
+}
+
+static void rionet_set_msglevel(struct net_device *ndev, u32 value)
+{
+ struct rionet_private *rnet = ndev->priv;
+
+ rnet->msg_enable = value;
+}
+
+static u32 rionet_get_link(struct net_device *ndev)
+{
+ return netif_carrier_ok(ndev);
+}
+
+static struct ethtool_ops rionet_ethtool_ops = {
+ .get_drvinfo = rionet_get_drvinfo,
+ .get_msglevel = rionet_get_msglevel,
+ .set_msglevel = rionet_set_msglevel,
+ .get_link = rionet_get_link,
+};
+
+static int rionet_setup_netdev(struct rio_mport *mport)
+{
+ int rc = 0;
+ struct net_device *ndev = NULL;
+ struct rionet_private *rnet;
+ u16 device_id;
+
+ /* Allocate our net_device structure */
+ ndev = alloc_etherdev(sizeof(struct rionet_private));
+ if (ndev == NULL) {
+ printk(KERN_INFO "%s: could not allocate ethernet device.\n",
+ DRV_NAME);
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ /*
+ * XXX hack, store point a static at ndev so we can get it...
+ * Perhaps need an array of these that the handler can
+ * index via the mbox number.
+ */
+ sndev = ndev;
+
+ /* Set up private area */
+ rnet = (struct rionet_private *)ndev->priv;
+ rnet->mport = mport;
+
+ /* Set the default MAC address */
+ device_id = rio_local_get_device_id(mport);
+ ndev->dev_addr[0] = 0x00;
+ ndev->dev_addr[1] = 0x01;
+ ndev->dev_addr[2] = 0x00;
+ ndev->dev_addr[3] = 0x01;
+ ndev->dev_addr[4] = device_id >> 8;
+ ndev->dev_addr[5] = device_id & 0xff;
+
+ /* Fill in the driver function table */
+ ndev->open = &rionet_open;
+ ndev->hard_start_xmit = &rionet_start_xmit;
+ ndev->stop = &rionet_close;
+ ndev->get_stats = &rionet_stats;
+ ndev->change_mtu = &rionet_change_mtu;
+ ndev->set_mac_address = &rionet_set_mac_address;
+ ndev->set_multicast_list = &rionet_set_multicast_list;
+ ndev->do_ioctl = &rionet_ioctl;
+ SET_ETHTOOL_OPS(ndev, &rionet_ethtool_ops);
+
+ ndev->mtu = RIO_MAX_MSG_SIZE - 14;
+
+ SET_MODULE_OWNER(ndev);
+
+ rc = register_netdev(ndev);
+ if (rc != 0)
+ goto out;
+
+ printk("%s: %s %s Version %s, MAC %02x:%02x:%02x:%02x:%02x:%02x\n",
+ ndev->name,
+ DRV_NAME,
+ DRV_DESC,
+ DRV_VERSION,
+ ndev->dev_addr[0], ndev->dev_addr[1], ndev->dev_addr[2],
+ ndev->dev_addr[3], ndev->dev_addr[4], ndev->dev_addr[5]);
+
+ out:
+ return rc;
+}
+
+/*
+ * XXX Make multi-net safe
+ */
+static int rionet_probe(struct rio_dev *rdev, const struct rio_device_id *id)
+{
+ int rc = -ENODEV;
+ u32 lpef, lsrc_ops, ldst_ops;
+ struct rionet_peer *peer;
+
+ /* If local device is not rionet capable, give up quickly */
+ if (!rionet_capable)
+ goto out;
+
+ /*
+ * First time through, make sure local device is rionet
+ * capable, setup netdev, and set flags so this is skipped
+ * on later probes
+ */
+ if (!rionet_check) {
+ rio_local_read_config_32(rdev->net->hport, RIO_PEF_CAR, &lpef);
+ rio_local_read_config_32(rdev->net->hport, RIO_SRC_OPS_CAR,
+ &lsrc_ops);
+ rio_local_read_config_32(rdev->net->hport, RIO_DST_OPS_CAR,
+ &ldst_ops);
+ if (!is_rionet_capable(lpef, lsrc_ops, ldst_ops)) {
+ printk(KERN_ERR
+ "%s: local device is not network capable\n",
+ DRV_NAME);
+ rionet_check = 1;
+ rionet_capable = 0;
+ goto out;
+ }
+
+ rc = rionet_setup_netdev(rdev->net->hport);
+ rionet_check = 1;
+ }
+
+ /*
+ * If the remote device has mailbox/doorbell capabilities,
+ * add it to the peer list.
+ */
+ if (dev_rionet_capable(rdev)) {
+ if (!(peer = kmalloc(sizeof(struct rionet_peer), GFP_KERNEL))) {
+ rc = -ENOMEM;
+ goto out;
+ }
+ peer->rdev = rdev;
+ list_add_tail(&peer->node, &rionet_peers);
+ }
+
+ out:
+ return rc;
+}
+
+static struct rio_device_id rionet_id_table[] = {
+ {RIO_DEVICE(RIO_ANY_ID, RIO_ANY_ID)}
+};
+
+static struct rio_driver rionet_driver = {
+ .name = "rionet",
+ .id_table = rionet_id_table,
+ .probe = rionet_probe,
+ .remove = rionet_remove,
+};
+
+static int __init rionet_init(void)
+{
+ return rio_register_driver(&rionet_driver);
+}
+
+static void __exit rionet_exit(void)
+{
+ rio_unregister_driver(&rionet_driver);
+}
+
+module_init(rionet_init);
+module_exit(rionet_exit);
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: David S. Miller @ 2005-06-02 21:40 UTC (permalink / raw)
To: shemminger
Cc: john.ronciak, hadi, jdmason, mitch.a.williams, netdev,
Robert.Olsson, ganesh.venkatesan, jesse.brandeburg
In-Reply-To: <20050602143126.7c302cfd@dxpl.pdx.osdl.net>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Thu, 2 Jun 2005 14:31:26 -0700
> For networking the problem is worse, the "right" choice depends on workload
> and relationship between components in the system. I can't see how you could
> ever expect a driver specific solution.
I totally agree, even the mere concept of driver-centric decisions
in this area is pretty bogus.
> And for other workloads, and other systems (think about the Altix with
> long access latencies), your numbers will be wrong. Perhaps we need
> to quit trying for a perfect solution and just get a "good enough" one
> that works.
I don't understand why nobody is investigating doing this stuff
by generic measurements that the core kernel can perform.
The generic ->poll() runner code can say, wow it took N-usec to
process M packets, perhaps I should adjust the weight.
I haven't seen one concrete suggestion along those lines, yet that is
where the answer to this kind of stuff is. Those kinds of solutions
are completely CPU, memory, I/O bus, network device, and workload
independant.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Jon Mason @ 2005-06-02 21:51 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Ronciak, John, hadi, David S. Miller, Williams, Mitch A, netdev,
Robert.Olsson, Venkatesan, Ganesh, Brandeburg, Jesse
In-Reply-To: <20050602143126.7c302cfd@dxpl.pdx.osdl.net>
On Thursday 02 June 2005 04:31 pm, Stephen Hemminger wrote:
<...>
> For networking the problem is worse, the "right" choice depends on workload
> and relationship between components in the system. I can't see how you
> could ever expect a driver specific solution.
I think there is a way for a generic driver NAPI enhancement. That is to
modify the weight dependent on link speed.
Here is the problem as I see it, NAPI enablement for slow media speeds causes
unneeded strain on the system. This is because of the "weight" of NAPI.
Lets look at e1000 as an example. Currently the NAPI weight is 64,
regardless of link media speed. This weight is probably fine for a gigabit
link, but for 10/100 this is way to large. Thus causing interrupts to be
enabled/disabled after every poll/interrupt. Lots of overhead, and not very
smart. Why not have the driver set the weight to 16/32 respectively for the
weight (or better yet, have someone run numbers to find weight that are
closer to what the adapter can actually use)? While these numbers may not be
optimal for every system, this is much better that the current system, and
would only require 5 or so extra lines of code per NAPI enabled driver.
For those who want to have an optimal weight for their tuned system, let them
use the /proc entry that is being proposed.
Thanks,
Jon
^ permalink raw reply
* Re: [PATCH][5/5] RapidIO support: net driver over messaging
From: Stephen Hemminger @ 2005-06-02 22:05 UTC (permalink / raw)
To: Matt Porter
Cc: akpm, netdev, linux-kernel, torvalds, linuxppc-embedded, jgarzik
In-Reply-To: <20050602143404.F24818@cox.net>
How much is this like ethernet? does it still do ARP?
Can it do promiscious receive?
> +LIST_HEAD(rionet_peers);
Does this have to be global?
Not sure about the locking of this stuff, are you
relying on the RTNL?
> +
> +static int rionet_change_mtu(struct net_device *ndev, int new_mtu)
> +{
> + struct rionet_private *rnet = ndev->priv;
> +
> + if (netif_msg_drv(rnet))
> + printk(KERN_WARNING
> + "%s: rionet_change_mtu(): not implemented\n", DRV_NAME);
> +
> + return 0;
> +}
If you can allow any mtu then don't need this at all.
Or if you are limited then better return an error for bad values.
> +static void rionet_set_multicast_list(struct net_device *ndev)
> +{
> + struct rionet_private *rnet = ndev->priv;
> +
> + if (netif_msg_drv(rnet))
> + printk(KERN_WARNING
> + "%s: rionet_set_multicast_list(): not implemented\n",
> + DRV_NAME);
> +}
If you can't handle it then just leave dev->set_multicast_list
as NULL and all attempts to add or delete will get -EINVAL
> +
> +static int rionet_open(struct net_device *ndev)
> +{
> + /* Initialize inbound message ring */
> + for (i = 0; i < RIONET_RX_RING_SIZE; i++)
> + rnet->rx_skb[i] = NULL;
> + rnet->rx_slot = 0;
> + rionet_rx_fill(ndev, 0);
> +
> + rnet->tx_slot = 0;
> + rnet->tx_cnt = 0;
> + rnet->ack_slot = 0;
> +
> + spin_lock_init(&rnet->lock);
> +
> + rnet->msg_enable = RIONET_DEFAULT_MSGLEVEL;
Better to do all initialization of the per device data
in the place it is allocated (rio_setup_netdev)
> +
> +static int rionet_ioctl(struct net_device *ndev, struct ifreq *rq, int cmd)
> +{
> + return -EOPNOTSUPP;
> +}
Unneeded, if dev->do_ioctl is NULL, then all private ioctl's will
return -EINVAL that is what you want.
> +
> +static u32 rionet_get_link(struct net_device *ndev)
> +{
> + return netif_carrier_ok(ndev);
> +}
Use ethtool_op_get_link
> +
> +static int rionet_setup_netdev(struct rio_mport *mport)
> +{
> + int rc = 0;
> + struct net_device *ndev = NULL;
> + struct rionet_private *rnet;
> + u16 device_id;
> +
> + /* Allocate our net_device structure */
> + ndev = alloc_etherdev(sizeof(struct rionet_private));
> + if (ndev == NULL) {
> + printk(KERN_INFO "%s: could not allocate ethernet device.\n",
> + DRV_NAME);
> + rc = -ENOMEM;
> + goto out;
> + }
> +
> + /*
> + * XXX hack, store point a static at ndev so we can get it...
> + * Perhaps need an array of these that the handler can
> + * index via the mbox number.
> + */
> + sndev = ndev;
> +
> + /* Set up private area */
> + rnet = (struct rionet_private *)ndev->priv;
> + rnet->mport = mport;
> +
> + /* Set the default MAC address */
> + device_id = rio_local_get_device_id(mport);
> + ndev->dev_addr[0] = 0x00;
> + ndev->dev_addr[1] = 0x01;
> + ndev->dev_addr[2] = 0x00;
> + ndev->dev_addr[3] = 0x01;
> + ndev->dev_addr[4] = device_id >> 8;
> + ndev->dev_addr[5] = device_id & 0xff;
> +
> + /* Fill in the driver function table */
> + ndev->open = &rionet_open;
> + ndev->hard_start_xmit = &rionet_start_xmit;
> + ndev->stop = &rionet_close;
> + ndev->get_stats = &rionet_stats;
> + ndev->change_mtu = &rionet_change_mtu;
> + ndev->set_mac_address = &rionet_set_mac_address;
> + ndev->set_multicast_list = &rionet_set_multicast_list;
> + ndev->do_ioctl = &rionet_ioctl;
> + SET_ETHTOOL_OPS(ndev, &rionet_ethtool_ops);
> +
> + ndev->mtu = RIO_MAX_MSG_SIZE - 14;
> +
> + SET_MODULE_OWNER(ndev);
Can you set any ndev->features to get better performance.
Can you take >32bit data addresses? then set HIGHDMA
You are doing your on locking, can you use LLTX?
Does the hardware support scatter gather?
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: David S. Miller @ 2005-06-02 22:12 UTC (permalink / raw)
To: jdmason
Cc: shemminger, john.ronciak, hadi, mitch.a.williams, netdev,
Robert.Olsson, ganesh.venkatesan, jesse.brandeburg
In-Reply-To: <200506021651.49013.jdmason@us.ibm.com>
From: Jon Mason <jdmason@us.ibm.com>
Date: Thu, 2 Jun 2005 16:51:48 -0500
> Why not have the driver set the weight to 16/32 respectively for the
> weight (or better yet, have someone run numbers to find weight that
> are closer to what the adapter can actually use)? While these
> numbers may not be optimal for every system, this is much better
> that the current system, and would only require 5 or so extra lines
> of code per NAPI enabled driver.
Why do this when we can adjust the weight in one spot,
namely the upper level NAPI ->poll() running loop?
It can measure the overhead, how many packets processed, etc.
and make intelligent decisions based upon that. This is a CPU
speed, memory speed, I/O bus speed, and link speed agnostic
solution.
The driver need not take any part in this, and the scheme will
dynamically adjust to resource usage changes in the system.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Robert Olsson @ 2005-06-02 22:15 UTC (permalink / raw)
To: Jon Mason
Cc: Stephen Hemminger, Ronciak, John, hadi, David S. Miller,
Williams, Mitch A, netdev, Robert.Olsson, Venkatesan, Ganesh,
Brandeburg, Jesse
In-Reply-To: <200506021651.49013.jdmason@us.ibm.com>
Differentiate the meaning of weight a bit.
Let weight only limit the number of pkts we deliver per ->poll
Have some other mechanism or threshold to control when interrupts are
to be turned on.
The first approximation for this could be to poll as long as we see
any pkt on the RX ring. As interrupt seems expensive on all platforms.
Cheers.
--ro
Jon Mason writes:
> On Thursday 02 June 2005 04:31 pm, Stephen Hemminger wrote:
> <...>
> > For networking the problem is worse, the "right" choice depends on workload
> > and relationship between components in the system. I can't see how you
> > could ever expect a driver specific solution.
>
> I think there is a way for a generic driver NAPI enhancement. That is to
> modify the weight dependent on link speed.
>
> Here is the problem as I see it, NAPI enablement for slow media speeds causes
> unneeded strain on the system. This is because of the "weight" of NAPI.
> Lets look at e1000 as an example. Currently the NAPI weight is 64,
> regardless of link media speed. This weight is probably fine for a gigabit
> link, but for 10/100 this is way to large. Thus causing interrupts to be
> enabled/disabled after every poll/interrupt. Lots of overhead, and not very
> smart. Why not have the driver set the weight to 16/32 respectively for the
> weight (or better yet, have someone run numbers to find weight that are
> closer to what the adapter can actually use)? While these numbers may not be
> optimal for every system, this is much better that the current system, and
> would only require 5 or so extra lines of code per NAPI enabled driver.
>
> For those who want to have an optimal weight for their tuned system, let them
> use the /proc entry that is being proposed.
>
> Thanks,
> Jon
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Jon Mason @ 2005-06-02 22:19 UTC (permalink / raw)
To: David S. Miller
Cc: shemminger, john.ronciak, hadi, mitch.a.williams, netdev,
Robert.Olsson, ganesh.venkatesan, jesse.brandeburg
In-Reply-To: <20050602.151212.35014607.davem@davemloft.net>
On Thursday 02 June 2005 05:12 pm, David S. Miller wrote:
> From: Jon Mason <jdmason@us.ibm.com>
> Date: Thu, 2 Jun 2005 16:51:48 -0500
>
> > Why not have the driver set the weight to 16/32 respectively for the
> > weight (or better yet, have someone run numbers to find weight that
> > are closer to what the adapter can actually use)? While these
> > numbers may not be optimal for every system, this is much better
> > that the current system, and would only require 5 or so extra lines
> > of code per NAPI enabled driver.
>
> Why do this when we can adjust the weight in one spot,
> namely the upper level NAPI ->poll() running loop?
>
> It can measure the overhead, how many packets processed, etc.
> and make intelligent decisions based upon that. This is a CPU
> speed, memory speed, I/O bus speed, and link speed agnostic
> solution.
>
> The driver need not take any part in this, and the scheme will
> dynamically adjust to resource usage changes in the system.
Yes, a much better idea to do this generically. I 100% agree with you.
^ permalink raw reply
* RE: [PATCH 2.6.12-rc4] IPv4/IPv6: UDP Large Send Offload feature
From: Ravinandan Arakali @ 2005-06-02 23:18 UTC (permalink / raw)
To: 'David S. Miller'
Cc: jgarzik, netdev, raghavendra.koushik, leonid.grossman,
ananda.raju, rapuru.sriram
In-Reply-To: <20050527.120215.26278001.davem@davemloft.net>
David,
Since there seems to be pros and cons for both the approaches, we are
planning
to submit two separate patches(one for each approach). These patches also
include the ethtool changes. In terms of performance, we did not observe any
diff between the two approaches although the first approach(using SG)
minimizes
coalescing in driver.
Also, some changes will be required in the ethtool user-level utility.
I'm not sure if this is the right forum to submit patches for the ethtool
utility as well..
Thanks,
Ravi
-----Original Message-----
From: David S. Miller [mailto:davem@davemloft.net]
Sent: Friday, May 27, 2005 12:02 PM
To: ravinandan.arakali@neterion.com
Cc: jgarzik@pobox.com; netdev@oss.sgi.com;
raghavendra.koushik@neterion.com; leonid.grossman@neterion.com;
ananda.raju@neterion.com; rapuru.sriram@neterion.com
Subject: Re: [PATCH 2.6.12-rc4] IPv4/IPv6: UDP Large Send Offload
feature
From: "Ravinandan Arakali" <ravinandan.arakali@neterion.com>
Date: Fri, 27 May 2005 09:32:00 -0700
> Thanks for the quick feedback.
> At that time when we considered using skb_shinfo(skb)->fraglist,
> it contained fragments of MTU size. So, for a 60k udp datagram
> and 1500 MTU we will have 60k/1500 = 45 fragments which is
> more than MAX_SKB_FRAGS(18).
>
> However we will relook at fraglist for the possibility of increasing
> frag size to >MTU.
MAX_SKB_FRAGS controls the limit of skb_shinfo(skb)->frags[]
entries, not how many SKBs may be chained via
skb_shinfo(skb)->fraglist, there is no limit on the latter.
Note that there is much coalescing that can be performed on
the SKB list data areas, particularly if UDP sendfile() is
being used.
But such coalescing is messy to be performing inside of the
drivers. It may end up being the case that your approach
ends up being a better one for these reasons.
^ permalink raw reply
* Re: [PATCH 2.6.12-rc4] IPv4/IPv6: UDP Large Send Offload feature
From: David S. Miller @ 2005-06-02 23:22 UTC (permalink / raw)
To: ravinandan.arakali
Cc: jgarzik, netdev, raghavendra.koushik, leonid.grossman,
ananda.raju, rapuru.sriram
In-Reply-To: <003201c567c9$73322240$3910100a@pc.s2io.com>
From: "Ravinandan Arakali" <ravinandan.arakali@neterion.com>
Date: Thu, 2 Jun 2005 16:18:55 -0700
> Since there seems to be pros and cons for both the approaches, we are
> planning
> to submit two separate patches(one for each approach). These patches also
> include the ethtool changes. In terms of performance, we did not observe any
> diff between the two approaches although the first approach(using SG)
> minimizes
> coalescing in driver.
Ok. I think minimizing driver specific work is probably going
to make the SG approach more desirable, but we'll see.
> Also, some changes will be required in the ethtool user-level utility.
> I'm not sure if this is the right forum to submit patches for the ethtool
> utility as well..
Making sure jgarzik@pobox.com gets the patch is usually the way
to go wrt. ethtool submissions.
^ permalink raw reply
* Re: Comparison of several congestion control algorithms
From: David S. Miller @ 2005-06-02 23:35 UTC (permalink / raw)
To: baruch; +Cc: netdev, shemminger, doug.leith
In-Reply-To: <4298E045.9050009@ev-en.org>
From: Baruch Even <baruch@ev-en.org>
Date: Sat, 28 May 2005 22:19:01 +0100
> I wanted to point you to a comparison of congestion control algorithm
> done at the Hamilton Institute. These experiments compare Scalable-TCP,
> High-Speed TCP, FAST-TCP, BIC-TCP, H-TCP and Standard TCP. They compared
> fairness, compatibility with TCP and link utilisation.
>
> You can find the results and a report at http://hamilton.ie/net/eval/
Nice work, I enjoyed this paper very much.
There is something that none of these papers mention, but is essential
for interpreting results. Did you use interfaces with TSO enabled?
There is a very serious congestion window growth bug with TSO enabled
in the current 2.6.x tree. The problem is due to congestion window
validation. When we build TSO frames, even if we have packets to send,
we may defer a few frames until the full TSO packet can go out.
But this causes the congestion window validation checks in tcp_ack()
to not pass, and thus the congestion window does not grow.
I am going to have this fixed, but for now people should do congestion
window algorithm tests with TSO explicitly disabled on their interfaces.
^ permalink raw reply
* Re: [PATCH] shaper.c: fix locking
From: David S. Miller @ 2005-06-02 23:36 UTC (permalink / raw)
To: hch; +Cc: netdev
In-Reply-To: <20050601052149.GA11935@lst.de>
From: Christoph Hellwig <hch@lst.de>
Date: Wed, 1 Jun 2005 07:21:50 +0200
> On Tue, May 31, 2005 at 02:41:14PM -0700, David S. Miller wrote:
> > From: Christoph Hellwig <hch@lst.de>
> > Subject: [PATCH] shaper.c: fix locking
> > Date: Fri, 27 May 2005 13:54:50 +0200
> >
> > > o use a semaphore instead of an opencoded and racy lock
> > > o move locking out of shaper_kick and into the callers - most just
> > > released the lock before calling shaper_kick
> > > o remove in_interrupt() tests. from ->close we can always block, from
> > > ->hard_start_xmit and timer context never
> >
> > Do you really want to use a semaphore for a lock taken
> > %99 of the time in software IRQ context, which obviously
> > cannot sleep?
>
> I want to change as little as possible from the previous variant ;-)
Fair enough, patch applied. If this driver breaks as a result of
these changes, you get to keep the pieces ok? :-)
^ permalink raw reply
* Re: Comparison of several congestion control algorithms
From: Baruch Even @ 2005-06-02 23:50 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, shemminger, doug.leith
In-Reply-To: <20050602.163512.10298458.davem@davemloft.net>
David S. Miller wrote:
> From: Baruch Even <baruch@ev-en.org>
> Date: Sat, 28 May 2005 22:19:01 +0100
>
>
>>I wanted to point you to a comparison of congestion control algorithm
>>done at the Hamilton Institute. These experiments compare Scalable-TCP,
>>High-Speed TCP, FAST-TCP, BIC-TCP, H-TCP and Standard TCP. They compared
>> fairness, compatibility with TCP and link utilisation.
>>
>>You can find the results and a report at http://hamilton.ie/net/eval/
>
>
> Nice work, I enjoyed this paper very much.
>
> There is something that none of these papers mention, but is essential
> for interpreting results. Did you use interfaces with TSO enabled?
I did not do these experiments myself, but to the best of my knowledge,
none of the experiments done so far in Hamilton have used the TSO feature.
This is in part because of the start of the work that was based on 2.4
kernels and even as far as the 2.6.6 kernel which had disabled TSO once
it saw SACKs. This made TSO unusable for our needs.
AFAIK, the tests reported in that document used kernel 2.6.6.
Baruch
^ permalink raw reply
* Re: Comparison of several congestion control algorithms
From: David S. Miller @ 2005-06-02 23:53 UTC (permalink / raw)
To: baruch; +Cc: netdev, shemminger, doug.leith
In-Reply-To: <429F9B2F.8030507@ev-en.org>
From: Baruch Even <baruch@ev-en.org>
Date: Fri, 03 Jun 2005 00:50:07 +0100
> This is in part because of the start of the work that was based on 2.4
> kernels and even as far as the 2.6.6 kernel which had disabled TSO once
> it saw SACKs. This made TSO unusable for our needs.
>
> AFAIK, the tests reported in that document used kernel 2.6.6.
Sure SACKs turn off TSO currently, but you'll have them enabled
at the beginning until the first loss and this affects how fast
the cwnd will grow.
If you have e1000 cards, for example, you're getting TSO enabled
by default.
You really need to look into this, as it has a real and very
non-trivial effect on all of the results you obtained.
^ permalink raw reply
* RE: RFC: NAPI packet weighting patch
From: Ronciak, John @ 2005-06-03 0:11 UTC (permalink / raw)
To: Jon Mason, David S. Miller
Cc: shemminger, hadi, Williams, Mitch A, netdev, Robert.Olsson,
Venkatesan, Ganesh, Brandeburg, Jesse
I like this idea as well but I do an issue with it. How would this
stack code find out that the weight is too high and pacekts are being
dropped (not being polled fast enough)? It would have to check the
controller stats to see the error count increasing for some period. I'm
not sure this is workable unless we have some sort of feedback which the
driver could send up (or set) saying that this is happening and the
dynamic weight code could take into acount.
Comments?
Cheers,
John
> -----Original Message-----
> From: Jon Mason [mailto:jdmason@us.ibm.com]
> Sent: Thursday, June 02, 2005 3:20 PM
> To: David S. Miller
> Cc: shemminger@osdl.org; Ronciak, John; hadi@cyberus.ca;
> Williams, Mitch A; netdev@oss.sgi.com;
> Robert.Olsson@data.slu.se; Venkatesan, Ganesh; Brandeburg, Jesse
> Subject: Re: RFC: NAPI packet weighting patch
>
>
> On Thursday 02 June 2005 05:12 pm, David S. Miller wrote:
> > From: Jon Mason <jdmason@us.ibm.com>
> > Date: Thu, 2 Jun 2005 16:51:48 -0500
> >
> > > Why not have the driver set the weight to 16/32
> respectively for the
> > > weight (or better yet, have someone run numbers to find
> weight that
> > > are closer to what the adapter can actually use)? While these
> > > numbers may not be optimal for every system, this is much better
> > > that the current system, and would only require 5 or so
> extra lines
> > > of code per NAPI enabled driver.
> >
> > Why do this when we can adjust the weight in one spot,
> > namely the upper level NAPI ->poll() running loop?
> >
> > It can measure the overhead, how many packets processed, etc.
> > and make intelligent decisions based upon that. This is a CPU
> > speed, memory speed, I/O bus speed, and link speed agnostic
> > solution.
> >
> > The driver need not take any part in this, and the scheme will
> > dynamically adjust to resource usage changes in the system.
>
> Yes, a much better idea to do this generically. I 100% agree
> with you.
>
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: David S. Miller @ 2005-06-03 0:18 UTC (permalink / raw)
To: john.ronciak
Cc: jdmason, shemminger, hadi, mitch.a.williams, netdev,
Robert.Olsson, ganesh.venkatesan, jesse.brandeburg
In-Reply-To: <468F3FDA28AA87429AD807992E22D07E0450BFDB@orsmsx408>
From: "Ronciak, John" <john.ronciak@intel.com>
Date: Thu, 2 Jun 2005 17:11:20 -0700
> I like this idea as well but I do an issue with it. How would this
> stack code find out that the weight is too high and pacekts are being
> dropped (not being polled fast enough)? It would have to check the
> controller stats to see the error count increasing for some period. I'm
> not sure this is workable unless we have some sort of feedback which the
> driver could send up (or set) saying that this is happening and the
> dynamic weight code could take into acount.
What more do you need other than checking the statistics counter? The
drop statistics (the ones we care about) are incremented in real time
by the ->poll() code, so it's not like we have to trigger some
asynchronous event to get a current version of the number.
^ permalink raw reply
* [PATCH 2.6.12-rc4] IPv4/IPv6: USO v2, Scatter-gather approach
From: ravinandan.arakali @ 2005-06-03 0:41 UTC (permalink / raw)
To: davem, jgarzik, netdev
Cc: raghavendra.koushik, ravinandan.arakali, leonid.grossman,
ananda.raju, rapuru.sriram
Hi,
Attached below is version 2 of kernel patch for UDP Large send offload
feature. This patch uses the "Scatter-Gather" approach.
It also incorporates David Miller's comments on the first version.
Also, below is a "how-to" on changes required in network drivers to use
the USO interface.
UDP Large Send Offload (USO) Interface:
--------------------------------------
USO is a feature wherein the Linux kernel network stack will offload the
IP fragmentation functionality of large UDP datagram to hardware. This
will reduce the overhead of stack in fragmenting the large UDP datagram
to MTU sized packets.
1) Drivers indicate their capability of USO using
dev->features |= NETIF_F_USO | NETIF_F_HW_CSUM | NETIF_F_SG
NETIF_F_HW_CSUM is required for USO over ipv6.
2) USO packet will be submitted for transmission using driver xmit
routine. USO packet will have a non-zero value for
"skb_shinfo(skb)->uso_size"
skb_shinfo(skb)->uso_size will indicate the length of data part in each IP
fragment going out of the adapter after IP fragmentation by hardware.
skb->data will contain MAC/IP/UDP header and skb_shinfo(skb)->frags[]
contains the data payload. The skb->ip_summed will be set to CHECKSUM_HW
indicating that hardware has to do checksum calculation. Hardware should
compute the UDP checksum of complete datagram and also ip header checksum
of each fragmented IP packet.
For IPV6 the USO provides the fragment identification-id in
skb_shinfo(skb)->ip6_frag_id. The adapter should use this ID for generating
IPv6 fragments.
Signed-off-by: Ananda Raju <ananda.raju@neterion.com>
Signed-off-by: Ravinandan Arakali <ravinandan.arakali@neterion.com>
---
diff -uNr linux-2.6.12-rc4.org/include/linux/ethtool.h linux-2.6.12-rc4/include/linux/ethtool.h
--- linux-2.6.12-rc4.org/include/linux/ethtool.h 2005-06-01 19:56:58.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/ethtool.h 2005-06-01 19:51:47.000000000 +0545
@@ -260,6 +260,8 @@
int ethtool_op_set_sg(struct net_device *dev, u32 data);
u32 ethtool_op_get_tso(struct net_device *dev);
int ethtool_op_set_tso(struct net_device *dev, u32 data);
+u32 ethtool_op_get_uso(struct net_device *dev);
+int ethtool_op_set_uso(struct net_device *dev, u32 data);
/**
* ðtool_ops - Alter and report network device settings
@@ -289,6 +291,8 @@
* set_sg: Turn scatter-gather on or off
* get_tso: Report whether TCP segmentation offload is enabled
* set_tso: Turn TCP segmentation offload on or off
+ * get_uso: Report whether UDP large send offload is enabled
+ * set_uso: Turn UDP large send offload on or off
* self_test: Run specified self-tests
* get_strings: Return a set of strings that describe the requested objects
* phys_id: Identify the device
@@ -353,6 +357,8 @@
void (*get_ethtool_stats)(struct net_device *, struct ethtool_stats *, u64 *);
int (*begin)(struct net_device *);
void (*complete)(struct net_device *);
+ u32 (*get_uso)(struct net_device *);
+ int (*set_uso)(struct net_device *, u32);
};
/* CMDs currently supported */
@@ -388,6 +394,8 @@
#define ETHTOOL_GSTATS 0x0000001d /* get NIC-specific statistics */
#define ETHTOOL_GTSO 0x0000001e /* Get TSO enable (ethtool_value) */
#define ETHTOOL_STSO 0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GUSO 0x00000020 /* Get USO enable (ethtool_value) */
+#define ETHTOOL_SUSO 0x00000021 /* Set USO enable (ethtool_value) */
/* compatibility with older code */
#define SPARC_ETH_GSET ETHTOOL_GSET
diff -uNr linux-2.6.12-rc4.org/include/linux/netdevice.h linux-2.6.12-rc4/include/linux/netdevice.h
--- linux-2.6.12-rc4.org/include/linux/netdevice.h 2005-05-25 17:18:11.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/netdevice.h 2005-06-01 14:33:12.000000000 +0545
@@ -414,6 +414,7 @@
#define NETIF_F_VLAN_CHALLENGED 1024 /* Device cannot handle VLAN packets */
#define NETIF_F_TSO 2048 /* Can offload TCP/IP segmentation */
#define NETIF_F_LLTX 4096 /* LockLess TX */
+#define NETIF_F_USO 8192 /* Can offload UDP Large Send*/
/* Called after device is detached from network. */
void (*uninit)(struct net_device *dev);
diff -uNr linux-2.6.12-rc4.org/include/linux/skbuff.h linux-2.6.12-rc4/include/linux/skbuff.h
--- linux-2.6.12-rc4.org/include/linux/skbuff.h 2005-05-25 17:18:20.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/skbuff.h 2005-06-01 15:18:44.000000000 +0545
@@ -135,6 +135,8 @@
atomic_t dataref;
unsigned int nr_frags;
unsigned short tso_size;
+ unsigned short uso_size;
+ unsigned int ip6_frag_id;
unsigned short tso_segs;
struct sk_buff *frag_list;
skb_frag_t frags[MAX_SKB_FRAGS];
diff -uNr linux-2.6.12-rc4.org/include/net/sock.h linux-2.6.12-rc4/include/net/sock.h
--- linux-2.6.12-rc4.org/include/net/sock.h 2005-05-25 17:18:44.000000000 +0545
+++ linux-2.6.12-rc4/include/net/sock.h 2005-05-25 20:28:14.000000000 +0545
@@ -1296,5 +1296,11 @@
return -ENODEV;
}
#endif
+struct sk_buff *sock_append_data(struct sock *sk,
+ int getfrag(void *from, char *to, int offset, int len,
+ int odd, struct sk_buff *skb),
+ void *from, int length, int transhdrlen,
+ int hh_len, int fragheaderlen,
+ unsigned int flags,int *err);
#endif /* _SOCK_H */
diff -uNr linux-2.6.12-rc4.org/net/core/dev.c linux-2.6.12-rc4/net/core/dev.c
--- linux-2.6.12-rc4.org/net/core/dev.c 2005-06-01 14:35:01.000000000 +0545
+++ linux-2.6.12-rc4/net/core/dev.c 2005-06-01 19:46:03.000000000 +0545
@@ -2793,6 +2793,18 @@
dev->name);
dev->features &= ~NETIF_F_TSO;
}
+ if (dev->features & NETIF_F_USO) {
+ if (!(dev->features & NETIF_F_HW_CSUM)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_HW_CSUM feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ if (!(dev->features & NETIF_F_SG)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_SG feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ }
/*
* nil rebuild_header routine,
diff -uNr linux-2.6.12-rc4.org/net/core/ethtool.c linux-2.6.12-rc4/net/core/ethtool.c
--- linux-2.6.12-rc4.org/net/core/ethtool.c 2005-06-01 19:48:31.000000000 +0545
+++ linux-2.6.12-rc4/net/core/ethtool.c 2005-06-01 23:02:39.000000000 +0545
@@ -72,6 +72,21 @@
return 0;
}
+u32 ethtool_op_get_uso(struct net_device *dev)
+{
+ return (dev->features & NETIF_F_USO) != 0;
+}
+
+int ethtool_op_set_uso(struct net_device *dev, u32 data)
+{
+ if (data)
+ dev->features |= NETIF_F_USO;
+ else
+ dev->features &= ~NETIF_F_USO;
+
+ return 0;
+}
+
/* Handlers for each ethtool command */
static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)
@@ -460,6 +475,9 @@
err = dev->ethtool_ops->set_tso(dev, 0);
if (err)
return err;
+ err = dev->ethtool_ops->set_uso(dev, 0);
+ if (err)
+ return err;
}
return dev->ethtool_ops->set_sg(dev, data);
@@ -548,6 +566,39 @@
return dev->ethtool_ops->set_tso(dev, edata.data);
}
+static int ethtool_get_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata = { ETHTOOL_GTSO };
+
+ if (!dev->ethtool_ops->get_uso)
+ return -EOPNOTSUPP;
+
+ edata.data = dev->ethtool_ops->get_uso(dev);
+
+ if (copy_to_user(useraddr, &edata, sizeof(edata)))
+ return -EFAULT;
+ return 0;
+}
+
+static int ethtool_set_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata;
+
+ if (!dev->ethtool_ops->set_uso)
+ return -EOPNOTSUPP;
+
+ if (copy_from_user(&edata, useraddr, sizeof(edata)))
+ return -EFAULT;
+
+ if (edata.data && !(dev->features & NETIF_F_SG))
+ return -EINVAL;
+
+ if (edata.data && !(dev->features & NETIF_F_HW_CSUM))
+ return -EINVAL;
+
+ return dev->ethtool_ops->set_uso(dev, edata.data);
+}
+
static int ethtool_self_test(struct net_device *dev, char __user *useraddr)
{
struct ethtool_test test;
@@ -795,6 +846,12 @@
case ETHTOOL_GSTATS:
rc = ethtool_get_stats(dev, useraddr);
break;
+ case ETHTOOL_GUSO:
+ rc = ethtool_get_uso(dev, useraddr);
+ break;
+ case ETHTOOL_SUSO:
+ rc = ethtool_set_uso(dev, useraddr);
+ break;
default:
rc = -EOPNOTSUPP;
}
@@ -817,3 +874,6 @@
EXPORT_SYMBOL(ethtool_op_set_sg);
EXPORT_SYMBOL(ethtool_op_set_tso);
EXPORT_SYMBOL(ethtool_op_set_tx_csum);
+EXPORT_SYMBOL(ethtool_op_set_uso);
+EXPORT_SYMBOL(ethtool_op_get_uso);
+
diff -uNr linux-2.6.12-rc4.org/net/core/skbuff.c linux-2.6.12-rc4/net/core/skbuff.c
--- linux-2.6.12-rc4.org/net/core/skbuff.c 2005-05-25 20:25:35.000000000 +0545
+++ linux-2.6.12-rc4/net/core/skbuff.c 2005-06-01 14:34:27.000000000 +0545
@@ -159,6 +159,8 @@
skb_shinfo(skb)->tso_size = 0;
skb_shinfo(skb)->tso_segs = 0;
skb_shinfo(skb)->frag_list = NULL;
+ skb_shinfo(skb)->uso_size = 0;
+ skb_shinfo(skb)->ip6_frag_id = 0;
out:
return skb;
nodata:
diff -uNr linux-2.6.12-rc4.org/net/core/sock.c linux-2.6.12-rc4/net/core/sock.c
--- linux-2.6.12-rc4.org/net/core/sock.c 2005-05-25 20:25:47.000000000 +0545
+++ linux-2.6.12-rc4/net/core/sock.c 2005-06-01 19:40:03.000000000 +0545
@@ -1401,6 +1401,102 @@
EXPORT_SYMBOL(proto_unregister);
+/*
+ * sock_append_data - append the user data to a skb,
+ * sk - sock structure which contains skbs for transmission
+ * getfrag - The function to be called to get the data from the user.
+ * from - pointer to user message iov
+ * length - length of the iov message
+ * transhdrlen - transport header length
+ * hh_len - hardware header length
+ * fragheaderlen - length of the IP header
+ * flags - iov message flags
+ * err - Error code returned
+ *
+ * This procedure will allocate a skb enough to hold protocol headers and
+ * append the user data in the fragment part of the skb and add the skb to
+ * socket write queue
+ */
+struct sk_buff *sock_append_data(struct sock *sk,
+ int getfrag(void *from, char *to, int offset, int len,
+ int odd, struct sk_buff *skb),
+ void *from, int length, int transhdrlen,
+ int hh_len, int fragheaderlen,
+ unsigned int flags,int *err)
+{
+ struct sk_buff *skb;
+ int frg_cnt = 0;
+ skb_frag_t *frag = NULL;
+ struct page *page = NULL;
+ int copy, left;
+ int offset = 0;
+
+ if (skb_queue_len(&sk->sk_write_queue)) {
+ *err = -EOPNOTSUPP;
+ return NULL;
+ }
+
+ skb = sock_alloc_send_skb(sk,
+ hh_len + fragheaderlen + transhdrlen + 20,
+ (flags & MSG_DONTWAIT), err);
+ if (skb == NULL) {
+ *err = -ENOMEM;
+ return NULL;
+ }
+ /* reserve space for Hardware header */
+ skb_reserve(skb, hh_len);
+ /* create space for UDP/IP header */
+ skb_put(skb,fragheaderlen + transhdrlen);
+ /* initialize network header pointer */
+ skb->nh.raw = skb->data;
+ /* initialize protocol header pointer */
+ skb->h.raw = skb->data + fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ skb->csum = 0;
+ do {
+ copy = length;
+ if (frg_cnt >= MAX_SKB_FRAGS) {
+ *err = -EFAULT;
+ kfree_skb(skb);
+ return NULL;
+ }
+ page = alloc_pages(sk->sk_allocation, 0);
+ if (page == NULL) {
+ *err = -ENOMEM;
+ kfree_skb(skb);
+ return NULL;
+ }
+ sk->sk_sndmsg_page = page;
+ sk->sk_sndmsg_off = 0;
+ skb_fill_page_desc(skb, frg_cnt, page, 0, 0);
+ skb->truesize += PAGE_SIZE;
+ atomic_add(PAGE_SIZE, &sk->sk_wmem_alloc);
+ frg_cnt = skb_shinfo(skb)->nr_frags;
+ frag = &skb_shinfo(skb)->frags[frg_cnt - 1];
+ left = PAGE_SIZE - frag->page_offset;
+ if (copy > left)
+ copy = left;
+ if (getfrag(from, page_address(frag->page)+
+ frag->page_offset+frag->size,
+ offset, copy, 0, skb) < 0) {
+ *err = -EFAULT;
+ kfree_skb(skb);
+ return NULL;
+ }
+ sk->sk_sndmsg_off += copy;
+ frag->size += copy;
+ skb->len += copy;
+ skb->data_len += copy;
+ offset += copy;
+ length -= copy;
+ page = NULL;
+ } while (length > 0);
+ __skb_queue_tail(&sk->sk_write_queue, skb);
+ *err = 0;
+ return skb;
+}
+EXPORT_SYMBOL(sock_append_data);
+
#ifdef CONFIG_PROC_FS
static inline struct proto *__proto_head(void)
{
diff -uNr linux-2.6.12-rc4.org/net/ipv4/ip_output.c linux-2.6.12-rc4/net/ipv4/ip_output.c
--- linux-2.6.12-rc4.org/net/ipv4/ip_output.c 2005-05-25 20:26:07.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv4/ip_output.c 2005-06-02 22:04:59.000000000 +0545
@@ -291,7 +291,8 @@
{
IP_INC_STATS(IPSTATS_MIB_OUTREQUESTS);
- if (skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->tso_size)
+ if (skb->len > dst_mtu(skb->dst) &&
+ !(skb_shinfo(skb)->uso_size || skb_shinfo(skb)->tso_size))
return ip_fragment(skb, ip_finish_output);
else
return ip_finish_output(skb);
@@ -789,6 +790,28 @@
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ skb = sock_append_data(sk, getfrag, from,
+ (length - transhdrlen), transhdrlen,
+ hh_len, fragheaderlen, flags, &err);
+ if (skb != NULL) {
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen);
+ return 0;
+ } else if (err == -EOPNOTSUPP) {
+ /* There is not enough support do UPD LSO,
+ * so follow normal path
+ */
+ err = 0;
+ } else
+ goto error;
+ }
+
/* So, what's going on in the loop below?
*
* We use calculated fragment length to generate chained skb,
diff -uNr linux-2.6.12-rc4.org/net/ipv6/ip6_output.c linux-2.6.12-rc4/net/ipv6/ip6_output.c
--- linux-2.6.12-rc4.org/net/ipv6/ip6_output.c 2005-05-25 20:26:17.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv6/ip6_output.c 2005-06-02 22:05:24.000000000 +0545
@@ -147,7 +147,8 @@
int ip6_output(struct sk_buff *skb)
{
- if (skb->len > dst_mtu(skb->dst) || dst_allfrag(skb->dst))
+ if ((skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->uso_size) ||
+ dst_allfrag(skb->dst))
return ip6_fragment(skb, ip6_output2);
else
return ip6_output2(skb);
@@ -898,6 +899,33 @@
*/
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ skb = sock_append_data(sk, getfrag, from,
+ (length - transhdrlen), transhdrlen,
+ hh_len, fragheaderlen, flags, &err);
+ if (skb != NULL) {
+ struct frag_hdr fhdr;
+
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen -
+ sizeof(struct frag_hdr));
+ ipv6_select_ident(skb, &fhdr);
+ skb_shinfo(skb)->ip6_frag_id = fhdr.identification;
+ return 0;
+ } else if (err == -EOPNOTSUPP){
+ /* There is not enough support for UDP LSO,
+ * so follow normal path
+ */
+ err = 0;
+ } else
+ goto error;
+ }
if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL)
goto alloc_new_skb;
^ permalink raw reply
* [PATCH 2.6.12-rc4] IPv4/IPv6: USO v2, fragment list approach
From: ravinandan.arakali @ 2005-06-03 0:43 UTC (permalink / raw)
To: davem, jgarzik, netdev
Cc: raghavendra.koushik, ravinandan.arakali, leonid.grossman,
ananda.raju, rapuru.sriram
Hi,
Attached below is version 2 of kernel patch for UDP Large send offload
feature. This patch uses the "fragment list" approach.
It also incorporates David Miller's comments on the first version.
Also, below is a "how-to" on changes required in network drivers to use
the USO interface.
UDP Large Send Offload (USO) Interface:
---------------------------------------
USO is a feature wherein the Linux kernel network stack will offload the
IP fragmentation functionality of large UDP datagram to hardware. This will
reduce the overhead of stack in fragmenting the large UDP datagram to MTU
sized packets.
1) Drivers indicate their capability of USO using
dev->features |= NETIF_F_USO | NETIF_F_HW_CSUM | NETIF_F_FRAGLIST
NETIF_F_HW_CSUM is required for USO over IPv6.
2) USO packet will be submitted for transmission using driver xmit
routine. USO packet will have a non zero value for
"skb_shinfo(skb)->uso_size"
skb_shinfo(skb)->uso_size indicates the length of data part in each IP
fragment going out of the adapter after IP fragmentation by hardware.
skb->data and skb_shinfo(skb)->frag_list will contain complete large
UDP datagram.
The driver is required to traverse each skb in skb_shinfo(skb)->frag_list
to get complete UDP packet. The skb->ip_summed will be set to CHECKSUM_HW
indicating that hardware has to perform checksum calculation. Hardware should
compute the UDP checksum of complete UDP datagram and also ip header checksum
of each fragmented IP packet.
For IPV6 the USO provides the fragment identification id in
skb_shinfo(skb)->ip6_frag_id. The adapter should use this ID for generating
IPv6 fragments.
Signed-off-by: Ananda Raju <ananda.raju@neterion.com>
Signed-off-by: Ravinandan Arakali <ravinandan.arakali@neterion.com>
---
diff -uNr linux-2.6.12-rc4.org/include/linux/ethtool.h linux-2.6.12-rc4/include/linux/ethtool.h
--- linux-2.6.12-rc4.org/include/linux/ethtool.h 2005-06-02 16:55:51.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/ethtool.h 2005-06-02 16:56:46.000000000 +0545
@@ -260,6 +260,8 @@
int ethtool_op_set_sg(struct net_device *dev, u32 data);
u32 ethtool_op_get_tso(struct net_device *dev);
int ethtool_op_set_tso(struct net_device *dev, u32 data);
+u32 ethtool_op_get_uso(struct net_device *dev);
+int ethtool_op_set_uso(struct net_device *dev, u32 data);
/**
* ðtool_ops - Alter and report network device settings
@@ -289,6 +291,8 @@
* set_sg: Turn scatter-gather on or off
* get_tso: Report whether TCP segmentation offload is enabled
* set_tso: Turn TCP segmentation offload on or off
+ * get_uso: Report whether UDP large send offload is enabled
+ * set_uso: Turn UDP large send offload on or off
* self_test: Run specified self-tests
* get_strings: Return a set of strings that describe the requested objects
* phys_id: Identify the device
@@ -353,6 +357,8 @@
void (*get_ethtool_stats)(struct net_device *, struct ethtool_stats *, u64 *);
int (*begin)(struct net_device *);
void (*complete)(struct net_device *);
+ u32 (*get_uso)(struct net_device *);
+ int (*set_uso)(struct net_device *, u32);
};
/* CMDs currently supported */
@@ -388,6 +394,8 @@
#define ETHTOOL_GSTATS 0x0000001d /* get NIC-specific statistics */
#define ETHTOOL_GTSO 0x0000001e /* Get TSO enable (ethtool_value) */
#define ETHTOOL_STSO 0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GUSO 0x00000020 /* Get USO enable (ethtool_value) */
+#define ETHTOOL_SUSO 0x00000021 /* Set USO enable (ethtool_value) */
/* compatibility with older code */
#define SPARC_ETH_GSET ETHTOOL_GSET
diff -uNr linux-2.6.12-rc4.org/include/linux/netdevice.h linux-2.6.12-rc4/include/linux/netdevice.h
--- linux-2.6.12-rc4.org/include/linux/netdevice.h 2005-05-27 23:22:46.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/netdevice.h 2005-05-31 10:02:02.000000000 +0545
@@ -414,6 +414,7 @@
#define NETIF_F_VLAN_CHALLENGED 1024 /* Device cannot handle VLAN packets */
#define NETIF_F_TSO 2048 /* Can offload TCP/IP segmentation */
#define NETIF_F_LLTX 4096 /* LockLess TX */
+#define NETIF_F_USO 8192 /* Can offload UDP Large Send*/
/* Called after device is detached from network. */
void (*uninit)(struct net_device *dev);
diff -uNr linux-2.6.12-rc4.org/include/linux/skbuff.h linux-2.6.12-rc4/include/linux/skbuff.h
--- linux-2.6.12-rc4.org/include/linux/skbuff.h 2005-05-27 23:22:46.000000000 +0545
+++ linux-2.6.12-rc4/include/linux/skbuff.h 2005-06-02 20:27:43.000000000 +0545
@@ -136,6 +136,8 @@
unsigned int nr_frags;
unsigned short tso_size;
unsigned short tso_segs;
+ unsigned short uso_size;
+ unsigned int ip6_frag_id;
struct sk_buff *frag_list;
skb_frag_t frags[MAX_SKB_FRAGS];
};
diff -uNr linux-2.6.12-rc4.org/net/core/dev.c linux-2.6.12-rc4/net/core/dev.c
--- linux-2.6.12-rc4.org/net/core/dev.c 2005-05-28 01:49:18.000000000 +0545
+++ linux-2.6.12-rc4/net/core/dev.c 2005-05-31 22:57:22.000000000 +0545
@@ -2793,6 +2793,18 @@
dev->name);
dev->features &= ~NETIF_F_TSO;
}
+ if (dev->features & NETIF_F_USO) {
+ if(!(dev->features & NETIF_F_FRAGLIST)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_FRAGLIST feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ if(!(dev->features & NETIF_F_HW_CSUM)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_HW_CSUM feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ }
/*
* nil rebuild_header routine,
diff -uNr linux-2.6.12-rc4.org/net/core/ethtool.c linux-2.6.12-rc4/net/core/ethtool.c
--- linux-2.6.12-rc4.org/net/core/ethtool.c 2005-06-02 16:55:32.000000000 +0545
+++ linux-2.6.12-rc4/net/core/ethtool.c 2005-06-02 21:53:16.000000000 +0545
@@ -72,6 +72,21 @@
return 0;
}
+u32 ethtool_op_get_uso(struct net_device *dev)
+{
+ return (dev->features & NETIF_F_USO) != 0;
+}
+
+int ethtool_op_set_uso(struct net_device *dev, u32 data)
+{
+ if (data)
+ dev->features |= NETIF_F_USO;
+ else
+ dev->features &= ~NETIF_F_USO;
+
+ return 0;
+}
+
/* Handlers for each ethtool command */
static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)
@@ -548,6 +563,39 @@
return dev->ethtool_ops->set_tso(dev, edata.data);
}
+static int ethtool_get_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata = { ETHTOOL_GTSO };
+
+ if (!dev->ethtool_ops->get_uso)
+ return -EOPNOTSUPP;
+
+ edata.data = dev->ethtool_ops->get_uso(dev);
+
+ if (copy_to_user(useraddr, &edata, sizeof(edata)))
+ return -EFAULT;
+ return 0;
+}
+
+static int ethtool_set_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata;
+
+ if (!dev->ethtool_ops->set_uso)
+ return -EOPNOTSUPP;
+
+ if (copy_from_user(&edata, useraddr, sizeof(edata)))
+ return -EFAULT;
+
+ if (edata.data && !(dev->features & NETIF_F_FRAGLIST))
+ return -EINVAL;
+
+ if (edata.data && !(dev->features & NETIF_F_HW_CSUM))
+ return -EINVAL;
+
+ return dev->ethtool_ops->set_uso(dev, edata.data);
+}
+
static int ethtool_self_test(struct net_device *dev, char __user *useraddr)
{
struct ethtool_test test;
@@ -795,6 +843,12 @@
case ETHTOOL_GSTATS:
rc = ethtool_get_stats(dev, useraddr);
break;
+ case ETHTOOL_GUSO:
+ rc = ethtool_get_uso(dev, useraddr);
+ break;
+ case ETHTOOL_SUSO:
+ rc = ethtool_set_uso(dev, useraddr);
+ break;
default:
rc = -EOPNOTSUPP;
}
@@ -817,3 +871,6 @@
EXPORT_SYMBOL(ethtool_op_set_sg);
EXPORT_SYMBOL(ethtool_op_set_tso);
EXPORT_SYMBOL(ethtool_op_set_tx_csum);
+EXPORT_SYMBOL(ethtool_op_set_uso);
+EXPORT_SYMBOL(ethtool_op_get_uso);
+
diff -uNr linux-2.6.12-rc4.org/net/core/skbuff.c linux-2.6.12-rc4/net/core/skbuff.c
--- linux-2.6.12-rc4.org/net/core/skbuff.c 2005-05-27 23:22:46.000000000 +0545
+++ linux-2.6.12-rc4/net/core/skbuff.c 2005-06-02 20:27:27.000000000 +0545
@@ -159,6 +159,8 @@
skb_shinfo(skb)->tso_size = 0;
skb_shinfo(skb)->tso_segs = 0;
skb_shinfo(skb)->frag_list = NULL;
+ skb_shinfo(skb)->ip6_frag_id = 0;
+ skb_shinfo(skb)->uso_size = 0;
out:
return skb;
nodata:
diff -uNr linux-2.6.12-rc4.org/net/ipv4/ip_output.c linux-2.6.12-rc4/net/ipv4/ip_output.c
--- linux-2.6.12-rc4.org/net/ipv4/ip_output.c 2005-05-27 23:22:46.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv4/ip_output.c 2005-05-31 15:55:39.000000000 +0545
@@ -291,7 +291,8 @@
{
IP_INC_STATS(IPSTATS_MIB_OUTREQUESTS);
- if (skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->tso_size)
+ if (skb->len > dst_mtu(skb->dst) &&
+ !(skb_shinfo(skb)->tso_size || skb_shinfo(skb)->uso_size))
return ip_fragment(skb, ip_finish_output);
else
return ip_finish_output(skb);
@@ -768,7 +769,6 @@
mtu = inet->cork.fragsize;
}
hh_len = LL_RESERVED_SPACE(rt->u.dst.dev);
-
fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0);
maxfraglen = ((mtu - fragheaderlen) & ~7) + fragheaderlen;
@@ -864,6 +864,12 @@
skb->ip_summed = csummode;
skb->csum = 0;
skb_reserve(skb, hh_len);
+ if ((!offset) && (length > mtu) &&
+ (sk->sk_protocol == IPPROTO_UDP) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ skb_shinfo(skb)->uso_size = mtu - fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ }
/*
* Find where to start putting bytes.
diff -uNr linux-2.6.12-rc4.org/net/ipv4/udp.c linux-2.6.12-rc4/net/ipv4/udp.c
--- linux-2.6.12-rc4.org/net/ipv4/udp.c 2005-05-27 23:23:55.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv4/udp.c 2005-05-31 21:14:44.000000000 +0545
@@ -424,9 +424,10 @@
goto send;
}
- if (skb_queue_len(&sk->sk_write_queue) == 1) {
+ if ((skb_queue_len(&sk->sk_write_queue) == 1) ||
+ (skb_shinfo(skb)->uso_size)) {
/*
- * Only one fragment on the socket.
+ * Only one fragment on the socket or it is udp lso skb.
*/
if (skb->ip_summed == CHECKSUM_HW) {
skb->csum = offsetof(struct udphdr, check);
diff -uNr linux-2.6.12-rc4.org/net/ipv6/ip6_output.c linux-2.6.12-rc4/net/ipv6/ip6_output.c
--- linux-2.6.12-rc4.org/net/ipv6/ip6_output.c 2005-05-27 23:22:46.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv6/ip6_output.c 2005-06-02 20:27:55.000000000 +0545
@@ -147,7 +147,8 @@
int ip6_output(struct sk_buff *skb)
{
- if (skb->len > dst_mtu(skb->dst) || dst_allfrag(skb->dst))
+ if ((skb->len > dst_mtu(skb->dst) || dst_allfrag(skb->dst)) &&
+ !skb_shinfo(skb)->uso_size)
return ip6_fragment(skb, ip6_output2);
else
return ip6_output2(skb);
@@ -977,6 +978,19 @@
skb->csum = 0;
/* reserve for fragmentation */
skb_reserve(skb, hh_len+sizeof(struct frag_hdr));
+ if ((!offset) && (length > mtu) &&
+ (sk->sk_protocol == IPPROTO_UDP) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ struct frag_hdr fhdr;
+
+ skb_shinfo(skb)->uso_size =
+ (mtu - fragheaderlen -
+ sizeof(struct frag_hdr));
+ skb->ip_summed = CHECKSUM_HW;
+ ipv6_select_ident(skb, &fhdr);
+ skb_shinfo(skb)->ip6_frag_id =
+ fhdr.identification;
+ }
/*
* Find where to start putting bytes
diff -uNr linux-2.6.12-rc4.org/net/ipv6/udp.c linux-2.6.12-rc4/net/ipv6/udp.c
--- linux-2.6.12-rc4.org/net/ipv6/udp.c 2005-05-27 23:24:12.000000000 +0545
+++ linux-2.6.12-rc4/net/ipv6/udp.c 2005-05-31 17:32:31.000000000 +0545
@@ -590,7 +590,8 @@
goto send;
}
- if (skb_queue_len(&sk->sk_write_queue) == 1) {
+ if ((skb_queue_len(&sk->sk_write_queue) == 1) ||
+ (skb_shinfo(skb)->uso_size)) {
skb->csum = csum_partial((char *)uh,
sizeof(struct udphdr), skb->csum);
uh->check = csum_ipv6_magic(&fl->fl6_src,
^ permalink raw reply
* [PATCH 2.6.12-rc4] ethtool: Support for UDP Large Send Offload
From: ravinandan.arakali @ 2005-06-03 0:43 UTC (permalink / raw)
To: davem, jgarzik, netdev
Cc: raghavendra.koushik, ravinandan.arakali, leonid.grossman,
ananda.raju, rapuru.sriram
Hi,
Attached below is a patch on ethtool utility to support USO(UDP Large
Send Offload).
Pls review the patch.
Usage:
1. To view USO setting
# ethtool -k <ethernet_interface>
2. To set/unset USO
# ethtool -K <ethernet_interface> uso on|off
Signed-off-by: Ananda Raju <ananda.raju@neterion.com>
Signed-off-by: Ravinandan Arakali <ravinandan.arakali@neterion.com>
---
diff -uNr ethtool-3/ethtool-copy.h ethtool-3_uso/ethtool-copy.h
--- ethtool-3/ethtool-copy.h 2005-01-28 01:50:26.000000000 +0545
+++ ethtool-3_uso/ethtool-copy.h 2005-06-02 23:06:48.000000000 +0545
@@ -283,6 +283,8 @@
#define ETHTOOL_GSTATS 0x0000001d /* get NIC-specific statistics */
#define ETHTOOL_GTSO 0x0000001e /* Get TSO enable (ethtool_value) */
#define ETHTOOL_STSO 0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GUSO 0x00000020 /* Get USO enable (ethtool_value) */
+#define ETHTOOL_SUSO 0x00000021 /* Set USO enable (ethtool_value) */
/* compatibility with older code */
#define SPARC_ETH_GSET ETHTOOL_GSET
diff -uNr ethtool-3/ethtool.c ethtool-3_uso/ethtool.c
--- ethtool-3/ethtool.c 2005-01-28 04:19:29.000000000 +0545
+++ ethtool-3_uso/ethtool.c 2005-06-02 23:06:52.000000000 +0545
@@ -119,6 +119,7 @@
* [ tx on|off ] \
* [ sg on|off ] \
* [ tso on|off ]
+ * [ uso on|off ]
* ethtool -r DEVNAME
* ethtool -p DEVNAME [ %d ]
* ethtool -t DEVNAME [ online|offline ]
@@ -191,6 +192,7 @@
" [ tx on|off ] \\\n"
" [ sg on|off ] \\\n"
" [ tso on|off ]\n"
+ " [ uso on|off ]\n"
" ethtool -r DEVNAME\n"
" ethtool -p DEVNAME [ %%d ]\n"
" ethtool -t DEVNAME [online|(offline)]\n"
@@ -236,6 +238,7 @@
static int off_csum_tx_wanted = -1;
static int off_sg_wanted = -1;
static int off_tso_wanted = -1;
+static int off_uso_wanted = -1;
static struct ethtool_pauseparam epause;
static int gpause_changed = 0;
@@ -339,6 +342,7 @@
{ "tx", CMDL_BOOL, &off_csum_tx_wanted, NULL },
{ "sg", CMDL_BOOL, &off_sg_wanted, NULL },
{ "tso", CMDL_BOOL, &off_tso_wanted, NULL },
+ { "uso", CMDL_BOOL, &off_uso_wanted, NULL },
};
static struct cmdline_info cmdline_pause[] = {
@@ -1184,17 +1188,19 @@
return 0;
}
-static int dump_offload (int rx, int tx, int sg, int tso)
+static int dump_offload (int rx, int tx, int sg, int tso, int uso)
{
fprintf(stdout,
"rx-checksumming: %s\n"
"tx-checksumming: %s\n"
"scatter-gather: %s\n"
- "tcp segmentation offload: %s\n",
+ "tcp segmentation offload: %s\n"
+ "udp large send offload: %s\n",
rx ? "on" : "off",
tx ? "on" : "off",
sg ? "on" : "off",
- tso ? "on" : "off");
+ tso ? "on" : "off",
+ uso ? "on" : "off");
return 0;
}
@@ -1458,7 +1464,7 @@
static int do_goffload(int fd, struct ifreq *ifr)
{
struct ethtool_value eval;
- int err, allfail = 1, rx = 0, tx = 0, sg = 0, tso = 0;
+ int err, allfail = 1, rx = 0, tx = 0, sg = 0, tso = 0, uso = 0;
fprintf(stdout, "Offload parameters for %s:\n", devname);
@@ -1502,12 +1508,22 @@
allfail = 0;
}
+ eval.cmd = ETHTOOL_GUSO;
+ ifr->ifr_data = (caddr_t)&eval;
+ err = ioctl(fd, SIOCETHTOOL, ifr);
+ if (err)
+ perror("Cannot get device udp large send offload settings");
+ else {
+ uso = eval.data;
+ allfail = 0;
+ }
+
if (allfail) {
fprintf(stdout, "no offload info available\n");
return 83;
}
- return dump_offload(rx, tx, sg, tso);
+ return dump_offload(rx, tx, sg, tso, uso);
}
static int do_soffload(int fd, struct ifreq *ifr)
@@ -1562,6 +1578,17 @@
return 88;
}
}
+ if (off_uso_wanted >= 0) {
+ changed = 1;
+ eval.cmd = ETHTOOL_SUSO;
+ eval.data = (off_uso_wanted == 1);
+ ifr->ifr_data = (caddr_t)&eval;
+ err = ioctl(fd, SIOCETHTOOL, ifr);
+ if (err) {
+ perror("Cannot set device udp large send offload settings");
+ return 89;
+ }
+ }
if (!changed) {
fprintf(stdout, "no offload settings changed\n");
}
^ permalink raw reply
* Re: PATCH: ioctl send PID in netlink events
From: Thomas Graf @ 2005-06-03 1:00 UTC (permalink / raw)
To: jamal; +Cc: David S. Miller, netdev
In-Reply-To: <1117720349.6050.59.camel@localhost.localdomain>
> Index: net/ipv4/devinet.c
> ===================================================================
> --- e4f7366a04d973a42a948d3b4175d66e9adf143e/net/ipv4/devinet.c (mode:100644)
> +++ uncommitted/net/ipv4/devinet.c (mode:100644)
> @@ -236,6 +236,7 @@
> struct in_ifaddr *promote = NULL;
> struct in_ifaddr *ifa1 = *ifap;
>
> + printk("inet_del_ifa: pid %d\n",current->pid);
> ASSERT_RTNL();
>
> /* 1. Deleting primary ifaddr forces deletion all secondaries
> @@ -305,6 +306,7 @@
>
> ASSERT_RTNL();
>
> + printk("inet_insert_ifa: pid %d\n",current->pid);
> if (!ifa->ifa_local) {
> inet_free_ifa(ifa);
> return 0;
Don't you want to remove these?
> Index: net/ipv4/fib_semantics.c
> ===================================================================
> --- e4f7366a04d973a42a948d3b4175d66e9adf143e/net/ipv4/fib_semantics.c (mode:100644)
> +++ uncommitted/net/ipv4/fib_semantics.c (mode:100644)
> @@ -276,7 +276,7 @@
> struct nlmsghdr *n, struct netlink_skb_parms *req)
> {
> struct sk_buff *skb;
> - u32 pid = req ? req->pid : 0;
> + u32 pid = req ? req->pid : n->nlmsg_pid;
> int size = NLMSG_SPACE(sizeof(struct rtmsg)+256);
>
> skb = alloc_skb(size, GFP_KERNEL);
> @@ -1035,7 +1035,7 @@
> }
>
> nl->nlmsg_flags = NLM_F_REQUEST;
> - nl->nlmsg_pid = 0;
> + nl->nlmsg_pid = current->pid;
> nl->nlmsg_seq = 0;
> nl->nlmsg_len = NLMSG_LENGTH(sizeof(*rtm));
> if (cmd == SIOCDELRT) {
Neat ;->
^ permalink raw reply
* Re: PATCH: ioctl send PID in netlink events
From: jamal @ 2005-06-03 1:37 UTC (permalink / raw)
To: Thomas Graf; +Cc: David S. Miller, netdev
In-Reply-To: <20050603010059.GU15391@postel.suug.ch>
On Fri, 2005-03-06 at 03:00 +0200, Thomas Graf wrote:
> > Index: net/ipv4/devinet.c
> > ===================================================================
> > --- e4f7366a04d973a42a948d3b4175d66e9adf143e/net/ipv4/devinet.c (mode:100644)
> > +++ uncommitted/net/ipv4/devinet.c (mode:100644)
> > @@ -236,6 +236,7 @@
> > struct in_ifaddr *promote = NULL;
> > struct in_ifaddr *ifa1 = *ifap;
> >
> > + printk("inet_del_ifa: pid %d\n",current->pid);
> > ASSERT_RTNL();
> >
> > /* 1. Deleting primary ifaddr forces deletion all secondaries
> > @@ -305,6 +306,7 @@
> >
> > ASSERT_RTNL();
> >
> > + printk("inet_insert_ifa: pid %d\n",current->pid);
> > if (!ifa->ifa_local) {
> > inet_free_ifa(ifa);
> > return 0;
>
> Don't you want to remove these?
>
>
Yes, how did those get there? ;->
> > Index: net/ipv4/fib_semantics.c
> > ===================================================================
> > --- e4f7366a04d973a42a948d3b4175d66e9adf143e/net/ipv4/fib_semantics.c (mode:100644)
> > +++ uncommitted/net/ipv4/fib_semantics.c (mode:100644)
> > @@ -276,7 +276,7 @@
> > struct nlmsghdr *n, struct netlink_skb_parms *req)
> > {
> > struct sk_buff *skb;
> > - u32 pid = req ? req->pid : 0;
> > + u32 pid = req ? req->pid : n->nlmsg_pid;
> > int size = NLMSG_SPACE(sizeof(struct rtmsg)+256);
> >
> > skb = alloc_skb(size, GFP_KERNEL);
> > @@ -1035,7 +1035,7 @@
> > }
> >
> > nl->nlmsg_flags = NLM_F_REQUEST;
> > - nl->nlmsg_pid = 0;
> > + nl->nlmsg_pid = current->pid;
> > nl->nlmsg_seq = 0;
> > nl->nlmsg_len = NLMSG_LENGTH(sizeof(*rtm));
> > if (cmd == SIOCDELRT) {
>
> Neat ;->
The second one could probably use the new macros.
Maybe i will wait until Dave puts this in his tree and send a small
change; else you could send it.
cheers,
jamal
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: jamal @ 2005-06-03 2:32 UTC (permalink / raw)
To: David S. Miller
Cc: john.ronciak, jdmason, shemminger, mitch.a.williams, netdev,
Robert.Olsson, ganesh.venkatesan, jesse.brandeburg
In-Reply-To: <20050602.171812.48807872.davem@davemloft.net>
On Thu, 2005-02-06 at 17:18 -0700, David S. Miller wrote:
> From: "Ronciak, John" <john.ronciak@intel.com>
> Date: Thu, 2 Jun 2005 17:11:20 -0700
>
> > I like this idea as well but I do an issue with it. How would this
> > stack code find out that the weight is too high and pacekts are being
> > dropped (not being polled fast enough)? It would have to check the
> > controller stats to see the error count increasing for some period. I'm
> > not sure this is workable unless we have some sort of feedback which the
> > driver could send up (or set) saying that this is happening and the
> > dynamic weight code could take into acount.
>
> What more do you need other than checking the statistics counter? The
> drop statistics (the ones we care about) are incremented in real time
> by the ->poll() code, so it's not like we have to trigger some
> asynchronous event to get a current version of the number.
I am reading through all the emails and I think either the problem is
not being clearly stated or not understood. I was going to say "or i am
on crack "- but I know i am clean ;->
Heres what i think i saw as a flow of events:
Someone posted a theory that if you happen to reduce the weight
(iirc the reduction was via a shift) then the DRR would give less CPU
time cycle to the driver - Whats the big suprise there? thats DRR design
intent.
Stephen has a patch which allows people to reduce the weight.
DRR provides fairness. If you have 10 NICs coming at different wire
rates, the weights provide a fairness quota without caring about what
those speeds are. So it doesnt make any sense IMO to have the weight
based on what the NIC speed is. Infact i claim it is _nonsense_. You
dont need to factor speed. And the claim that DRR is not real world
is blasphemous.
Having said that:
I have a feeling that issue which is which is being waded around is the
amount that the softirq chews in the CPU (unfortunately a well known
issue) and to some extent the packet flow a specific driver chews
depending on the path it takes.
In other words, for DRR algorithm to enhance the fairness it should
consider not only fairness in the amounts of packets the driver injects
into the system but also the amount of CPU that driver chews. At the
moment we lump all drivers together as far as the CPU cycles are
concerned.
If we could narrow it down to this, then i think there is something that
could lead to meaningful discussion.
This, however, does not eradicate the need for DRR and is absolutely not
driver specific.
cheers,
jamal
^ permalink raw reply
* Re: [PATCH] shaper.c: fix locking
From: jamal @ 2005-06-03 2:36 UTC (permalink / raw)
To: David S. Miller; +Cc: hch, netdev
In-Reply-To: <20050602.163628.01205145.davem@davemloft.net>
On Thu, 2005-02-06 at 16:36 -0700, David S. Miller wrote:
> Fair enough, patch applied. If this driver breaks as a result of
> these changes, you get to keep the pieces ok? :-)
The question is anyone really using this driver? ;->
cheers,
jamal
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox