* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: Ben Greear @ 2005-09-13 16:35 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Ravikiran G Thirumalai, Andrew Morton, linux-kernel, dipankar,
bharata, shai, Rusty Russell, netdev, davem
In-Reply-To: <20050913092659.791bddec@localhost.localdomain>
Stephen Hemminger wrote:
> On Tue, 13 Sep 2005 09:10:12 -0700
> Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
>
>>The net_device has a refcnt used to keep track of it's uses.
>>This is used at the time of unregistering the network device
>>(module unloading ..) (see netdev_wait_allrefs) .
>>For loopback_dev , this refcnt increment/decrement is causing
>>unnecessary traffic on the interlink for NUMA system
>>affecting it's performance. This patch improves tbench numbers by 6% on a
>>8way x86 Xeon (x445).
>>
>
>
> Since when is bringing a network device up/down performance critical?
We grab and drop a reference for each poll of a device, roughly.
See dev_hold in _netif_rx_schedule(struct net_device *dev)
in include/netdevice.h, for instance.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: Stephen Hemminger @ 2005-09-13 16:46 UTC (permalink / raw)
To: Ben Greear
Cc: Ravikiran G Thirumalai, Andrew Morton, linux-kernel, dipankar,
bharata, shai, Rusty Russell, netdev, davem
In-Reply-To: <4326FFC2.7030803@candelatech.com>
On Tue, 13 Sep 2005 09:35:14 -0700
Ben Greear <greearb@candelatech.com> wrote:
> Stephen Hemminger wrote:
> > On Tue, 13 Sep 2005 09:10:12 -0700
> > Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> >
> >
> >>The net_device has a refcnt used to keep track of it's uses.
> >>This is used at the time of unregistering the network device
> >>(module unloading ..) (see netdev_wait_allrefs) .
> >>For loopback_dev , this refcnt increment/decrement is causing
> >>unnecessary traffic on the interlink for NUMA system
> >>affecting it's performance. This patch improves tbench numbers by 6% on a
> >>8way x86 Xeon (x445).
> >>
> >
> >
> > Since when is bringing a network device up/down performance critical?
>
> We grab and drop a reference for each poll of a device, roughly.
>
> See dev_hold in _netif_rx_schedule(struct net_device *dev)
> in include/netdevice.h, for instance.
Yeah, that would be an issue, especially since the rest of that
path is nicely per-cpu
^ permalink raw reply
* Re: masquerading failure for at least icmp and tcp+sack on amd64
From: Stephen Hemminger @ 2005-09-13 18:09 UTC (permalink / raw)
To: Patrick McHardy
Cc: Andrew Morton, netdev, Netfilter Development Mailinglist,
Marc Lehmann
In-Reply-To: <43243AB9.9000705@trash.net>
On Sun, 11 Sep 2005 16:10:01 +0200
Patrick McHardy <kaber@trash.net> wrote:
> Marc Lehmann wrote:
> > On Fri, Sep 09, 2005 at 01:41:34PM +0200, Patrick McHardy <kaber@trash.net> wrote:
> >
> >>What network driver are you using?
> >
> > Happens with both skge and sk98.
>
> Are you sure the same checksum error happens? sk98_lin never sets
> ip_summed to CHECKSUM_HW, it uses either CHECKSUM_UNNECESSARY for
> HW checksummed packets, in which case the check is skipped in
> ip_conntrack, or it uses CHECKSUM_NONE, in which case the checksum
> in the packet must be invalid if the check fails and the packet
> wouldn't be accepted by the final receipient anyway. skge uses
> CHECKSUM_HW for every packet, so they have nothing in common wrt.
> HW checksumming. This would normally mean we can rule out HW
> checksumming, but for some reason turning it off on skge seems
> to help in your case. Stephen, you're more familiar with the
> sk* drivers than me, anything I'm missing here?
>
Some background, the semantic of ip_summed is different on the
output than the input path. On input, it means a checksum is
available in skb->csum; and on output it means the packet is
destined for a device that can do hardware checksumming.
I have gotten reports of receive checksum errors on
some systems, it may be related to certain revisions of
hardware. It would be useful to see the message printed out
by the skge driver that shows chip and revision.
Also, on the input path for TCP and UDP, the code does not
depend on the hardware being correct, and if the checksum
is incorrect, it just prints a warning and does a software
checksum before deciding to drop.
Perhaps netfilter code needs to handle that case?
^ permalink raw reply
* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: Eric Dumazet @ 2005-09-13 18:27 UTC (permalink / raw)
To: Ravikiran G Thirumalai
Cc: Andrew Morton, linux-kernel, dipankar, bharata, shai,
Rusty Russell, netdev, davem
In-Reply-To: <20050913161012.GI3570@localhost.localdomain>
Ravikiran G Thirumalai a écrit :
> The net_device has a refcnt used to keep track of it's uses.
> This is used at the time of unregistering the network device
> (module unloading ..) (see netdev_wait_allrefs) .
> For loopback_dev , this refcnt increment/decrement is causing
> unnecessary traffic on the interlink for NUMA system
> affecting it's performance. This patch improves tbench numbers by 6% on a
> 8way x86 Xeon (x445).
===================================================================
> --- alloc_percpu-2.6.13.orig/include/linux/netdevice.h 2005-08-28 16:41:01.000000000 -0700
> +++ alloc_percpu-2.6.13/include/linux/netdevice.h 2005-09-12 11:54:21.000000000 -0700
> @@ -37,6 +37,7 @@
> #include <linux/config.h>
> #include <linux/device.h>
> #include <linux/percpu.h>
> +#include <linux/bigref.h>
>
> struct divert_blk;
> struct vlan_group;
> @@ -377,7 +378,7 @@
> /* device queue lock */
> spinlock_t queue_lock;
> /* Number of references to this device */
> - atomic_t refcnt;
> + struct bigref netdev_refcnt;
> /* delayed register/unregister */
> struct list_head todo_list;
> /* device name hash chain */
> @@ -677,11 +678,11 @@
Hum...
Did you tried to place refcnt/netdev_refcnt in a separate cache line than
queue_lock ? I got good results too...
> /* device queue lock */
> spinlock_t queue_lock;
> /* Number of references to this device */
> - atomic_t refcnt;
> + struct bigref netdev_refcnt ____cacheline_aligned_in_smp ;
> /* delayed register/unregister */
> struct list_head todo_list;
> /* device name hash chain */
Every time a cpu take the queue_lock spinlock, it exclusively gets one cache
line. If another cpu try to access netdev_refcnt, it has to grab this cache
line (even if properely per_cpu designed, there is still one shared field). In
fact the whole struct net_device should be re-ordered for SMP/NUMA performance.
Eric
^ permalink raw reply
* [RFC] hippi: change to not use skb private
From: Stephen Hemminger @ 2005-09-13 18:38 UTC (permalink / raw)
To: David S. Miller, jes; +Cc: netdev, linux-kernel
It looks like the following would fix hippi to not have to put
fields in sk_buff. The ifield looks appears to to be additional
header information that is being passed in the skb but could just
be put in the header.
Does anyone still have the hardware to test this? Is anyone
even using it on 2.6?
Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
Index: hippi-2.6.13/drivers/net/rrunner.c
===================================================================
--- hippi-2.6.13.orig/drivers/net/rrunner.c
+++ hippi-2.6.13/drivers/net/rrunner.c
@@ -1432,7 +1432,6 @@ static int rr_start_xmit(struct sk_buff
struct ring_ctrl *txctrl;
unsigned long flags;
u32 index, len = skb->len;
- u32 *ifield;
struct sk_buff *new_skb;
if (readl(®s->Mode) & FATAL_ERR)
@@ -1440,29 +1439,6 @@ static int rr_start_xmit(struct sk_buff
readl(®s->Fail1), readl(®s->Fail2));
/*
- * We probably need to deal with tbusy here to prevent overruns.
- */
-
- if (skb_headroom(skb) < 8){
- printk("incoming skb too small - reallocating\n");
- if (!(new_skb = dev_alloc_skb(len + 8))) {
- dev_kfree_skb(skb);
- netif_wake_queue(dev);
- return -EBUSY;
- }
- skb_reserve(new_skb, 8);
- skb_put(new_skb, len);
- memcpy(new_skb->data, skb->data, len);
- dev_kfree_skb(skb);
- skb = new_skb;
- }
-
- ifield = (u32 *)skb_push(skb, 8);
-
- ifield[0] = 0;
- ifield[1] = skb->private.ifield;
-
- /*
* We don't need the lock before we are actually going to start
* fiddling with the control blocks.
*/
@@ -1475,7 +1451,7 @@ static int rr_start_xmit(struct sk_buff
rrpriv->tx_skbuff[index] = skb;
set_rraddr(&rrpriv->tx_ring[index].addr, pci_map_single(
rrpriv->pci_dev, skb->data, len + 8, PCI_DMA_TODEVICE));
- rrpriv->tx_ring[index].size = len + 8; /* include IFIELD */
+ rrpriv->tx_ring[index].size = len;
rrpriv->tx_ring[index].mode = PACKET_START | PACKET_END;
txctrl->pi = (index + 1) % TX_RING_ENTRIES;
wmb();
Index: hippi-2.6.13/include/linux/skbuff.h
===================================================================
--- hippi-2.6.13.orig/include/linux/skbuff.h
+++ hippi-2.6.13/include/linux/skbuff.h
@@ -194,7 +194,6 @@ struct skb_shared_info {
* @nfct: Associated connection, if any
* @nfctinfo: Relationship of this skb to the connection
* @nf_bridge: Saved data about a bridged frame - see br_netfilter.c
- * @private: Data which is private to the HIPPI implementation
* @tc_index: Traffic control index
* @tc_verd: traffic control verdict
* @tc_classid: traffic control classid
@@ -267,11 +266,7 @@ struct sk_buff {
struct nf_bridge_info *nf_bridge;
#endif
#endif /* CONFIG_NETFILTER */
-#if defined(CONFIG_HIPPI)
- union {
- __u32 ifield;
- } private;
-#endif
+
#ifdef CONFIG_NET_SCHED
__u32 tc_index; /* traffic control index */
#ifdef CONFIG_NET_CLS_ACT
Index: hippi-2.6.13/net/802/hippi.c
===================================================================
--- hippi-2.6.13.orig/net/802/hippi.c
+++ hippi-2.6.13/net/802/hippi.c
@@ -50,7 +50,8 @@ static int hippi_header(struct sk_buff *
unsigned short type, void *daddr, void *saddr,
unsigned len)
{
- struct hippi_hdr *hip = (struct hippi_hdr *)skb_push(skb, HIPPI_HLEN);
+ struct hippi_hdr *hip = (struct hippi_hdr *)skb_push(skb, HIPPI_HLEN + 8);
+ u32 *ifield = (u32 *) (hip + 1);
if (!len){
len = skb->len - HIPPI_HLEN;
@@ -81,13 +82,14 @@ static int hippi_header(struct sk_buff *
hip->snap.oui[2] = 0x00;
hip->snap.ethertype = htons(type);
- if (daddr)
- {
+ memset(ifield, 0, 2*sizeof(u32));
+ if (daddr) {
memcpy(hip->le.dest_switch_addr, daddr + 3, 3);
- memcpy(&skb->private.ifield, daddr + 2, 4);
- return HIPPI_HLEN;
+ memcpy(ifield+1, daddr + 2, 4);
+ return HIPPI_HLEN + 8;
}
- return -((int)HIPPI_HLEN);
+
+ return -((int)HIPPI_HLEN - 8);
}
@@ -200,7 +202,7 @@ static void hippi_setup(struct net_devic
* still need a fake ARPHRD to make ifconfig and friends play ball.
*/
dev->type = ARPHRD_HIPPI;
- dev->hard_header_len = HIPPI_HLEN;
+ dev->hard_header_len = HIPPI_HLEN + 8;
dev->mtu = 65280;
dev->addr_len = HIPPI_ALEN;
dev->tx_queue_len = 25 /* 5 */;
Index: hippi-2.6.13/net/core/skbuff.c
===================================================================
--- hippi-2.6.13.orig/net/core/skbuff.c
+++ hippi-2.6.13/net/core/skbuff.c
@@ -370,9 +370,6 @@ struct sk_buff *skb_clone(struct sk_buff
nf_bridge_get(skb->nf_bridge);
#endif
#endif /*CONFIG_NETFILTER*/
-#if defined(CONFIG_HIPPI)
- C(private);
-#endif
#ifdef CONFIG_NET_SCHED
C(tc_index);
#ifdef CONFIG_NET_CLS_ACT
^ permalink raw reply
* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: Ravikiran G Thirumalai @ 2005-09-13 18:53 UTC (permalink / raw)
To: Eric Dumazet
Cc: Andrew Morton, linux-kernel, dipankar, bharata, shai,
Rusty Russell, netdev, davem
In-Reply-To: <43271A28.9090301@cosmosbay.com>
On Tue, Sep 13, 2005 at 08:27:52PM +0200, Eric Dumazet wrote:
> Ravikiran G Thirumalai a écrit :
>
> Hum...
>
> Did you tried to place refcnt/netdev_refcnt in a separate cache line than
> queue_lock ? I got good results too...
>
> > /* device queue lock */
> > spinlock_t queue_lock;
> > /* Number of references to this device */
> > - atomic_t refcnt;
> > + struct bigref netdev_refcnt ____cacheline_aligned_in_smp ;
> > /* delayed register/unregister */
> > struct list_head todo_list;
> > /* device name hash chain */
>
> Every time a cpu take the queue_lock spinlock, it exclusively gets one
> cache line. If another cpu try to access netdev_refcnt, it has to grab this
> cache line (even if properely per_cpu designed, there is still one shared
> field). In fact the whole struct net_device should be re-ordered for
> SMP/NUMA performance.
I agree. Maybe placing the queue_lock in a different cacheline is the
right approach?
Thanks,
Kiran
^ permalink raw reply
* Re: [patch 9/11] net: dst_entry.refcount, use, lastuse to use alloc_percpu
From: David S. Miller @ 2005-09-13 20:24 UTC (permalink / raw)
To: kiran; +Cc: akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913161708.GK3570@localhost.localdomain>
There is no way in the world this enormous amount of NUMA
complexity is being added to the destination cache layer.
Sorry.
^ permalink raw reply
* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: David S. Miller @ 2005-09-13 20:26 UTC (permalink / raw)
To: shemminger
Cc: kiran, akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913092659.791bddec@localhost.localdomain>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Tue, 13 Sep 2005 09:26:59 -0700
> Since when is bringing a network device up/down performance critical?
The issue is the dev_get()'s that occur all over the place
to during packet transmit/receive, that's what they are
trying to address.
I'm still against all of these invasive NUMA changes to the
networking though, they are simply too ugly and special cased
to consider seriously.
^ permalink raw reply
* Re: [RFC] hippi: change to not use skb private
From: David S. Miller @ 2005-09-13 20:54 UTC (permalink / raw)
To: shemminger; +Cc: jes, netdev, linux-kernel
In-Reply-To: <20050913113858.440d3a0f@localhost.localdomain>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Tue, 13 Sep 2005 11:38:58 -0700
> It looks like the following would fix hippi to not have to put
> fields in sk_buff. The ifield looks appears to to be additional
> header information that is being passed in the skb but could just
> be put in the header.
I can't see any problem with this patch, and it fixes
all the realloc stuff that has to potentially occur
in the transmit path.
^ permalink raw reply
* Re: masquerading failure for at least icmp and tcp+sack on amd64
From: David S. Miller @ 2005-09-13 20:59 UTC (permalink / raw)
To: shemminger; +Cc: akpm, netdev, netfilter-devel, kaber, schmorp
In-Reply-To: <20050913110902.0ad58b90@localhost.localdomain>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Tue, 13 Sep 2005 11:09:02 -0700
> Also, on the input path for TCP and UDP, the code does not
> depend on the hardware being correct, and if the checksum
> is incorrect, it just prints a warning and does a software
> checksum before deciding to drop.
> Perhaps netfilter code needs to handle that case?
I personally think netfilter should do so.
^ permalink raw reply
* Re: [patch 9/11] net: dst_entry.refcount, use, lastuse to use alloc_percpu
From: Ravikiran G Thirumalai @ 2005-09-13 22:07 UTC (permalink / raw)
To: David S. Miller
Cc: akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913.132442.53540386.davem@davemloft.net>
On Tue, Sep 13, 2005 at 01:24:42PM -0700, David S. Miller wrote:
>
> There is no way in the world this enormous amount of NUMA
> complexity is being added to the destination cache layer.
Agreed the dst changes are ugly; that can be worked on. But the
cacheline bouncing problem on the atomic_t dst_entry refcounter has been
around for quite a while -- even on SMPs, not just NUMA. We need a solution
for that. I thought you were against the dst_entry bloat caused by the
previous version of the dst patch. alloc_percpu takes that away. You had
concerns about workloads with low route locality. Unfortunately we don't have
access to infrastructure setup for such tests :(
As for the ugliness, would something on the lines of net_device refcounter
patch in the series above be acceptable?
Thanks,
Kiran
^ permalink raw reply
* Re: [patch 9/11] net: dst_entry.refcount, use, lastuse to use alloc_percpu
From: David S. Miller @ 2005-09-13 22:12 UTC (permalink / raw)
To: kiran; +Cc: akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913220737.GA6249@localhost.localdomain>
From: Ravikiran G Thirumalai <kiran@scalex86.org>
Date: Tue, 13 Sep 2005 15:07:37 -0700
> Agreed the dst changes are ugly; that can be worked on. But the
> cacheline bouncing problem on the atomic_t dst_entry refcounter has
> been around for quite a while -- even on SMPs, not just NUMA. We
> need a solution for that. I thought you were against the dst_entry
> bloat caused by the previous version of the dst patch. alloc_percpu
> takes that away. You had concerns about workloads with low route
> locality. Unfortunately we don't have access to infrastructure setup
> for such tests :(
You don't have two computers connected on a network?
All you need is that, load a bunch of routes into one system that
point to an IP address which you just force an ARP entry for (so it
just gets lost in the ether) and then generate a rDOS workload through
it from another machine using pktgen.
I'm fine with funny per-cpu memory allocation strategies, perhaps
(would have to see a patch doing _only_ that to be sure).
But using bigrefs, no way. We have enough trouble making the data
structures small without adding bloat like that. A busy server can
have hundreds of thousands of dst cache entries active on it, and they
chew up enough memory as is.
^ permalink raw reply
* Re: Route cache performance
From: Simon Kirby @ 2005-09-13 22:14 UTC (permalink / raw)
To: Alexey Kuznetsov, Robert Olsson, Eric Dumazet, netdev
In-Reply-To: <20050907195911.GA8382@yakov.inr.ac.ru>
On Wed, Sep 07, 2005 at 11:59:11PM +0400, Alexey Kuznetsov wrote:
> Hello!
>
> > Yes, setting maxbatch to 10000 also results in working gc,
>
> Could you try lower values? F.e. I guess 300 or a little more
> (it is netdev_max_backlog) should be enough.
300 seems to be sufficient, but I'm not sure what this depends on (load,
HZ, timing of some sort?). See below for full tests.
> > for the normal case also hurts the DoS case...and it really hurts when
> > the when the DoS case is the normal case.
>
> 5.7% is not "really hurts" yet. :-)
I decided to try out FreeBSD in comparison as I've heard people saying
that it handles this case quite well. The results are interesting.
FreeBSD seems to have a route cache; however, it keys only on
destination. When a new destination is seen, the route table entry that
matched is "cloned" so that the MTU, etc., is copied, the dst rewritten
to the exact IP (as opposed to a network route), and path MTU discovery
results are maintained in this entry, keyed by destination address only.
I'm not sure if Linux could work in the same way with the source routing
tables enabled, but perhaps it's possible to either disable the source
side of the route cache when policy routing is disabled. Or perhaps a
route cache hash could be instantiated per route table or something.
Actually, is there ever a valid case where the source needs to be tracked
in the route cache when policy routing is disabled? A local socket will
track MSS correctly while a forwarded packet will create or use an entry
without touching it, so I don't see why not.
Anyway, spoofed source or not go the same speed through FreeBSD. Also,
there is a "fastforwarding" sysctl that sends forwarded packets from
the input interrupt/poll without queueing them in a soft interrupt
("NETISR").
Polling mode on FreeBSD isn't as nice as NAPI in that it's fully manual
on or off, and when it's on it triggers entirely from the timer interrupt
unless told to also trigger from the idle loop. The user/kernel balancing
is also manual but I can't seem to get it to forward as fast as with it
disabled no matter how I adjust it.
TEST RESULTS
------------
All Linux tests with NAPI enabled and the e1000 driver native to that
kernel unless otherwise specified. maxbatch does not exist in kernels
< 2.6.9, and rhash_size does not exist in 2.4.
Sender: 367 Mbps, 717883 pps valid src/dst, 64 byte (Ethernet) packets
2.4.27-rc1: 297 Mbps forwarded (w/idle time?!)
2.4.31: 296 Mbps forwarded (w/idle time?!)
2.6.13-rc6: 173 Mbps forwarded
FreeBSD 5.4-RELEASE (HZ=1000): 103 Mbps forwarded (dead userland)
`- net.inet.ip.fastforwarding=1: 282 Mbps forwarded (dead userland)
`- kern.polling.enable=1: 75.3 Mbps forwarded
`- kern.polling.idle_poll=1: 226 Mbps forwarded
Sender: 348 Mbps, 680416 pps random src, valid dst, 64 bytes
(All FreeBSD tests have identical results.)
2.4.27-rc1: 122 Mbps forwarded
2.4.27-rc1 gc_elasticity=1: 182 Mbps forwarded
2.4.27-rc1+2.4.31_e1000: 117 Mbps forwarded
2.4.27-rc1+2.4.31_e1000 gc_elasticity=1: 170 Mbps forwarded
2.4.31: 95.1 Mbps forwarded
2.4.31 gc_elasticity=1: 122 Mbps forwarded
2.6.13-rc6: <1 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=30: <1 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=60: 1.5 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=100: 2.6 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=150: 3.8 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=200: 6.9 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=250: 15.4 Mbps forwarded (dst overflow)
2.6.13-rc6 maxbatch=300: 58.6 Mbps forwarded (gc balanced)
2.6.13-rc6 maxbatch=350: 60.5 Mbps forwarded
2.6.13-rc6 maxbatch=400: 59.4 Mbps forwarded
2.6.13-rc6 maxbatch=450: 59.1 Mbps forwarded
2.6.13-rc6 maxbatch=500: 62.0 Mbps forwarded
2.6.13-rc6 maxbatch=550: 61.9 Mbps forwarded
2.6.13-rc6 maxbatch=1000: 61.4 Mbps forwarded
2.6.13-rc6 maxbatch=2000: 60.2 Mbps forwarded
2.6.13-rc6 maxbatch=3000: 60.1 Mbps forwarded
2.6.13-rc6 maxbatch=5000: 59.1 Mbps forwarded
2.6.13-rc6 maxbatch=MAXINT: 59.1 Mbps forwarded
2.6.13-rc6 dst_free: 66.0 Mbps forwarded
2.6.13-rc6 dst_free max_size=rhash_size: 79.2 Mbps forwarded
------------
2.6 definitely has better dst cache gc balancing than 2.4. I can set
the max_size=rhash_size in 2.6.13-rc6 and it will just work, even without
adjusting gc_elasticity or gc_thresh. In 2.4.27 and 2.4.31, the only
parameter that appears to help is gc_elasticity. If I just adjust
max_size, it overflows and falls over.
I note that the actual read copy update "maxbatch" limit was added in
2.6.9. Before then, it seems there was no limit (infinite). Was it
added for latency reasons?
Time permitting, I'd also like to run some profiles. It's interesting
to note that 2.6 is slower at forwarding even straight duplicate small
packets. We should definitely get to the bottom of that.
Simon-
^ permalink raw reply
* Re: [patch 7/11] net: Use bigrefs for net_device.refcount
From: Ravikiran G Thirumalai @ 2005-09-13 22:16 UTC (permalink / raw)
To: David S. Miller
Cc: shemminger, akpm, linux-kernel, dipankar, bharata, shai, rusty,
netdev
In-Reply-To: <20050913.132607.113443001.davem@davemloft.net>
On Tue, Sep 13, 2005 at 01:26:07PM -0700, David S. Miller wrote:
> From: Stephen Hemminger <shemminger@osdl.org>
> Date: Tue, 13 Sep 2005 09:26:59 -0700
>
> > Since when is bringing a network device up/down performance critical?
>
> The issue is the dev_get()'s that occur all over the place
> to during packet transmit/receive, that's what they are
> trying to address.
>
> I'm still against all of these invasive NUMA changes to the
> networking though, they are simply too ugly and special cased
> to consider seriously.
All of them or the dst ones? Hopefully the netdevice refcounter patch
is not ugly or complicated as the dst ones? And why are they special cased?
Are networking workloads with high route locality not interesting?
Thanks,
Kiran
^ permalink raw reply
* Re: [patch 9/11] net: dst_entry.refcount, use, lastuse to use alloc_percpu
From: Ravikiran G Thirumalai @ 2005-09-13 23:17 UTC (permalink / raw)
To: David S. Miller
Cc: akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913.151216.48124942.davem@davemloft.net>
On Tue, Sep 13, 2005 at 03:12:16PM -0700, David S. Miller wrote:
> From: Ravikiran G Thirumalai <kiran@scalex86.org>
> Date: Tue, 13 Sep 2005 15:07:37 -0700
> ...
> But using bigrefs, no way. We have enough trouble making the data
> structures small without adding bloat like that. A busy server can
> have hundreds of thousands of dst cache entries active on it, and they
> chew up enough memory as is.
>
But even 1 Million dst cache entries would be 16+4 MB additional for a 4 cpu
box....is that too much? The alloc_percpu reimplementation interleaves
objects on cache lines, unlike the existing implementation which pads per-cpu
objects to cache lines...
If you are referring to embedded routing devices,
would they use CONFIG_NUMA or CONFIG_SMP?? (bigrefs nicely fold back to
regular atomic_t s on UPs)
Thanks,
Kiran
^ permalink raw reply
* Re: [patch 9/11] net: dst_entry.refcount, use, lastuse to use alloc_percpu
From: David S. Miller @ 2005-09-13 23:27 UTC (permalink / raw)
To: kiran; +Cc: akpm, linux-kernel, dipankar, bharata, shai, rusty, netdev
In-Reply-To: <20050913231717.GC6249@localhost.localdomain>
From: Ravikiran G Thirumalai <kiran@scalex86.org>
Date: Tue, 13 Sep 2005 16:17:17 -0700
> But even 1 Million dst cache entries would be 16+4 MB additional for
> a 4 cpu box....is that too much?
Absolutely.
Per-cpu counters are great for things like single instance
statistics et al. But once you start doing them per-object
that's out of control bloat as far as I'm concerned.
^ permalink raw reply
* Re: [PATCH 15/29] ieee80211 Renamed ieee80211_hdr to ieee80211_hdr_4addr
From: Michael Wu @ 2005-09-13 23:45 UTC (permalink / raw)
To: netdev; +Cc: ieee80211-devel, James Ketrenos
In-Reply-To: <43275856.7010305@linux.intel.com>
This patch plus patch 5 looks very similar to my management frame patch
except..
On Tuesday 13 September 2005 18:53, James Ketrenos wrote:
> @@ -552,6 +577,17 @@ struct ieee80211_authentication {
> struct ieee80211_info_element info_element[0];
> } __attribute__ ((packed));
>
> +struct ieee80211_disassoc {
Add alias for deauthentication frame here?
> + struct ieee80211_hdr_3addr header;
> + u16 reason_code;
If we do not append _code to "status", why append _code to "reason"?
> + struct ieee80211_info_element info_element[0];
> +} __attribute__ ((packed));
> +
> +struct ieee80211_probe_request {
> + struct ieee80211_hdr_3addr header;
> + struct ieee80211_info_element info_element[0];
> +} __attribute__ ((packed));
> +
> struct ieee80211_probe_response {
> struct ieee80211_hdr_3addr header;
> u32 time_stamp[2];
> @@ -560,14 +596,25 @@ struct ieee80211_probe_response {
> struct ieee80211_info_element info_element[0];
> } __attribute__ ((packed));
>
> -struct ieee80211_assoc_request_frame {
> +/* Alias beacon for probe_response */
> +#define ieee80211_beacon ieee80211_probe_response
> +
> +struct ieee80211_assoc_request {
> + struct ieee80211_hdr_3addr header;
> + u16 capability;
> + u16 listen_interval;
> + struct ieee80211_info_element info_element[0];
> +} __attribute__ ((packed));
> +
> +struct ieee80211_reassoc_request {
> + struct ieee80211_hdr_3addr header;
> __le16 capability;
> __le16 listen_interval;
> u8 current_ap[ETH_ALEN];
> struct ieee80211_info_element info_element[0];
> } __attribute__ ((packed));
>
> -struct ieee80211_assoc_response_frame {
> +struct ieee80211_assoc_response {
> struct ieee80211_hdr_3addr header;
> __le16 capability;
> __le16 status;
^ permalink raw reply
* Re: [PATCH 25/29] ieee80211 use endian-aware types
From: Michael Wu @ 2005-09-13 23:48 UTC (permalink / raw)
To: netdev; +Cc: James Ketrenos, ieee80211-devel
In-Reply-To: <43276290.2010809@linux.intel.com>
On Tuesday 13 September 2005 19:36, James Ketrenos wrote:
> ieee80211: use endian-aware types
>
> From: Michael Wu <flamingice@sourmilk.net>
>
> This patch:
> - fixes misc. whitespace/comments
> - replaces u16 with __le16/__be16 where appropriate
>
> Signed-off-by: Michael Wu <flamingice@sourmilk.net>
> Signed-off-by: Jiri Benc <jbenc@suse.cz>
>
> Signed-off-by: James Ketrenos <jketreno@linux.intel.com>
>
The original patch already went in, and this patch looks nothing like the one
I sent in.
-Michael Wu
^ permalink raw reply
* Re: [PATCH 23/29] ieee80211 Added ieee80211_radiotap.h
From: Michael Wu @ 2005-09-13 23:50 UTC (permalink / raw)
To: netdev
In-Reply-To: <43276231.1000908@linux.intel.com>
On Tuesday 13 September 2005 19:35, James Ketrenos wrote:
> Added ieee80211_radiotap.h to enhance statistic reporting to user space
> from wireless drivers.
>
> Signed-off-by: James Ketrenos <jketreno@linux.intel.com>
I'm not sure, but didn't Mike Kershaw create this patch? I was expecting a
signed-off-by line by him.
-Michael Wu
^ permalink raw reply
* Re: [PATCH 2.6.13] IPv4/IPv6: USO Scatter-gather approac
From: Jeff Garzik @ 2005-09-14 0:24 UTC (permalink / raw)
To: ravinandan.arakali
Cc: netdev, raghavendra.koushik, leonid.grossman, rapuru.sriram,
ananda.raju
In-Reply-To: <200509140655.j8E6t1jx003507@localhost.localdomain>
Please resend to netdev@vger.kernel.org, so that I may properly comment.
netdev@oss.sgi.com has been retired.
Thanks,
Jeff
^ permalink raw reply
* Re: masquerading failure for at least icmp and tcp+sack on amd64
From: Patrick McHardy @ 2005-09-14 1:10 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Andrew Morton, netdev, Netfilter Development Mailinglist,
Marc Lehmann
In-Reply-To: <20050913110902.0ad58b90@localhost.localdomain>
Stephen Hemminger wrote:
> Some background, the semantic of ip_summed is different on the
> output than the input path. On input, it means a checksum is
> available in skb->csum; and on output it means the packet is
> destined for a device that can do hardware checksumming.
Yep, so far the netfilter code should be correct since my batch
of HW checksum fixes.
> I have gotten reports of receive checksum errors on
> some systems, it may be related to certain revisions of
> hardware. It would be useful to see the message printed out
> by the skge driver that shows chip and revision.
That may be the reason. I've audited most relevant code-paths
for this case and they seem to be mostly OK. There are a couple
of cases in the ppp code I'm not sure about yet, but I couldn't
trigger any errors in my test-setup. Anyway they don't seem to
be related to this problem since it also happens with sk98_lin
which doesn't set CHECKSUM_HW.
> Also, on the input path for TCP and UDP, the code does not
> depend on the hardware being correct, and if the checksum
> is incorrect, it just prints a warning and does a software
> checksum before deciding to drop.
> Perhaps netfilter code needs to handle that case?
Yes, this is not handled so far. But is looks like there is
some other problem because it also happens with sk98_lin.
Thanks for your help Stephen.
^ permalink raw reply
* Re: masquerading failure for at least icmp and tcp+sack on amd64
From: Patrick McHardy @ 2005-09-14 1:13 UTC (permalink / raw)
To: David S. Miller; +Cc: akpm, netdev, netfilter-devel, schmorp, shemminger
In-Reply-To: <20050913.135954.111731835.davem@davemloft.net>
David S. Miller wrote:
> From: Stephen Hemminger <shemminger@osdl.org>
> Date: Tue, 13 Sep 2005 11:09:02 -0700
>
>
>>Also, on the input path for TCP and UDP, the code does not
>>depend on the hardware being correct, and if the checksum
>>is incorrect, it just prints a warning and does a software
>>checksum before deciding to drop.
>>Perhaps netfilter code needs to handle that case?
>
> I personally think netfilter should do so.
I agree. One thing I've planned for some silent moment is
to clean up the entire netfilter checksumming code (there's
lots of small duplicated chunks). Probably at least some
of it will also be applicable for the remaining stack.
^ permalink raw reply
* Re: masquerading failure for at least icmp and tcp+sack on amd64
From: David S. Miller @ 2005-09-14 3:41 UTC (permalink / raw)
To: kaber; +Cc: akpm, netdev, netfilter-devel, schmorp, shemminger
In-Reply-To: <43277943.8050700@trash.net>
From: Patrick McHardy <kaber@trash.net>
Date: Wed, 14 Sep 2005 03:13:39 +0200
> David S. Miller wrote:
> > I personally think netfilter should do so.
>
> I agree. One thing I've planned for some silent moment is
> to clean up the entire netfilter checksumming code (there's
> lots of small duplicated chunks). Probably at least some
> of it will also be applicable for the remaining stack.
There is another thing I thought about today, and that is
to automatically handle this CHECKSUM_HW stuff when the
skb->data area is COW'd via pskb_expand_head() or similar.
I don't know how well that would work, but if it did then
we could consolidate all of this stuff into one spot which
is always nice.
^ permalink raw reply
* [PATCH 2.6.13] IPv4/IPv6: USO Scatter-gather approac
From: ravinandan.arakali @ 2005-09-14 6:55 UTC (permalink / raw)
To: jgarzik, netdev
Cc: raghavendra.koushik, ravinandan.arakali, leonid.grossman,
rapuru.sriram, ananda.raju
Hi,
Attached below is kernel patch with UPD large send offload which address the
sendfile() syscall also. This patch uses scatter-gather support of skb to
generate large UDP datagram.
Below is a "how-to" on changes required in network drivers to use the USO
interface.
UDP Large Send Offload (USO) Interface:
--------------------------------------
USO is a feature wherein the Linux kernel network stack will offload the IP
fragmentation functionality of large UDP datagram to hardware. This will reduce
the overhead of stack in fragmenting the large UDP datagram to MTU sized packets
1) Drivers indicate their capability of USO using
dev->features |= NETIF_F_USO | NETIF_F_HW_CSUM | NETIF_F_SG
NETIF_F_HW_CSUM is required for USO over ipv6.
2) USO packet will be submitted for transmission using driver xmit routine.
USO packet will have a non-zero value for
"skb_shinfo(skb)->uso_size"
skb_shinfo(skb)->uso_size will indicate the length of data part in each IP
fragment going out of the adapter after IP fragmentation by hardware.
skb->data will contain MAC/IP/UDP header and skb_shinfo(skb)->frags[]
contains the data payload. The skb->ip_summed will be set to CHECKSUM_HW
indicating that hardware has to do checksum calculation. Hardware should
compute the UDP checksum of complete datagram and also ip header checksum of
each fragmented IP packet.
For IPV6 the USO provides the fragment identification-id in
skb_shinfo(skb)->ip6_frag_id. The adapter should use this ID for generating
IPv6 fragments.
Signed-off-by: Ananda Raju <ananda.raju@neterion.com>
---
diff -uNr linux-2.6.13/include/linux/ethtool.h linux-2.6.13_uso/include/linux/ethtool.h
--- linux-2.6.13/include/linux/ethtool.h 2005-09-07 06:36:15.000000000 -0700
+++ linux-2.6.13_uso/include/linux/ethtool.h 2005-09-07 06:32:29.000000000 -0700
@@ -261,6 +261,8 @@
int ethtool_op_set_sg(struct net_device *dev, u32 data);
u32 ethtool_op_get_tso(struct net_device *dev);
int ethtool_op_set_tso(struct net_device *dev, u32 data);
+u32 ethtool_op_get_uso(struct net_device *dev);
+int ethtool_op_set_uso(struct net_device *dev, u32 data);
/**
* ðtool_ops - Alter and report network device settings
@@ -290,6 +292,8 @@
* set_sg: Turn scatter-gather on or off
* get_tso: Report whether TCP segmentation offload is enabled
* set_tso: Turn TCP segmentation offload on or off
+ * get_uso: Report whether UDP large send offload is enabled
+ * set_uso: Turn UDP large send offload on or off
* self_test: Run specified self-tests
* get_strings: Return a set of strings that describe the requested objects
* phys_id: Identify the device
@@ -354,6 +358,8 @@
void (*get_ethtool_stats)(struct net_device *, struct ethtool_stats *, u64 *);
int (*begin)(struct net_device *);
void (*complete)(struct net_device *);
+ u32 (*get_uso)(struct net_device *);
+ int (*set_uso)(struct net_device *, u32);
};
/* CMDs currently supported */
@@ -389,6 +395,8 @@
#define ETHTOOL_GSTATS 0x0000001d /* get NIC-specific statistics */
#define ETHTOOL_GTSO 0x0000001e /* Get TSO enable (ethtool_value) */
#define ETHTOOL_STSO 0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GUSO 0x00000020 /* Get USO enable (ethtool_value) */
+#define ETHTOOL_SUSO 0x00000021 /* Set USO enable (ethtool_value) */
/* compatibility with older code */
#define SPARC_ETH_GSET ETHTOOL_GSET
diff -uNr linux-2.6.13/include/linux/netdevice.h linux-2.6.13_uso/include/linux/netdevice.h
--- linux-2.6.13/include/linux/netdevice.h 2005-09-07 04:20:51.000000000 -0700
+++ linux-2.6.13_uso/include/linux/netdevice.h 2005-09-07 04:22:51.000000000 -0700
@@ -408,6 +408,7 @@
#define NETIF_F_VLAN_CHALLENGED 1024 /* Device cannot handle VLAN packets */
#define NETIF_F_TSO 2048 /* Can offload TCP/IP segmentation */
#define NETIF_F_LLTX 4096 /* LockLess TX */
+#define NETIF_F_USO 8192 /* Can offload UDP Large Send*/
/* Called after device is detached from network. */
void (*uninit)(struct net_device *dev);
diff -uNr linux-2.6.13/include/linux/skbuff.h linux-2.6.13_uso/include/linux/skbuff.h
--- linux-2.6.13/include/linux/skbuff.h 2005-09-07 04:20:56.000000000 -0700
+++ linux-2.6.13_uso/include/linux/skbuff.h 2005-09-07 04:22:58.000000000 -0700
@@ -137,6 +137,8 @@
unsigned int nr_frags;
unsigned short tso_size;
unsigned short tso_segs;
+ unsigned short uso_size;
+ unsigned int ip6_frag_id;
struct sk_buff *frag_list;
skb_frag_t frags[MAX_SKB_FRAGS];
};
@@ -327,6 +329,11 @@
extern void skb_under_panic(struct sk_buff *skb, int len,
void *here);
+extern int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
+ int getfrag(void *from, char *to, int offset,
+ int len,int odd, struct sk_buff *skb),
+ void *from, int length);
+
struct skb_seq_state
{
__u32 lower_offset;
diff -uNr linux-2.6.13/net/core/dev.c linux-2.6.13_uso/net/core/dev.c
--- linux-2.6.13/net/core/dev.c 2005-09-07 06:36:25.000000000 -0700
+++ linux-2.6.13_uso/net/core/dev.c 2005-09-07 06:32:02.000000000 -0700
@@ -2706,6 +2706,25 @@
dev->name);
dev->features &= ~NETIF_F_TSO;
}
+ /* TSO requires that SG is present as well. */
+ if ((dev->features & NETIF_F_TSO) &&
+ !(dev->features & NETIF_F_SG)) {
+ printk("%s: Dropping NETIF_F_TSO since no SG feature.\n",
+ dev->name);
+ dev->features &= ~NETIF_F_TSO;
+ }
+ if (dev->features & NETIF_F_USO) {
+ if (!(dev->features & NETIF_F_HW_CSUM)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_HW_CSUM feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ if (!(dev->features & NETIF_F_SG)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_SG feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ }
/*
* nil rebuild_header routine,
diff -uNr linux-2.6.13/net/core/ethtool.c linux-2.6.13_uso/net/core/ethtool.c
--- linux-2.6.13/net/core/ethtool.c 2005-09-07 06:36:34.000000000 -0700
+++ linux-2.6.13_uso/net/core/ethtool.c 2005-09-07 06:32:15.000000000 -0700
@@ -81,6 +81,20 @@
return 0;
}
+u32 ethtool_op_get_uso(struct net_device *dev)
+{
+ return (dev->features & NETIF_F_USO) != 0;
+}
+
+int ethtool_op_set_uso(struct net_device *dev, u32 data)
+{
+ if (data)
+ dev->features |= NETIF_F_USO;
+ else
+ dev->features &= ~NETIF_F_USO;
+ return 0;
+}
+
/* Handlers for each ethtool command */
static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)
@@ -469,6 +483,9 @@
err = dev->ethtool_ops->set_tso(dev, 0);
if (err)
return err;
+ err = dev->ethtool_ops->set_uso(dev, 0);
+ if (err)
+ return err;
}
return dev->ethtool_ops->set_sg(dev, data);
@@ -557,6 +574,32 @@
return dev->ethtool_ops->set_tso(dev, edata.data);
}
+static int ethtool_get_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata = { ETHTOOL_GTSO };
+
+ if (!dev->ethtool_ops->get_uso)
+ return -EOPNOTSUPP;
+ edata.data = dev->ethtool_ops->get_uso(dev);
+ if (copy_to_user(useraddr, &edata, sizeof(edata)))
+ return -EFAULT;
+ return 0;
+}
+static int ethtool_set_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata;
+
+ if (!dev->ethtool_ops->set_uso)
+ return -EOPNOTSUPP;
+ if (copy_from_user(&edata, useraddr, sizeof(edata)))
+ return -EFAULT;
+ if (edata.data && !(dev->features & NETIF_F_SG))
+ return -EINVAL;
+ if (edata.data && !(dev->features & NETIF_F_HW_CSUM))
+ return -EINVAL;
+ return dev->ethtool_ops->set_uso(dev, edata.data);
+}
+
static int ethtool_self_test(struct net_device *dev, char __user *useraddr)
{
struct ethtool_test test;
@@ -806,6 +849,12 @@
case ETHTOOL_GSTATS:
rc = ethtool_get_stats(dev, useraddr);
break;
+ case ETHTOOL_GUSO:
+ rc = ethtool_get_uso(dev, useraddr);
+ break;
+ case ETHTOOL_SUSO:
+ rc = ethtool_set_uso(dev, useraddr);
+ break;
default:
rc = -EOPNOTSUPP;
}
@@ -833,3 +882,5 @@
EXPORT_SYMBOL(ethtool_op_set_tso);
EXPORT_SYMBOL(ethtool_op_set_tx_csum);
EXPORT_SYMBOL(ethtool_op_set_tx_hw_csum);
+EXPORT_SYMBOL(ethtool_op_set_uso);
+EXPORT_SYMBOL(ethtool_op_get_uso);
diff -uNr linux-2.6.13/net/core/skbuff.c linux-2.6.13_uso/net/core/skbuff.c
--- linux-2.6.13/net/core/skbuff.c 2005-09-07 04:21:30.000000000 -0700
+++ linux-2.6.13_uso/net/core/skbuff.c 2005-09-07 06:38:57.000000000 -0700
@@ -159,6 +159,8 @@
skb_shinfo(skb)->tso_size = 0;
skb_shinfo(skb)->tso_segs = 0;
skb_shinfo(skb)->frag_list = NULL;
+ skb_shinfo(skb)->uso_size = 0;
+ skb_shinfo(skb)->ip6_frag_id = 0;
out:
return skb;
nodata:
@@ -1654,6 +1656,64 @@
return textsearch_find(config, state);
}
+/*
+ * skb_append_datato_frags - append the user data to a skb,
+ * sk - sock structure which contains skbs for transmission
+ * getfrag - The function to be called to get the data from the user.
+ * from - pointer to user message iov
+ * length - length of the iov message
+ *
+ * This procedure will allocate a skb enough to hold protocol headers and
+ * append the user data in the fragment part of the skb and add the skb to
+ * socket write queue
+ */
+int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
+ int getfrag(void *from, char *to, int offset,
+ int len,int odd, struct sk_buff *skb),
+ void *from, int length)
+{
+ int frg_cnt = 0;
+ skb_frag_t *frag = NULL;
+ struct page *page = NULL;
+ int copy, left;
+ int offset = 0;
+ do {
+ frg_cnt = skb_shinfo(skb)->nr_frags;
+ if (frg_cnt >= MAX_SKB_FRAGS) {
+ kfree_skb(skb);
+ return -EFAULT;
+ }
+ page = alloc_pages(sk->sk_allocation, 0);
+ if (page == NULL) {
+ kfree_skb(skb);
+ return -ENOMEM;
+ }
+ sk->sk_sndmsg_page = page;
+ sk->sk_sndmsg_off = 0;
+ skb_fill_page_desc(skb, frg_cnt, page, 0, 0);
+ frg_cnt = skb_shinfo(skb)->nr_frags;
+ atomic_add(PAGE_SIZE, &sk->sk_wmem_alloc);
+ skb->truesize += PAGE_SIZE;
+ frag = &skb_shinfo(skb)->frags[frg_cnt - 1];
+ left = PAGE_SIZE - frag->page_offset;
+ copy = (length > left)? left : length;
+ if (getfrag(from, page_address(frag->page) +
+ frag->page_offset+frag->size,
+ offset, copy, 0, skb) < 0) {
+ kfree_skb(skb);
+ return -EFAULT;
+ }
+ sk->sk_sndmsg_off += copy;
+ frag->size += copy;
+ skb->len += copy;
+ skb->data_len += copy;
+ offset += copy;
+ length -= copy;
+ page = NULL;
+ } while (length > 0);
+ return 0;
+}
+
void __init skb_init(void)
{
skbuff_head_cache = kmem_cache_create("skbuff_head_cache",
@@ -1696,3 +1756,4 @@
EXPORT_SYMBOL(skb_seq_read);
EXPORT_SYMBOL(skb_abort_seq_read);
EXPORT_SYMBOL(skb_find_text);
+EXPORT_SYMBOL(skb_append_datato_frags);
diff -uNr linux-2.6.13/net/ipv4/ip_output.c linux-2.6.13_uso/net/ipv4/ip_output.c
--- linux-2.6.13/net/ipv4/ip_output.c 2005-09-07 04:21:46.000000000 -0700
+++ linux-2.6.13_uso/net/ipv4/ip_output.c 2005-09-13 07:12:05.000000000 -0700
@@ -280,7 +280,8 @@
{
IP_INC_STATS(IPSTATS_MIB_OUTREQUESTS);
- if (skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->tso_size)
+ if (skb->len > dst_mtu(skb->dst) &&
+ !(skb_shinfo(skb)->uso_size || skb_shinfo(skb)->tso_size))
return ip_fragment(skb, ip_finish_output);
else
return ip_finish_output(skb);
@@ -781,6 +782,46 @@
csummode = CHECKSUM_HW;
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL) {
+ skb = sock_alloc_send_skb(sk,
+ hh_len + fragheaderlen + transhdrlen + 20,
+ (flags & MSG_DONTWAIT), &err);
+ if (skb == NULL)
+ goto error;
+ /* reserve space for Hardware header */
+ skb_reserve(skb, hh_len);
+ /* create space for UDP/IP header */
+ skb_put(skb,fragheaderlen + transhdrlen);
+ /* initialize network header pointer */
+ skb->nh.raw = skb->data;
+ /* initialize protocol header pointer */
+ skb->h.raw = skb->data + fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ skb->csum = 0;
+ sk->sk_sndmsg_off = 0;
+ }
+ err = skb_append_datato_frags(sk,skb, getfrag, from,
+ (length - transhdrlen));
+ if (!err) {
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen);
+ __skb_queue_tail(&sk->sk_write_queue, skb);
+ return 0;
+ } else {
+ /* There is not enough support do UPD LSO,
+ * so follow normal path
+ */
+ kfree_skb(skb);
+ goto error;
+ }
+ }
+
/* So, what's going on in the loop below?
*
@@ -1012,14 +1053,23 @@
return -EINVAL;
inet->cork.length += size;
+ if ((sk->sk_protocol == IPPROTO_UDP) &&
+ (rt->u.dst.dev->features & NETIF_F_USO))
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen);
+
while (size > 0) {
int i;
- /* Check if the remaining data fits into current packet. */
- len = mtu - skb->len;
- if (len < size)
- len = maxfraglen - skb->len;
+ if (skb_shinfo(skb)->uso_size) {
+ len = size;
+ } else {
+
+ /* Check if the remaining data fits into current packet. */
+ len = mtu - skb->len;
+ if (len < size)
+ len = maxfraglen - skb->len;
+ }
if (len <= 0) {
struct sk_buff *skb_prev;
char *data;
diff -uNr linux-2.6.13/net/ipv6/ip6_output.c linux-2.6.13_uso/net/ipv6/ip6_output.c
--- linux-2.6.13/net/ipv6/ip6_output.c 2005-09-07 04:21:57.000000000 -0700
+++ linux-2.6.13_uso/net/ipv6/ip6_output.c 2005-09-13 07:11:10.000000000 -0700
@@ -147,7 +147,8 @@
int ip6_output(struct sk_buff *skb)
{
- if (skb->len > dst_mtu(skb->dst) || dst_allfrag(skb->dst))
+ if ((skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->uso_size) ||
+ dst_allfrag(skb->dst))
return ip6_fragment(skb, ip6_output2);
else
return ip6_output2(skb);
@@ -893,6 +894,50 @@
*/
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL) {
+ skb = sock_alloc_send_skb(sk,
+ hh_len + fragheaderlen + transhdrlen + 20,
+ (flags & MSG_DONTWAIT), &err);
+ if (skb == NULL)
+ goto error;
+ /* reserve space for Hardware header */
+ skb_reserve(skb, hh_len);
+ /* create space for UDP/IP header */
+ skb_put(skb,fragheaderlen + transhdrlen);
+ /* initialize network header pointer */
+ skb->nh.raw = skb->data;
+ /* initialize protocol header pointer */
+ skb->h.raw = skb->data + fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ skb->csum = 0;
+ sk->sk_sndmsg_off = 0;
+ }
+ err = skb_append_datato_frags(sk,skb, getfrag, from,
+ (length - transhdrlen));
+ if (!err) {
+ struct frag_hdr fhdr;
+
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen) -
+ sizeof(struct frag_hdr);
+ ipv6_select_ident(skb, &fhdr);
+ skb_shinfo(skb)->ip6_frag_id = fhdr.identification;
+ __skb_queue_tail(&sk->sk_write_queue, skb);
+ return 0;
+ } else {
+ /* There is not enough support do UPD LSO,
+ * so follow normal path
+ */
+ kfree_skb(skb);
+ goto error;
+ }
+ }
if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL)
goto alloc_new_skb;
^ permalink raw reply
* [PATCH 2.6.13] IPv4/IPv6: USO Scatter-gather approac
From: ananda.raju @ 2005-09-14 6:56 UTC (permalink / raw)
To: jgarzik, netdev
Cc: raghavendra.koushik, ravinandan.arakali, leonid.grossman,
rapuru.sriram, ananda.raju
Hi,
Attached below is kernel patch with UPD large send offload which address the
sendfile() syscall also. This patch uses scatter-gather support of skb to
generate large UDP datagram.
Below is a "how-to" on changes required in network drivers to use the USO
interface.
UDP Large Send Offload (USO) Interface:
--------------------------------------
USO is a feature wherein the Linux kernel network stack will offload the IP
fragmentation functionality of large UDP datagram to hardware. This will reduce
the overhead of stack in fragmenting the large UDP datagram to MTU sized packets
1) Drivers indicate their capability of USO using
dev->features |= NETIF_F_USO | NETIF_F_HW_CSUM | NETIF_F_SG
NETIF_F_HW_CSUM is required for USO over ipv6.
2) USO packet will be submitted for transmission using driver xmit routine.
USO packet will have a non-zero value for
"skb_shinfo(skb)->uso_size"
skb_shinfo(skb)->uso_size will indicate the length of data part in each IP
fragment going out of the adapter after IP fragmentation by hardware.
skb->data will contain MAC/IP/UDP header and skb_shinfo(skb)->frags[]
contains the data payload. The skb->ip_summed will be set to CHECKSUM_HW
indicating that hardware has to do checksum calculation. Hardware should
compute the UDP checksum of complete datagram and also ip header checksum of
each fragmented IP packet.
For IPV6 the USO provides the fragment identification-id in
skb_shinfo(skb)->ip6_frag_id. The adapter should use this ID for generating
IPv6 fragments.
Signed-off-by: Ananda Raju <ananda.raju@neterion.com>
---
diff -uNr linux-2.6.13/include/linux/ethtool.h linux-2.6.13_uso/include/linux/ethtool.h
--- linux-2.6.13/include/linux/ethtool.h 2005-09-07 06:36:15.000000000 -0700
+++ linux-2.6.13_uso/include/linux/ethtool.h 2005-09-07 06:32:29.000000000 -0700
@@ -261,6 +261,8 @@
int ethtool_op_set_sg(struct net_device *dev, u32 data);
u32 ethtool_op_get_tso(struct net_device *dev);
int ethtool_op_set_tso(struct net_device *dev, u32 data);
+u32 ethtool_op_get_uso(struct net_device *dev);
+int ethtool_op_set_uso(struct net_device *dev, u32 data);
/**
* ðtool_ops - Alter and report network device settings
@@ -290,6 +292,8 @@
* set_sg: Turn scatter-gather on or off
* get_tso: Report whether TCP segmentation offload is enabled
* set_tso: Turn TCP segmentation offload on or off
+ * get_uso: Report whether UDP large send offload is enabled
+ * set_uso: Turn UDP large send offload on or off
* self_test: Run specified self-tests
* get_strings: Return a set of strings that describe the requested objects
* phys_id: Identify the device
@@ -354,6 +358,8 @@
void (*get_ethtool_stats)(struct net_device *, struct ethtool_stats *, u64 *);
int (*begin)(struct net_device *);
void (*complete)(struct net_device *);
+ u32 (*get_uso)(struct net_device *);
+ int (*set_uso)(struct net_device *, u32);
};
/* CMDs currently supported */
@@ -389,6 +395,8 @@
#define ETHTOOL_GSTATS 0x0000001d /* get NIC-specific statistics */
#define ETHTOOL_GTSO 0x0000001e /* Get TSO enable (ethtool_value) */
#define ETHTOOL_STSO 0x0000001f /* Set TSO enable (ethtool_value) */
+#define ETHTOOL_GUSO 0x00000020 /* Get USO enable (ethtool_value) */
+#define ETHTOOL_SUSO 0x00000021 /* Set USO enable (ethtool_value) */
/* compatibility with older code */
#define SPARC_ETH_GSET ETHTOOL_GSET
diff -uNr linux-2.6.13/include/linux/netdevice.h linux-2.6.13_uso/include/linux/netdevice.h
--- linux-2.6.13/include/linux/netdevice.h 2005-09-07 04:20:51.000000000 -0700
+++ linux-2.6.13_uso/include/linux/netdevice.h 2005-09-07 04:22:51.000000000 -0700
@@ -408,6 +408,7 @@
#define NETIF_F_VLAN_CHALLENGED 1024 /* Device cannot handle VLAN packets */
#define NETIF_F_TSO 2048 /* Can offload TCP/IP segmentation */
#define NETIF_F_LLTX 4096 /* LockLess TX */
+#define NETIF_F_USO 8192 /* Can offload UDP Large Send*/
/* Called after device is detached from network. */
void (*uninit)(struct net_device *dev);
diff -uNr linux-2.6.13/include/linux/skbuff.h linux-2.6.13_uso/include/linux/skbuff.h
--- linux-2.6.13/include/linux/skbuff.h 2005-09-07 04:20:56.000000000 -0700
+++ linux-2.6.13_uso/include/linux/skbuff.h 2005-09-07 04:22:58.000000000 -0700
@@ -137,6 +137,8 @@
unsigned int nr_frags;
unsigned short tso_size;
unsigned short tso_segs;
+ unsigned short uso_size;
+ unsigned int ip6_frag_id;
struct sk_buff *frag_list;
skb_frag_t frags[MAX_SKB_FRAGS];
};
@@ -327,6 +329,11 @@
extern void skb_under_panic(struct sk_buff *skb, int len,
void *here);
+extern int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
+ int getfrag(void *from, char *to, int offset,
+ int len,int odd, struct sk_buff *skb),
+ void *from, int length);
+
struct skb_seq_state
{
__u32 lower_offset;
diff -uNr linux-2.6.13/net/core/dev.c linux-2.6.13_uso/net/core/dev.c
--- linux-2.6.13/net/core/dev.c 2005-09-07 06:36:25.000000000 -0700
+++ linux-2.6.13_uso/net/core/dev.c 2005-09-07 06:32:02.000000000 -0700
@@ -2706,6 +2706,25 @@
dev->name);
dev->features &= ~NETIF_F_TSO;
}
+ /* TSO requires that SG is present as well. */
+ if ((dev->features & NETIF_F_TSO) &&
+ !(dev->features & NETIF_F_SG)) {
+ printk("%s: Dropping NETIF_F_TSO since no SG feature.\n",
+ dev->name);
+ dev->features &= ~NETIF_F_TSO;
+ }
+ if (dev->features & NETIF_F_USO) {
+ if (!(dev->features & NETIF_F_HW_CSUM)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_HW_CSUM feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ if (!(dev->features & NETIF_F_SG)) {
+ printk("%s: Dropping NETIF_F_USO since no ", dev->name);
+ printk("NETIF_F_SG feature.\n");
+ dev->features &= ~NETIF_F_USO;
+ }
+ }
/*
* nil rebuild_header routine,
diff -uNr linux-2.6.13/net/core/ethtool.c linux-2.6.13_uso/net/core/ethtool.c
--- linux-2.6.13/net/core/ethtool.c 2005-09-07 06:36:34.000000000 -0700
+++ linux-2.6.13_uso/net/core/ethtool.c 2005-09-07 06:32:15.000000000 -0700
@@ -81,6 +81,20 @@
return 0;
}
+u32 ethtool_op_get_uso(struct net_device *dev)
+{
+ return (dev->features & NETIF_F_USO) != 0;
+}
+
+int ethtool_op_set_uso(struct net_device *dev, u32 data)
+{
+ if (data)
+ dev->features |= NETIF_F_USO;
+ else
+ dev->features &= ~NETIF_F_USO;
+ return 0;
+}
+
/* Handlers for each ethtool command */
static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)
@@ -469,6 +483,9 @@
err = dev->ethtool_ops->set_tso(dev, 0);
if (err)
return err;
+ err = dev->ethtool_ops->set_uso(dev, 0);
+ if (err)
+ return err;
}
return dev->ethtool_ops->set_sg(dev, data);
@@ -557,6 +574,32 @@
return dev->ethtool_ops->set_tso(dev, edata.data);
}
+static int ethtool_get_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata = { ETHTOOL_GTSO };
+
+ if (!dev->ethtool_ops->get_uso)
+ return -EOPNOTSUPP;
+ edata.data = dev->ethtool_ops->get_uso(dev);
+ if (copy_to_user(useraddr, &edata, sizeof(edata)))
+ return -EFAULT;
+ return 0;
+}
+static int ethtool_set_uso(struct net_device *dev, char __user *useraddr)
+{
+ struct ethtool_value edata;
+
+ if (!dev->ethtool_ops->set_uso)
+ return -EOPNOTSUPP;
+ if (copy_from_user(&edata, useraddr, sizeof(edata)))
+ return -EFAULT;
+ if (edata.data && !(dev->features & NETIF_F_SG))
+ return -EINVAL;
+ if (edata.data && !(dev->features & NETIF_F_HW_CSUM))
+ return -EINVAL;
+ return dev->ethtool_ops->set_uso(dev, edata.data);
+}
+
static int ethtool_self_test(struct net_device *dev, char __user *useraddr)
{
struct ethtool_test test;
@@ -806,6 +849,12 @@
case ETHTOOL_GSTATS:
rc = ethtool_get_stats(dev, useraddr);
break;
+ case ETHTOOL_GUSO:
+ rc = ethtool_get_uso(dev, useraddr);
+ break;
+ case ETHTOOL_SUSO:
+ rc = ethtool_set_uso(dev, useraddr);
+ break;
default:
rc = -EOPNOTSUPP;
}
@@ -833,3 +882,5 @@
EXPORT_SYMBOL(ethtool_op_set_tso);
EXPORT_SYMBOL(ethtool_op_set_tx_csum);
EXPORT_SYMBOL(ethtool_op_set_tx_hw_csum);
+EXPORT_SYMBOL(ethtool_op_set_uso);
+EXPORT_SYMBOL(ethtool_op_get_uso);
diff -uNr linux-2.6.13/net/core/skbuff.c linux-2.6.13_uso/net/core/skbuff.c
--- linux-2.6.13/net/core/skbuff.c 2005-09-07 04:21:30.000000000 -0700
+++ linux-2.6.13_uso/net/core/skbuff.c 2005-09-07 06:38:57.000000000 -0700
@@ -159,6 +159,8 @@
skb_shinfo(skb)->tso_size = 0;
skb_shinfo(skb)->tso_segs = 0;
skb_shinfo(skb)->frag_list = NULL;
+ skb_shinfo(skb)->uso_size = 0;
+ skb_shinfo(skb)->ip6_frag_id = 0;
out:
return skb;
nodata:
@@ -1654,6 +1656,64 @@
return textsearch_find(config, state);
}
+/*
+ * skb_append_datato_frags - append the user data to a skb,
+ * sk - sock structure which contains skbs for transmission
+ * getfrag - The function to be called to get the data from the user.
+ * from - pointer to user message iov
+ * length - length of the iov message
+ *
+ * This procedure will allocate a skb enough to hold protocol headers and
+ * append the user data in the fragment part of the skb and add the skb to
+ * socket write queue
+ */
+int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
+ int getfrag(void *from, char *to, int offset,
+ int len,int odd, struct sk_buff *skb),
+ void *from, int length)
+{
+ int frg_cnt = 0;
+ skb_frag_t *frag = NULL;
+ struct page *page = NULL;
+ int copy, left;
+ int offset = 0;
+ do {
+ frg_cnt = skb_shinfo(skb)->nr_frags;
+ if (frg_cnt >= MAX_SKB_FRAGS) {
+ kfree_skb(skb);
+ return -EFAULT;
+ }
+ page = alloc_pages(sk->sk_allocation, 0);
+ if (page == NULL) {
+ kfree_skb(skb);
+ return -ENOMEM;
+ }
+ sk->sk_sndmsg_page = page;
+ sk->sk_sndmsg_off = 0;
+ skb_fill_page_desc(skb, frg_cnt, page, 0, 0);
+ frg_cnt = skb_shinfo(skb)->nr_frags;
+ atomic_add(PAGE_SIZE, &sk->sk_wmem_alloc);
+ skb->truesize += PAGE_SIZE;
+ frag = &skb_shinfo(skb)->frags[frg_cnt - 1];
+ left = PAGE_SIZE - frag->page_offset;
+ copy = (length > left)? left : length;
+ if (getfrag(from, page_address(frag->page) +
+ frag->page_offset+frag->size,
+ offset, copy, 0, skb) < 0) {
+ kfree_skb(skb);
+ return -EFAULT;
+ }
+ sk->sk_sndmsg_off += copy;
+ frag->size += copy;
+ skb->len += copy;
+ skb->data_len += copy;
+ offset += copy;
+ length -= copy;
+ page = NULL;
+ } while (length > 0);
+ return 0;
+}
+
void __init skb_init(void)
{
skbuff_head_cache = kmem_cache_create("skbuff_head_cache",
@@ -1696,3 +1756,4 @@
EXPORT_SYMBOL(skb_seq_read);
EXPORT_SYMBOL(skb_abort_seq_read);
EXPORT_SYMBOL(skb_find_text);
+EXPORT_SYMBOL(skb_append_datato_frags);
diff -uNr linux-2.6.13/net/ipv4/ip_output.c linux-2.6.13_uso/net/ipv4/ip_output.c
--- linux-2.6.13/net/ipv4/ip_output.c 2005-09-07 04:21:46.000000000 -0700
+++ linux-2.6.13_uso/net/ipv4/ip_output.c 2005-09-13 07:12:05.000000000 -0700
@@ -280,7 +280,8 @@
{
IP_INC_STATS(IPSTATS_MIB_OUTREQUESTS);
- if (skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->tso_size)
+ if (skb->len > dst_mtu(skb->dst) &&
+ !(skb_shinfo(skb)->uso_size || skb_shinfo(skb)->tso_size))
return ip_fragment(skb, ip_finish_output);
else
return ip_finish_output(skb);
@@ -781,6 +782,46 @@
csummode = CHECKSUM_HW;
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL) {
+ skb = sock_alloc_send_skb(sk,
+ hh_len + fragheaderlen + transhdrlen + 20,
+ (flags & MSG_DONTWAIT), &err);
+ if (skb == NULL)
+ goto error;
+ /* reserve space for Hardware header */
+ skb_reserve(skb, hh_len);
+ /* create space for UDP/IP header */
+ skb_put(skb,fragheaderlen + transhdrlen);
+ /* initialize network header pointer */
+ skb->nh.raw = skb->data;
+ /* initialize protocol header pointer */
+ skb->h.raw = skb->data + fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ skb->csum = 0;
+ sk->sk_sndmsg_off = 0;
+ }
+ err = skb_append_datato_frags(sk,skb, getfrag, from,
+ (length - transhdrlen));
+ if (!err) {
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen);
+ __skb_queue_tail(&sk->sk_write_queue, skb);
+ return 0;
+ } else {
+ /* There is not enough support do UPD LSO,
+ * so follow normal path
+ */
+ kfree_skb(skb);
+ goto error;
+ }
+ }
+
/* So, what's going on in the loop below?
*
@@ -1012,14 +1053,23 @@
return -EINVAL;
inet->cork.length += size;
+ if ((sk->sk_protocol == IPPROTO_UDP) &&
+ (rt->u.dst.dev->features & NETIF_F_USO))
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen);
+
while (size > 0) {
int i;
- /* Check if the remaining data fits into current packet. */
- len = mtu - skb->len;
- if (len < size)
- len = maxfraglen - skb->len;
+ if (skb_shinfo(skb)->uso_size) {
+ len = size;
+ } else {
+
+ /* Check if the remaining data fits into current packet. */
+ len = mtu - skb->len;
+ if (len < size)
+ len = maxfraglen - skb->len;
+ }
if (len <= 0) {
struct sk_buff *skb_prev;
char *data;
diff -uNr linux-2.6.13/net/ipv6/ip6_output.c linux-2.6.13_uso/net/ipv6/ip6_output.c
--- linux-2.6.13/net/ipv6/ip6_output.c 2005-09-07 04:21:57.000000000 -0700
+++ linux-2.6.13_uso/net/ipv6/ip6_output.c 2005-09-13 07:11:10.000000000 -0700
@@ -147,7 +147,8 @@
int ip6_output(struct sk_buff *skb)
{
- if (skb->len > dst_mtu(skb->dst) || dst_allfrag(skb->dst))
+ if ((skb->len > dst_mtu(skb->dst) && !skb_shinfo(skb)->uso_size) ||
+ dst_allfrag(skb->dst))
return ip6_fragment(skb, ip6_output2);
else
return ip6_output2(skb);
@@ -893,6 +894,50 @@
*/
inet->cork.length += length;
+ if (((length > mtu) && (sk->sk_protocol == IPPROTO_UDP)) &&
+ (rt->u.dst.dev->features & NETIF_F_USO)) {
+ /* There is support for UDP large send offload by network
+ * device, so create one single skb packet containing complete
+ * udp datagram
+ */
+ if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL) {
+ skb = sock_alloc_send_skb(sk,
+ hh_len + fragheaderlen + transhdrlen + 20,
+ (flags & MSG_DONTWAIT), &err);
+ if (skb == NULL)
+ goto error;
+ /* reserve space for Hardware header */
+ skb_reserve(skb, hh_len);
+ /* create space for UDP/IP header */
+ skb_put(skb,fragheaderlen + transhdrlen);
+ /* initialize network header pointer */
+ skb->nh.raw = skb->data;
+ /* initialize protocol header pointer */
+ skb->h.raw = skb->data + fragheaderlen;
+ skb->ip_summed = CHECKSUM_HW;
+ skb->csum = 0;
+ sk->sk_sndmsg_off = 0;
+ }
+ err = skb_append_datato_frags(sk,skb, getfrag, from,
+ (length - transhdrlen));
+ if (!err) {
+ struct frag_hdr fhdr;
+
+ /* specify the length of each IP datagram fragment*/
+ skb_shinfo(skb)->uso_size = (mtu - fragheaderlen) -
+ sizeof(struct frag_hdr);
+ ipv6_select_ident(skb, &fhdr);
+ skb_shinfo(skb)->ip6_frag_id = fhdr.identification;
+ __skb_queue_tail(&sk->sk_write_queue, skb);
+ return 0;
+ } else {
+ /* There is not enough support do UPD LSO,
+ * so follow normal path
+ */
+ kfree_skb(skb);
+ goto error;
+ }
+ }
if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL)
goto alloc_new_skb;
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox