* Re: [RFC PATCH 7/9] GSO: Support partial segmentation offload
From: Alexander Duyck @ 2016-03-23 22:36 UTC (permalink / raw)
To: Edward Cree
Cc: Or Gerlitz, Alexander Duyck, Netdev, David Miller, Tom Herbert
In-Reply-To: <56F30523.8080108@solarflare.com>
On Wed, Mar 23, 2016 at 2:05 PM, Edward Cree <ecree@solarflare.com> wrote:
> On 23/03/16 18:06, Alexander Duyck wrote:
>> On Wed, Mar 23, 2016 at 9:27 AM, Edward Cree <ecree@solarflare.com> wrote:
>>> My belief is that my way is (in the long run) simpler: ultimately it gets
>>> rid of per-protocol GSO callbacks entirely. Every header gets written
>>> correctly* when the packet initially traverses the stack, and then at
>>> transmit time you either hand that off to TSO, or do the equivalent thing
>>> in software: segment at the TCP layer while treating everything above it
>>> as an opaque pseudo-L2 header.
>> I'm pretty sure that isn't a safe approach to take. With GSO we are
>> holding off until we are about to hand the packets off to the device
>> before we segment it. If we do it earlier it will lead to more issues
>> as you could have the packet route off to somewhere you were not
>> expecting and having it already "soft segmented" could lead to issues
>> where the packet would no longer be coherent.
> Ah, yes, that is a potential problem. And now that I think about it, we
> might not know what the MSS is until segmentation time, either, even if
> we did know for sure we would want to segment.
> So my approach doesn't work after all, the superframe has to be coherent
> when it traverses the stack.
Right.
>>> You mean not supported by offloads, right? We can still _create_ a nested
>>> tunnel device, it just won't use hardware offloads. And in the long run,
>>> if we can make tunnel-agnostic TSO as I'm proposing, we should be able to
>>> support it for nested tunnels too (the giant L2 header just gets more giant).
>> Right. Basically what will currently happen is that if you were to
>> encapsulate an ipip tunnel inside of a VXLAN for instance the GSO
>> would kick in before the VXLAN tunnel has even added the outer headers
>> because the VXLAN netdev does not advertise NETIF_F_GSO_IPIP. That is
>> the kind of thing we want to have happen though anyway since a
>> physical device wouldn't know how to deal with such a scenario anyway.
> I disagree. Surely we should be able to "soft segment" the packet just
> before we give it to the physical device, and then tell it to do dumb copying
> of both the VXLAN and IPIP headers? At this point, we don't have the problem
> you identified above, because we've arrived at the device now.
One issue here is that all levels of IP headers would have to have the
DF bit set. I don't think that happens right now.
> So we can chase through some per-protocol callbacks to shorten all the outer
> lengths and adjust all the outer checksums, then hand it to the device for
> TSO. The device is treating the extra headers as an opaque blob, so it
> doesn't know or care whether it's one layer of encapsulation or forty-two.
So if we do pure software offloads this is doable. However the GSO
flags are meant to have hardware feature equivalents. The problem is
if you combine an IPIP and VXLAN header how do you know what header is
what and which order things are in, and what is the likelihood of
having a device that would get things right when dealing with 3 levels
of IP headers. This is one of the reasons why we don't support
multiple levels of tunnels in the GSO code. GSO is just meant to be a
fall-back for hardware offloads.
>>> I think an L2 header extension is a better semantic match for what's
>>> happening (the tunnel is an L2 device and we're sending L3 packets over
>>> it). But why does it matter? Are you giving the hardware the L2 and
>>> L3 headers in separate DMA descriptors or something? The way I see it,
>>> all hardware needs to be told is "where to start TSO from", and how it
>>> thinks of the stuff before that doesn't matter, because it's not
>>> supposed to touch it anyway.
>> The problem is setups like VFs where they want to know where the
>> network header starts so they know where to insert a VLAN tag. Most
>> hardware supports IP options or IPv6 extension headers. What I am
>> doing is exploiting that in the case of the Intel hardware by telling
>> it the IP header is the outer IP header and the transport header is
>> the inner transport header. Then all I have to deal with is the fact
>> that hardware will try to compute the entire IPv4 header checksum over
>> the range so I cancel that out by seeding the IP checksum with the
>> lco_csum added to the inner pseudo-header checksum.
> Ok, it sounds like the interface to Intel hardware is just Very Different
> to Solarflare hardware on this point: we don't tell our hardware anything
> about where the various headers start, it just parses them to figure it
> out. (And for new-style TSO we'd tell it where the TCP header starts, as
> I described before.)
That is kind of what I figured. So does that mean for IPv6 you guys
are parsing through extension headers? I believe that is one of the
reasons why Intel did things the way they did is to avoid having to
parse through any IPv4 options or IPv6 extension headers.
> But this sounds like a driver-level thing: you have to undo some of what
> your hardware will do because you're having to lie to it about what you're
> giving it. So that all happens in the driver, the stack's GSO code isn't
> affected. In which case I'm much less bothered by this; I thought it was
> an assumption you were baking into the stack. As far as I'm concerned you
> can do whatever ugly hacks you like in Intel drivers, as long as I don't
> have to do them in sfc ;)
The thing is I have to have the hardware recompute the outer IPv4
checksum because it is updating the ID field. If I were able to leave
the IP ID static then I wouldn't need to update the checksum. The
only bit that makes it ugly is the fact that the hardware is being
misled so it thinks the IPv4 header has 50 bytes of options when those
actually represent the outer L4 through to the inner L3 headers.
> Although, why is your device computing the IPv4 header checksum? Those
> aren't supposed to be offloaded, the stack always already filled them in
> in software. (I think sfc makes the same mistake actually.) Is there not
> a way for you to tell your device to skip IP header checksum offload?
> Then you wouldn't have this problem in the first place, and you could tell
> it the IP header was as big as you like without having to seed it with
> this correction value.
It is because we are rewriting the IP ID. We cannot have a static
checksum if the IP ID is getting updated.
>>> Like I say, I'm assuming we'll start setting DF on outer frames.
>>> Besides, it doesn't matter what happens "in transit" - as long as the
>>> outer headers aren't touched by the transmitting NIC, the network can
>>> do what it likes to them afterwards, without it breaking our TSO code.
>> The IP ID bit is going to be something where I don't want to break
>> things. One of the things I have seen is there ends up being a number
>> of occasions where VXLAN gets fragmented due to incorrectly configured
>> MTU. I would rather not have it get completely screwed up when this
>> occurs. A performance hit for fragmenting is one thing. Having
>> people start complaining because previously working tunnels suddenly
>> stop functioning is another. The fact is the whole point of VXLAN is
>> to support sending the tunneled frames between data centers. With the
>> DF bit set on a tunnel header we end up making it so that frames get
>> dropped instead of fragmented which would be a change of behavior.
> I agree this isn't something we can do silently. But we _can_ make it a
> condition for enabling gso-partial. And I think it's a necessary
> condition for truly generic TSO. Sure, your 'L3 extension header' works
> fine for a single tunnel. But if you nest tunnels, you now need to
> update the outer _and_ middle IP IDs, and you can't do that because you
> only have one L3 header pointer.
This is getting away from the 'less is more' concept. If we are doing
multiple levels of tunnels we have already made things far too
complicated and it is unlikely hardware will ever support anything
like that.
> OTOH, for a single tunnel I think we could implement your 'L3 extension
> header' trick in firmware, by making it always parse the outer packet up
> to the outer L3 header and increment the IP ID in that. So I could live
> with that approach if necessary.
Good to hear.
>> Yes, I am very confident of that. For Intel hardware the outer VLAN
>> tag would be the one inserted via software, the inner VLAN tag is the
>> one inserted via hardware.
> That's really weird; why does it do that?
> For sfc, the only reason we do hardware VLANs at all is to transparently
> tag a VF (or non-primary PF) that's being passed through into an
> (untrusted) VM. For that use case you'd always want to insert the outer
> VLAN tag, because otherwise the VM can escape the isolation by inserting
> a different VLAN tag in software.
The problem is I think the feature was implemented long before any of
the SR-IOV stuff was added and it has been carried that way through
the igb and ixgbe drivers. I haven't looked at i40e but it doesn't
report doing STAG so I don't know how it handles QinQ.
>>> _However_, if we don't need to update the IP IDs, then we can just take
>>> the offset of the inner L4 header, and it doesn't make any difference
>>> whether you choose to think of the stuff before that as L2 + giant L3
>>> or as giant L2 + normal L3, because it's not part of the OS->NIC
>>> interface (which is just "L4 starts here"). If your NIC needs to be
>>> told where the outer L3 starts as well, then, I guess that's just a
>>> wart you need in your drivers. You have skb->network_header, so that
>>> shouldn't be difficult - that will always point to the outer one.
>> Right. That is kind of what I was going for with this setup. The
>> only issue is that the VXLAN tunnels not setting the DF bit kind of
>> get in the way of the giant L3 approach.
Sorry meant giant L2 approach here, not L3.
> On the contrary; your giant L3 approach is exactly what solves this case
> (for non-nested tunnels) - the hardware has the outer IP and inner TCP
> header offsets, which are exactly the two headers it needs to alter.
> And if the hardware is sensible, it won't try to re-checksum the whole
> giant L3 header, it'll just decrement the (outer) IP checksum to account
> for incrementing the (outer) IP ID. If the hardware isn't sensible,
> then you have to play games like you are doing in the Intel drivers ;)
Agreed.
>> Dealing with the outer header needing the DF bit is something I have
>> left unaddressed at this point. The question would be what is the
>> correct approach to take for all this. I know RFC2003 for IPv4 in
>> IPv4 says you must set the DF bit if the inner header has the DF bit
>> set. I'm just wondering if we can apply the same type of logic to GRE
>> and UDP tunnels.
> I wonder if _not_ setting the DF bit in the outer header can harm TCP
> congestion control at all? If so, then we'd pretty much have to set it
> in that case.
Really the only concern with DF based on RFC 2003 is for path MTU
discovery. What happens right now if you set the VXLAN MTU
incorrectly is you end up taking a performance hit as the tunneled
frames are fragmented and have to be reassembled on the other end. It
ends up being a performance hit due to the fragmentation. I might
need to look into this. It is possible that we are already doing the
wrong stuff anyway with this and might need to look at setting the DF
bit anyway.
>> My concern is that I am not sure how much value there is to add with
>> this type of code if the hardware is parsing headers. In the case of
>> most of the Intel parts you specify offsets for the network and
>> transport headers so it gives you some degree of flexibility. If
>> however the hardware parses headers it becomes problematic as we can
>> only support protocols that can be parsed by the hardware.
> Solarflare parts will _normally_ parse headers to get that information.
> But, when doing TSO, we do get a chance to specify some extra
> information, in the TSO option descriptor. Enough of the datapath is
> under firmware control that that should be enough; as long as the outer
> frame is IP over Ethernet, the hardware will parse that fine, and we
> *should* be able to make it just accept that it doesn't know what's
> going on between that and the start of the TCP header. And, it
> shouldn't matter that the hardware can parse some types of tunnel
> headers, because we'll just tell it to ignore that.
Right. Just skipping the tunnel headers makes it quite a bit easier.
> Of course, that means changing the firmware; luckily we haven't got any
> parts in the wild doing tunnel offloads yet, so we still have a chance
> to do that without needing driver code to work around our past
> mistakes...
>
> But this stuff does definitely add value for us, it means we could TSO
> any tunnel type whatsoever; even nested tunnels as long as only the
> outermost IP ID needs to change.
Right. In your case it sounds like you would have the advantage of
just having to run essentially two counters, one increments the IPv4
ID and the other decrements the IPv4 checksum. Beyond that the outer
headers wouldn't need to change at all.
The only other issue would be determining how the inner pseudo-header
checksum is updated. If you were parsing out header fields from the
IP header previously to generate it you would instead need to update
things so that you could use the partial checksum that is already
stored in the TCP header checksum field.
- Alex
^ permalink raw reply
* [PATCH RFC 0/9] udp: GRO in UDP sockets and fast encap_rcv
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
This patch set does two things:
- Add GRO functions (gro_receive and gro_complete) to UDP sockets.
- Implement "fast receive encapsulation".
Add GRO functions (gro_receive and gro_complete) to UDP sockets. In
udp_gro_receive and udp_gro_complete a socket lookup is done instead of
looking up the port number in udp_offloads. If a socket is found and
there is are gro functions for it then those are called. This feature
allows binding GRO functions to more than just a port number.
Eventually, we will be able to use this technique to allow application
defined GRO for an application protocol by attaching BPF porgrams to UDP
sockets for doing GRO.
Fast receive encapsulation allows encap_rcv to be called for a socket
without taking the reference to the socket. The callee can assume it
was called under rcu_read_lock, but must assume that no reference to the
socket is held (so the socket structures should not be written to and
pointer not saved).
In order to implement these function, no socket reference lookup
functions were added in ipv4/udp.c and ipv6/udp.c. Also, inet_iif and
references to skb_dst() were changed to check that dst is set in skbuf
before derefencing. In the GRO path there is now a UDP socket lookup
performed before dst is set, to the get the device in that case we
simply use skb->dev.
Tested:
Ran various combinations of VXLAN and GUE TCP_STREAM and TCP_RR tests.
DId not see any regressions. perf shows a that the atomic opertions in
taking a reference to UDP sockets in lookup is eliminated for
encapsulation.
Also ran plain UDP_RR. DId not see any regression there.
Tom Herbert (9):
net: Check skb_dst for NULL in inet_iif
udp: Add noreference lookup functions
net: Add fast receive encapsulation
udp: Add GRO functions to UDP socket
udp: Add socket based GRO and fast receive encap to tunnel config
vxlan: change vxlan to use UDP socket GRO
fou: change to use UDP socket GRO and fast rcv encap
geneve: change to use UDP socket GRO
udp: Remove udp_offloads
drivers/net/geneve.c | 29 ++----
drivers/net/vxlan.c | 31 ++----
include/linux/netdevice.h | 17 ----
include/linux/udp.h | 13 ++-
include/net/protocol.h | 3 -
include/net/route.h | 7 +-
include/net/udp.h | 19 +++-
include/net/udp_tunnel.h | 8 ++
include/net/vxlan.h | 1 -
net/ipv4/fou.c | 49 ++++------
net/ipv4/udp.c | 237 +++++++++++++++++++++++++++++-----------------
net/ipv4/udp_offload.c | 113 ++++------------------
net/ipv4/udp_tunnel.c | 3 +
net/ipv6/Makefile | 5 +-
net/ipv6/af_inet6.c | 8 ++
net/ipv6/ip6_offload.c | 2 -
net/ipv6/ip6_offload.h | 3 +-
net/ipv6/udp.c | 166 +++++++++++++++++---------------
net/ipv6/udp_offload.c | 11 ++-
19 files changed, 360 insertions(+), 365 deletions(-)
--
2.8.0.rc2
^ permalink raw reply
* [PATCH RFC 3/9] net: Add fast receive encapsulation
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
This patch allows fast receive encapsulation processing. A configuration
flag, encap_fast, may be set in a UDP socket. When this flag is set
encap_rcv may be called without taking a reference to the the
encapsulation socket (which is usually unnecessary since the
encapsulation socket is not written to or saved in an skbuff).
In udp.c the logic to handle encapsulated packets is changed. When
receive a packet:
1) Perform a noref socket lookup.
2) If found socket is an encapsulation socket and encap_fast is set
call encap_rcv with taking a reference
3) If further processing is needed, including calling encap_rcv
when encap_fast is not set, the take a reference to the socket
This patch adds udp_encap_rcv_check to check and run encap_rcv.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/linux/udp.h | 5 +-
net/ipv4/udp.c | 165 ++++++++++++++++++++++++++++++----------------------
net/ipv6/udp.c | 91 +++++++++++------------------
3 files changed, 133 insertions(+), 128 deletions(-)
diff --git a/include/linux/udp.h b/include/linux/udp.h
index 87c0949..f58213e 100644
--- a/include/linux/udp.h
+++ b/include/linux/udp.h
@@ -49,7 +49,8 @@ struct udp_sock {
unsigned int corkflag; /* Cork is required */
__u8 encap_type; /* Is this an Encapsulation socket? */
unsigned char no_check6_tx:1,/* Send zero UDP6 checksums on TX? */
- no_check6_rx:1;/* Allow zero UDP6 checksums on RX? */
+ no_check6_rx:1,/* Allow zero UDP6 checksums on RX? */
+ encap_fast:1; /* Can call encap_rcv wihout ref */
/*
* Following member retains the information to create a UDP header
* when the socket is uncorked.
@@ -98,6 +99,8 @@ static inline bool udp_get_no_check6_rx(struct sock *sk)
return udp_sk(sk)->no_check6_rx;
}
+int udp_encap_rcv_check(struct sock *sk, struct sk_buff *skb);
+
#define udp_portaddr_for_each_entry(__sk, node, list) \
hlist_nulls_for_each_entry(__sk, node, list, __sk_common.skc_portaddr_node)
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 324d008..cb13ec0 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -678,17 +678,6 @@ struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
}
EXPORT_SYMBOL_GPL(udp4_lib_lookup);
-static inline struct sock *__udp4_lib_lookup_skb(struct sk_buff *skb,
- __be16 sport, __be16 dport,
- struct udp_table *udptable)
-{
- const struct iphdr *iph = ip_hdr(skb);
-
- return __udp4_lib_lookup(dev_net(skb_dst(skb)->dev), iph->saddr, sport,
- iph->daddr, dport, inet_iif(skb),
- udptable, skb);
-}
-
static inline struct sock *__udp4_lib_lookup_skb_noref(struct sk_buff *skb,
__be16 sport, __be16 dport,
struct udp_table *udptable)
@@ -1611,9 +1600,65 @@ void udp_encap_enable(void)
EXPORT_SYMBOL(udp_encap_enable);
/* returns:
+ * =0 if skb was successfully passed to the encap
+ * handler or was discarded by it.
+ * >0 if skb should be passed on to UDP.
+ * <0 if skb should be resubmitted as proto -N
+ *
+ * Note that in the success and error cases, the skb is assumed to
+ * have either been requeued or freed.
+ */
+int udp_encap_rcv_check(struct sock *sk, struct sk_buff *skb)
+{
+ struct udp_sock *up = udp_sk(sk);
+ int is_udplite = IS_UDPLITE(sk);
+
+ int (*encap_rcv)(struct sock *sk, struct sk_buff *skb);
+
+ if (!static_key_false(&udp_encap_needed))
+ return 1;
+ /* This is an encapsulation socket so pass the skb to
+ * the socket's udp_encap_rcv() hook. Otherwise, just
+ * fall through and pass this up the UDP socket.
+ * up->encap_rcv() returns the following value:
+ * =0 if skb was successfully passed to the encap
+ * handler or was discarded by it.
+ * >0 if skb should be passed on to UDP.
+ * <0 if skb should be resubmitted as proto -N
+ */
+
+ /* if we're overly short, let UDP handle it */
+ encap_rcv = ACCESS_ONCE(up->encap_rcv);
+ if (skb->len > sizeof(struct udphdr) && encap_rcv) {
+ int ret;
+
+ /* Verify checksum before giving to encap */
+ if (udp_lib_checksum_complete(skb))
+ goto csum_error;
+
+ ret = encap_rcv(sk, skb);
+ if (ret <= 0) {
+ UDP_INC_STATS_BH(sock_net(sk),
+ UDP_MIB_INDATAGRAMS,
+ is_udplite);
+ return ret;
+ }
+ }
+
+ return 1; /* Continue UDP processing */
+
+csum_error:
+ UDP_INC_STATS_BH(sock_net(sk), UDP_MIB_CSUMERRORS, is_udplite);
+ UDP_INC_STATS_BH(sock_net(sk), UDP_MIB_INERRORS, is_udplite);
+ atomic_inc(&sk->sk_drops);
+ kfree_skb(skb);
+ return 0; /* Dropped */
+}
+EXPORT_SYMBOL(udp_encap_rcv_check);
+
+/* returns:
* -1: error
* 0: success
- * >0: "udp encap" protocol resubmission
*
* Note that in the success and error cases, the skb is assumed to
* have either been requeued or freed.
@@ -1621,8 +1666,8 @@ EXPORT_SYMBOL(udp_encap_enable);
int udp_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
{
struct udp_sock *up = udp_sk(sk);
- int rc;
int is_udplite = IS_UDPLITE(sk);
+ int rc;
/*
* Charge it to the socket, dropping if the queue is full.
@@ -1631,41 +1676,6 @@ int udp_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
goto drop;
nf_reset(skb);
- if (static_key_false(&udp_encap_needed) && up->encap_type) {
- int (*encap_rcv)(struct sock *sk, struct sk_buff *skb);
-
- /*
- * This is an encapsulation socket so pass the skb to
- * the socket's udp_encap_rcv() hook. Otherwise, just
- * fall through and pass this up the UDP socket.
- * up->encap_rcv() returns the following value:
- * =0 if skb was successfully passed to the encap
- * handler or was discarded by it.
- * >0 if skb should be passed on to UDP.
- * <0 if skb should be resubmitted as proto -N
- */
-
- /* if we're overly short, let UDP handle it */
- encap_rcv = ACCESS_ONCE(up->encap_rcv);
- if (skb->len > sizeof(struct udphdr) && encap_rcv) {
- int ret;
-
- /* Verify checksum before giving to encap */
- if (udp_lib_checksum_complete(skb))
- goto csum_error;
-
- ret = encap_rcv(sk, skb);
- if (ret <= 0) {
- UDP_INC_STATS_BH(sock_net(sk),
- UDP_MIB_INDATAGRAMS,
- is_udplite);
- return -ret;
- }
- }
-
- /* FALLTHROUGH -- it's a UDP Packet */
- }
-
/*
* UDP-Lite specific tests, ignored on UDP sockets
*/
@@ -1864,10 +1874,14 @@ static inline int udp4_csum_init(struct sk_buff *skb, struct udphdr *uh,
inet_compute_pseudo);
}
-/*
- * All we need to do is get the socket, and then do a checksum.
+/* Process a received UDP packet. Validate the packet and checksum,
+ * lookup a socket, either receive packet on that socket or call
+ * the specified encapsulation receive function.
+ *
+ * Returns:
+ * 0: packet was received or consumed
+ * <0: "udp encap" protocol resubmission
*/
-
int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
int proto)
{
@@ -1905,26 +1919,20 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
sk = skb_steal_sock(skb);
if (sk) {
struct dst_entry *dst = skb_dst(skb);
- int ret;
if (unlikely(sk->sk_rx_dst != dst))
udp_sk_rx_dst_set(sk, dst);
- ret = udp_queue_rcv_skb(sk, skb);
- sock_put(sk);
- /* a return value > 0 means to resubmit the input, but
- * it wants the return to be -protocol, or 0
- */
- if (ret > 0)
- return -ret;
- return 0;
+ goto have_ref_sock;
}
if (rt->rt_flags & (RTCF_BROADCAST|RTCF_MULTICAST))
return __udp4_lib_mcast_deliver(net, skb, uh,
saddr, daddr, udptable, proto);
- sk = __udp4_lib_lookup_skb(skb, uh->source, uh->dest, udptable);
+ rcu_read_lock();
+ /* Don't take socket reference unless we need to */
+ sk = __udp4_lib_lookup_skb_noref(skb, uh->source, uh->dest, udptable);
if (sk) {
int ret;
@@ -1932,15 +1940,34 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check,
inet_compute_pseudo);
- ret = udp_queue_rcv_skb(sk, skb);
- sock_put(sk);
+ if (udp_sk(sk)->encap_type && udp_sk(sk)->encap_fast) {
+ ret = udp_encap_rcv_check(sk, skb);
+ if (ret <= 0) {
+ rcu_read_unlock();
+ return ret;
+ }
+ }
- /* a return value > 0 means to resubmit the input, but
- * it wants the return to be -protocol, or 0
- */
- if (ret > 0)
- return -ret;
- return 0;
+ /* Okay, need reference for futher processing */
+ sk = udp_get_ref(sk);
+ rcu_read_unlock();
+
+ if (sk) {
+have_ref_sock:
+ if (udp_sk(sk)->encap_type && !udp_sk(sk)->encap_fast) {
+ /* Did not check for encap yet */
+ ret = udp_encap_rcv_check(sk, skb);
+ if (ret <= 0) {
+ sock_put(sk);
+ return ret;
+ }
+ }
+ ret = udp_queue_rcv_skb(sk, skb);
+ sock_put(sk);
+ return ret;
+ }
+ } else {
+ rcu_read_unlock();
}
if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb))
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 281469c..cbcac8f 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -386,21 +386,6 @@ struct sock *__udp6_lib_lookup(struct net *net,
}
EXPORT_SYMBOL_GPL(__udp6_lib_lookup);
-static struct sock *__udp6_lib_lookup_skb(struct sk_buff *skb,
- __be16 sport, __be16 dport,
- struct udp_table *udptable)
-{
- struct sock *sk;
- const struct ipv6hdr *iph = ipv6_hdr(skb);
-
- sk = skb_steal_sock(skb);
- if (unlikely(sk))
- return sk;
- return __udp6_lib_lookup(dev_net(skb_dst(skb)->dev), &iph->saddr, sport,
- &iph->daddr, dport, inet6_iif(skb),
- udptable, skb);
-}
-
struct sock *udp6_lib_lookup(struct net *net, const struct in6_addr *saddr, __be16 sport,
const struct in6_addr *daddr, __be16 dport, int dif)
{
@@ -676,41 +661,6 @@ int udpv6_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
if (!xfrm6_policy_check(sk, XFRM_POLICY_IN, skb))
goto drop;
- if (static_key_false(&udpv6_encap_needed) && up->encap_type) {
- int (*encap_rcv)(struct sock *sk, struct sk_buff *skb);
-
- /*
- * This is an encapsulation socket so pass the skb to
- * the socket's udp_encap_rcv() hook. Otherwise, just
- * fall through and pass this up the UDP socket.
- * up->encap_rcv() returns the following value:
- * =0 if skb was successfully passed to the encap
- * handler or was discarded by it.
- * >0 if skb should be passed on to UDP.
- * <0 if skb should be resubmitted as proto -N
- */
-
- /* if we're overly short, let UDP handle it */
- encap_rcv = ACCESS_ONCE(up->encap_rcv);
- if (skb->len > sizeof(struct udphdr) && encap_rcv) {
- int ret;
-
- /* Verify checksum before giving to encap */
- if (udp_lib_checksum_complete(skb))
- goto csum_error;
-
- ret = encap_rcv(sk, skb);
- if (ret <= 0) {
- UDP_INC_STATS_BH(sock_net(sk),
- UDP_MIB_INDATAGRAMS,
- is_udplite);
- return -ret;
- }
- }
-
- /* FALLTHROUGH -- it's a UDP Packet */
- }
-
/*
* UDP-Lite specific tests, ignored on UDP sockets (see net/ipv4/udp.c).
*/
@@ -944,7 +894,13 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
* check socket cache ... must talk to Alan about his plans
* for sock caches... i'll skip this for now.
*/
- sk = __udp6_lib_lookup_skb(skb, uh->source, uh->dest, udptable);
+ sk = skb_steal_sock(skb);
+ if (unlikely(sk))
+ goto have_ref_sock;
+
+ rcu_read_lock();
+ /* Don't take socket reference unless we need to */
+ sk = __udp6_lib_lookup_skb_noref(skb, uh->source, uh->dest, udptable);
if (sk) {
int ret;
@@ -958,14 +914,33 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check,
ip6_compute_pseudo);
- ret = udpv6_queue_rcv_skb(sk, skb);
- sock_put(sk);
-
- /* a return value > 0 means to resubmit the input */
- if (ret > 0)
- return ret;
+ if (udp_sk(sk)->encap_type && udp_sk(sk)->encap_fast) {
+ ret = udp_encap_rcv_check(sk, skb);
+ if (ret) {
+ rcu_read_unlock();
+ return ret > 0 ? -ret : 0;
+ }
+ }
- return 0;
+ /* Okay, need reference for futher processing */
+ sk = udp_get_ref(sk);
+ rcu_read_unlock();
+
+ if (sk) {
+have_ref_sock:
+ if (udp_sk(sk)->encap_type && !udp_sk(sk)->encap_fast) {
+ /* Did not check for encap yet */
+ ret = udp_encap_rcv_check(sk, skb);
+ if (ret)
+ goto out;
+ }
+ ret = udpv6_queue_rcv_skb(sk, skb);
+out:
+ sock_put(sk);
+ return ret > 0 ? -ret : 0;
+ }
+ } else {
+ rcu_read_unlock();
}
if (!uh->check) {
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 4/9] udp: Add GRO functions to UDP socket
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
This patch adds GRO functions (gro_receive and gro_complete) to UDP
sockets. udp_gro_receive is changed to perfrom a noref socket lookup
on a packet. If a socket is found the related GRO functions are called.
This features obsoletes using UDP offload infrastructure for GRO
(udp_offload). This has the advantage of not being limited to provide
offload on a per port basis, GRO is now applied to whatever individual
UDP sockets are bound to. This also allows the possbility of
"application defined GRO"-- that is we can attach something like
a BPF program to a UDP socket to perfrom GRO on an application
layer protocol.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/linux/udp.h | 8 ++++++++
include/net/udp.h | 7 +++++--
net/ipv4/udp_offload.c | 52 +++++++++++++++++++-------------------------------
net/ipv6/Makefile | 5 +++--
net/ipv6/af_inet6.c | 8 ++++++++
net/ipv6/ip6_offload.c | 2 --
net/ipv6/ip6_offload.h | 3 ++-
net/ipv6/udp_offload.c | 11 ++++++++---
8 files changed, 54 insertions(+), 42 deletions(-)
diff --git a/include/linux/udp.h b/include/linux/udp.h
index f58213e..e905103 100644
--- a/include/linux/udp.h
+++ b/include/linux/udp.h
@@ -72,6 +72,14 @@ struct udp_sock {
*/
int (*encap_rcv)(struct sock *sk, struct sk_buff *skb);
void (*encap_destroy)(struct sock *sk);
+
+ /* GRO functions for UDP socket */
+ struct sk_buff ** (*gro_receive)(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb);
+ int (*gro_complete)(struct sock *sk,
+ struct sk_buff *skb,
+ int nhoff);
};
static inline struct udp_sock *udp_sk(const struct sock *sk)
diff --git a/include/net/udp.h b/include/net/udp.h
index 2a6f7b2..94e0ab6 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -158,9 +158,12 @@ static inline __sum16 udp_v4_check(int len, __be32 saddr,
void udp_set_csum(bool nocheck, struct sk_buff *skb,
__be32 saddr, __be32 daddr, int len);
+typedef struct sock *(*udp_lookup_t)(struct sk_buff *skb, __be16 sport,
+ __be16 dport);
+
struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb,
- struct udphdr *uh);
-int udp_gro_complete(struct sk_buff *skb, int nhoff);
+ struct udphdr *uh, udp_lookup_t lookup);
+int udp_gro_complete(struct sk_buff *skb, int nhoff, udp_lookup_t lookup);
static inline struct udphdr *udp_gro_udphdr(struct sk_buff *skb)
{
diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c
index 0e89f80..dce43c9 100644
--- a/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -179,6 +179,7 @@ out_unlock:
return segs;
}
+EXPORT_SYMBOL(skb_udp_tunnel_segment);
static struct sk_buff *udp4_ufo_fragment(struct sk_buff *skb,
netdev_features_t features)
@@ -304,13 +305,13 @@ unlock:
EXPORT_SYMBOL(udp_del_offload);
struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb,
- struct udphdr *uh)
+ struct udphdr *uh, udp_lookup_t lookup)
{
- struct udp_offload_priv *uo_priv;
struct sk_buff *p, **pp = NULL;
struct udphdr *uh2;
unsigned int off = skb_gro_offset(skb);
int flush = 1;
+ struct sock *sk;
if (NAPI_GRO_CB(skb)->udp_mark ||
(skb->ip_summed != CHECKSUM_PARTIAL &&
@@ -322,13 +323,11 @@ struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb,
NAPI_GRO_CB(skb)->udp_mark = 1;
rcu_read_lock();
- uo_priv = rcu_dereference(udp_offload_base);
- for (; uo_priv != NULL; uo_priv = rcu_dereference(uo_priv->next)) {
- if (net_eq(read_pnet(&uo_priv->net), dev_net(skb->dev)) &&
- uo_priv->offload->port == uh->dest &&
- uo_priv->offload->callbacks.gro_receive)
- goto unflush;
- }
+ sk = (*lookup)(skb, uh->source, uh->dest);
+
+ if (sk && udp_sk(sk)->gro_receive)
+ goto unflush;
+
goto out_unlock;
unflush:
@@ -352,9 +351,7 @@ unflush:
skb_gro_pull(skb, sizeof(struct udphdr)); /* pull encapsulating udp header */
skb_gro_postpull_rcsum(skb, uh, sizeof(struct udphdr));
- NAPI_GRO_CB(skb)->proto = uo_priv->offload->ipproto;
- pp = uo_priv->offload->callbacks.gro_receive(head, skb,
- uo_priv->offload);
+ pp = udp_sk(sk)->gro_receive(sk, head, skb);
out_unlock:
rcu_read_unlock();
@@ -362,6 +359,7 @@ out:
NAPI_GRO_CB(skb)->flush |= flush;
return pp;
}
+EXPORT_SYMBOL(udp_gro_receive);
static struct sk_buff **udp4_gro_receive(struct sk_buff **head,
struct sk_buff *skb)
@@ -383,39 +381,28 @@ static struct sk_buff **udp4_gro_receive(struct sk_buff **head,
inet_gro_compute_pseudo);
skip:
NAPI_GRO_CB(skb)->is_ipv6 = 0;
- return udp_gro_receive(head, skb, uh);
+ return udp_gro_receive(head, skb, uh, udp4_lib_lookup_skb_noref);
flush:
NAPI_GRO_CB(skb)->flush = 1;
return NULL;
}
-int udp_gro_complete(struct sk_buff *skb, int nhoff)
+int udp_gro_complete(struct sk_buff *skb, int nhoff,
+ udp_lookup_t lookup)
{
- struct udp_offload_priv *uo_priv;
__be16 newlen = htons(skb->len - nhoff);
struct udphdr *uh = (struct udphdr *)(skb->data + nhoff);
int err = -ENOSYS;
+ struct sock *sk;
uh->len = newlen;
rcu_read_lock();
-
- uo_priv = rcu_dereference(udp_offload_base);
- for (; uo_priv != NULL; uo_priv = rcu_dereference(uo_priv->next)) {
- if (net_eq(read_pnet(&uo_priv->net), dev_net(skb->dev)) &&
- uo_priv->offload->port == uh->dest &&
- uo_priv->offload->callbacks.gro_complete)
- break;
- }
-
- if (uo_priv) {
- NAPI_GRO_CB(skb)->proto = uo_priv->offload->ipproto;
- err = uo_priv->offload->callbacks.gro_complete(skb,
- nhoff + sizeof(struct udphdr),
- uo_priv->offload);
- }
-
+ sk = (*lookup)(skb, uh->source, uh->dest);
+ if (sk && udp_sk(sk)->gro_complete)
+ err = udp_sk(sk)->gro_complete(sk, skb,
+ nhoff + sizeof(struct udphdr));
rcu_read_unlock();
if (skb->remcsum_offload)
@@ -426,6 +413,7 @@ int udp_gro_complete(struct sk_buff *skb, int nhoff)
return err;
}
+EXPORT_SYMBOL(udp_gro_complete);
static int udp4_gro_complete(struct sk_buff *skb, int nhoff)
{
@@ -440,7 +428,7 @@ static int udp4_gro_complete(struct sk_buff *skb, int nhoff)
skb_shinfo(skb)->gso_type |= SKB_GSO_UDP_TUNNEL;
}
- return udp_gro_complete(skb, nhoff);
+ return udp_gro_complete(skb, nhoff, udp4_lib_lookup_skb_noref);
}
static const struct net_offload udpv4_offload = {
diff --git a/net/ipv6/Makefile b/net/ipv6/Makefile
index 2fbd90b..5e9d6bf 100644
--- a/net/ipv6/Makefile
+++ b/net/ipv6/Makefile
@@ -8,9 +8,10 @@ ipv6-objs := af_inet6.o anycast.o ip6_output.o ip6_input.o addrconf.o \
addrlabel.o \
route.o ip6_fib.o ipv6_sockglue.o ndisc.o udp.o udplite.o \
raw.o icmp.o mcast.o reassembly.o tcp_ipv6.o ping.o \
- exthdrs.o datagram.o ip6_flowlabel.o inet6_connection_sock.o
+ exthdrs.o datagram.o ip6_flowlabel.o inet6_connection_sock.o \
+ udp_offload.o
-ipv6-offload := ip6_offload.o tcpv6_offload.o udp_offload.o exthdrs_offload.o
+ipv6-offload := ip6_offload.o tcpv6_offload.o exthdrs_offload.o
ipv6-$(CONFIG_SYSCTL) = sysctl_net_ipv6.o
ipv6-$(CONFIG_IPV6_MROUTE) += ip6mr.o
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index b11c37c..9a2fec5 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -64,6 +64,8 @@
#include <asm/uaccess.h>
#include <linux/mroute6.h>
+#include "ip6_offload.h"
+
MODULE_AUTHOR("Cast of dozens");
MODULE_DESCRIPTION("IPv6 protocol stack for Linux");
MODULE_LICENSE("GPL");
@@ -958,6 +960,10 @@ static int __init inet6_init(void)
if (err)
goto udplitev6_fail;
+ err = udpv6_offload_init();
+ if (err)
+ goto udpv6_offload_fail;
+
err = tcpv6_init();
if (err)
goto tcpv6_fail;
@@ -987,6 +993,8 @@ pingv6_fail:
ipv6_packet_fail:
tcpv6_exit();
tcpv6_fail:
+ udpv6_offload_exit();
+udpv6_offload_fail:
udplitev6_exit();
udplitev6_fail:
udpv6_exit();
diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
index eeca943..d96c5e2 100644
--- a/net/ipv6/ip6_offload.c
+++ b/net/ipv6/ip6_offload.c
@@ -312,8 +312,6 @@ static int __init ipv6_offload_init(void)
if (tcpv6_offload_init() < 0)
pr_crit("%s: Cannot add TCP protocol offload\n", __func__);
- if (udp_offload_init() < 0)
- pr_crit("%s: Cannot add UDP protocol offload\n", __func__);
if (ipv6_exthdrs_offload_init() < 0)
pr_crit("%s: Cannot add EXTHDRS protocol offload\n", __func__);
diff --git a/net/ipv6/ip6_offload.h b/net/ipv6/ip6_offload.h
index 2e155c6..96b40e4 100644
--- a/net/ipv6/ip6_offload.h
+++ b/net/ipv6/ip6_offload.h
@@ -12,7 +12,8 @@
#define __ip6_offload_h
int ipv6_exthdrs_offload_init(void);
-int udp_offload_init(void);
+int udpv6_offload_init(void);
+int udpv6_offload_exit(void);
int tcpv6_offload_init(void);
#endif
diff --git a/net/ipv6/udp_offload.c b/net/ipv6/udp_offload.c
index 2b0fbe6..4a822f6 100644
--- a/net/ipv6/udp_offload.c
+++ b/net/ipv6/udp_offload.c
@@ -153,7 +153,7 @@ static struct sk_buff **udp6_gro_receive(struct sk_buff **head,
skip:
NAPI_GRO_CB(skb)->is_ipv6 = 1;
- return udp_gro_receive(head, skb, uh);
+ return udp_gro_receive(head, skb, uh, udp6_lib_lookup_skb_noref);
flush:
NAPI_GRO_CB(skb)->flush = 1;
@@ -173,7 +173,7 @@ static int udp6_gro_complete(struct sk_buff *skb, int nhoff)
skb_shinfo(skb)->gso_type |= SKB_GSO_UDP_TUNNEL;
}
- return udp_gro_complete(skb, nhoff);
+ return udp_gro_complete(skb, nhoff, udp6_lib_lookup_skb_noref);
}
static const struct net_offload udpv6_offload = {
@@ -184,7 +184,12 @@ static const struct net_offload udpv6_offload = {
},
};
-int __init udp_offload_init(void)
+int udpv6_offload_init(void)
{
return inet6_add_offload(&udpv6_offload, IPPROTO_UDP);
}
+
+int udpv6_offload_exit(void)
+{
+ return inet6_del_offload(&udpv6_offload, IPPROTO_UDP);
+}
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 6/9] vxlan: change vxlan to use UDP socket GRO
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
Adapt vxlan_gro_receive, vxlan_gro_complete to take a socket argument.
Set these functions in tunnel_config. Don't set udp_offloads any more.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
drivers/net/vxlan.c | 31 +++++++++----------------------
include/net/vxlan.h | 1 -
2 files changed, 9 insertions(+), 23 deletions(-)
diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 800106a7..d27aceb 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -551,16 +551,15 @@ static struct vxlanhdr *vxlan_gro_remcsum(struct sk_buff *skb,
return vh;
}
-static struct sk_buff **vxlan_gro_receive(struct sk_buff **head,
- struct sk_buff *skb,
- struct udp_offload *uoff)
+static struct sk_buff **vxlan_gro_receive(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb)
{
struct sk_buff *p, **pp = NULL;
struct vxlanhdr *vh, *vh2;
unsigned int hlen, off_vx;
int flush = 1;
- struct vxlan_sock *vs = container_of(uoff, struct vxlan_sock,
- udp_offloads);
+ struct vxlan_sock *vs = rcu_dereference_sk_user_data(sk);
__be32 flags;
struct gro_remcsum grc;
@@ -613,8 +612,7 @@ out:
return pp;
}
-static int vxlan_gro_complete(struct sk_buff *skb, int nhoff,
- struct udp_offload *uoff)
+static int vxlan_gro_complete(struct sock *sk, struct sk_buff *skb, int nhoff)
{
udp_tunnel_gro_complete(skb, nhoff);
@@ -629,13 +627,6 @@ static void vxlan_notify_add_rx_port(struct vxlan_sock *vs)
struct net *net = sock_net(sk);
sa_family_t sa_family = vxlan_get_sk_family(vs);
__be16 port = inet_sk(sk)->inet_sport;
- int err;
-
- if (sa_family == AF_INET) {
- err = udp_add_offload(net, &vs->udp_offloads);
- if (err)
- pr_warn("vxlan: udp_add_offload failed with status %d\n", err);
- }
rcu_read_lock();
for_each_netdev_rcu(net, dev) {
@@ -662,9 +653,6 @@ static void vxlan_notify_del_rx_port(struct vxlan_sock *vs)
port);
}
rcu_read_unlock();
-
- if (sa_family == AF_INET)
- udp_del_offload(&vs->udp_offloads);
}
/* Add new entry to forwarding table -- assumes lock held */
@@ -2643,21 +2631,20 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6,
atomic_set(&vs->refcnt, 1);
vs->flags = (flags & VXLAN_F_RCV_FLAGS);
- /* Initialize the vxlan udp offloads structure */
- vs->udp_offloads.port = port;
- vs->udp_offloads.callbacks.gro_receive = vxlan_gro_receive;
- vs->udp_offloads.callbacks.gro_complete = vxlan_gro_complete;
-
spin_lock(&vn->sock_lock);
hlist_add_head_rcu(&vs->hlist, vs_head(net, port));
vxlan_notify_add_rx_port(vs);
spin_unlock(&vn->sock_lock);
/* Mark socket as an encapsulation socket. */
+ memset(&tunnel_cfg, 0, sizeof(tunnel_cfg));
tunnel_cfg.sk_user_data = vs;
tunnel_cfg.encap_type = 1;
tunnel_cfg.encap_rcv = vxlan_rcv;
tunnel_cfg.encap_destroy = NULL;
+ tunnel_cfg.encap_fast = 0;
+ tunnel_cfg.gro_receive = vxlan_gro_receive;
+ tunnel_cfg.gro_complete = vxlan_gro_complete;
setup_udp_tunnel_sock(net, sock, &tunnel_cfg);
diff --git a/include/net/vxlan.h b/include/net/vxlan.h
index a763c96..b4ba1d73 100644
--- a/include/net/vxlan.h
+++ b/include/net/vxlan.h
@@ -131,7 +131,6 @@ struct vxlan_sock {
struct rcu_head rcu;
struct hlist_head vni_list[VNI_HASH_SIZE];
atomic_t refcnt;
- struct udp_offload udp_offloads;
u32 flags;
};
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 8/9] geneve: change to use UDP socket GRO
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
Adapt geneve_gro_receive, geneve_gro_complete to take a socket argument.
Set these functions in tunnel_config. Don't set udp_offloads any more.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
drivers/net/geneve.c | 29 +++++++++--------------------
1 file changed, 9 insertions(+), 20 deletions(-)
diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c
index 192631a..6caf9ed 100644
--- a/drivers/net/geneve.c
+++ b/drivers/net/geneve.c
@@ -87,7 +87,6 @@ struct geneve_sock {
struct socket *sock;
struct rcu_head rcu;
int refcnt;
- struct udp_offload udp_offloads;
struct hlist_head vni_list[VNI_HASH_SIZE];
u32 flags;
};
@@ -409,14 +408,6 @@ static void geneve_notify_add_rx_port(struct geneve_sock *gs)
struct net *net = sock_net(sk);
sa_family_t sa_family = geneve_get_sk_family(gs);
__be16 port = inet_sk(sk)->inet_sport;
- int err;
-
- if (sa_family == AF_INET) {
- err = udp_add_offload(sock_net(sk), &gs->udp_offloads);
- if (err)
- pr_warn("geneve: udp_add_offload failed with status %d\n",
- err);
- }
rcu_read_lock();
for_each_netdev_rcu(net, dev) {
@@ -432,9 +423,9 @@ static int geneve_hlen(struct genevehdr *gh)
return sizeof(*gh) + gh->opt_len * 4;
}
-static struct sk_buff **geneve_gro_receive(struct sk_buff **head,
- struct sk_buff *skb,
- struct udp_offload *uoff)
+static struct sk_buff **geneve_gro_receive(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb)
{
struct sk_buff *p, **pp = NULL;
struct genevehdr *gh, *gh2;
@@ -495,8 +486,8 @@ out:
return pp;
}
-static int geneve_gro_complete(struct sk_buff *skb, int nhoff,
- struct udp_offload *uoff)
+static int geneve_gro_complete(struct sock *sk, struct sk_buff *skb,
+ int nhoff)
{
struct genevehdr *gh;
struct packet_offload *ptype;
@@ -545,14 +536,15 @@ static struct geneve_sock *geneve_socket_create(struct net *net, __be16 port,
INIT_HLIST_HEAD(&gs->vni_list[h]);
/* Initialize the geneve udp offloads structure */
- gs->udp_offloads.port = port;
- gs->udp_offloads.callbacks.gro_receive = geneve_gro_receive;
- gs->udp_offloads.callbacks.gro_complete = geneve_gro_complete;
geneve_notify_add_rx_port(gs);
/* Mark socket as an encapsulation socket */
+ memset(&tunnel_cfg, 0, sizeof(tunnel_cfg));
tunnel_cfg.sk_user_data = gs;
tunnel_cfg.encap_type = 1;
+ tunnel_cfg.encap_fast = 0;
+ tunnel_cfg.gro_receive = geneve_gro_receive;
+ tunnel_cfg.gro_complete = geneve_gro_complete;
tunnel_cfg.encap_rcv = geneve_udp_encap_recv;
tunnel_cfg.encap_destroy = NULL;
setup_udp_tunnel_sock(net, sock, &tunnel_cfg);
@@ -576,9 +568,6 @@ static void geneve_notify_del_rx_port(struct geneve_sock *gs)
}
rcu_read_unlock();
-
- if (sa_family == AF_INET)
- udp_del_offload(&gs->udp_offloads);
}
static void __geneve_sock_release(struct geneve_sock *gs)
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 1/9] net: Check skb_dst for NULL in inet_iif
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
In inet_iif check if skb_rtable is NULL for the skb and return
skb->skb_iif if it is.
This change allows inet_iif to be called before the dst
information has been set in the skb (e.g. when doing socket based
UDP GRO).
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/net/route.h | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/net/route.h b/include/net/route.h
index 9b0a523..f4b11ee 100644
--- a/include/net/route.h
+++ b/include/net/route.h
@@ -322,10 +322,11 @@ static inline struct rtable *ip_route_newports(struct flowi4 *fl4, struct rtable
static inline int inet_iif(const struct sk_buff *skb)
{
- int iif = skb_rtable(skb)->rt_iif;
+ struct rtable *rt = skb_rtable(skb);
+
+ if (rt && rt->rt_iif)
+ return rt->rt_iif;
- if (iif)
- return iif;
return skb->skb_iif;
}
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 5/9] udp: Add socket based GRO and fast receive encap to tunnel config
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
Add gro_receive, gro_complete and encap_fast to
struct udp_tunnel_sock_cfg.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/net/udp_tunnel.h | 8 ++++++++
net/ipv4/udp_tunnel.c | 3 +++
2 files changed, 11 insertions(+)
diff --git a/include/net/udp_tunnel.h b/include/net/udp_tunnel.h
index b831140..3c534e7 100644
--- a/include/net/udp_tunnel.h
+++ b/include/net/udp_tunnel.h
@@ -64,13 +64,21 @@ static inline int udp_sock_create(struct net *net,
typedef int (*udp_tunnel_encap_rcv_t)(struct sock *sk, struct sk_buff *skb);
typedef void (*udp_tunnel_encap_destroy_t)(struct sock *sk);
+typedef struct sk_buff **(*udp_tunnel_gro_receive_t)(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb);
+typedef int (*udp_tunnel_gro_complete_t)(struct sock *sk, struct sk_buff *skb,
+ int nhoff);
struct udp_tunnel_sock_cfg {
void *sk_user_data; /* user data used by encap_rcv call back */
/* Used for setting up udp_sock fields, see udp.h for details */
__u8 encap_type;
+ __u8 encap_fast:1;
udp_tunnel_encap_rcv_t encap_rcv;
udp_tunnel_encap_destroy_t encap_destroy;
+ udp_tunnel_gro_receive_t gro_receive;
+ udp_tunnel_gro_complete_t gro_complete;
};
/* Setup the given (UDP) sock to receive UDP encapsulated packets */
diff --git a/net/ipv4/udp_tunnel.c b/net/ipv4/udp_tunnel.c
index 96599d1..b5b13f8 100644
--- a/net/ipv4/udp_tunnel.c
+++ b/net/ipv4/udp_tunnel.c
@@ -69,6 +69,9 @@ void setup_udp_tunnel_sock(struct net *net, struct socket *sock,
udp_sk(sk)->encap_type = cfg->encap_type;
udp_sk(sk)->encap_rcv = cfg->encap_rcv;
udp_sk(sk)->encap_destroy = cfg->encap_destroy;
+ udp_sk(sk)->encap_fast = cfg->encap_fast;
+ udp_sk(sk)->gro_receive = cfg->gro_receive;
+ udp_sk(sk)->gro_complete = cfg->gro_complete;
udp_tunnel_encap_enable(sock);
}
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 7/9] fou: change to use UDP socket GRO and fast rcv encap
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
Adapt gue_gro_receive, gue_gro_complete to take a socket argument.
Set the encap_fast flag on fou sockets. Don't set udp_offloads
any more.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
net/ipv4/fou.c | 49 ++++++++++++++++++-------------------------------
1 file changed, 18 insertions(+), 31 deletions(-)
diff --git a/net/ipv4/fou.c b/net/ipv4/fou.c
index 7804842..a90d179 100644
--- a/net/ipv4/fou.c
+++ b/net/ipv4/fou.c
@@ -22,7 +22,6 @@ struct fou {
u8 flags;
__be16 port;
u16 type;
- struct udp_offload udp_offloads;
struct list_head list;
struct rcu_head rcu;
};
@@ -177,13 +176,13 @@ drop:
return 0;
}
-static struct sk_buff **fou_gro_receive(struct sk_buff **head,
- struct sk_buff *skb,
- struct udp_offload *uoff)
+static struct sk_buff **fou_gro_receive(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb)
{
const struct net_offload *ops;
struct sk_buff **pp = NULL;
- u8 proto = NAPI_GRO_CB(skb)->proto;
+ u8 proto = fou_from_sock(sk)->protocol;
const struct net_offload **offloads;
rcu_read_lock();
@@ -200,11 +199,11 @@ out_unlock:
return pp;
}
-static int fou_gro_complete(struct sk_buff *skb, int nhoff,
- struct udp_offload *uoff)
+static int fou_gro_complete(struct sock *sk, struct sk_buff *skb,
+ int nhoff)
{
const struct net_offload *ops;
- u8 proto = NAPI_GRO_CB(skb)->proto;
+ u8 proto = fou_from_sock(sk)->protocol;
int err = -ENOSYS;
const struct net_offload **offloads;
@@ -247,9 +246,9 @@ static struct guehdr *gue_gro_remcsum(struct sk_buff *skb, unsigned int off,
return guehdr;
}
-static struct sk_buff **gue_gro_receive(struct sk_buff **head,
- struct sk_buff *skb,
- struct udp_offload *uoff)
+static struct sk_buff **gue_gro_receive(struct sock *sk,
+ struct sk_buff **head,
+ struct sk_buff *skb)
{
const struct net_offload **offloads;
const struct net_offload *ops;
@@ -260,7 +259,7 @@ static struct sk_buff **gue_gro_receive(struct sk_buff **head,
void *data;
u16 doffset = 0;
int flush = 1;
- struct fou *fou = container_of(uoff, struct fou, udp_offloads);
+ struct fou *fou = fou_from_sock(sk);
struct gro_remcsum grc;
skb_gro_remcsum_init(&grc);
@@ -361,8 +360,7 @@ out:
return pp;
}
-static int gue_gro_complete(struct sk_buff *skb, int nhoff,
- struct udp_offload *uoff)
+static int gue_gro_complete(struct sock *sk, struct sk_buff *skb, int nhoff)
{
const struct net_offload **offloads;
struct guehdr *guehdr = (struct guehdr *)(skb->data + nhoff);
@@ -410,10 +408,7 @@ static int fou_add_to_port_list(struct net *net, struct fou *fou)
static void fou_release(struct fou *fou)
{
struct socket *sock = fou->sock;
- struct sock *sk = sock->sk;
- if (sk->sk_family == AF_INET)
- udp_del_offload(&fou->udp_offloads);
list_del(&fou->list);
udp_tunnel_sock_release(sock);
@@ -422,12 +417,10 @@ static void fou_release(struct fou *fou)
static int fou_encap_init(struct sock *sk, struct fou *fou, struct fou_cfg *cfg)
{
- udp_sk(sk)->encap_rcv = fou_udp_recv;
fou->protocol = cfg->protocol;
- fou->udp_offloads.callbacks.gro_receive = fou_gro_receive;
- fou->udp_offloads.callbacks.gro_complete = fou_gro_complete;
- fou->udp_offloads.port = cfg->udp_config.local_udp_port;
- fou->udp_offloads.ipproto = cfg->protocol;
+ udp_sk(sk)->encap_rcv = fou_udp_recv;
+ udp_sk(sk)->gro_receive = fou_gro_receive;
+ udp_sk(sk)->gro_complete = fou_gro_complete;
return 0;
}
@@ -435,9 +428,8 @@ static int fou_encap_init(struct sock *sk, struct fou *fou, struct fou_cfg *cfg)
static int gue_encap_init(struct sock *sk, struct fou *fou, struct fou_cfg *cfg)
{
udp_sk(sk)->encap_rcv = gue_udp_recv;
- fou->udp_offloads.callbacks.gro_receive = gue_gro_receive;
- fou->udp_offloads.callbacks.gro_complete = gue_gro_complete;
- fou->udp_offloads.port = cfg->udp_config.local_udp_port;
+ udp_sk(sk)->gro_receive = gue_gro_receive;
+ udp_sk(sk)->gro_complete = gue_gro_complete;
return 0;
}
@@ -487,6 +479,7 @@ static int fou_create(struct net *net, struct fou_cfg *cfg,
fou->type = cfg->type;
udp_sk(sk)->encap_type = 1;
+ udp_sk(sk)->encap_fast = 1;
udp_encap_enable();
sk->sk_user_data = fou;
@@ -496,12 +489,6 @@ static int fou_create(struct net *net, struct fou_cfg *cfg,
sk->sk_allocation = GFP_ATOMIC;
- if (cfg->udp_config.family == AF_INET) {
- err = udp_add_offload(net, &fou->udp_offloads);
- if (err)
- goto error;
- }
-
err = fou_add_to_port_list(net, fou);
if (err)
goto error;
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 2/9] udp: Add noreference lookup functions
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
This patches adds udp6_lib_lookup_skb_noref, udp4_lib_lookup_skb_noref
and related support to allow a caller to lookup a UDP socket
without automatically taking a reference. The lookup and caller
use of the socket must be done under rcu_read_lock.
This feature will be used in a fast receive encapsulation path and
also when performing GRO through callout in the UDP socket.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/net/udp.h | 12 ++++++++
net/ipv4/udp.c | 88 +++++++++++++++++++++++++++++++++++++++----------------
net/ipv6/udp.c | 85 ++++++++++++++++++++++++++++++++++++++---------------
3 files changed, 136 insertions(+), 49 deletions(-)
diff --git a/include/net/udp.h b/include/net/udp.h
index 92927f7..2a6f7b2 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -260,6 +260,8 @@ struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
__be32 daddr, __be16 dport, int dif,
struct udp_table *tbl, struct sk_buff *skb);
+struct sock *udp4_lib_lookup_skb_noref(struct sk_buff *skb,
+ __be16 sport, __be16 dport);
struct sock *udp6_lib_lookup(struct net *net,
const struct in6_addr *saddr, __be16 sport,
const struct in6_addr *daddr, __be16 dport,
@@ -269,6 +271,16 @@ struct sock *__udp6_lib_lookup(struct net *net,
const struct in6_addr *daddr, __be16 dport,
int dif, struct udp_table *tbl,
struct sk_buff *skb);
+struct sock *udp6_lib_lookup_skb_noref(struct sk_buff *skb,
+ __be16 sport, __be16 dport);
+
+static inline struct sock *udp_get_ref(struct sock *sk)
+{
+ if (unlikely(!atomic_inc_not_zero_hint(&sk->sk_refcnt, 2)))
+ return NULL;
+
+ return sk;
+}
/*
* SNMP statistics for UDP and UDP-Lite
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 836abe5..324d008 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -494,7 +494,7 @@ static struct sock *udp4_lib_lookup2(struct net *net,
__be32 saddr, __be16 sport,
__be32 daddr, unsigned int hnum, int dif,
struct udp_hslot *hslot2, unsigned int slot2,
- struct sk_buff *skb)
+ struct sk_buff *skb, bool get_ref)
{
struct sock *sk, *result;
struct hlist_nulls_node *node;
@@ -544,12 +544,14 @@ begin:
goto begin;
if (result) {
found:
- if (unlikely(!atomic_inc_not_zero_hint(&result->sk_refcnt, 2)))
- result = NULL;
- else if (unlikely(compute_score2(result, net, saddr, sport,
- daddr, hnum, dif) < badness)) {
- sock_put(result);
- goto begin;
+ if (get_ref) {
+ if (!udp_get_ref(result)) {
+ result = NULL;
+ } else if (unlikely(compute_score2(result, net, saddr,
+ sport, daddr, hnum, dif) < badness)) {
+ sock_put(result);
+ goto begin;
+ }
}
}
return result;
@@ -558,9 +560,11 @@ found:
/* UDP is nearly always wildcards out the wazoo, it makes no sense to try
* harder than this. -DaveM
*/
-struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
+/* called with read_rcu_lock() */
+static struct sock *___udp4_lib_lookup(struct net *net, __be32 saddr,
__be16 sport, __be32 daddr, __be16 dport,
- int dif, struct udp_table *udptable, struct sk_buff *skb)
+ int dif, struct udp_table *udptable, struct sk_buff *skb,
+ bool get_ref)
{
struct sock *sk, *result;
struct hlist_nulls_node *node;
@@ -571,7 +575,6 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
bool select_ok = true;
u32 hash = 0;
- rcu_read_lock();
if (hslot->count > 10) {
hash2 = udp4_portaddr_hash(net, daddr, hnum);
slot2 = hash2 & udptable->mask;
@@ -581,7 +584,7 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
result = udp4_lib_lookup2(net, saddr, sport,
daddr, hnum, dif,
- hslot2, slot2, skb);
+ hslot2, slot2, skb, get_ref);
if (!result) {
hash2 = udp4_portaddr_hash(net, htonl(INADDR_ANY), hnum);
slot2 = hash2 & udptable->mask;
@@ -591,9 +594,8 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
result = udp4_lib_lookup2(net, saddr, sport,
htonl(INADDR_ANY), hnum, dif,
- hslot2, slot2, skb);
+ hslot2, slot2, skb, get_ref);
}
- rcu_read_unlock();
return result;
}
begin:
@@ -639,19 +641,43 @@ begin:
if (result) {
found:
- if (unlikely(!atomic_inc_not_zero_hint(&result->sk_refcnt, 2)))
- result = NULL;
- else if (unlikely(compute_score(result, net, saddr, hnum, sport,
- daddr, dport, dif) < badness)) {
- sock_put(result);
- goto begin;
+ if (get_ref) {
+ if (!udp_get_ref(result)) {
+ result = NULL;
+ } else if (unlikely(compute_score(result, net, saddr,
+ hnum, sport, daddr, dport,
+ dif) < badness)) {
+ sock_put(result);
+ goto begin;
+ }
}
}
- rcu_read_unlock();
return result;
}
+
+struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
+ __be16 sport, __be32 daddr, __be16 dport,
+ int dif, struct udp_table *udptable, struct sk_buff *skb)
+{
+ struct sock *sk;
+
+ rcu_read_lock();
+ sk = ___udp4_lib_lookup(net, saddr, sport, daddr, dport, dif, udptable,
+ skb, true);
+ rcu_read_unlock();
+
+ return sk;
+}
EXPORT_SYMBOL_GPL(__udp4_lib_lookup);
+struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
+ __be32 daddr, __be16 dport, int dif)
+{
+ return __udp4_lib_lookup(net, saddr, sport, daddr, dport, dif,
+ &udp_table, NULL);
+}
+EXPORT_SYMBOL_GPL(udp4_lib_lookup);
+
static inline struct sock *__udp4_lib_lookup_skb(struct sk_buff *skb,
__be16 sport, __be16 dport,
struct udp_table *udptable)
@@ -663,13 +689,24 @@ static inline struct sock *__udp4_lib_lookup_skb(struct sk_buff *skb,
udptable, skb);
}
-struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
- __be32 daddr, __be16 dport, int dif)
+static inline struct sock *__udp4_lib_lookup_skb_noref(struct sk_buff *skb,
+ __be16 sport, __be16 dport,
+ struct udp_table *udptable)
{
- return __udp4_lib_lookup(net, saddr, sport, daddr, dport, dif,
- &udp_table, NULL);
+ const struct iphdr *iph = ip_hdr(skb);
+ struct net_device *dev = skb_dst(skb) ? skb_dst(skb)->dev : skb->dev;
+
+ return ___udp4_lib_lookup(dev_net(dev), iph->saddr, sport,
+ iph->daddr, dport, inet_iif(skb),
+ udptable, skb, false);
}
-EXPORT_SYMBOL_GPL(udp4_lib_lookup);
+
+struct sock *udp4_lib_lookup_skb_noref(struct sk_buff *skb,
+ __be16 sport, __be16 dport)
+{
+ return __udp4_lib_lookup_skb_noref(skb, sport, dport, &udp_table);
+}
+EXPORT_SYMBOL_GPL(udp4_lib_lookup_skb_noref);
static inline bool __udp_is_mcast_sock(struct net *net, struct sock *sk,
__be16 loc_port, __be32 loc_addr,
@@ -1563,7 +1600,6 @@ static int __udp_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
}
return 0;
-
}
static struct static_key udp_encap_needed __read_mostly;
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index fd25e44..281469c 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -210,7 +210,7 @@ static struct sock *udp6_lib_lookup2(struct net *net,
const struct in6_addr *saddr, __be16 sport,
const struct in6_addr *daddr, unsigned int hnum, int dif,
struct udp_hslot *hslot2, unsigned int slot2,
- struct sk_buff *skb)
+ struct sk_buff *skb, bool get_ref)
{
struct sock *sk, *result;
struct hlist_nulls_node *node;
@@ -261,22 +261,25 @@ begin:
if (result) {
found:
- if (unlikely(!atomic_inc_not_zero_hint(&result->sk_refcnt, 2)))
- result = NULL;
- else if (unlikely(compute_score2(result, net, saddr, sport,
- daddr, hnum, dif) < badness)) {
- sock_put(result);
- goto begin;
+ if (get_ref) {
+ if (!udp_get_ref(result)) {
+ result = NULL;
+ } else if (unlikely(compute_score2(result, net, saddr,
+ sport, daddr, hnum, dif) < badness)) {
+ sock_put(result);
+ goto begin;
+ }
}
}
return result;
}
-struct sock *__udp6_lib_lookup(struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, __be16 dport,
- int dif, struct udp_table *udptable,
- struct sk_buff *skb)
+/* called with read_rcu_lock() */
+struct sock *___udp6_lib_lookup(struct net *net,
+ const struct in6_addr *saddr, __be16 sport,
+ const struct in6_addr *daddr, __be16 dport,
+ int dif, struct udp_table *udptable,
+ struct sk_buff *skb, bool get_ref)
{
struct sock *sk, *result;
struct hlist_nulls_node *node;
@@ -287,7 +290,6 @@ struct sock *__udp6_lib_lookup(struct net *net,
bool select_ok = true;
u32 hash = 0;
- rcu_read_lock();
if (hslot->count > 10) {
hash2 = udp6_portaddr_hash(net, daddr, hnum);
slot2 = hash2 & udptable->mask;
@@ -297,7 +299,7 @@ struct sock *__udp6_lib_lookup(struct net *net,
result = udp6_lib_lookup2(net, saddr, sport,
daddr, hnum, dif,
- hslot2, slot2, skb);
+ hslot2, slot2, skb, get_ref);
if (!result) {
hash2 = udp6_portaddr_hash(net, &in6addr_any, hnum);
slot2 = hash2 & udptable->mask;
@@ -307,9 +309,8 @@ struct sock *__udp6_lib_lookup(struct net *net,
result = udp6_lib_lookup2(net, saddr, sport,
&in6addr_any, hnum, dif,
- hslot2, slot2, skb);
+ hslot2, slot2, skb, get_ref);
}
- rcu_read_unlock();
return result;
}
begin:
@@ -354,17 +355,35 @@ begin:
if (result) {
found:
- if (unlikely(!atomic_inc_not_zero_hint(&result->sk_refcnt, 2)))
- result = NULL;
- else if (unlikely(compute_score(result, net, hnum, saddr, sport,
- daddr, dport, dif) < badness)) {
- sock_put(result);
- goto begin;
+ if (get_ref) {
+ if (!udp_get_ref(result)) {
+ result = NULL;
+ } else if (unlikely(compute_score(result, net, hnum,
+ saddr, sport, daddr, dport,
+ dif) < badness)) {
+ sock_put(result);
+ goto begin;
+ }
}
}
- rcu_read_unlock();
return result;
}
+
+struct sock *__udp6_lib_lookup(struct net *net,
+ const struct in6_addr *saddr, __be16 sport,
+ const struct in6_addr *daddr, __be16 dport,
+ int dif, struct udp_table *udptable,
+ struct sk_buff *skb)
+{
+ struct sock *sk;
+
+ rcu_read_lock();
+ sk = ___udp6_lib_lookup(net, saddr, sport, daddr, dport, dif,
+ udptable, skb, true);
+ rcu_read_unlock();
+
+ return sk;
+}
EXPORT_SYMBOL_GPL(__udp6_lib_lookup);
static struct sock *__udp6_lib_lookup_skb(struct sk_buff *skb,
@@ -389,6 +408,26 @@ struct sock *udp6_lib_lookup(struct net *net, const struct in6_addr *saddr, __be
}
EXPORT_SYMBOL_GPL(udp6_lib_lookup);
+static inline struct sock *__udp6_lib_lookup_skb_noref(struct sk_buff *skb,
+ __be16 sport,
+ __be16 dport,
+ struct udp_table *udptable)
+{
+ const struct ipv6hdr *iph = ipv6_hdr(skb);
+ struct net_device *dev = skb_dst(skb) ? skb_dst(skb)->dev : skb->dev;
+
+ return ___udp6_lib_lookup(dev_net(dev), &iph->saddr, sport,
+ &iph->daddr, dport, inet6_iif(skb),
+ udptable, skb, false);
+}
+
+struct sock *udp6_lib_lookup_skb_noref(struct sk_buff *skb, __be16 sport,
+ __be16 dport)
+{
+ return __udp6_lib_lookup_skb_noref(skb, sport, dport, &udp_table);
+}
+EXPORT_SYMBOL(udp6_lib_lookup_skb_noref);
+
/*
* This should be easy, if there is something there we
* return it, otherwise we block.
--
2.8.0.rc2
^ permalink raw reply related
* [PATCH RFC 9/9] udp: Remove udp_offloads
From: Tom Herbert @ 2016-03-23 22:36 UTC (permalink / raw)
To: davem, netdev; +Cc: kernel-team
In-Reply-To: <1458772618-845742-1-git-send-email-tom@herbertland.com>
Now that the UDP encapsulation GRO functions have been moved to the UDP
socket we not longer need the udp_offload insfrastructure so removing it.
Signed-off-by: Tom Herbert <tom@herbertland.com>
---
include/linux/netdevice.h | 17 -------------
include/net/protocol.h | 3 ---
net/ipv4/udp_offload.c | 63 -----------------------------------------------
3 files changed, 83 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index be693b3..9262ac9 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2152,23 +2152,6 @@ struct packet_offload {
struct list_head list;
};
-struct udp_offload;
-
-struct udp_offload_callbacks {
- struct sk_buff **(*gro_receive)(struct sk_buff **head,
- struct sk_buff *skb,
- struct udp_offload *uoff);
- int (*gro_complete)(struct sk_buff *skb,
- int nhoff,
- struct udp_offload *uoff);
-};
-
-struct udp_offload {
- __be16 port;
- u8 ipproto;
- struct udp_offload_callbacks callbacks;
-};
-
/* often modified stats are per cpu, other are shared (netdev->stats) */
struct pcpu_sw_netstats {
u64 rx_packets;
diff --git a/include/net/protocol.h b/include/net/protocol.h
index da689f5..bf36ca3 100644
--- a/include/net/protocol.h
+++ b/include/net/protocol.h
@@ -107,9 +107,6 @@ int inet_del_offload(const struct net_offload *prot, unsigned char num);
void inet_register_protosw(struct inet_protosw *p);
void inet_unregister_protosw(struct inet_protosw *p);
-int udp_add_offload(struct net *net, struct udp_offload *prot);
-void udp_del_offload(struct udp_offload *prot);
-
#if IS_ENABLED(CONFIG_IPV6)
int inet6_add_protocol(const struct inet6_protocol *prot, unsigned char num);
int inet6_del_protocol(const struct inet6_protocol *prot, unsigned char num);
diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c
index dce43c9..320c91e 100644
--- a/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -14,18 +14,6 @@
#include <net/udp.h>
#include <net/protocol.h>
-static DEFINE_SPINLOCK(udp_offload_lock);
-static struct udp_offload_priv __rcu *udp_offload_base __read_mostly;
-
-#define udp_deref_protected(X) rcu_dereference_protected(X, lockdep_is_held(&udp_offload_lock))
-
-struct udp_offload_priv {
- struct udp_offload *offload;
- possible_net_t net;
- struct rcu_head rcu;
- struct udp_offload_priv __rcu *next;
-};
-
static struct sk_buff *__skb_udp_tunnel_segment(struct sk_buff *skb,
netdev_features_t features,
struct sk_buff *(*gso_inner_segment)(struct sk_buff *skb,
@@ -254,56 +242,6 @@ out:
return segs;
}
-int udp_add_offload(struct net *net, struct udp_offload *uo)
-{
- struct udp_offload_priv *new_offload = kzalloc(sizeof(*new_offload), GFP_ATOMIC);
-
- if (!new_offload)
- return -ENOMEM;
-
- write_pnet(&new_offload->net, net);
- new_offload->offload = uo;
-
- spin_lock(&udp_offload_lock);
- new_offload->next = udp_offload_base;
- rcu_assign_pointer(udp_offload_base, new_offload);
- spin_unlock(&udp_offload_lock);
-
- return 0;
-}
-EXPORT_SYMBOL(udp_add_offload);
-
-static void udp_offload_free_routine(struct rcu_head *head)
-{
- struct udp_offload_priv *ou_priv = container_of(head, struct udp_offload_priv, rcu);
- kfree(ou_priv);
-}
-
-void udp_del_offload(struct udp_offload *uo)
-{
- struct udp_offload_priv __rcu **head = &udp_offload_base;
- struct udp_offload_priv *uo_priv;
-
- spin_lock(&udp_offload_lock);
-
- uo_priv = udp_deref_protected(*head);
- for (; uo_priv != NULL;
- uo_priv = udp_deref_protected(*head)) {
- if (uo_priv->offload == uo) {
- rcu_assign_pointer(*head,
- udp_deref_protected(uo_priv->next));
- goto unlock;
- }
- head = &uo_priv->next;
- }
- pr_warn("udp_del_offload: didn't find offload for port %d\n", ntohs(uo->port));
-unlock:
- spin_unlock(&udp_offload_lock);
- if (uo_priv)
- call_rcu(&uo_priv->rcu, udp_offload_free_routine);
-}
-EXPORT_SYMBOL(udp_del_offload);
-
struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb,
struct udphdr *uh, udp_lookup_t lookup)
{
@@ -327,7 +265,6 @@ struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb,
if (sk && udp_sk(sk)->gro_receive)
goto unflush;
-
goto out_unlock;
unflush:
--
2.8.0.rc2
^ permalink raw reply related
* Re: [PATCH RFC 2/9] udp: Add noreference lookup functions
From: Eric Dumazet @ 2016-03-23 22:59 UTC (permalink / raw)
To: Tom Herbert; +Cc: davem, netdev, kernel-team
In-Reply-To: <1458772618-845742-3-git-send-email-tom@herbertland.com>
On Wed, 2016-03-23 at 15:36 -0700, Tom Herbert wrote:
> This patches adds udp6_lib_lookup_skb_noref, udp4_lib_lookup_skb_noref
> and related support to allow a caller to lookup a UDP socket
> without automatically taking a reference. The lookup and caller
> use of the socket must be done under rcu_read_lock.
>
> This feature will be used in a fast receive encapsulation path and
> also when performing GRO through callout in the UDP socket.
>
> Signed-off-by: Tom Herbert <tom@herbertland.com>
> ---
As already mentioned in the past, you can not do a UDP lookup without
taking a reference, because we use SLAB_DESTROY_BY_RCU rules which are
strict.
Only taking a reference and re-doing the keys check can make sure the
lookup result makes sense (as we check multiple words and there is no
way it can be done atomically)
This is explained in include/linux/slab.h and
Documentation/RCU/rculist_nulls.txt
If you want that to happen, we need first to not use
SLAB_DESTROY_BY_RCU.
Then, we do not need to take a reference at all, even for the regular
UDP stack unicast receive path.
^ permalink raw reply
* Re: [RFC PATCH 7/9] GSO: Support partial segmentation offload
From: Edward Cree @ 2016-03-23 23:00 UTC (permalink / raw)
To: Alexander Duyck
Cc: Or Gerlitz, Alexander Duyck, Netdev, David Miller, Tom Herbert
In-Reply-To: <CAKgT0UfgKwSRjMwRHbDNawAyJ3jvz937j1jxGvJWRaD39bSSeQ@mail.gmail.com>
On 23/03/16 22:36, Alexander Duyck wrote:
> On Wed, Mar 23, 2016 at 2:05 PM, Edward Cree <ecree@solarflare.com> wrote:
>> I disagree. Surely we should be able to "soft segment" the packet just
>> before we give it to the physical device, and then tell it to do dumb copying
>> of both the VXLAN and IPIP headers? At this point, we don't have the problem
>> you identified above, because we've arrived at the device now.
> One issue here is that all levels of IP headers would have to have the
> DF bit set. I don't think that happens right now.
Yes, that's still a requirement. (Well, except for the outermost IP header.)
>> So we can chase through some per-protocol callbacks to shorten all the outer
>> lengths and adjust all the outer checksums, then hand it to the device for
>> TSO. The device is treating the extra headers as an opaque blob, so it
>> doesn't know or care whether it's one layer of encapsulation or forty-two.
> So if we do pure software offloads this is doable. However the GSO
> flags are meant to have hardware feature equivalents. The problem is
> if you combine an IPIP and VXLAN header how do you know what header is
> what and which order things are in, and what is the likelihood of
> having a device that would get things right when dealing with 3 levels
> of IP headers. This is one of the reasons why we don't support
> multiple levels of tunnels in the GSO code. GSO is just meant to be a
> fall-back for hardware offloads.
Right, but if the hardware does things "the new way" it should work fine:
Packet still starts with Eth + IP. Packet still has TCP headers at some
specified offset. So it all works, as long as you don't have to update
any IP IDs except possibly the outermost one.
>> Ok, it sounds like the interface to Intel hardware is just Very Different
>> to Solarflare hardware on this point: we don't tell our hardware anything
>> about where the various headers start, it just parses them to figure it
>> out. (And for new-style TSO we'd tell it where the TCP header starts, as
>> I described before.)
> That is kind of what I figured. So does that mean for IPv6 you guys
> are parsing through extension headers? I believe that is one of the
> reasons why Intel did things the way they did is to avoid having to
> parse through any IPv4 options or IPv6 extension headers.
I believe so, but I'd have to check with our firmware team to be sure.
The hardware needs to have that capability for RX processing, where it
wants to figure out things like the l4proto for IPv6: you have to walk
the extension headers until you get a layer 4 nexthdr. I wonder how
Intel manage without that?
>> I agree this isn't something we can do silently. But we _can_ make it a
>> condition for enabling gso-partial. And I think it's a necessary
>> condition for truly generic TSO. Sure, your 'L3 extension header' works
>> fine for a single tunnel. But if you nest tunnels, you now need to
>> update the outer _and_ middle IP IDs, and you can't do that because you
>> only have one L3 header pointer.
> This is getting away from the 'less is more' concept. If we are doing
> multiple levels of tunnels we have already made things far too
> complicated and it is unlikely hardware will ever support anything
> like that.
That's not how I understood the concept. I parsed it as "if hardware knows
less, we can get more out of it", i.e. by having the hardware blithely paste
together whatever headers you give it, you can support things like nested
tunnels. As long as your 'middle' IP header has DF set, this can be done
without the hardware needing to know a thing about it. And while we don't
need to implement that straight away, we should care to design our
interfaces to ensure we can do that in the future without too much trouble.
>> Of course, that means changing the firmware; luckily we haven't got any
>> parts in the wild doing tunnel offloads yet, so we still have a chance
>> to do that without needing driver code to work around our past
>> mistakes...
>>
>> But this stuff does definitely add value for us, it means we could TSO
>> any tunnel type whatsoever; even nested tunnels as long as only the
>> outermost IP ID needs to change.
> Right. In your case it sounds like you would have the advantage of
> just having to run essentially two counters, one increments the IPv4
> ID and the other decrements the IPv4 checksum. Beyond that the outer
> headers wouldn't need to change at all.
Exactly.
> The only other issue would be determining how the inner pseudo-header
> checksum is updated. If you were parsing out header fields from the
> IP header previously to generate it you would instead need to update
> things so that you could use the partial checksum that is already
> stored in the TCP header checksum field.
Right, but again that's sufficiently under firmware control (AFAIK) that
that should just be a SMOP for the firmware. Though I will ask about
that tomorrow, just in case.
-Ed
^ permalink raw reply
* Re: linux-next: manual merge of the rdma tree with the net-next tree
From: Or Gerlitz @ 2016-03-23 23:04 UTC (permalink / raw)
To: Linus Torvalds
Cc: Doug Ledford, Stephen Rothwell, David Miller, Network Development,
linux-next, Linux Kernel Mailing List
In-Reply-To: <CA+55aFyi31eRO6nO1-kz=-kq4XqyyA=VagbsiaO4ZPu1aLmnuA@mail.gmail.com>
On Wed, Mar 16, 2016 at 7:44 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Wed, Mar 16, 2016 at 10:35 AM, Doug Ledford <dledford@redhat.com> wrote:
>> On 3/16/2016 1:18 PM, Linus Torvalds wrote:
>>> On Tue, Mar 15, 2016 at 5:58 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>>>> I fixed it up (see below) and can carry the fix as necessary (no action
>>>> is required).
>>> Side note: can you change this wording for your manual merge script?
>>> Last merge window (or was it the one before it?) we had confusion with
>>> people who thought that "no action is required" means "you can just
>>> ignore this entirely".
>> I certainly didn't take it that way regardless of the wording.
> It was Or Gerlitz. You were cc'd, since it was the whole rdma Mellanox
> mess. I quote from that thread:
>
> "> However, the fact that it got resolved in linux-next is purely
> > informational. It doesn't "fix" the conflict - it just means that both
> > sides should have gotten informed about it. That doesn't mean that the
> > conflict goes away or becomes better.
>
> That's news to me. When such things happen and caught by Stephen, we
> are getting an email saying something like
>
> "Today's linux-next merge of the infiniband tree got a conflict
> between commit X from net-next tree and commit Y from the infiniband
> tree. I fixed it up (see below) and can carry the fix as necessary (no
> action is required)."
>
> Also asked around a bit and got to learn on Stephen using git rerere,
> so all (no action needed note + seeing git rerere in action...) that
> leaded me to think that indeed no action is required from our side,
> but after reading your email (twice, so far), I realized that this was
> wrong conclusion."
> So that whole "no action is required" wording very much has caused
> confusion before in the rdma camp.
> Let's fix the wording. I'm indeed hopeful that the rdma camp is now
> keenly aware of the issues, but that doesn't change the fact that the
> wording has been problematic.
>> "[...] The Mellanox people are on my xxit-list until they show that they can
>> actually act like responsible people [...]"
Linus,
As I wrote you in that other thread you were quoting from there,
following to the happenings mentioned there, we took responsibility
and made bunch of corrective actions within Mellanox which got us to a
point where there was only one rdma/netdev-next conflict for the 4.6
merge window.
I know there's history here, and in the 4.5 cycle things were much
worse, but I still wanted to put things in their more precise place,
if you don't mind.
Or.
^ permalink raw reply
* Re: [PATCH v8 net-next] ravb: Add dma queue interrupt support
From: Yoshihiro Kaneko @ 2016-03-23 23:07 UTC (permalink / raw)
To: David Miller
Cc: netdev, Sergei Shtylyov, Simon Horman, Magnus Damm,
linux-renesas-soc
In-Reply-To: <20160322.155555.77700411133852493.davem@redhat.com>
2016-03-23 4:55 GMT+09:00 David Miller <davem@redhat.com>:
> From: Yoshihiro Kaneko <ykaneko0929@gmail.com>
> Date: Wed, 23 Mar 2016 00:22:00 +0900
>
>> From: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
>>
>> This patch supports the following interrupts.
>>
>> - One interrupt for multiple (timestamp, error, gPTP)
>> - One interrupt for emac
>> - Four interrupts for dma queue (best effort rx/tx, network control rx/tx)
>>
>> This patch improve efficiency of the interrupt handler by adding the
>> interrupt handler corresponding to each interrupt source described
>> above. Additionally, it reduces the number of times of the access to
>> EthernetAVB IF.
>> Also this patch prevent this driver depends on the whim of a boot loader.
>>
>> [ykaneko0929@gmail.com: define bit names of registers]
>> [ykaneko0929@gmail.com: add comment for gen3 only registers]
>> [ykaneko0929@gmail.com: fix coding style]
>> [ykaneko0929@gmail.com: update changelog]
>> [ykaneko0929@gmail.com: gen3: fix initialization of interrupts]
>> [ykaneko0929@gmail.com: gen3: fix clearing interrupts]
>> [ykaneko0929@gmail.com: gen3: add helper function for request_irq()]
>> [ykaneko0929@gmail.com: gen3: remove IRQF_SHARED flag for request_irq()]
>> [ykaneko0929@gmail.com: revert ravb_close() and ravb_ptp_stop()]
>> [ykaneko0929@gmail.com: avoid calling free_irq() to non-hooked interrupts]
>> [ykaneko0929@gmail.com: make NC/BE interrupt handler a function]
>> [ykaneko0929@gmail.com: make timestamp interrupt handler a function]
>> [ykaneko0929@gmail.com: timestamp interrupt is handled in multiple
>> interrupt handler instead of dma queue interrupt handler]
>> Signed-off-by: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
>> Signed-off-by: Yoshihiro Kaneko <ykaneko0929@gmail.com>
>
> Sorry, it is not appropriate to submit new features and major optimizations
> at this time.
>
> Please wait until some reasonable time after the merge window closes to
> resubmit this.
I'm sorry to bother you.
I will re-post this patch after net-next reopened.
>
> Thanks.
Best regards,
kaneko
^ permalink raw reply
* Re: [RFC PATCH 7/9] GSO: Support partial segmentation offload
From: Alexander Duyck @ 2016-03-23 23:15 UTC (permalink / raw)
To: Edward Cree
Cc: Or Gerlitz, Alexander Duyck, Netdev, David Miller, Tom Herbert
In-Reply-To: <56F3200C.20200@solarflare.com>
On Wed, Mar 23, 2016 at 4:00 PM, Edward Cree <ecree@solarflare.com> wrote:
> On 23/03/16 22:36, Alexander Duyck wrote:
>> On Wed, Mar 23, 2016 at 2:05 PM, Edward Cree <ecree@solarflare.com> wrote:
>>> I disagree. Surely we should be able to "soft segment" the packet just
>>> before we give it to the physical device, and then tell it to do dumb copying
>>> of both the VXLAN and IPIP headers? At this point, we don't have the problem
>>> you identified above, because we've arrived at the device now.
>> One issue here is that all levels of IP headers would have to have the
>> DF bit set. I don't think that happens right now.
> Yes, that's still a requirement. (Well, except for the outermost IP header.)
>>> So we can chase through some per-protocol callbacks to shorten all the outer
>>> lengths and adjust all the outer checksums, then hand it to the device for
>>> TSO. The device is treating the extra headers as an opaque blob, so it
>>> doesn't know or care whether it's one layer of encapsulation or forty-two.
>> So if we do pure software offloads this is doable. However the GSO
>> flags are meant to have hardware feature equivalents. The problem is
>> if you combine an IPIP and VXLAN header how do you know what header is
>> what and which order things are in, and what is the likelihood of
>> having a device that would get things right when dealing with 3 levels
>> of IP headers. This is one of the reasons why we don't support
>> multiple levels of tunnels in the GSO code. GSO is just meant to be a
>> fall-back for hardware offloads.
> Right, but if the hardware does things "the new way" it should work fine:
> Packet still starts with Eth + IP. Packet still has TCP headers at some
> specified offset. So it all works, as long as you don't have to update
> any IP IDs except possibly the outermost one.
Right, but the problem becomes how do you identify what tunnel wants
what. So for example we could theoretically have a UDP tunnel in a
UDP with checksum. How would we tell which one want to have the
checksum set and which one doesn't? The fact is we cannot. You are
looking too far ahead. We haven't gotten to tunnel in tunnel yet.
The approach as it stands doesn't have any issues that necessarily
prevent that as long as the outer is the only IP ID that has to
increment, but we don't support anything like that now so we don't
need to worry about it too much.
>>> Ok, it sounds like the interface to Intel hardware is just Very Different
>>> to Solarflare hardware on this point: we don't tell our hardware anything
>>> about where the various headers start, it just parses them to figure it
>>> out. (And for new-style TSO we'd tell it where the TCP header starts, as
>>> I described before.)
>> That is kind of what I figured. So does that mean for IPv6 you guys
>> are parsing through extension headers? I believe that is one of the
>> reasons why Intel did things the way they did is to avoid having to
>> parse through any IPv4 options or IPv6 extension headers.
> I believe so, but I'd have to check with our firmware team to be sure.
> The hardware needs to have that capability for RX processing, where it
> wants to figure out things like the l4proto for IPv6: you have to walk
> the extension headers until you get a layer 4 nexthdr. I wonder how
> Intel manage without that?
They have some parsing in the Rx. That is one of the reasons why
there was all the arguing about adding GENEVE port numbers a few
months ago. They just don't make use of it in the Tx path with the
exception of the fm10k parts.
>>> I agree this isn't something we can do silently. But we _can_ make it a
>>> condition for enabling gso-partial. And I think it's a necessary
>>> condition for truly generic TSO. Sure, your 'L3 extension header' works
>>> fine for a single tunnel. But if you nest tunnels, you now need to
>>> update the outer _and_ middle IP IDs, and you can't do that because you
>>> only have one L3 header pointer.
>> This is getting away from the 'less is more' concept. If we are doing
>> multiple levels of tunnels we have already made things far too
>> complicated and it is unlikely hardware will ever support anything
>> like that.
> That's not how I understood the concept. I parsed it as "if hardware knows
> less, we can get more out of it", i.e. by having the hardware blithely paste
> together whatever headers you give it, you can support things like nested
> tunnels. As long as your 'middle' IP header has DF set, this can be done
> without the hardware needing to know a thing about it. And while we don't
> need to implement that straight away, we should care to design our
> interfaces to ensure we can do that in the future without too much trouble.
The design as is does nothing to prevent that. One of the reasons why
I prefer to keep the outer IP ID incrementing is in order to support
that kind of concept. Also it shields us a bit as we usually cannot
control the network between the tunnel endpoints since it is usually
traversing a WAN. What we need to do though is go through and see if
we can get away with something like "if inner IP DF is set the outer
IP DF bit must be set" kind of logic for GRE and UDP tunnels. If we
can push that then it will allow us to essentially fix all the tunnel
logic in one shot since TCP requires DF bit be set so all levels of
headers would have the DF bit set.
>>> Of course, that means changing the firmware; luckily we haven't got any
>>> parts in the wild doing tunnel offloads yet, so we still have a chance
>>> to do that without needing driver code to work around our past
>>> mistakes...
>>>
>>> But this stuff does definitely add value for us, it means we could TSO
>>> any tunnel type whatsoever; even nested tunnels as long as only the
>>> outermost IP ID needs to change.
>> Right. In your case it sounds like you would have the advantage of
>> just having to run essentially two counters, one increments the IPv4
>> ID and the other decrements the IPv4 checksum. Beyond that the outer
>> headers wouldn't need to change at all.
> Exactly.
>> The only other issue would be determining how the inner pseudo-header
>> checksum is updated. If you were parsing out header fields from the
>> IP header previously to generate it you would instead need to update
>> things so that you could use the partial checksum that is already
>> stored in the TCP header checksum field.
> Right, but again that's sufficiently under firmware control (AFAIK) that
> that should just be a SMOP for the firmware. Though I will ask about
> that tomorrow, just in case.
There shouldn't be much to it. In the case of the Intel parts they
want the length cancelled out of the checksum by the driver and they
they fold it back in via hardware. I would imagine that your hardware
could probably do something similar or may already be doing it since
the length has to be handled differently for IPv4 vs IPv6.
- Alex
^ permalink raw reply
* Re: [PATCH RFC 2/9] udp: Add noreference lookup functions
From: Tom Herbert @ 2016-03-23 23:17 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S. Miller, Linux Kernel Network Developers, Kernel Team
In-Reply-To: <1458773986.10868.54.camel@edumazet-glaptop3.roam.corp.google.com>
On Wed, Mar 23, 2016 at 3:59 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Wed, 2016-03-23 at 15:36 -0700, Tom Herbert wrote:
>> This patches adds udp6_lib_lookup_skb_noref, udp4_lib_lookup_skb_noref
>> and related support to allow a caller to lookup a UDP socket
>> without automatically taking a reference. The lookup and caller
>> use of the socket must be done under rcu_read_lock.
>>
>> This feature will be used in a fast receive encapsulation path and
>> also when performing GRO through callout in the UDP socket.
>>
>> Signed-off-by: Tom Herbert <tom@herbertland.com>
>> ---
>
> As already mentioned in the past, you can not do a UDP lookup without
> taking a reference, because we use SLAB_DESTROY_BY_RCU rules which are
> strict.
>
> Only taking a reference and re-doing the keys check can make sure the
> lookup result makes sense (as we check multiple words and there is no
> way it can be done atomically)
>
> This is explained in include/linux/slab.h and
> Documentation/RCU/rculist_nulls.txt
>
> If you want that to happen, we need first to not use
> SLAB_DESTROY_BY_RCU.
>
> Then, we do not need to take a reference at all, even for the regular
> UDP stack unicast receive path.
>
Thanks Eric. Do you think it is reasonable to not use
SLAB_DESTROY_BY_RCU for UDP since it probably has a much lower
allocation rate than what we'd see in TCP?
Tom
>
>
^ permalink raw reply
* Re: linux-next: manual merge of the rdma tree with the net-next tree
From: Linus Torvalds @ 2016-03-23 23:23 UTC (permalink / raw)
To: Or Gerlitz
Cc: Doug Ledford, Stephen Rothwell, David Miller, Network Development,
linux-next, Linux Kernel Mailing List
In-Reply-To: <CAJ3xEMgth+KpQdcGi3Y8s9_0GJrEESCiEU_8uFGWrMOLHwOdHA@mail.gmail.com>
On Wed, Mar 23, 2016 at 4:04 PM, Or Gerlitz <gerlitz.or@gmail.com> wrote:
>
> I know there's history here, and in the 4.5 cycle things were much
> worse, but I still wanted to put things in their more precise place,
> if you don't mind.
We'll see how things shape up in the future. Once bitten, twice shy,
as they say.
Please make sure that Mellanox will not be a pain going forward, and
everything will be forgiven/forgotten.
Linus
^ permalink raw reply
* Re: [PATCH RFC 2/9] udp: Add noreference lookup functions
From: Eric Dumazet @ 2016-03-23 23:28 UTC (permalink / raw)
To: Tom Herbert; +Cc: David S. Miller, Linux Kernel Network Developers, Kernel Team
In-Reply-To: <CALx6S35pPLoyY6hiOg1WKKe4FUATgJQncWvoxJe-Z2NoOMe2eA@mail.gmail.com>
On Wed, 2016-03-23 at 16:17 -0700, Tom Herbert wrote:
> Thanks Eric. Do you think it is reasonable to not use
> SLAB_DESTROY_BY_RCU for UDP since it probably has a much lower
> allocation rate than what we'd see in TCP?
This is absolutely reasonable ;)
^ permalink raw reply
* Re: [PATCH] ipv6: Fix the pmtu path for connected UDP socket
From: Wei Wang @ 2016-03-23 23:57 UTC (permalink / raw)
To: Eric Dumazet, Martin KaFai Lau, Cong Wang, Eric Dumazet
Cc: Wei Wang, David Miller, Linux Kernel Network Developers
In-Reply-To: <1458689814.10868.29.camel@edumazet-glaptop3.roam.corp.google.com>
What about something like this:
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index ed44663..21b4102 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -1394,6 +1394,19 @@ static void ip6_rt_update_pmtu(struct dst_entry
*dst, struct sock *sk,
__ip6_rt_update_pmtu(dst, sk, skb ? ipv6_hdr(skb) : NULL, mtu);
}
+static void ip6_fill_in_flow(struct flowi6 *fl6, struct net *net,
+ struct sk_buff *skb, int oif, u32 mark)
+{
+ const struct ipv6hdr *iph = (struct ipv6hdr *) skb->data;
+
+ memset(fl6, 0, sizeof(fl6));
+ fl6->flowi6_oif = oif;
+ fl6->flowi6_mark = mark ? mark : IP6_REPLY_MARK(net, skb->mark);
+ fl6->daddr = iph->daddr;
+ fl6->saddr = iph->saddr;
+ fl6->flowlabel = ip6_flowinfo(iph);
+}
+
void ip6_update_pmtu(struct sk_buff *skb, struct net *net, __be32 mtu,
int oif, u32 mark)
{
@@ -1401,13 +1414,7 @@ void ip6_update_pmtu(struct sk_buff *skb,
struct net *net, __be32 mtu,
struct dst_entry *dst;
struct flowi6 fl6;
- memset(&fl6, 0, sizeof(fl6));
- fl6.flowi6_oif = oif;
- fl6.flowi6_mark = mark ? mark : IP6_REPLY_MARK(net, skb->mark);
- fl6.daddr = iph->daddr;
- fl6.saddr = iph->saddr;
- fl6.flowlabel = ip6_flowinfo(iph);
-
+ ip6_fill_in_flow(&fl6, net, skb, oif, mark);
dst = ip6_route_output(net, NULL, &fl6);
if (!dst->error)
__ip6_rt_update_pmtu(dst, NULL, iph, ntohl(mtu));
@@ -1417,8 +1424,22 @@ EXPORT_SYMBOL_GPL(ip6_update_pmtu);
void ip6_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, __be32 mtu)
{
- ip6_update_pmtu(skb, sock_net(sk), mtu,
+ struct ipv6_pinfo *np = inet6_sk(sk);
+ struct dst_entry *dst_new;
+ struct flowi6 fl6;
+ struct net *net = sock_net(sk);
+
+ ip6_update_pmtu(skb, net, mtu,
+ sk->sk_bound_dev_if, sk->sk_mark);
+
+ if (sk->sk_state == TCP_ESTABLISHED &&
+ !sk_dst_check(sk, np->dst_cookie)) {
+ ip6_fill_in_flow(&fl6, net, skb,
sk->sk_bound_dev_if, sk->sk_mark);
+ dst_new = ip6_route_output(net, NULL, &fl6);
+ if (!IS_ERR(dst_new))
+ ip6_dst_store(sk, dst_new, NULL, NULL);
+ }
}
EXPORT_SYMBOL_GPL(ip6_sk_update_pmtu);
Thanks.
Wei
On Tue, Mar 22, 2016 at 4:36 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Tue, 2016-03-22 at 13:13 -0700, Cong Wang wrote:
>> On Tue, Mar 22, 2016 at 11:03 AM, Wei Wang <tracywwnj@gmail.com> wrote:
>> > Thanks Martin and Cong.
>> >
>> > I guess then we are going with the following fix in ip6_sk_update_pmtu():
>> > 1. call ip6_upate_pmtu() as it is
>> > 2. do a dst_check()
>> > 3. re-lookup() if it is invalid
>> > 4. and then do a ip6_dst_store()/dst_set
>>
>> Exactly, please try the attached patch. Note I did nothing more than a
>> compile test.
>>
>> Does it make sense to you now?
>
>
> Hard to reply on your patch as it was not inlined.
>
> 1) Lot of code duplication, for some reason I do not yet understand.
>
> ip6_sk_update_pmtu() and ip6_update_pmtu() will basically do the same
> thing...
>
> 2)
>
> + if (sk->sk_state == TCP_ESTABLISHED)
> + ip6_dst_store(sk, dst, &iph->daddr, &iph->saddr);
> +out:
>
>
> ip6_dst_store() will do :
>
> np->daddr_cache = daddr; (&iph->daddr)
> np->saddr_cache = saddr; (&iph->saddr)
>
> So when skb is freed, daddr_cache & saddr_cache point to freed data.
>
>
>
>
^ permalink raw reply related
* Re: [PATCH] net: Fix typos and whitespace.
From: Tom Herbert @ 2016-03-24 0:28 UTC (permalink / raw)
To: David Miller; +Cc: bhelgaas, Linux Kernel Network Developers, LKML
In-Reply-To: <20160323.142703.2259993852905081527.davem@davemloft.net>
On Wed, Mar 23, 2016 at 11:27 AM, David Miller <davem@davemloft.net> wrote:
> From: Bjorn Helgaas <bhelgaas@google.com>
> Date: Wed, 23 Mar 2016 08:45:30 -0500
>
>> Fix typos. Capitalize CPU, NAPI, RCU consistently. Align structure
>> indentation. No functional change intended; only comment and whitespace
>> changes.
>>
>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>
> Does not apply to the current 'net' tree, please respin.
Why is this for net and not net-next?
^ permalink raw reply
* Re: [PATCH net v2] xfrm: Fix crash observed during device unregistration and decryption
From: Herbert Xu @ 2016-03-24 0:45 UTC (permalink / raw)
To: Eric Dumazet
Cc: Subash Abhinov Kasiviswanathan, 'Steffen Klassert',
netdev, jeromes
In-Reply-To: <1458754165.10868.41.camel@edumazet-glaptop3.roam.corp.google.com>
On Wed, Mar 23, 2016 at 10:29:25AM -0700, Eric Dumazet wrote:
>
> OK, but before calling netif_rx() are we properly testing dev->flags
> IFF_UP status ?
>
> Otherwise, we still allow packets being queued after flush_backlog() had
> been called.
That's the first thing enqueue_to_backlog tests.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] net: Fix typos and whitespace.
From: David Miller @ 2016-03-24 0:55 UTC (permalink / raw)
To: tom; +Cc: bhelgaas, netdev, linux-kernel
In-Reply-To: <CALx6S34KC2cGwvW0RPE_9CEABhrTOrbhkBYb5Eg3PpQ8rspsAg@mail.gmail.com>
From: Tom Herbert <tom@herbertland.com>
Date: Wed, 23 Mar 2016 17:28:20 -0700
> On Wed, Mar 23, 2016 at 11:27 AM, David Miller <davem@davemloft.net> wrote:
>> From: Bjorn Helgaas <bhelgaas@google.com>
>> Date: Wed, 23 Mar 2016 08:45:30 -0500
>>
>>> Fix typos. Capitalize CPU, NAPI, RCU consistently. Align structure
>>> indentation. No functional change intended; only comment and whitespace
>>> changes.
>>>
>>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>
>> Does not apply to the current 'net' tree, please respin.
>
> Why is this for net and not net-next?
Fixing comment typos is always appropriate, as they enhance
understanding of the code and make it easier to maintain.
^ permalink raw reply
* Re: [RFC PATCH 7/9] GSO: Support partial segmentation offload
From: Jesse Gross @ 2016-03-24 1:37 UTC (permalink / raw)
To: Alexander Duyck
Cc: Tom Herbert, Alexander Duyck, Edward Cree,
Linux Kernel Network Developers, David S. Miller
In-Reply-To: <CAKgT0Uc9FNJ59529V8=0vtZnVij+JrZ2s-uOpFjnvbiqNHdVYQ@mail.gmail.com>
On Wed, Mar 23, 2016 at 11:19 AM, Alexander Duyck
<alexander.duyck@gmail.com> wrote:
> On Wed, Mar 23, 2016 at 10:09 AM, Tom Herbert <tom@herbertland.com> wrote:
>> Can you add some description about strategy for dealing with ip_id?
>
> Yeah. I still need to add more documentation. I just didn't want to
> get into details on it until we have finalized things a bit more. I'm
> still wondering if we should follow the pattern of ipip tunnels in
> terms of setting the DF bit if the inner header has the DF bit set.
> If we end up needing to add code to do that then it makes it so that
> the ip_id value can be fixed for both inner and outer which makes the
> segmentation much simpler since the only header that would ever really
> need to be updated would be the transport header in order to get the
> checksum correct.
I tried to do this years ago but in practice it broke things.
There's enough middleboxes/firewalls/etc. out there that filter ICMP
messages that path MTU discovery isn't necessarily reliable. And while
you might argue that if the box is breaking things then the same would
be true for the original, unencapsulated TCP stream but a lot of times
there are some other hacks built in (like MSS clamping) that make
assumptions that the traffic is TCP. So at the minimum it is generally
good to have an option to force the DF bit off.
That being said, I actually think that it is good to have the DF bit
on by default for encapsulation headers being added. Unintentional
(and perhaps multiple layers of) fragmentation usually results in
unuseably bad performance and so it best to try to correct it,
hopefully automatically in most cases. And, of course, this is the
direction that IPv6 has already gone. If we can assume that this is
the most common case then in practice we can keep the outer headers
constant for the high performance path.
To me, incrementing the inner IP really seems the best choice. The
inner header is most likely someone else's traffic so it best to not
mess with that whereas the outer headers are likely ours and we know
the parameters for them (and can set the DF bit as we believe is
correct). Also, if you are looking forward to the future as far as
stacking multiple layers of tunnels, I think the only consistent thing
to do is have the inner ID increment and all of the tunnel headers
stay fixed - it is hard to justify why the first tunnel header should
increment but not the second one. And finally, as a nice bonus, this
is what the GRO code has been expecting already so you won't cause any
performance regressions with existing systems.
^ permalink raw reply
* Re: [PATCH net v2] xfrm: Fix crash observed during device unregistration and decryption
From: Eric Dumazet @ 2016-03-24 1:39 UTC (permalink / raw)
To: Herbert Xu
Cc: Subash Abhinov Kasiviswanathan, 'Steffen Klassert',
netdev, jeromes
In-Reply-To: <20160324004514.GA3110@gondor.apana.org.au>
On Thu, 2016-03-24 at 08:45 +0800, Herbert Xu wrote:
> On Wed, Mar 23, 2016 at 10:29:25AM -0700, Eric Dumazet wrote:
> >
> > OK, but before calling netif_rx() are we properly testing dev->flags
> > IFF_UP status ?
> >
> > Otherwise, we still allow packets being queued after flush_backlog() had
> > been called.
>
> That's the first thing enqueue_to_backlog tests.
>
> Cheers,
Seems to be very recent stuff ( commit
e9e4dd3267d0c5234c5c0f47440456b10875dec9 in linux-4.2)
In the old days the test was done in callers, since in most cases NIC
drivers do not need it.
Lets make sure this was backported to all stable trees.
And then we probably can cleanup some callers as well.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox