From: 배석진 <soukjin.bae@samsung.com>
To: "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: [Kernel][NET] Bug report on packet defragmenting
Date: Thu, 08 Nov 2018 10:29:27 +0900 [thread overview]
Message-ID: <20181108012927epcms1p47f719c1908da64a378690362901644ee@epcms1p4> (raw)
In-Reply-To: CGME20181108012927epcms1p47f719c1908da64a378690362901644ee@epcms1p4
Hello,
This is bae working on Samsung Elec.
We got the problem that fragmented SIP packet couldn't be deliverd to user layer.
And found that they were stoled at HOOK function, ipv6_defrag.
In condition with SMP and RPS.
After first fragmented packet, they have no further network header except ip.
But __skb_flow_dissect function using the port field to determine hash key, 'ports'.
So each packet get different hash key, and be sent to different core.
Although hash is different, selected cpu could be same. but it just lucky. [exam 2]
And addition, when those packets arrived with little time gap.
They became ran the ipv6_defrag hook simultaneously in each core.
So they each be treated to first fragmented packet.
And they can't merged to original packet, and can't be deliverd to upper. [exam 1]
If ipv6_defrag hook is not excuted simultaneously, then it's ok.
ipv6_defrag hook can handle that. [exam 3]
We'll skip 'ports' setting when the packet was fragmented.
Because of IPv6 SIP invite packet is usally fragmented, this problem is very often.
>From be74b56861cf76a16d0f2d054d468c584ed67cce Mon Sep 17 00:00:00 2001
From: soukjin bae <soukjin.bae@samsung.com>
Date: Thu, 8 Nov 2018 09:52:29 +0900
Subject: [PATCH] flow_dissector: don't refer port field in fragmented packet
After first fragmented packet, they have no further network header except ip.
So when try to refer port field in nexthdr, they got the garbage from payload.
Skip port set when the packet was fragmented.
Signed-off-by: soukjin bae <soukjin.bae@samsung.com>
---
net/core/flow_dissector.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 676f3ad629f9..928df25129ba 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -1166,8 +1166,8 @@ bool __skb_flow_dissect(const struct sk_buff *skb,
break;
}
- if (dissector_uses_key(flow_dissector,
- FLOW_DISSECTOR_KEY_PORTS)) {
+ if (dissector_uses_key(flow_dissector, FLOW_DISSECTOR_KEY_PORTS)
+ && !(key_control->flags & FLOW_DIS_IS_FRAGMENT)) {
key_ports = skb_flow_dissector_target(flow_dissector,
FLOW_DISSECTOR_KEY_PORTS,
target_container);
--
2.13.0
[exam 1]
# tcp dump #
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:11.938345 2001:4430:5:401::53 2001:4430:140:a635::3cd:9f3 IPv6 396 IPv6 fragment
0010 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 05 88 50 2a 54 16 -> ipv6 hdr with fragment hdr
0040 5d 6a a0 da 01 14 26 5a 85 ba 51 9f 17 75 04 9c -> fragmented payload area. ipv6_defrag using this area as port
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:11.937654 2001:4430:5:401::53 7538 2001:4430:140:a635::3cd:9f3 6300 SIP/SDP 1480 Request: INVITE
0010 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 00 01 50 2a 54 16 -> ipv6 hdr with fragment hdr
0040 00 00 c4 c3 00 00 00 06 da 4d d7 26 a1 d7 64 c6 -> UDP hdr, right value for port. but only just this packet can be refer.
# kernel log #
11-07 13:40:12.296 I[3: swapper/3: 0] LNK-RX(1464): 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01 ... --> our NIC log when pkt rcv
11-07 13:40:12.297 I[3: swapper/3: 0] __skb_flow_dissect: ports: c3c40000 --> right value for port
11-07 13:40:12.297 I[3: swapper/3: 0] get_rps_cpu: cpu = 2 (hash:2758499534)
11-07 13:40:12.297 I[3: swapper/3: 0] LNK-RX(380): 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01 ...
11-07 13:40:12.297 I[3: swapper/3: 0] __skb_flow_dissect: ports: daa06a5d --> but at here...
11-07 13:40:12.298 I[3: swapper/3: 0] get_rps_cpu: cpu = 1 (hash:791526712) --> so this pkt has different hash, cpu
11-07 13:40:12.298 I[2: swapper/2: 0] ipv6_defrag+++
11-07 13:40:12.298 I[1: swapper/1: 0] ipv6_defrag+++ --> go into ipv6_defrag at same time
11-07 13:40:12.298 I[1: swapper/1: 0] ipv6_defrag: EINPROGRESS
11-07 13:40:12.298 I[2: swapper/2: 0] ipv6_defrag: EINPROGRESS --> both packet was treated to first frag
[exam 2]
# tcp dump #
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:13.947576 2001:4430:5:401::53 2001:4430:140:a635::3cd:9f3 IPv6 1480 IPv6 fragment
0010 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 00 01 50 2a 54 1c
0040 00 00 c4 c3 00 00 00 07 2e bb 86 7f 97 4f 58 7c
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:13.948379 2001:4430:5:401::53 7538 2001:4430:140:a635::3cd:9f3 6300 SIP/SDP 396 Request: INVITE
0010 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 05 88 50 2a 54 1c
0040 24 fd 06 d4 b9 23 5b c4 49 9e 1f 3e be f5 12 67
# kernel log #
11-07 13:40:14.306 I[3: swapper/3: 0] LNK-RX(1464): 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01 ...
11-07 13:40:14.306 I[3: swapper/3: 0] __skb_flow_dissect: ports: c3c40000
11-07 13:40:14.307 I[3: swapper/3: 0] get_rps_cpu: cpu = 2 (hash:2758499534)
11-07 13:40:14.307 I[3: swapper/3: 0] LNK-RX(380): 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01 ...
11-07 13:40:14.307 I[3: swapper/3: 0] __skb_flow_dissect: ports: d406fd24
11-07 13:40:14.308 I[3: swapper/3: 0] get_rps_cpu: cpu = 2 (hash:2624600197) --> different hash, but same cpu by unhash.
11-07 13:40:14.308 I[2: swapper/2: 0] ipv6_defrag+++
11-07 13:40:14.308 I[2: swapper/2: 0] ipv6_defrag: EINPROGRESS
11-07 13:40:14.308 I[2: swapper/2: 0] ipv6_defrag+++
11-07 13:40:14.309 I[2: swapper/2: 0] UDP: __udp_enqueue_schedule_skb: qlen=1 --> deliverd to upper
[exam 3]
# tcp dump #
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:05.514191 2001:4430:5:401::53 2001:4430:140:a635::3cd:9f3 IPv6 1480 IPv6 fragment
0010 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 00 01 50 2a 54 03
0040 00 00 c4 c3 00 00 00 03 27 e0 dd cc fe 77 a0 64
<TIME> <Source> <S.Port> <Destination> <D.Port> <Proto> <Len> <Info>
13:40:05.517187 2001:4430:5:401::53 7538 2001:4430:140:a635::3cd:9f3 6300 SIP/SDP 396 Request: INVITE
0010 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01
0020 00 00 00 00 00 00 00 53 20 01 44 30 01 40 a6 35
0030 00 00 00 00 03 cd 09 f3 32 00 05 88 50 2a 54 03
0040 b8 d2 31 bd b3 d7 89 d2 d3 07 99 36 28 3c 37 a4
# kernel log #
11-07 13:40:05.872 I[3: swapper/3: 0] LNK-RX(1464): 6b 80 00 00 05 90 2c 3d 20 01 44 30 00 05 04 01 ...
11-07 13:40:05.874 I[3: swapper/3: 0] __skb_flow_dissect: ports: c3c40000
11-07 13:40:05.875 I[3: swapper/3: 0] get_rps_cpu: cpu = 2 (hash:2758499534)
11-07 13:40:05.876 I[3: swapper/3: 0] LNK-RX(380): 6b 80 00 00 01 54 2c 3d 20 01 44 30 00 05 04 01 ...
11-07 13:40:05.878 I[3: swapper/3: 0] __skb_flow_dissect: ports: bd31d2b8
11-07 13:40:05.878 I[3: swapper/3: 0] get_rps_cpu: cpu = 3 (hash:3167003083) --> different cpu
11-07 13:40:05.879 I[2: swapper/2: 0] ipv6_defrag+++
11-07 13:40:05.879 I[2: swapper/2: 0] ipv6_defrag: EINPROGRESS
11-07 13:40:05.881 I[3: ksoftirqd/3: 33] ipv6_defrag+++ --> but successfully handled by ipv6_defrag
11-07 13:40:05.883 I[3: ksoftirqd/3: 33] UDP: __udp_enqueue_schedule_skb: qlen=1 --> deliverd to upper
next parent reply other threads:[~2018-11-08 11:02 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20181108012927epcms1p47f719c1908da64a378690362901644ee@epcms1p4>
2018-11-08 1:29 ` 배석진 [this message]
2018-11-08 1:43 ` [Kernel][NET] Bug report on packet defragmenting Eric Dumazet
2018-11-08 2:05 ` 배석진
2018-11-08 3:24 ` (2) " Eric Dumazet
2018-11-08 3:56 ` Eric Dumazet
2018-11-08 4:10 ` 배석진
2018-11-08 4:26 ` (2) " Eric Dumazet
2018-11-08 6:13 ` Eric Dumazet
2018-11-08 7:58 ` 배석진
2018-11-08 15:12 ` (2) " Eric Dumazet
2018-11-09 0:42 ` 배석진
2018-11-09 1:58 ` (2) " Eric Dumazet
2018-11-09 2:24 ` 배석진
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181108012927epcms1p47f719c1908da64a378690362901644ee@epcms1p4 \
--to=soukjin.bae@samsung.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox