From: maowenan <maowenan@huawei.com>
To: Eric Dumazet <edumazet@google.com>, <gregkh@linux-foundation.org>
Cc: <mkubecek@suse.cz>, David Woodhouse <dwmw2@infradead.org>,
netdev <netdev@vger.kernel.org>,
Eric Dumazet <eric.dumazet@gmail.com>,
David Miller <davem@davemloft.net>,
Yuchung Cheng <ycheng@google.com>, <jdw@amazon.de>,
<stable@vger.kernel.org>, <tiwai@suse.de>
Subject: Re: [PATCH stable 4.4 0/9] fix SegmentSmack in stable branch (CVE-2018-5390)
Date: Fri, 14 Sep 2018 10:24:40 +0800 [thread overview]
Message-ID: <3f82db5d-c6e8-e7f5-71a6-c5d14d1b9461@huawei.com> (raw)
In-Reply-To: <CANn89iLVjY+KWs3y=cYW_OpwCLAOt9=x=iTnsPatwzCMYxUpTA@mail.gmail.com>
On 2018/9/13 20:44, Eric Dumazet wrote:
> On Thu, Sep 13, 2018 at 5:32 AM Greg KH <gregkh@linux-foundation.org> wrote:
>>
>> On Thu, Aug 16, 2018 at 05:24:09PM +0200, Greg KH wrote:
>>> On Thu, Aug 16, 2018 at 02:33:56PM +0200, Michal Kubecek wrote:
>>>> On Thu, Aug 16, 2018 at 08:05:50PM +0800, maowenan wrote:
>>>>> On 2018/8/16 19:39, Michal Kubecek wrote:
>>>>>>
>>>>>> I suspect you may be doing something wrong with your tests. I checked
>>>>>> the segmentsmack testcase and the CPU utilization on receiving side
>>>>>> (with sending 10 times as many packets as default) went down from ~100%
>>>>>> to ~3% even when comparing what is in stable 4.4 now against older 4.4
>>>>>> kernel.
>>>>>
>>>>> There seems no obvious problem when you send packets with default
>>>>> parameter in Segmentsmack POC, Which is also very related with your
>>>>> server's hardware configuration. Please try with below parameter to
>>>>> form OFO packets
>>>>
>>>> I did and even with these (questionable, see below) changes, I did not
>>>> get more than 10% (of one core) by receiving ksoftirqd.
>>>>
>>>>> for (i = 0; i < 1024; i++) // 128->1024
>>>> ...
>>>>> usleep(10*1000); // Adjust this and packet count to match the target!, sleep 100ms->10ms
>>>>
>>>> The comment in the testcase source suggests to do _one_ of these two
>>>> changes so that you generate 10 times as many packets as the original
>>>> testcase. You did both so that you end up sending 102400 packets per
>>>> second. With 55 byte long packets, this kind of attack requires at least
>>>> 5.5 MB/s (44 Mb/s) of throughput. This is no longer a "low packet rate
>>>> DoS", I'm afraid.
>>>>
>>>> Anyway, even at this rate, I only get ~10% of one core (Intel E5-2697).
>>>>
>>>> What I can see, though, is that with current stable 4.4 code, modified
>>>> testcase which sends something like
>>>>
>>>> 2:3, 3:4, ..., 3001:3002, 3003:3004, 3004:3005, ... 6001:6002, ...
>>>>
>>>> I quickly eat 6 MB of memory for receive queue of one socket while
>>>> earlier 4.4 kernels only take 200-300 KB. I didn't test latest 4.4 with
>>>> Takashi's follow-up yet but I'm pretty sure it will help while
>>>> preserving nice performance when using the original segmentsmack
>>>> testcase (with increased packet ratio).
>>>
>>> Ok, for now I've applied Takashi's fix to the 4.4 stable queue and will
>>> push out a new 4.4-rc later tonight. Can everyone standardize on that
>>> and test and let me know if it does, or does not, fix the reported
>>> issues?
>>>
>>> If not, we can go from there and evaluate this much larger patch series.
>>> But let's try the simple thing first.
>>
>> So, is the issue still present on the latest 4.4 release? Has anyone
>> tested it? If not, I'm more than willing to look at backported patches,
>> but I want to ensure that they really are needed here.
>>
>> thanks,
>
> Honestly, TCP stack without rb-tree for the OOO queue is vulnerable,
> even with non malicious sender,
> but with big enough TCP receive window and a not favorable network.
>
> So a malicious peer can definitely send packets needed to make TCP
> stack behave in O(N), which is pretty bad if N is big...
>
> 9f5afeae51526b3ad7b7cb21ee8b145ce6ea7a7a ("tcp: use an RB tree for ooo
> receive queue")
> was proven to be almost bug free [1], and should be backported if possible.
>
> [1] bug fixed :
> 76f0dcbb5ae1a7c3dbeec13dd98233b8e6b0b32a tcp: fix a stale ooo_last_skb
> after a replace
Thank you for Eric's suggestion, I will do some work to backport them.
>
> .
>
prev parent reply other threads:[~2018-09-14 2:24 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-16 2:50 [PATCH stable 4.4 0/9] fix SegmentSmack in stable branch (CVE-2018-5390) Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 1/9] Revert "tcp: detect malicious patterns in tcp_collapse_ofo_queue()" Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 2/9] Revert "tcp: avoid collapses in tcp_prune_queue() if possible" Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 3/9] tcp: increment sk_drops for dropped rx packets Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 4/9] tcp: use an RB tree for ooo receive queue Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 5/9] tcp: free batches of packets in tcp_prune_ofo_queue() Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 6/9] tcp: avoid collapses in tcp_prune_queue() if possible Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 7/9] tcp: detect malicious patterns in tcp_collapse_ofo_queue() Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 8/9] tcp: call tcp_drop() from tcp_data_queue_ofo() Mao Wenan
2018-08-16 2:50 ` [PATCH stable 4.4 9/9] tcp: add tcp_ooo_try_coalesce() helper Mao Wenan
2018-08-16 6:16 ` [PATCH stable 4.4 0/9] fix SegmentSmack in stable branch (CVE-2018-5390) Michal Kubecek
2018-08-16 6:42 ` maowenan
2018-08-16 6:52 ` Michal Kubecek
2018-08-16 7:19 ` maowenan
2018-08-16 7:23 ` Michal Kubecek
2018-08-16 7:39 ` maowenan
2018-08-16 7:44 ` Michal Kubecek
2018-08-16 7:55 ` maowenan
2018-08-16 11:39 ` Michal Kubecek
2018-08-16 12:05 ` maowenan
2018-08-16 12:33 ` Michal Kubecek
2018-08-16 15:24 ` Greg KH
2018-08-16 16:06 ` Michal Kubecek
2018-08-16 16:20 ` Greg KH
2018-08-17 2:48 ` maowenan
2018-09-13 12:32 ` Greg KH
2018-09-13 12:44 ` Eric Dumazet
2018-09-14 2:24 ` maowenan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3f82db5d-c6e8-e7f5-71a6-c5d14d1b9461@huawei.com \
--to=maowenan@huawei.com \
--cc=davem@davemloft.net \
--cc=dwmw2@infradead.org \
--cc=edumazet@google.com \
--cc=eric.dumazet@gmail.com \
--cc=gregkh@linux-foundation.org \
--cc=jdw@amazon.de \
--cc=mkubecek@suse.cz \
--cc=netdev@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=tiwai@suse.de \
--cc=ycheng@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox