From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F18FBC3A59B for ; Fri, 30 Aug 2019 19:56:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C236E23431 for ; Fri, 30 Aug 2019 19:56:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a36Vr/CJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728079AbfH3T43 (ORCPT ); Fri, 30 Aug 2019 15:56:29 -0400 Received: from mail-ot1-f68.google.com ([209.85.210.68]:47024 "EHLO mail-ot1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728053AbfH3T42 (ORCPT ); Fri, 30 Aug 2019 15:56:28 -0400 Received: by mail-ot1-f68.google.com with SMTP id z17so8013562otk.13 for ; Fri, 30 Aug 2019 12:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=YpoG8jIkxXjVtNoO0jl5YfFLEKuc9+sp9Ac02XuP3Ps=; b=a36Vr/CJaChQDoNbnl6ccll5pu/0k7QFH0f27qCbfPfZKiYEU1v8BdPbx+W+mBlt4w auABZ8XMci4HZjR1mWftvuHRKe0P2tXYHEfI8ZZ3eSrif+VE784nGtt+h7+LpljndpiE zLo1zTMlGGo4riIl+U5+6epBdb+6BcoF69TngP8UyzFzlxteFODCXMpHuBhsaXtP4q9X knQ/r4pQGIK9N8HTcpMCkfn6KfhbBh2M0l1eJAXx5oNED6eEPatQdxgPTfa4h5cL7a7r rqLhoaRUxOjau6FtaNFzaTRI0IMswPWkNDw4W7MSVZcIdhfLYhBIc98mVORujYoRzShN t9UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=YpoG8jIkxXjVtNoO0jl5YfFLEKuc9+sp9Ac02XuP3Ps=; b=KxbmszElqp1ugmtSeTLiy7IT5K4DBVs6sClb/ulpe+7kJy1mFFt837YaUSYDlJn12i LU3LVEVY2BDmeKVYRaWpR2hXWCcZgX+BUn+CPAxxzk4vf7dzwSIT6j7VFqfvMNFUN8Wj 4GsF47j+v2JmXHx28FvYxYwElYNW0SvV/ayg5d2YgPHbi2olk3JgoCqvBVbsBsV7XNCy lHqwNhnGewHNsQaR1zi8A5dvznBMYJzrF/tjqnB2qmelW7un2nzNsyNye3BCIi1TRzkm vZqiZmrHQHys9sX8TD9Hayv0hN9kHUuQTtAX5ONnikVd0NtiWalP+bLnd0Gq9MgyJcui Xr+g== X-Gm-Message-State: APjAAAWZSv5s4EGLCszsRmJMKNqUvLRg6apBvlNHhJhJaH7+grCwGM+O JNJNySNzSzFfGXAPzzdsLhF3Hvhb X-Google-Smtp-Source: APXvYqyfV1+PxblVZieIWJjd6aoVdgJdsPbLV75/WG16XnGaC9nkwY3ZHYgqRQrjGLtsfX86KArppA== X-Received: by 2002:a05:6830:1151:: with SMTP id x17mr8322726otq.270.1567194987743; Fri, 30 Aug 2019 12:56:27 -0700 (PDT) Received: from [192.168.1.249] (cpe-70-114-247-242.austin.res.rr.com. [70.114.247.242]) by smtp.googlemail.com with ESMTPSA id n29sm1811067oij.50.2019.08.30.12.56.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 30 Aug 2019 12:56:18 -0700 (PDT) Subject: Re: [RFCv2 2/4] nl80211: Support >4096 byte NEW_WIPHY event nlmsg To: Johannes Berg , linux-wireless@vger.kernel.org References: <20190816192703.12445-1-denkenz@gmail.com> <20190816192703.12445-2-denkenz@gmail.com> <34a0b0652403edcb630f65a240a30dfddb82950c.camel@sipsolutions.net> From: Denis Kenzior Message-ID: Date: Fri, 30 Aug 2019 14:56:17 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <34a0b0652403edcb630f65a240a30dfddb82950c.camel@sipsolutions.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Hi Johannes, On 8/30/19 4:36 AM, Johannes Berg wrote: > On Fri, 2019-08-16 at 14:27 -0500, Denis Kenzior wrote: >> For historical reasons, NEW_WIPHY messages generated by dumps or >> GET_WIPHY commands were limited to 4096 bytes due to userspace tools >> using limited buffers. > > I think now that I've figured out why, it'd be good to note that it > wasn't due to userspace tools, but rather due to the default netlink > dump skb allocation at the time, prior to commit 9063e21fb026 > ("netlink: autosize skb lengthes"). > Sure, will take care of it. >> Once the sizes NEW_WIPHY messages exceeded these >> sizes, split dumps were introduced. All any non-legacy data was added >> only to messages using split-dumps (including filtered dumps). >> >> However, split-dumping has quite a significant overhead. On cards >> tested, split dumps generated message sizes 1.7-1.8x compared to >> non-split dumps, while still comfortably fitting into an 8k buffer. The >> kernel now expects userspace to provide 16k buffers by default, and 32k >> buffers are possible. >> >> Introduce a concept of a large message, so that if the kernel detects >> that userspace has provided a buffer of sufficient size, a non-split >> message could be generated. > > So, there's still a wrinkle with this. Larger SKB allocation can fail, > and instead of returning an error to userspace, the kernel will allocate > a smaller SKB instead. > > With genetlink, we currently don't even have a way of controlling the > minimum allocation that's always required. > > Since we already have basically all of the mechanics, I'd say perhaps a > better concept would be to "split when necessary", aborting if split > isn't supported. > > IOW, do something like > > ... nl80211_send_wiphy(...) > { > [...] > > switch (state->split_start) { > [...] > case : > [...] // put stuff > state->split_start++; > state->skb_end = nlmsg_get_pos(skb); > /* fall through */ > case : > [...] > } > > finish: > genlmsg_end(msg, hdr); > return 0; > nla_put_failure: > if (state->split_start < 9) { > genlmsg_cancel(msg, hdr); > return -EMSGSIZE; > } > nlmsg_trim(msg, state->skb_end); > goto finish; > } > > > That way, we fill each SKB as much as possible, up to 32k if userspace > provided big enough buffers *and* we could allocate the SKB. > > > Your userspace would still set the split flag, and thus be compatible > with all kinds of options: > * really old kernel not supporting split > * older kernel sending many messages > * kernel after this change packing more into one message > * even if allocating big SKBs failed > What I was thinking was to attempt to build a large message first and if that fails to fail over to the old logic. But I think what you propose is even better. I'll incorporate this feedback into the next version. Regards, -Denis