From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C533DC43457 for ; Tue, 13 Oct 2020 06:55:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 174EB20797 for ; Tue, 13 Oct 2020 06:55:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DVCqZc0K" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 174EB20797 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 06D1D940007; Tue, 13 Oct 2020 02:55:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 01F9B900002; Tue, 13 Oct 2020 02:55:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E28B8940007; Tue, 13 Oct 2020 02:55:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id B65E5900002 for ; Tue, 13 Oct 2020 02:55:35 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3DD04181AC217 for ; Tue, 13 Oct 2020 06:55:35 +0000 (UTC) X-FDA: 77365991430.20.paste34_2202bb427201 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 1A41F180C07AB for ; Tue, 13 Oct 2020 06:55:35 +0000 (UTC) X-HE-Tag: paste34_2202bb427201 X-Filterd-Recvd-Size: 7010 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Tue, 13 Oct 2020 06:55:34 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id s9so10488684wro.8 for ; Mon, 12 Oct 2020 23:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=tsRtK+RphjJoix8LNFIUyEqIBIL/eXaIC2bZ4t/CBOQ=; b=DVCqZc0KrivZAyxTL0AvxUJIK8ar/eLOwEdWUo/cKCbAoTHgn33/8+mW9OHucS84v6 0imUvQqQAst6kKaHRL53ucde/VGeMSByqHyllJ/uDQpUysG6+31rPcgj26UxVsnLdPw8 bfyqYHaZd6F7DTxDh4M9MqsWd1Fqr7QO6Eib0jiaFCXEG3HkYbI14lKYhDar8x0Cdl8s pB7FD0Rhk2JUG7jdZ45/x7EknGoQjDpzV2vQ7D5UdDPRn5/rJbsCpCJQU5neobKMaKUQ U6QTs0HJiG92p59SleNX/m+xsMzOj0Pb+03mhokRCjuiTEZ/11U7eLxoWvknFy0pXz3f EJ2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=tsRtK+RphjJoix8LNFIUyEqIBIL/eXaIC2bZ4t/CBOQ=; b=Ugo/zfXSVFm4OoGmq6dN1CQAM3ZxiEGFf0+pfbq6MgZKbJUl883wiDfmD9iqR6gQWh RlWERDetBiImXK+Xu0GGcZkZ2OGQEBQWtq0cOrfprKNMiq6B5+XsgE+UadoP8s9bau38 /Q7s8ni4jTzKxkhTXRyaibngGg9E5M+1AhV/uuH5nGPaNZqQIFy1g9AlzbwkX4WzVb6l BCdwPo1cdzpV9yubdIdiFqFXB5Lulta4An80HAq24JvGX5P9bF2x5ROqNncR/yFafIJy cwHUvc76syiElZ3tYgf6SJvjr2CS5xMTIwZHZKbP9Wn1KLKowNHRBT/ZMrh9gfk9svTo aN/A== X-Gm-Message-State: AOAM530CPhMyG6EHpNuioBYoC0ly0Is572Yc0h/NZ8JGB3xpZMpcynHt oewVY6OJmV4uxYnSa/nb4tvY3x52zH4= X-Google-Smtp-Source: ABdhPJyQXupiGXk53k8bnCs10W8RAjL7Zi7PA+tSuK0vnrjR6U4YpJdR1j2LtYICyCerFkByTf/QBw== X-Received: by 2002:adf:e412:: with SMTP id g18mr11015720wrm.211.1602572132746; Mon, 12 Oct 2020 23:55:32 -0700 (PDT) Received: from [192.168.8.147] ([37.171.149.236]) by smtp.gmail.com with ESMTPSA id f14sm26683540wme.22.2020.10.12.23.55.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Oct 2020 23:55:32 -0700 (PDT) Subject: Re: [External] Re: [PATCH] mm: proc: add Sock to /proc/meminfo To: Muchun Song , Eric Dumazet Cc: Eric Dumazet , Cong Wang , Greg KH , rafael@kernel.org, "Michael S. Tsirkin" , Jason Wang , David Miller , Jakub Kicinski , Alexey Dobriyan , Andrew Morton , Alexey Kuznetsov , Hideaki YOSHIFUJI , Steffen Klassert , Herbert Xu , Shakeel Butt , Will Deacon , Michal Hocko , Roman Gushchin , Neil Brown , rppt@kernel.org, Sami Tolvanen , "Kirill A. Shutemov" , Feng Tang , Paolo Abeni , Willem de Bruijn , Randy Dunlap , Florian Westphal , gustavoars@kernel.org, Pablo Neira Ayuso , Dexuan Cui , Jakub Sitnicki , Peter Zijlstra , Christian Brauner , "Eric W. Biederman" , Thomas Gleixner , dave@stgolabs.net, Michel Lespinasse , Jann Horn , chenqiwu@xiaomi.com, christophe.leroy@c-s.fr, Minchan Kim , Martin KaFai Lau , Alexei Starovoitov , Daniel Borkmann , Miaohe Lin , Kees Cook , LKML , virtualization@lists.linux-foundation.org, Linux Kernel Network Developers , linux-fsdevel , linux-mm References: <20201010103854.66746-1-songmuchun@bytedance.com> <9262ea44-fc3a-0b30-54dd-526e16df85d1@gmail.com> From: Eric Dumazet Message-ID: <93d5ad90-2bd5-07ad-618e-456ed2e6da87@gmail.com> Date: Tue, 13 Oct 2020 08:55:29 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/12/20 11:53 AM, Muchun Song wrote: > We are not complaining about TCP using too much memory, but how do > we know that TCP uses a lot of memory. When I firstly face this problem, > I do not know who uses the 25GB memory and it is not shown in the /proc/meminfo. > If we can know the amount memory of the socket buffer via /proc/meminfo, we > may not need to spend a lot of time troubleshooting this problem. Not everyone > knows that a lot of memory may be used here. But I believe many people > should know /proc/meminfo to confirm memory users. Adding yet another operations in networking fast path is a high cost to pay just to add one extra line in /proc/meminfo, while /proc/net/sockstat is already a good proxy, with per protocol details, instead of a single bucket. I reiterate that zero copy would make this counter out of sync, unless special support is added (adding yet another operations ?) Also your patch does not address gazillions of page allocations from drivers in RX path. Here at Google the majority of networking mm usage when hosts are under stress is in RX path, when out of order queues start to grow in TCP sockets. Allocations in TX path were greatly reduced and optimally sized with the introduction of /proc/sys/net/ipv4/tcp_notsent_lowat. We have gazillions of put_page()/__free_page()/__free_pages()/alloc_page()/... all over the places, adding yet another tracking of "this page is used by networking stacks" is going to be quite a big project. I thought memcg was a better goal in the long run, lets focus on it.