From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A65CCA481 for ; Fri, 3 Jun 2022 18:11:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89E3A6B0072; Fri, 3 Jun 2022 14:11:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84BC16B0073; Fri, 3 Jun 2022 14:11:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E6D58D0001; Fri, 3 Jun 2022 14:11:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5D3FA6B0072 for ; Fri, 3 Jun 2022 14:11:52 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 31DE5617CF for ; Fri, 3 Jun 2022 18:11:52 +0000 (UTC) X-FDA: 79537718064.18.88CA094 Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.187]) by imf07.hostedemail.com (Postfix) with ESMTP id 63D0D40002 for ; Fri, 3 Jun 2022 18:11:38 +0000 (UTC) Received: from mail-yw1-f178.google.com ([209.85.128.178]) by mrelayeu.kundenserver.de (mreue010 [213.165.67.97]) with ESMTPSA (Nemesis) id 1Mvs2R-1ngTfW3hHL-00sy6a for ; Fri, 03 Jun 2022 20:11:50 +0200 Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-30c143c41e5so90386437b3.3 for ; Fri, 03 Jun 2022 11:11:49 -0700 (PDT) X-Gm-Message-State: AOAM533admj+fuYu9e/NlddVPOu3B0BuHwxCI8VzcLegJx/eo2FzLHfn 5XhiohPzBOqoYkZqWtBet33ecAdSUMII8dvxHWQ= X-Google-Smtp-Source: ABdhPJyPx59bJ78LaaObb3dJKSJ0uJ/vq5ZzSQOt+iF0NVQ5NcXvxuv91TRpb5URyUqSZSk6MNG98rWb8KJos1p12es= X-Received: by 2002:a0d:fc83:0:b0:2e5:b0f4:c125 with SMTP id m125-20020a0dfc83000000b002e5b0f4c125mr13094816ywf.347.1654279908444; Fri, 03 Jun 2022 11:11:48 -0700 (PDT) MIME-Version: 1.0 References: <70b4e1e46d9d63275a0dfe90f96f40ea14d89f0c.camel@infinera.com> <88dfec5a1c98f4eb71e23cafe89db4395ea12811.camel@infinera.com> In-Reply-To: From: Arnd Bergmann Date: Fri, 3 Jun 2022 20:11:31 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Finding kernel RAM consumers ? To: Matthew Wilcox Cc: Joakim Tjernlund , "linux-mm@kvack.org" , Linux ARM Content-Type: text/plain; charset="UTF-8" X-Provags-ID: V03:K1:z4xMjKakmg4uiVL2hxpYo8Idi9rtTccs8PH3nSVxsxxO0AwROjI kpTQ0fUbEhWsbKztc9kQhEg14c76BwgCiAmlapVOEydqoirLGiK98JHP4F6E16EovNBxpDQ XTHPQnFAjbbEoWGn1MJ6SUSzD4/T+EW8dRdb+a9YJn47DEI+q5yiUfVMovXL0fesEFfq7ZR kZkuYTA5mHkgaCM8tZFag== X-UI-Out-Filterresults: notjunk:1;V03:K0:nkDWgxClfoc=:ZEuRTGsjbtrv3AJaFgpFfX FXqwQd10lIJ9ssmyv37RBw1pg/xp1ABFU+8eiXr76hZeGDlL0YkrjGmVljU4yczTMZQ/mF8sD 7dsJYdsoIFmc36cAQ2ukfHXYzQSyxzda73NNxuKKT5YipVbIPHWn05H/d4NQYsnN2ndkOo5Ku YxQYJL8QeUFEJI0fPfy+oTTFogjXhvHuR8DamYCNPH4vQ/pGtaJ98e4WEeZy4zVYEp8JeYEKp RGi0WrJduO5YTX8fZWtYrEsfcx5Dmo0sMY/2c4UmYwxWLCf8848zZlndDsyztQ0f1AYMIYbw9 Au7TMhvMpSbEt8FFh13+SMqVJtpfvpUmpAzzVj/T54iOM4kzUqM3a54ac0z2xKcFWkaC5ff/1 OUmHhhmlZ+ORM+hGibISXNgbowab2RWRZ4gBU+XOeCyXy50MdnUXguaWkh/PnBMAy7BHEg6gB J71qKX2HPBsk7XlDvmZSUbHdqF70pgWvhkbt7TkHR+ZbTH3ZXI8mtKaI1nEUimqumwBtUoNzf Y+swMadbcAh8+VlLazIRUksenkTSTe/tic0CrHqf6aASiUFNjWHvRJdpvZ6rVfhRzlRBpczMu ECSijNijqcHM/CNnSwRE1t5QWOFFjO7v+GkfJWgRE3fOdWrJvWhDyF0x+2iY4V9g60dtWfR/+ Y8QpTdpE6a+6PgHVe/zAqXNDjvMGcJYaS0e4uXuEb8pvPO34wxb0RHWKe+my/j0L1f/6zCObZ 3MUMtCWxWwwesNAZLikso9Uct2h1LnCSyGaEDA== X-Stat-Signature: e7k5w6ad9aiygquj4c9y1a967zfs8hwy Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=none; spf=none (imf07.hostedemail.com: domain of arnd@arndb.de has no SPF policy when checking 212.227.126.187) smtp.mailfrom=arnd@arndb.de X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 63D0D40002 X-HE-Tag: 1654279898-969513 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 3, 2022 at 7:29 PM Matthew Wilcox wrote: > On Fri, Jun 03, 2022 at 05:26:33PM +0000, Joakim Tjernlund wrote: > > On Fri, 2022-06-03 at 08:49 +0200, Joakim Tjernlund wrote: > > > On Thu, 2022-06-02 at 21:11 +0100, Matthew Wilcox wrote: > > > > On Thu, Jun 02, 2022 at 07:24:23PM +0000, Joakim Tjernlund wrote: > > > > > We have this small embedded target(aarch64) with 32 MB of RAM where the kernel consumes 14420K: > > > > > Memory: 22444K/36864K available (3584K kernel code, 698K rwdata, 936K rodata, 320K init, 255K bss, 14420K reserved, 0K cma-reserved) > > > > > > > > > > I want to track down were most of this RAM is consumed so I can trim away some MBs > > > > > but I am having a hard time finding may way. > > > > > Is there some tool/kernel config that can help me with that? > > > > > > Those are interesting, thanks. > > > In my case it is the amount of work space RAM allocated that is a bit much. The kernel code/data > > > is 5+MB but the total need is 14MB, 9MB is buffer and similar. I'm fairly sure you are in new territory here. The arm64 kernel is just not really optimized for small memory configurations. Even for 32-bit Arm, 32MB is considered too small for most workloads these days, though we do have some very limited support for machines with as little as 2MB of SRAM. The smallest arm64 machines that I can think of being supported in the kernel today have at least 256MB. > > Found something, arm64 only supports mem model SPARSEMEM_EXTREME and it uses most of my memory. > > Tried to force SPARSEMEM_STATIC but that didn't boot, is STATIC even cheaper ? > > That sounds like a question for the arm64 maintainers >From what I understand, the 'static' variant uses a static allocation in .bss, so that may end up consuming most of your RAM even if it works normally. If all memory is contiguous, then using flatmem may be more appropriate. You may also be able to save some memory for sparsemem by reducing the size of the physical address space. CONFIG_ARM64_PA_BITS is normally '48', but reducing it to '40' may help (provided you can get that to boot). Most of the smaller CPUs only support a 40 bit physical address space, so having another option for this is probably sensible. I think this is a case of "patches welcome". Nobody has really needed this so far, but as even the smaller machines are slowly migrating from 32-bit to 64-bit cores, optimizing this will get interesting for more developers. There are probably other low-hanging fruit that you can address after figuring out. One observation I made is that modern kernels don't seem to deal as well as older ones with low-memory situations, so even if you manage to free up most of your 32MB, it might still not work reliably. Arnd