From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21E3CC47253 for ; Fri, 1 May 2020 19:05:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA03621835 for ; Fri, 1 May 2020 19:05:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kGJQFdoJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA03621835 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8AD1B8E0005; Fri, 1 May 2020 15:05:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 85B458E0001; Fri, 1 May 2020 15:05:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7718D8E0005; Fri, 1 May 2020 15:05:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 5F9388E0001 for ; Fri, 1 May 2020 15:05:52 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 18215181AC9BF for ; Fri, 1 May 2020 19:05:52 +0000 (UTC) X-FDA: 76769079744.22.level04_8a7e6b9d44550 X-HE-Tag: level04_8a7e6b9d44550 X-Filterd-Recvd-Size: 6317 Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Fri, 1 May 2020 19:05:51 +0000 (UTC) Received: by mail-lj1-f193.google.com with SMTP id f18so3457045lja.13 for ; Fri, 01 May 2020 12:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=f6OIEvFqPmReWl2w17b5UbQj6gAIYoXFfjhT0r9r6O4=; b=kGJQFdoJS6gengiFpUslXTRx+vhgDrp8c6gVnsbYng32emFpwMWtaGIrvU4aJ/Ul4b Gm3v3f2e7JZ2tJD10ZgQRAhZQy7IjYeor31Wq9N0pGrpVZPWI4yWG1+OnJRU84U/2Xlh Evs+WRH5Q673H+ZdW5xsHPI5FH46xBgK9p9IDxqxqg60RfqMzWGBiyz0Km4LR9cRgx01 HIGlFqVXRISTEyXZXEPDL9fNLA1NGY0hZDMT9OE5K0OoY6aocazBaSQifQTTwVITF+zt vMqGOH6WCusfmrkbo5qzhy87m2HFAvknSr8junxy/mpqhqES0zIl+dskyhldT/MNYpwv tR0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=f6OIEvFqPmReWl2w17b5UbQj6gAIYoXFfjhT0r9r6O4=; b=TpjSQZDqYyNr/KGF+dLyCB5FO8FswJBX9kssswLbfvUXIf0n2JqjFctcZli2JS5jem hdFr9ZSsaIp7pfGNVxwrN9Dmnczd0qvH+KtGj3SqUadTOiXwuvVAEmVLZXhOzpDPRkKN WnlM+gCgOgDt3+9SB/W7aC97uORKo698eKuNb9BM4xobvT+IeUvdLTZ+z7t+Prgoz8hI EVFEvBdMBp4il+Z2hA0eb0Ma9qO+bjL56USsqYCf4M+4bUharPHdLgzpJuroPq2SJUeP A8pB15FUp5RQscajNo+w5+Ou3z9Gq5ngPS+RvWT++Hfe05G4DQ3SPM8DoyyY6Cn/cRI1 5L6Q== X-Gm-Message-State: AGi0PuaG7DK8djn4QC3U0GGHZagroW7679rYyMX60ONmaVhTBNsB/ygf CZ4vnddxD1PmkZGo7nIlMC0n8oQ4 X-Google-Smtp-Source: APiQypJSK/f5sm17b+w+6f9C+LcoLSzfuQov6KemVwsoHjuZfnodm1X784NRxN7yR7P7JIVmBbsGdA== X-Received: by 2002:a2e:3508:: with SMTP id z8mr2878934ljz.32.1588359949407; Fri, 01 May 2020 12:05:49 -0700 (PDT) Received: from [192.168.1.38] (88-114-211-119.elisa-laajakaista.fi. [88.114.211.119]) by smtp.gmail.com with ESMTPSA id d5sm2726397lfn.56.2020.05.01.12.05.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 01 May 2020 12:05:48 -0700 (PDT) Subject: Re: Tmpfs size accounting misses memory allocations To: Hugh Dickins Cc: Helge Deller , Pete Zaitcev , linux-mm@kvack.org References: <63ab63bd-d6c6-a96a-9667-bbe8edb7cedb@gmail.com> <38759fbe-77a1-f105-ccde-9529170bd9a0@gmail.com> From: Topi Miettinen Message-ID: Date: Fri, 1 May 2020 22:05:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1.5.2020 6.14, Hugh Dickins wrote: > On Tue, 28 Apr 2020, Topi Miettinen wrote: >> On 28.4.2020 4.34, Hugh Dickins wrote: >>> On Sat, 25 Apr 2020, Topi Miettinen wrote: >>>> Hi, >>>> >>>> It seems that tmpfs does not count memory which is allocated for short >>>> symlinks or xattrs towards size= limit. >>> >>> Yes, you are right. And that is why tmpfs does not (so far) support >>> user xattrs, because the unprivileged user could take up too much >>> memory that way. >>> >>>> I guess the fix would be to change >>>> shmem_sb_info->{used_blocks,max_blocks} to use bytes as units (instead of >>>> blocks) and then add accounting and checks to shmem_symlink() and >>>> shmem_initxattrs(). Would a patch for that be acceptable? >>> >>> Thank you for offering, but I don't think a patch for exactly that would >>> be acceptable. Because "size=" has just been another way of expressing >>> "nr_blocks=" ever since it was added in 2.4.4, and tmpfs has never >>> counted the kernel metadata towards its data blocks limit. >>> >>> You could certainly argue that it should have done from the start; but >>> in order to keep the accounting suitably simple (pages rather than bytes) >>> it never did. And I believe there are many users who expect a tmpfs of a >>> certain size to be able to accommodate data of that size, who would not >>> care to have to change their scripts or apps to meet a lower limitation. >>> >>>> >>>> Another issue is that inodes aren't counted towards size= limit either, >>>> but >>>> perhaps that's intentional because there's nr_inodes= mount option for >>>> exactly that. >>> >>> Yes, tmpfs lets the nr_inodes limit be used to constrain the kernel >>> metadata (and tmpfs has a peculiarity, that it actually counts hard >>> links out of nr_inodes, in order to limit the memory spent on dentries). >>> >>> I doubt the nr_inodes limit is depended upon so critically as the >>> nr_blocks, and I think we might extend it (say, consider each 1 of >>> nr_inodes to grant approximately 1kB of unswappable lowmem metadata) >>> to enable limited user xattrs: a patch along those lines might well >>> be acceptable. >> >> I'm interested in restricting the amount of memory allocated to tmpfs mounts >> in the system rather than granting more. I've seen a system lock up because >> tmpfs mounts consumed the entire memory. Possible contributing factors could >> be use of LVM and encryption for the swap. > > Yes, it is too easy to get into a terrible state that way. With OOM > killer doing no good at all, because it's busy killing processes, which > does nothing to free the memory held by tmpfs files. I've never found > a good answer to that in general, though marking files as suitable for > truncation on OOM has been useful in special cases. It seems that similar state can be reached also when an unprivileged process allocates and maps lots of SysV shm (if the limit isn't set, which seems to be the case at least for Debian). -Topi