From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EE69C48BF5 for ; Fri, 16 Feb 2024 21:11:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 572918D0003; Fri, 16 Feb 2024 16:11:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5222C8D0002; Fri, 16 Feb 2024 16:11:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C2E18D0003; Fri, 16 Feb 2024 16:11:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 245B78D0002 for ; Fri, 16 Feb 2024 16:11:26 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BBBC1120145 for ; Fri, 16 Feb 2024 21:11:25 +0000 (UTC) X-FDA: 81798912930.27.C98E462 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf26.hostedemail.com (Postfix) with ESMTP id 9DB66140007 for ; Fri, 16 Feb 2024 21:11:23 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Cath589n; spf=pass (imf26.hostedemail.com: domain of adrianvovk@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=adrianvovk@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708117883; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cei1sBmTCzFrZDMNjx54WFqFzKt6CM7WEQ8atTA9u6Q=; b=KABcfEiWE+vtubUF5aEswTuaohTLjk4EsqVda9cMbR2jMwd9WRr+tgtgPTBtT04jU4fsfX C+rPzQ++z88ScB4mhVtlRFAHn+yG+9EQRYOaKCJgcCydskZmBvnDPfgKqBFbZSHGtcU8dy V91wSokrv8olSWIh4siZtvqjq3KLYUw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708117883; a=rsa-sha256; cv=none; b=TZzMBejg1F1hU5QCdYuBwly6p5hWEV3XGLwkw/N3ESsloeX33/eEdws0XFFkxAmXh8vloU 3Hmdhk8FpC3N9rRF7o2q30cdvHquwI3cabyt8o2e8k8TD56aFhgPuX0LRGGGVvcy4TJn0y ucPhchR6CswWMT3jUWqXrySWvrIZ1j8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Cath589n; spf=pass (imf26.hostedemail.com: domain of adrianvovk@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=adrianvovk@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-68f1423353aso8688196d6.1 for ; Fri, 16 Feb 2024 13:11:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708117883; x=1708722683; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=cei1sBmTCzFrZDMNjx54WFqFzKt6CM7WEQ8atTA9u6Q=; b=Cath589nf9tpNNY97jzIqAMBUvahGqHH75zaZfaLN+ul6dbMS0BkRMfYTumBDZY9tX LIOcalaqbJQK4aHDLoRL+jnPnRk15lrTiw8GFtc8cXltMYFsa4EEIXcXBnNv7eI82hsD P2qtI7KAf+wU++3pKuDslWBtN7oMQqOP4TPZQatORmbEQ3GpvsbU69enqO9hz2cMWlfg vZ8VXTHkRpqrgR8KVRlUtRstxZt8CYcqZfQsu0jU77+TruDR+dCbM/rhT9Th6K5+n0LD FjKAAbDvZFX5W7g5bgWrpXEG/LdPU6ijSspFMnfyqM3wjyIDcjmcnYMikaVJ0cxuSilb vVmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708117883; x=1708722683; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cei1sBmTCzFrZDMNjx54WFqFzKt6CM7WEQ8atTA9u6Q=; b=t3wE01xQz/AOHNhUzfzjB860uV7VYncEOTGKIMrIeITTdX2zRekO+583QTNT8PWsdb /Pr90qQdexYdVSMg5GkdRA3c1KAz6qd0j4tujNIHzVZzA851hngwk4HhYATr9VSa/ybR z7Tc1/1B+zX7BEQiNvvkKUaGPE7xBLlBateSw8Pu8bDNtSqu0EDs6Kk5tkLIDmikZjgu cqv2u7aHlYE03j5yR4DwT9HI7t4t2nUHcHHEY/Rz62KpzCerZyLi/TObTI739o0QGM1Y CrNkFmTC9Ea6Ns3TMHT1X6jLh9sFqNQi8qH9NYplWgubLG1U6WTLV22aIx0LhDaP5JsL nB3Q== X-Forwarded-Encrypted: i=1; AJvYcCWJCCTqtqxKpGg9LTDYekkz45i7EwqBKAqCvNBfKvVb2LVoYSjOLunKWLGkxx8etAPUZsEIo+XVGqd5L9qOKpd/23E= X-Gm-Message-State: AOJu0Yy90tbr9rW/swqDmpoyjAEVtMYMFtDEzFS7JW+LM/dsLDIAfYQu 2+DjqKeiMPIi+71ffnNBX70mLhUw1vFmuh7MCLLXDFWL7Ra6hmdA X-Google-Smtp-Source: AGHT+IGGIyZYXqrrPYvYwagwXBP6J2L6EbwB9vuTUs8ITekdgs15UOF+4pXhOOBIbV91a80Xa+NuYA== X-Received: by 2002:a05:6214:d81:b0:68f:385d:1f9b with SMTP id e1-20020a0562140d8100b0068f385d1f9bmr2295034qve.21.1708117882573; Fri, 16 Feb 2024 13:11:22 -0800 (PST) Received: from [10.56.180.189] (184-057-057-014.res.spectrum.com. [184.57.57.14]) by smtp.gmail.com with ESMTPSA id c12-20020a0cf2cc000000b0068f1258a16asm284591qvm.42.2024.02.16.13.11.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 16 Feb 2024 13:11:22 -0800 (PST) Message-ID: <67eef60c-b0fe-4034-a2e5-b09c7ef38a5a@gmail.com> Date: Fri, 16 Feb 2024 16:11:20 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: init_on_alloc digression: [LSF/MM/BPF TOPIC] Dropping page cache of individual fs Content-Language: en-US To: John Hubbard , Dave Chinner Cc: Jan Kara , Matthew Wilcox , Christian Brauner , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-btrfs@vger.kernel.org, linux-block@vger.kernel.org, Christoph Hellwig References: <20240116-tagelang-zugnummer-349edd1b5792@brauner> <20240116114519.jcktectmk2thgagw@quack3> <20240117-tupfen-unqualifiziert-173af9bc68c8@brauner> <20240117143528.idmyeadhf4yzs5ck@quack3> <3107a023-3173-4b3d-9623-71812b1e7eb6@gmail.com> <20240215135709.4zmfb7qlerztbq6b@quack3> <10c3b162-265b-442b-80e9-8563c0168a8b@gmail.com> From: Adrian Vovk In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Stat-Signature: gcrub6a1gcgkawxkhdhmmjz9dbgntch6 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 9DB66140007 X-Rspam-User: X-HE-Tag: 1708117883-685516 X-HE-Meta: U2FsdGVkX1+Q5yjrKjjIhBDKbSM8XeTKE9aTEJLnewf233Stv/0r7Zu3nvTss1enr5mVlNUtx7KTlMSj/0Z47IsG4NEZgbKSdoIyvOhLJe8krdSggeizPlT1PklmjdpCmOb4RK1Dul+SILDSIRJCYXaZbBBibXhgvCZzt+jvhfSA16tKSPzY5R7QRF1LFlfOxBIV9vSe+Mo6v4uHfJrJYNCfHbTl0xePHv33VcDPCEUAPzNjBy5SS0VQFo0mJccwwE+UjPNHkpR720wSXFUx83ef32xDRqlbik0kM31fieqSdctCwiW30vOyxYGf+uw0a9PBdRtHApioq/t09NcqESrTbIaKLD2KimMrTNlgZj7hn+pvJkHbJmhnMZQNhOZ15w+q18ceuAuE5zUyi/J/P6fAq3Q7TNqLoAANELN/ZhgAgzOiInlS0Vv8AokdHJQIGH1sWeOrYqxK/NUKMV9hKJjJp67sBiieSibkJqpI6Gp0eT+6IjrtMr+WUVZV4ucczim8Lr0FNetKxhrYNh5CvkdITZG0QAb/o1ZX/D7bIPJVDP0q93krHjXMqCdvACp9A4FdxR0lyE8VM2dD1BCN8hAw1L1CmmTrVl1man7s7Xe2RuHuah/hAe/gOUEgx4Tjq9Fx/o338108ml8/GJA8Vfw1S8SxP+qhF/do/OgQCg9wTV+Uf5ActLIN0soi4Awp+GQ2UURJbQRuNnTf+6oM9L0IkwMmrcTHxgCYZwm2W/Op+azWtWrquCURtsSoRWmhPylOF8viazoW2vG6hv8QvrlPntcPzy123I0b0bwMdaKYP1o8m1IlrozkeUHJmUQHl4/xkTyKoi9Uur7ELP45CQfxCK+VZDSKCmAcTfb6qoLQ4i2Au4SO5hh9FApi6Pmz3stI5XftEzEIV4d4ikhcDh6dBF5OvcwNPVH6UUVEhXBzQojEc4ZlEQSbQ4cXPBpwjowbREgwdto23OyqLdy aU9NJVRF +Ga2SCYChqiFc8YJa4+XehvpRSMGsq3mJJcZU3ejfk8PmZROkjHFyGT6BFZz8piaIgYVOtx09HWW8Ns6zNNt2a6grGNCD/GYJBzgc5u0HE8NeaEsa5AmfzFl95sFajEwXT4LeWseRq8ngrI/miMYhYE0Q+CS/G/ZLyU3Pqc6WccObxEhso9YXexCxiODlpIPzNqUNfUo20qGFZXDr0fG0SwDpWcczwDIggW3ekIPP5edS6+b6u4VMYTxI7eh6SM6FooHBFZw2YTIjeNUHPmy2mDILe9Jm1XAEQ5Vp2o4nL9wriXWJhLWgMN9aQKSiMCnRiSLYqB8VreVM6ZDjl4oq++H7MnSon9HsXTsg8y+W5NiqvJbvf5YCLumAMTCXT7m3lYiz1HyG2k1ZB2vOo1A2GyJG+MTdqfKvgsHXyOiPmM/+P6wNmDDAIUHqzrkl9VF6veYLECUx8ZOiFkYlihuroL0zuiJC76xdTMD2CqxFEqPAhFKe9G5MYPUAFO/eKUdOvQLkQHBzOjUrecVUfUQZzmacLnmgRrzh3JFg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/16/24 15:38, John Hubbard wrote: > On 2/15/24 17:14, Adrian Vovk wrote: > ... >>> Typical distro configuration is: >>> >>> $ sudo dmesg |grep auto-init >>> [    0.018882] mem auto-init: stack:all(zero), heap alloc:on, heap >>> free:off >>> $ >>> >>> So this kernel zeroes all stack memory, page and heap memory on >>> allocation, and does nothing on free... >> >> I see. Thank you for all the information. >> >> So ~5% performance penalty isn't trivial, especially to protect against > > And it's more like 600% or more, on some systems. For example, imagine if > someone had a memory-coherent system that included both CPUs and GPUs, > each with their own NUMA memory nodes. The GPU has fast DMA engines that > can zero a lot of that memory very very quickly, order(s) of magnitude > faster than the CPU can clear it. > > So, the GPU driver is going to clear that memory before handing it > out to user space, and all is well so far. > > But init_on_alloc forces the CPU to clear the memory first, because of > the belief here that this is somehow required in order to get defense > in depth. (True, if you can convince yourself that some parts of the > kernel are in a different trust boundary than others. I lack faith > here and am not a believer in such make belief boundaries.) As far as I can tell init_on_alloc isn't about drawing a trust boundary between parts of the kernel, but about hardening the kernel against mistakes made by developers, i.e. if they forget to initialize some memory. If the memory isn't zero'd and the developer forgets to initialize it, then potentially memory under user control (from page cache or so) can control flow of execution in the kernel. Thus, zeroing out the memory provides a second layer of defense even in situations where the first layer (not using uninitialized memory) failed. Thus, defense in depth. Is this just an NVIDIA embedded thing (AFAIK your desktop/laptop cards don't share memory with the CPU), or would it affect something like Intel/AMD APUs as well? If the GPU is so much faster at zeroing out blocks of memory in these systems, maybe the kernel should use the GPU's DMA engine whenever it needs to zero out some blocks of memory (I'm joking, mostly; I can imagine it's not quite so simple) > Anyway, this situation has wasted much time, and at this point, I > wish I could delete the whole init_on_alloc feature. > > Just in case you wanted an alt perspective. :) This is all good to know, thanks. I'm not particularly interested in init_on_alloc since it doesn't help against cold-boot scenarios. Does init_on_free have similar performance issues on such systems? (i.e. are you often freeing memory and then immediately allocating the same memory in the GPU driver?) Either way, I'd much prefer to have both turned off and only zero out free'd memory periodically / on user request. Not on every allocation/free. > thanks, Best, Adrian