From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2C8EC64ED6 for ; Tue, 28 Feb 2023 12:07:38 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4PQx2J6y5sz3cML for ; Tue, 28 Feb 2023 23:07:36 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=gby5KpPW; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2a00:1450:4864:20::333; helo=mail-wm1-x333.google.com; envelope-from=punit.agrawal@bytedance.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=gby5KpPW; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4PQx1H5Slyz30Kr for ; Tue, 28 Feb 2023 23:06:41 +1100 (AEDT) Received: by mail-wm1-x333.google.com with SMTP id k37so6268295wms.0 for ; Tue, 28 Feb 2023 04:06:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=gby5KpPWtmDSuPSdmLVwOeyPJ75gi22wCTpni2UDLR3cqXxtkvNMB/VpBEeY9idtBP K/U8Vws19YkUgCdu+jDSNh2NeL453+iLuB/IpzCtIjSYL2zdJ/2dNnkTN2bSYLEHnq5P G642ZJC6fjauM2jjg21KMN1NQUFc/Sc/SO7y7B7QDnxxOfrauBykv/6LXVaK5Xc+k22j J7cN/VGbytB9c9UP3wvl4eb82JvIDf1II0/X5nVPbSMdwSwHGP9hF6iQJiFK/bcGD7R2 1HPZJRfW4Atk/ucwXBlYO3GuWpMJaS/2o4mxzVy5GC3hbCFS4KTT9YBiWHlCVJEbWlEc vuTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=z9kjWT6zE7jCUt9iB2JJ0VU1QVHi+P1JD9Rmb3c4XaSkXTRbXVFE5HAxTqMEEVbptB hTf5SImKmz+UiUH/sBa5UAoB1/CXNVdUOInJtDyYjImJNhI2lF80UAUPRDX/hek684pz +wf8FZgkVLTQSzHexaEkMGYe+W0+/uzF2nzskM/9BuVHvvV5nw4mj8+OmcLeHYUgT8YG lyFFnWR0uXbM1/Z2uCy4YVqMuvP7FEluSyxuyJonE+ivuK15cvqa/d0Rwly6qwbSjmAH jD913ZmnGrYjiwmGC3wVMkFfo8aP2EWzyLDyMDnDz268oi6df7kWcQNQlSw1taQbuxpB 2Qpw== X-Gm-Message-State: AO0yUKVoHPLRfwbhbLy5AB2eLW4k9XV5OK88ghdhy35EhmctPTiPACtj 12VIVFz1Wdi03MwxAdo7mtJ7fg== X-Google-Smtp-Source: AK7set9K5fr49pjDNXd0jk3vZV9+2nq0FdwHqxbkZhn7OHdaHusnSbCsOF6tn3AUWfDtVfZGp5A/wg== X-Received: by 2002:a05:600c:19ca:b0:3ea:f883:53ea with SMTP id u10-20020a05600c19ca00b003eaf88353eamr1904265wmq.7.1677585994998; Tue, 28 Feb 2023 04:06:34 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id l1-20020a1ced01000000b003ea57808179sm15482212wmh.38.2023.02.28.04.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 04:06:34 -0800 (PST) From: Punit Agrawal To: Punit Agrawal Subject: Re: [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> <87r0uqq1f9.fsf@stealth> Date: Tue, 28 Feb 2023 12:06:34 +0000 In-Reply-To: <87r0uqq1f9.fsf@stealth> (Punit Agrawal's message of "Wed, 15 Feb 2023 17:32:58 +0000") Message-ID: <87fsaqouyd.fsf_-_@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: michel@lespinasse.org, joelaf@google.com, songliubraving@fb.com, mhocko@suse.com, leewalsh@google.com, david@redhat.com, peterz@infradead.org, bigeasy@linutronix.de, peterx@redhat.com, dhowells@redhat.com, linux-mm@kvack.org, edumazet@google.com, jglisse@google.com, will@kernel.org, arjunroy@google.com, dave@stgolabs.net, minchan@google.com, x86@kernel.org, hughd@google.com, willy@infradead.org, gurua@google.com, mingo@redhat.com, linux-arm-kernel@lists.infradead.org, rientjes@google.com, axelrasmussen@google.com, kernel-team@android.com, soheil@google.com, paulmck@kernel.org, jannh@google.com, liam.howlett@oracle.com, shakeelb@google.com, luto@kernel.org, gthelen@google.com, ldufour@linux.ibm.com, Suren Baghdasaryan , vbabka@suse.cz, posk@google.com, lstoakes@gmail.com, peterjung1337@gmail.com, linuxppc-dev@lists.ozlabs.org, kent.overstreet@linux.dev, linux-kernel@vger.kernel.org, hannes@cmpxchg.org, akpm@linux-foundation.org, tatashin@google.com, mgorman@tech singularity.net, rppt@kernel.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Punit Agrawal writes: > Suren Baghdasaryan writes: > >> Previous version: >> v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.c= om/ >> RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.c= om/ >> >> LWN article describing the feature: >> https://lwn.net/Articles/906852/ >> >> Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM >> last year [2], which concluded with suggestion that =E2=80=9Ca reader/wr= iter >> semaphore could be put into the VMA itself; that would have the effect of >> using the VMA as a sort of range lock. There would still be contention at >> the VMA level, but it would be an improvement.=E2=80=9D This patchset im= plements >> this suggested approach. > > I took the patches for a spin on a 2-socket 32 core (64 threads) system > with Intel 8336C (Ice Lake) and 512GB of RAM. > > For the initial testing, "pft-threads" from the mm-tests suite[0] was > used. The test mmaps a memory region (~100GB on the test system) and > triggers access by a number of threads executing in parallel. For each > degree of parallelism, the test is repeated 10 times to get a better > feel for the behaviour. Below is an excerpt of the harmonic mean > reported by 'compare_kernel' script[1] included with mm-tests. > > The first column is results for mm-unstable as of 2023-02-10, the second > column is the patches posted here while the third column includes > optimizations to reclaim some of the observed regression. > > From the results, there is a drop in page fault/second for low number of > CPUs but good improvement with higher CPUs. > > 6.2.0-rc4 6.2.0-rc= 4 6.2.0-rc4 > mm-unstable-20230210 pvl-v= 2 pvl-v2+opt > > Hmean faults/cpu-1 898792.9338 ( 0.00%) 894597.0474 * -0.47%= * 895933.2782 * -0.32%* > Hmean faults/cpu-4 751903.9803 ( 0.00%) 677764.2975 * -9.86%= * 688643.8163 * -8.41%* > Hmean faults/cpu-7 612275.5663 ( 0.00%) 565363.4137 * -7.66%= * 597538.9396 * -2.41%* > Hmean faults/cpu-12 434460.9074 ( 0.00%) 410974.2708 * -5.41%= * 452501.4290 * 4.15%* > Hmean faults/cpu-21 291475.5165 ( 0.00%) 293936.8460 ( 0.84%= ) 308712.2434 * 5.91%* > Hmean faults/cpu-30 218021.3980 ( 0.00%) 228265.0559 * 4.70%= * 241897.5225 * 10.95%* > Hmean faults/cpu-48 141798.5030 ( 0.00%) 162322.5972 * 14.47%= * 166081.9459 * 17.13%* > Hmean faults/cpu-79 90060.9577 ( 0.00%) 107028.7779 * 18.84%= * 109810.4488 * 21.93%* > Hmean faults/cpu-110 64729.3561 ( 0.00%) 80597.7246 * 24.51%= * 83134.0679 * 28.43%* > Hmean faults/cpu-128 55740.1334 ( 0.00%) 68395.4426 * 22.70%= * 69248.2836 * 24.23%* > > Hmean faults/sec-1 898781.7694 ( 0.00%) 894247.3174 * -0.50%= * 894440.3118 * -0.48%* > Hmean faults/sec-4 2965588.9697 ( 0.00%) 2683651.5664 * -9.51%= * 2726450.9710 * -8.06%* > Hmean faults/sec-7 4144512.3996 ( 0.00%) 3891644.2128 * -6.10%= * 4099918.8601 ( -1.08%) > Hmean faults/sec-12 4969513.6934 ( 0.00%) 4829731.4355 * -2.81%= * 5264682.7371 * 5.94%* > Hmean faults/sec-21 5814379.4789 ( 0.00%) 5941405.3116 * 2.18%= * 6263716.3903 * 7.73%* > Hmean faults/sec-30 6153685.3709 ( 0.00%) 6489311.6634 * 5.45%= * 6910843.5858 * 12.30%* > Hmean faults/sec-48 6197953.1327 ( 0.00%) 7216320.7727 * 16.43%= * 7412782.2927 * 19.60%* > Hmean faults/sec-79 6167135.3738 ( 0.00%) 7425927.1022 * 20.41%= * 7637042.2198 * 23.83%* > Hmean faults/sec-110 6264768.2247 ( 0.00%) 7813329.3863 * 24.72%= * 7984344.4005 * 27.45%* > Hmean faults/sec-128 6460727.8216 ( 0.00%) 7875664.8999 * 21.90%= * 8049910.3601 * 24.60%* The above workload represent the worst case with regards to per-VMA locks as it creates a single large VMA. As a follow-up, I modified pft[2] to create a VMA per thread to understand the behaviour in scenarios where per-VMA locks should show the most benefit. 6.2.0-rc4 6.2.0-rc4 = 6.2.0-rc4 mm-unstable-20230210 pvl-v2 = pvl-v2+opt Hmean faults/cpu-1 905497.4354 ( 0.00%) 888736.5570 * -1.85%* = 888695.2675 * -1.86%* Hmean faults/cpu-4 758519.2719 ( 0.00%) 812103.1991 * 7.06%* = 825077.9277 * 8.77%* Hmean faults/cpu-7 617153.8038 ( 0.00%) 729943.4518 * 18.28%* = 770872.3161 * 24.91%* Hmean faults/cpu-12 424848.5266 ( 0.00%) 550357.2856 * 29.54%* = 597478.5634 * 40.63%* Hmean faults/cpu-21 290142.9988 ( 0.00%) 383668.3190 * 32.23%* = 433376.8959 * 49.37%* Hmean faults/cpu-30 218705.2915 ( 0.00%) 299888.5533 * 37.12%* = 342640.6153 * 56.67%* Hmean faults/cpu-48 142842.3372 ( 0.00%) 206498.2605 * 44.56%* = 240306.3442 * 68.23%* Hmean faults/cpu-79 90706.1425 ( 0.00%) 160006.6800 * 76.40%* = 185298.4326 * 104.28%* Hmean faults/cpu-110 67011.9297 ( 0.00%) 143536.0062 * 114.19%* = 162688.8015 * 142.78%* Hmean faults/cpu-128 55986.4986 ( 0.00%) 136550.8760 * 143.90%* = 152718.8713 * 172.78%* Hmean faults/sec-1 905492.1265 ( 0.00%) 887244.6592 * -2.02%* = 887775.6079 * -1.96%* Hmean faults/sec-4 2994284.4204 ( 0.00%) 3154236.9408 * 5.34%* = 3221994.8465 * 7.60%* Hmean faults/sec-7 4177411.3461 ( 0.00%) 4933286.4045 * 18.09%* = 5202347.2077 * 24.54%* Hmean faults/sec-12 4892848.3633 ( 0.00%) 6054577.0988 * 23.74%* = 6511987.1142 * 33.09%* Hmean faults/sec-21 5823534.1820 ( 0.00%) 7637637.4162 * 31.15%* = 8553362.3513 * 46.88%* Hmean faults/sec-30 6247210.8414 ( 0.00%) 8598150.6717 * 37.63%* = 9799696.0945 * 56.87%* Hmean faults/sec-48 6274617.1419 ( 0.00%) 9467132.3699 * 50.88%* = 11049401.9072 * 76.10%* Hmean faults/sec-79 6187291.4971 ( 0.00%) 11919062.5284 * 92.64%* = 13420825.3820 * 116.91%* Hmean faults/sec-110 6454542.3239 ( 0.00%) 15050228.1869 * 133.17%* = 16667873.7618 * 158.23%* Hmean faults/sec-128 6472970.8548 ( 0.00%) 16647275.6575 * 157.18%* = 18680029.3714 * 188.59%* As expected, the tests highlight the improved scalability as core count increases. > [0] https://github.com/gormanm/mmtests > [1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh [2] https://github.com/gormanm/pft/pull/1/commits/8fe554a3d8b4f5947cd00d4b4= 6f97178b8ba8752 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0517BC7EE2E for ; Tue, 28 Feb 2023 12:07:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:In-Reply-To: Date:References:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cVe08czO1SntCIf7qfDYFBYq1zs3PkdbXoyBuzVkYeU=; b=GaSSK6gDOBq+CA Ne3BBsYgzeWXzv4VbLUMgISa/+NN/zoAbPKNf04SnWJ3YuqVqd2Begfl3VLDR9sycMrrWPl6ZSsPj Qy23XRuSp/MD5r5kN6G22atCrnLbcKIqMrxHc2OOQ0AvCORgQqa3hULxwAeSiB4sNXXBXirSwe0Ql PMC+OAjedK7D12NhOhOd8UYckch+AkyYG/ZexesOsICCaxjIIWuo/9h4yN0DfiviFdiTww+a7xVpN zoPvFayOhww0wdNZ2NI7gII8rVOYa6q6DN6bAFmffaj5bMGeoO4xf37Eul4p9UWzh6bspwMoo3s+c p5OpFWjU/w2tyEL5C3ww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pWyko-00D6ym-4D; Tue, 28 Feb 2023 12:06:46 +0000 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pWykk-00D6w2-5r for linux-arm-kernel@lists.infradead.org; Tue, 28 Feb 2023 12:06:44 +0000 Received: by mail-wm1-x330.google.com with SMTP id c18so6224249wmr.3 for ; Tue, 28 Feb 2023 04:06:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=gby5KpPWtmDSuPSdmLVwOeyPJ75gi22wCTpni2UDLR3cqXxtkvNMB/VpBEeY9idtBP K/U8Vws19YkUgCdu+jDSNh2NeL453+iLuB/IpzCtIjSYL2zdJ/2dNnkTN2bSYLEHnq5P G642ZJC6fjauM2jjg21KMN1NQUFc/Sc/SO7y7B7QDnxxOfrauBykv/6LXVaK5Xc+k22j J7cN/VGbytB9c9UP3wvl4eb82JvIDf1II0/X5nVPbSMdwSwHGP9hF6iQJiFK/bcGD7R2 1HPZJRfW4Atk/ucwXBlYO3GuWpMJaS/2o4mxzVy5GC3hbCFS4KTT9YBiWHlCVJEbWlEc vuTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=Hl6uqUPJ2N1lY/lgYiYuJ1WvE3LW6MeuZHDV7umlGxtqBCN2NzzAb+Oa1t2dx2z8ce m08ivTF7feuOhqpWhQDiFgUe0X7i8ex7/Q17LJdFIg6Ikr3Dfm+wBJE8rjJ1NNBg5kwe 5nbPMMu3I0z3DQes//4Qpu6g2hphqZByADhqkneXJuZu9tYnl7jQTTBjjlpqR8+yuO3s lOX+i9GQ/s4S7zOARzOjkPIpFe8T64Lu5GJyWIsiSKgf6EZKhzantUeKZbDuFCYtkqZO dARznGU+qhjiVF0uiVAqvRmG65QOn6Ng9jUFJApDM9Pgey88L1NOfrL7RVVo4xWIUqMi R+Pw== X-Gm-Message-State: AO0yUKUnDpF57LvbtADWFciuF3fsmilTpDGgYIVq4iv/6kh4Sd7J5KIg jyBpyFR2wgWI2WUsM4BiN0+aMA== X-Google-Smtp-Source: AK7set9K5fr49pjDNXd0jk3vZV9+2nq0FdwHqxbkZhn7OHdaHusnSbCsOF6tn3AUWfDtVfZGp5A/wg== X-Received: by 2002:a05:600c:19ca:b0:3ea:f883:53ea with SMTP id u10-20020a05600c19ca00b003eaf88353eamr1904265wmq.7.1677585994998; Tue, 28 Feb 2023 04:06:34 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id l1-20020a1ced01000000b003ea57808179sm15482212wmh.38.2023.02.28.04.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 04:06:34 -0800 (PST) From: Punit Agrawal To: Punit Agrawal Cc: Suren Baghdasaryan , akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> <87r0uqq1f9.fsf@stealth> Date: Tue, 28 Feb 2023 12:06:34 +0000 In-Reply-To: <87r0uqq1f9.fsf@stealth> (Punit Agrawal's message of "Wed, 15 Feb 2023 17:32:58 +0000") Message-ID: <87fsaqouyd.fsf_-_@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230228_040642_528735_4C5C8589 X-CRM114-Status: GOOD ( 19.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org UHVuaXQgQWdyYXdhbCA8cHVuaXQuYWdyYXdhbEBieXRlZGFuY2UuY29tPiB3cml0ZXM6Cgo+IFN1 cmVuIEJhZ2hkYXNhcnlhbiA8c3VyZW5iQGdvb2dsZS5jb20+IHdyaXRlczoKPgo+PiBQcmV2aW91 cyB2ZXJzaW9uOgo+PiB2MTogaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvYWxsLzIwMjMwMTA5MjA1 MzM2LjM2NjU5MzctMS1zdXJlbmJAZ29vZ2xlLmNvbS8KPj4gUkZDOiBodHRwczovL2xvcmUua2Vy bmVsLm9yZy9hbGwvMjAyMjA5MDExNzM1MTYuNzAyMTIyLTEtc3VyZW5iQGdvb2dsZS5jb20vCj4+ Cj4+IExXTiBhcnRpY2xlIGRlc2NyaWJpbmcgdGhlIGZlYXR1cmU6Cj4+IGh0dHBzOi8vbHduLm5l dC9BcnRpY2xlcy85MDY4NTIvCj4+Cj4+IFBlci12bWEgbG9ja3MgaWRlYSB0aGF0IHdhcyBkaXNj dXNzZWQgZHVyaW5nIFNQRiBbMV0gZGlzY3Vzc2lvbiBhdCBMU0YvTU0KPj4gbGFzdCB5ZWFyIFsy XSwgd2hpY2ggY29uY2x1ZGVkIHdpdGggc3VnZ2VzdGlvbiB0aGF0IOKAnGEgcmVhZGVyL3dyaXRl cgo+PiBzZW1hcGhvcmUgY291bGQgYmUgcHV0IGludG8gdGhlIFZNQSBpdHNlbGY7IHRoYXQgd291 bGQgaGF2ZSB0aGUgZWZmZWN0IG9mCj4+IHVzaW5nIHRoZSBWTUEgYXMgYSBzb3J0IG9mIHJhbmdl IGxvY2suIFRoZXJlIHdvdWxkIHN0aWxsIGJlIGNvbnRlbnRpb24gYXQKPj4gdGhlIFZNQSBsZXZl bCwgYnV0IGl0IHdvdWxkIGJlIGFuIGltcHJvdmVtZW50LuKAnSBUaGlzIHBhdGNoc2V0IGltcGxl bWVudHMKPj4gdGhpcyBzdWdnZXN0ZWQgYXBwcm9hY2guCj4KPiBJIHRvb2sgdGhlIHBhdGNoZXMg Zm9yIGEgc3BpbiBvbiBhIDItc29ja2V0IDMyIGNvcmUgKDY0IHRocmVhZHMpIHN5c3RlbQo+IHdp dGggSW50ZWwgODMzNkMgKEljZSBMYWtlKSBhbmQgNTEyR0Igb2YgUkFNLgo+Cj4gRm9yIHRoZSBp bml0aWFsIHRlc3RpbmcsICJwZnQtdGhyZWFkcyIgZnJvbSB0aGUgbW0tdGVzdHMgc3VpdGVbMF0g d2FzCj4gdXNlZC4gVGhlIHRlc3QgbW1hcHMgYSBtZW1vcnkgcmVnaW9uICh+MTAwR0Igb24gdGhl IHRlc3Qgc3lzdGVtKSBhbmQKPiB0cmlnZ2VycyBhY2Nlc3MgYnkgYSBudW1iZXIgb2YgdGhyZWFk cyBleGVjdXRpbmcgaW4gcGFyYWxsZWwuIEZvciBlYWNoCj4gZGVncmVlIG9mIHBhcmFsbGVsaXNt LCB0aGUgdGVzdCBpcyByZXBlYXRlZCAxMCB0aW1lcyB0byBnZXQgYSBiZXR0ZXIKPiBmZWVsIGZv ciB0aGUgYmVoYXZpb3VyLiBCZWxvdyBpcyBhbiBleGNlcnB0IG9mIHRoZSBoYXJtb25pYyBtZWFu Cj4gcmVwb3J0ZWQgYnkgJ2NvbXBhcmVfa2VybmVsJyBzY3JpcHRbMV0gaW5jbHVkZWQgd2l0aCBt bS10ZXN0cy4KPgo+IFRoZSBmaXJzdCBjb2x1bW4gaXMgcmVzdWx0cyBmb3IgbW0tdW5zdGFibGUg YXMgb2YgMjAyMy0wMi0xMCwgdGhlIHNlY29uZAo+IGNvbHVtbiBpcyB0aGUgcGF0Y2hlcyBwb3N0 ZWQgaGVyZSB3aGlsZSB0aGUgdGhpcmQgY29sdW1uIGluY2x1ZGVzCj4gb3B0aW1pemF0aW9ucyB0 byByZWNsYWltIHNvbWUgb2YgdGhlIG9ic2VydmVkIHJlZ3Jlc3Npb24uCj4KPiBGcm9tIHRoZSBy ZXN1bHRzLCB0aGVyZSBpcyBhIGRyb3AgaW4gcGFnZSBmYXVsdC9zZWNvbmQgZm9yIGxvdyBudW1i ZXIgb2YKPiBDUFVzIGJ1dCBnb29kIGltcHJvdmVtZW50IHdpdGggaGlnaGVyIENQVXMuCj4KPiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgNi4yLjAtcmM0ICAgICAgICAg ICAgICAgIDYuMi4wLXJjNCAgICAgICAgICAgICAgICA2LjIuMC1yYzQKPiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIG1tLXVuc3RhYmxlLTIwMjMwMjEwICAgICAgICAgICAgICAgICAgIHB2 bC12MiAgICAgICAgICAgICAgIHB2bC12MitvcHQKPgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTEg ICAgIDg5ODc5Mi45MzM4ICggICAwLjAwJSkgICA4OTQ1OTcuMDQ3NCAqICAtMC40NyUqICAgODk1 OTMzLjI3ODIgKiAgLTAuMzIlKgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTQgICAgIDc1MTkwMy45 ODAzICggICAwLjAwJSkgICA2Nzc3NjQuMjk3NSAqICAtOS44NiUqICAgNjg4NjQzLjgxNjMgKiAg LTguNDElKgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTcgICAgIDYxMjI3NS41NjYzICggICAwLjAw JSkgICA1NjUzNjMuNDEzNyAqICAtNy42NiUqICAgNTk3NTM4LjkzOTYgKiAgLTIuNDElKgo+IEht ZWFuICAgICBmYXVsdHMvY3B1LTEyICAgIDQzNDQ2MC45MDc0ICggICAwLjAwJSkgICA0MTA5NzQu MjcwOCAqICAtNS40MSUqICAgNDUyNTAxLjQyOTAgKiAgIDQuMTUlKgo+IEhtZWFuICAgICBmYXVs dHMvY3B1LTIxICAgIDI5MTQ3NS41MTY1ICggICAwLjAwJSkgICAyOTM5MzYuODQ2MCAoICAgMC44 NCUpICAgMzA4NzEyLjI0MzQgKiAgIDUuOTElKgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTMwICAg IDIxODAyMS4zOTgwICggICAwLjAwJSkgICAyMjgyNjUuMDU1OSAqICAgNC43MCUqICAgMjQxODk3 LjUyMjUgKiAgMTAuOTUlKgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTQ4ICAgIDE0MTc5OC41MDMw ICggICAwLjAwJSkgICAxNjIzMjIuNTk3MiAqICAxNC40NyUqICAgMTY2MDgxLjk0NTkgKiAgMTcu MTMlKgo+IEhtZWFuICAgICBmYXVsdHMvY3B1LTc5ICAgICA5MDA2MC45NTc3ICggICAwLjAwJSkg ICAxMDcwMjguNzc3OSAqICAxOC44NCUqICAgMTA5ODEwLjQ0ODggKiAgMjEuOTMlKgo+IEhtZWFu ICAgICBmYXVsdHMvY3B1LTExMCAgICA2NDcyOS4zNTYxICggICAwLjAwJSkgICAgODA1OTcuNzI0 NiAqICAyNC41MSUqICAgIDgzMTM0LjA2NzkgKiAgMjguNDMlKgo+IEhtZWFuICAgICBmYXVsdHMv Y3B1LTEyOCAgICA1NTc0MC4xMzM0ICggICAwLjAwJSkgICAgNjgzOTUuNDQyNiAqICAyMi43MCUq ICAgIDY5MjQ4LjI4MzYgKiAgMjQuMjMlKgo+Cj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtMSAgICAg ODk4NzgxLjc2OTQgKCAgIDAuMDAlKSAgIDg5NDI0Ny4zMTc0ICogIC0wLjUwJSogICA4OTQ0NDAu MzExOCAqICAtMC40OCUqCj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtNCAgICAyOTY1NTg4Ljk2OTcg KCAgIDAuMDAlKSAgMjY4MzY1MS41NjY0ICogIC05LjUxJSogIDI3MjY0NTAuOTcxMCAqICAtOC4w NiUqCj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtNyAgICA0MTQ0NTEyLjM5OTYgKCAgIDAuMDAlKSAg Mzg5MTY0NC4yMTI4ICogIC02LjEwJSogIDQwOTk5MTguODYwMSAoICAtMS4wOCUpCj4gSG1lYW4g ICAgIGZhdWx0cy9zZWMtMTIgICA0OTY5NTEzLjY5MzQgKCAgIDAuMDAlKSAgNDgyOTczMS40MzU1 ICogIC0yLjgxJSogIDUyNjQ2ODIuNzM3MSAqICAgNS45NCUqCj4gSG1lYW4gICAgIGZhdWx0cy9z ZWMtMjEgICA1ODE0Mzc5LjQ3ODkgKCAgIDAuMDAlKSAgNTk0MTQwNS4zMTE2ICogICAyLjE4JSog IDYyNjM3MTYuMzkwMyAqICAgNy43MyUqCj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtMzAgICA2MTUz Njg1LjM3MDkgKCAgIDAuMDAlKSAgNjQ4OTMxMS42NjM0ICogICA1LjQ1JSogIDY5MTA4NDMuNTg1 OCAqICAxMi4zMCUqCj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtNDggICA2MTk3OTUzLjEzMjcgKCAg IDAuMDAlKSAgNzIxNjMyMC43NzI3ICogIDE2LjQzJSogIDc0MTI3ODIuMjkyNyAqICAxOS42MCUq Cj4gSG1lYW4gICAgIGZhdWx0cy9zZWMtNzkgICA2MTY3MTM1LjM3MzggKCAgIDAuMDAlKSAgNzQy NTkyNy4xMDIyICogIDIwLjQxJSogIDc2MzcwNDIuMjE5OCAqICAyMy44MyUqCj4gSG1lYW4gICAg IGZhdWx0cy9zZWMtMTEwICA2MjY0NzY4LjIyNDcgKCAgIDAuMDAlKSAgNzgxMzMyOS4zODYzICog IDI0LjcyJSogIDc5ODQzNDQuNDAwNSAqICAyNy40NSUqCj4gSG1lYW4gICAgIGZhdWx0cy9zZWMt MTI4ICA2NDYwNzI3LjgyMTYgKCAgIDAuMDAlKSAgNzg3NTY2NC44OTk5ICogIDIxLjkwJSogIDgw NDk5MTAuMzYwMSAqICAyNC42MCUqCgoKVGhlIGFib3ZlIHdvcmtsb2FkIHJlcHJlc2VudCB0aGUg d29yc3QgY2FzZSB3aXRoIHJlZ2FyZHMgdG8gcGVyLVZNQQpsb2NrcyBhcyBpdCBjcmVhdGVzIGEg c2luZ2xlIGxhcmdlIFZNQS4gQXMgYSBmb2xsb3ctdXAsIEkgbW9kaWZpZWQKcGZ0WzJdIHRvIGNy ZWF0ZSBhIFZNQSBwZXIgdGhyZWFkIHRvIHVuZGVyc3RhbmQgdGhlIGJlaGF2aW91ciBpbgpzY2Vu YXJpb3Mgd2hlcmUgcGVyLVZNQSBsb2NrcyBzaG91bGQgc2hvdyB0aGUgbW9zdCBiZW5lZml0LgoK ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDYuMi4wLXJjNCAgICAgICAg ICAgICAgICA2LjIuMC1yYzQgICAgICAgICAgICAgICAgNi4yLjAtcmM0CiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgbW0tdW5zdGFibGUtMjAyMzAyMTAgICAgICAgICAgICAgICAgICAgcHZs LXYyICAgICAgICAgICAgICAgcHZsLXYyK29wdAoKSG1lYW4gICAgIGZhdWx0cy9jcHUtMSAgICAg OTA1NDk3LjQzNTQgKCAgIDAuMDAlKSAgIDg4ODczNi41NTcwICogIC0xLjg1JSogICA4ODg2OTUu MjY3NSAqICAtMS44NiUqCkhtZWFuICAgICBmYXVsdHMvY3B1LTQgICAgIDc1ODUxOS4yNzE5ICgg ICAwLjAwJSkgICA4MTIxMDMuMTk5MSAqICAgNy4wNiUqICAgODI1MDc3LjkyNzcgKiAgIDguNzcl KgpIbWVhbiAgICAgZmF1bHRzL2NwdS03ICAgICA2MTcxNTMuODAzOCAoICAgMC4wMCUpICAgNzI5 OTQzLjQ1MTggKiAgMTguMjglKiAgIDc3MDg3Mi4zMTYxICogIDI0LjkxJSoKSG1lYW4gICAgIGZh dWx0cy9jcHUtMTIgICAgNDI0ODQ4LjUyNjYgKCAgIDAuMDAlKSAgIDU1MDM1Ny4yODU2ICogIDI5 LjU0JSogICA1OTc0NzguNTYzNCAqICA0MC42MyUqCkhtZWFuICAgICBmYXVsdHMvY3B1LTIxICAg IDI5MDE0Mi45OTg4ICggICAwLjAwJSkgICAzODM2NjguMzE5MCAqICAzMi4yMyUqICAgNDMzMzc2 Ljg5NTkgKiAgNDkuMzclKgpIbWVhbiAgICAgZmF1bHRzL2NwdS0zMCAgICAyMTg3MDUuMjkxNSAo ICAgMC4wMCUpICAgMjk5ODg4LjU1MzMgKiAgMzcuMTIlKiAgIDM0MjY0MC42MTUzICogIDU2LjY3 JSoKSG1lYW4gICAgIGZhdWx0cy9jcHUtNDggICAgMTQyODQyLjMzNzIgKCAgIDAuMDAlKSAgIDIw NjQ5OC4yNjA1ICogIDQ0LjU2JSogICAyNDAzMDYuMzQ0MiAqICA2OC4yMyUqCkhtZWFuICAgICBm YXVsdHMvY3B1LTc5ICAgICA5MDcwNi4xNDI1ICggICAwLjAwJSkgICAxNjAwMDYuNjgwMCAqICA3 Ni40MCUqICAgMTg1Mjk4LjQzMjYgKiAxMDQuMjglKgpIbWVhbiAgICAgZmF1bHRzL2NwdS0xMTAg ICAgNjcwMTEuOTI5NyAoICAgMC4wMCUpICAgMTQzNTM2LjAwNjIgKiAxMTQuMTklKiAgIDE2MjY4 OC44MDE1ICogMTQyLjc4JSoKSG1lYW4gICAgIGZhdWx0cy9jcHUtMTI4ICAgIDU1OTg2LjQ5ODYg KCAgIDAuMDAlKSAgIDEzNjU1MC44NzYwICogMTQzLjkwJSogICAxNTI3MTguODcxMyAqIDE3Mi43 OCUqCgpIbWVhbiAgICAgZmF1bHRzL3NlYy0xICAgICA5MDU0OTIuMTI2NSAoICAgMC4wMCUpICAg ODg3MjQ0LjY1OTIgKiAgLTIuMDIlKiAgIDg4Nzc3NS42MDc5ICogIC0xLjk2JSoKSG1lYW4gICAg IGZhdWx0cy9zZWMtNCAgICAyOTk0Mjg0LjQyMDQgKCAgIDAuMDAlKSAgMzE1NDIzNi45NDA4ICog ICA1LjM0JSogIDMyMjE5OTQuODQ2NSAqICAgNy42MCUqCkhtZWFuICAgICBmYXVsdHMvc2VjLTcg ICAgNDE3NzQxMS4zNDYxICggICAwLjAwJSkgIDQ5MzMyODYuNDA0NSAqICAxOC4wOSUqICA1MjAy MzQ3LjIwNzcgKiAgMjQuNTQlKgpIbWVhbiAgICAgZmF1bHRzL3NlYy0xMiAgIDQ4OTI4NDguMzYz MyAoICAgMC4wMCUpICA2MDU0NTc3LjA5ODggKiAgMjMuNzQlKiAgNjUxMTk4Ny4xMTQyICogIDMz LjA5JSoKSG1lYW4gICAgIGZhdWx0cy9zZWMtMjEgICA1ODIzNTM0LjE4MjAgKCAgIDAuMDAlKSAg NzYzNzYzNy40MTYyICogIDMxLjE1JSogIDg1NTMzNjIuMzUxMyAqICA0Ni44OCUqCkhtZWFuICAg ICBmYXVsdHMvc2VjLTMwICAgNjI0NzIxMC44NDE0ICggICAwLjAwJSkgIDg1OTgxNTAuNjcxNyAq ICAzNy42MyUqICA5Nzk5Njk2LjA5NDUgKiAgNTYuODclKgpIbWVhbiAgICAgZmF1bHRzL3NlYy00 OCAgIDYyNzQ2MTcuMTQxOSAoICAgMC4wMCUpICA5NDY3MTMyLjM2OTkgKiAgNTAuODglKiAxMTA0 OTQwMS45MDcyICogIDc2LjEwJSoKSG1lYW4gICAgIGZhdWx0cy9zZWMtNzkgICA2MTg3MjkxLjQ5 NzEgKCAgIDAuMDAlKSAxMTkxOTA2Mi41Mjg0ICogIDkyLjY0JSogMTM0MjA4MjUuMzgyMCAqIDEx Ni45MSUqCkhtZWFuICAgICBmYXVsdHMvc2VjLTExMCAgNjQ1NDU0Mi4zMjM5ICggICAwLjAwJSkg MTUwNTAyMjguMTg2OSAqIDEzMy4xNyUqIDE2NjY3ODczLjc2MTggKiAxNTguMjMlKgpIbWVhbiAg ICAgZmF1bHRzL3NlYy0xMjggIDY0NzI5NzAuODU0OCAoICAgMC4wMCUpIDE2NjQ3Mjc1LjY1NzUg KiAxNTcuMTglKiAxODY4MDAyOS4zNzE0ICogMTg4LjU5JSoKCkFzIGV4cGVjdGVkLCB0aGUgdGVz dHMgaGlnaGxpZ2h0IHRoZSBpbXByb3ZlZCBzY2FsYWJpbGl0eSBhcyBjb3JlIGNvdW50CmluY3Jl YXNlcy4KCj4gWzBdIGh0dHBzOi8vZ2l0aHViLmNvbS9nb3JtYW5tL21tdGVzdHMKPiBbMV0gaHR0 cHM6Ly9naXRodWIuY29tL2dvcm1hbm0vbW10ZXN0cy9ibG9iL21hc3Rlci9jb21wYXJlLWtlcm5l bHMuc2gKClsyXSBodHRwczovL2dpdGh1Yi5jb20vZ29ybWFubS9wZnQvcHVsbC8xL2NvbW1pdHMv OGZlNTU0YTNkOGI0ZjU5NDdjZDAwZDRiNDZmOTcxNzhiOGJhODc1MgoKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5n IGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5p bmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81D95C64ED6 for ; Tue, 28 Feb 2023 12:06:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D16566B0071; Tue, 28 Feb 2023 07:06:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C9F796B0072; Tue, 28 Feb 2023 07:06:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1A3F6B0073; Tue, 28 Feb 2023 07:06:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9AF126B0071 for ; Tue, 28 Feb 2023 07:06:40 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 415981A14C6 for ; Tue, 28 Feb 2023 12:06:40 +0000 (UTC) X-FDA: 80516573760.20.EE1201C Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf26.hostedemail.com (Postfix) with ESMTP id CE6F014001A for ; Tue, 28 Feb 2023 12:06:36 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=gby5KpPW; spf=pass (imf26.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677585997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=Wn/rrtIkZiWEmxyiau3i2qv6aGEcrx0nrLOCFowL8lLWUtiBYzVdl2S3mgbeWfgU/Me/Vv +qlouD63OGtRNT5eIZxzdSIBKSHAPWq7u2w4AI111bt5NIUc3hAmWK1Mgg/U76xW45cRZW Okp6R1kfxrrJxJeZrZV8Q5FGZ2yYRFw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=gby5KpPW; spf=pass (imf26.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677585997; a=rsa-sha256; cv=none; b=c0P2fLfvRVKUduGb5M46og0oFuHbVL/uLYgYxn3oSlEurYWPev03kSqxP/2iv07y1V2KZY lupCM+W7a+df0gzhgKlIgFs41qOZdC1wyK9eUUZBIPGlhZIYzV2yQmWkDYC/HvKYrzlFZF JJb4BLU2lRK5s1TFszrs8gqtZOpgvVc= Received: by mail-wm1-f44.google.com with SMTP id j3so6238493wms.2 for ; Tue, 28 Feb 2023 04:06:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=gby5KpPWtmDSuPSdmLVwOeyPJ75gi22wCTpni2UDLR3cqXxtkvNMB/VpBEeY9idtBP K/U8Vws19YkUgCdu+jDSNh2NeL453+iLuB/IpzCtIjSYL2zdJ/2dNnkTN2bSYLEHnq5P G642ZJC6fjauM2jjg21KMN1NQUFc/Sc/SO7y7B7QDnxxOfrauBykv/6LXVaK5Xc+k22j J7cN/VGbytB9c9UP3wvl4eb82JvIDf1II0/X5nVPbSMdwSwHGP9hF6iQJiFK/bcGD7R2 1HPZJRfW4Atk/ucwXBlYO3GuWpMJaS/2o4mxzVy5GC3hbCFS4KTT9YBiWHlCVJEbWlEc vuTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677585995; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bp3F4cfna3ERELV0qv1FC8LtmHZqyDHs34ywT+7WJr4=; b=Zedptzk7+o6Tupp3a3Po39j05fR+8WRm37tXDCqxOyWe/EvQI1T/uUB2AGLXFckXnv pKa5YNKCoBDVuQP/CLK2unIe8+fH80yVOy+ZKsXQl3UzZP8OFscxxSSK8mG4AGvDX754 Fff7vheuhHjxokQUdv/TgWpxiOvVSoFF05rRxs/yrxZ3kj8+YEsQIx+QhZ9HnYHZJNf5 ms3f1f1RrOA4LH2qwD/s1xftCjMR3tBql7ap0hXsN0QdiLY9fiWtY0BvkaqmaelRcvpv g2tYJhsZBiEyOZDnqih9s294sbR0PAFgbbD64sSlkNtHZtaTd0iersfFtJ/IsIYAlIuD tkMg== X-Gm-Message-State: AO0yUKVqqewpeH6o17uaDLjJ5h0vU34McWpsVj6Fp2boeh6I+YBIwX6F T5AxKy7MSqAwh9ARb8Aev7//rw== X-Google-Smtp-Source: AK7set9K5fr49pjDNXd0jk3vZV9+2nq0FdwHqxbkZhn7OHdaHusnSbCsOF6tn3AUWfDtVfZGp5A/wg== X-Received: by 2002:a05:600c:19ca:b0:3ea:f883:53ea with SMTP id u10-20020a05600c19ca00b003eaf88353eamr1904265wmq.7.1677585994998; Tue, 28 Feb 2023 04:06:34 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id l1-20020a1ced01000000b003ea57808179sm15482212wmh.38.2023.02.28.04.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 04:06:34 -0800 (PST) From: Punit Agrawal To: Punit Agrawal Cc: Suren Baghdasaryan , akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> <87r0uqq1f9.fsf@stealth> Date: Tue, 28 Feb 2023 12:06:34 +0000 In-Reply-To: <87r0uqq1f9.fsf@stealth> (Punit Agrawal's message of "Wed, 15 Feb 2023 17:32:58 +0000") Message-ID: <87fsaqouyd.fsf_-_@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CE6F014001A X-Stat-Signature: cugw55pm9wmaywpzpwadpabka616umfr X-Rspam-User: X-HE-Tag: 1677585996-441218 X-HE-Meta: U2FsdGVkX18mYOm9hupGRTwdwVO4zjkV4g6ULpkBHa5zMiOnLIgdmTMsliOciLpxCEljooWZLKWwDxsu6Hwb1fBAoXtMgGqIPCivxw+tTgMNK1HyVVz2Qrms29wYvDkMRNyYCNUlDxyQY/noskzSyySq4AlJLkPgjgnu6EPXduHe4qXlovIHZagBMyu/W/xJNAXtkoydtsyFJBVTyQU/k+Ar8AeNWJqCFoQxlwfqEKuvm04hju88lg3t+b2G0GAzyZkkbl8PUOHs55BGZtDwntDEPewi3GjD72J4eG19rulGXwo9MTx6lyvvcxf6DjUHg3ToJBe0pa/nR4x8/rnj/S4KQyiopF4Sx4qY9Mg1KmaESkSNlezbw2fPb+IilTkvaaslvtBFufFv/g4IKdKPHcsSfOdW4RvZ2Z3lMnl8r4eSh0goS4SKsqqygqEvuLX/26Y5YQAiErkF7FLPyxwhb5o4q7JzLAQR6TYmShOQjJguRoFc+AdKqthyChN7UoTXbZ8Pih0Xi725JM80/5V+n1VeR1RBGTwv2BEieRYgR49gDq2+wVSLdvQcWc5uIVKtO8YErrAKnKsiHMACKC6yKEr87pVoyRjouvwDE63AUEwRhyMDoJTMYSelFp/YniULuvMJeB9YVFn7WE3Y+XzAmw7VX8+A4mgBwPHITzpMtYubogck238Plt5pFHIURD/+uWwFx1R1K9oHEx2LZN7uO0EGrHbqR/ioRaxtWmFoFKOWwc/eYx126bEvhCaVOAuIH8IYJsnNAvMkYAGPCddRjxX82bjqz++TDYtHcDwFkbsPVWscrtRx7MHo7LDhmXJf0h8xdAM482RZ9Q9/eCTaeo7PSnqnWAdcQc0xevShI+NgM8y2CzcRxrP87ly0YCan9ybHppyjsD/zGQkb2QqoMX8AyCxeKpnoE6xoLEFUPXqCNz7UcESCqfSq0R5Y8nto5qsWJvCCWMbK0tmIeQ1 xpk52IFI WtekHtMdPwbOoF3MkurZIfSyTmvSlNThyisZYFkp46w65CsP/3O/gokKNNOdBmzSysdmYc0TIGTMHooNABIlIrIfFhanNM9JlfNTFtrUYeJqxXh0hhAi43RNbfgpuZQEeHES5gVUofet0CQTjWs0V1p11HSb+f5gE5PCYP9a7Na3TcERcSYvPrhJvHC2KF09fl0H22+nu0kqeBHu9P8Dt7btXJ65I5k8aBQ2LmZf7I0eWswMnFGHdn1S5o72bYXbeiexm6zPhSLidYAyqnVC2uo90reDKVThYhjWx2WXBsY9TfC4DNt/+mTVo7il3e/FOleMhd9FC/bGw2o2voNoqaHN0sRRQ0cMTqgDkFCvYyQ/mOzx+mutsR+jVIgHaragEKHrMIfjwzBYmn0jgj9Gcz25oc9EdMBU31juYInyviVk+xVacd73lEWmjgzUpbUrUQIaJBMN35TTrXOKrhJSRCS61yA+iyH1z9nVsxmseQbCe/B8h0n7VoMEqI8CHNPofQ+7Bzt6F0AGLKRTuz3ijf7oYWFsBz5hK15bxn7IDh/ujsfr46Bg7rN7KWgADly+Y9yWfSiFXGtGy9MqfAr+1XzSDXWfg92mXDVvqGac8Cq+LD3p0/OVw6Xw2ow== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Punit Agrawal writes: > Suren Baghdasaryan writes: > >> Previous version: >> v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.c= om/ >> RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.c= om/ >> >> LWN article describing the feature: >> https://lwn.net/Articles/906852/ >> >> Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM >> last year [2], which concluded with suggestion that =E2=80=9Ca reader/wr= iter >> semaphore could be put into the VMA itself; that would have the effect of >> using the VMA as a sort of range lock. There would still be contention at >> the VMA level, but it would be an improvement.=E2=80=9D This patchset im= plements >> this suggested approach. > > I took the patches for a spin on a 2-socket 32 core (64 threads) system > with Intel 8336C (Ice Lake) and 512GB of RAM. > > For the initial testing, "pft-threads" from the mm-tests suite[0] was > used. The test mmaps a memory region (~100GB on the test system) and > triggers access by a number of threads executing in parallel. For each > degree of parallelism, the test is repeated 10 times to get a better > feel for the behaviour. Below is an excerpt of the harmonic mean > reported by 'compare_kernel' script[1] included with mm-tests. > > The first column is results for mm-unstable as of 2023-02-10, the second > column is the patches posted here while the third column includes > optimizations to reclaim some of the observed regression. > > From the results, there is a drop in page fault/second for low number of > CPUs but good improvement with higher CPUs. > > 6.2.0-rc4 6.2.0-rc= 4 6.2.0-rc4 > mm-unstable-20230210 pvl-v= 2 pvl-v2+opt > > Hmean faults/cpu-1 898792.9338 ( 0.00%) 894597.0474 * -0.47%= * 895933.2782 * -0.32%* > Hmean faults/cpu-4 751903.9803 ( 0.00%) 677764.2975 * -9.86%= * 688643.8163 * -8.41%* > Hmean faults/cpu-7 612275.5663 ( 0.00%) 565363.4137 * -7.66%= * 597538.9396 * -2.41%* > Hmean faults/cpu-12 434460.9074 ( 0.00%) 410974.2708 * -5.41%= * 452501.4290 * 4.15%* > Hmean faults/cpu-21 291475.5165 ( 0.00%) 293936.8460 ( 0.84%= ) 308712.2434 * 5.91%* > Hmean faults/cpu-30 218021.3980 ( 0.00%) 228265.0559 * 4.70%= * 241897.5225 * 10.95%* > Hmean faults/cpu-48 141798.5030 ( 0.00%) 162322.5972 * 14.47%= * 166081.9459 * 17.13%* > Hmean faults/cpu-79 90060.9577 ( 0.00%) 107028.7779 * 18.84%= * 109810.4488 * 21.93%* > Hmean faults/cpu-110 64729.3561 ( 0.00%) 80597.7246 * 24.51%= * 83134.0679 * 28.43%* > Hmean faults/cpu-128 55740.1334 ( 0.00%) 68395.4426 * 22.70%= * 69248.2836 * 24.23%* > > Hmean faults/sec-1 898781.7694 ( 0.00%) 894247.3174 * -0.50%= * 894440.3118 * -0.48%* > Hmean faults/sec-4 2965588.9697 ( 0.00%) 2683651.5664 * -9.51%= * 2726450.9710 * -8.06%* > Hmean faults/sec-7 4144512.3996 ( 0.00%) 3891644.2128 * -6.10%= * 4099918.8601 ( -1.08%) > Hmean faults/sec-12 4969513.6934 ( 0.00%) 4829731.4355 * -2.81%= * 5264682.7371 * 5.94%* > Hmean faults/sec-21 5814379.4789 ( 0.00%) 5941405.3116 * 2.18%= * 6263716.3903 * 7.73%* > Hmean faults/sec-30 6153685.3709 ( 0.00%) 6489311.6634 * 5.45%= * 6910843.5858 * 12.30%* > Hmean faults/sec-48 6197953.1327 ( 0.00%) 7216320.7727 * 16.43%= * 7412782.2927 * 19.60%* > Hmean faults/sec-79 6167135.3738 ( 0.00%) 7425927.1022 * 20.41%= * 7637042.2198 * 23.83%* > Hmean faults/sec-110 6264768.2247 ( 0.00%) 7813329.3863 * 24.72%= * 7984344.4005 * 27.45%* > Hmean faults/sec-128 6460727.8216 ( 0.00%) 7875664.8999 * 21.90%= * 8049910.3601 * 24.60%* The above workload represent the worst case with regards to per-VMA locks as it creates a single large VMA. As a follow-up, I modified pft[2] to create a VMA per thread to understand the behaviour in scenarios where per-VMA locks should show the most benefit. 6.2.0-rc4 6.2.0-rc4 = 6.2.0-rc4 mm-unstable-20230210 pvl-v2 = pvl-v2+opt Hmean faults/cpu-1 905497.4354 ( 0.00%) 888736.5570 * -1.85%* = 888695.2675 * -1.86%* Hmean faults/cpu-4 758519.2719 ( 0.00%) 812103.1991 * 7.06%* = 825077.9277 * 8.77%* Hmean faults/cpu-7 617153.8038 ( 0.00%) 729943.4518 * 18.28%* = 770872.3161 * 24.91%* Hmean faults/cpu-12 424848.5266 ( 0.00%) 550357.2856 * 29.54%* = 597478.5634 * 40.63%* Hmean faults/cpu-21 290142.9988 ( 0.00%) 383668.3190 * 32.23%* = 433376.8959 * 49.37%* Hmean faults/cpu-30 218705.2915 ( 0.00%) 299888.5533 * 37.12%* = 342640.6153 * 56.67%* Hmean faults/cpu-48 142842.3372 ( 0.00%) 206498.2605 * 44.56%* = 240306.3442 * 68.23%* Hmean faults/cpu-79 90706.1425 ( 0.00%) 160006.6800 * 76.40%* = 185298.4326 * 104.28%* Hmean faults/cpu-110 67011.9297 ( 0.00%) 143536.0062 * 114.19%* = 162688.8015 * 142.78%* Hmean faults/cpu-128 55986.4986 ( 0.00%) 136550.8760 * 143.90%* = 152718.8713 * 172.78%* Hmean faults/sec-1 905492.1265 ( 0.00%) 887244.6592 * -2.02%* = 887775.6079 * -1.96%* Hmean faults/sec-4 2994284.4204 ( 0.00%) 3154236.9408 * 5.34%* = 3221994.8465 * 7.60%* Hmean faults/sec-7 4177411.3461 ( 0.00%) 4933286.4045 * 18.09%* = 5202347.2077 * 24.54%* Hmean faults/sec-12 4892848.3633 ( 0.00%) 6054577.0988 * 23.74%* = 6511987.1142 * 33.09%* Hmean faults/sec-21 5823534.1820 ( 0.00%) 7637637.4162 * 31.15%* = 8553362.3513 * 46.88%* Hmean faults/sec-30 6247210.8414 ( 0.00%) 8598150.6717 * 37.63%* = 9799696.0945 * 56.87%* Hmean faults/sec-48 6274617.1419 ( 0.00%) 9467132.3699 * 50.88%* = 11049401.9072 * 76.10%* Hmean faults/sec-79 6187291.4971 ( 0.00%) 11919062.5284 * 92.64%* = 13420825.3820 * 116.91%* Hmean faults/sec-110 6454542.3239 ( 0.00%) 15050228.1869 * 133.17%* = 16667873.7618 * 158.23%* Hmean faults/sec-128 6472970.8548 ( 0.00%) 16647275.6575 * 157.18%* = 18680029.3714 * 188.59%* As expected, the tests highlight the improved scalability as core count increases. > [0] https://github.com/gormanm/mmtests > [1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh [2] https://github.com/gormanm/pft/pull/1/commits/8fe554a3d8b4f5947cd00d4b4= 6f97178b8ba8752