From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFA06C636D7 for ; Wed, 15 Feb 2023 17:34:05 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4PH4v020Jlz3f32 for ; Thu, 16 Feb 2023 04:34:04 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=bytedance-com.20210112.gappssmtp.com header.i=@bytedance-com.20210112.gappssmtp.com header.a=rsa-sha256 header.s=20210112 header.b=wTYzp+HL; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2a00:1450:4864:20::42f; helo=mail-wr1-x42f.google.com; envelope-from=punit.agrawal@bytedance.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance-com.20210112.gappssmtp.com header.i=@bytedance-com.20210112.gappssmtp.com header.a=rsa-sha256 header.s=20210112 header.b=wTYzp+HL; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4PH4sw28rFz3cd2 for ; Thu, 16 Feb 2023 04:33:06 +1100 (AEDT) Received: by mail-wr1-x42f.google.com with SMTP id y1so19961869wru.2 for ; Wed, 15 Feb 2023 09:33:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=wTYzp+HLVi+lx88ricSM/4zA8JGJ0VYAUK8sP1YwmHt5MGHWabBceqwlwxcEPxzMvc magXOb6suv1DXlOStB5PZvc2zBV9hlnvnH4H50+kYlzyuV0Uu06GW3ZzJsNjP1wjmEm/ /g6mWpsZrj7fTLUZSrzvfBri7ZHgBO27wAq3dbmuY1x4ZZ0PkgpMBAjk0VwNa34aASAE A6rGzpy27hnSGLZSLkcS6pi1DiFjFqnQuJ3Rc47jhXlM2nYrZ2UnOXk5csTd2qgzNdtr 43SrIB3xEnsd/MLqrCKN/npLj7g40d4R1m+2AC2+7QkNOt17ZlxEe1FozUu7SHAncT2A rrYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=tuAXYk6LcC6p/DdWGgZ67bPGVOqxERr+m2fg8SRF/eeQGr544bXSke/i0+9xc5n73g L+DmKm4gv5QeyfAMpQV+3dIKMLeHRKXqGVjgLBDeGAJkl4f8bQE3ySJ8ULpKDgIGbZrV Ll0Xd7ckB+pJwvIUpB2DS2YVuSE0jYIf3FdN53vlggb6Uiiv3yb5aDirBEqbsu0PMAPk 4wueHRd5dw1KlbKeLGngazjcURbPP70maw87H7tFj4uZkYZ9MJMxgnH3Z76wwXah7XA7 lzcVVwgKCSEtEgGvgoMYdojkAMsFUFlDDm8z8N40+kZaAbbeWVqBkTqZi/17ly+2j1Ap 1TTA== X-Gm-Message-State: AO0yUKXGIozr2J4hrXI9AUIj8yCpy1bb4qSZg9iI0eupDQq+3L2qwVIP 5asunJdEZ1YiBLKvfKYp2kqM6w== X-Google-Smtp-Source: AK7set9kT9H6NMj2/S3WBxDWHmAmNxq0wUR9ec7mDgnR1zjNhxV91JrWyeNEVDzBJm76EQ32XvPpCg== X-Received: by 2002:a5d:4b03:0:b0:2c5:644c:3e89 with SMTP id v3-20020a5d4b03000000b002c5644c3e89mr2357955wrq.67.1676482380110; Wed, 15 Feb 2023 09:33:00 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id f26-20020a5d58fa000000b002c5526680cbsm10288128wrd.92.2023.02.15.09.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 09:32:59 -0800 (PST) From: Punit Agrawal To: Suren Baghdasaryan Subject: Re: [External] [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> Date: Wed, 15 Feb 2023 17:32:58 +0000 In-Reply-To: <20230127194110.533103-1-surenb@google.com> (Suren Baghdasaryan's message of "Fri, 27 Jan 2023 11:40:37 -0800") Message-ID: <87r0uqq1f9.fsf@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: michel@lespinasse.org, joelaf@google.com, songliubraving@fb.com, mhocko@suse.com, leewalsh@google.com, david@redhat.com, peterz@infradead.org, bigeasy@linutronix.de, peterx@redhat.com, dhowells@redhat.com, linux-mm@kvack.org, edumazet@google.com, jglisse@google.com, punit.agrawal@bytedance.com, will@kernel.org, arjunroy@google.com, dave@stgolabs.net, minchan@google.com, x86@kernel.org, hughd@google.com, willy@infradead.org, gurua@google.com, mingo@redhat.com, linux-arm-kernel@lists.infradead.org, rientjes@google.com, axelrasmussen@google.com, kernel-team@android.com, soheil@google.com, paulmck@kernel.org, jannh@google.com, liam.howlett@oracle.com, shakeelb@google.com, luto@kernel.org, gthelen@google.com, ldufour@linux.ibm.com, vbabka@suse.cz, posk@google.com, lstoakes@gmail.com, peterjung1337@gmail.com, linuxppc-dev@lists.ozlabs.org, kent.overstreet@linux.dev, linux-kernel@vger.kernel.org, hannes@cmpxchg.org, akpm@linux-foundation.org, tatashin@google.com, mgorman@techsingularity .net, rppt@kernel.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Suren Baghdasaryan writes: > Previous version: > v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.co= m/ > RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.co= m/ > > LWN article describing the feature: > https://lwn.net/Articles/906852/ > > Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM > last year [2], which concluded with suggestion that =E2=80=9Ca reader/wri= ter > semaphore could be put into the VMA itself; that would have the effect of > using the VMA as a sort of range lock. There would still be contention at > the VMA level, but it would be an improvement.=E2=80=9D This patchset imp= lements > this suggested approach. I took the patches for a spin on a 2-socket 32 core (64 threads) system with Intel 8336C (Ice Lake) and 512GB of RAM. For the initial testing, "pft-threads" from the mm-tests suite[0] was used. The test mmaps a memory region (~100GB on the test system) and triggers access by a number of threads executing in parallel. For each degree of parallelism, the test is repeated 10 times to get a better feel for the behaviour. Below is an excerpt of the harmonic mean reported by 'compare_kernel' script[1] included with mm-tests. The first column is results for mm-unstable as of 2023-02-10, the second column is the patches posted here while the third column includes optimizations to reclaim some of the observed regression. >From the results, there is a drop in page fault/second for low number of CPUs but good improvement with higher CPUs. 6.2.0-rc4 6.2.0-rc4 = 6.2.0-rc4 mm-unstable-20230210 pvl-v2 = pvl-v2+opt Hmean faults/cpu-1 898792.9338 ( 0.00%) 894597.0474 * -0.47%* = 895933.2782 * -0.32%* Hmean faults/cpu-4 751903.9803 ( 0.00%) 677764.2975 * -9.86%* = 688643.8163 * -8.41%* Hmean faults/cpu-7 612275.5663 ( 0.00%) 565363.4137 * -7.66%* = 597538.9396 * -2.41%* Hmean faults/cpu-12 434460.9074 ( 0.00%) 410974.2708 * -5.41%* = 452501.4290 * 4.15%* Hmean faults/cpu-21 291475.5165 ( 0.00%) 293936.8460 ( 0.84%) = 308712.2434 * 5.91%* Hmean faults/cpu-30 218021.3980 ( 0.00%) 228265.0559 * 4.70%* = 241897.5225 * 10.95%* Hmean faults/cpu-48 141798.5030 ( 0.00%) 162322.5972 * 14.47%* = 166081.9459 * 17.13%* Hmean faults/cpu-79 90060.9577 ( 0.00%) 107028.7779 * 18.84%* = 109810.4488 * 21.93%* Hmean faults/cpu-110 64729.3561 ( 0.00%) 80597.7246 * 24.51%* = 83134.0679 * 28.43%* Hmean faults/cpu-128 55740.1334 ( 0.00%) 68395.4426 * 22.70%* = 69248.2836 * 24.23%* Hmean faults/sec-1 898781.7694 ( 0.00%) 894247.3174 * -0.50%* = 894440.3118 * -0.48%* Hmean faults/sec-4 2965588.9697 ( 0.00%) 2683651.5664 * -9.51%* = 2726450.9710 * -8.06%* Hmean faults/sec-7 4144512.3996 ( 0.00%) 3891644.2128 * -6.10%* = 4099918.8601 ( -1.08%) Hmean faults/sec-12 4969513.6934 ( 0.00%) 4829731.4355 * -2.81%* = 5264682.7371 * 5.94%* Hmean faults/sec-21 5814379.4789 ( 0.00%) 5941405.3116 * 2.18%* = 6263716.3903 * 7.73%* Hmean faults/sec-30 6153685.3709 ( 0.00%) 6489311.6634 * 5.45%* = 6910843.5858 * 12.30%* Hmean faults/sec-48 6197953.1327 ( 0.00%) 7216320.7727 * 16.43%* = 7412782.2927 * 19.60%* Hmean faults/sec-79 6167135.3738 ( 0.00%) 7425927.1022 * 20.41%* = 7637042.2198 * 23.83%* Hmean faults/sec-110 6264768.2247 ( 0.00%) 7813329.3863 * 24.72%* = 7984344.4005 * 27.45%* Hmean faults/sec-128 6460727.8216 ( 0.00%) 7875664.8999 * 21.90%* = 8049910.3601 * 24.60%* [0] https://github.com/gormanm/mmtests [1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5FA38C636D4 for ; Wed, 15 Feb 2023 17:34:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:In-Reply-To: Date:References:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6Z7ep0nfthGyT34ILpmbebQ8kuuTjsgUtv+UbZtZOxI=; b=QK64TVXlvNHVq6 vYtokFfAJKBXPVSScCW7FidzC1RqUhh/HYMDtMycsp52mzoGrYsss4wMRccLTo9yuw/wJnByJynvn fjJo62+Rqtbjy5p2u2lyWO81QEndHrJS2inoCoBlPAZpAnBH8dmbMolTvMMB0NdxLFugpwTBm7DvX K7hSbCK0T1FPQ8pU0bJA/x//VNIaBveMi3JsPYjFCd/CVn+tb5V3rOumnMTSuRZpPbZ5z7xbske9d 0t6PiOS7wSdNQd8XQjKeqllt36/mm/Kb+Rjv7jwFYENEARw+gfpPYL3cpvHBu5i+kMLkSSDoWLmzW D0tKMoMXnOJZfHBTFjNQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSLec-006hbB-Bn; Wed, 15 Feb 2023 17:33:14 +0000 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSLeW-006hY6-Ug for linux-arm-kernel@lists.infradead.org; Wed, 15 Feb 2023 17:33:12 +0000 Received: by mail-wr1-x42e.google.com with SMTP id bk16so19925191wrb.11 for ; Wed, 15 Feb 2023 09:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=wTYzp+HLVi+lx88ricSM/4zA8JGJ0VYAUK8sP1YwmHt5MGHWabBceqwlwxcEPxzMvc magXOb6suv1DXlOStB5PZvc2zBV9hlnvnH4H50+kYlzyuV0Uu06GW3ZzJsNjP1wjmEm/ /g6mWpsZrj7fTLUZSrzvfBri7ZHgBO27wAq3dbmuY1x4ZZ0PkgpMBAjk0VwNa34aASAE A6rGzpy27hnSGLZSLkcS6pi1DiFjFqnQuJ3Rc47jhXlM2nYrZ2UnOXk5csTd2qgzNdtr 43SrIB3xEnsd/MLqrCKN/npLj7g40d4R1m+2AC2+7QkNOt17ZlxEe1FozUu7SHAncT2A rrYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=tJBhaEozuuLEyfw+A/AuJbf46j/gb7oZcUwpcajGjLLada/rD1/fjXii0NQ/Ls8Kcz 80b8sXzahrwGHAqfUjjBcwn7zjP448pDlzE8iPwxyms8JJfKKW6nxczXnIZc5V6G2g8A sjByDJWwrz2m3JfdCIzlf72DcKx9S4hwtymEIrCnsN15OzJZkunhF9if2uh064jouH+e rQ0hT94Gtae/zSFjot8R/SQ5MvbFla9LVSett0CnvwH1/+a4yLtqTsEFeitlGy0FBptp wnxxVrTp38XPgXJG1v6KlqolDv5EWYfW886Lza8363eiI+DSAp7phJ71na0PHlt6ZyLs 7ugA== X-Gm-Message-State: AO0yUKU+0RpTCI/hYQzL+6/+UNczHiP1M0BbJQQ19OHLkrMHwkClw98M YqQDARkaHS9f0C7vtwRLGAQUiQ== X-Google-Smtp-Source: AK7set9kT9H6NMj2/S3WBxDWHmAmNxq0wUR9ec7mDgnR1zjNhxV91JrWyeNEVDzBJm76EQ32XvPpCg== X-Received: by 2002:a5d:4b03:0:b0:2c5:644c:3e89 with SMTP id v3-20020a5d4b03000000b002c5644c3e89mr2357955wrq.67.1676482380110; Wed, 15 Feb 2023 09:33:00 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id f26-20020a5d58fa000000b002c5526680cbsm10288128wrd.92.2023.02.15.09.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 09:32:59 -0800 (PST) From: Punit Agrawal To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [External] [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> Date: Wed, 15 Feb 2023 17:32:58 +0000 In-Reply-To: <20230127194110.533103-1-surenb@google.com> (Suren Baghdasaryan's message of "Fri, 27 Jan 2023 11:40:37 -0800") Message-ID: <87r0uqq1f9.fsf@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230215_093309_011024_4319D412 X-CRM114-Status: GOOD ( 13.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org U3VyZW4gQmFnaGRhc2FyeWFuIDxzdXJlbmJAZ29vZ2xlLmNvbT4gd3JpdGVzOgoKPiBQcmV2aW91 cyB2ZXJzaW9uOgo+IHYxOiBodHRwczovL2xvcmUua2VybmVsLm9yZy9hbGwvMjAyMzAxMDkyMDUz MzYuMzY2NTkzNy0xLXN1cmVuYkBnb29nbGUuY29tLwo+IFJGQzogaHR0cHM6Ly9sb3JlLmtlcm5l bC5vcmcvYWxsLzIwMjIwOTAxMTczNTE2LjcwMjEyMi0xLXN1cmVuYkBnb29nbGUuY29tLwo+Cj4g TFdOIGFydGljbGUgZGVzY3JpYmluZyB0aGUgZmVhdHVyZToKPiBodHRwczovL2x3bi5uZXQvQXJ0 aWNsZXMvOTA2ODUyLwo+Cj4gUGVyLXZtYSBsb2NrcyBpZGVhIHRoYXQgd2FzIGRpc2N1c3NlZCBk dXJpbmcgU1BGIFsxXSBkaXNjdXNzaW9uIGF0IExTRi9NTQo+IGxhc3QgeWVhciBbMl0sIHdoaWNo IGNvbmNsdWRlZCB3aXRoIHN1Z2dlc3Rpb24gdGhhdCDigJxhIHJlYWRlci93cml0ZXIKPiBzZW1h cGhvcmUgY291bGQgYmUgcHV0IGludG8gdGhlIFZNQSBpdHNlbGY7IHRoYXQgd291bGQgaGF2ZSB0 aGUgZWZmZWN0IG9mCj4gdXNpbmcgdGhlIFZNQSBhcyBhIHNvcnQgb2YgcmFuZ2UgbG9jay4gVGhl cmUgd291bGQgc3RpbGwgYmUgY29udGVudGlvbiBhdAo+IHRoZSBWTUEgbGV2ZWwsIGJ1dCBpdCB3 b3VsZCBiZSBhbiBpbXByb3ZlbWVudC7igJ0gVGhpcyBwYXRjaHNldCBpbXBsZW1lbnRzCj4gdGhp cyBzdWdnZXN0ZWQgYXBwcm9hY2guCgpJIHRvb2sgdGhlIHBhdGNoZXMgZm9yIGEgc3BpbiBvbiBh IDItc29ja2V0IDMyIGNvcmUgKDY0IHRocmVhZHMpIHN5c3RlbQp3aXRoIEludGVsIDgzMzZDIChJ Y2UgTGFrZSkgYW5kIDUxMkdCIG9mIFJBTS4KCkZvciB0aGUgaW5pdGlhbCB0ZXN0aW5nLCAicGZ0 LXRocmVhZHMiIGZyb20gdGhlIG1tLXRlc3RzIHN1aXRlWzBdIHdhcwp1c2VkLiBUaGUgdGVzdCBt bWFwcyBhIG1lbW9yeSByZWdpb24gKH4xMDBHQiBvbiB0aGUgdGVzdCBzeXN0ZW0pIGFuZAp0cmln Z2VycyBhY2Nlc3MgYnkgYSBudW1iZXIgb2YgdGhyZWFkcyBleGVjdXRpbmcgaW4gcGFyYWxsZWwu IEZvciBlYWNoCmRlZ3JlZSBvZiBwYXJhbGxlbGlzbSwgdGhlIHRlc3QgaXMgcmVwZWF0ZWQgMTAg dGltZXMgdG8gZ2V0IGEgYmV0dGVyCmZlZWwgZm9yIHRoZSBiZWhhdmlvdXIuIEJlbG93IGlzIGFu IGV4Y2VycHQgb2YgdGhlIGhhcm1vbmljIG1lYW4KcmVwb3J0ZWQgYnkgJ2NvbXBhcmVfa2VybmVs JyBzY3JpcHRbMV0gaW5jbHVkZWQgd2l0aCBtbS10ZXN0cy4KClRoZSBmaXJzdCBjb2x1bW4gaXMg cmVzdWx0cyBmb3IgbW0tdW5zdGFibGUgYXMgb2YgMjAyMy0wMi0xMCwgdGhlIHNlY29uZApjb2x1 bW4gaXMgdGhlIHBhdGNoZXMgcG9zdGVkIGhlcmUgd2hpbGUgdGhlIHRoaXJkIGNvbHVtbiBpbmNs dWRlcwpvcHRpbWl6YXRpb25zIHRvIHJlY2xhaW0gc29tZSBvZiB0aGUgb2JzZXJ2ZWQgcmVncmVz c2lvbi4KCkZyb20gdGhlIHJlc3VsdHMsIHRoZXJlIGlzIGEgZHJvcCBpbiBwYWdlIGZhdWx0L3Nl Y29uZCBmb3IgbG93IG51bWJlciBvZgpDUFVzIGJ1dCBnb29kIGltcHJvdmVtZW50IHdpdGggaGln aGVyIENQVXMuCgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgNi4yLjAt cmM0ICAgICAgICAgICAgICAgIDYuMi4wLXJjNCAgICAgICAgICAgICAgICA2LjIuMC1yYzQKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBtbS11bnN0YWJsZS0yMDIzMDIxMCAgICAgICAgICAg ICAgICAgICBwdmwtdjIgICAgICAgICAgICAgICBwdmwtdjIrb3B0CgpIbWVhbiAgICAgZmF1bHRz L2NwdS0xICAgICA4OTg3OTIuOTMzOCAoICAgMC4wMCUpICAgODk0NTk3LjA0NzQgKiAgLTAuNDcl KiAgIDg5NTkzMy4yNzgyICogIC0wLjMyJSoKSG1lYW4gICAgIGZhdWx0cy9jcHUtNCAgICAgNzUx OTAzLjk4MDMgKCAgIDAuMDAlKSAgIDY3Nzc2NC4yOTc1ICogIC05Ljg2JSogICA2ODg2NDMuODE2 MyAqICAtOC40MSUqCkhtZWFuICAgICBmYXVsdHMvY3B1LTcgICAgIDYxMjI3NS41NjYzICggICAw LjAwJSkgICA1NjUzNjMuNDEzNyAqICAtNy42NiUqICAgNTk3NTM4LjkzOTYgKiAgLTIuNDElKgpI bWVhbiAgICAgZmF1bHRzL2NwdS0xMiAgICA0MzQ0NjAuOTA3NCAoICAgMC4wMCUpICAgNDEwOTc0 LjI3MDggKiAgLTUuNDElKiAgIDQ1MjUwMS40MjkwICogICA0LjE1JSoKSG1lYW4gICAgIGZhdWx0 cy9jcHUtMjEgICAgMjkxNDc1LjUxNjUgKCAgIDAuMDAlKSAgIDI5MzkzNi44NDYwICggICAwLjg0 JSkgICAzMDg3MTIuMjQzNCAqICAgNS45MSUqCkhtZWFuICAgICBmYXVsdHMvY3B1LTMwICAgIDIx ODAyMS4zOTgwICggICAwLjAwJSkgICAyMjgyNjUuMDU1OSAqICAgNC43MCUqICAgMjQxODk3LjUy MjUgKiAgMTAuOTUlKgpIbWVhbiAgICAgZmF1bHRzL2NwdS00OCAgICAxNDE3OTguNTAzMCAoICAg MC4wMCUpICAgMTYyMzIyLjU5NzIgKiAgMTQuNDclKiAgIDE2NjA4MS45NDU5ICogIDE3LjEzJSoK SG1lYW4gICAgIGZhdWx0cy9jcHUtNzkgICAgIDkwMDYwLjk1NzcgKCAgIDAuMDAlKSAgIDEwNzAy OC43Nzc5ICogIDE4Ljg0JSogICAxMDk4MTAuNDQ4OCAqICAyMS45MyUqCkhtZWFuICAgICBmYXVs dHMvY3B1LTExMCAgICA2NDcyOS4zNTYxICggICAwLjAwJSkgICAgODA1OTcuNzI0NiAqICAyNC41 MSUqICAgIDgzMTM0LjA2NzkgKiAgMjguNDMlKgpIbWVhbiAgICAgZmF1bHRzL2NwdS0xMjggICAg NTU3NDAuMTMzNCAoICAgMC4wMCUpICAgIDY4Mzk1LjQ0MjYgKiAgMjIuNzAlKiAgICA2OTI0OC4y ODM2ICogIDI0LjIzJSoKCkhtZWFuICAgICBmYXVsdHMvc2VjLTEgICAgIDg5ODc4MS43Njk0ICgg ICAwLjAwJSkgICA4OTQyNDcuMzE3NCAqICAtMC41MCUqICAgODk0NDQwLjMxMTggKiAgLTAuNDgl KgpIbWVhbiAgICAgZmF1bHRzL3NlYy00ICAgIDI5NjU1ODguOTY5NyAoICAgMC4wMCUpICAyNjgz NjUxLjU2NjQgKiAgLTkuNTElKiAgMjcyNjQ1MC45NzEwICogIC04LjA2JSoKSG1lYW4gICAgIGZh dWx0cy9zZWMtNyAgICA0MTQ0NTEyLjM5OTYgKCAgIDAuMDAlKSAgMzg5MTY0NC4yMTI4ICogIC02 LjEwJSogIDQwOTk5MTguODYwMSAoICAtMS4wOCUpCkhtZWFuICAgICBmYXVsdHMvc2VjLTEyICAg NDk2OTUxMy42OTM0ICggICAwLjAwJSkgIDQ4Mjk3MzEuNDM1NSAqICAtMi44MSUqICA1MjY0Njgy LjczNzEgKiAgIDUuOTQlKgpIbWVhbiAgICAgZmF1bHRzL3NlYy0yMSAgIDU4MTQzNzkuNDc4OSAo ICAgMC4wMCUpICA1OTQxNDA1LjMxMTYgKiAgIDIuMTglKiAgNjI2MzcxNi4zOTAzICogICA3Ljcz JSoKSG1lYW4gICAgIGZhdWx0cy9zZWMtMzAgICA2MTUzNjg1LjM3MDkgKCAgIDAuMDAlKSAgNjQ4 OTMxMS42NjM0ICogICA1LjQ1JSogIDY5MTA4NDMuNTg1OCAqICAxMi4zMCUqCkhtZWFuICAgICBm YXVsdHMvc2VjLTQ4ICAgNjE5Nzk1My4xMzI3ICggICAwLjAwJSkgIDcyMTYzMjAuNzcyNyAqICAx Ni40MyUqICA3NDEyNzgyLjI5MjcgKiAgMTkuNjAlKgpIbWVhbiAgICAgZmF1bHRzL3NlYy03OSAg IDYxNjcxMzUuMzczOCAoICAgMC4wMCUpICA3NDI1OTI3LjEwMjIgKiAgMjAuNDElKiAgNzYzNzA0 Mi4yMTk4ICogIDIzLjgzJSoKSG1lYW4gICAgIGZhdWx0cy9zZWMtMTEwICA2MjY0NzY4LjIyNDcg KCAgIDAuMDAlKSAgNzgxMzMyOS4zODYzICogIDI0LjcyJSogIDc5ODQzNDQuNDAwNSAqICAyNy40 NSUqCkhtZWFuICAgICBmYXVsdHMvc2VjLTEyOCAgNjQ2MDcyNy44MjE2ICggICAwLjAwJSkgIDc4 NzU2NjQuODk5OSAqICAyMS45MCUqICA4MDQ5OTEwLjM2MDEgKiAgMjQuNjAlKgoKWzBdIGh0dHBz Oi8vZ2l0aHViLmNvbS9nb3JtYW5tL21tdGVzdHMKWzFdIGh0dHBzOi8vZ2l0aHViLmNvbS9nb3Jt YW5tL21tdGVzdHMvYmxvYi9tYXN0ZXIvY29tcGFyZS1rZXJuZWxzLnNoCgpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxp bmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3Rz LmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3EA1C636CC for ; Wed, 15 Feb 2023 17:33:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68BEA6B0074; Wed, 15 Feb 2023 12:33:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 614536B0075; Wed, 15 Feb 2023 12:33:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48F2B6B0078; Wed, 15 Feb 2023 12:33:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3674B6B0074 for ; Wed, 15 Feb 2023 12:33:06 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 03834A7489 for ; Wed, 15 Feb 2023 17:33:05 +0000 (UTC) X-FDA: 80470221972.17.7A9760F Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by imf19.hostedemail.com (Postfix) with ESMTP id 024661A0002 for ; Wed, 15 Feb 2023 17:33:01 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=wTYzp+HL; spf=pass (imf19.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676482383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=F4dFsWZFAOTLnpOUzobeX8Kp+9koFTLXf+ogjVZW7fDujdW8V8uHWecfYcItbYZL5eyWQR CB6HwSvGZf3fB37ebEf6xcglwU3TtYyTn3wM3ycOLNsI+1EiOmlthMW8Bgv380Hj8z81Nr 3sKh3WvXG/jFGxECso+rjk3+bsdJMTU= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=wTYzp+HL; spf=pass (imf19.hostedemail.com: domain of punit.agrawal@bytedance.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=punit.agrawal@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676482383; a=rsa-sha256; cv=none; b=gOjSVR/M3DyW7WXpDT+N6JkETNsw9MP16aaVO3tZTyWf4i8Vma3MUJVczhS2TEZdbLtQQp OX9j/SCzUsBtZWPw88A3HyJ6NP8KZALio1kUeo9pro1JBdjQ4T9z5gDVEdDko07bFBDSIq fxIj+yHXn1TOklEuzxM+FaVL5vB0YII= Received: by mail-wr1-f54.google.com with SMTP id l2so3312714wry.0 for ; Wed, 15 Feb 2023 09:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=wTYzp+HLVi+lx88ricSM/4zA8JGJ0VYAUK8sP1YwmHt5MGHWabBceqwlwxcEPxzMvc magXOb6suv1DXlOStB5PZvc2zBV9hlnvnH4H50+kYlzyuV0Uu06GW3ZzJsNjP1wjmEm/ /g6mWpsZrj7fTLUZSrzvfBri7ZHgBO27wAq3dbmuY1x4ZZ0PkgpMBAjk0VwNa34aASAE A6rGzpy27hnSGLZSLkcS6pi1DiFjFqnQuJ3Rc47jhXlM2nYrZ2UnOXk5csTd2qgzNdtr 43SrIB3xEnsd/MLqrCKN/npLj7g40d4R1m+2AC2+7QkNOt17ZlxEe1FozUu7SHAncT2A rrYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=0t94xwbIE7oAhUJFQ2BQHFhw0oZE0qnNECyAb2ZFKHQ=; b=Eu8NvPTlU8RZunFA+UlpwsoyWLDCGgXic94yXG86mIZd6rJnWcX3X3bUonoLRaX3rE 5jm7j2r+RVsy3SBtN3kKpz1FYUX/E2tOIgfsBmwglOO2fhRQeNfM7PwiZt56pLXUyzTG Kr0tr41JxKwMIw9aG2vJc5BpmeXdwBLZD2m6O1R/h+uFV61hY0G22AUNW6n3jBDrP3Dl B6y6xtFWsVyfFB88m1iC0acKiP0b8NPCPsDJ8IMQ6yC0G8zIXn7fzFI3ms2zrvXI+6qu ssIfpGtnUVZrvTMWJ1ITMfilQYBsZK/C0mBBfwghcIEEfmV8afBqrc/71GqY8XKlRayH VQUw== X-Gm-Message-State: AO0yUKXM2TJa2KSQQ783y7x8FxArQOvfjLMNZRV/Y0Mr5Z8lHpHpuCON jjc0o1GGh854rVRuQUpshnI2LQ== X-Google-Smtp-Source: AK7set9kT9H6NMj2/S3WBxDWHmAmNxq0wUR9ec7mDgnR1zjNhxV91JrWyeNEVDzBJm76EQ32XvPpCg== X-Received: by 2002:a5d:4b03:0:b0:2c5:644c:3e89 with SMTP id v3-20020a5d4b03000000b002c5644c3e89mr2357955wrq.67.1676482380110; Wed, 15 Feb 2023 09:33:00 -0800 (PST) Received: from localhost ([95.148.15.48]) by smtp.gmail.com with ESMTPSA id f26-20020a5d58fa000000b002c5526680cbsm10288128wrd.92.2023.02.15.09.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 09:32:59 -0800 (PST) From: Punit Agrawal To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [External] [PATCH v2 00/33] Per-VMA locks References: <20230127194110.533103-1-surenb@google.com> Date: Wed, 15 Feb 2023 17:32:58 +0000 In-Reply-To: <20230127194110.533103-1-surenb@google.com> (Suren Baghdasaryan's message of "Fri, 27 Jan 2023 11:40:37 -0800") Message-ID: <87r0uqq1f9.fsf@stealth> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 024661A0002 X-Rspam-User: X-Stat-Signature: 1mdm3x71z3bu9fz6d3u14jh9pb7k768u X-HE-Tag: 1676482381-374033 X-HE-Meta: U2FsdGVkX1+4PnMVIYhfMw/AImUxcGSsx2dKqsYOeeHp1zC/K46ETtcLWn9PXdm5ptOhpjEpX4UiakW+WFWCqMc6rBa1x8fvQW+a7jE4zfd0ggw6KNhMa6kUmwkFFrSgg8WSXBcrVGTe5U4al5fwkPYTgHSWunLei8BlF0sOtMkX1pxuIwz0HaJdmYp8KqnuM2fJ+qYgHR1wa96bvxhHlydlPPiaIkwc+z3rPGGyXVDpH6XwDJcCG2EsnRqh8j6ZImorK5NXKJCZIAXDcRn59yZ+5uq+HgiuKPu52Ij2EFI1JSwGCFR59MVVUNH99fJ2l1LpKJVGyv0flnFSjbn2DpEAYMz8MYGFN3vKNK393OKE1o7JL8epZl95X/kKe5WvSBdNNRuHYVh239GMYt6FyXhGyvtc8PnrRyc7VEYli76YBeEck3ZsrIaN6PsJXcKmTj+fUVikFJRwcRwfQHjL8gkBVYnKoRWocdfYe0xbgZcbsmcorqOZu1C1QkqGiY006G48SjdqIbc7CST6jTKaag8cGsc4TNPHibor0pNOv4mcnJA2kIk8zkWvWDjOpIc33nJNDLrIQ5oPGs9hsRdVwtQH1GSLZOK25po/RZDidKiZQDlNsYwK5EKvo7lhg+R4BB15+VyT6b7HTcB0qtLnGRnO0qw5rH2n0krsy9ciWJ/BVfoyPBa7/pCC8TigqUglepgk2ydlLbajv8dr2l4SZnLsSOA5GpQtKcQlK+W6jNaiHii83zcwWsG1ijdZ/Dh2T4zDJiLvzEjNmuFM0y4/hF9zOz7rM5SPcE8wY6vIrI+TnUJH0gxmhiRnPeCqR/Gs1cljLl1c4h66V2/QHMnqrvqABGpsQ0YrnCgYgD4rV6hrsHb2KbBWNCkCoQ8VV0E5F5WbCGuTZ9lD00LL+1zQ3o5WaKxMj4ekTdq1uwKLC3NqusWamBu1Y2OiCRiHBkn5ZdfpCmbtve0KtfSANxT Dk4qd/84 4k2io90J7XBGU/uwETSyBvcnIbEC62tZyzM/Qy1k/RaESwehSOynILkDXqBMZlya5898Cy7VzwHIauPlF0yaXsOBZrmMbbmHdB56qqAzK41DHoHXyJvnqLsdlK9jeI7u4ailCUgNURakK0qRXtoOtxX41tkHvSPVGXCbZ9otn04wx4BwomIjqmuPTaTTCsgyEP8EdKw6EqjyU78jLT61DzktxJydKDNN5f9TJHzE3BW0TJ5gbMgPoT32l+/rQdPT6yPZiCJpbjmfu3bUpzsa7hQ18RfKXEBMYBV3t/z1E4XSiXnaZFRWHW6n1hiNbNvykT9N+8/edVHw4JdRH6dWMleD1eWkXvGvxhmYlh15tW93G++yFEt7G+OeHphnIj7GUDUM5Ac+nzF+9OVSIpkUyQ0M7o/kf0gd7i/ZyhhiyTleT0VVWF6bHXnq+AsGltUGDPVDIPMdlnsePKnG7ZmHlGicsu0zN/Ntz0TJIgkd5xhzVzGptDmMMP5u9khuvdm9wN7ZfjQ4AQQFVQDsTBQ1pRXCbwOAVHywgcKPBhrc1KHJapndln0cUt1xEGcyMnYnR1ZGCZ2zRblyfI6FfUNxvt1AoCnaTTL44lc0B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Suren Baghdasaryan writes: > Previous version: > v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.co= m/ > RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.co= m/ > > LWN article describing the feature: > https://lwn.net/Articles/906852/ > > Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM > last year [2], which concluded with suggestion that =E2=80=9Ca reader/wri= ter > semaphore could be put into the VMA itself; that would have the effect of > using the VMA as a sort of range lock. There would still be contention at > the VMA level, but it would be an improvement.=E2=80=9D This patchset imp= lements > this suggested approach. I took the patches for a spin on a 2-socket 32 core (64 threads) system with Intel 8336C (Ice Lake) and 512GB of RAM. For the initial testing, "pft-threads" from the mm-tests suite[0] was used. The test mmaps a memory region (~100GB on the test system) and triggers access by a number of threads executing in parallel. For each degree of parallelism, the test is repeated 10 times to get a better feel for the behaviour. Below is an excerpt of the harmonic mean reported by 'compare_kernel' script[1] included with mm-tests. The first column is results for mm-unstable as of 2023-02-10, the second column is the patches posted here while the third column includes optimizations to reclaim some of the observed regression. >From the results, there is a drop in page fault/second for low number of CPUs but good improvement with higher CPUs. 6.2.0-rc4 6.2.0-rc4 = 6.2.0-rc4 mm-unstable-20230210 pvl-v2 = pvl-v2+opt Hmean faults/cpu-1 898792.9338 ( 0.00%) 894597.0474 * -0.47%* = 895933.2782 * -0.32%* Hmean faults/cpu-4 751903.9803 ( 0.00%) 677764.2975 * -9.86%* = 688643.8163 * -8.41%* Hmean faults/cpu-7 612275.5663 ( 0.00%) 565363.4137 * -7.66%* = 597538.9396 * -2.41%* Hmean faults/cpu-12 434460.9074 ( 0.00%) 410974.2708 * -5.41%* = 452501.4290 * 4.15%* Hmean faults/cpu-21 291475.5165 ( 0.00%) 293936.8460 ( 0.84%) = 308712.2434 * 5.91%* Hmean faults/cpu-30 218021.3980 ( 0.00%) 228265.0559 * 4.70%* = 241897.5225 * 10.95%* Hmean faults/cpu-48 141798.5030 ( 0.00%) 162322.5972 * 14.47%* = 166081.9459 * 17.13%* Hmean faults/cpu-79 90060.9577 ( 0.00%) 107028.7779 * 18.84%* = 109810.4488 * 21.93%* Hmean faults/cpu-110 64729.3561 ( 0.00%) 80597.7246 * 24.51%* = 83134.0679 * 28.43%* Hmean faults/cpu-128 55740.1334 ( 0.00%) 68395.4426 * 22.70%* = 69248.2836 * 24.23%* Hmean faults/sec-1 898781.7694 ( 0.00%) 894247.3174 * -0.50%* = 894440.3118 * -0.48%* Hmean faults/sec-4 2965588.9697 ( 0.00%) 2683651.5664 * -9.51%* = 2726450.9710 * -8.06%* Hmean faults/sec-7 4144512.3996 ( 0.00%) 3891644.2128 * -6.10%* = 4099918.8601 ( -1.08%) Hmean faults/sec-12 4969513.6934 ( 0.00%) 4829731.4355 * -2.81%* = 5264682.7371 * 5.94%* Hmean faults/sec-21 5814379.4789 ( 0.00%) 5941405.3116 * 2.18%* = 6263716.3903 * 7.73%* Hmean faults/sec-30 6153685.3709 ( 0.00%) 6489311.6634 * 5.45%* = 6910843.5858 * 12.30%* Hmean faults/sec-48 6197953.1327 ( 0.00%) 7216320.7727 * 16.43%* = 7412782.2927 * 19.60%* Hmean faults/sec-79 6167135.3738 ( 0.00%) 7425927.1022 * 20.41%* = 7637042.2198 * 23.83%* Hmean faults/sec-110 6264768.2247 ( 0.00%) 7813329.3863 * 24.72%* = 7984344.4005 * 27.45%* Hmean faults/sec-128 6460727.8216 ( 0.00%) 7875664.8999 * 21.90%* = 8049910.3601 * 24.60%* [0] https://github.com/gormanm/mmtests [1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh