From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82250EB64D8 for ; Thu, 22 Jun 2023 13:11:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE0178D0002; Thu, 22 Jun 2023 09:11:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C69368D0001; Thu, 22 Jun 2023 09:11:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE3188D0002; Thu, 22 Jun 2023 09:11:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9D64D8D0001 for ; Thu, 22 Jun 2023 09:11:44 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6F02B120B4D for ; Thu, 22 Jun 2023 13:11:44 +0000 (UTC) X-FDA: 80930420928.08.1568F68 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 22C271A0002 for ; Thu, 22 Jun 2023 13:11:40 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CLGYB8PI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of ypodemsk@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=ypodemsk@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687439501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sL8itRGmxApUKDOrvea4cRz5mnSUjX/8ZLxFKQlLox8=; b=xYl2TZ2F1TVU5HFiTrX/ygwZqr2G2Up3Bi6x3+mHpYXrpvPG1PcySRH9LDq3RtvzU0iPwA DJ/9xH/4mfbRha0TNNGVk1b85Nj48C7A5hh2aQ6YVT9v16C9YYxWugP6/dtjHYfMFUf8o1 EIT947MJ6CuDEN0+fLMSV7RzZPWDCto= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CLGYB8PI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of ypodemsk@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=ypodemsk@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687439501; a=rsa-sha256; cv=none; b=UVWHBj/euaw9CNV0BoLDgdv61CnGM2SPLPL9jiLrjGkd5faciObUuaK/q+FENSSRpDtgUS IS6hLN/dzYKMM80NuPhNN1idnXBRy3kxliEqZITyaOaErNx6gRAAt1Paw8F7nGb8RlqeVv woVv4ePU4iaJT2rCAbYjM8Ia9IZ1+XY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687439500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sL8itRGmxApUKDOrvea4cRz5mnSUjX/8ZLxFKQlLox8=; b=CLGYB8PI6b4R3Ja0/X/dITVP9zfCpmEHxZYg+ZbdgX79gZVY5007ypxpd3+obrThZJlrRD wuctEKM8wYyPmExS8rzxvvDFH2KVAaucv5HiN9FMN+JdvwwYmEKEayqkZD8G36FkK+bJBn 5n71BY3nNxQBeloGOZmih1Ym6iMvPwo= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-583-6YAaFnvxPrmoOqODzUUwSA-1; Thu, 22 Jun 2023 09:11:39 -0400 X-MC-Unique: 6YAaFnvxPrmoOqODzUUwSA-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2b45bf52675so52691491fa.3 for ; Thu, 22 Jun 2023 06:11:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687439496; x=1690031496; h=content-transfer-encoding:mime-version:references:in-reply-to:date :cc:to:from:subject:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=sL8itRGmxApUKDOrvea4cRz5mnSUjX/8ZLxFKQlLox8=; b=Z8vjElRUYeRnI1tr6Hc5XSI93M+BjSalPIu/ueJYRYyW2NDXXcS02fVy36E7L5Xwfi uLvLvA9b4i/AOAUGoNaqYnpe8AZF8izPH2KPOtrujw5aafQLTQQIVRZ1g0eMCffwkrLR wtI1Ice2090rMo1gNrjZrLFSqd9eAVDrQchyLcHp2tIV0QAnG3EFEQBeuXwlmiRn4ZAy RtCBjXEaXZQGF2LYpNyuAB3GDzOfkURmE5/jlKRluJO5i6E2g2D8LBflQ7ybjVcLkumK 1GD+3CJ+3r+kkCuJFfxJezK5Ba4Q+DkFjp+DSLFXLcLdSEZZocHgIqGAjxtlcORhzZPV PDzA== X-Gm-Message-State: AC+VfDzjy1CHsnFqZqpGjlY+fG54Ai2KlLf/jpQ0PStxzYyDw93KkcKk Q2gn+K3oVqw02zuLhao6W4IEoSBqkrp4Qf1LGRW0j3CcX0oQ1MN2PO7Zk+G6dTDdZmaKdRghJJE Exyi3BmaxCP4= X-Received: by 2002:a19:7710:0:b0:4f8:6800:86f6 with SMTP id s16-20020a197710000000b004f8680086f6mr7663752lfc.49.1687439496651; Thu, 22 Jun 2023 06:11:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6XeMu+ILD2rIWpjmZgWGZlKWI9+Th3UDxS632DqbBLE4GiDliqEnF0ulmrbywCCnR/Rw0JbA== X-Received: by 2002:a19:7710:0:b0:4f8:6800:86f6 with SMTP id s16-20020a197710000000b004f8680086f6mr7663723lfc.49.1687439496259; Thu, 22 Jun 2023 06:11:36 -0700 (PDT) Received: from ypodemsk.tlv.csb (IGLD-84-229-250-192.inter.net.il. [84.229.250.192]) by smtp.gmail.com with ESMTPSA id k37-20020a05600c1ca500b003f9b3829269sm2706502wms.2.2023.06.22.06.11.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 06:11:35 -0700 (PDT) Message-ID: <7a9f193e6fa9db1d5fa0eb4a91927a866909f13c.camel@redhat.com> Subject: Re: [PATCH v2 0/2] send tlb_remove_table_smp_sync IPI only to necessary CPUs From: ypodemsk@redhat.com To: Peter Zijlstra Cc: mtosatti@redhat.com, ppandit@redhat.com, david@redhat.com, linux@armlinux.org.uk, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, keescook@chromium.org, paulmck@kernel.org, frederic@kernel.org, will@kernel.org, ardb@kernel.org, samitolvanen@google.com, juerg.haefliger@canonical.com, arnd@arndb.de, rmk+kernel@armlinux.org.uk, geert+renesas@glider.be, linus.walleij@linaro.org, akpm@linux-foundation.org, sebastian.reichel@collabora.com, rppt@kernel.org, aneesh.kumar@linux.ibm.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Thu, 22 Jun 2023 16:11:32 +0300 In-Reply-To: <20230621074337.GF2046280@hirez.programming.kicks-ass.net> References: <20230620144618.125703-1-ypodemsk@redhat.com> <20230621074337.GF2046280@hirez.programming.kicks-ass.net> X-Mailer: Evolution 3.28.5 (3.28.5-22.el8) Mime-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 22C271A0002 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 7yyrw1whtctyw9bsj1pbkgp6rjedtgxc X-HE-Tag: 1687439500-273440 X-HE-Meta: U2FsdGVkX1/f1dTQGugHWpyJXeQdlHZQOxSY4S8d6SdTbBLtyVCJqoNNzk/B+G2rJ9Sg2Mq4m8nv2SqP/T40Q1DEDVZcUamEEbPUnaglmqabpEyo2h/G6a2viVdDX16cav6ey/heWo75gfoNSSq713aN3D7D9uQHmnEU7sfrWqPc87eKxAZBUO2ext2PivgSXQj0XBYbEW3tDyEO5/hnIzv12I2IC/tw29tAI6CT+pnVN3+CqOljo5YVa3uipvgg//feomWgogEerZjv2pSKhWTyTuLN0Byk3SRWAb9+ozLk0tXWOSqJupuu0N5LsEKxd1nq8ENgxjkUA6boVp1imAIUYhPCcRkdQe2bxIVHv66lucb/JWUyxzFGaNIY5Iqc4w0Jprc/dMFuMcCRY7IRZz34tP5XcE13SNE6qSXFcMVaXXDe8tRs0pI75djfolUOReUBDHM/m7hMoh2+EdcDPYq3Ih5ESuSCbWdF4zt6k1DVgE/R/HLt/YPukA2UkWKSWMfBfq/MkvqngxDG8JJOoUaiiPReki6FFqfh4MYPvxeWi5h66ehvWdEQ9rYYedcRakHoRSBh+vj2uRfJAdgwyc4C6svuB76TfSPaENU3VRJIWGRFes5IvbfKb/0j7VrCfWiWEJpJiGpck/ZtdFtdhRd199tWMRb228DLgUZnPW91qtfopoEVwYj4QThB2NF66Q3vC32nKFSgWaaY4YM1XRIPzxNi1WU2wDK+x5yC6wir1rGgufWxL3zMdQfv1wHcByyAUmoIl3Y+dpPMvJjZU/l6G7o8UMyp5RdGbjz1+s3rIc2F9czpmUbFUBydTYpgZ3GW5IalTRCyPkfjXUfP6AgXIZbyeXs1XpPfWT8Sq0vwJlir4IEh5iAfB/w7/aD99GtH0Z9EBW237OGeCdPF6jJgwUN/HcnAH7zW1+ZsmiYgyT0gzbepTQm9ExBcsg19yyWsblbH4cG3Z/ngLS5 bgNQeD2V ROXnGAIhXGwoU6ci/XF9nLz3HdwJ/cK91OYYKrja2e6IpxJvrFdqYnwdXr08jDG8AI9PV00oSc3Zyz5/G0hfPr3k8eM3iN+W0PvTZnmVuswJaRDdrttCsegDUU1L/iDEcTZjBcOOIRzC0eDH7bhIC1Nt/ScDuPRB3g92Pgzr+NqHbGi5aM/8WByxqlu1SCwBxvxqwPTs3qoao4px3U2McU3sMdrnHMxK6LmJPYZBMJzRKkKsCVy070/wKvahf+DRyE+oThSwn0SJJ9P0cwLmnOBwa3obdM2WkoOqqpVwBCVfYmJ69mKTC8c2vk7QlI6dDmEr+3NnPMEHEnL5rw3qRMONmw1vh45LCZCNEsp0sgJoAqAVG1SmrD4H0aTmkUVy5UyZdAw4HHMI7pL0g9qcPPFA9YHESx8eZ7PpL7WVgq19jY73kSZI/I68OSow1igEfA6bad1GGCx6/FWZ7Qc+fN7SNF95/dStaNoo10zy9fSUCOzQvZW2b3PfimzxZ1Sym1Jch5ihr3W8YxLbR8ci9mgKFag== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2023-06-21 at 09:43 +0200, Peter Zijlstra wrote: > On Tue, Jun 20, 2023 at 05:46:16PM +0300, Yair Podemsky wrote: > > Currently the tlb_remove_table_smp_sync IPI is sent to all CPUs > > indiscriminately, this causes unnecessary work and delays notable > > in > > real-time use-cases and isolated cpus. > > By limiting the IPI to only be sent to cpus referencing the > > effected > > mm. > > a config to differentiate architectures that support mm_cpumask > > from > > those that don't will allow safe usage of this feature. > > > > changes from -v1: > > - Previous version included a patch to only send the IPI to CPU's > > with > > context_tracking in the kernel space, this was removed due to race > > condition concerns. > > - for archs that do not maintain mm_cpumask the mask used should be > > cpu_online_mask (Peter Zijlstra). > > > > Would it not be much better to fix the root cause? As per the last > time, > there's patches that cure the thp abuse of this. > Hi Peter, Thanks for your reply. There are two code paths leading to this IPI, one is the thp, But the other is the failure to allocate page in tlb_remove_table, It is the the second path that we are most interested in as it was found to cause interference in a real time process for a client (That system did not have thp). So while curing thp abuses is a good thing, it will not unfortunately solve our root cause. If you have any idea of how to remove the tlb_remove_table_sync_one() usage in the tlb_remove_table()->tlb_remove_table_one() call path -- the usage that's relevant for us -- that would be great. As long as we can't remove that, I'm afraid all we can do is optimize for it to not broadcast an IPI to all CPUs in the system, as done in this patch. Thanks, Yair