From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50C26C77B6C for ; Wed, 5 Apr 2023 10:44:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237811AbjDEKoF (ORCPT ); Wed, 5 Apr 2023 06:44:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237588AbjDEKoE (ORCPT ); Wed, 5 Apr 2023 06:44:04 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98E694ECA; Wed, 5 Apr 2023 03:44:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2975763C57; Wed, 5 Apr 2023 10:44:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BFC61C433EF; Wed, 5 Apr 2023 10:44:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680691441; bh=+ZO/yBxfJv0GaHczUgdJTBFHBjfQ/ykADeJttq42HmQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RfEi0EWHrVtgDNTDmAkQHpOA1xGMHUdirDcpsD08NWpiRE8In3RAHPfhmz+mLo3Re keFWgaWXAHpSzJgLMWdEzHtCNaP6XaC77hbIAZwK+4PpDX/Zwb9pTItZlq4DbE4Bcj gF37NoMLKD33dyuHM4yV5O3ouVk+s6lLupFGmAHvGBDdJp1MrGsgJ5ZzVLp7GXS7g2 UIOMTpAu0+CUNt+73VKyRli+76KWSC33YcG5QeaD76fiXSWmghliODfhBl98LZE/dA PhaBHuqHXYQ037jMpcPmRNTtYCqZtKeIbh0pk5/nikjbQBZDel/38qAxiwMpnOZnDc uqZAC/jHMG+Ow== Date: Wed, 5 Apr 2023 12:43:58 +0200 From: Frederic Weisbecker To: Yair Podemsky Cc: linux@armlinux.org.uk, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, will@kernel.org, aneesh.kumar@linux.ibm.com, akpm@linux-foundation.org, peterz@infradead.org, arnd@arndb.de, keescook@chromium.org, paulmck@kernel.org, jpoimboe@kernel.org, samitolvanen@google.com, ardb@kernel.org, juerg.haefliger@canonical.com, rmk+kernel@armlinux.org.uk, geert+renesas@glider.be, tony@atomide.com, linus.walleij@linaro.org, sebastian.reichel@collabora.com, nick.hawkins@hpe.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, mtosatti@redhat.com, vschneid@redhat.com, dhildenb@redhat.com, alougovs@redhat.com Subject: Re: [PATCH 3/3] mm/mmu_gather: send tlb_remove_table_smp_sync IPI only to CPUs in kernel mode Message-ID: References: <20230404134224.137038-1-ypodemsk@redhat.com> <20230404134224.137038-4-ypodemsk@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230404134224.137038-4-ypodemsk@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-arch@vger.kernel.org On Tue, Apr 04, 2023 at 04:42:24PM +0300, Yair Podemsky wrote: > @@ -191,6 +192,20 @@ static void tlb_remove_table_smp_sync(void *arg) > /* Simply deliver the interrupt */ > } > > + > +#ifdef CONFIG_CONTEXT_TRACKING > +static bool cpu_in_kernel(int cpu, void *info) > +{ > + struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu); Like Peter said, an smp_mb() is required here before the read (unless there is already one between the page table modification and that ct->state read?). So that you have this pairing: WRITE page_table WRITE ct->state smp_mb() smp_mb() // implied by atomic_fetch_or READ ct->state READ page_table > + int state = atomic_read(&ct->state); > + /* will return true only for cpus in kernel space */ > + return state & CT_STATE_MASK == CONTEXT_KERNEL; > +} Also note that this doesn't stricly prevent userspace from being interrupted. You may well observe the CPU in kernel but it may receive the IPI later after switching to userspace. We could arrange for avoiding that with marking ct->state with a pending work bit to flush upon user entry/exit but that's a bit more overhead so I first need to know about your expectations here, ie: can you tolerate such an occasional interruption or not? Thanks. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 27AFDC761A6 for ; Wed, 5 Apr 2023 10:44:57 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4Ps1VH3f5Hz3cgT for ; Wed, 5 Apr 2023 20:44:55 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RfEi0EWH; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=frederic@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=RfEi0EWH; dkim-atps=neutral Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Ps1TK65hmz3c6Y for ; Wed, 5 Apr 2023 20:44:05 +1000 (AEST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0DD036247E; Wed, 5 Apr 2023 10:44:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BFC61C433EF; Wed, 5 Apr 2023 10:44:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680691441; bh=+ZO/yBxfJv0GaHczUgdJTBFHBjfQ/ykADeJttq42HmQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RfEi0EWHrVtgDNTDmAkQHpOA1xGMHUdirDcpsD08NWpiRE8In3RAHPfhmz+mLo3Re keFWgaWXAHpSzJgLMWdEzHtCNaP6XaC77hbIAZwK+4PpDX/Zwb9pTItZlq4DbE4Bcj gF37NoMLKD33dyuHM4yV5O3ouVk+s6lLupFGmAHvGBDdJp1MrGsgJ5ZzVLp7GXS7g2 UIOMTpAu0+CUNt+73VKyRli+76KWSC33YcG5QeaD76fiXSWmghliODfhBl98LZE/dA PhaBHuqHXYQ037jMpcPmRNTtYCqZtKeIbh0pk5/nikjbQBZDel/38qAxiwMpnOZnDc uqZAC/jHMG+Ow== Date: Wed, 5 Apr 2023 12:43:58 +0200 From: Frederic Weisbecker To: Yair Podemsky Subject: Re: [PATCH 3/3] mm/mmu_gather: send tlb_remove_table_smp_sync IPI only to CPUs in kernel mode Message-ID: References: <20230404134224.137038-1-ypodemsk@redhat.com> <20230404134224.137038-4-ypodemsk@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230404134224.137038-4-ypodemsk@redhat.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: geert+renesas@glider.be, peterz@infradead.org, linus.walleij@linaro.org, dave.hansen@linux.intel.com, sebastian.reichel@collabora.com, linux-mm@kvack.org, hpa@zytor.com, sparclinux@vger.kernel.org, agordeev@linux.ibm.com, will@kernel.org, ardb@kernel.org, linux-arch@vger.kernel.org, linux-s390@vger.kernel.org, vschneid@redhat.com, arnd@arndb.de, paulmck@kernel.org, aneesh.kumar@linux.ibm.com, x86@kernel.org, linux@armlinux.org.uk, tony@atomide.com, mingo@redhat.com, samitolvanen@google.com, borntraeger@linux.ibm.com, keescook@chromium.org, gor@linux.ibm.com, hca@linux.ibm.com, npiggin@gmail.com, rmk+kernel@armlinux.org.uk, bp@alien8.de, nick.hawkins@hpe.com, tglx@linutronix.de, jpoimboe@kernel.org, linux-arm-kernel@lists.infradead.org, alougovs@redhat.com, mtosatti@redhat.com, linux-kernel@vger.kernel.org, juerg.haefliger@canonical.com, svens@linux.ibm.com, dhildenb@redhat.com, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org, davem@davemloft.net Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue, Apr 04, 2023 at 04:42:24PM +0300, Yair Podemsky wrote: > @@ -191,6 +192,20 @@ static void tlb_remove_table_smp_sync(void *arg) > /* Simply deliver the interrupt */ > } > > + > +#ifdef CONFIG_CONTEXT_TRACKING > +static bool cpu_in_kernel(int cpu, void *info) > +{ > + struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu); Like Peter said, an smp_mb() is required here before the read (unless there is already one between the page table modification and that ct->state read?). So that you have this pairing: WRITE page_table WRITE ct->state smp_mb() smp_mb() // implied by atomic_fetch_or READ ct->state READ page_table > + int state = atomic_read(&ct->state); > + /* will return true only for cpus in kernel space */ > + return state & CT_STATE_MASK == CONTEXT_KERNEL; > +} Also note that this doesn't stricly prevent userspace from being interrupted. You may well observe the CPU in kernel but it may receive the IPI later after switching to userspace. We could arrange for avoiding that with marking ct->state with a pending work bit to flush upon user entry/exit but that's a bit more overhead so I first need to know about your expectations here, ie: can you tolerate such an occasional interruption or not? Thanks. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62E9FC77B60 for ; Wed, 5 Apr 2023 10:44:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4GG9H67eQAM2UvRSQ47+7b8cdCibq9Y9lFt/KSxp5W0=; b=ZjYImzBEmkELc9 af72pj2wlDvln0bfVRi37Qa9nOE2H+K0OyGzvu2HHqqD2g7rBRlyuexOmZ1MBsylUnnfeOfdRQHZP fA6pkXfagEMe51ZGUGH7gwbdez06iSCWEE3kXM5d4mZ1/YPD6pF+QFVOFzCuU5R6zHiVyGiNmHiqi ffS4a4CUPNXADTWD880KeJsDpfW8LnJWcaECmaTEh/euuaY/5/Vyjrv3TV6ziqG50LpqDltDQfDGb EfDtrPEze5FZLDXls61i14J8jkduGtoWMk76f3ERLWG1KxoH6LROy1pFxRuynkslEgOmsU+JjHDg/ I1Yo8emegYmljZMmSocg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pk0ca-0047mS-35; Wed, 05 Apr 2023 10:44:08 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pk0cW-0047ka-1k for linux-arm-kernel@lists.infradead.org; Wed, 05 Apr 2023 10:44:06 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0DD036247E; Wed, 5 Apr 2023 10:44:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BFC61C433EF; Wed, 5 Apr 2023 10:44:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680691441; bh=+ZO/yBxfJv0GaHczUgdJTBFHBjfQ/ykADeJttq42HmQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RfEi0EWHrVtgDNTDmAkQHpOA1xGMHUdirDcpsD08NWpiRE8In3RAHPfhmz+mLo3Re keFWgaWXAHpSzJgLMWdEzHtCNaP6XaC77hbIAZwK+4PpDX/Zwb9pTItZlq4DbE4Bcj gF37NoMLKD33dyuHM4yV5O3ouVk+s6lLupFGmAHvGBDdJp1MrGsgJ5ZzVLp7GXS7g2 UIOMTpAu0+CUNt+73VKyRli+76KWSC33YcG5QeaD76fiXSWmghliODfhBl98LZE/dA PhaBHuqHXYQ037jMpcPmRNTtYCqZtKeIbh0pk5/nikjbQBZDel/38qAxiwMpnOZnDc uqZAC/jHMG+Ow== Date: Wed, 5 Apr 2023 12:43:58 +0200 From: Frederic Weisbecker To: Yair Podemsky Cc: linux@armlinux.org.uk, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, will@kernel.org, aneesh.kumar@linux.ibm.com, akpm@linux-foundation.org, peterz@infradead.org, arnd@arndb.de, keescook@chromium.org, paulmck@kernel.org, jpoimboe@kernel.org, samitolvanen@google.com, ardb@kernel.org, juerg.haefliger@canonical.com, rmk+kernel@armlinux.org.uk, geert+renesas@glider.be, tony@atomide.com, linus.walleij@linaro.org, sebastian.reichel@collabora.com, nick.hawkins@hpe.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, mtosatti@redhat.com, vschneid@redhat.com, dhildenb@redhat.com, alougovs@redhat.com Subject: Re: [PATCH 3/3] mm/mmu_gather: send tlb_remove_table_smp_sync IPI only to CPUs in kernel mode Message-ID: References: <20230404134224.137038-1-ypodemsk@redhat.com> <20230404134224.137038-4-ypodemsk@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230404134224.137038-4-ypodemsk@redhat.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230405_034404_621406_C537D241 X-CRM114-Status: GOOD ( 16.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Apr 04, 2023 at 04:42:24PM +0300, Yair Podemsky wrote: > @@ -191,6 +192,20 @@ static void tlb_remove_table_smp_sync(void *arg) > /* Simply deliver the interrupt */ > } > > + > +#ifdef CONFIG_CONTEXT_TRACKING > +static bool cpu_in_kernel(int cpu, void *info) > +{ > + struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu); Like Peter said, an smp_mb() is required here before the read (unless there is already one between the page table modification and that ct->state read?). So that you have this pairing: WRITE page_table WRITE ct->state smp_mb() smp_mb() // implied by atomic_fetch_or READ ct->state READ page_table > + int state = atomic_read(&ct->state); > + /* will return true only for cpus in kernel space */ > + return state & CT_STATE_MASK == CONTEXT_KERNEL; > +} Also note that this doesn't stricly prevent userspace from being interrupted. You may well observe the CPU in kernel but it may receive the IPI later after switching to userspace. We could arrange for avoiding that with marking ct->state with a pending work bit to flush upon user entry/exit but that's a bit more overhead so I first need to know about your expectations here, ie: can you tolerate such an occasional interruption or not? Thanks. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel