From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CF39C4338F for ; Tue, 17 Aug 2021 16:31:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3314A60FD7 for ; Tue, 17 Aug 2021 16:31:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231145AbhHQQb1 (ORCPT ); Tue, 17 Aug 2021 12:31:27 -0400 Received: from gate.crashing.org ([63.228.1.57]:38467 "EHLO gate.crashing.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230386AbhHQQ3z (ORCPT ); Tue, 17 Aug 2021 12:29:55 -0400 Received: from gate.crashing.org (localhost.localdomain [127.0.0.1]) by gate.crashing.org (8.14.1/8.14.1) with ESMTP id 17HGMea1021252; Tue, 17 Aug 2021 11:22:40 -0500 Received: (from segher@localhost) by gate.crashing.org (8.14.1/8.14.1/Submit) id 17HGMdsF021251; Tue, 17 Aug 2021 11:22:39 -0500 X-Authentication-Warning: gate.crashing.org: segher set sender to segher@kernel.crashing.org using -f Date: Tue, 17 Aug 2021 11:22:39 -0500 From: Segher Boessenkool To: Christophe Leroy Cc: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , userm57@yahoo.com, fthain@linux-m68k.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] powerpc/32s: Fix random crashes by adding isync() after locking/unlocking KUEP Message-ID: <20210817162239.GF1583@gate.crashing.org> References: <1d28441dd80845e6428d693c0724cb6457247466.1629211378.git.christophe.leroy@csgroup.eu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1d28441dd80845e6428d693c0724cb6457247466.1629211378.git.christophe.leroy@csgroup.eu> User-Agent: Mutt/1.4.2.3i Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Aug 17, 2021 at 02:43:15PM +0000, Christophe Leroy wrote: > Commit b5efec00b671 ("powerpc/32s: Move KUEP locking/unlocking in C") > removed the 'isync' instruction after adding/removing NX bit in user > segments. The reasoning behind this change was that when setting the > NX bit we don't mind it taking effect with delay as the kernel never > executes text from userspace, and when clearing the NX bit this is > to return to userspace and then the 'rfi' should synchronise the > context. > > However, it looks like on book3s/32 having a hash page table, at least > on the G3 processor, we get an unexpected fault from userspace, then > this is followed by something wrong in the verification of MSR_PR > at end of another interrupt. > > This is fixed by adding back the removed isync() following update > of NX bit in user segment registers. Only do it for cores with an > hash table, as 603 cores don't exhibit that problem and the two isync > increase ./null_syscall selftest by 6 cycles on an MPC 832x. > > First problem: unexpected PROTFAULT > > [ 62.896426] WARNING: CPU: 0 PID: 1660 at arch/powerpc/mm/fault.c:354 do_page_fault+0x6c/0x5b0 > [ 62.918111] Modules linked in: > [ 62.923350] CPU: 0 PID: 1660 Comm: Xorg Not tainted 5.13.0-pmac-00028-gb3c15b60339a #40 > [ 62.943476] NIP: c001b5c8 LR: c001b6f8 CTR: 00000000 > [ 62.954714] REGS: e2d09e40 TRAP: 0700 Not tainted (5.13.0-pmac-00028-gb3c15b60339a) That is not a protection fault. What causes this? A CSI (like isync) is required both before and after mtsr. It may work on some cores without -- what part of that is luck, if there is anything that guarantees it, is anyone's guess :-/ > @@ -28,6 +30,8 @@ static inline void kuep_lock(void) > return; > > update_user_segments(mfsr(0) | SR_NX); > + if (mmu_has_feature(MMU_FTR_HPTE_TABLE)) > + isync(); /* Context sync required after mtsr() */ > } This needs a comment why you are not doing this for systems without hardware page table walk, at the least? Segher