From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 351FFC2D0E4 for ; Tue, 24 Nov 2020 02:48:56 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD6D22071E for ; Tue, 24 Nov 2020 02:48:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="g530sbSI"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="VOkJWl6X" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD6D22071E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Mime-Version:References:In-Reply-To:Message-Id: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qkAOJSZ1TPr7DQbCIux6PU6wOxp9XXgezoUqqtw86GM=; b=g530sbSIXLoXOVCNuLwJESmV7 OAqQHBSqzVwUEbvtM4usPC7GGUs99s7eVHdBay/2BbnFgEzdHX4xdrwj5LdJSKOtE+E7I1527w5Hq ZXkLpw3Q7uCzOTdXf9D16l7ZzPtGuJEjUu60mBQS1wzneYIU2ZIo+eEMXayrlOAh20hCEg0yYV2dt VAbMlsNBRwVolUC9cjO58L0Vg6fyyE4uvX0hDSRk19GRDLay4t+zievaCYC53CzjdZ6FkvRhLKxyV qQTuXi17JwLuuUf/QthyOY+Hu1QCb3etb386e7BCwD7qn4ejPKmNcxXTCMWliBGyuoSUXu3SdW9Yr OxfSTWlRQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1khONS-0006TP-Uv; Tue, 24 Nov 2020 02:48:22 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1khONQ-0006T1-HK for linux-arm-kernel@lists.infradead.org; Tue, 24 Nov 2020 02:48:21 +0000 Received: from devnote2 (NE2965lan1.rev.em-net.ne.jp [210.141.244.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1B9242067C; Tue, 24 Nov 2020 02:48:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1606186098; bh=DifF5KiKF9Z3qJbFxfsfNCx0cjrKfrDIQko7nyOBRt8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=VOkJWl6XDSNIbKYMcde50Gro6w7QUypJjHsyluqR/zIlw9ii5zh9boF9afY1z1euE VgtBkuz2XjYmCONqpy834L3SMNeL1QmMGbW4I/bHIgYweQKOGXMiy/PHGTASM9LQTN GaiP4Rj4rInVn5lXauCbp7qyRdXmN2prO8AlwDyo= Date: Tue, 24 Nov 2020 11:48:13 +0900 From: Masami Hiramatsu To: Will Deacon Subject: Re: [PATCH] arm64: don't preempt_disable in do_debug_exception Message-Id: <20201124114813.677e57eb591218a98b70af0e@kernel.org> In-Reply-To: <20200626095551.GA9312@willie-the-truck> References: <1592501369-27645-1-git-send-email-paul.gortmaker@windriver.com> <20200623155900.GA4777@willie-the-truck> <20200623165557.GA12767@C02TD0UTHF1T.local> <20200626095551.GA9312@willie-the-truck> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201123_214820_747712_BC79365C X-CRM114-Status: GOOD ( 36.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Jean-Philippe Brucker , Catalin Marinas , Naresh Kamboju , stable@vger.kernel.org, Paul Gortmaker , James Morse , Masami Hiramatsu , "Paul E . McKenney" , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 26 Jun 2020 10:55:54 +0100 Will Deacon wrote: > On Tue, Jun 23, 2020 at 05:55:57PM +0100, Mark Rutland wrote: > > On Tue, Jun 23, 2020 at 04:59:01PM +0100, Will Deacon wrote: > > > On Thu, Jun 18, 2020 at 01:29:29PM -0400, Paul Gortmaker wrote: > > > > BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:975 > > > > in_atomic(): 1, irqs_disabled(): 0, pid: 35658, name: gdbtest > > > > Preemption disabled at: > > > > [] do_debug_exception+0x38/0x1a4 > > > > Call trace: > > > > dump_backtrace+0x0/0x138 > > > > show_stack+0x24/0x30 > > > > dump_stack+0x94/0xbc > > > > ___might_sleep+0x13c/0x168 > > > > rt_spin_lock+0x40/0x80 > > > > do_force_sig_info+0x30/0xe0 > > > > force_sig_fault+0x64/0x90 > > > > arm64_force_sig_fault+0x50/0x80 > > > > send_user_sigtrap+0x50/0x80 > > > > brk_handler+0x98/0xc8 > > > > do_debug_exception+0x70/0x1a4 > > > > el0_dbg+0x18/0x20 > > > > > > > > The reproducer was basically an automated gdb test that set a breakpoint > > > > on a simple "hello world" program and then quit gdb once the breakpoint > > > > was hit - i.e. "(gdb) A debugging session is active. Quit anyway? " > > > > > > Hmm, the debug exception handler path was definitely written with the > > > expectation that preemption is disabled, so this is unfortunate. For > > > exceptions from kernelspace, we need to keep that guarantee as we implement > > > things like BUG() using this path. For exceptions from userspace, it's > > > plausible that we could re-enable preemption, but then we should also > > > re-enable interrupts and debug exceptions too because we don't > > > context-switch pstate in switch_to() and we would end up with holes in our > > > kernel debug coverage (and these might be fatal if e.g. single step doesn't > > > work in a kprobe OOL buffer). However, that then means that any common code > > > when handling user and kernel debug exceptions needs to be re-entrant, > > > which it probably isn't at the moment (I haven't checked). > > > > I'm pretty certain existing code is not reentrant, and regardless it's > > going to be a mess to reason about this generally if we have to undo our > > strict exception nesting rules. > > Are these rules written down somewhere? I'll need to update them if we > get this working for preempt-rt (and we should try to do that). > > > I reckon we need to treat this like an NMI instead -- is that plausible? > > I don't think so. It's very much a synchronous exception, and delivering a > signal to the exceptional context doesn't feel like an NMI to me. There's > also a fair amount of code that can run in debug context (hw_breakpoint, > kprobes, uprobes, kasan) which might not be happy to suddenly be in an > NMI-like environment. Furthermore, the masking rules are different depending > on what triggers the exception. > > One of the things I've started looking at is ripping out our dodgy > hw_breakpoint code so that kernel debug exceptions are easier to reason > about. Specifically, I think we end up with something like: > > - On taking a non-debug exception from EL0, unmask D as soon as we can. > > - On taking a debug exception from EL0, unmask {D,I} and invoke user > handlers. I think this always means SIGTRAP, apart from uprobes. > This will mean making those paths preemptible, as I don't think they > are right now (e.g. traversing the callback hooks uses an RCU-protected > list). > > - On taking a non-debug, non-fatal synchronous exception from EL1, unmask > D as soon as we can (i.e. we step into these exceptions). Fatal exceptions > can obviously leave D masked. To make clear, the BRK exception will be non-fatal synchronous exception, correct? If so, would you mean single-stepping into these exception handlers too? As we discussed in another thread, after the BRK only kprobes is merged, I'm OK for this. But also we need to care about the BRK recursive call. If someone puts a kprobe in the single-step handler, we can break into the other break handler is running. (kprobes itself can handle this case, because it sets the current_kprobe as the recursion-detect flag) > > - On taking an interrupt from EL1, stash MDSCR_EL1.SS in a pcpu variable and > clear the register bit if it was set. Then unmask only D and leave I set. On > return from the exception, set D and restore MDSCR_EL1.SS. If we decide to > reschedule, unmask D (i.e. we only step into interrupts if we need a > reschedule. Alternatively, we could skip the reschedule if we were > stepping.) This sounds good to me (context-based single-stepping). Thank you, > > - On taking a debug exception from EL1, leave {D,I} set. Watchpoints on > uaccess are silently stepped over. > > Thoughts? We could probably simplify this if we could state that stepping an > instruction in kernel space could only ever be interrupted by an interrupt. > That's probably true for kprobes, but relying on it feels like it might bite > us later on. > > Will -- Masami Hiramatsu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel