From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D757C61D97 for ; Thu, 23 Nov 2023 17:45:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=0wUpz+pOfFpzQRMzmsoQD74PYIHTnfIynrk0rhPE6FI=; b=w3j sQH3Uff70zRN5NWcoBjY/rNfzjNGoMg+QP88fYiGW7ItyKCqJGfe8hC3Cb9pJ3W25QUp2VOnO5xup Ljq8pDsNTfE5RjX+e4PAajNZIes7UEguSbzBfiFSoYb89QCap0b+iaBPT62T6VqEh5J8tfL0vDZkS oOEkl7j7qKSn4mry8vimC2auW44jILDO6zm+y2G4JWh4bXl7waRCAp9g267erifRSO904noQ3jyVZ wrL9JJC7leaWYYONXmrXOEqLCqMldMl+EBjIzvAsUD1Ob8JE+QX7HyQ2Bpdeq0hlPekTVcqwMeZBj Z6rHITY0+Irq68b2vqgOSjD7nnxaASQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r6Dku-005SCa-2y; Thu, 23 Nov 2023 17:44:48 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r6Dks-005SAP-00 for linux-arm-kernel@lists.infradead.org; Thu, 23 Nov 2023 17:44:47 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5c994880223so15471627b3.0 for ; Thu, 23 Nov 2023 09:44:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1700761483; x=1701366283; darn=lists.infradead.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=A1drm/XxoRhT1Y4OhS52yTKtUYazjfVWJpVHhF1knKg=; b=iZbBGfHmR8fO3fK5cGyJ6GaJ0JPoz6r1pCKataDfppVKbJc8yLh++/cfiMkH9LwqOB RYhnS/9FyY0l27nYokFwHmaGKM1EtvmS2vp9lbwhtdmiwVKEooWe/k++arcMq30VlRde WNy0oHqtpJPBj3kRCTVZ29+vGSLhXu1Q/m/8KUJ9IcclzLDRqutqptD+fOc03usFtxrS krg3M1GC8eOq6w+IOKWiciYXcJaFxlSuQuxDGOK2eUivXlYZmt/oQC43EBUTNiaeyPFw zjqtcL1ZOrH47BdDwUIxXwTVTpCU2I8xXzrsgYZXUytGtrUJhL7XDhX0krFdn24kV+1h gwRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700761483; x=1701366283; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=A1drm/XxoRhT1Y4OhS52yTKtUYazjfVWJpVHhF1knKg=; b=f9MLHZIXiIogcrS2c8CHj+HPY/ZN48rL6nQdELgpXg8ivof1m4TBY303VnZ0nERxtk t7ai9y32WsraPbQ2FEqXLCq1OggDOe1D7DSAVHtUOnwWOgL5Wg16iJQUhS1DrimOifzO hp+4IADki7AzypiUPMdoCRuXinKmanwTRXRh2jh+B6UeQIzPcGL0NmYyPVhhCP4e14JN jExQtSnyl/1j3lBJzGCVgKu0aLRSvsq0UvVj+oqLjQjntrChZtdwgYYCPmIZbLAJApu4 C4Uo4uBu+/riQlyyf7aUpwt7nUORgLMd+Y51eA23Yb+unuwtZ6glFlBLrq25UDB5c/cd pIYQ== X-Gm-Message-State: AOJu0Yw7dtBnFBDFUaJPKGRhxIhxgV3IS5sOC1VkvFb9qOrtY+2zO3Py 8luox8Slsrka6Y2+KszF9/iHIbc3/3Te+6nYvmmA4cZgsq2ctRWFWA5oPUsLlkmfDWzhCyQRSUj jzCQfzAe3dhXOSs1HrLwC37ZVbdES1qNV9GPBNmMFO4TCuFoKfn725j+glmWqay98zs4AimG+Vu o= X-Google-Smtp-Source: AGHT+IGik5qZQtC31htAJz0B3lUREVaS97lquiJioB5nL2R3zYQssE+OFvG5bGPCuQ0qt3AanKuCiAsh X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a25:cf8e:0:b0:da0:5d66:4ded with SMTP id f136-20020a25cf8e000000b00da05d664dedmr184ybg.2.1700761483242; Thu, 23 Nov 2023 09:44:43 -0800 (PST) Date: Thu, 23 Nov 2023 18:44:34 +0100 Mime-Version: 1.0 X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3470; i=ardb@kernel.org; h=from:subject; bh=fvu73qQ/L1cEVHGLK0S21tYlAlGDstp1MYIVmKjeihk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JITW+v8m74v8+JrO6tyqsPS287zU3bw19tjdY+lbItVsR7 Oz//67vKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABMRuczIMHGyrrFJ40HnU7ei PM+qPM/YEO77eOXmt3Pmf1BtvDdVwZKR4YTx89+ro7mlDsYVx808YT3heK3hkRjXJuWjj0KKn16 fxg4A X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog Message-ID: <20231123174433.737171-6-ardb@google.com> Subject: [PATCH v2 0/4] arm64: Run kernel mode NEON with preemption enabled From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown , Eric Biggers X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231123_094446_060627_25DE8BAE X-CRM114-Status: GOOD ( 18.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Currently, kernel mode NEON (SIMD) support is implemented in a way that requires preemption to be disabled while the SIMD registers are live. The reason for this is that those registers are not in the set that is preserved/restored on exception entry/exit and context switch, as this would impact performance generally, even for workloads where kernel mode SIMD is not the bottleneck. However, doing substantial work with preemption disabled is not great, as it affects scheduling latency, which is especially problematic for real-time use cases. So ideally, we should keep preemption enabled when we can, and find another way to ensure that this does not corrupt the NEON register state of in-kernel SIMD users. This series implements a suggestion by Mark Rutland, and introduces a thread_info flag TIF_USING_KMODE_NEON, which indicates to the thread switch machinery that the task in question has live kernel mode SIMD state which needs to be preserved and restored. The space needed for this is allocated in thread_struct. (*) Given that currently, we run kernel mode NEON with softirqs disabled (to avoid the need for preserving kernel mode NEON context belonging to task context while the SIMD unit is being used by code running in softirq context), just removing the preempt_disable/enable calls is not sufficient, and we also need to leave softirqs enabled. This means that we may need to preserve kernel mode NEON state not only on a context switch, but also when code running in softirq context takes ownership of the SIMD unit, but this is straight-forward once we add the scratch space to thread_struct. (*) We might decide to allocate this space (~512 bytes) dynamically, if the thread_struct memory footprint causes issues. However, we should also explore doing the same for the user space FPSIMD state, as kernel threads never return to user space and have no need for this allocation. v2: - tweak some commit logs for clarity - integrate with the existing lazy restore logic - add Mark's R-b to patch #1 Cc: Marc Zyngier Cc: Will Deacon Cc: Mark Rutland Cc: Kees Cook Cc: Catalin Marinas Cc: Mark Brown Cc: Eric Biggers Ard Biesheuvel (4): arm64: fpsimd: Drop unneeded 'busy' flag arm64: fpsimd: Preserve/restore kernel mode NEON at context switch arm64: fpsimd: Implement lazy restore for kernel mode FPSIMD arm64: crypto: Remove conditional yield logic arch/arm64/crypto/aes-glue.c | 21 +-- arch/arm64/crypto/aes-modes.S | 2 - arch/arm64/crypto/sha1-ce-core.S | 2 - arch/arm64/crypto/sha1-ce-glue.c | 19 +-- arch/arm64/crypto/sha2-ce-core.S | 2 - arch/arm64/crypto/sha2-ce-glue.c | 19 +-- arch/arm64/crypto/sha3-ce-core.S | 4 +- arch/arm64/crypto/sha3-ce-glue.c | 14 +- arch/arm64/crypto/sha512-ce-core.S | 2 - arch/arm64/crypto/sha512-ce-glue.c | 16 +- arch/arm64/include/asm/assembler.h | 29 ---- arch/arm64/include/asm/processor.h | 3 + arch/arm64/include/asm/simd.h | 11 +- arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/asm-offsets.c | 4 - arch/arm64/kernel/fpsimd.c | 162 +++++++++++++------- 16 files changed, 138 insertions(+), 173 deletions(-) -- 2.43.0.rc1.413.gea7ed67945-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel