From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36E6DC43216 for ; Mon, 26 Jul 2021 14:11:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1AD8860F44 for ; Mon, 26 Jul 2021 14:11:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234316AbhGZNb2 (ORCPT ); Mon, 26 Jul 2021 09:31:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:42720 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234183AbhGZNbU (ORCPT ); Mon, 26 Jul 2021 09:31:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2864660F44; Mon, 26 Jul 2021 14:11:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627308708; bh=5kcXJiHE0PQpWAyz9VUZ4OV94qxkV3Y2oDR5o1D2FOk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=edl2wetoyjeM475gLpRFpus5Lw3iCGQUSLyjx59TAayU1mfJZVjoegxXh7crKj/Fv DwRwg4HYWIj5I71VkADRX3WqOpuJxMCbJF9wvdQ/3HcoITTfnTUPrfhDgCKIcIijFi 1DcvZTIZa1t1cfhx3LFoLesc4BGq76w5KtdQf/UzLnVL5XuuAr7FlYvDLXvHmx70GZ OfzBk+yr2qlGUc4S8Hla4OaI7+ZIuF5yR6gqAQh6tR7OSl1KkFmAS6evg30YOf4E+T MBpnsrMi+Trz1apkV3orFWPMK/wMVeMuI6J9P8UasfJa9G3HpEtTGBFpj2MT6ehu0I qMfyHcos58hyg== From: Arnd Bergmann To: Russell King Cc: Arnd Bergmann , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, Alexander Viro , Linus Walleij , Christoph Hellwig Subject: [PATCH v5 01/10] mm/maccess: fix unaligned copy_{from,to}_kernel_nofault Date: Mon, 26 Jul 2021 16:11:32 +0200 Message-Id: <20210726141141.2839385-2-arnd@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210726141141.2839385-1-arnd@kernel.org> References: <20210726141141.2839385-1-arnd@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Arnd Bergmann On machines such as ARMv5 that trap unaligned accesses, these two functions can be slow when each access needs to be emulated, or they might not work at all. Change them so that each loop is only used when both the src and dst pointers are naturally aligned. Reviewed-by: Christoph Hellwig Reviewed-by: Linus Walleij Signed-off-by: Arnd Bergmann --- mm/maccess.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/maccess.c b/mm/maccess.c index 3bd70405f2d8..d3f1a1f0b1c1 100644 --- a/mm/maccess.c +++ b/mm/maccess.c @@ -24,13 +24,21 @@ bool __weak copy_from_kernel_nofault_allowed(const void *unsafe_src, long copy_from_kernel_nofault(void *dst, const void *src, size_t size) { + unsigned long align = 0; + + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)) + align = (unsigned long)dst | (unsigned long)src; + if (!copy_from_kernel_nofault_allowed(src, size)) return -ERANGE; pagefault_disable(); - copy_from_kernel_nofault_loop(dst, src, size, u64, Efault); - copy_from_kernel_nofault_loop(dst, src, size, u32, Efault); - copy_from_kernel_nofault_loop(dst, src, size, u16, Efault); + if (!(align & 7)) + copy_from_kernel_nofault_loop(dst, src, size, u64, Efault); + if (!(align & 3)) + copy_from_kernel_nofault_loop(dst, src, size, u32, Efault); + if (!(align & 1)) + copy_from_kernel_nofault_loop(dst, src, size, u16, Efault); copy_from_kernel_nofault_loop(dst, src, size, u8, Efault); pagefault_enable(); return 0; @@ -50,10 +58,18 @@ EXPORT_SYMBOL_GPL(copy_from_kernel_nofault); long copy_to_kernel_nofault(void *dst, const void *src, size_t size) { + unsigned long align = 0; + + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)) + align = (unsigned long)dst | (unsigned long)src; + pagefault_disable(); - copy_to_kernel_nofault_loop(dst, src, size, u64, Efault); - copy_to_kernel_nofault_loop(dst, src, size, u32, Efault); - copy_to_kernel_nofault_loop(dst, src, size, u16, Efault); + if (!(align & 7)) + copy_to_kernel_nofault_loop(dst, src, size, u64, Efault); + if (!(align & 3)) + copy_to_kernel_nofault_loop(dst, src, size, u32, Efault); + if (!(align & 1)) + copy_to_kernel_nofault_loop(dst, src, size, u16, Efault); copy_to_kernel_nofault_loop(dst, src, size, u8, Efault); pagefault_enable(); return 0; -- 2.29.2