From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E434FC433E1 for ; Mon, 15 Jun 2020 13:27:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C39D7207BB for ; Mon, 15 Jun 2020 13:27:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592227666; bh=KiwUQT9DqNGgzQev4bi3zf2yZCEMD/qEftIYgUxUgtk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=AUaIJeSDJDS0WK5W7efp4MUlhTnx19bNRjhM5veYt/1OF/x6AgXFi/tc7tFn2XMwO 4/VpPNuhVHum0JJs8FNKMUpwfkEMral9VzCdSpRtCBe9Hbh4G+Zr5L/3TMeBstY5l2 pxQ4cMi42cAmp949yvdiGsUhtZnTKokmJA3cv434= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730476AbgFON1p (ORCPT ); Mon, 15 Jun 2020 09:27:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:58598 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730342AbgFON1l (ORCPT ); Mon, 15 Jun 2020 09:27:41 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8CB8C207DA; Mon, 15 Jun 2020 13:27:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592227660; bh=KiwUQT9DqNGgzQev4bi3zf2yZCEMD/qEftIYgUxUgtk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=naTHiI+C89GafUlY4vag/hTbnLHeBe5OOspNwKT+HBEyiPW+Q1V3kGXQf1XE5PA7k JCgjcO/NKcpVrZzcMUg6QnzdeufD+Vo741HkPYY+thkkrT1vjfVtcFkJcIaq/biET7 5HBYvNz0ecrg+OzF1mDQAQEmPavnAtFDPCN7PEr8= Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jkp9H-0036w9-3n; Mon, 15 Jun 2020 14:27:39 +0100 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Andre Przywara , Christoffer Dall , Dave Martin , Jintack Lim , Alexandru Elisei , George Cherian , "Zengtao (B)" , Andrew Scull , Will Deacon , Catalin Marinas , Mark Rutland , James Morse , Julien Thierry , Suzuki K Poulose , kernel-team@android.com Subject: [PATCH v2 04/17] arm64: Add level-hinted TLB invalidation helper Date: Mon, 15 Jun 2020 14:27:06 +0100 Message-Id: <20200615132719.1932408-5-maz@kernel.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200615132719.1932408-1-maz@kernel.org> References: <20200615132719.1932408-1-maz@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, andre.przywara@arm.com, christoffer.dall@arm.com, Dave.Martin@arm.com, jintack@cs.columbia.edu, alexandru.elisei@arm.com, gcherian@marvell.com, prime.zeng@hisilicon.com, ascull@google.com, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a level-hinted TLB invalidation helper that only gets used if ARMv8.4-TTL gets detected. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/stage2_pgtable.h | 9 +++++ arch/arm64/include/asm/tlbflush.h | 45 +++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h index b767904f28b1..996bf98f0cab 100644 --- a/arch/arm64/include/asm/stage2_pgtable.h +++ b/arch/arm64/include/asm/stage2_pgtable.h @@ -256,4 +256,13 @@ stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end) return (boundary - 1 < end - 1) ? boundary : end; } +/* + * Level values for the ARMv8.4-TTL extension, mapping PUD/PMD/PTE and + * the architectural page-table level. + */ +#define S2_NO_LEVEL_HINT 0 +#define S2_PUD_LEVEL 1 +#define S2_PMD_LEVEL 2 +#define S2_PTE_LEVEL 3 + #endif /* __ARM64_S2_PGTABLE_H_ */ diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index bc3949064725..e05c31fd0bbc 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -10,6 +10,7 @@ #ifndef __ASSEMBLY__ +#include #include #include #include @@ -59,6 +60,50 @@ __ta; \ }) +/* + * Level-based TLBI operations. + * + * When ARMv8.4-TTL exists, TLBI operations take an additional hint for + * the level at which the invalidation must take place. If the level is + * wrong, no invalidation may take place. In the case where the level + * cannot be easily determined, a 0 value for the level parameter will + * perform a non-hinted invalidation. + * + * For Stage-2 invalidation, use the level values provided to that effect + * in asm/stage2_pgtable.h. + */ +#define TLBI_TTL_MASK GENMASK_ULL(47, 44) +#define TLBI_TTL_PS_4K 1 +#define TLBI_TTL_PS_16K 2 +#define TLBI_TTL_PS_64K 3 + +#define __tlbi_level(op, addr, level) \ + do { \ + u64 arg = addr; \ + \ + if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && \ + level) { \ + u64 ttl = level & 3; \ + \ + switch (PAGE_SIZE) { \ + case SZ_4K: \ + ttl |= TLBI_TTL_PS_4K << 2; \ + break; \ + case SZ_16K: \ + ttl |= TLBI_TTL_PS_16K << 2; \ + break; \ + case SZ_64K: \ + ttl |= TLBI_TTL_PS_64K << 2; \ + break; \ + } \ + \ + arg &= ~TLBI_TTL_MASK; \ + arg |= FIELD_PREP(TLBI_TTL_MASK, ttl); \ + } \ + \ + __tlbi(op, arg); \ + } while(0) + /* * TLB Invalidation * ================ -- 2.27.0