From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.28.71.27 with SMTP id u27csp2109153wma; Sat, 17 Feb 2018 11:19:56 -0800 (PST) X-Google-Smtp-Source: AH8x227YC9hs7uTHuIhFXG5GU56Fh+qXqiFyfoRO89ex73rGIbmaYSI77Bjs76Sfhz93dgRo3iJe X-Received: by 10.37.181.17 with SMTP id p17mr7376708ybj.481.1518895196871; Sat, 17 Feb 2018 11:19:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518895196; cv=none; d=google.com; s=arc-20160816; b=lAA5i1et/YyOZogJzF9/vkm1mfKiKG1hCERlLmM+IAQofRS0oD6AwOsKshRsOd/9pR 65UMUqnpuS3HGvRAm3UPabWJBJftwt8su8XFLMGjQs2lwbqPgLPkOeVK7b/ddtrVkx2T p31vimKcNQY8id05v0CEubwK9xNa7N5hFeE8fTjOrfdwAd/bp4rKmNCvZXD1Sz8/ZcYh Wny+RVD67614u8DvvweUrmAr8gUXyFyzD9QZ/qWprxlfGYl5myt34Vyu4ceOVc8R0LXB UUqeuEDRdPXZSIuk2AS3e/zQkks7bTnu8QijW93A2iXyQk/SJv+bRf3WKPD/+nNXw5bI Qoww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:arc-authentication-results; bh=loVmdVNJ2wx45rD1Nxk1sERsEiXVydg80iOXQvSL6ew=; b=i1z7wjxenCTcgCXsVxtlVZGryNZneJNQVqT3QuEfTQmI1GDf1Ctnyjz4l1U0pLPiCR DLDMvWMkY9RqRZ2rMoAGcoIURxLaTN6p167n9rg8tyuStY12qDWlPjo7K41lkoiG4gu5 BOJRCKwC1zle8kej6R+1EO7QE24ekROmhvLZT/TU60umSXTmd3k1P5uNXC6UFtoIhRjX Fv+kMGPJPyYVsGyMQeblHVReDg1/nkgatY0aQ4GAY72aZZKR4Jj8OMQB9hBgFM6/pvjd f2nHZjYDBLxps1R05g5ZWdddqR9pFCT4bAkOZLv3zl0IPcHnaY22Zgky5kkASkfyDyvf /kqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id e1si1057816ybm.181.2018.02.17.11.19.56 for (version=TLS1 cipher=AES128-SHA bits=128/128); Sat, 17 Feb 2018 11:19:56 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from localhost ([::1]:49481 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1en81c-0007G9-6U for alex.bennee@linaro.org; Sat, 17 Feb 2018 14:19:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1en7W1-0003GA-Vg for qemu-arm@nongnu.org; Sat, 17 Feb 2018 13:47:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1en7Vz-0000i3-G4 for qemu-arm@nongnu.org; Sat, 17 Feb 2018 13:47:18 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:36078 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1en7Vz-0000gi-Ay; Sat, 17 Feb 2018 13:47:15 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C6821EAE83; Sat, 17 Feb 2018 18:47:14 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-21.ams2.redhat.com [10.36.116.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id 108CF2024CA2; Sat, 17 Feb 2018 18:47:11 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, peter.maydell@linaro.org, qemu-arm@nongnu.org, qemu-devel@nongnu.org, prem.mallappa@gmail.com, alex.williamson@redhat.com Date: Sat, 17 Feb 2018 19:46:45 +0100 Message-Id: <1518893216-9983-4-git-send-email-eric.auger@redhat.com> In-Reply-To: <1518893216-9983-1-git-send-email-eric.auger@redhat.com> References: <1518893216-9983-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Sat, 17 Feb 2018 18:47:14 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Sat, 17 Feb 2018 18:47:14 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'eric.auger@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-arm] [PATCH v9 03/14] hw/arm/smmu-common: VMSAv8-64 page table walk X-BeenThere: qemu-arm@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mst@redhat.com, jean-philippe.brucker@arm.com, tn@semihalf.com, peterx@redhat.com, linuc.decode@gmail.com, bharat.bhushan@nxp.com, christoffer.dall@linaro.org Errors-To: qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org Sender: "Qemu-arm" X-TUID: abltVLHs+pKU This patch implements the page table walk for VMSAv8-64. Signed-off-by: Eric Auger --- v8 -> v9: - remove guest error log on PTE fetch fault - rename trace functions - fix smmu_page_walk_level_res_invalid_pte last arg - fix PTE_ADDRESS - turn functions into macros - make sure to return the actual pte access permission into tlbe->perm - change proto of smmu_ptw* v7 -> v8: - rework get_pte - use LOG_LEVEL_ERROR - remove error checking in get_block_pte_address - page table walk simplified (no VFIO replay anymore) - handle PTW error events - use dma_memory_read v6 -> v7: - fix wrong error handling in walk_page_table - check perm in smmu_translate v5 -> v6: - use IOMMUMemoryRegion - remove initial_lookup_level() - fix block replay v4 -> v5: - add initial level in translation config - implement block pte - rename must_translate into nofail - introduce call_entry_hook - small changes to dynamic traces - smmu_page_walk code moved from smmuv3.c to this file - remove smmu_translate* v3 -> v4: - reworked page table walk to prepare for VFIO integration (capability to scan a range of IOVA). Same function is used for translate for a single iova. This is largely inspired from intel_iommu.c - as the translate function was not straightforward to me, I tried to stick more closely to the VMSA spec. - remove support of nested stage (kernel driver does not support it anyway) - use error_report and trace events - add aa64[] field in SMMUTransCfg --- hw/arm/smmu-common.c | 232 +++++++++++++++++++++++++++++++++++++++++++ hw/arm/smmu-internal.h | 96 ++++++++++++++++++ hw/arm/trace-events | 10 ++ include/hw/arm/smmu-common.h | 6 ++ 4 files changed, 344 insertions(+) create mode 100644 hw/arm/smmu-internal.h diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index d0516dc..24cc4ba 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -27,6 +27,238 @@ #include "qemu/error-report.h" #include "hw/arm/smmu-common.h" +#include "smmu-internal.h" + +/* VMSAv8-64 Translation */ + +/** + * get_pte - Get the content of a page table entry located t + * @base_addr[@index] + */ +static int get_pte(dma_addr_t baseaddr, uint32_t index, uint64_t *pte, + SMMUPTWEventInfo *info) +{ + int ret; + dma_addr_t addr = baseaddr + index * sizeof(*pte); + + ret = dma_memory_read(&address_space_memory, addr, + (uint8_t *)pte, sizeof(*pte)); + + if (ret != MEMTX_OK) { + info->type = SMMU_PTW_ERR_WALK_EABT; + info->addr = addr; + return -EINVAL; + } + trace_smmu_get_pte(baseaddr, index, addr, *pte); + return 0; +} + +/* VMSAv8-64 Translation Table Format Descriptor Decoding */ + +/** + * get_page_pte_address - returns the L3 descriptor output address, + * ie. the page frame + * ARM ARM spec: Figure D4-17 VMSAv8-64 level 3 descriptor format + */ +static inline hwaddr get_page_pte_address(uint64_t pte, int granule_sz) +{ + return PTE_ADDRESS(pte, granule_sz); +} + +/** + * get_table_pte_address - return table descriptor output address, + * ie. address of next level table + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats + */ +static inline hwaddr get_table_pte_address(uint64_t pte, int granule_sz) +{ + return PTE_ADDRESS(pte, granule_sz); +} + +/** + * get_block_pte_address - return block descriptor output address and block size + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats + */ +static hwaddr get_block_pte_address(uint64_t pte, int level, int granule_sz, + uint64_t *bsz) +{ + int n = 0; + + switch (granule_sz) { + case 12: + if (level == 1) { + n = 30; + } else if (level == 2) { + n = 21; + } + break; + case 14: + if (level == 2) { + n = 25; + } + break; + case 16: + if (level == 2) { + n = 29; + } + break; + } + if (!n) { + error_setg(&error_fatal, + "wrong granule/level combination (%d/%d)", + granule_sz, level); + } + *bsz = 1 << n; + return PTE_ADDRESS(pte, n); +} + +static inline bool check_perm(int access_attrs, int mem_attrs) +{ + if (((access_attrs & IOMMU_RO) && !(mem_attrs & IOMMU_RO)) || + ((access_attrs & IOMMU_WO) && !(mem_attrs & IOMMU_WO))) { + return false; + } + return true; +} + +SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) +{ + if (!extract64(iova, 64 - cfg->tt[0].tsz, cfg->tt[0].tsz - cfg->tbi)) { + return &cfg->tt[0]; + } + return &cfg->tt[1]; +} + +/** + * smmu_ptw_64 - VMSAv8-64 Walk of the page tables for a given IOVA + * @cfg: translation config + * @iova: iova to translate + * @perm: access type + * @tlbe: IOMMUTLBEntry (out) + * @info: handle to an error info + * + * Return 0 on success, < 0 on error. In case of error, @info is filled + * and tlbe->perm is set to IOMMU_NONE. + * Upon success, @tlbe is filled with translated_addr and entry + * permission rights. + */ +static int smmu_ptw_64(SMMUTransCfg *cfg, + dma_addr_t iova, IOMMUAccessFlags perm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +{ + dma_addr_t baseaddr; + int stage = cfg->stage; + SMMUTransTableInfo *tt = select_tt(cfg, iova); + uint8_t level; + uint8_t granule_sz; + + if (tt->disabled) { + info->type = SMMU_PTW_ERR_TRANSLATION; + goto error; + } + + level = tt->initial_level; + granule_sz = tt->granule_sz; + baseaddr = extract64(tt->ttb, 0, 48); + + tlbe->iova = iova; + tlbe->addr_mask = (1 << tt->granule_sz) - 1; + + while (level <= 3) { + uint64_t subpage_size = 1ULL << level_shift(level, granule_sz); + uint64_t mask = subpage_size - 1; + uint32_t offset = iova_level_offset(iova, level, granule_sz); + uint64_t pte; + dma_addr_t pte_addr = baseaddr + offset * sizeof(pte); + uint8_t ap; + + if (get_pte(baseaddr, offset, &pte, info)) { + goto error; + } + trace_smmu_ptw_level(level, iova, subpage_size, + baseaddr, offset, pte); + + if (is_invalid_pte(pte) || is_reserved_pte(pte, level)) { + trace_smmu_ptw_invalid_pte(stage, level, baseaddr, + pte_addr, offset, pte); + info->type = SMMU_PTW_ERR_TRANSLATION; + goto error; + } + + if (is_page_pte(pte, level)) { + uint64_t gpa = get_page_pte_address(pte, granule_sz); + + ap = PTE_AP(pte); + if (is_permission_fault(ap, perm)) { + info->type = SMMU_PTW_ERR_PERMISSION; + goto error; + } + + tlbe->translated_addr = gpa + (iova & mask); + tlbe->perm = PTE_AP_TO_PERM(ap); + trace_smmu_ptw_page_pte(stage, level, iova, + baseaddr, pte_addr, pte, gpa); + return 0; + } + if (is_block_pte(pte, level)) { + uint64_t block_size; + hwaddr gpa = get_block_pte_address(pte, level, granule_sz, + &block_size); + + ap = PTE_AP(pte); + if (is_permission_fault(ap, perm)) { + info->type = SMMU_PTW_ERR_PERMISSION; + goto error; + } + + trace_smmu_ptw_block_pte(stage, level, baseaddr, + pte_addr, pte, iova, gpa, + (int)(block_size >> 20)); + + tlbe->translated_addr = gpa + (iova & mask); + tlbe->perm = PTE_AP_TO_PERM(ap); + return 0; + } + + /* table pte */ + ap = PTE_APTABLE(pte); + + if (is_permission_fault(ap, perm)) { + info->type = SMMU_PTW_ERR_PERMISSION; + goto error; + } + baseaddr = get_table_pte_address(pte, granule_sz); + level++; + } + + info->type = SMMU_PTW_ERR_TRANSLATION; + +error: + tlbe->perm = IOMMU_NONE; + return -EINVAL; +} + +/** + * smmu_ptw - Walk the page tables for an IOVA, according to @cfg + * + * @cfg: translation configuration + * @iova: iova to translate + * @perm: tentative access type + * @tlbe: returned entry + * @info: ptw event handle + * + * return 0 on success + */ +int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) +{ + if (!cfg->aa64) { + error_setg(&error_fatal, + "SMMUv3 model does not support VMSAv8-32 page walk yet"); + } + + return smmu_ptw_64(cfg, iova, perm, tlbe, info); +} SMMUPciBus *smmu_find_as_from_bus_num(SMMUState *s, uint8_t bus_num) { diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h new file mode 100644 index 0000000..3ed97ee --- /dev/null +++ b/hw/arm/smmu-internal.h @@ -0,0 +1,96 @@ +/* + * ARM SMMU support - Internal API + * + * Copyright (c) 2017 Red Hat, Inc. + * Copyright (C) 2014-2016 Broadcom Corporation + * Written by Prem Mallappa, Eric Auger + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#ifndef HW_ARM_SMMU_INTERNAL_H +#define HW_ARM_SMMU_INTERNAL_H + +#define ARM_LPAE_MAX_ADDR_BITS 48 +#define ARM_LPAE_MAX_LEVELS 4 + +/* PTE Manipulation */ + +#define ARM_LPAE_PTE_TYPE_SHIFT 0 +#define ARM_LPAE_PTE_TYPE_MASK 0x3 + +#define ARM_LPAE_PTE_TYPE_BLOCK 1 +#define ARM_LPAE_PTE_TYPE_TABLE 3 + +#define ARM_LPAE_L3_PTE_TYPE_RESERVED 1 +#define ARM_LPAE_L3_PTE_TYPE_PAGE 3 + +#define ARM_LPAE_PTE_VALID (1 << 0) + +#define PTE_ADDRESS(pte, shift) \ + (extract64(pte, shift, 47 - shift + 1) << shift) + +#define is_invalid_pte(pte) (!(pte & ARM_LPAE_PTE_VALID)) + +#define is_reserved_pte(pte, level) \ + ((level == 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_L3_PTE_TYPE_RESERVED)) + +#define is_block_pte(pte, level) \ + ((level < 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_BLOCK)) + +#define is_table_pte(pte, level) \ + ((level < 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_TABLE)) + +#define is_page_pte(pte, level) \ + ((level == 3) && \ + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_L3_PTE_TYPE_PAGE)) + +#define PTE_AP(pte) \ + (extract64(pte, 6, 2)) + +#define PTE_APTABLE(pte) \ + (extract64(pte, 61, 2)) + +#define is_permission_fault(ap, perm) \ + (((perm) & IOMMU_WO) && ((ap) & 0x2)) + +#define PTE_AP_TO_PERM(ap) \ + (IOMMU_ACCESS_FLAG(true, !((ap) & 0x2))) + +/* Level Indexing */ + +static inline int level_shift(int level, int granule_sz) +{ + return granule_sz + (3 - level) * (granule_sz - 3); +} + +static inline uint64_t level_page_mask(int level, int granule_sz) +{ + return ~((1ULL << level_shift(level, granule_sz)) - 1); +} + +/** + * TODO: handle the case where the level resolves less than + * granule_sz -3 IA bits. + */ +static inline +uint64_t iova_level_offset(uint64_t iova, int level, int granule_sz) +{ + return (iova >> level_shift(level, granule_sz)) & + ((1ULL << (granule_sz - 3)) - 1); +} + +#endif diff --git a/hw/arm/trace-events b/hw/arm/trace-events index 193063e..3584974 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -2,3 +2,13 @@ # hw/arm/virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." + +# hw/arm/smmu-common.c + +smmu_page_walk(int stage, uint64_t baseaddr, int first_level, uint64_t start, uint64_t end) "stage=%d, baseaddr=0x%"PRIx64", first level=%d, start=0x%"PRIx64", end=0x%"PRIx64 +smmu_lookup_table(int level, uint64_t baseaddr, int granule_sz, uint64_t start, uint64_t end, int flags, uint64_t subpage_size) "level=%d baseaddr=0x%"PRIx64" granule=%d, start=0x%"PRIx64" end=0x%"PRIx64" flags=%d subpage_size=0x%lx" +smmu_ptw_level(int level, uint64_t iova, size_t subpage_size, uint64_t baseaddr, uint32_t offset, uint64_t pte) "level=%d iova=0x%lx subpage_sz=0x%lx baseaddr=0x%"PRIx64" offset=%d => pte=0x%lx" +smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint32_t offset, uint64_t pte) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" offset=%d pte=0x%"PRIx64 +smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=%d level=%d iova=0x%"PRIx64" base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" page address = 0x%"PRIx64 +smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" iova=0x%"PRIx64" block address = 0x%"PRIx64" block size = %d MiB" +smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "baseaddr=0x%"PRIx64" index=0x%x, pteaddr=0x%"PRIx64", pte=0x%"PRIx64 diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index aee96c2..0fb27f7 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -127,4 +127,10 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev) { return ((pci_bus_num(sdev->bus) & 0xff) << 8) | sdev->devfn; } + +int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, + IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info); + +SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova); + #endif /* HW_ARM_SMMU_COMMON */ -- 2.5.5