* [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one
@ 2015-02-25 18:52 Julien Grall
2015-02-25 18:52 ` [PATCH v4 1/8] xen/iommu: arm: Remove temporary the SMMU driver Julien Grall
` (8 more replies)
0 siblings, 9 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, manish.jaggi, Julien Grall, tim, ian.campbell
Hello,
The SMMU driver has diverged from Linux. Having our own driver doesn't make
any benefits and make difficult to backport fixes and/or porting features
such as PCI.
With this series, the code of the SMMU drivers (copied from Linux) is mostly
not modified. If it's the case a comment /* Xen: ... */ has been added to
explain why. The driver is based on Linux 3.19. Even if they reworked the SMMU
code in 4.0 (page table code has been moved in a separate library), I think
we should stay on 3.19 version. Mainly because I don't want to spend again a
week to resync and Linux drivers can change fast.
To make the change obvious, the resync of the SMMU code is made in several
steps:
1) Revert the current SMMU driver (patch #1)
2) Import as it is the driver from Linux (patch #5)
3) Changes for Xen (patch #6)
I also took the opportunity of the resync to consolidate the iommu ops in a
single set. When I added the IOMMU set to handle platform device passthrough
(ops assign_dt_device and reassign_dt_device), I didn't think about merging
the ops with the PCI one. In fact Linux is using a signel set and have only
few lines per driver specific to each set (PCI or platform device).
A branch is available with all changes:
git://xenbits.xen.org/people/julieng/xen-unstable.git branch smmu-rework-v4
Major changes in v4:
- Drop the support of midway in the patch which will be upstreamed.
- Re-order/re-work the SMMU changes in order to modify less the Linux code.
Major changes in v3:
- Add back coherent walk support (see patch #13)
Major changes in v2:
- Introduce the geneirc struct to describe device on ARM. Alias it
to pci_dev on x86.
- Defer the introduction of PCI in the generic device later
For all changes see each patches.
I have summarized below the patch acked/modified:
A: Patch acked
M: Patch modified in this version
Sincerely yours,
Julien Grall (8):
xen/iommu: arm: Remove temporary the SMMU driver
xen/arm: Introduce a generic way to describe device
xen/iommu: Consolidate device assignment ops into a single set
xen/arm: Describe devices supported by a driver with dt_device_match
xen/iommu: arm: Import the SMMU driver from Linux
xen/iommu: smmu: Add Xen specific code to be able to use the driver
xen/iommu: smmu: Advertise when the SMMU support coherent table walk
DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU
xen/arch/arm/device.c | 21 +-
xen/arch/arm/gic-v2.c | 10 +-
xen/arch/arm/gic-v3.c | 8 +-
xen/common/device_tree.c | 3 +
xen/drivers/char/exynos4210-uart.c | 8 +-
xen/drivers/char/ns16550.c | 12 +-
xen/drivers/char/omap-uart.c | 8 +-
xen/drivers/char/pl011.c | 8 +-
xen/drivers/char/scif-uart.c | 8 +-
xen/drivers/passthrough/arm/smmu.c | 4057 +++++++++++++++++++++------------
xen/drivers/passthrough/device_tree.c | 5 +-
xen/drivers/passthrough/pci.c | 20 +-
xen/include/asm-arm/device.h | 30 +-
xen/include/asm-arm/gic.h | 15 +-
xen/include/asm-x86/device.h | 25 +
xen/include/xen/device_tree.h | 13 +
xen/include/xen/iommu.h | 18 +-
xen/include/xen/pci.h | 1 +
18 files changed, 2672 insertions(+), 1598 deletions(-)
create mode 100644 xen/include/asm-x86/device.h
--
2.1.4
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v4 1/8] xen/iommu: arm: Remove temporary the SMMU driver
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-02-25 18:52 ` [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device Julien Grall
` (7 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, manish.jaggi, Julien Grall, tim,
stefano.stabellini
The current SMMU driver has completly diverged. That makes me hard to
maintain.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
Currently none of the platform used on OSStest has SMMU nodes
described in the device tree. Therefore, the bisector won't be
impacted by this changeset.
Changes in v4:
- Add Ian's ack
Changes in v3:
- Add Stefano's ack
---
xen/drivers/passthrough/arm/Makefile | 1 -
xen/drivers/passthrough/arm/smmu.c | 1784 ----------------------------------
2 files changed, 1785 deletions(-)
delete mode 100644 xen/drivers/passthrough/arm/smmu.c
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index f4cd26e..0484b79 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -1,2 +1 @@
obj-y += iommu.o
-obj-y += smmu.o
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
deleted file mode 100644
index 42bde75..0000000
--- a/xen/drivers/passthrough/arm/smmu.c
+++ /dev/null
@@ -1,1784 +0,0 @@
-/*
- * IOMMU API for ARM architected SMMU implementations.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
- *
- * Based on Linux drivers/iommu/arm-smmu.c (commit 89a23cd)
- * Copyright (C) 2013 ARM Limited
- *
- * Author: Will Deacon <will.deacon@arm.com>
- *
- * Xen modification:
- * Julien Grall <julien.grall@linaro.org>
- * Copyright (C) 2014 Linaro Limited.
- *
- * This driver currently supports:
- * - SMMUv1 and v2 implementations (didn't try v2 SMMU)
- * - Stream-matching and stream-indexing
- * - v7/v8 long-descriptor format
- * - Non-secure access to the SMMU
- * - 4k pages, p2m shared with the processor
- * - Up to 40-bit addressing
- * - Context fault reporting
- */
-
-#include <xen/config.h>
-#include <xen/delay.h>
-#include <xen/errno.h>
-#include <xen/irq.h>
-#include <xen/lib.h>
-#include <xen/list.h>
-#include <xen/mm.h>
-#include <xen/vmap.h>
-#include <xen/rbtree.h>
-#include <xen/sched.h>
-#include <asm/atomic.h>
-#include <asm/device.h>
-#include <asm/io.h>
-#include <asm/platform.h>
-
-/* Driver options */
-#define SMMU_OPT_SECURE_CONFIG_ACCESS (1 << 0)
-
-/* Maximum number of stream IDs assigned to a single device */
-#define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
-
-/* Maximum stream ID */
-#define SMMU_MAX_STREAMIDS (PAGE_SIZE_64K - 1)
-
-/* Maximum number of context banks per SMMU */
-#define SMMU_MAX_CBS 128
-
-/* Maximum number of mapping groups per SMMU */
-#define SMMU_MAX_SMRS 128
-
-/* SMMU global address space */
-#define SMMU_GR0(smmu) ((smmu)->base)
-#define SMMU_GR1(smmu) ((smmu)->base + (smmu)->pagesize)
-
-/*
- * SMMU global address space with conditional offset to access secure aliases of
- * non-secure registers (e.g. nsCR0: 0x400, nsGFSR: 0x448, nsGFSYNR0: 0x450)
- */
-#define SMMU_GR0_NS(smmu) \
- ((smmu)->base + \
- ((smmu->options & SMMU_OPT_SECURE_CONFIG_ACCESS) \
- ? 0x400 : 0))
-
-/* Page table bits */
-#define SMMU_PTE_PAGE (((pteval_t)3) << 0)
-#define SMMU_PTE_CONT (((pteval_t)1) << 52)
-#define SMMU_PTE_AF (((pteval_t)1) << 10)
-#define SMMU_PTE_SH_NS (((pteval_t)0) << 8)
-#define SMMU_PTE_SH_OS (((pteval_t)2) << 8)
-#define SMMU_PTE_SH_IS (((pteval_t)3) << 8)
-
-#if PAGE_SIZE == PAGE_SIZE_4K
-#define SMMU_PTE_CONT_ENTRIES 16
-#elif PAGE_SIZE == PAGE_SIZE_64K
-#define SMMU_PTE_CONT_ENTRIES 32
-#else
-#define SMMU_PTE_CONT_ENTRIES 1
-#endif
-
-#define SMMU_PTE_CONT_SIZE (PAGE_SIZE * SMMU_PTE_CONT_ENTRIES)
-#define SMMU_PTE_CONT_MASK (~(SMMU_PTE_CONT_SIZE - 1))
-#define SMMU_PTE_HWTABLE_SIZE (PTRS_PER_PTE * sizeof(pte_t))
-
-/* Stage-1 PTE */
-#define SMMU_PTE_AP_UNPRIV (((pteval_t)1) << 6)
-#define SMMU_PTE_AP_RDONLY (((pteval_t)2) << 6)
-#define SMMU_PTE_ATTRINDX_SHIFT 2
-#define SMMU_PTE_nG (((pteval_t)1) << 11)
-
-/* Stage-2 PTE */
-#define SMMU_PTE_HAP_FAULT (((pteval_t)0) << 6)
-#define SMMU_PTE_HAP_READ (((pteval_t)1) << 6)
-#define SMMU_PTE_HAP_WRITE (((pteval_t)2) << 6)
-#define SMMU_PTE_MEMATTR_OIWB (((pteval_t)0xf) << 2)
-#define SMMU_PTE_MEMATTR_NC (((pteval_t)0x5) << 2)
-#define SMMU_PTE_MEMATTR_DEV (((pteval_t)0x1) << 2)
-
-/* Configuration registers */
-#define SMMU_GR0_sCR0 0x0
-#define SMMU_sCR0_CLIENTPD (1 << 0)
-#define SMMU_sCR0_GFRE (1 << 1)
-#define SMMU_sCR0_GFIE (1 << 2)
-#define SMMU_sCR0_GCFGFRE (1 << 4)
-#define SMMU_sCR0_GCFGFIE (1 << 5)
-#define SMMU_sCR0_USFCFG (1 << 10)
-#define SMMU_sCR0_VMIDPNE (1 << 11)
-#define SMMU_sCR0_PTM (1 << 12)
-#define SMMU_sCR0_FB (1 << 13)
-#define SMMU_sCR0_BSU_SHIFT 14
-#define SMMU_sCR0_BSU_MASK 0x3
-
-/* Identification registers */
-#define SMMU_GR0_ID0 0x20
-#define SMMU_GR0_ID1 0x24
-#define SMMU_GR0_ID2 0x28
-#define SMMU_GR0_ID3 0x2c
-#define SMMU_GR0_ID4 0x30
-#define SMMU_GR0_ID5 0x34
-#define SMMU_GR0_ID6 0x38
-#define SMMU_GR0_ID7 0x3c
-#define SMMU_GR0_sGFSR 0x48
-#define SMMU_GR0_sGFSYNR0 0x50
-#define SMMU_GR0_sGFSYNR1 0x54
-#define SMMU_GR0_sGFSYNR2 0x58
-#define SMMU_GR0_PIDR0 0xfe0
-#define SMMU_GR0_PIDR1 0xfe4
-#define SMMU_GR0_PIDR2 0xfe8
-
-#define SMMU_ID0_S1TS (1 << 30)
-#define SMMU_ID0_S2TS (1 << 29)
-#define SMMU_ID0_NTS (1 << 28)
-#define SMMU_ID0_SMS (1 << 27)
-#define SMMU_ID0_PTFS_SHIFT 24
-#define SMMU_ID0_PTFS_MASK 0x2
-#define SMMU_ID0_PTFS_V8_ONLY 0x2
-#define SMMU_ID0_CTTW (1 << 14)
-#define SMMU_ID0_NUMIRPT_SHIFT 16
-#define SMMU_ID0_NUMIRPT_MASK 0xff
-#define SMMU_ID0_NUMSMRG_SHIFT 0
-#define SMMU_ID0_NUMSMRG_MASK 0xff
-
-#define SMMU_ID1_PAGESIZE (1 << 31)
-#define SMMU_ID1_NUMPAGENDXB_SHIFT 28
-#define SMMU_ID1_NUMPAGENDXB_MASK 7
-#define SMMU_ID1_NUMS2CB_SHIFT 16
-#define SMMU_ID1_NUMS2CB_MASK 0xff
-#define SMMU_ID1_NUMCB_SHIFT 0
-#define SMMU_ID1_NUMCB_MASK 0xff
-
-#define SMMU_ID2_OAS_SHIFT 4
-#define SMMU_ID2_OAS_MASK 0xf
-#define SMMU_ID2_IAS_SHIFT 0
-#define SMMU_ID2_IAS_MASK 0xf
-#define SMMU_ID2_UBS_SHIFT 8
-#define SMMU_ID2_UBS_MASK 0xf
-#define SMMU_ID2_PTFS_4K (1 << 12)
-#define SMMU_ID2_PTFS_16K (1 << 13)
-#define SMMU_ID2_PTFS_64K (1 << 14)
-
-#define SMMU_PIDR2_ARCH_SHIFT 4
-#define SMMU_PIDR2_ARCH_MASK 0xf
-
-/* Global TLB invalidation */
-#define SMMU_GR0_STLBIALL 0x60
-#define SMMU_GR0_TLBIVMID 0x64
-#define SMMU_GR0_TLBIALLNSNH 0x68
-#define SMMU_GR0_TLBIALLH 0x6c
-#define SMMU_GR0_sTLBGSYNC 0x70
-#define SMMU_GR0_sTLBGSTATUS 0x74
-#define SMMU_sTLBGSTATUS_GSACTIVE (1 << 0)
-#define SMMU_TLB_LOOP_TIMEOUT 1000000 /* 1s! */
-
-/* Stream mapping registers */
-#define SMMU_GR0_SMR(n) (0x800 + ((n) << 2))
-#define SMMU_SMR_VALID (1 << 31)
-#define SMMU_SMR_MASK_SHIFT 16
-#define SMMU_SMR_MASK_MASK 0x7fff
-#define SMMU_SMR_ID_SHIFT 0
-#define SMMU_SMR_ID_MASK 0x7fff
-
-#define SMMU_GR0_S2CR(n) (0xc00 + ((n) << 2))
-#define SMMU_S2CR_CBNDX_SHIFT 0
-#define SMMU_S2CR_CBNDX_MASK 0xff
-#define SMMU_S2CR_TYPE_SHIFT 16
-#define SMMU_S2CR_TYPE_MASK 0x3
-#define SMMU_S2CR_TYPE_TRANS (0 << SMMU_S2CR_TYPE_SHIFT)
-#define SMMU_S2CR_TYPE_BYPASS (1 << SMMU_S2CR_TYPE_SHIFT)
-#define SMMU_S2CR_TYPE_FAULT (2 << SMMU_S2CR_TYPE_SHIFT)
-
-/* Context bank attribute registers */
-#define SMMU_GR1_CBAR(n) (0x0 + ((n) << 2))
-#define SMMU_CBAR_VMID_SHIFT 0
-#define SMMU_CBAR_VMID_MASK 0xff
-#define SMMU_CBAR_S1_MEMATTR_SHIFT 12
-#define SMMU_CBAR_S1_MEMATTR_MASK 0xf
-#define SMMU_CBAR_S1_MEMATTR_WB 0xf
-#define SMMU_CBAR_TYPE_SHIFT 16
-#define SMMU_CBAR_TYPE_MASK 0x3
-#define SMMU_CBAR_TYPE_S2_TRANS (0 << SMMU_CBAR_TYPE_SHIFT)
-#define SMMU_CBAR_TYPE_S1_TRANS_S2_BYPASS (1 << SMMU_CBAR_TYPE_SHIFT)
-#define SMMU_CBAR_TYPE_S1_TRANS_S2_FAULT (2 << SMMU_CBAR_TYPE_SHIFT)
-#define SMMU_CBAR_TYPE_S1_TRANS_S2_TRANS (3 << SMMU_CBAR_TYPE_SHIFT)
-#define SMMU_CBAR_IRPTNDX_SHIFT 24
-#define SMMU_CBAR_IRPTNDX_MASK 0xff
-
-#define SMMU_GR1_CBA2R(n) (0x800 + ((n) << 2))
-#define SMMU_CBA2R_RW64_32BIT (0 << 0)
-#define SMMU_CBA2R_RW64_64BIT (1 << 0)
-
-/* Translation context bank */
-#define SMMU_CB_BASE(smmu) ((smmu)->base + ((smmu)->size >> 1))
-#define SMMU_CB(smmu, n) ((n) * (smmu)->pagesize)
-
-#define SMMU_CB_SCTLR 0x0
-#define SMMU_CB_RESUME 0x8
-#define SMMU_CB_TCR2 0x10
-#define SMMU_CB_TTBR0_LO 0x20
-#define SMMU_CB_TTBR0_HI 0x24
-#define SMMU_CB_TCR 0x30
-#define SMMU_CB_S1_MAIR0 0x38
-#define SMMU_CB_FSR 0x58
-#define SMMU_CB_FAR_LO 0x60
-#define SMMU_CB_FAR_HI 0x64
-#define SMMU_CB_FSYNR0 0x68
-#define SMMU_CB_S1_TLBIASID 0x610
-
-#define SMMU_SCTLR_S1_ASIDPNE (1 << 12)
-#define SMMU_SCTLR_CFCFG (1 << 7)
-#define SMMU_SCTLR_CFIE (1 << 6)
-#define SMMU_SCTLR_CFRE (1 << 5)
-#define SMMU_SCTLR_E (1 << 4)
-#define SMMU_SCTLR_AFE (1 << 2)
-#define SMMU_SCTLR_TRE (1 << 1)
-#define SMMU_SCTLR_M (1 << 0)
-#define SMMU_SCTLR_EAE_SBOP (SMMU_SCTLR_AFE | SMMU_SCTLR_TRE)
-
-#define SMMU_RESUME_RETRY (0 << 0)
-#define SMMU_RESUME_TERMINATE (1 << 0)
-
-#define SMMU_TCR_EAE (1 << 31)
-
-#define SMMU_TCR_PASIZE_SHIFT 16
-#define SMMU_TCR_PASIZE_MASK 0x7
-
-#define SMMU_TCR_TG0_4K (0 << 14)
-#define SMMU_TCR_TG0_64K (1 << 14)
-
-#define SMMU_TCR_SH0_SHIFT 12
-#define SMMU_TCR_SH0_MASK 0x3
-#define SMMU_TCR_SH_NS 0
-#define SMMU_TCR_SH_OS 2
-#define SMMU_TCR_SH_IS 3
-
-#define SMMU_TCR_ORGN0_SHIFT 10
-#define SMMU_TCR_IRGN0_SHIFT 8
-#define SMMU_TCR_RGN_MASK 0x3
-#define SMMU_TCR_RGN_NC 0
-#define SMMU_TCR_RGN_WBWA 1
-#define SMMU_TCR_RGN_WT 2
-#define SMMU_TCR_RGN_WB 3
-
-#define SMMU_TCR_SL0_SHIFT 6
-#define SMMU_TCR_SL0_MASK 0x3
-#define SMMU_TCR_SL0_LVL_2 0
-#define SMMU_TCR_SL0_LVL_1 1
-
-#define SMMU_TCR_T1SZ_SHIFT 16
-#define SMMU_TCR_T0SZ_SHIFT 0
-#define SMMU_TCR_SZ_MASK 0xf
-
-#define SMMU_TCR2_SEP_SHIFT 15
-#define SMMU_TCR2_SEP_MASK 0x7
-
-#define SMMU_TCR2_PASIZE_SHIFT 0
-#define SMMU_TCR2_PASIZE_MASK 0x7
-
-/* Common definitions for PASize and SEP fields */
-#define SMMU_TCR2_ADDR_32 0
-#define SMMU_TCR2_ADDR_36 1
-#define SMMU_TCR2_ADDR_40 2
-#define SMMU_TCR2_ADDR_42 3
-#define SMMU_TCR2_ADDR_44 4
-#define SMMU_TCR2_ADDR_48 5
-
-#define SMMU_TTBRn_HI_ASID_SHIFT 16
-
-#define SMMU_MAIR_ATTR_SHIFT(n) ((n) << 3)
-#define SMMU_MAIR_ATTR_MASK 0xff
-#define SMMU_MAIR_ATTR_DEVICE 0x04
-#define SMMU_MAIR_ATTR_NC 0x44
-#define SMMU_MAIR_ATTR_WBRWA 0xff
-#define SMMU_MAIR_ATTR_IDX_NC 0
-#define SMMU_MAIR_ATTR_IDX_CACHE 1
-#define SMMU_MAIR_ATTR_IDX_DEV 2
-
-#define SMMU_FSR_MULTI (1 << 31)
-#define SMMU_FSR_SS (1 << 30)
-#define SMMU_FSR_UUT (1 << 8)
-#define SMMU_FSR_ASF (1 << 7)
-#define SMMU_FSR_TLBLKF (1 << 6)
-#define SMMU_FSR_TLBMCF (1 << 5)
-#define SMMU_FSR_EF (1 << 4)
-#define SMMU_FSR_PF (1 << 3)
-#define SMMU_FSR_AFF (1 << 2)
-#define SMMU_FSR_TF (1 << 1)
-
-#define SMMU_FSR_IGN (SMMU_FSR_AFF | SMMU_FSR_ASF | \
- SMMU_FSR_TLBMCF | SMMU_FSR_TLBLKF)
-#define SMMU_FSR_FAULT (SMMU_FSR_MULTI | SMMU_FSR_SS | \
- SMMU_FSR_UUT | SMMU_FSR_EF | \
- SMMU_FSR_PF | SMMU_FSR_TF | \
- SMMU_FSR_IGN)
-
-#define SMMU_FSYNR0_WNR (1 << 4)
-
-#define smmu_print(dev, lvl, fmt, ...) \
- printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev->node), ## __VA_ARGS__)
-
-#define smmu_err(dev, fmt, ...) smmu_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
-
-#define smmu_dbg(dev, fmt, ...) \
- smmu_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__)
-
-#define smmu_info(dev, fmt, ...) \
- smmu_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__)
-
-#define smmu_warn(dev, fmt, ...) \
- smmu_print(dev, XENLOG_WARNING, fmt, ## __VA_ARGS__)
-
-struct arm_smmu_device {
- const struct dt_device_node *node;
-
- void __iomem *base;
- unsigned long size;
- unsigned long pagesize;
-
-#define SMMU_FEAT_COHERENT_WALK (1 << 0)
-#define SMMU_FEAT_STREAM_MATCH (1 << 1)
-#define SMMU_FEAT_TRANS_S1 (1 << 2)
-#define SMMU_FEAT_TRANS_S2 (1 << 3)
-#define SMMU_FEAT_TRANS_NESTED (1 << 4)
- u32 features;
- u32 options;
- int version;
-
- u32 num_context_banks;
- u32 num_s2_context_banks;
- DECLARE_BITMAP(context_map, SMMU_MAX_CBS);
- atomic_t irptndx;
-
- u32 num_mapping_groups;
- DECLARE_BITMAP(smr_map, SMMU_MAX_SMRS);
-
- unsigned long input_size;
- unsigned long s1_output_size;
- unsigned long s2_output_size;
-
- u32 num_global_irqs;
- u32 num_context_irqs;
- unsigned int *irqs;
-
- u32 smr_mask_mask;
- u32 smr_id_mask;
-
- unsigned long *sids;
-
- struct list_head list;
- struct rb_root masters;
-};
-
-struct arm_smmu_smr {
- u8 idx;
- u16 mask;
- u16 id;
-};
-
-#define INVALID_IRPTNDX 0xff
-
-#define SMMU_CB_ASID(cfg) ((cfg)->cbndx)
-#define SMMU_CB_VMID(cfg) ((cfg)->cbndx + 1)
-
-struct arm_smmu_domain_cfg {
- struct arm_smmu_device *smmu;
- u8 cbndx;
- u8 irptndx;
- u32 cbar;
- /* Domain associated to this device */
- struct domain *domain;
- /* List of master which use this structure */
- struct list_head masters;
-
- /* Used to link domain context for a same domain */
- struct list_head list;
-};
-
-struct arm_smmu_master {
- const struct dt_device_node *dt_node;
-
- /*
- * The following is specific to the master's position in the
- * SMMU chain.
- */
- struct rb_node node;
- u32 num_streamids;
- u16 streamids[MAX_MASTER_STREAMIDS];
- int num_s2crs;
-
- struct arm_smmu_smr *smrs;
- struct arm_smmu_domain_cfg *cfg;
-
- /* Used to link masters in a same domain context */
- struct list_head list;
-};
-
-static LIST_HEAD(arm_smmu_devices);
-
-struct arm_smmu_domain {
- spinlock_t lock;
- struct list_head contexts;
-};
-
-struct arm_smmu_option_prop {
- u32 opt;
- const char *prop;
-};
-
-static const struct arm_smmu_option_prop arm_smmu_options [] __initconst =
-{
- { SMMU_OPT_SECURE_CONFIG_ACCESS, "calxeda,smmu-secure-config-access" },
- { 0, NULL},
-};
-
-static void __init check_driver_options(struct arm_smmu_device *smmu)
-{
- int i = 0;
-
- do {
- if ( dt_property_read_bool(smmu->node, arm_smmu_options[i].prop) )
- {
- smmu->options |= arm_smmu_options[i].opt;
- smmu_dbg(smmu, "option %s\n", arm_smmu_options[i].prop);
- }
- } while ( arm_smmu_options[++i].opt );
-}
-
-static void arm_smmu_context_fault(int irq, void *data,
- struct cpu_user_regs *regs)
-{
- u32 fsr, far, fsynr;
- uint64_t iova;
- struct arm_smmu_domain_cfg *cfg = data;
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *cb_base;
-
- cb_base = SMMU_CB_BASE(smmu) + SMMU_CB(smmu, cfg->cbndx);
- fsr = readl_relaxed(cb_base + SMMU_CB_FSR);
-
- if ( !(fsr & SMMU_FSR_FAULT) )
- return;
-
- if ( fsr & SMMU_FSR_IGN )
- smmu_err(smmu, "Unexpected context fault (fsr 0x%u)\n", fsr);
-
- fsynr = readl_relaxed(cb_base + SMMU_CB_FSYNR0);
- far = readl_relaxed(cb_base + SMMU_CB_FAR_LO);
- iova = far;
- far = readl_relaxed(cb_base + SMMU_CB_FAR_HI);
- iova |= ((uint64_t)far << 32);
-
- smmu_err(smmu, "Unhandled context fault for domain %u\n",
- cfg->domain->domain_id);
- smmu_err(smmu, "\tFSR 0x%x, IOVA 0x%"PRIx64", FSYNR 0x%x, CB %d\n",
- fsr, iova, fsynr, cfg->cbndx);
-
- /* Clear the faulting FSR */
- writel(fsr, cb_base + SMMU_CB_FSR);
-
- /* Terminate any stalled transactions */
- if ( fsr & SMMU_FSR_SS )
- writel_relaxed(SMMU_RESUME_TERMINATE, cb_base + SMMU_CB_RESUME);
-}
-
-static void arm_smmu_global_fault(int irq, void *data,
- struct cpu_user_regs *regs)
-{
- u32 gfsr, gfsynr0, gfsynr1, gfsynr2;
- struct arm_smmu_device *smmu = data;
- void __iomem *gr0_base = SMMU_GR0_NS(smmu);
-
- gfsr = readl_relaxed(gr0_base + SMMU_GR0_sGFSR);
- gfsynr0 = readl_relaxed(gr0_base + SMMU_GR0_sGFSYNR0);
- gfsynr1 = readl_relaxed(gr0_base + SMMU_GR0_sGFSYNR1);
- gfsynr2 = readl_relaxed(gr0_base + SMMU_GR0_sGFSYNR2);
-
- if ( !gfsr )
- return;
-
- smmu_err(smmu, "Unexpected global fault, this could be serious\n");
- smmu_err(smmu,
- "\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n",
- gfsr, gfsynr0, gfsynr1, gfsynr2);
- writel(gfsr, gr0_base + SMMU_GR0_sGFSR);
-}
-
-static struct arm_smmu_master *
-find_smmu_master(struct arm_smmu_device *smmu,
- const struct dt_device_node *dev_node)
-{
- struct rb_node *node = smmu->masters.rb_node;
-
- while ( node )
- {
- struct arm_smmu_master *master;
-
- master = container_of(node, struct arm_smmu_master, node);
-
- if ( dev_node < master->dt_node )
- node = node->rb_left;
- else if ( dev_node > master->dt_node )
- node = node->rb_right;
- else
- return master;
- }
-
- return NULL;
-}
-
-static __init int insert_smmu_master(struct arm_smmu_device *smmu,
- struct arm_smmu_master *master)
-{
- struct rb_node **new, *parent;
-
- new = &smmu->masters.rb_node;
- parent = NULL;
- while ( *new )
- {
- struct arm_smmu_master *this;
-
- this = container_of(*new, struct arm_smmu_master, node);
-
- parent = *new;
- if ( master->dt_node < this->dt_node )
- new = &((*new)->rb_left);
- else if (master->dt_node > this->dt_node)
- new = &((*new)->rb_right);
- else
- return -EEXIST;
- }
-
- rb_link_node(&master->node, parent, new);
- rb_insert_color(&master->node, &smmu->masters);
- return 0;
-}
-
-static __init int register_smmu_master(struct arm_smmu_device *smmu,
- struct dt_phandle_args *masterspec)
-{
- int i, sid;
- struct arm_smmu_master *master;
- int rc = 0;
-
- smmu_dbg(smmu, "Try to add master %s\n", masterspec->np->name);
-
- master = find_smmu_master(smmu, masterspec->np);
- if ( master )
- {
- smmu_err(smmu,
- "rejecting multiple registrations for master device %s\n",
- masterspec->np->name);
- return -EBUSY;
- }
-
- if ( masterspec->args_count > MAX_MASTER_STREAMIDS )
- {
- smmu_err(smmu,
- "reached maximum number (%d) of stream IDs for master device %s\n",
- MAX_MASTER_STREAMIDS, masterspec->np->name);
- return -ENOSPC;
- }
-
- master = xzalloc(struct arm_smmu_master);
- if ( !master )
- return -ENOMEM;
-
- INIT_LIST_HEAD(&master->list);
- master->dt_node = masterspec->np;
- master->num_streamids = masterspec->args_count;
-
- dt_device_set_protected(masterspec->np);
-
- for ( i = 0; i < master->num_streamids; ++i )
- {
- sid = masterspec->args[i];
- if ( test_and_set_bit(sid, smmu->sids) )
- {
- smmu_err(smmu, "duplicate stream ID (%d)\n", sid);
- xfree(master);
- return -EEXIST;
- }
- master->streamids[i] = masterspec->args[i];
- }
-
- rc = insert_smmu_master(smmu, master);
- /* Insertion should never fail */
- ASSERT(rc == 0);
-
- return 0;
-}
-
-static int __arm_smmu_alloc_bitmap(unsigned long *map, int start, int end)
-{
- int idx;
-
- do
- {
- idx = find_next_zero_bit(map, end, start);
- if ( idx == end )
- return -ENOSPC;
- } while ( test_and_set_bit(idx, map) );
-
- return idx;
-}
-
-static void __arm_smmu_free_bitmap(unsigned long *map, int idx)
-{
- clear_bit(idx, map);
-}
-
-static void arm_smmu_tlb_sync(struct arm_smmu_device *smmu)
-{
- int count = 0;
- void __iomem *gr0_base = SMMU_GR0(smmu);
-
- writel_relaxed(0, gr0_base + SMMU_GR0_sTLBGSYNC);
- while ( readl_relaxed(gr0_base + SMMU_GR0_sTLBGSTATUS) &
- SMMU_sTLBGSTATUS_GSACTIVE )
- {
- cpu_relax();
- if ( ++count == SMMU_TLB_LOOP_TIMEOUT )
- {
- smmu_err(smmu, "TLB sync timed out -- SMMU may be deadlocked\n");
- return;
- }
- udelay(1);
- }
-}
-
-static void arm_smmu_tlb_inv_context(struct arm_smmu_domain_cfg *cfg)
-{
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *base = SMMU_GR0(smmu);
-
- writel_relaxed(SMMU_CB_VMID(cfg),
- base + SMMU_GR0_TLBIVMID);
-
- arm_smmu_tlb_sync(smmu);
-}
-
-static void arm_smmu_iotlb_flush_all(struct domain *d)
-{
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
- struct arm_smmu_domain_cfg *cfg;
-
- spin_lock(&smmu_domain->lock);
- list_for_each_entry(cfg, &smmu_domain->contexts, list)
- arm_smmu_tlb_inv_context(cfg);
- spin_unlock(&smmu_domain->lock);
-}
-
-static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn,
- unsigned int page_count)
-{
- /* ARM SMMU v1 doesn't have flush by VMA and VMID */
- arm_smmu_iotlb_flush_all(d);
-}
-
-static int determine_smr_mask(struct arm_smmu_device *smmu,
- struct arm_smmu_master *master,
- struct arm_smmu_smr *smr, int start, int order)
-{
- u16 i, zero_bits_mask, one_bits_mask, const_mask;
- int nr;
-
- nr = 1 << order;
-
- if ( nr == 1 )
- {
- /* no mask, use streamid to match and be done with it */
- smr->mask = 0;
- smr->id = master->streamids[start];
- return 0;
- }
-
- zero_bits_mask = 0;
- one_bits_mask = 0xffff;
- for ( i = start; i < start + nr; i++)
- {
- zero_bits_mask |= master->streamids[i]; /* const 0 bits */
- one_bits_mask &= master->streamids[i]; /* const 1 bits */
- }
- zero_bits_mask = ~zero_bits_mask;
-
- /* bits having constant values (either 0 or 1) */
- const_mask = zero_bits_mask | one_bits_mask;
-
- i = hweight16(~const_mask);
- if ( (1 << i) == nr )
- {
- smr->mask = ~const_mask;
- smr->id = one_bits_mask;
- }
- else
- /* no usable mask for this set of streamids */
- return 1;
-
- if ( ((smr->mask & smmu->smr_mask_mask) != smr->mask) ||
- ((smr->id & smmu->smr_id_mask) != smr->id) )
- /* insufficient number of mask/id bits */
- return 1;
-
- return 0;
-}
-
-static int determine_smr_mapping(struct arm_smmu_device *smmu,
- struct arm_smmu_master *master,
- struct arm_smmu_smr *smrs, int max_smrs)
-{
- int nr_sid, nr, i, bit, start;
-
- /*
- * This function is called only once -- when a master is added
- * to a domain. If master->num_s2crs != 0 then this master
- * was already added to a domain.
- */
- BUG_ON(master->num_s2crs);
-
- start = nr = 0;
- nr_sid = master->num_streamids;
- do
- {
- /*
- * largest power-of-2 number of streamids for which to
- * determine a usable mask/id pair for stream matching
- */
- bit = fls(nr_sid);
- if (!bit)
- return 0;
-
- /*
- * iterate over power-of-2 numbers to determine
- * largest possible mask/id pair for stream matching
- * of next 2**i streamids
- */
- for ( i = bit - 1; i >= 0; i-- )
- {
- if( !determine_smr_mask(smmu, master,
- &smrs[master->num_s2crs],
- start, i))
- break;
- }
-
- if ( i < 0 )
- goto out;
-
- nr = 1 << i;
- nr_sid -= nr;
- start += nr;
- master->num_s2crs++;
- } while ( master->num_s2crs <= max_smrs );
-
-out:
- if ( nr_sid )
- {
- /* not enough mapping groups available */
- master->num_s2crs = 0;
- return -ENOSPC;
- }
-
- return 0;
-}
-
-static int arm_smmu_master_configure_smrs(struct arm_smmu_device *smmu,
- struct arm_smmu_master *master)
-{
- int i, max_smrs, ret;
- struct arm_smmu_smr *smrs;
- void __iomem *gr0_base = SMMU_GR0(smmu);
-
- if ( !(smmu->features & SMMU_FEAT_STREAM_MATCH) )
- return 0;
-
- if ( master->smrs )
- return -EEXIST;
-
- max_smrs = min(smmu->num_mapping_groups, master->num_streamids);
- smrs = xmalloc_array(struct arm_smmu_smr, max_smrs);
- if ( !smrs )
- {
- smmu_err(smmu, "failed to allocated %d SMRs for master %s\n",
- max_smrs, dt_node_name(master->dt_node));
- return -ENOMEM;
- }
-
- ret = determine_smr_mapping(smmu, master, smrs, max_smrs);
- if ( ret )
- goto err_free_smrs;
-
- /* Allocate the SMRs on the root SMMU */
- for ( i = 0; i < master->num_s2crs; ++i )
- {
- int idx = __arm_smmu_alloc_bitmap(smmu->smr_map, 0,
- smmu->num_mapping_groups);
- if ( idx < 0 )
- {
- smmu_err(smmu, "failed to allocate free SMR\n");
- goto err_free_bitmap;
- }
- smrs[i].idx = idx;
- }
-
- /* It worked! Now, poke the actual hardware */
- for ( i = 0; i < master->num_s2crs; ++i )
- {
- u32 reg = SMMU_SMR_VALID | smrs[i].id << SMMU_SMR_ID_SHIFT |
- smrs[i].mask << SMMU_SMR_MASK_SHIFT;
- smmu_dbg(smmu, "SMR%d: 0x%x\n", smrs[i].idx, reg);
- writel_relaxed(reg, gr0_base + SMMU_GR0_SMR(smrs[i].idx));
- }
-
- master->smrs = smrs;
- return 0;
-
-err_free_bitmap:
- while (--i >= 0)
- __arm_smmu_free_bitmap(smmu->smr_map, smrs[i].idx);
- master->num_s2crs = 0;
-err_free_smrs:
- xfree(smrs);
- return -ENOSPC;
-}
-
-/* Forward declaration */
-static void arm_smmu_destroy_domain_context(struct arm_smmu_domain_cfg *cfg);
-
-static int arm_smmu_domain_add_master(struct domain *d,
- struct arm_smmu_domain_cfg *cfg,
- struct arm_smmu_master *master)
-{
- int i, ret;
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *gr0_base = SMMU_GR0(smmu);
- struct arm_smmu_smr *smrs = master->smrs;
-
- if ( master->cfg )
- return -EBUSY;
-
- ret = arm_smmu_master_configure_smrs(smmu, master);
- if ( ret )
- return ret;
-
- /* Now we're at the root, time to point at our context bank */
- if ( !master->num_s2crs )
- master->num_s2crs = master->num_streamids;
-
- for ( i = 0; i < master->num_s2crs; ++i )
- {
- u32 idx, s2cr;
-
- idx = smrs ? smrs[i].idx : master->streamids[i];
- s2cr = (SMMU_S2CR_TYPE_TRANS << SMMU_S2CR_TYPE_SHIFT) |
- (cfg->cbndx << SMMU_S2CR_CBNDX_SHIFT);
- smmu_dbg(smmu, "S2CR%d: 0x%x\n", idx, s2cr);
- writel_relaxed(s2cr, gr0_base + SMMU_GR0_S2CR(idx));
- }
-
- master->cfg = cfg;
- list_add(&master->list, &cfg->masters);
-
- return 0;
-}
-
-static void arm_smmu_domain_remove_master(struct arm_smmu_master *master)
-{
- int i;
- struct arm_smmu_domain_cfg *cfg = master->cfg;
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *gr0_base = SMMU_GR0(smmu);
- struct arm_smmu_smr *smrs = master->smrs;
-
- /*
- * We *must* clear the S2CR first, because freeing the SMR means
- * that it can be reallocated immediately
- */
- for ( i = 0; i < master->num_streamids; ++i )
- {
- u16 sid = master->streamids[i];
- writel_relaxed(SMMU_S2CR_TYPE_FAULT,
- gr0_base + SMMU_GR0_S2CR(sid));
- }
-
- /* Invalidate the SMRs before freeing back to the allocator */
- for (i = 0; i < master->num_s2crs; ++i) {
- u8 idx = smrs[i].idx;
- writel_relaxed(~SMMU_SMR_VALID, gr0_base + SMMU_GR0_SMR(idx));
- __arm_smmu_free_bitmap(smmu->smr_map, idx);
- }
-
- master->smrs = NULL;
- master->num_s2crs = 0;
- xfree(smrs);
-
- master->cfg = NULL;
- list_del(&master->list);
- INIT_LIST_HEAD(&master->list);
-}
-
-static void arm_smmu_init_context_bank(struct arm_smmu_domain_cfg *cfg)
-{
- u32 reg;
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *cb_base, *gr1_base;
- paddr_t p2maddr;
-
- ASSERT(cfg->domain != NULL);
- p2maddr = page_to_maddr(cfg->domain->arch.p2m.root);
-
- gr1_base = SMMU_GR1(smmu);
- cb_base = SMMU_CB_BASE(smmu) + SMMU_CB(smmu, cfg->cbndx);
-
- /* CBAR */
- reg = cfg->cbar;
- if ( smmu->version == 1 )
- reg |= cfg->irptndx << SMMU_CBAR_IRPTNDX_SHIFT;
-
- reg |= SMMU_CB_VMID(cfg) << SMMU_CBAR_VMID_SHIFT;
- writel_relaxed(reg, gr1_base + SMMU_GR1_CBAR(cfg->cbndx));
-
- if ( smmu->version > 1 )
- {
- /* CBA2R */
-#ifdef CONFIG_ARM_64
- reg = SMMU_CBA2R_RW64_64BIT;
-#else
- reg = SMMU_CBA2R_RW64_32BIT;
-#endif
- writel_relaxed(reg, gr1_base + SMMU_GR1_CBA2R(cfg->cbndx));
- }
-
- /* TTBR0 */
- reg = (p2maddr & ((1ULL << 32) - 1));
- writel_relaxed(reg, cb_base + SMMU_CB_TTBR0_LO);
- reg = (p2maddr >> 32);
- writel_relaxed(reg, cb_base + SMMU_CB_TTBR0_HI);
-
- /*
- * TCR
- * We use long descriptor, with inner-shareable WBWA tables in TTBR0.
- */
- if ( smmu->version > 1 )
- {
- /* 4K Page Table */
- if ( PAGE_SIZE == PAGE_SIZE_4K )
- reg = SMMU_TCR_TG0_4K;
- else
- reg = SMMU_TCR_TG0_64K;
-
- switch ( smmu->s2_output_size ) {
- case 32:
- reg |= (SMMU_TCR2_ADDR_32 << SMMU_TCR_PASIZE_SHIFT);
- break;
- case 36:
- reg |= (SMMU_TCR2_ADDR_36 << SMMU_TCR_PASIZE_SHIFT);
- break;
- case 40:
- reg |= (SMMU_TCR2_ADDR_40 << SMMU_TCR_PASIZE_SHIFT);
- break;
- case 42:
- reg |= (SMMU_TCR2_ADDR_42 << SMMU_TCR_PASIZE_SHIFT);
- break;
- case 44:
- reg |= (SMMU_TCR2_ADDR_44 << SMMU_TCR_PASIZE_SHIFT);
- break;
- case 48:
- reg |= (SMMU_TCR2_ADDR_48 << SMMU_TCR_PASIZE_SHIFT);
- break;
- }
- }
- else
- reg = 0;
-
- /* The attribute to walk the page table should be the same as VTCR_EL2 */
- reg |= SMMU_TCR_EAE |
- (SMMU_TCR_SH_IS << SMMU_TCR_SH0_SHIFT) |
- (SMMU_TCR_RGN_WBWA << SMMU_TCR_ORGN0_SHIFT) |
- (SMMU_TCR_RGN_WBWA << SMMU_TCR_IRGN0_SHIFT) |
- (SMMU_TCR_SL0_LVL_1 << SMMU_TCR_SL0_SHIFT) |
- /* T0SZ=(1)100 = -8 ( 32 -(-8) = 40 bit physical addresses ) */
- (0x18 << SMMU_TCR_T0SZ_SHIFT);
- writel_relaxed(reg, cb_base + SMMU_CB_TCR);
-
- /* SCTLR */
- reg = SMMU_SCTLR_CFCFG |
- SMMU_SCTLR_CFIE |
- SMMU_SCTLR_CFRE |
- SMMU_SCTLR_M |
- SMMU_SCTLR_EAE_SBOP;
-
- writel_relaxed(reg, cb_base + SMMU_CB_SCTLR);
-}
-
-static struct arm_smmu_domain_cfg *
-arm_smmu_alloc_domain_context(struct domain *d,
- struct arm_smmu_device *smmu)
-{
- unsigned int irq;
- int ret, start;
- struct arm_smmu_domain_cfg *cfg;
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
-
- ASSERT(spin_is_locked(&smmu_domain->lock));
-
- cfg = xzalloc(struct arm_smmu_domain_cfg);
- if ( !cfg )
- return NULL;
-
- /* Master already initialized to another domain ... */
- if ( cfg->domain != NULL )
- goto out_free_mem;
-
- cfg->cbar = SMMU_CBAR_TYPE_S2_TRANS;
- start = 0;
-
- ret = __arm_smmu_alloc_bitmap(smmu->context_map, start,
- smmu->num_context_banks);
- if ( ret < 0 )
- goto out_free_mem;
-
- cfg->cbndx = ret;
- if ( smmu->version == 1 )
- {
- cfg->irptndx = atomic_inc_return(&smmu->irptndx);
- cfg->irptndx %= smmu->num_context_irqs;
- }
- else
- cfg->irptndx = cfg->cbndx;
-
- irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
- ret = request_irq(irq, IRQF_SHARED, arm_smmu_context_fault,
- "arm-smmu-context-fault", cfg);
- if ( ret )
- {
- smmu_err(smmu, "failed to request context IRQ %d (%u)\n",
- cfg->irptndx, irq);
- cfg->irptndx = INVALID_IRPTNDX;
- goto out_free_context;
- }
-
- cfg->domain = d;
- cfg->smmu = smmu;
- if ( smmu->features & SMMU_FEAT_COHERENT_WALK )
- iommu_set_feature(d, IOMMU_FEAT_COHERENT_WALK);
-
- arm_smmu_init_context_bank(cfg);
- list_add(&cfg->list, &smmu_domain->contexts);
- INIT_LIST_HEAD(&cfg->masters);
-
- return cfg;
-
-out_free_context:
- __arm_smmu_free_bitmap(smmu->context_map, cfg->cbndx);
-out_free_mem:
- xfree(cfg);
-
- return NULL;
-}
-
-static void arm_smmu_destroy_domain_context(struct arm_smmu_domain_cfg *cfg)
-{
- struct domain *d = cfg->domain;
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
- struct arm_smmu_device *smmu = cfg->smmu;
- void __iomem *cb_base;
- unsigned int irq;
-
- ASSERT(spin_is_locked(&smmu_domain->lock));
- BUG_ON(!list_empty(&cfg->masters));
-
- /* Disable the context bank and nuke the TLB before freeing it */
- cb_base = SMMU_CB_BASE(smmu) + SMMU_CB(smmu, cfg->cbndx);
- writel_relaxed(0, cb_base + SMMU_CB_SCTLR);
- arm_smmu_tlb_inv_context(cfg);
-
- if ( cfg->irptndx != INVALID_IRPTNDX )
- {
- irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
- release_irq(irq, cfg);
- }
-
- __arm_smmu_free_bitmap(smmu->context_map, cfg->cbndx);
- list_del(&cfg->list);
- xfree(cfg);
-}
-
-static struct arm_smmu_device *
-arm_smmu_find_smmu_by_dev(const struct dt_device_node *dev)
-{
- struct arm_smmu_device *smmu;
- struct arm_smmu_master *master = NULL;
-
- list_for_each_entry( smmu, &arm_smmu_devices, list )
- {
- master = find_smmu_master(smmu, dev);
- if ( master )
- break;
- }
-
- if ( !master )
- return NULL;
-
- return smmu;
-}
-
-static int arm_smmu_attach_dev(struct domain *d,
- const struct dt_device_node *dev)
-{
- struct arm_smmu_device *smmu = arm_smmu_find_smmu_by_dev(dev);
- struct arm_smmu_master *master;
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
- struct arm_smmu_domain_cfg *cfg = NULL;
- struct arm_smmu_domain_cfg *curr;
- int ret;
-
- printk(XENLOG_DEBUG "arm-smmu: attach %s to domain %d\n",
- dt_node_full_name(dev), d->domain_id);
-
- if ( !smmu )
- {
- printk(XENLOG_ERR "%s: cannot attach to SMMU, is it on the same bus?\n",
- dt_node_full_name(dev));
- return -ENODEV;
- }
-
- master = find_smmu_master(smmu, dev);
- BUG_ON(master == NULL);
-
- /* Check if the device is already assigned to someone */
- if ( master->cfg )
- return -EBUSY;
-
- spin_lock(&smmu_domain->lock);
- list_for_each_entry( curr, &smmu_domain->contexts, list )
- {
- if ( curr->smmu == smmu )
- {
- cfg = curr;
- break;
- }
- }
-
- if ( !cfg )
- {
- cfg = arm_smmu_alloc_domain_context(d, smmu);
- if ( !cfg )
- {
- smmu_err(smmu, "unable to allocate context for domain %u\n",
- d->domain_id);
- spin_unlock(&smmu_domain->lock);
- return -ENOMEM;
- }
- }
- spin_unlock(&smmu_domain->lock);
-
- ret = arm_smmu_domain_add_master(d, cfg, master);
- if ( ret )
- {
- spin_lock(&smmu_domain->lock);
- if ( list_empty(&cfg->masters) )
- arm_smmu_destroy_domain_context(cfg);
- spin_unlock(&smmu_domain->lock);
- }
-
- return ret;
-}
-
-static int arm_smmu_detach_dev(struct domain *d,
- const struct dt_device_node *dev)
-{
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
- struct arm_smmu_master *master;
- struct arm_smmu_device *smmu = arm_smmu_find_smmu_by_dev(dev);
- struct arm_smmu_domain_cfg *cfg;
-
- printk(XENLOG_DEBUG "arm-smmu: detach %s to domain %d\n",
- dt_node_full_name(dev), d->domain_id);
-
- if ( !smmu )
- {
- printk(XENLOG_ERR "%s: cannot find the SMMU, is it on the same bus?\n",
- dt_node_full_name(dev));
- return -ENODEV;
- }
-
- master = find_smmu_master(smmu, dev);
- BUG_ON(master == NULL);
-
- cfg = master->cfg;
-
- /* Sanity check to avoid removing a device that doesn't belong to
- * the domain
- */
- if ( !cfg || cfg->domain != d )
- {
- printk(XENLOG_ERR "%s: was not attach to domain %d\n",
- dt_node_full_name(dev), d->domain_id);
- return -ESRCH;
- }
-
- arm_smmu_domain_remove_master(master);
-
- spin_lock(&smmu_domain->lock);
- if ( list_empty(&cfg->masters) )
- arm_smmu_destroy_domain_context(cfg);
- spin_unlock(&smmu_domain->lock);
-
- return 0;
-}
-
-static int arm_smmu_reassign_dt_dev(struct domain *s, struct domain *t,
- const struct dt_device_node *dev)
-{
- int ret = 0;
-
- /* Don't allow remapping on other domain than hwdom */
- if ( t != hardware_domain )
- return -EPERM;
-
- if ( t == s )
- return 0;
-
- ret = arm_smmu_detach_dev(s, dev);
- if ( ret )
- return ret;
-
- ret = arm_smmu_attach_dev(t, dev);
-
- return ret;
-}
-
-static __init int arm_smmu_id_size_to_bits(int size)
-{
- switch ( size )
- {
- case 0:
- return 32;
- case 1:
- return 36;
- case 2:
- return 40;
- case 3:
- return 42;
- case 4:
- return 44;
- case 5:
- default:
- return 48;
- }
-}
-
-static __init int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
-{
- unsigned long size;
- void __iomem *gr0_base = SMMU_GR0(smmu);
- u32 id;
-
- smmu_info(smmu, "probing hardware configuration...\n");
-
- /*
- * Primecell ID
- */
- id = readl_relaxed(gr0_base + SMMU_GR0_PIDR2);
- smmu->version = ((id >> SMMU_PIDR2_ARCH_SHIFT) & SMMU_PIDR2_ARCH_MASK) + 1;
- smmu_info(smmu, "SMMUv%d with:\n", smmu->version);
-
- /* ID0 */
- id = readl_relaxed(gr0_base + SMMU_GR0_ID0);
-#ifndef CONFIG_ARM_64
- if ( ((id >> SMMU_ID0_PTFS_SHIFT) & SMMU_ID0_PTFS_MASK) ==
- SMMU_ID0_PTFS_V8_ONLY )
- {
- smmu_err(smmu, "\tno v7 descriptor support!\n");
- return -ENODEV;
- }
-#endif
- if ( id & SMMU_ID0_S1TS )
- {
- smmu->features |= SMMU_FEAT_TRANS_S1;
- smmu_info(smmu, "\tstage 1 translation\n");
- }
-
- if ( id & SMMU_ID0_S2TS )
- {
- smmu->features |= SMMU_FEAT_TRANS_S2;
- smmu_info(smmu, "\tstage 2 translation\n");
- }
-
- if ( id & SMMU_ID0_NTS )
- {
- smmu->features |= SMMU_FEAT_TRANS_NESTED;
- smmu_info(smmu, "\tnested translation\n");
- }
-
- if ( !(smmu->features &
- (SMMU_FEAT_TRANS_S1 | SMMU_FEAT_TRANS_S2 |
- SMMU_FEAT_TRANS_NESTED)) )
- {
- smmu_err(smmu, "\tno translation support!\n");
- return -ENODEV;
- }
-
- /* We need at least support for Stage 2 */
- if ( !(smmu->features & SMMU_FEAT_TRANS_S2) )
- {
- smmu_err(smmu, "\tno stage 2 translation!\n");
- return -ENODEV;
- }
-
- if ( id & SMMU_ID0_CTTW )
- {
- smmu->features |= SMMU_FEAT_COHERENT_WALK;
- smmu_info(smmu, "\tcoherent table walk\n");
- }
-
- if ( id & SMMU_ID0_SMS )
- {
- u32 smr, sid, mask;
-
- smmu->features |= SMMU_FEAT_STREAM_MATCH;
- smmu->num_mapping_groups = (id >> SMMU_ID0_NUMSMRG_SHIFT) &
- SMMU_ID0_NUMSMRG_MASK;
- if ( smmu->num_mapping_groups == 0 )
- {
- smmu_err(smmu,
- "stream-matching supported, but no SMRs present!\n");
- return -ENODEV;
- }
-
- smr = SMMU_SMR_MASK_MASK << SMMU_SMR_MASK_SHIFT;
- smr |= (SMMU_SMR_ID_MASK << SMMU_SMR_ID_SHIFT);
- writel_relaxed(smr, gr0_base + SMMU_GR0_SMR(0));
- smr = readl_relaxed(gr0_base + SMMU_GR0_SMR(0));
-
- mask = (smr >> SMMU_SMR_MASK_SHIFT) & SMMU_SMR_MASK_MASK;
- sid = (smr >> SMMU_SMR_ID_SHIFT) & SMMU_SMR_ID_MASK;
- if ( (mask & sid) != sid )
- {
- smmu_err(smmu,
- "SMR mask bits (0x%x) insufficient for ID field (0x%x)\n",
- mask, sid);
- return -ENODEV;
- }
- smmu->smr_mask_mask = mask;
- smmu->smr_id_mask = sid;
-
- smmu_info(smmu,
- "\tstream matching with %u register groups, mask 0x%x\n",
- smmu->num_mapping_groups, mask);
- }
-
- /* ID1 */
- id = readl_relaxed(gr0_base + SMMU_GR0_ID1);
- smmu->pagesize = (id & SMMU_ID1_PAGESIZE) ? PAGE_SIZE_64K : PAGE_SIZE_4K;
-
- /* Check for size mismatch of SMMU address space from mapped region */
- size = 1 << (((id >> SMMU_ID1_NUMPAGENDXB_SHIFT) &
- SMMU_ID1_NUMPAGENDXB_MASK) + 1);
- size *= (smmu->pagesize << 1);
- if ( smmu->size != size )
- smmu_warn(smmu, "SMMU address space size (0x%lx) differs "
- "from mapped region size (0x%lx)!\n", size, smmu->size);
-
- smmu->num_s2_context_banks = (id >> SMMU_ID1_NUMS2CB_SHIFT) &
- SMMU_ID1_NUMS2CB_MASK;
- smmu->num_context_banks = (id >> SMMU_ID1_NUMCB_SHIFT) &
- SMMU_ID1_NUMCB_MASK;
- if ( smmu->num_s2_context_banks > smmu->num_context_banks )
- {
- smmu_err(smmu, "impossible number of S2 context banks!\n");
- return -ENODEV;
- }
- smmu_info(smmu, "\t%u context banks (%u stage-2 only)\n",
- smmu->num_context_banks, smmu->num_s2_context_banks);
-
- /* ID2 */
- id = readl_relaxed(gr0_base + SMMU_GR0_ID2);
- size = arm_smmu_id_size_to_bits((id >> SMMU_ID2_IAS_SHIFT) &
- SMMU_ID2_IAS_MASK);
-
- /*
- * Stage-1 output limited by stage-2 input size due to VTCR_EL2
- * setup (see setup_virt_paging)
- */
- /* Current maximum output size of 40 bits */
- smmu->s1_output_size = min(40UL, size);
-
- /* The stage-2 output mask is also applied for bypass */
- size = arm_smmu_id_size_to_bits((id >> SMMU_ID2_OAS_SHIFT) &
- SMMU_ID2_OAS_MASK);
- smmu->s2_output_size = min((unsigned long)PADDR_BITS, size);
-
- if ( smmu->version == 1 )
- smmu->input_size = 32;
- else
- {
-#ifdef CONFIG_ARM_64
- size = (id >> SMMU_ID2_UBS_SHIFT) & SMMU_ID2_UBS_MASK;
- size = min(39, arm_smmu_id_size_to_bits(size));
-#else
- size = 32;
-#endif
- smmu->input_size = size;
-
- if ( (PAGE_SIZE == PAGE_SIZE_4K && !(id & SMMU_ID2_PTFS_4K) ) ||
- (PAGE_SIZE == PAGE_SIZE_64K && !(id & SMMU_ID2_PTFS_64K)) ||
- (PAGE_SIZE != PAGE_SIZE_4K && PAGE_SIZE != PAGE_SIZE_64K) )
- {
- smmu_err(smmu, "CPU page size 0x%lx unsupported\n",
- PAGE_SIZE);
- return -ENODEV;
- }
- }
-
- smmu_info(smmu, "\t%lu-bit VA, %lu-bit IPA, %lu-bit PA\n",
- smmu->input_size, smmu->s1_output_size, smmu->s2_output_size);
- return 0;
-}
-
-static __init void arm_smmu_device_reset(struct arm_smmu_device *smmu)
-{
- void __iomem *gr0_base = SMMU_GR0(smmu);
- void __iomem *cb_base;
- int i = 0;
- u32 reg;
-
- smmu_dbg(smmu, "device reset\n");
-
- /* Clear Global FSR */
- reg = readl_relaxed(SMMU_GR0_NS(smmu) + SMMU_GR0_sGFSR);
- writel(reg, SMMU_GR0_NS(smmu) + SMMU_GR0_sGFSR);
-
- /* Mark all SMRn as invalid and all S2CRn as fault */
- for ( i = 0; i < smmu->num_mapping_groups; ++i )
- {
- writel_relaxed(~SMMU_SMR_VALID, gr0_base + SMMU_GR0_SMR(i));
- writel_relaxed(SMMU_S2CR_TYPE_FAULT, gr0_base + SMMU_GR0_S2CR(i));
- }
-
- /* Make sure all context banks are disabled and clear CB_FSR */
- for ( i = 0; i < smmu->num_context_banks; ++i )
- {
- cb_base = SMMU_CB_BASE(smmu) + SMMU_CB(smmu, i);
- writel_relaxed(0, cb_base + SMMU_CB_SCTLR);
- writel_relaxed(SMMU_FSR_FAULT, cb_base + SMMU_CB_FSR);
- }
-
- /* Invalidate the TLB, just in case */
- writel_relaxed(0, gr0_base + SMMU_GR0_STLBIALL);
- writel_relaxed(0, gr0_base + SMMU_GR0_TLBIALLH);
- writel_relaxed(0, gr0_base + SMMU_GR0_TLBIALLNSNH);
-
- reg = readl_relaxed(SMMU_GR0_NS(smmu) + SMMU_GR0_sCR0);
-
- /* Enable fault reporting */
- reg |= (SMMU_sCR0_GFRE | SMMU_sCR0_GFIE |
- SMMU_sCR0_GCFGFRE | SMMU_sCR0_GCFGFIE);
-
- /* Disable TLB broadcasting. */
- reg |= (SMMU_sCR0_VMIDPNE | SMMU_sCR0_PTM);
-
- /* Enable client access, generate a fault if no mapping is found */
- reg &= ~(SMMU_sCR0_CLIENTPD);
- reg |= SMMU_sCR0_USFCFG;
-
- /* Disable forced broadcasting */
- reg &= ~SMMU_sCR0_FB;
-
- /* Don't upgrade barriers when client devices are not mapped to
- * a translation context banks (just here for clarity as Xen policy
- * is to deny invalid transaction). */
- reg &= ~(SMMU_sCR0_BSU_MASK << SMMU_sCR0_BSU_SHIFT);
-
- /* Push the button */
- arm_smmu_tlb_sync(smmu);
- writel_relaxed(reg, SMMU_GR0_NS(smmu) + SMMU_GR0_sCR0);
-}
-
-static int arm_smmu_iommu_domain_init(struct domain *d)
-{
- struct arm_smmu_domain *smmu_domain;
-
- smmu_domain = xzalloc(struct arm_smmu_domain);
- if ( !smmu_domain )
- return -ENOMEM;
-
- spin_lock_init(&smmu_domain->lock);
- INIT_LIST_HEAD(&smmu_domain->contexts);
-
- domain_hvm_iommu(d)->arch.priv = smmu_domain;
-
- return 0;
-}
-
-static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d)
-{
-}
-
-static void arm_smmu_iommu_domain_teardown(struct domain *d)
-{
- struct arm_smmu_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
-
- ASSERT(list_empty(&smmu_domain->contexts));
- xfree(smmu_domain);
-}
-
-static int arm_smmu_map_page(struct domain *d, unsigned long gfn,
- unsigned long mfn, unsigned int flags)
-{
- p2m_type_t t;
-
- /* Grant mappings can be used for DMA requests. The dev_bus_addr returned by
- * the hypercall is the MFN (not the IPA). For device protected by
- * an IOMMU, Xen needs to add a 1:1 mapping in the domain p2m to
- * allow DMA request to work.
- * This is only valid when the domain is directed mapped. Hence this
- * function should only be used by gnttab code with gfn == mfn.
- */
- BUG_ON(!is_domain_direct_mapped(d));
- BUG_ON(mfn != gfn);
-
- /* We only support readable and writable flags */
- if ( !(flags & (IOMMUF_readable | IOMMUF_writable)) )
- return -EINVAL;
-
- t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro;
-
- /* The function guest_physmap_add_entry replaces the current mapping
- * if there is already one...
- */
- return guest_physmap_add_entry(d, gfn, mfn, 0, t);
-}
-
-static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn)
-{
- /* This function should only be used by gnttab code when the domain
- * is direct mapped
- */
- if ( !is_domain_direct_mapped(d) )
- return -EINVAL;
-
- guest_physmap_remove_page(d, gfn, gfn, 0);
-
- return 0;
-}
-
-static const struct iommu_ops arm_smmu_iommu_ops = {
- .init = arm_smmu_iommu_domain_init,
- .hwdom_init = arm_smmu_iommu_hwdom_init,
- .teardown = arm_smmu_iommu_domain_teardown,
- .iotlb_flush = arm_smmu_iotlb_flush,
- .iotlb_flush_all = arm_smmu_iotlb_flush_all,
- .assign_dt_device = arm_smmu_attach_dev,
- .reassign_dt_device = arm_smmu_reassign_dt_dev,
- .map_page = arm_smmu_map_page,
- .unmap_page = arm_smmu_unmap_page,
-};
-
-static int __init smmu_init(struct dt_device_node *dev,
- const void *data)
-{
- struct arm_smmu_device *smmu;
- int res;
- u64 addr, size;
- unsigned int num_irqs, i;
- struct dt_phandle_args masterspec;
- struct rb_node *node;
-
- /* Even if the device can't be initialized, we don't want to give
- * the smmu device to dom0.
- */
- dt_device_set_used_by(dev, DOMID_XEN);
-
- smmu = xzalloc(struct arm_smmu_device);
- if ( !smmu )
- {
- printk(XENLOG_ERR "%s: failed to allocate arm_smmu_device\n",
- dt_node_full_name(dev));
- return -ENOMEM;
- }
-
- smmu->node = dev;
- check_driver_options(smmu);
-
- res = dt_device_get_address(smmu->node, 0, &addr, &size);
- if ( res )
- {
- smmu_err(smmu, "unable to retrieve the base address of the SMMU\n");
- goto out_err;
- }
-
- smmu->base = ioremap_nocache(addr, size);
- if ( !smmu->base )
- {
- smmu_err(smmu, "unable to map the SMMU memory\n");
- goto out_err;
- }
-
- smmu->size = size;
-
- if ( !dt_property_read_u32(smmu->node, "#global-interrupts",
- &smmu->num_global_irqs) )
- {
- smmu_err(smmu, "missing #global-interrupts\n");
- goto out_unmap;
- }
-
- num_irqs = dt_number_of_irq(smmu->node);
- if ( num_irqs > smmu->num_global_irqs )
- smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;
-
- if ( !smmu->num_context_irqs )
- {
- smmu_err(smmu, "found %d interrupts but expected at least %d\n",
- num_irqs, smmu->num_global_irqs + 1);
- goto out_unmap;
- }
-
- smmu->irqs = xzalloc_array(unsigned int, num_irqs);
- if ( !smmu->irqs )
- {
- smmu_err(smmu, "failed to allocated %d irqs\n", num_irqs);
- goto out_unmap;
- }
-
- for ( i = 0; i < num_irqs; i++ )
- {
- res = platform_get_irq(smmu->node, i);
- if ( res < 0 )
- {
- smmu_err(smmu, "failed to get irq index %d\n", i);
- goto out_free_irqs;
- }
- smmu->irqs[i] = res;
- }
-
- smmu->sids = xzalloc_array(unsigned long,
- BITS_TO_LONGS(SMMU_MAX_STREAMIDS));
- if ( !smmu->sids )
- {
- smmu_err(smmu, "failed to allocated bitmap for stream ID tracking\n");
- goto out_free_masters;
- }
-
-
- i = 0;
- smmu->masters = RB_ROOT;
- while ( !dt_parse_phandle_with_args(smmu->node, "mmu-masters",
- "#stream-id-cells", i, &masterspec) )
- {
- res = register_smmu_master(smmu, &masterspec);
- if ( res )
- {
- smmu_err(smmu, "failed to add master %s\n",
- masterspec.np->name);
- goto out_free_masters;
- }
- i++;
- }
-
- smmu_info(smmu, "registered %d master devices\n", i);
-
- res = arm_smmu_device_cfg_probe(smmu);
- if ( res )
- {
- smmu_err(smmu, "failed to probe the SMMU\n");
- goto out_free_masters;
- }
-
- if ( smmu->version > 1 &&
- smmu->num_context_banks != smmu->num_context_irqs )
- {
- smmu_err(smmu,
- "found only %d context interrupt(s) but %d required\n",
- smmu->num_context_irqs, smmu->num_context_banks);
- goto out_free_masters;
- }
-
- smmu_dbg(smmu, "register global IRQs handler\n");
-
- for ( i = 0; i < smmu->num_global_irqs; ++i )
- {
- smmu_dbg(smmu, "\t- global IRQ %u\n", smmu->irqs[i]);
- res = request_irq(smmu->irqs[i], IRQF_SHARED, arm_smmu_global_fault,
- "arm-smmu global fault", smmu);
- if ( res )
- {
- smmu_err(smmu, "failed to request global IRQ %d (%u)\n",
- i, smmu->irqs[i]);
- goto out_release_irqs;
- }
- }
-
- INIT_LIST_HEAD(&smmu->list);
- list_add(&smmu->list, &arm_smmu_devices);
-
- arm_smmu_device_reset(smmu);
-
- iommu_set_ops(&arm_smmu_iommu_ops);
-
- /* sids field can be freed... */
- xfree(smmu->sids);
- smmu->sids = NULL;
-
- return 0;
-
-out_release_irqs:
- while (i--)
- release_irq(smmu->irqs[i], smmu);
-
-out_free_masters:
- for ( node = rb_first(&smmu->masters); node; node = rb_next(node) )
- {
- struct arm_smmu_master *master;
-
- master = container_of(node, struct arm_smmu_master, node);
- xfree(master);
- }
-
- xfree(smmu->sids);
-
-out_free_irqs:
- xfree(smmu->irqs);
-
-out_unmap:
- iounmap(smmu->base);
-
-out_err:
- xfree(smmu);
-
- return -ENODEV;
-}
-
-static const char * const smmu_dt_compat[] __initconst =
-{
- "arm,mmu-400",
- NULL
-};
-
-DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU)
- .compatible = smmu_dt_compat,
- .init = smmu_init,
-DT_DEVICE_END
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
2015-02-25 18:52 ` [PATCH v4 1/8] xen/iommu: arm: Remove temporary the SMMU driver Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-03-30 9:51 ` Manish Jaggi
2015-02-25 18:52 ` [PATCH v4 3/8] xen/iommu: Consolidate device assignment ops into a single set Julien Grall
` (6 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: Keir Fraser, ian.campbell, Stefano Stabellini, Andrew Cooper,
manish.jaggi, Julien Grall, tim, stefano.stabellini, Jan Beulich
Currently, Xen is supporting PCI and Platform device (based on Device Tree).
While Xen only supports Platform device on ARM, Xen will gain support of
PCI soon.
Some drivers, such as IOMMU drivers, may handle PCI and platform device in
the same way. Only few lines of code differs.
Rather than requesting to provide 2 set of functions (one for PCI and
one for platform device), introduce a generic structure "device" which
is embedded in each specialized device.
As x86 only supports PCI, introduce a new type device_t which will be an
alias to pci_dev for this architecture. It will avoid to add a new field
for this place.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Keir Fraser <keir@xen.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
---
Changes in v4:
- Add Jan and Stefano's ack
- Typoes
Changes in v3:
- Drop common/device.c as it wasn't used after v1
Changes in v2:
- As x86 will only support PCI, only introduce the generic
device on ARM
- Introduce a typedef device_t to be used in common code
- Drop the PCI code for ARM for now. It will be reintroduced
when PCI support will be added
- s#asm/device.h#xen/device.h# is not anymore needed
---
xen/common/device_tree.c | 3 +++
xen/include/asm-arm/device.h | 26 ++++++++++++++++++++++++++
xen/include/asm-x86/device.h | 25 +++++++++++++++++++++++++
xen/include/xen/device_tree.h | 13 +++++++++++++
xen/include/xen/iommu.h | 1 +
xen/include/xen/pci.h | 1 +
6 files changed, 69 insertions(+)
create mode 100644 xen/include/asm-x86/device.h
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 34a1b9e..d1c716f 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1454,6 +1454,9 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
((char *)pp->value)[sz - 1] = 0;
dt_dprintk("fixed up name for %s -> %s\n", pathp,
(char *)pp->value);
+ /* Generic device initialization */
+ np->dev.type = DEV_DT;
+ np->dev.of_node = np;
}
}
if ( allnextpp )
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index b6b32bc..1cedd49 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -2,8 +2,34 @@
#define __ASM_ARM_DEVICE_H
#include <xen/init.h>
+
+enum device_type
+{
+ DEV_DT,
+};
+
+struct dev_archdata {
+ void *iommu; /* IOMMU private data */
+};
+
+/* struct device - The basic device structure */
+struct device
+{
+ enum device_type type;
+#ifdef HAS_DEVICE_TREE
+ struct dt_device_node *of_node; /* Used by drivers imported from Linux */
+#endif
+ struct dev_archdata archdata;
+};
+
+typedef struct device device_t;
+
#include <xen/device_tree.h>
+/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
+#define dev_is_pci(dev) ((void)(dev), 0)
+#define dev_is_dt(dev) ((dev->type == DEV_DT)
+
enum device_class
{
DEVICE_SERIAL,
diff --git a/xen/include/asm-x86/device.h b/xen/include/asm-x86/device.h
new file mode 100644
index 0000000..f2acc7e
--- /dev/null
+++ b/xen/include/asm-x86/device.h
@@ -0,0 +1,25 @@
+#ifndef __ASM_X86_DEVICE_H
+#define __ASM_X86_DEVICE_H
+
+#include <xen/pci.h>
+
+/*
+ * x86 only supports PCI. Therefore it's possible to directly use
+ * pci_dev to avoid adding new field.
+ */
+
+typedef struct pci_dev device_t;
+
+#define dev_is_pci(dev) ((void)(dev), 1)
+#define pci_to_dev(pci) (pci)
+
+#endif /* __ASM_X86_DEVICE_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 6502369..c8a0375 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -11,7 +11,9 @@
#define __XEN_DEVICE_TREE_H__
#include <asm/byteorder.h>
+#include <asm/device.h>
#include <public/xen.h>
+#include <xen/kernel.h>
#include <xen/init.h>
#include <xen/string.h>
#include <xen/types.h>
@@ -80,8 +82,19 @@ struct dt_device_node {
/* IOMMU specific fields */
bool is_protected;
struct list_head domain_list;
+
+ struct device dev;
};
+#define dt_to_dev(dt_node) (&(dt_node)->dev)
+
+static inline struct dt_device_node *dev_to_dt(struct device *dev)
+{
+ ASSERT(dev->type == DEV_DT);
+
+ return container_of(dev, struct dt_device_node, dev);
+}
+
#define MAX_PHANDLE_ARGS 16
struct dt_phandle_args {
struct dt_device_node *np;
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..ecb2627 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -25,6 +25,7 @@
#include <xen/pci.h>
#include <public/hvm/ioreq.h>
#include <public/domctl.h>
+#include <asm/device.h>
#include <asm/iommu.h>
extern bool_t iommu_enable, iommu_enabled;
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index 5f295f3..3988ee68 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -13,6 +13,7 @@
#include <xen/irq.h>
#include <xen/pci_regs.h>
#include <xen/pfn.h>
+#include <asm/device.h>
#include <asm/pci.h>
/*
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 3/8] xen/iommu: Consolidate device assignment ops into a single set
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
2015-02-25 18:52 ` [PATCH v4 1/8] xen/iommu: arm: Remove temporary the SMMU driver Julien Grall
2015-02-25 18:52 ` [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-02-25 18:52 ` [PATCH v4 4/8] xen/arm: Describe devices supported by a driver with dt_device_match Julien Grall
` (5 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, manish.jaggi, Julien Grall, tim,
stefano.stabellini, Jan Beulich
On ARM, the way to assign device tree node is exactly the same as PCI.
Futhermore, all devices can be represented by a 'device_t'.
Therefore there is no need to add separate ops.
The x86 iommu drivers has not been modified to replace 'struct pci_dev'
by "device_t" because the latter is an alias of the former.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>\
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v4:
- Add Ian's ack
Changes in v3:
- Add Stefano and Jan's ack
Changes in v2:
- Use the newly type device_t
- It's not neccessary to modify the x86 iommu drivers
---
xen/drivers/passthrough/device_tree.c | 5 +++--
xen/drivers/passthrough/pci.c | 20 +++++++++++---------
xen/include/xen/iommu.h | 17 ++++++-----------
3 files changed, 20 insertions(+), 22 deletions(-)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 3e47df5..377d41d 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -41,7 +41,7 @@ int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev)
if ( !list_empty(&dev->domain_list) )
goto fail;
- rc = hd->platform_ops->assign_dt_device(d, dev);
+ rc = hd->platform_ops->assign_device(d, 0, dt_to_dev(dev));
if ( rc )
goto fail;
@@ -68,7 +68,8 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev)
spin_lock(&dtdevs_lock);
- rc = hd->platform_ops->reassign_dt_device(d, hardware_domain, dev);
+ rc = hd->platform_ops->reassign_device(d, hardware_domain,
+ 0, dt_to_dev(dev));
if ( rc )
goto fail;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index b93e7d8..4b83583 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1254,7 +1254,7 @@ int iommu_add_device(struct pci_dev *pdev)
if ( !iommu_enabled || !hd->platform_ops )
return 0;
- rc = hd->platform_ops->add_device(pdev->devfn, pdev);
+ rc = hd->platform_ops->add_device(pdev->devfn, pci_to_dev(pdev));
if ( rc || !pdev->phantom_stride )
return rc;
@@ -1263,7 +1263,7 @@ int iommu_add_device(struct pci_dev *pdev)
devfn += pdev->phantom_stride;
if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) )
return 0;
- rc = hd->platform_ops->add_device(devfn, pdev);
+ rc = hd->platform_ops->add_device(devfn, pci_to_dev(pdev));
if ( rc )
printk(XENLOG_WARNING "IOMMU: add %04x:%02x:%02x.%u failed (%d)\n",
pdev->seg, pdev->bus, PCI_SLOT(devfn), PCI_FUNC(devfn), rc);
@@ -1284,7 +1284,7 @@ int iommu_enable_device(struct pci_dev *pdev)
!hd->platform_ops->enable_device )
return 0;
- return hd->platform_ops->enable_device(pdev);
+ return hd->platform_ops->enable_device(pci_to_dev(pdev));
}
int iommu_remove_device(struct pci_dev *pdev)
@@ -1306,7 +1306,7 @@ int iommu_remove_device(struct pci_dev *pdev)
devfn += pdev->phantom_stride;
if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) )
break;
- rc = hd->platform_ops->remove_device(devfn, pdev);
+ rc = hd->platform_ops->remove_device(devfn, pci_to_dev(pdev));
if ( !rc )
continue;
@@ -1315,7 +1315,7 @@ int iommu_remove_device(struct pci_dev *pdev)
return rc;
}
- return hd->platform_ops->remove_device(pdev->devfn, pdev);
+ return hd->platform_ops->remove_device(pdev->devfn, pci_to_dev(pdev));
}
/*
@@ -1383,7 +1383,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
pdev->fault.count = 0;
- if ( (rc = hd->platform_ops->assign_device(d, devfn, pdev)) )
+ if ( (rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev))) )
goto done;
for ( ; pdev->phantom_stride; rc = 0 )
@@ -1391,7 +1391,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
devfn += pdev->phantom_stride;
if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) )
break;
- rc = hd->platform_ops->assign_device(d, devfn, pdev);
+ rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev));
if ( rc )
printk(XENLOG_G_WARNING "d%d: assign %04x:%02x:%02x.%u failed (%d)\n",
d->domain_id, seg, bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
@@ -1426,7 +1426,8 @@ int deassign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
devfn += pdev->phantom_stride;
if ( PCI_SLOT(devfn) != PCI_SLOT(pdev->devfn) )
break;
- ret = hd->platform_ops->reassign_device(d, hardware_domain, devfn, pdev);
+ ret = hd->platform_ops->reassign_device(d, hardware_domain, devfn,
+ pci_to_dev(pdev));
if ( !ret )
continue;
@@ -1436,7 +1437,8 @@ int deassign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
}
devfn = pdev->devfn;
- ret = hd->platform_ops->reassign_device(d, hardware_domain, devfn, pdev);
+ ret = hd->platform_ops->reassign_device(d, hardware_domain, devfn,
+ pci_to_dev(pdev));
if ( ret )
{
dprintk(XENLOG_G_ERR,
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index ecb2627..bf4aff0 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -124,22 +124,17 @@ struct page_info;
struct iommu_ops {
int (*init)(struct domain *d);
void (*hwdom_init)(struct domain *d);
-#ifdef HAS_PCI
- int (*add_device)(u8 devfn, struct pci_dev *);
- int (*enable_device)(struct pci_dev *pdev);
- int (*remove_device)(u8 devfn, struct pci_dev *);
- int (*assign_device)(struct domain *, u8 devfn, struct pci_dev *);
+ int (*add_device)(u8 devfn, device_t *dev);
+ int (*enable_device)(device_t *dev);
+ int (*remove_device)(u8 devfn, device_t *dev);
+ int (*assign_device)(struct domain *, u8 devfn, device_t *dev);
int (*reassign_device)(struct domain *s, struct domain *t,
- u8 devfn, struct pci_dev *);
+ u8 devfn, device_t *dev);
+#ifdef HAS_PCI
int (*get_device_group_id)(u16 seg, u8 bus, u8 devfn);
int (*update_ire_from_msi)(struct msi_desc *msi_desc, struct msi_msg *msg);
void (*read_msi_from_ire)(struct msi_desc *msi_desc, struct msi_msg *msg);
#endif /* HAS_PCI */
-#ifdef HAS_DEVICE_TREE
- int (*assign_dt_device)(struct domain *d, const struct dt_device_node *dev);
- int (*reassign_dt_device)(struct domain *s, struct domain *t,
- const struct dt_device_node *dev);
-#endif
void (*teardown)(struct domain *d);
int (*map_page)(struct domain *d, unsigned long gfn, unsigned long mfn,
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 4/8] xen/arm: Describe devices supported by a driver with dt_device_match
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (2 preceding siblings ...)
2015-02-25 18:52 ` [PATCH v4 3/8] xen/iommu: Consolidate device assignment ops into a single set Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-02-25 18:52 ` [PATCH v4 5/8] xen/iommu: arm: Import the SMMU driver from Linux Julien Grall
` (4 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, manish.jaggi, Julien Grall, tim,
stefano.stabellini
Xen is currently using a list of compatible strings to match drivers again
device nodes. This leads to having double definitions in the GIC
code.
Furthermore Linux drivers are using dt_device_match (actually called
of_device_id in Linux) to list device supported by the drivers.
Remove the exisiting compatible field and replace with a dt_match field
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v4:
- s/dt_match_node/dt_device_match/ in the commit message
- Make the SCIF UART driver use the new API (was merged recently)
- Typoes and clarify the commit message
- Add Ian's ack
Changes in v3:
- Add Stefano's ack
---
xen/arch/arm/device.c | 21 ++-------------------
xen/arch/arm/gic-v2.c | 10 ++++------
xen/arch/arm/gic-v3.c | 8 ++++----
xen/drivers/char/exynos4210-uart.c | 8 ++++----
xen/drivers/char/ns16550.c | 12 ++++++------
xen/drivers/char/omap-uart.c | 8 ++++----
xen/drivers/char/pl011.c | 8 ++++----
xen/drivers/char/scif-uart.c | 8 ++++----
xen/include/asm-arm/device.h | 4 ++--
xen/include/asm-arm/gic.h | 15 +++++----------
10 files changed, 39 insertions(+), 63 deletions(-)
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 1f9dbf7..675784c 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -23,23 +23,6 @@
extern const struct device_desc _sdevice[], _edevice[];
-static bool_t __init device_is_compatible(const struct device_desc *desc,
- const struct dt_device_node *dev)
-{
- const char *const *compat;
-
- if ( !desc->compatible )
- return 0;
-
- for ( compat = desc->compatible; *compat; compat++ )
- {
- if ( dt_device_is_compatible(dev, *compat) )
- return 1;
- }
-
- return 0;
-}
-
int __init device_init(struct dt_device_node *dev, enum device_class class,
const void *data)
{
@@ -55,7 +38,7 @@ int __init device_init(struct dt_device_node *dev, enum device_class class,
if ( desc->class != class )
continue;
- if ( device_is_compatible(desc, dev) )
+ if ( dt_match_node(desc->dt_match, dev) )
{
ASSERT(desc->init != NULL);
@@ -75,7 +58,7 @@ enum device_class device_get_class(const struct dt_device_node *dev)
for ( desc = _sdevice; desc != _edevice; desc++ )
{
- if ( device_is_compatible(desc, dev) )
+ if ( dt_match_node(desc->dt_match, dev) )
return desc->class;
}
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index c05b64a..e551549 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -776,16 +776,14 @@ static int __init gicv2_init(struct dt_device_node *node, const void *data)
return 0;
}
-static const char * const gicv2_dt_compat[] __initconst =
+static const struct dt_device_match gicv2_dt_match[] __initconst =
{
- DT_COMPAT_GIC_CORTEX_A15,
- DT_COMPAT_GIC_CORTEX_A7,
- DT_COMPAT_GIC_400,
- NULL
+ DT_MATCH_GIC_V2,
+ { /* sentinel */ },
};
DT_DEVICE_START(gicv2, "GICv2", DEVICE_GIC)
- .compatible = gicv2_dt_compat,
+ .dt_match = gicv2_dt_match,
.init = gicv2_init,
DT_DEVICE_END
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 1558e17..66cc2a2 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1320,14 +1320,14 @@ static int __init gicv3_init(struct dt_device_node *node, const void *data)
return res;
}
-static const char * const gicv3_dt_compat[] __initconst =
+static const struct dt_device_match gicv3_dt_match[] __initconst =
{
- DT_COMPAT_GIC_V3,
- NULL
+ DT_MATCH_GIC_V3,
+ { /* sentinel */ },
};
DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
- .compatible = gicv3_dt_compat,
+ .dt_match = gicv3_dt_match,
.init = gicv3_init,
DT_DEVICE_END
diff --git a/xen/drivers/char/exynos4210-uart.c b/xen/drivers/char/exynos4210-uart.c
index cba8729..bac1c2b 100644
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -352,14 +352,14 @@ static int __init exynos4210_uart_init(struct dt_device_node *dev,
return 0;
}
-static const char * const exynos4210_dt_compat[] __initconst =
+static const struct dt_device_match exynos4210_dt_match[] __initconst =
{
- "samsung,exynos4210-uart",
- NULL
+ DT_MATCH_COMPATIBLE("samsung,exynos4210-uart"),
+ { /* sentinel */ },
};
DT_DEVICE_START(exynos4210, "Exynos 4210 UART", DEVICE_SERIAL)
- .compatible = exynos4210_dt_compat,
+ .dt_match = exynos4210_dt_match,
.init = exynos4210_uart_init,
DT_DEVICE_END
diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index 161b251..d443880 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -1185,16 +1185,16 @@ static int __init ns16550_uart_dt_init(struct dt_device_node *dev,
return 0;
}
-static const char * const ns16550_dt_compat[] __initconst =
+static const struct dt_device_match ns16550_dt_match[] __initconst =
{
- "ns16550",
- "ns16550a",
- "snps,dw-apb-uart",
- NULL
+ DT_MATCH_COMPATIBLE("ns16550"),
+ DT_MATCH_COMPATIBLE("ns16550a"),
+ DT_MATCH_COMPATIBLE("snps,dw-apb-uart"),
+ { /* sentinel */ },
};
DT_DEVICE_START(ns16550, "NS16550 UART", DEVICE_SERIAL)
- .compatible = ns16550_dt_compat,
+ .dt_match = ns16550_dt_match,
.init = ns16550_uart_dt_init,
DT_DEVICE_END
diff --git a/xen/drivers/char/omap-uart.c b/xen/drivers/char/omap-uart.c
index 16d1454..d8f64ea 100644
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -350,14 +350,14 @@ static int __init omap_uart_init(struct dt_device_node *dev,
return 0;
}
-static const char * const omap_uart_dt_compat[] __initconst =
+static const struct dt_device_match omap_uart_dt_match[] __initconst =
{
- "ti,omap4-uart",
- NULL
+ DT_MATCH_COMPATIBLE("ti,omap4-uart"),
+ { /* sentinel */ },
};
DT_DEVICE_START(omap_uart, "OMAP UART", DEVICE_SERIAL)
- .compatible = omap_uart_dt_compat,
+ .dt_match = omap_uart_dt_match,
.init = omap_uart_init,
DT_DEVICE_END
diff --git a/xen/drivers/char/pl011.c b/xen/drivers/char/pl011.c
index 57274d9..67e6df5 100644
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -285,14 +285,14 @@ static int __init pl011_uart_init(struct dt_device_node *dev,
return 0;
}
-static const char * const pl011_dt_compat[] __initconst =
+static const struct dt_device_match pl011_dt_match[] __initconst =
{
- "arm,pl011",
- NULL
+ DT_MATCH_COMPATIBLE("arm,pl011"),
+ { /* sentinel */ },
};
DT_DEVICE_START(pl011, "PL011 UART", DEVICE_SERIAL)
- .compatible = pl011_dt_compat,
+ .dt_match = pl011_dt_match,
.init = pl011_uart_init,
DT_DEVICE_END
diff --git a/xen/drivers/char/scif-uart.c b/xen/drivers/char/scif-uart.c
index 1a1c496..51a2233 100644
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -346,14 +346,14 @@ static int __init scif_uart_init(struct dt_device_node *dev,
return 0;
}
-static const char * const scif_uart_dt_compat[] __initconst =
+static const struct dt_device_match scif_uart_dt_match[] __initconst =
{
- "renesas,scif",
- NULL
+ DT_MATCH_COMPATIBLE("renesas,scif"),
+ { /* sentinel */ },
};
DT_DEVICE_START(scif_uart, "SCIF UART", DEVICE_SERIAL)
- .compatible = scif_uart_dt_compat,
+ .dt_match = scif_uart_dt_match,
.init = scif_uart_init,
DT_DEVICE_END
diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
index 1cedd49..a72f7c9 100644
--- a/xen/include/asm-arm/device.h
+++ b/xen/include/asm-arm/device.h
@@ -44,8 +44,8 @@ struct device_desc {
const char *name;
/* Device class */
enum device_class class;
- /* Array of device tree 'compatible' strings */
- const char *const *compatible;
+ /* List of devices supported by this driver */
+ const struct dt_device_match *dt_match;
/* Device initialization */
int (*init)(struct dt_device_node *dev, const void *data);
};
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 0396a8e..6f5767f 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -153,17 +153,12 @@
#include <xen/irq.h>
#include <asm-arm/vgic.h>
-#define DT_COMPAT_GIC_400 "arm,gic-400"
-#define DT_COMPAT_GIC_CORTEX_A15 "arm,cortex-a15-gic"
-#define DT_COMPAT_GIC_CORTEX_A7 "arm,cortex-a7-gic"
+#define DT_MATCH_GIC_V2 \
+ DT_MATCH_COMPATIBLE("arm,cortex-a15-gic"), \
+ DT_MATCH_COMPATIBLE("arm,cortex-a7-gic"), \
+ DT_MATCH_COMPATIBLE("arm,gic-400")
-#define DT_MATCH_GIC_V2 DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_CORTEX_A15), \
- DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_CORTEX_A7), \
- DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_400)
-
-#define DT_COMPAT_GIC_V3 "arm,gic-v3"
-
-#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE(DT_COMPAT_GIC_V3)
+#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
/*
* GICv3 registers that needs to be saved/restored
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 5/8] xen/iommu: arm: Import the SMMU driver from Linux
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (3 preceding siblings ...)
2015-02-25 18:52 ` [PATCH v4 4/8] xen/arm: Describe devices supported by a driver with dt_device_match Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-02-25 18:52 ` [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Julien Grall
` (3 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, manish.jaggi, Julien Grall, tim,
stefano.stabellini
Based on commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3.
It's a verbatim of the Linux SMMU drivers code. No Xen code has yet been added
and the file is not built.
Compare to the previous drivers it gains support of PCI. Though it will
need a bit of plumbing for Xen.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v4:
- Add Ian's ack
Changes in v3:
- Update the commit message based on Stefano's review
- Add Stefano's ack
Changes in v2:
- Fix typo in the Signed-off-by
---
xen/drivers/passthrough/arm/smmu.c | 2193 ++++++++++++++++++++++++++++++++++++
1 file changed, 2193 insertions(+)
create mode 100644 xen/drivers/passthrough/arm/smmu.c
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
new file mode 100644
index 0000000..6cd47b7
--- /dev/null
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -0,0 +1,2193 @@
+/*
+ * IOMMU API for ARM architected SMMU implementations.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) 2013 ARM Limited
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ *
+ * This driver currently supports:
+ * - SMMUv1 and v2 implementations
+ * - Stream-matching and stream-indexing
+ * - v7/v8 long-descriptor format
+ * - Non-secure access to the SMMU
+ * - 4k and 64k pages, with contiguous pte hints.
+ * - Up to 48-bit addressing (dependent on VA_BITS)
+ * - Context fault reporting
+ */
+
+#define pr_fmt(fmt) "arm-smmu: " fmt
+
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include <linux/amba/bus.h>
+
+#include <asm/pgalloc.h>
+
+/* Maximum number of stream IDs assigned to a single device */
+#define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
+
+/* Maximum number of context banks per SMMU */
+#define ARM_SMMU_MAX_CBS 128
+
+/* Maximum number of mapping groups per SMMU */
+#define ARM_SMMU_MAX_SMRS 128
+
+/* SMMU global address space */
+#define ARM_SMMU_GR0(smmu) ((smmu)->base)
+#define ARM_SMMU_GR1(smmu) ((smmu)->base + (1 << (smmu)->pgshift))
+
+/*
+ * SMMU global address space with conditional offset to access secure
+ * aliases of non-secure registers (e.g. nsCR0: 0x400, nsGFSR: 0x448,
+ * nsGFSYNR0: 0x450)
+ */
+#define ARM_SMMU_GR0_NS(smmu) \
+ ((smmu)->base + \
+ ((smmu->options & ARM_SMMU_OPT_SECURE_CFG_ACCESS) \
+ ? 0x400 : 0))
+
+/* Page table bits */
+#define ARM_SMMU_PTE_XN (((pteval_t)3) << 53)
+#define ARM_SMMU_PTE_CONT (((pteval_t)1) << 52)
+#define ARM_SMMU_PTE_AF (((pteval_t)1) << 10)
+#define ARM_SMMU_PTE_SH_NS (((pteval_t)0) << 8)
+#define ARM_SMMU_PTE_SH_OS (((pteval_t)2) << 8)
+#define ARM_SMMU_PTE_SH_IS (((pteval_t)3) << 8)
+#define ARM_SMMU_PTE_PAGE (((pteval_t)3) << 0)
+
+#if PAGE_SIZE == SZ_4K
+#define ARM_SMMU_PTE_CONT_ENTRIES 16
+#elif PAGE_SIZE == SZ_64K
+#define ARM_SMMU_PTE_CONT_ENTRIES 32
+#else
+#define ARM_SMMU_PTE_CONT_ENTRIES 1
+#endif
+
+#define ARM_SMMU_PTE_CONT_SIZE (PAGE_SIZE * ARM_SMMU_PTE_CONT_ENTRIES)
+#define ARM_SMMU_PTE_CONT_MASK (~(ARM_SMMU_PTE_CONT_SIZE - 1))
+
+/* Stage-1 PTE */
+#define ARM_SMMU_PTE_AP_UNPRIV (((pteval_t)1) << 6)
+#define ARM_SMMU_PTE_AP_RDONLY (((pteval_t)2) << 6)
+#define ARM_SMMU_PTE_ATTRINDX_SHIFT 2
+#define ARM_SMMU_PTE_nG (((pteval_t)1) << 11)
+
+/* Stage-2 PTE */
+#define ARM_SMMU_PTE_HAP_FAULT (((pteval_t)0) << 6)
+#define ARM_SMMU_PTE_HAP_READ (((pteval_t)1) << 6)
+#define ARM_SMMU_PTE_HAP_WRITE (((pteval_t)2) << 6)
+#define ARM_SMMU_PTE_MEMATTR_OIWB (((pteval_t)0xf) << 2)
+#define ARM_SMMU_PTE_MEMATTR_NC (((pteval_t)0x5) << 2)
+#define ARM_SMMU_PTE_MEMATTR_DEV (((pteval_t)0x1) << 2)
+
+/* Configuration registers */
+#define ARM_SMMU_GR0_sCR0 0x0
+#define sCR0_CLIENTPD (1 << 0)
+#define sCR0_GFRE (1 << 1)
+#define sCR0_GFIE (1 << 2)
+#define sCR0_GCFGFRE (1 << 4)
+#define sCR0_GCFGFIE (1 << 5)
+#define sCR0_USFCFG (1 << 10)
+#define sCR0_VMIDPNE (1 << 11)
+#define sCR0_PTM (1 << 12)
+#define sCR0_FB (1 << 13)
+#define sCR0_BSU_SHIFT 14
+#define sCR0_BSU_MASK 0x3
+
+/* Identification registers */
+#define ARM_SMMU_GR0_ID0 0x20
+#define ARM_SMMU_GR0_ID1 0x24
+#define ARM_SMMU_GR0_ID2 0x28
+#define ARM_SMMU_GR0_ID3 0x2c
+#define ARM_SMMU_GR0_ID4 0x30
+#define ARM_SMMU_GR0_ID5 0x34
+#define ARM_SMMU_GR0_ID6 0x38
+#define ARM_SMMU_GR0_ID7 0x3c
+#define ARM_SMMU_GR0_sGFSR 0x48
+#define ARM_SMMU_GR0_sGFSYNR0 0x50
+#define ARM_SMMU_GR0_sGFSYNR1 0x54
+#define ARM_SMMU_GR0_sGFSYNR2 0x58
+#define ARM_SMMU_GR0_PIDR0 0xfe0
+#define ARM_SMMU_GR0_PIDR1 0xfe4
+#define ARM_SMMU_GR0_PIDR2 0xfe8
+
+#define ID0_S1TS (1 << 30)
+#define ID0_S2TS (1 << 29)
+#define ID0_NTS (1 << 28)
+#define ID0_SMS (1 << 27)
+#define ID0_PTFS_SHIFT 24
+#define ID0_PTFS_MASK 0x2
+#define ID0_PTFS_V8_ONLY 0x2
+#define ID0_CTTW (1 << 14)
+#define ID0_NUMIRPT_SHIFT 16
+#define ID0_NUMIRPT_MASK 0xff
+#define ID0_NUMSIDB_SHIFT 9
+#define ID0_NUMSIDB_MASK 0xf
+#define ID0_NUMSMRG_SHIFT 0
+#define ID0_NUMSMRG_MASK 0xff
+
+#define ID1_PAGESIZE (1 << 31)
+#define ID1_NUMPAGENDXB_SHIFT 28
+#define ID1_NUMPAGENDXB_MASK 7
+#define ID1_NUMS2CB_SHIFT 16
+#define ID1_NUMS2CB_MASK 0xff
+#define ID1_NUMCB_SHIFT 0
+#define ID1_NUMCB_MASK 0xff
+
+#define ID2_OAS_SHIFT 4
+#define ID2_OAS_MASK 0xf
+#define ID2_IAS_SHIFT 0
+#define ID2_IAS_MASK 0xf
+#define ID2_UBS_SHIFT 8
+#define ID2_UBS_MASK 0xf
+#define ID2_PTFS_4K (1 << 12)
+#define ID2_PTFS_16K (1 << 13)
+#define ID2_PTFS_64K (1 << 14)
+
+#define PIDR2_ARCH_SHIFT 4
+#define PIDR2_ARCH_MASK 0xf
+
+/* Global TLB invalidation */
+#define ARM_SMMU_GR0_STLBIALL 0x60
+#define ARM_SMMU_GR0_TLBIVMID 0x64
+#define ARM_SMMU_GR0_TLBIALLNSNH 0x68
+#define ARM_SMMU_GR0_TLBIALLH 0x6c
+#define ARM_SMMU_GR0_sTLBGSYNC 0x70
+#define ARM_SMMU_GR0_sTLBGSTATUS 0x74
+#define sTLBGSTATUS_GSACTIVE (1 << 0)
+#define TLB_LOOP_TIMEOUT 1000000 /* 1s! */
+
+/* Stream mapping registers */
+#define ARM_SMMU_GR0_SMR(n) (0x800 + ((n) << 2))
+#define SMR_VALID (1 << 31)
+#define SMR_MASK_SHIFT 16
+#define SMR_MASK_MASK 0x7fff
+#define SMR_ID_SHIFT 0
+#define SMR_ID_MASK 0x7fff
+
+#define ARM_SMMU_GR0_S2CR(n) (0xc00 + ((n) << 2))
+#define S2CR_CBNDX_SHIFT 0
+#define S2CR_CBNDX_MASK 0xff
+#define S2CR_TYPE_SHIFT 16
+#define S2CR_TYPE_MASK 0x3
+#define S2CR_TYPE_TRANS (0 << S2CR_TYPE_SHIFT)
+#define S2CR_TYPE_BYPASS (1 << S2CR_TYPE_SHIFT)
+#define S2CR_TYPE_FAULT (2 << S2CR_TYPE_SHIFT)
+
+/* Context bank attribute registers */
+#define ARM_SMMU_GR1_CBAR(n) (0x0 + ((n) << 2))
+#define CBAR_VMID_SHIFT 0
+#define CBAR_VMID_MASK 0xff
+#define CBAR_S1_BPSHCFG_SHIFT 8
+#define CBAR_S1_BPSHCFG_MASK 3
+#define CBAR_S1_BPSHCFG_NSH 3
+#define CBAR_S1_MEMATTR_SHIFT 12
+#define CBAR_S1_MEMATTR_MASK 0xf
+#define CBAR_S1_MEMATTR_WB 0xf
+#define CBAR_TYPE_SHIFT 16
+#define CBAR_TYPE_MASK 0x3
+#define CBAR_TYPE_S2_TRANS (0 << CBAR_TYPE_SHIFT)
+#define CBAR_TYPE_S1_TRANS_S2_BYPASS (1 << CBAR_TYPE_SHIFT)
+#define CBAR_TYPE_S1_TRANS_S2_FAULT (2 << CBAR_TYPE_SHIFT)
+#define CBAR_TYPE_S1_TRANS_S2_TRANS (3 << CBAR_TYPE_SHIFT)
+#define CBAR_IRPTNDX_SHIFT 24
+#define CBAR_IRPTNDX_MASK 0xff
+
+#define ARM_SMMU_GR1_CBA2R(n) (0x800 + ((n) << 2))
+#define CBA2R_RW64_32BIT (0 << 0)
+#define CBA2R_RW64_64BIT (1 << 0)
+
+/* Translation context bank */
+#define ARM_SMMU_CB_BASE(smmu) ((smmu)->base + ((smmu)->size >> 1))
+#define ARM_SMMU_CB(smmu, n) ((n) * (1 << (smmu)->pgshift))
+
+#define ARM_SMMU_CB_SCTLR 0x0
+#define ARM_SMMU_CB_RESUME 0x8
+#define ARM_SMMU_CB_TTBCR2 0x10
+#define ARM_SMMU_CB_TTBR0_LO 0x20
+#define ARM_SMMU_CB_TTBR0_HI 0x24
+#define ARM_SMMU_CB_TTBCR 0x30
+#define ARM_SMMU_CB_S1_MAIR0 0x38
+#define ARM_SMMU_CB_FSR 0x58
+#define ARM_SMMU_CB_FAR_LO 0x60
+#define ARM_SMMU_CB_FAR_HI 0x64
+#define ARM_SMMU_CB_FSYNR0 0x68
+#define ARM_SMMU_CB_S1_TLBIASID 0x610
+
+#define SCTLR_S1_ASIDPNE (1 << 12)
+#define SCTLR_CFCFG (1 << 7)
+#define SCTLR_CFIE (1 << 6)
+#define SCTLR_CFRE (1 << 5)
+#define SCTLR_E (1 << 4)
+#define SCTLR_AFE (1 << 2)
+#define SCTLR_TRE (1 << 1)
+#define SCTLR_M (1 << 0)
+#define SCTLR_EAE_SBOP (SCTLR_AFE | SCTLR_TRE)
+
+#define RESUME_RETRY (0 << 0)
+#define RESUME_TERMINATE (1 << 0)
+
+#define TTBCR_EAE (1 << 31)
+
+#define TTBCR_PASIZE_SHIFT 16
+#define TTBCR_PASIZE_MASK 0x7
+
+#define TTBCR_TG0_4K (0 << 14)
+#define TTBCR_TG0_64K (1 << 14)
+
+#define TTBCR_SH0_SHIFT 12
+#define TTBCR_SH0_MASK 0x3
+#define TTBCR_SH_NS 0
+#define TTBCR_SH_OS 2
+#define TTBCR_SH_IS 3
+
+#define TTBCR_ORGN0_SHIFT 10
+#define TTBCR_IRGN0_SHIFT 8
+#define TTBCR_RGN_MASK 0x3
+#define TTBCR_RGN_NC 0
+#define TTBCR_RGN_WBWA 1
+#define TTBCR_RGN_WT 2
+#define TTBCR_RGN_WB 3
+
+#define TTBCR_SL0_SHIFT 6
+#define TTBCR_SL0_MASK 0x3
+#define TTBCR_SL0_LVL_2 0
+#define TTBCR_SL0_LVL_1 1
+
+#define TTBCR_T1SZ_SHIFT 16
+#define TTBCR_T0SZ_SHIFT 0
+#define TTBCR_SZ_MASK 0xf
+
+#define TTBCR2_SEP_SHIFT 15
+#define TTBCR2_SEP_MASK 0x7
+
+#define TTBCR2_PASIZE_SHIFT 0
+#define TTBCR2_PASIZE_MASK 0x7
+
+/* Common definitions for PASize and SEP fields */
+#define TTBCR2_ADDR_32 0
+#define TTBCR2_ADDR_36 1
+#define TTBCR2_ADDR_40 2
+#define TTBCR2_ADDR_42 3
+#define TTBCR2_ADDR_44 4
+#define TTBCR2_ADDR_48 5
+
+#define TTBRn_HI_ASID_SHIFT 16
+
+#define MAIR_ATTR_SHIFT(n) ((n) << 3)
+#define MAIR_ATTR_MASK 0xff
+#define MAIR_ATTR_DEVICE 0x04
+#define MAIR_ATTR_NC 0x44
+#define MAIR_ATTR_WBRWA 0xff
+#define MAIR_ATTR_IDX_NC 0
+#define MAIR_ATTR_IDX_CACHE 1
+#define MAIR_ATTR_IDX_DEV 2
+
+#define FSR_MULTI (1 << 31)
+#define FSR_SS (1 << 30)
+#define FSR_UUT (1 << 8)
+#define FSR_ASF (1 << 7)
+#define FSR_TLBLKF (1 << 6)
+#define FSR_TLBMCF (1 << 5)
+#define FSR_EF (1 << 4)
+#define FSR_PF (1 << 3)
+#define FSR_AFF (1 << 2)
+#define FSR_TF (1 << 1)
+
+#define FSR_IGN (FSR_AFF | FSR_ASF | \
+ FSR_TLBMCF | FSR_TLBLKF)
+#define FSR_FAULT (FSR_MULTI | FSR_SS | FSR_UUT | \
+ FSR_EF | FSR_PF | FSR_TF | FSR_IGN)
+
+#define FSYNR0_WNR (1 << 4)
+
+static int force_stage;
+module_param_named(force_stage, force_stage, int, S_IRUGO | S_IWUSR);
+MODULE_PARM_DESC(force_stage,
+ "Force SMMU mappings to be installed at a particular stage of translation. A value of '1' or '2' forces the corresponding stage. All other values are ignored (i.e. no stage is forced). Note that selecting a specific stage will disable support for nested translation.");
+
+enum arm_smmu_arch_version {
+ ARM_SMMU_V1 = 1,
+ ARM_SMMU_V2,
+};
+
+struct arm_smmu_smr {
+ u8 idx;
+ u16 mask;
+ u16 id;
+};
+
+struct arm_smmu_master_cfg {
+ int num_streamids;
+ u16 streamids[MAX_MASTER_STREAMIDS];
+ struct arm_smmu_smr *smrs;
+};
+
+struct arm_smmu_master {
+ struct device_node *of_node;
+ struct rb_node node;
+ struct arm_smmu_master_cfg cfg;
+};
+
+struct arm_smmu_device {
+ struct device *dev;
+
+ void __iomem *base;
+ unsigned long size;
+ unsigned long pgshift;
+
+#define ARM_SMMU_FEAT_COHERENT_WALK (1 << 0)
+#define ARM_SMMU_FEAT_STREAM_MATCH (1 << 1)
+#define ARM_SMMU_FEAT_TRANS_S1 (1 << 2)
+#define ARM_SMMU_FEAT_TRANS_S2 (1 << 3)
+#define ARM_SMMU_FEAT_TRANS_NESTED (1 << 4)
+ u32 features;
+
+#define ARM_SMMU_OPT_SECURE_CFG_ACCESS (1 << 0)
+ u32 options;
+ enum arm_smmu_arch_version version;
+
+ u32 num_context_banks;
+ u32 num_s2_context_banks;
+ DECLARE_BITMAP(context_map, ARM_SMMU_MAX_CBS);
+ atomic_t irptndx;
+
+ u32 num_mapping_groups;
+ DECLARE_BITMAP(smr_map, ARM_SMMU_MAX_SMRS);
+
+ unsigned long s1_input_size;
+ unsigned long s1_output_size;
+ unsigned long s2_input_size;
+ unsigned long s2_output_size;
+
+ u32 num_global_irqs;
+ u32 num_context_irqs;
+ unsigned int *irqs;
+
+ struct list_head list;
+ struct rb_root masters;
+};
+
+struct arm_smmu_cfg {
+ u8 cbndx;
+ u8 irptndx;
+ u32 cbar;
+ pgd_t *pgd;
+};
+#define INVALID_IRPTNDX 0xff
+
+#define ARM_SMMU_CB_ASID(cfg) ((cfg)->cbndx)
+#define ARM_SMMU_CB_VMID(cfg) ((cfg)->cbndx + 1)
+
+enum arm_smmu_domain_stage {
+ ARM_SMMU_DOMAIN_S1 = 0,
+ ARM_SMMU_DOMAIN_S2,
+ ARM_SMMU_DOMAIN_NESTED,
+};
+
+struct arm_smmu_domain {
+ struct arm_smmu_device *smmu;
+ struct arm_smmu_cfg cfg;
+ enum arm_smmu_domain_stage stage;
+ spinlock_t lock;
+};
+
+static DEFINE_SPINLOCK(arm_smmu_devices_lock);
+static LIST_HEAD(arm_smmu_devices);
+
+struct arm_smmu_option_prop {
+ u32 opt;
+ const char *prop;
+};
+
+static struct arm_smmu_option_prop arm_smmu_options[] = {
+ { ARM_SMMU_OPT_SECURE_CFG_ACCESS, "calxeda,smmu-secure-config-access" },
+ { 0, NULL},
+};
+
+static void parse_driver_options(struct arm_smmu_device *smmu)
+{
+ int i = 0;
+
+ do {
+ if (of_property_read_bool(smmu->dev->of_node,
+ arm_smmu_options[i].prop)) {
+ smmu->options |= arm_smmu_options[i].opt;
+ dev_notice(smmu->dev, "option %s\n",
+ arm_smmu_options[i].prop);
+ }
+ } while (arm_smmu_options[++i].opt);
+}
+
+static struct device_node *dev_get_dev_node(struct device *dev)
+{
+ if (dev_is_pci(dev)) {
+ struct pci_bus *bus = to_pci_dev(dev)->bus;
+
+ while (!pci_is_root_bus(bus))
+ bus = bus->parent;
+ return bus->bridge->parent->of_node;
+ }
+
+ return dev->of_node;
+}
+
+static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu,
+ struct device_node *dev_node)
+{
+ struct rb_node *node = smmu->masters.rb_node;
+
+ while (node) {
+ struct arm_smmu_master *master;
+
+ master = container_of(node, struct arm_smmu_master, node);
+
+ if (dev_node < master->of_node)
+ node = node->rb_left;
+ else if (dev_node > master->of_node)
+ node = node->rb_right;
+ else
+ return master;
+ }
+
+ return NULL;
+}
+
+static struct arm_smmu_master_cfg *
+find_smmu_master_cfg(struct device *dev)
+{
+ struct arm_smmu_master_cfg *cfg = NULL;
+ struct iommu_group *group = iommu_group_get(dev);
+
+ if (group) {
+ cfg = iommu_group_get_iommudata(group);
+ iommu_group_put(group);
+ }
+
+ return cfg;
+}
+
+static int insert_smmu_master(struct arm_smmu_device *smmu,
+ struct arm_smmu_master *master)
+{
+ struct rb_node **new, *parent;
+
+ new = &smmu->masters.rb_node;
+ parent = NULL;
+ while (*new) {
+ struct arm_smmu_master *this
+ = container_of(*new, struct arm_smmu_master, node);
+
+ parent = *new;
+ if (master->of_node < this->of_node)
+ new = &((*new)->rb_left);
+ else if (master->of_node > this->of_node)
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&master->node, parent, new);
+ rb_insert_color(&master->node, &smmu->masters);
+ return 0;
+}
+
+static int register_smmu_master(struct arm_smmu_device *smmu,
+ struct device *dev,
+ struct of_phandle_args *masterspec)
+{
+ int i;
+ struct arm_smmu_master *master;
+
+ master = find_smmu_master(smmu, masterspec->np);
+ if (master) {
+ dev_err(dev,
+ "rejecting multiple registrations for master device %s\n",
+ masterspec->np->name);
+ return -EBUSY;
+ }
+
+ if (masterspec->args_count > MAX_MASTER_STREAMIDS) {
+ dev_err(dev,
+ "reached maximum number (%d) of stream IDs for master device %s\n",
+ MAX_MASTER_STREAMIDS, masterspec->np->name);
+ return -ENOSPC;
+ }
+
+ master = devm_kzalloc(dev, sizeof(*master), GFP_KERNEL);
+ if (!master)
+ return -ENOMEM;
+
+ master->of_node = masterspec->np;
+ master->cfg.num_streamids = masterspec->args_count;
+
+ for (i = 0; i < master->cfg.num_streamids; ++i) {
+ u16 streamid = masterspec->args[i];
+
+ if (!(smmu->features & ARM_SMMU_FEAT_STREAM_MATCH) &&
+ (streamid >= smmu->num_mapping_groups)) {
+ dev_err(dev,
+ "stream ID for master device %s greater than maximum allowed (%d)\n",
+ masterspec->np->name, smmu->num_mapping_groups);
+ return -ERANGE;
+ }
+ master->cfg.streamids[i] = streamid;
+ }
+ return insert_smmu_master(smmu, master);
+}
+
+static struct arm_smmu_device *find_smmu_for_device(struct device *dev)
+{
+ struct arm_smmu_device *smmu;
+ struct arm_smmu_master *master = NULL;
+ struct device_node *dev_node = dev_get_dev_node(dev);
+
+ spin_lock(&arm_smmu_devices_lock);
+ list_for_each_entry(smmu, &arm_smmu_devices, list) {
+ master = find_smmu_master(smmu, dev_node);
+ if (master)
+ break;
+ }
+ spin_unlock(&arm_smmu_devices_lock);
+
+ return master ? smmu : NULL;
+}
+
+static int __arm_smmu_alloc_bitmap(unsigned long *map, int start, int end)
+{
+ int idx;
+
+ do {
+ idx = find_next_zero_bit(map, end, start);
+ if (idx == end)
+ return -ENOSPC;
+ } while (test_and_set_bit(idx, map));
+
+ return idx;
+}
+
+static void __arm_smmu_free_bitmap(unsigned long *map, int idx)
+{
+ clear_bit(idx, map);
+}
+
+/* Wait for any pending TLB invalidations to complete */
+static void arm_smmu_tlb_sync(struct arm_smmu_device *smmu)
+{
+ int count = 0;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+
+ writel_relaxed(0, gr0_base + ARM_SMMU_GR0_sTLBGSYNC);
+ while (readl_relaxed(gr0_base + ARM_SMMU_GR0_sTLBGSTATUS)
+ & sTLBGSTATUS_GSACTIVE) {
+ cpu_relax();
+ if (++count == TLB_LOOP_TIMEOUT) {
+ dev_err_ratelimited(smmu->dev,
+ "TLB sync timed out -- SMMU may be deadlocked\n");
+ return;
+ }
+ udelay(1);
+ }
+}
+
+static void arm_smmu_tlb_inv_context(struct arm_smmu_domain *smmu_domain)
+{
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ void __iomem *base = ARM_SMMU_GR0(smmu);
+ bool stage1 = cfg->cbar != CBAR_TYPE_S2_TRANS;
+
+ if (stage1) {
+ base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
+ writel_relaxed(ARM_SMMU_CB_ASID(cfg),
+ base + ARM_SMMU_CB_S1_TLBIASID);
+ } else {
+ base = ARM_SMMU_GR0(smmu);
+ writel_relaxed(ARM_SMMU_CB_VMID(cfg),
+ base + ARM_SMMU_GR0_TLBIVMID);
+ }
+
+ arm_smmu_tlb_sync(smmu);
+}
+
+static irqreturn_t arm_smmu_context_fault(int irq, void *dev)
+{
+ int flags, ret;
+ u32 fsr, far, fsynr, resume;
+ unsigned long iova;
+ struct iommu_domain *domain = dev;
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ void __iomem *cb_base;
+
+ cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
+ fsr = readl_relaxed(cb_base + ARM_SMMU_CB_FSR);
+
+ if (!(fsr & FSR_FAULT))
+ return IRQ_NONE;
+
+ if (fsr & FSR_IGN)
+ dev_err_ratelimited(smmu->dev,
+ "Unexpected context fault (fsr 0x%x)\n",
+ fsr);
+
+ fsynr = readl_relaxed(cb_base + ARM_SMMU_CB_FSYNR0);
+ flags = fsynr & FSYNR0_WNR ? IOMMU_FAULT_WRITE : IOMMU_FAULT_READ;
+
+ far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_LO);
+ iova = far;
+#ifdef CONFIG_64BIT
+ far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_HI);
+ iova |= ((unsigned long)far << 32);
+#endif
+
+ if (!report_iommu_fault(domain, smmu->dev, iova, flags)) {
+ ret = IRQ_HANDLED;
+ resume = RESUME_RETRY;
+ } else {
+ dev_err_ratelimited(smmu->dev,
+ "Unhandled context fault: iova=0x%08lx, fsynr=0x%x, cb=%d\n",
+ iova, fsynr, cfg->cbndx);
+ ret = IRQ_NONE;
+ resume = RESUME_TERMINATE;
+ }
+
+ /* Clear the faulting FSR */
+ writel(fsr, cb_base + ARM_SMMU_CB_FSR);
+
+ /* Retry or terminate any stalled transactions */
+ if (fsr & FSR_SS)
+ writel_relaxed(resume, cb_base + ARM_SMMU_CB_RESUME);
+
+ return ret;
+}
+
+static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
+{
+ u32 gfsr, gfsynr0, gfsynr1, gfsynr2;
+ struct arm_smmu_device *smmu = dev;
+ void __iomem *gr0_base = ARM_SMMU_GR0_NS(smmu);
+
+ gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+ gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR0);
+ gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR1);
+ gfsynr2 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR2);
+
+ if (!gfsr)
+ return IRQ_NONE;
+
+ dev_err_ratelimited(smmu->dev,
+ "Unexpected global fault, this could be serious\n");
+ dev_err_ratelimited(smmu->dev,
+ "\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n",
+ gfsr, gfsynr0, gfsynr1, gfsynr2);
+
+ writel(gfsr, gr0_base + ARM_SMMU_GR0_sGFSR);
+ return IRQ_HANDLED;
+}
+
+static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
+ size_t size)
+{
+ unsigned long offset = (unsigned long)addr & ~PAGE_MASK;
+
+
+ /* Ensure new page tables are visible to the hardware walker */
+ if (smmu->features & ARM_SMMU_FEAT_COHERENT_WALK) {
+ dsb(ishst);
+ } else {
+ /*
+ * If the SMMU can't walk tables in the CPU caches, treat them
+ * like non-coherent DMA since we need to flush the new entries
+ * all the way out to memory. There's no possibility of
+ * recursion here as the SMMU table walker will not be wired
+ * through another SMMU.
+ */
+ dma_map_page(smmu->dev, virt_to_page(addr), offset, size,
+ DMA_TO_DEVICE);
+ }
+}
+
+static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
+{
+ u32 reg;
+ bool stage1;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ void __iomem *cb_base, *gr0_base, *gr1_base;
+
+ gr0_base = ARM_SMMU_GR0(smmu);
+ gr1_base = ARM_SMMU_GR1(smmu);
+ stage1 = cfg->cbar != CBAR_TYPE_S2_TRANS;
+ cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
+
+ /* CBAR */
+ reg = cfg->cbar;
+ if (smmu->version == ARM_SMMU_V1)
+ reg |= cfg->irptndx << CBAR_IRPTNDX_SHIFT;
+
+ /*
+ * Use the weakest shareability/memory types, so they are
+ * overridden by the ttbcr/pte.
+ */
+ if (stage1) {
+ reg |= (CBAR_S1_BPSHCFG_NSH << CBAR_S1_BPSHCFG_SHIFT) |
+ (CBAR_S1_MEMATTR_WB << CBAR_S1_MEMATTR_SHIFT);
+ } else {
+ reg |= ARM_SMMU_CB_VMID(cfg) << CBAR_VMID_SHIFT;
+ }
+ writel_relaxed(reg, gr1_base + ARM_SMMU_GR1_CBAR(cfg->cbndx));
+
+ if (smmu->version > ARM_SMMU_V1) {
+ /* CBA2R */
+#ifdef CONFIG_64BIT
+ reg = CBA2R_RW64_64BIT;
+#else
+ reg = CBA2R_RW64_32BIT;
+#endif
+ writel_relaxed(reg,
+ gr1_base + ARM_SMMU_GR1_CBA2R(cfg->cbndx));
+
+ /* TTBCR2 */
+ switch (smmu->s1_input_size) {
+ case 32:
+ reg = (TTBCR2_ADDR_32 << TTBCR2_SEP_SHIFT);
+ break;
+ case 36:
+ reg = (TTBCR2_ADDR_36 << TTBCR2_SEP_SHIFT);
+ break;
+ case 39:
+ case 40:
+ reg = (TTBCR2_ADDR_40 << TTBCR2_SEP_SHIFT);
+ break;
+ case 42:
+ reg = (TTBCR2_ADDR_42 << TTBCR2_SEP_SHIFT);
+ break;
+ case 44:
+ reg = (TTBCR2_ADDR_44 << TTBCR2_SEP_SHIFT);
+ break;
+ case 48:
+ reg = (TTBCR2_ADDR_48 << TTBCR2_SEP_SHIFT);
+ break;
+ }
+
+ switch (smmu->s1_output_size) {
+ case 32:
+ reg |= (TTBCR2_ADDR_32 << TTBCR2_PASIZE_SHIFT);
+ break;
+ case 36:
+ reg |= (TTBCR2_ADDR_36 << TTBCR2_PASIZE_SHIFT);
+ break;
+ case 39:
+ case 40:
+ reg |= (TTBCR2_ADDR_40 << TTBCR2_PASIZE_SHIFT);
+ break;
+ case 42:
+ reg |= (TTBCR2_ADDR_42 << TTBCR2_PASIZE_SHIFT);
+ break;
+ case 44:
+ reg |= (TTBCR2_ADDR_44 << TTBCR2_PASIZE_SHIFT);
+ break;
+ case 48:
+ reg |= (TTBCR2_ADDR_48 << TTBCR2_PASIZE_SHIFT);
+ break;
+ }
+
+ if (stage1)
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBCR2);
+ }
+
+ /* TTBR0 */
+ arm_smmu_flush_pgtable(smmu, cfg->pgd,
+ PTRS_PER_PGD * sizeof(pgd_t));
+ reg = __pa(cfg->pgd);
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
+ reg = (phys_addr_t)__pa(cfg->pgd) >> 32;
+ if (stage1)
+ reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
+
+ /*
+ * TTBCR
+ * We use long descriptor, with inner-shareable WBWA tables in TTBR0.
+ */
+ if (smmu->version > ARM_SMMU_V1) {
+ if (PAGE_SIZE == SZ_4K)
+ reg = TTBCR_TG0_4K;
+ else
+ reg = TTBCR_TG0_64K;
+
+ if (!stage1) {
+ reg |= (64 - smmu->s2_input_size) << TTBCR_T0SZ_SHIFT;
+
+ switch (smmu->s2_output_size) {
+ case 32:
+ reg |= (TTBCR2_ADDR_32 << TTBCR_PASIZE_SHIFT);
+ break;
+ case 36:
+ reg |= (TTBCR2_ADDR_36 << TTBCR_PASIZE_SHIFT);
+ break;
+ case 40:
+ reg |= (TTBCR2_ADDR_40 << TTBCR_PASIZE_SHIFT);
+ break;
+ case 42:
+ reg |= (TTBCR2_ADDR_42 << TTBCR_PASIZE_SHIFT);
+ break;
+ case 44:
+ reg |= (TTBCR2_ADDR_44 << TTBCR_PASIZE_SHIFT);
+ break;
+ case 48:
+ reg |= (TTBCR2_ADDR_48 << TTBCR_PASIZE_SHIFT);
+ break;
+ }
+ } else {
+ reg |= (64 - smmu->s1_input_size) << TTBCR_T0SZ_SHIFT;
+ }
+ } else {
+ reg = 0;
+ }
+
+ reg |= TTBCR_EAE |
+ (TTBCR_SH_IS << TTBCR_SH0_SHIFT) |
+ (TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) |
+ (TTBCR_RGN_WBWA << TTBCR_IRGN0_SHIFT);
+
+ if (!stage1)
+ reg |= (TTBCR_SL0_LVL_1 << TTBCR_SL0_SHIFT);
+
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBCR);
+
+ /* MAIR0 (stage-1 only) */
+ if (stage1) {
+ reg = (MAIR_ATTR_NC << MAIR_ATTR_SHIFT(MAIR_ATTR_IDX_NC)) |
+ (MAIR_ATTR_WBRWA << MAIR_ATTR_SHIFT(MAIR_ATTR_IDX_CACHE)) |
+ (MAIR_ATTR_DEVICE << MAIR_ATTR_SHIFT(MAIR_ATTR_IDX_DEV));
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_S1_MAIR0);
+ }
+
+ /* SCTLR */
+ reg = SCTLR_CFCFG | SCTLR_CFIE | SCTLR_CFRE | SCTLR_M | SCTLR_EAE_SBOP;
+ if (stage1)
+ reg |= SCTLR_S1_ASIDPNE;
+#ifdef __BIG_ENDIAN
+ reg |= SCTLR_E;
+#endif
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_SCTLR);
+}
+
+static int arm_smmu_init_domain_context(struct iommu_domain *domain,
+ struct arm_smmu_device *smmu)
+{
+ int irq, start, ret = 0;
+ unsigned long flags;
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+
+ spin_lock_irqsave(&smmu_domain->lock, flags);
+ if (smmu_domain->smmu)
+ goto out_unlock;
+
+ /*
+ * Mapping the requested stage onto what we support is surprisingly
+ * complicated, mainly because the spec allows S1+S2 SMMUs without
+ * support for nested translation. That means we end up with the
+ * following table:
+ *
+ * Requested Supported Actual
+ * S1 N S1
+ * S1 S1+S2 S1
+ * S1 S2 S2
+ * S1 S1 S1
+ * N N N
+ * N S1+S2 S2
+ * N S2 S2
+ * N S1 S1
+ *
+ * Note that you can't actually request stage-2 mappings.
+ */
+ if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1))
+ smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
+ if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S2))
+ smmu_domain->stage = ARM_SMMU_DOMAIN_S1;
+
+ switch (smmu_domain->stage) {
+ case ARM_SMMU_DOMAIN_S1:
+ cfg->cbar = CBAR_TYPE_S1_TRANS_S2_BYPASS;
+ start = smmu->num_s2_context_banks;
+ break;
+ case ARM_SMMU_DOMAIN_NESTED:
+ /*
+ * We will likely want to change this if/when KVM gets
+ * involved.
+ */
+ case ARM_SMMU_DOMAIN_S2:
+ cfg->cbar = CBAR_TYPE_S2_TRANS;
+ start = 0;
+ break;
+ default:
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ ret = __arm_smmu_alloc_bitmap(smmu->context_map, start,
+ smmu->num_context_banks);
+ if (IS_ERR_VALUE(ret))
+ goto out_unlock;
+
+ cfg->cbndx = ret;
+ if (smmu->version == ARM_SMMU_V1) {
+ cfg->irptndx = atomic_inc_return(&smmu->irptndx);
+ cfg->irptndx %= smmu->num_context_irqs;
+ } else {
+ cfg->irptndx = cfg->cbndx;
+ }
+
+ ACCESS_ONCE(smmu_domain->smmu) = smmu;
+ arm_smmu_init_context_bank(smmu_domain);
+ spin_unlock_irqrestore(&smmu_domain->lock, flags);
+
+ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
+ ret = request_irq(irq, arm_smmu_context_fault, IRQF_SHARED,
+ "arm-smmu-context-fault", domain);
+ if (IS_ERR_VALUE(ret)) {
+ dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
+ cfg->irptndx, irq);
+ cfg->irptndx = INVALID_IRPTNDX;
+ }
+
+ return 0;
+
+out_unlock:
+ spin_unlock_irqrestore(&smmu_domain->lock, flags);
+ return ret;
+}
+
+static void arm_smmu_destroy_domain_context(struct iommu_domain *domain)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ void __iomem *cb_base;
+ int irq;
+
+ if (!smmu)
+ return;
+
+ /* Disable the context bank and nuke the TLB before freeing it. */
+ cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
+ writel_relaxed(0, cb_base + ARM_SMMU_CB_SCTLR);
+ arm_smmu_tlb_inv_context(smmu_domain);
+
+ if (cfg->irptndx != INVALID_IRPTNDX) {
+ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
+ free_irq(irq, domain);
+ }
+
+ __arm_smmu_free_bitmap(smmu->context_map, cfg->cbndx);
+}
+
+static int arm_smmu_domain_init(struct iommu_domain *domain)
+{
+ struct arm_smmu_domain *smmu_domain;
+ pgd_t *pgd;
+
+ /*
+ * Allocate the domain and initialise some of its data structures.
+ * We can't really do anything meaningful until we've added a
+ * master.
+ */
+ smmu_domain = kzalloc(sizeof(*smmu_domain), GFP_KERNEL);
+ if (!smmu_domain)
+ return -ENOMEM;
+
+ pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL);
+ if (!pgd)
+ goto out_free_domain;
+ smmu_domain->cfg.pgd = pgd;
+
+ spin_lock_init(&smmu_domain->lock);
+ domain->priv = smmu_domain;
+ return 0;
+
+out_free_domain:
+ kfree(smmu_domain);
+ return -ENOMEM;
+}
+
+static void arm_smmu_free_ptes(pmd_t *pmd)
+{
+ pgtable_t table = pmd_pgtable(*pmd);
+
+ __free_page(table);
+}
+
+static void arm_smmu_free_pmds(pud_t *pud)
+{
+ int i;
+ pmd_t *pmd, *pmd_base = pmd_offset(pud, 0);
+
+ pmd = pmd_base;
+ for (i = 0; i < PTRS_PER_PMD; ++i) {
+ if (pmd_none(*pmd))
+ continue;
+
+ arm_smmu_free_ptes(pmd);
+ pmd++;
+ }
+
+ pmd_free(NULL, pmd_base);
+}
+
+static void arm_smmu_free_puds(pgd_t *pgd)
+{
+ int i;
+ pud_t *pud, *pud_base = pud_offset(pgd, 0);
+
+ pud = pud_base;
+ for (i = 0; i < PTRS_PER_PUD; ++i) {
+ if (pud_none(*pud))
+ continue;
+
+ arm_smmu_free_pmds(pud);
+ pud++;
+ }
+
+ pud_free(NULL, pud_base);
+}
+
+static void arm_smmu_free_pgtables(struct arm_smmu_domain *smmu_domain)
+{
+ int i;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ pgd_t *pgd, *pgd_base = cfg->pgd;
+
+ /*
+ * Recursively free the page tables for this domain. We don't
+ * care about speculative TLB filling because the tables should
+ * not be active in any context bank at this point (SCTLR.M is 0).
+ */
+ pgd = pgd_base;
+ for (i = 0; i < PTRS_PER_PGD; ++i) {
+ if (pgd_none(*pgd))
+ continue;
+ arm_smmu_free_puds(pgd);
+ pgd++;
+ }
+
+ kfree(pgd_base);
+}
+
+static void arm_smmu_domain_destroy(struct iommu_domain *domain)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+
+ /*
+ * Free the domain resources. We assume that all devices have
+ * already been detached.
+ */
+ arm_smmu_destroy_domain_context(domain);
+ arm_smmu_free_pgtables(smmu_domain);
+ kfree(smmu_domain);
+}
+
+static int arm_smmu_master_configure_smrs(struct arm_smmu_device *smmu,
+ struct arm_smmu_master_cfg *cfg)
+{
+ int i;
+ struct arm_smmu_smr *smrs;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+
+ if (!(smmu->features & ARM_SMMU_FEAT_STREAM_MATCH))
+ return 0;
+
+ if (cfg->smrs)
+ return -EEXIST;
+
+ smrs = kmalloc_array(cfg->num_streamids, sizeof(*smrs), GFP_KERNEL);
+ if (!smrs) {
+ dev_err(smmu->dev, "failed to allocate %d SMRs\n",
+ cfg->num_streamids);
+ return -ENOMEM;
+ }
+
+ /* Allocate the SMRs on the SMMU */
+ for (i = 0; i < cfg->num_streamids; ++i) {
+ int idx = __arm_smmu_alloc_bitmap(smmu->smr_map, 0,
+ smmu->num_mapping_groups);
+ if (IS_ERR_VALUE(idx)) {
+ dev_err(smmu->dev, "failed to allocate free SMR\n");
+ goto err_free_smrs;
+ }
+
+ smrs[i] = (struct arm_smmu_smr) {
+ .idx = idx,
+ .mask = 0, /* We don't currently share SMRs */
+ .id = cfg->streamids[i],
+ };
+ }
+
+ /* It worked! Now, poke the actual hardware */
+ for (i = 0; i < cfg->num_streamids; ++i) {
+ u32 reg = SMR_VALID | smrs[i].id << SMR_ID_SHIFT |
+ smrs[i].mask << SMR_MASK_SHIFT;
+ writel_relaxed(reg, gr0_base + ARM_SMMU_GR0_SMR(smrs[i].idx));
+ }
+
+ cfg->smrs = smrs;
+ return 0;
+
+err_free_smrs:
+ while (--i >= 0)
+ __arm_smmu_free_bitmap(smmu->smr_map, smrs[i].idx);
+ kfree(smrs);
+ return -ENOSPC;
+}
+
+static void arm_smmu_master_free_smrs(struct arm_smmu_device *smmu,
+ struct arm_smmu_master_cfg *cfg)
+{
+ int i;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+ struct arm_smmu_smr *smrs = cfg->smrs;
+
+ if (!smrs)
+ return;
+
+ /* Invalidate the SMRs before freeing back to the allocator */
+ for (i = 0; i < cfg->num_streamids; ++i) {
+ u8 idx = smrs[i].idx;
+
+ writel_relaxed(~SMR_VALID, gr0_base + ARM_SMMU_GR0_SMR(idx));
+ __arm_smmu_free_bitmap(smmu->smr_map, idx);
+ }
+
+ cfg->smrs = NULL;
+ kfree(smrs);
+}
+
+static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain,
+ struct arm_smmu_master_cfg *cfg)
+{
+ int i, ret;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+
+ /* Devices in an IOMMU group may already be configured */
+ ret = arm_smmu_master_configure_smrs(smmu, cfg);
+ if (ret)
+ return ret == -EEXIST ? 0 : ret;
+
+ for (i = 0; i < cfg->num_streamids; ++i) {
+ u32 idx, s2cr;
+
+ idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i];
+ s2cr = S2CR_TYPE_TRANS |
+ (smmu_domain->cfg.cbndx << S2CR_CBNDX_SHIFT);
+ writel_relaxed(s2cr, gr0_base + ARM_SMMU_GR0_S2CR(idx));
+ }
+
+ return 0;
+}
+
+static void arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain,
+ struct arm_smmu_master_cfg *cfg)
+{
+ int i;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+
+ /* An IOMMU group is torn down by the first device to be removed */
+ if ((smmu->features & ARM_SMMU_FEAT_STREAM_MATCH) && !cfg->smrs)
+ return;
+
+ /*
+ * We *must* clear the S2CR first, because freeing the SMR means
+ * that it can be re-allocated immediately.
+ */
+ for (i = 0; i < cfg->num_streamids; ++i) {
+ u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i];
+
+ writel_relaxed(S2CR_TYPE_BYPASS,
+ gr0_base + ARM_SMMU_GR0_S2CR(idx));
+ }
+
+ arm_smmu_master_free_smrs(smmu, cfg);
+}
+
+static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
+{
+ int ret;
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_device *smmu, *dom_smmu;
+ struct arm_smmu_master_cfg *cfg;
+
+ smmu = find_smmu_for_device(dev);
+ if (!smmu) {
+ dev_err(dev, "cannot attach to SMMU, is it on the same bus?\n");
+ return -ENXIO;
+ }
+
+ if (dev->archdata.iommu) {
+ dev_err(dev, "already attached to IOMMU domain\n");
+ return -EEXIST;
+ }
+
+ /*
+ * Sanity check the domain. We don't support domains across
+ * different SMMUs.
+ */
+ dom_smmu = ACCESS_ONCE(smmu_domain->smmu);
+ if (!dom_smmu) {
+ /* Now that we have a master, we can finalise the domain */
+ ret = arm_smmu_init_domain_context(domain, smmu);
+ if (IS_ERR_VALUE(ret))
+ return ret;
+
+ dom_smmu = smmu_domain->smmu;
+ }
+
+ if (dom_smmu != smmu) {
+ dev_err(dev,
+ "cannot attach to SMMU %s whilst already attached to domain on SMMU %s\n",
+ dev_name(smmu_domain->smmu->dev), dev_name(smmu->dev));
+ return -EINVAL;
+ }
+
+ /* Looks ok, so add the device to the domain */
+ cfg = find_smmu_master_cfg(dev);
+ if (!cfg)
+ return -ENODEV;
+
+ ret = arm_smmu_domain_add_master(smmu_domain, cfg);
+ if (!ret)
+ dev->archdata.iommu = domain;
+ return ret;
+}
+
+static void arm_smmu_detach_dev(struct iommu_domain *domain, struct device *dev)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_master_cfg *cfg;
+
+ cfg = find_smmu_master_cfg(dev);
+ if (!cfg)
+ return;
+
+ dev->archdata.iommu = NULL;
+ arm_smmu_domain_remove_master(smmu_domain, cfg);
+}
+
+static bool arm_smmu_pte_is_contiguous_range(unsigned long addr,
+ unsigned long end)
+{
+ return !(addr & ~ARM_SMMU_PTE_CONT_MASK) &&
+ (addr + ARM_SMMU_PTE_CONT_SIZE <= end);
+}
+
+static int arm_smmu_alloc_init_pte(struct arm_smmu_device *smmu, pmd_t *pmd,
+ unsigned long addr, unsigned long end,
+ unsigned long pfn, int prot, int stage)
+{
+ pte_t *pte, *start;
+ pteval_t pteval = ARM_SMMU_PTE_PAGE | ARM_SMMU_PTE_AF;
+
+ if (pmd_none(*pmd)) {
+ /* Allocate a new set of tables */
+ pgtable_t table = alloc_page(GFP_ATOMIC|__GFP_ZERO);
+
+ if (!table)
+ return -ENOMEM;
+
+ arm_smmu_flush_pgtable(smmu, page_address(table), PAGE_SIZE);
+ pmd_populate(NULL, pmd, table);
+ arm_smmu_flush_pgtable(smmu, pmd, sizeof(*pmd));
+ }
+
+ if (stage == 1) {
+ pteval |= ARM_SMMU_PTE_AP_UNPRIV | ARM_SMMU_PTE_nG;
+ if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
+ pteval |= ARM_SMMU_PTE_AP_RDONLY;
+
+ if (prot & IOMMU_CACHE)
+ pteval |= (MAIR_ATTR_IDX_CACHE <<
+ ARM_SMMU_PTE_ATTRINDX_SHIFT);
+ } else {
+ pteval |= ARM_SMMU_PTE_HAP_FAULT;
+ if (prot & IOMMU_READ)
+ pteval |= ARM_SMMU_PTE_HAP_READ;
+ if (prot & IOMMU_WRITE)
+ pteval |= ARM_SMMU_PTE_HAP_WRITE;
+ if (prot & IOMMU_CACHE)
+ pteval |= ARM_SMMU_PTE_MEMATTR_OIWB;
+ else
+ pteval |= ARM_SMMU_PTE_MEMATTR_NC;
+ }
+
+ if (prot & IOMMU_NOEXEC)
+ pteval |= ARM_SMMU_PTE_XN;
+
+ /* If no access, create a faulting entry to avoid TLB fills */
+ if (!(prot & (IOMMU_READ | IOMMU_WRITE)))
+ pteval &= ~ARM_SMMU_PTE_PAGE;
+
+ pteval |= ARM_SMMU_PTE_SH_IS;
+ start = pmd_page_vaddr(*pmd) + pte_index(addr);
+ pte = start;
+
+ /*
+ * Install the page table entries. This is fairly complicated
+ * since we attempt to make use of the contiguous hint in the
+ * ptes where possible. The contiguous hint indicates a series
+ * of ARM_SMMU_PTE_CONT_ENTRIES ptes mapping a physically
+ * contiguous region with the following constraints:
+ *
+ * - The region start is aligned to ARM_SMMU_PTE_CONT_SIZE
+ * - Each pte in the region has the contiguous hint bit set
+ *
+ * This complicates unmapping (also handled by this code, when
+ * neither IOMMU_READ or IOMMU_WRITE are set) because it is
+ * possible, yet highly unlikely, that a client may unmap only
+ * part of a contiguous range. This requires clearing of the
+ * contiguous hint bits in the range before installing the new
+ * faulting entries.
+ *
+ * Note that re-mapping an address range without first unmapping
+ * it is not supported, so TLB invalidation is not required here
+ * and is instead performed at unmap and domain-init time.
+ */
+ do {
+ int i = 1;
+
+ pteval &= ~ARM_SMMU_PTE_CONT;
+
+ if (arm_smmu_pte_is_contiguous_range(addr, end)) {
+ i = ARM_SMMU_PTE_CONT_ENTRIES;
+ pteval |= ARM_SMMU_PTE_CONT;
+ } else if (pte_val(*pte) &
+ (ARM_SMMU_PTE_CONT | ARM_SMMU_PTE_PAGE)) {
+ int j;
+ pte_t *cont_start;
+ unsigned long idx = pte_index(addr);
+
+ idx &= ~(ARM_SMMU_PTE_CONT_ENTRIES - 1);
+ cont_start = pmd_page_vaddr(*pmd) + idx;
+ for (j = 0; j < ARM_SMMU_PTE_CONT_ENTRIES; ++j)
+ pte_val(*(cont_start + j)) &=
+ ~ARM_SMMU_PTE_CONT;
+
+ arm_smmu_flush_pgtable(smmu, cont_start,
+ sizeof(*pte) *
+ ARM_SMMU_PTE_CONT_ENTRIES);
+ }
+
+ do {
+ *pte = pfn_pte(pfn, __pgprot(pteval));
+ } while (pte++, pfn++, addr += PAGE_SIZE, --i);
+ } while (addr != end);
+
+ arm_smmu_flush_pgtable(smmu, start, sizeof(*pte) * (pte - start));
+ return 0;
+}
+
+static int arm_smmu_alloc_init_pmd(struct arm_smmu_device *smmu, pud_t *pud,
+ unsigned long addr, unsigned long end,
+ phys_addr_t phys, int prot, int stage)
+{
+ int ret;
+ pmd_t *pmd;
+ unsigned long next, pfn = __phys_to_pfn(phys);
+
+#ifndef __PAGETABLE_PMD_FOLDED
+ if (pud_none(*pud)) {
+ pmd = (pmd_t *)get_zeroed_page(GFP_ATOMIC);
+ if (!pmd)
+ return -ENOMEM;
+
+ arm_smmu_flush_pgtable(smmu, pmd, PAGE_SIZE);
+ pud_populate(NULL, pud, pmd);
+ arm_smmu_flush_pgtable(smmu, pud, sizeof(*pud));
+
+ pmd += pmd_index(addr);
+ } else
+#endif
+ pmd = pmd_offset(pud, addr);
+
+ do {
+ next = pmd_addr_end(addr, end);
+ ret = arm_smmu_alloc_init_pte(smmu, pmd, addr, next, pfn,
+ prot, stage);
+ phys += next - addr;
+ pfn = __phys_to_pfn(phys);
+ } while (pmd++, addr = next, addr < end);
+
+ return ret;
+}
+
+static int arm_smmu_alloc_init_pud(struct arm_smmu_device *smmu, pgd_t *pgd,
+ unsigned long addr, unsigned long end,
+ phys_addr_t phys, int prot, int stage)
+{
+ int ret = 0;
+ pud_t *pud;
+ unsigned long next;
+
+#ifndef __PAGETABLE_PUD_FOLDED
+ if (pgd_none(*pgd)) {
+ pud = (pud_t *)get_zeroed_page(GFP_ATOMIC);
+ if (!pud)
+ return -ENOMEM;
+
+ arm_smmu_flush_pgtable(smmu, pud, PAGE_SIZE);
+ pgd_populate(NULL, pgd, pud);
+ arm_smmu_flush_pgtable(smmu, pgd, sizeof(*pgd));
+
+ pud += pud_index(addr);
+ } else
+#endif
+ pud = pud_offset(pgd, addr);
+
+ do {
+ next = pud_addr_end(addr, end);
+ ret = arm_smmu_alloc_init_pmd(smmu, pud, addr, next, phys,
+ prot, stage);
+ phys += next - addr;
+ } while (pud++, addr = next, addr < end);
+
+ return ret;
+}
+
+static int arm_smmu_handle_mapping(struct arm_smmu_domain *smmu_domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ int ret, stage;
+ unsigned long end;
+ phys_addr_t input_mask, output_mask;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+ pgd_t *pgd = cfg->pgd;
+ unsigned long flags;
+
+ if (cfg->cbar == CBAR_TYPE_S2_TRANS) {
+ stage = 2;
+ input_mask = (1ULL << smmu->s2_input_size) - 1;
+ output_mask = (1ULL << smmu->s2_output_size) - 1;
+ } else {
+ stage = 1;
+ input_mask = (1ULL << smmu->s1_input_size) - 1;
+ output_mask = (1ULL << smmu->s1_output_size) - 1;
+ }
+
+ if (!pgd)
+ return -EINVAL;
+
+ if (size & ~PAGE_MASK)
+ return -EINVAL;
+
+ if ((phys_addr_t)iova & ~input_mask)
+ return -ERANGE;
+
+ if (paddr & ~output_mask)
+ return -ERANGE;
+
+ spin_lock_irqsave(&smmu_domain->lock, flags);
+ pgd += pgd_index(iova);
+ end = iova + size;
+ do {
+ unsigned long next = pgd_addr_end(iova, end);
+
+ ret = arm_smmu_alloc_init_pud(smmu, pgd, iova, next, paddr,
+ prot, stage);
+ if (ret)
+ goto out_unlock;
+
+ paddr += next - iova;
+ iova = next;
+ } while (pgd++, iova != end);
+
+out_unlock:
+ spin_unlock_irqrestore(&smmu_domain->lock, flags);
+
+ return ret;
+}
+
+static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+
+ if (!smmu_domain)
+ return -ENODEV;
+
+ return arm_smmu_handle_mapping(smmu_domain, iova, paddr, size, prot);
+}
+
+static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ int ret;
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+
+ ret = arm_smmu_handle_mapping(smmu_domain, iova, 0, size, 0);
+ arm_smmu_tlb_inv_context(smmu_domain);
+ return ret ? 0 : size;
+}
+
+static phys_addr_t arm_smmu_iova_to_phys(struct iommu_domain *domain,
+ dma_addr_t iova)
+{
+ pgd_t *pgdp, pgd;
+ pud_t pud;
+ pmd_t pmd;
+ pte_t pte;
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+ struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+
+ pgdp = cfg->pgd;
+ if (!pgdp)
+ return 0;
+
+ pgd = *(pgdp + pgd_index(iova));
+ if (pgd_none(pgd))
+ return 0;
+
+ pud = *pud_offset(&pgd, iova);
+ if (pud_none(pud))
+ return 0;
+
+ pmd = *pmd_offset(&pud, iova);
+ if (pmd_none(pmd))
+ return 0;
+
+ pte = *(pmd_page_vaddr(pmd) + pte_index(iova));
+ if (pte_none(pte))
+ return 0;
+
+ return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK);
+}
+
+static bool arm_smmu_capable(enum iommu_cap cap)
+{
+ switch (cap) {
+ case IOMMU_CAP_CACHE_COHERENCY:
+ /*
+ * Return true here as the SMMU can always send out coherent
+ * requests.
+ */
+ return true;
+ case IOMMU_CAP_INTR_REMAP:
+ return true; /* MSIs are just memory writes */
+ case IOMMU_CAP_NOEXEC:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data)
+{
+ *((u16 *)data) = alias;
+ return 0; /* Continue walking */
+}
+
+static void __arm_smmu_release_pci_iommudata(void *data)
+{
+ kfree(data);
+}
+
+static int arm_smmu_add_device(struct device *dev)
+{
+ struct arm_smmu_device *smmu;
+ struct arm_smmu_master_cfg *cfg;
+ struct iommu_group *group;
+ void (*releasefn)(void *) = NULL;
+ int ret;
+
+ smmu = find_smmu_for_device(dev);
+ if (!smmu)
+ return -ENODEV;
+
+ group = iommu_group_alloc();
+ if (IS_ERR(group)) {
+ dev_err(dev, "Failed to allocate IOMMU group\n");
+ return PTR_ERR(group);
+ }
+
+ if (dev_is_pci(dev)) {
+ struct pci_dev *pdev = to_pci_dev(dev);
+
+ cfg = kzalloc(sizeof(*cfg), GFP_KERNEL);
+ if (!cfg) {
+ ret = -ENOMEM;
+ goto out_put_group;
+ }
+
+ cfg->num_streamids = 1;
+ /*
+ * Assume Stream ID == Requester ID for now.
+ * We need a way to describe the ID mappings in FDT.
+ */
+ pci_for_each_dma_alias(pdev, __arm_smmu_get_pci_sid,
+ &cfg->streamids[0]);
+ releasefn = __arm_smmu_release_pci_iommudata;
+ } else {
+ struct arm_smmu_master *master;
+
+ master = find_smmu_master(smmu, dev->of_node);
+ if (!master) {
+ ret = -ENODEV;
+ goto out_put_group;
+ }
+
+ cfg = &master->cfg;
+ }
+
+ iommu_group_set_iommudata(group, cfg, releasefn);
+ ret = iommu_group_add_device(group, dev);
+
+out_put_group:
+ iommu_group_put(group);
+ return ret;
+}
+
+static void arm_smmu_remove_device(struct device *dev)
+{
+ iommu_group_remove_device(dev);
+}
+
+static int arm_smmu_domain_get_attr(struct iommu_domain *domain,
+ enum iommu_attr attr, void *data)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+
+ switch (attr) {
+ case DOMAIN_ATTR_NESTING:
+ *(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED);
+ return 0;
+ default:
+ return -ENODEV;
+ }
+}
+
+static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
+ enum iommu_attr attr, void *data)
+{
+ struct arm_smmu_domain *smmu_domain = domain->priv;
+
+ switch (attr) {
+ case DOMAIN_ATTR_NESTING:
+ if (smmu_domain->smmu)
+ return -EPERM;
+ if (*(int *)data)
+ smmu_domain->stage = ARM_SMMU_DOMAIN_NESTED;
+ else
+ smmu_domain->stage = ARM_SMMU_DOMAIN_S1;
+
+ return 0;
+ default:
+ return -ENODEV;
+ }
+}
+
+static const struct iommu_ops arm_smmu_ops = {
+ .capable = arm_smmu_capable,
+ .domain_init = arm_smmu_domain_init,
+ .domain_destroy = arm_smmu_domain_destroy,
+ .attach_dev = arm_smmu_attach_dev,
+ .detach_dev = arm_smmu_detach_dev,
+ .map = arm_smmu_map,
+ .unmap = arm_smmu_unmap,
+ .map_sg = default_iommu_map_sg,
+ .iova_to_phys = arm_smmu_iova_to_phys,
+ .add_device = arm_smmu_add_device,
+ .remove_device = arm_smmu_remove_device,
+ .domain_get_attr = arm_smmu_domain_get_attr,
+ .domain_set_attr = arm_smmu_domain_set_attr,
+ .pgsize_bitmap = (SECTION_SIZE |
+ ARM_SMMU_PTE_CONT_SIZE |
+ PAGE_SIZE),
+};
+
+static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
+{
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+ void __iomem *cb_base;
+ int i = 0;
+ u32 reg;
+
+ /* clear global FSR */
+ reg = readl_relaxed(ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sGFSR);
+ writel(reg, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sGFSR);
+
+ /* Mark all SMRn as invalid and all S2CRn as bypass */
+ for (i = 0; i < smmu->num_mapping_groups; ++i) {
+ writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i));
+ writel_relaxed(S2CR_TYPE_BYPASS,
+ gr0_base + ARM_SMMU_GR0_S2CR(i));
+ }
+
+ /* Make sure all context banks are disabled and clear CB_FSR */
+ for (i = 0; i < smmu->num_context_banks; ++i) {
+ cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, i);
+ writel_relaxed(0, cb_base + ARM_SMMU_CB_SCTLR);
+ writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+ }
+
+ /* Invalidate the TLB, just in case */
+ writel_relaxed(0, gr0_base + ARM_SMMU_GR0_STLBIALL);
+ writel_relaxed(0, gr0_base + ARM_SMMU_GR0_TLBIALLH);
+ writel_relaxed(0, gr0_base + ARM_SMMU_GR0_TLBIALLNSNH);
+
+ reg = readl_relaxed(ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
+
+ /* Enable fault reporting */
+ reg |= (sCR0_GFRE | sCR0_GFIE | sCR0_GCFGFRE | sCR0_GCFGFIE);
+
+ /* Disable TLB broadcasting. */
+ reg |= (sCR0_VMIDPNE | sCR0_PTM);
+
+ /* Enable client access, but bypass when no mapping is found */
+ reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
+
+ /* Disable forced broadcasting */
+ reg &= ~sCR0_FB;
+
+ /* Don't upgrade barriers */
+ reg &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
+
+ /* Push the button */
+ arm_smmu_tlb_sync(smmu);
+ writel(reg, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
+}
+
+static int arm_smmu_id_size_to_bits(int size)
+{
+ switch (size) {
+ case 0:
+ return 32;
+ case 1:
+ return 36;
+ case 2:
+ return 40;
+ case 3:
+ return 42;
+ case 4:
+ return 44;
+ case 5:
+ default:
+ return 48;
+ }
+}
+
+static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
+{
+ unsigned long size;
+ void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
+ u32 id;
+
+ dev_notice(smmu->dev, "probing hardware configuration...\n");
+ dev_notice(smmu->dev, "SMMUv%d with:\n", smmu->version);
+
+ /* ID0 */
+ id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID0);
+#ifndef CONFIG_64BIT
+ if (((id >> ID0_PTFS_SHIFT) & ID0_PTFS_MASK) == ID0_PTFS_V8_ONLY) {
+ dev_err(smmu->dev, "\tno v7 descriptor support!\n");
+ return -ENODEV;
+ }
+#endif
+
+ /* Restrict available stages based on module parameter */
+ if (force_stage == 1)
+ id &= ~(ID0_S2TS | ID0_NTS);
+ else if (force_stage == 2)
+ id &= ~(ID0_S1TS | ID0_NTS);
+
+ if (id & ID0_S1TS) {
+ smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
+ dev_notice(smmu->dev, "\tstage 1 translation\n");
+ }
+
+ if (id & ID0_S2TS) {
+ smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
+ dev_notice(smmu->dev, "\tstage 2 translation\n");
+ }
+
+ if (id & ID0_NTS) {
+ smmu->features |= ARM_SMMU_FEAT_TRANS_NESTED;
+ dev_notice(smmu->dev, "\tnested translation\n");
+ }
+
+ if (!(smmu->features &
+ (ARM_SMMU_FEAT_TRANS_S1 | ARM_SMMU_FEAT_TRANS_S2))) {
+ dev_err(smmu->dev, "\tno translation support!\n");
+ return -ENODEV;
+ }
+
+ if (id & ID0_CTTW) {
+ smmu->features |= ARM_SMMU_FEAT_COHERENT_WALK;
+ dev_notice(smmu->dev, "\tcoherent table walk\n");
+ }
+
+ if (id & ID0_SMS) {
+ u32 smr, sid, mask;
+
+ smmu->features |= ARM_SMMU_FEAT_STREAM_MATCH;
+ smmu->num_mapping_groups = (id >> ID0_NUMSMRG_SHIFT) &
+ ID0_NUMSMRG_MASK;
+ if (smmu->num_mapping_groups == 0) {
+ dev_err(smmu->dev,
+ "stream-matching supported, but no SMRs present!\n");
+ return -ENODEV;
+ }
+
+ smr = SMR_MASK_MASK << SMR_MASK_SHIFT;
+ smr |= (SMR_ID_MASK << SMR_ID_SHIFT);
+ writel_relaxed(smr, gr0_base + ARM_SMMU_GR0_SMR(0));
+ smr = readl_relaxed(gr0_base + ARM_SMMU_GR0_SMR(0));
+
+ mask = (smr >> SMR_MASK_SHIFT) & SMR_MASK_MASK;
+ sid = (smr >> SMR_ID_SHIFT) & SMR_ID_MASK;
+ if ((mask & sid) != sid) {
+ dev_err(smmu->dev,
+ "SMR mask bits (0x%x) insufficient for ID field (0x%x)\n",
+ mask, sid);
+ return -ENODEV;
+ }
+
+ dev_notice(smmu->dev,
+ "\tstream matching with %u register groups, mask 0x%x",
+ smmu->num_mapping_groups, mask);
+ } else {
+ smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) &
+ ID0_NUMSIDB_MASK;
+ }
+
+ /* ID1 */
+ id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID1);
+ smmu->pgshift = (id & ID1_PAGESIZE) ? 16 : 12;
+
+ /* Check for size mismatch of SMMU address space from mapped region */
+ size = 1 <<
+ (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
+ size *= 2 << smmu->pgshift;
+ if (smmu->size != size)
+ dev_warn(smmu->dev,
+ "SMMU address space size (0x%lx) differs from mapped region size (0x%lx)!\n",
+ size, smmu->size);
+
+ smmu->num_s2_context_banks = (id >> ID1_NUMS2CB_SHIFT) &
+ ID1_NUMS2CB_MASK;
+ smmu->num_context_banks = (id >> ID1_NUMCB_SHIFT) & ID1_NUMCB_MASK;
+ if (smmu->num_s2_context_banks > smmu->num_context_banks) {
+ dev_err(smmu->dev, "impossible number of S2 context banks!\n");
+ return -ENODEV;
+ }
+ dev_notice(smmu->dev, "\t%u context banks (%u stage-2 only)\n",
+ smmu->num_context_banks, smmu->num_s2_context_banks);
+
+ /* ID2 */
+ id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID2);
+ size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK);
+ smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
+
+ /* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */
+#ifdef CONFIG_64BIT
+ smmu->s2_input_size = min_t(unsigned long, VA_BITS, size);
+#else
+ smmu->s2_input_size = min(32UL, size);
+#endif
+
+ /* The stage-2 output mask is also applied for bypass */
+ size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK);
+ smmu->s2_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
+
+ if (smmu->version == ARM_SMMU_V1) {
+ smmu->s1_input_size = 32;
+ } else {
+#ifdef CONFIG_64BIT
+ size = (id >> ID2_UBS_SHIFT) & ID2_UBS_MASK;
+ size = min(VA_BITS, arm_smmu_id_size_to_bits(size));
+#else
+ size = 32;
+#endif
+ smmu->s1_input_size = size;
+
+ if ((PAGE_SIZE == SZ_4K && !(id & ID2_PTFS_4K)) ||
+ (PAGE_SIZE == SZ_64K && !(id & ID2_PTFS_64K)) ||
+ (PAGE_SIZE != SZ_4K && PAGE_SIZE != SZ_64K)) {
+ dev_err(smmu->dev, "CPU page size 0x%lx unsupported\n",
+ PAGE_SIZE);
+ return -ENODEV;
+ }
+ }
+
+ if (smmu->features & ARM_SMMU_FEAT_TRANS_S1)
+ dev_notice(smmu->dev, "\tStage-1: %lu-bit VA -> %lu-bit IPA\n",
+ smmu->s1_input_size, smmu->s1_output_size);
+
+ if (smmu->features & ARM_SMMU_FEAT_TRANS_S2)
+ dev_notice(smmu->dev, "\tStage-2: %lu-bit IPA -> %lu-bit PA\n",
+ smmu->s2_input_size, smmu->s2_output_size);
+
+ return 0;
+}
+
+static const struct of_device_id arm_smmu_of_match[] = {
+ { .compatible = "arm,smmu-v1", .data = (void *)ARM_SMMU_V1 },
+ { .compatible = "arm,smmu-v2", .data = (void *)ARM_SMMU_V2 },
+ { .compatible = "arm,mmu-400", .data = (void *)ARM_SMMU_V1 },
+ { .compatible = "arm,mmu-401", .data = (void *)ARM_SMMU_V1 },
+ { .compatible = "arm,mmu-500", .data = (void *)ARM_SMMU_V2 },
+ { },
+};
+MODULE_DEVICE_TABLE(of, arm_smmu_of_match);
+
+static int arm_smmu_device_dt_probe(struct platform_device *pdev)
+{
+ const struct of_device_id *of_id;
+ struct resource *res;
+ struct arm_smmu_device *smmu;
+ struct device *dev = &pdev->dev;
+ struct rb_node *node;
+ struct of_phandle_args masterspec;
+ int num_irqs, i, err;
+
+ smmu = devm_kzalloc(dev, sizeof(*smmu), GFP_KERNEL);
+ if (!smmu) {
+ dev_err(dev, "failed to allocate arm_smmu_device\n");
+ return -ENOMEM;
+ }
+ smmu->dev = dev;
+
+ of_id = of_match_node(arm_smmu_of_match, dev->of_node);
+ smmu->version = (enum arm_smmu_arch_version)of_id->data;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ smmu->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(smmu->base))
+ return PTR_ERR(smmu->base);
+ smmu->size = resource_size(res);
+
+ if (of_property_read_u32(dev->of_node, "#global-interrupts",
+ &smmu->num_global_irqs)) {
+ dev_err(dev, "missing #global-interrupts property\n");
+ return -ENODEV;
+ }
+
+ num_irqs = 0;
+ while ((res = platform_get_resource(pdev, IORESOURCE_IRQ, num_irqs))) {
+ num_irqs++;
+ if (num_irqs > smmu->num_global_irqs)
+ smmu->num_context_irqs++;
+ }
+
+ if (!smmu->num_context_irqs) {
+ dev_err(dev, "found %d interrupts but expected at least %d\n",
+ num_irqs, smmu->num_global_irqs + 1);
+ return -ENODEV;
+ }
+
+ smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
+ GFP_KERNEL);
+ if (!smmu->irqs) {
+ dev_err(dev, "failed to allocate %d irqs\n", num_irqs);
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < num_irqs; ++i) {
+ int irq = platform_get_irq(pdev, i);
+
+ if (irq < 0) {
+ dev_err(dev, "failed to get irq index %d\n", i);
+ return -ENODEV;
+ }
+ smmu->irqs[i] = irq;
+ }
+
+ err = arm_smmu_device_cfg_probe(smmu);
+ if (err)
+ return err;
+
+ i = 0;
+ smmu->masters = RB_ROOT;
+ while (!of_parse_phandle_with_args(dev->of_node, "mmu-masters",
+ "#stream-id-cells", i,
+ &masterspec)) {
+ err = register_smmu_master(smmu, dev, &masterspec);
+ if (err) {
+ dev_err(dev, "failed to add master %s\n",
+ masterspec.np->name);
+ goto out_put_masters;
+ }
+
+ i++;
+ }
+ dev_notice(dev, "registered %d master devices\n", i);
+
+ parse_driver_options(smmu);
+
+ if (smmu->version > ARM_SMMU_V1 &&
+ smmu->num_context_banks != smmu->num_context_irqs) {
+ dev_err(dev,
+ "found only %d context interrupt(s) but %d required\n",
+ smmu->num_context_irqs, smmu->num_context_banks);
+ err = -ENODEV;
+ goto out_put_masters;
+ }
+
+ for (i = 0; i < smmu->num_global_irqs; ++i) {
+ err = request_irq(smmu->irqs[i],
+ arm_smmu_global_fault,
+ IRQF_SHARED,
+ "arm-smmu global fault",
+ smmu);
+ if (err) {
+ dev_err(dev, "failed to request global IRQ %d (%u)\n",
+ i, smmu->irqs[i]);
+ goto out_free_irqs;
+ }
+ }
+
+ INIT_LIST_HEAD(&smmu->list);
+ spin_lock(&arm_smmu_devices_lock);
+ list_add(&smmu->list, &arm_smmu_devices);
+ spin_unlock(&arm_smmu_devices_lock);
+
+ arm_smmu_device_reset(smmu);
+ return 0;
+
+out_free_irqs:
+ while (i--)
+ free_irq(smmu->irqs[i], smmu);
+
+out_put_masters:
+ for (node = rb_first(&smmu->masters); node; node = rb_next(node)) {
+ struct arm_smmu_master *master
+ = container_of(node, struct arm_smmu_master, node);
+ of_node_put(master->of_node);
+ }
+
+ return err;
+}
+
+static int arm_smmu_device_remove(struct platform_device *pdev)
+{
+ int i;
+ struct device *dev = &pdev->dev;
+ struct arm_smmu_device *curr, *smmu = NULL;
+ struct rb_node *node;
+
+ spin_lock(&arm_smmu_devices_lock);
+ list_for_each_entry(curr, &arm_smmu_devices, list) {
+ if (curr->dev == dev) {
+ smmu = curr;
+ list_del(&smmu->list);
+ break;
+ }
+ }
+ spin_unlock(&arm_smmu_devices_lock);
+
+ if (!smmu)
+ return -ENODEV;
+
+ for (node = rb_first(&smmu->masters); node; node = rb_next(node)) {
+ struct arm_smmu_master *master
+ = container_of(node, struct arm_smmu_master, node);
+ of_node_put(master->of_node);
+ }
+
+ if (!bitmap_empty(smmu->context_map, ARM_SMMU_MAX_CBS))
+ dev_err(dev, "removing device with active domains!\n");
+
+ for (i = 0; i < smmu->num_global_irqs; ++i)
+ free_irq(smmu->irqs[i], smmu);
+
+ /* Turn the thing off */
+ writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
+ return 0;
+}
+
+static struct platform_driver arm_smmu_driver = {
+ .driver = {
+ .name = "arm-smmu",
+ .of_match_table = of_match_ptr(arm_smmu_of_match),
+ },
+ .probe = arm_smmu_device_dt_probe,
+ .remove = arm_smmu_device_remove,
+};
+
+static int __init arm_smmu_init(void)
+{
+ struct device_node *np;
+ int ret;
+
+ /*
+ * Play nice with systems that don't have an ARM SMMU by checking that
+ * an ARM SMMU exists in the system before proceeding with the driver
+ * and IOMMU bus operation registration.
+ */
+ np = of_find_matching_node(NULL, arm_smmu_of_match);
+ if (!np)
+ return 0;
+
+ of_node_put(np);
+
+ ret = platform_driver_register(&arm_smmu_driver);
+ if (ret)
+ return ret;
+
+ /* Oh, for a proper bus abstraction */
+ if (!iommu_present(&platform_bus_type))
+ bus_set_iommu(&platform_bus_type, &arm_smmu_ops);
+
+#ifdef CONFIG_ARM_AMBA
+ if (!iommu_present(&amba_bustype))
+ bus_set_iommu(&amba_bustype, &arm_smmu_ops);
+#endif
+
+#ifdef CONFIG_PCI
+ if (!iommu_present(&pci_bus_type))
+ bus_set_iommu(&pci_bus_type, &arm_smmu_ops);
+#endif
+
+ return 0;
+}
+
+static void __exit arm_smmu_exit(void)
+{
+ return platform_driver_unregister(&arm_smmu_driver);
+}
+
+subsys_initcall(arm_smmu_init);
+module_exit(arm_smmu_exit);
+
+MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
+MODULE_AUTHOR("Will Deacon <will.deacon@arm.com>");
+MODULE_LICENSE("GPL v2");
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (4 preceding siblings ...)
2015-02-25 18:52 ` [PATCH v4 5/8] xen/iommu: arm: Import the SMMU driver from Linux Julien Grall
@ 2015-02-25 18:52 ` Julien Grall
2015-03-02 7:03 ` Manish Jaggi
2015-03-02 13:17 ` Ian Campbell
2015-02-25 18:53 ` [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk Julien Grall
` (2 subsequent siblings)
8 siblings, 2 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:52 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, manish.jaggi, Julien Grall, tim, ian.campbell
The main goal is to modify as little the Linux code to be able to port
easily new feature added in Linux repo for the driver.
To achieve that we:
- Add helpers to Linux function not implemented on Xen
- Add callbacks used by Xen to do our own stuff and call Linux ones
- Only modify when required the code which comes from Linux. If so a
comment has been added with /* Xen: ... */ explaining why it's
necessary.
The support for PCI has been commented because it's not yet supported by
Xen ARM and therefore won't compile.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v4:
- Re-shuffle the code to rationalize the number of lines changed
in the Linux SMMU code
- Use #if 0 /* Xen: ... */ rather than using 2 lines
- Add a bunch of definitions to avoid some #if 0
- Remove the "hack" for the Midway SMMU
Changes in v2:
- Add the ACCESS_ONCE definition in the drivers. The patch to
introduce the one in common code has been dropped.
- The include xen/device.h has been dropped in favor of
asm/device.h
---
xen/drivers/passthrough/arm/Makefile | 1 +
xen/drivers/passthrough/arm/smmu.c | 661 ++++++++++++++++++++++++++++++++---
2 files changed, 617 insertions(+), 45 deletions(-)
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index 0484b79..f4cd26e 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -1 +1,2 @@
obj-y += iommu.o
+obj-y += smmu.o
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 6cd47b7..d01a26a 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -18,6 +18,13 @@
*
* Author: Will Deacon <will.deacon@arm.com>
*
+ * Based on Linux drivers/iommu/arm-smmu.c
+ * => commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3
+ *
+ * Xen modification:
+ * Julien Grall <julien.grall@linaro.org>
+ * Copyright (C) 2014 Linaro Limited.
+ *
* This driver currently supports:
* - SMMUv1 and v2 implementations
* - Stream-matching and stream-indexing
@@ -28,25 +35,287 @@
* - Context fault reporting
*/
-#define pr_fmt(fmt) "arm-smmu: " fmt
-#include <linux/delay.h>
-#include <linux/dma-mapping.h>
-#include <linux/err.h>
-#include <linux/interrupt.h>
-#include <linux/io.h>
-#include <linux/iommu.h>
-#include <linux/mm.h>
-#include <linux/module.h>
-#include <linux/of.h>
-#include <linux/pci.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-#include <linux/spinlock.h>
+#include <xen/config.h>
+#include <xen/delay.h>
+#include <xen/errno.h>
+#include <xen/err.h>
+#include <xen/irq.h>
+#include <xen/lib.h>
+#include <xen/list.h>
+#include <xen/mm.h>
+#include <xen/vmap.h>
+#include <xen/rbtree.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <asm/atomic.h>
+#include <asm/device.h>
+#include <asm/io.h>
+#include <asm/platform.h>
+
+/* Xen: The below defines are redefined within the file. Undef it */
+#undef SCTLR_AFE
+#undef SCTLR_TRE
+#undef SCTLR_M
+#undef TTBCR_EAE
+
+/* Alias to Xen device tree helpers */
+#define device_node dt_device_node
+#define of_phandle_args dt_phandle_args
+#define of_device_id dt_device_match
+#define of_match_node dt_match_node
+#define of_property_read_u32(np, pname, out) (!dt_property_read_u32(np, pname, out))
+#define of_property_read_bool dt_property_read_bool
+#define of_parse_phandle_with_args dt_parse_phandle_with_args
+
+/* Xen: Helpers to get device MMIO and IRQs */
+struct resource
+{
+ u64 addr;
+ u64 size;
+ unsigned int type;
+};
+
+#define resource_size(res) (res)->size;
+
+#define platform_device dt_device_node
+
+#define IORESOURCE_MEM 0
+#define IORESOURCE_IRQ 1
+
+static struct resource *platform_get_resource(struct platform_device *pdev,
+ unsigned int type,
+ unsigned int num)
+{
+ /*
+ * The resource is only used between 2 calls of platform_get_resource.
+ * It's quite ugly but it's avoid to add too much code in the part
+ * imported from Linux
+ */
+ static struct resource res;
+ int ret = 0;
+
+ res.type = type;
+
+ switch (type) {
+ case IORESOURCE_MEM:
+ ret = dt_device_get_address(pdev, num, &res.addr, &res.size);
+
+ return ((ret) ? NULL : &res);
+
+ case IORESOURCE_IRQ:
+ ret = platform_get_irq(pdev, num);
+ if (ret < 0)
+ return NULL;
+
+ res.addr = ret;
+ res.size = 1;
+
+ return &res;
+
+ default:
+ return NULL;
+ }
+}
+
+/* Xen: Helpers for IRQ functions */
+#define request_irq(irq, func, flags, name, dev) request_irq(irq, flags, func, name, dev)
+#define free_irq release_irq
+
+enum irqreturn {
+ IRQ_NONE = (0 << 0),
+ IRQ_HANDLED = (1 << 0),
+};
+
+typedef enum irqreturn irqreturn_t;
+
+/* Device logger functions
+ * TODO: Handle PCI
+ */
+#define dev_print(dev, lvl, fmt, ...) \
+ printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__)
+
+#define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__)
+#define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__)
+#define dev_warn(dev, fmt, ...) dev_print(dev, XENLOG_WARNING, fmt, ## __VA_ARGS__)
+#define dev_err(dev, fmt, ...) dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
+
+#define dev_err_ratelimited(dev, fmt, ...) \
+ dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
+
+#define dev_name(dev) dt_node_full_name(dev_to_dt(dev))
+
+/* Alias to Xen allocation helpers */
+#define kfree xfree
+#define kmalloc(size, flags) _xmalloc(size, sizeof(void *))
+#define kzalloc(size, flags) _xzalloc(size, sizeof(void *))
+#define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void *))
+#define kmalloc_array(size, n, flags) _xmalloc_array(size, sizeof(void *), n)
+
+static void __iomem *devm_ioremap_resource(struct device *dev,
+ struct resource *res)
+{
+ void __iomem *ptr;
+
+ if (!res || res->type != IORESOURCE_MEM) {
+ dev_err(dev, "Invalid resource\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ ptr = ioremap_nocache(res->addr, res->size);
+ if (!ptr) {
+ dev_err(dev,
+ "ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n",
+ res->addr, res->size);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ return ptr;
+}
+
+/* Xen doesn't handle IOMMU fault */
+#define report_iommu_fault(...) 1
+
+#define IOMMU_FAULT_READ 0
+#define IOMMU_FAULT_WRITE 1
+
+/*
+ * Xen: PCI functions
+ * TODO: It should be implemented when PCI will be supported
+ */
+#define to_pci_dev(dev) (NULL)
+static inline int pci_for_each_dma_alias(struct pci_dev *pdev,
+ int (*fn) (struct pci_dev *pdev,
+ u16 alias, void *data),
+ void *data)
+{
+ BUG();
+ return 0;
+}
+
+/* Xen: misc */
+#define PHYS_MASK_SHIFT PADDR_BITS
+typedef paddr_t phys_addr_t;
+
+#ifdef CONFIG_ARM_64
+# define CONFIG_64BIT
+#endif
+
+#define VA_BITS 0 /* Only used for configuring stage-1 input size */
+
+/* The macro ACCESS_ONCE start to be replaced in Linux in favor of
+ * {READ, WRITE}_ONCE. Rather than introducing in the common code, keep a
+ * version here. We will have to drop it when the SMMU code in Linux will
+ * switch to {READ, WRITE}_ONCE.
+ */
+#define __ACCESS_ONCE(x) ({ \
+ __maybe_unused typeof(x) __var = 0; \
+ (volatile typeof(x) *)&(x); })
+#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
+
+#define MODULE_DEVICE_TABLE(type, name)
+#define module_param_named(name, value, type, perm)
+#define MODULE_PARM_DESC(_parm, desc)
+
+/* Xen: Dummy iommu_domain */
+struct iommu_domain
+{
+ /* Runtime SMMU configuration for this iommu_domain */
+ struct arm_smmu_domain *priv;
+
+ /* Used to link iommu_domain contexts for a same domain.
+ * There is at least one per-SMMU to used by the domain.
+ * */
+ struct list_head list;
+};
+
+/* Xen: Describes informations required for a Xen domain */
+struct arm_smmu_xen_domain {
+ spinlock_t lock;
+ /* List of context (i.e iommu_domain) associated to this domain */
+ struct list_head contexts;
+};
+
+/*
+ * Xen: Information about each device stored in dev->archdata.iommu
+ *
+ * Initially dev->archdata.iommu only stores the iommu_domain (runtime
+ * configuration of the SMMU) but, on Xen, we also have to store the
+ * iommu_group (list of streamIDs associated to the device).
+ *
+ * This is because Linux has a field iommu_group in the struct device. On Xen,
+ * that would require to move so hackery (dummy iommu_group) in a more generic
+ * place.
+ * */
+struct arm_smmu_xen_device {
+ struct iommu_domain *domain;
+ struct iommu_group *group;
+};
+
+#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu)
+#define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
+#define dev_iommu_group(dev) (dev_archdata(dev)->group)
+
+/* Xen: Dummy iommu_group */
+struct iommu_group
+{
+ /* Streamids of the device */
+ struct arm_smmu_master_cfg *cfg;
+
+ atomic_t ref;
+};
+
+static struct iommu_group *iommu_group_alloc(void)
+{
+ struct iommu_group *group = xzalloc(struct iommu_group);
+
+ if (!group)
+ return ERR_PTR(-ENOMEM);
-#include <linux/amba/bus.h>
+ atomic_set(&group->ref, 1);
-#include <asm/pgalloc.h>
+ return group;
+}
+
+static void iommu_group_put(struct iommu_group *group)
+{
+ if (atomic_dec_and_test(&group->ref))
+ xfree(group);
+}
+
+static void iommu_group_set_iommudata(struct iommu_group *group,
+ struct arm_smmu_master_cfg *cfg,
+ void (*releasefn)(void *))
+{
+ /* TODO: Store the releasefn for the PCI */
+ ASSERT(releasefn == NULL);
+
+ group->cfg = cfg;
+}
+
+static int iommu_group_add_device(struct iommu_group *group,
+ struct device *dev)
+{
+ dev_iommu_group(dev) = group;
+
+ atomic_inc(&group->ref);
+
+ return 0;
+}
+
+static struct iommu_group *iommu_group_get(struct device *dev)
+{
+ struct iommu_group *group = dev_iommu_group(dev);
+
+ if (group)
+ atomic_inc(&group->ref);
+
+ return group;
+}
+
+#define iommu_group_get_iommudata(group) (group)->cfg
+
+/***** Start of Linux SMMU code *****/
/* Maximum number of stream IDs assigned to a single device */
#define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
@@ -397,7 +666,9 @@ struct arm_smmu_cfg {
u8 cbndx;
u8 irptndx;
u32 cbar;
- pgd_t *pgd;
+
+ /* Xen: Domain associated to this configuration */
+ struct domain *domain;
};
#define INVALID_IRPTNDX 0xff
@@ -446,6 +717,7 @@ static void parse_driver_options(struct arm_smmu_device *smmu)
static struct device_node *dev_get_dev_node(struct device *dev)
{
+#if 0 /* Xen: TODO: Add support for PCI */
if (dev_is_pci(dev)) {
struct pci_bus *bus = to_pci_dev(dev)->bus;
@@ -453,6 +725,7 @@ static struct device_node *dev_get_dev_node(struct device *dev)
bus = bus->parent;
return bus->bridge->parent->of_node;
}
+#endif
return dev->of_node;
}
@@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
master->of_node = masterspec->np;
master->cfg.num_streamids = masterspec->args_count;
+ /* Xen: Let Xen knows that the device is protected by an SMMU */
+ dt_device_set_protected(masterspec->np);
+
for (i = 0; i < master->cfg.num_streamids; ++i) {
u16 streamid = masterspec->args[i];
@@ -712,6 +988,24 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
return IRQ_HANDLED;
}
+/* Xen: Interrupt handlers wrapper */
+static void arm_smmu_context_fault_xen(int irq, void *dev,
+ struct cpu_user_regs *regs)
+{
+ arm_smmu_context_fault(irq, dev);
+}
+
+#define arm_smmu_context_fault arm_smmu_context_fault_xen
+
+static void arm_smmu_global_fault_xen(int irq, void *dev,
+ struct cpu_user_regs *regs)
+{
+ arm_smmu_global_fault(irq, dev);
+}
+
+#define arm_smmu_global_fault arm_smmu_global_fault_xen
+
+#if 0 /* Xen: Page tables are shared with the processor */
static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
size_t size)
{
@@ -733,6 +1027,7 @@ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
DMA_TO_DEVICE);
}
}
+#endif
static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
{
@@ -741,6 +1036,7 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
struct arm_smmu_device *smmu = smmu_domain->smmu;
void __iomem *cb_base, *gr0_base, *gr1_base;
+ paddr_t p2maddr;
gr0_base = ARM_SMMU_GR0(smmu);
gr1_base = ARM_SMMU_GR1(smmu);
@@ -824,11 +1120,16 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
}
/* TTBR0 */
- arm_smmu_flush_pgtable(smmu, cfg->pgd,
- PTRS_PER_PGD * sizeof(pgd_t));
- reg = __pa(cfg->pgd);
+ /* Xen: The page table is shared with the P2M code */
+ ASSERT(smmu_domain->cfg.domain != NULL);
+ p2maddr = page_to_maddr(smmu_domain->cfg.domain->arch.p2m.root);
+
+ dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n",
+ smmu_domain->cfg.domain->domain_id, p2maddr);
+
+ reg = (p2maddr & ((1ULL << 32) - 1));
writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
- reg = (phys_addr_t)__pa(cfg->pgd) >> 32;
+ reg = (p2maddr >> 32);
if (stage1)
reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
@@ -873,6 +1174,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
reg = 0;
}
+ /* Xen: The attributes to walk the page table should be the same as
+ * VTCR_EL2. Currently doesn't differ from Linux ones.
+ */
reg |= TTBCR_EAE |
(TTBCR_SH_IS << TTBCR_SH0_SHIFT) |
(TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) |
@@ -1015,7 +1319,6 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain)
static int arm_smmu_domain_init(struct iommu_domain *domain)
{
struct arm_smmu_domain *smmu_domain;
- pgd_t *pgd;
/*
* Allocate the domain and initialise some of its data structures.
@@ -1026,20 +1329,12 @@ static int arm_smmu_domain_init(struct iommu_domain *domain)
if (!smmu_domain)
return -ENOMEM;
- pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL);
- if (!pgd)
- goto out_free_domain;
- smmu_domain->cfg.pgd = pgd;
-
spin_lock_init(&smmu_domain->lock);
domain->priv = smmu_domain;
return 0;
-
-out_free_domain:
- kfree(smmu_domain);
- return -ENOMEM;
}
+#if 0 /* Xen: Page tables are shared with the processor */
static void arm_smmu_free_ptes(pmd_t *pmd)
{
pgtable_t table = pmd_pgtable(*pmd);
@@ -1102,6 +1397,7 @@ static void arm_smmu_free_pgtables(struct arm_smmu_domain *smmu_domain)
kfree(pgd_base);
}
+#endif
static void arm_smmu_domain_destroy(struct iommu_domain *domain)
{
@@ -1112,7 +1408,6 @@ static void arm_smmu_domain_destroy(struct iommu_domain *domain)
* already been detached.
*/
arm_smmu_destroy_domain_context(domain);
- arm_smmu_free_pgtables(smmu_domain);
kfree(smmu_domain);
}
@@ -1229,11 +1524,12 @@ static void arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain,
/*
* We *must* clear the S2CR first, because freeing the SMR means
* that it can be re-allocated immediately.
+ * Xen: Unlike Linux, any access to non-configured stream will fault.
*/
for (i = 0; i < cfg->num_streamids; ++i) {
u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i];
- writel_relaxed(S2CR_TYPE_BYPASS,
+ writel_relaxed(S2CR_TYPE_FAULT,
gr0_base + ARM_SMMU_GR0_S2CR(idx));
}
@@ -1253,7 +1549,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
return -ENXIO;
}
- if (dev->archdata.iommu) {
+ if (dev_iommu_domain(dev)) {
dev_err(dev, "already attached to IOMMU domain\n");
return -EEXIST;
}
@@ -1285,8 +1581,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
return -ENODEV;
ret = arm_smmu_domain_add_master(smmu_domain, cfg);
+
if (!ret)
- dev->archdata.iommu = domain;
+ dev_iommu_domain(dev) = domain;
return ret;
}
@@ -1299,10 +1596,14 @@ static void arm_smmu_detach_dev(struct iommu_domain *domain, struct device *dev)
if (!cfg)
return;
- dev->archdata.iommu = NULL;
+ dev_iommu_domain(dev) = NULL;
arm_smmu_domain_remove_master(smmu_domain, cfg);
}
+#if 0 /*
+ * Xen: The page table is shared with the processor, therefore
+ * helpers to implement separate is not necessary.
+ */
static bool arm_smmu_pte_is_contiguous_range(unsigned long addr,
unsigned long end)
{
@@ -1591,7 +1892,9 @@ static phys_addr_t arm_smmu_iova_to_phys(struct iommu_domain *domain,
return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK);
}
+#endif
+#if 0 /* Xen: arm_smmu_capable is not used at the moment */
static bool arm_smmu_capable(enum iommu_cap cap)
{
switch (cap) {
@@ -1609,6 +1912,7 @@ static bool arm_smmu_capable(enum iommu_cap cap)
return false;
}
}
+#endif
static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data)
{
@@ -1676,6 +1980,7 @@ out_put_group:
return ret;
}
+#if 0 /* Xen: We don't support remove device for now. Will be useful for PCI */
static void arm_smmu_remove_device(struct device *dev)
{
iommu_group_remove_device(dev);
@@ -1733,6 +2038,7 @@ static const struct iommu_ops arm_smmu_ops = {
ARM_SMMU_PTE_CONT_SIZE |
PAGE_SIZE),
};
+#endif
static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
{
@@ -1748,7 +2054,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
/* Mark all SMRn as invalid and all S2CRn as bypass */
for (i = 0; i < smmu->num_mapping_groups; ++i) {
writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i));
- writel_relaxed(S2CR_TYPE_BYPASS,
+ /*
+ * Xen: Unlike Linux, any access to a non-configure stream
+ * will fault by default.
+ */
+ writel_relaxed(S2CR_TYPE_FAULT,
gr0_base + ARM_SMMU_GR0_S2CR(i));
}
@@ -1774,6 +2084,8 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
/* Enable client access, but bypass when no mapping is found */
reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
+ /* Xen: Unlike Linux, generate a fault when no mapping is found */
+ reg |= sCR0_USFCFG;
/* Disable forced broadcasting */
reg &= ~sCR0_FB;
@@ -1882,7 +2194,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
}
dev_notice(smmu->dev,
- "\tstream matching with %u register groups, mask 0x%x",
+ "\tstream matching with %u register groups, mask 0x%x\n",
smmu->num_mapping_groups, mask);
} else {
smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) &
@@ -1917,12 +2229,16 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK);
smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
+ /* Xen: Stage-2 input size is not restricted */
+ smmu->s2_input_size = size;
+#if 0
/* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */
#ifdef CONFIG_64BIT
smmu->s2_input_size = min_t(unsigned long, VA_BITS, size);
#else
smmu->s2_input_size = min(32UL, size);
#endif
+#endif
/* The stage-2 output mask is also applied for bypass */
size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK);
@@ -1969,6 +2285,10 @@ static const struct of_device_id arm_smmu_of_match[] = {
};
MODULE_DEVICE_TABLE(of, arm_smmu_of_match);
+/*
+ * Xen: We don't have refcount for allocated memory so manually free memory
+ * when an error occured.
+ */
static int arm_smmu_device_dt_probe(struct platform_device *pdev)
{
const struct of_device_id *of_id;
@@ -1991,14 +2311,17 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
smmu->base = devm_ioremap_resource(dev, res);
- if (IS_ERR(smmu->base))
- return PTR_ERR(smmu->base);
+ if (IS_ERR(smmu->base)) {
+ err = PTR_ERR(smmu->base);
+ goto out_free;
+ }
smmu->size = resource_size(res);
if (of_property_read_u32(dev->of_node, "#global-interrupts",
&smmu->num_global_irqs)) {
dev_err(dev, "missing #global-interrupts property\n");
- return -ENODEV;
+ err = -ENODEV;
+ goto out_free;
}
num_irqs = 0;
@@ -2011,14 +2334,16 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
if (!smmu->num_context_irqs) {
dev_err(dev, "found %d interrupts but expected at least %d\n",
num_irqs, smmu->num_global_irqs + 1);
- return -ENODEV;
+ err = -ENODEV;
+ goto out_free;
}
smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
GFP_KERNEL);
if (!smmu->irqs) {
dev_err(dev, "failed to allocate %d irqs\n", num_irqs);
- return -ENOMEM;
+ err = -ENOMEM;
+ goto out_free;
}
for (i = 0; i < num_irqs; ++i) {
@@ -2026,7 +2351,8 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
if (irq < 0) {
dev_err(dev, "failed to get irq index %d\n", i);
- return -ENODEV;
+ err = -ENODEV;
+ goto out_free;
}
smmu->irqs[i] = irq;
}
@@ -2091,12 +2417,19 @@ out_put_masters:
for (node = rb_first(&smmu->masters); node; node = rb_next(node)) {
struct arm_smmu_master *master
= container_of(node, struct arm_smmu_master, node);
- of_node_put(master->of_node);
+ kfree(master);
}
+out_free:
+ kfree(smmu->irqs);
+ if (!IS_ERR(smmu->base))
+ iounmap(smmu->base);
+ kfree(smmu);
+
return err;
}
+#if 0 /* Xen: We never remove SMMU */
static int arm_smmu_device_remove(struct platform_device *pdev)
{
int i;
@@ -2191,3 +2524,241 @@ module_exit(arm_smmu_exit);
MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
MODULE_AUTHOR("Will Deacon <will.deacon@arm.com>");
MODULE_LICENSE("GPL v2");
+#endif
+
+/***** Start of Xen specific code *****/
+
+/* Xen only supports stage-2 translation, so force the value to 2. */
+static int force_stage = 2;
+
+static void arm_smmu_iotlb_flush_all(struct domain *d)
+{
+ struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
+ struct iommu_domain *cfg;
+
+ spin_lock(&smmu_domain->lock);
+ list_for_each_entry(cfg, &smmu_domain->contexts, list) {
+ /*
+ * Only invalidate the context when SMMU is present.
+ * This is because the context initialization is delayed
+ * until a master has been added.
+ */
+ if (unlikely(!ACCESS_ONCE(cfg->priv->smmu)))
+ continue;
+ arm_smmu_tlb_inv_context(cfg->priv);
+ }
+ spin_unlock(&smmu_domain->lock);
+}
+
+static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn,
+ unsigned int page_count)
+{
+ /* ARM SMMU v1 doesn't have flush by VMA and VMID */
+ arm_smmu_iotlb_flush_all(d);
+}
+
+static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
+ struct device *dev)
+{
+ struct iommu_domain *domain;
+ struct arm_smmu_xen_domain *xen_domain;
+ int ret;
+
+ xen_domain = domain_hvm_iommu(d)->arch.priv;
+
+ if (!dev->archdata.iommu) {
+ dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device);
+ if (!dev->archdata.iommu)
+ return -ENOMEM;
+ }
+
+ if (!dev_iommu_group(dev)) {
+ ret = arm_smmu_add_device(dev);
+ if (ret)
+ return ret;
+ }
+
+ /*
+ * TODO: Share the context bank (i.e iommu_domain) when the device is
+ * under the same SMMU as another device assigned to this domain.
+ * Would it useful for PCI
+ */
+ domain = xzalloc(struct iommu_domain);
+ if (!domain)
+ return -ENOMEM;
+
+ ret = arm_smmu_domain_init(domain);
+ if (ret)
+ goto err_dom_init;
+
+ domain->priv->cfg.domain = d;
+
+ ret = arm_smmu_attach_dev(domain, dev);
+ if (ret)
+ goto err_attach_dev;
+
+ spin_lock(&xen_domain->lock);
+ /* Chain the new context to the domain */
+ list_add(&domain->list, &xen_domain->contexts);
+ spin_unlock(&xen_domain->lock);
+
+ return 0;
+
+err_attach_dev:
+ arm_smmu_domain_destroy(domain);
+err_dom_init:
+ xfree(domain);
+
+ return ret;
+}
+
+static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
+{
+ struct iommu_domain *domain = dev_iommu_domain(dev);
+ struct arm_smmu_xen_domain *xen_domain;
+
+ xen_domain = domain_hvm_iommu(d)->arch.priv;
+
+ if (!domain || domain->priv->cfg.domain != d) {
+ dev_err(dev, " not attached to domain %d\n", d->domain_id);
+ return -ESRCH;
+ }
+
+ arm_smmu_detach_dev(domain, dev);
+
+ spin_lock(&xen_domain->lock);
+ list_del(&domain->list);
+ spin_unlock(&xen_domain->lock);
+
+ arm_smmu_domain_destroy(domain);
+ xfree(domain);
+
+ return 0;
+}
+
+static int arm_smmu_reassign_dev(struct domain *s, struct domain *t,
+ u8 devfn, struct device *dev)
+{
+ int ret = 0;
+
+ /* Don't allow remapping on other domain than hwdom */
+ if (t != hardware_domain)
+ return -EPERM;
+
+ if (t == s)
+ return 0;
+
+ ret = arm_smmu_deassign_dev(s, dev);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int arm_smmu_iommu_domain_init(struct domain *d)
+{
+ struct arm_smmu_xen_domain *xen_domain;
+
+ xen_domain = xzalloc(struct arm_smmu_xen_domain);
+ if ( !xen_domain )
+ return -ENOMEM;
+
+ spin_lock_init(&xen_domain->lock);
+ INIT_LIST_HEAD(&xen_domain->contexts);
+
+ domain_hvm_iommu(d)->arch.priv = xen_domain;
+
+ return 0;
+}
+
+static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d)
+{
+}
+
+static void arm_smmu_iommu_domain_teardown(struct domain *d)
+{
+ struct arm_smmu_xen_domain *xen_domain = domain_hvm_iommu(d)->arch.priv;
+
+ ASSERT(list_empty(&xen_domain->contexts));
+ xfree(xen_domain);
+}
+
+static int arm_smmu_map_page(struct domain *d, unsigned long gfn,
+ unsigned long mfn, unsigned int flags)
+{
+ p2m_type_t t;
+
+ /*
+ * Grant mappings can be used for DMA requests. The dev_bus_addr
+ * returned by the hypercall is the MFN (not the IPA). For device
+ * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain
+ * p2m to allow DMA request to work.
+ * This is only valid when the domain is directed mapped. Hence this
+ * function should only be used by gnttab code with gfn == mfn.
+ */
+ BUG_ON(!is_domain_direct_mapped(d));
+ BUG_ON(mfn != gfn);
+
+ /* We only support readable and writable flags */
+ if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
+ return -EINVAL;
+
+ t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro;
+
+ /*
+ * The function guest_physmap_add_entry replaces the current mapping
+ * if there is already one...
+ */
+ return guest_physmap_add_entry(d, gfn, mfn, 0, t);
+}
+
+static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn)
+{
+ /*
+ * This function should only be used by gnttab code when the domain
+ * is direct mapped
+ */
+ if ( !is_domain_direct_mapped(d) )
+ return -EINVAL;
+
+ guest_physmap_remove_page(d, gfn, gfn, 0);
+
+ return 0;
+}
+
+static const struct iommu_ops arm_smmu_iommu_ops = {
+ .init = arm_smmu_iommu_domain_init,
+ .hwdom_init = arm_smmu_iommu_hwdom_init,
+ .teardown = arm_smmu_iommu_domain_teardown,
+ .iotlb_flush = arm_smmu_iotlb_flush,
+ .iotlb_flush_all = arm_smmu_iotlb_flush_all,
+ .assign_device = arm_smmu_assign_dev,
+ .reassign_device = arm_smmu_reassign_dev,
+ .map_page = arm_smmu_map_page,
+ .unmap_page = arm_smmu_unmap_page,
+};
+
+static __init int arm_smmu_dt_init(struct dt_device_node *dev,
+ const void *data)
+{
+ int rc;
+
+ /*
+ * Even if the device can't be initialized, we don't want to
+ * give the SMMU device to dom0.
+ */
+ dt_device_set_used_by(dev, DOMID_XEN);
+
+ rc = arm_smmu_device_dt_probe(dev);
+ if (rc)
+ return rc;
+
+ iommu_set_ops(&arm_smmu_iommu_ops);
+
+ return 0;
+}
+
+DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU)
+ .dt_match = arm_smmu_of_match,
+ .init = arm_smmu_dt_init,
+DT_DEVICE_END
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (5 preceding siblings ...)
2015-02-25 18:52 ` [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Julien Grall
@ 2015-02-25 18:53 ` Julien Grall
2015-03-02 13:10 ` Ian Campbell
2015-02-25 18:53 ` [PATCH v4 8/8] DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU Julien Grall
2015-03-02 14:47 ` [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Ian Campbell
8 siblings, 1 reply; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:53 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, manish.jaggi, Julien Grall, tim, ian.campbell
When SMMU doesn't support coherent table walk, Xen may need to clean
updated PT (see commit 4c5f4cb "xen/arm: p2m: Clean cache PT when the
IOMMU doesn't support coherent walk").
If one SMMU of the platform doesn't support coherent table walk, the
feature is disabled for the whole platform. This is because device is
assigned to a domain after the page table are populated.
This could impact performance on domain which doesn't use device
passthrough. But, as the spec strongly recommends the support of this
feature for maintstream platform, I expect server will always have SMMUs
supporting coherent table walk. If not, we may need to enable this feature
per-domain.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
I've just noticed that the support on the previous driver (i.e in
Xen 4.5) may incorrectly expose this feature when all the SMMUs is
not supporting coherent table walk.
I'm not sure, if I should send a patch for it.
Also I didn't squash this patch into "xen/iommu: smmu: Add Xen specific
code to be able to use the driver" to help for review and to catch
possible error in this patch.
Changes in v4:
- Browse the list to retrieve the last SMMU added.
- Take the spinlock
Changes in v3:
- Patch added
---
xen/drivers/passthrough/arm/smmu.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index d01a26a..b478463 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -2531,6 +2531,13 @@ MODULE_LICENSE("GPL v2");
/* Xen only supports stage-2 translation, so force the value to 2. */
static int force_stage = 2;
+/*
+ * Platform features. It indicates the list of features supported by all the
+ * SMMUs.
+ * Actually we only care about coherent table walk.
+ */
+static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK;
+
static void arm_smmu_iotlb_flush_all(struct domain *d)
{
struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
@@ -2668,6 +2675,13 @@ static int arm_smmu_iommu_domain_init(struct domain *d)
domain_hvm_iommu(d)->arch.priv = xen_domain;
+ /*
+ * The feature coherent walk can be enabled only when all SMMUs
+ * support it.
+ */
+ if (platform_features & ARM_SMMU_FEAT_COHERENT_WALK)
+ iommu_set_feature(d, IOMMU_FEAT_COHERENT_WALK);
+
return 0;
}
@@ -2742,6 +2756,7 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
const void *data)
{
int rc;
+ struct arm_smmu_device *smmu;
/*
* Even if the device can't be initialized, we don't want to
@@ -2755,6 +2770,20 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
iommu_set_ops(&arm_smmu_iommu_ops);
+ /* Find the last SMMU added and retrieve its features. */
+ spin_lock(&arm_smmu_devices_lock);
+ list_for_each_entry(smmu, &arm_smmu_devices, list) {
+ if (smmu->dev == &dev->dev)
+ goto found;
+ }
+
+ BUG();
+
+found:
+ platform_features &= smmu->features;
+
+ spin_unlock(&arm_smmu_devices_lock);
+
return 0;
}
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v4 8/8] DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (6 preceding siblings ...)
2015-02-25 18:53 ` [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk Julien Grall
@ 2015-02-25 18:53 ` Julien Grall
2015-03-02 14:47 ` [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Ian Campbell
8 siblings, 0 replies; 19+ messages in thread
From: Julien Grall @ 2015-02-25 18:53 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, manish.jaggi, Julien Grall, tim, ian.campbell
Basic changes to support SMMUs which protect the network cards. The one
for the SATA is more complicate to support because it requires to
support stream ID matching.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
xen/drivers/passthrough/arm/smmu.c | 32 +++++++++++++++++++++++++++-----
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index b478463..0a1ba4b 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -915,7 +915,7 @@ static irqreturn_t arm_smmu_context_fault(int irq, void *dev)
{
int flags, ret;
u32 fsr, far, fsynr, resume;
- unsigned long iova;
+ paddr_t iova;
struct iommu_domain *domain = dev;
struct arm_smmu_domain *smmu_domain = domain->priv;
struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
@@ -938,17 +938,16 @@ static irqreturn_t arm_smmu_context_fault(int irq, void *dev)
far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_LO);
iova = far;
-#ifdef CONFIG_64BIT
+ /* Xen: The fault address maybe higher than 32 bits on arm32 */
far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_HI);
- iova |= ((unsigned long)far << 32);
-#endif
+ iova |= ((paddr_t)far << 32);
if (!report_iommu_fault(domain, smmu->dev, iova, flags)) {
ret = IRQ_HANDLED;
resume = RESUME_RETRY;
} else {
dev_err_ratelimited(smmu->dev,
- "Unhandled context fault: iova=0x%08lx, fsynr=0x%x, cb=%d\n",
+ "Unhandled context fault: iova=0x%"PRIpaddr", fsynr=0x%x, cb=%d\n",
iova, fsynr, cfg->cbndx);
ret = IRQ_NONE;
resume = RESUME_TERMINATE;
@@ -1171,7 +1170,16 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
reg |= (64 - smmu->s1_input_size) << TTBCR_T0SZ_SHIFT;
}
} else {
+#if CONFIG_ARM_32
+ /* Xen: Midway is using 40-bit address space. Though it may
+ * not work on other ARM32 platform with SMMU-v1.
+ * TODO: A quirk may be necessary if we have to support
+ * other ARM32 platform with SMMU-v1.
+ */
+ reg = 0x18 << TTBCR_T0SZ_SHIFT;
+#else
reg = 0;
+#endif
}
/* Xen: The attributes to walk the page table should be the same as
@@ -2240,6 +2248,20 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
#endif
#endif
+ /*
+ * Xen: SMMU v1: Only 40 bits input address size is supported for
+ * arm32. See arm_smmu_init_context_bank
+ */
+#ifdef CONFIG_ARM_32
+ if ( smmu->version == ARM_SMMU_V1 && smmu->s2_input_size != 40 )
+ {
+ dev_err(smmu->dev,
+ "Stage-2 Input size %ld not supported for SMMUv1\n",
+ smmu->s2_input_size);
+ return -ENODEV;;
+ }
+#endif
+
/* The stage-2 output mask is also applied for bypass */
size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK);
smmu->s2_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-02-25 18:52 ` [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Julien Grall
@ 2015-03-02 7:03 ` Manish Jaggi
2015-03-04 2:08 ` Manish Jaggi
2015-03-02 13:17 ` Ian Campbell
1 sibling, 1 reply; 19+ messages in thread
From: Manish Jaggi @ 2015-03-02 7:03 UTC (permalink / raw)
To: Julien Grall, xen-devel
Cc: stefano.stabellini, manish.jaggi, tim, ian.campbell
On Thursday 26 February 2015 12:22 AM, Julien Grall wrote:
> The main goal is to modify as little the Linux code to be able to port
> easily new feature added in Linux repo for the driver.
>
> To achieve that we:
> - Add helpers to Linux function not implemented on Xen
> - Add callbacks used by Xen to do our own stuff and call Linux ones
> - Only modify when required the code which comes from Linux. If so a
> comment has been added with /* Xen: ... */ explaining why it's
> necessary.
>
> The support for PCI has been commented because it's not yet supported by
> Xen ARM and therefore won't compile.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
> Changes in v4:
> - Re-shuffle the code to rationalize the number of lines changed
> in the Linux SMMU code
> - Use #if 0 /* Xen: ... */ rather than using 2 lines
> - Add a bunch of definitions to avoid some #if 0
> - Remove the "hack" for the Midway SMMU
>
> Changes in v2:
> - Add the ACCESS_ONCE definition in the drivers. The patch to
> introduce the one in common code has been dropped.
> - The include xen/device.h has been dropped in favor of
> asm/device.h
> ---
> xen/drivers/passthrough/arm/Makefile | 1 +
> xen/drivers/passthrough/arm/smmu.c | 661 ++++++++++++++++++++++++++++++++---
> 2 files changed, 617 insertions(+), 45 deletions(-)
Hi Julien,
Few comments (first level)
a) What is the difference between assign / attach / add. There are
different apis and I get lost reading the code between them.
Either provide a description or use a better nomenclature.
b) In which case a xen domain be having multiple contexts. Even though
it is technically possible but there is no defined method to create a
different context in non-pci or pci-passthrough.
Could you please add some detail.
-mj
>
> diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
> index 0484b79..f4cd26e 100644
> --- a/xen/drivers/passthrough/arm/Makefile
> +++ b/xen/drivers/passthrough/arm/Makefile
> @@ -1 +1,2 @@
> obj-y += iommu.o
> +obj-y += smmu.o
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index 6cd47b7..d01a26a 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -18,6 +18,13 @@
> *
> * Author: Will Deacon <will.deacon@arm.com>
> *
> + * Based on Linux drivers/iommu/arm-smmu.c
> + * => commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3
> + *
> + * Xen modification:
> + * Julien Grall <julien.grall@linaro.org>
> + * Copyright (C) 2014 Linaro Limited.
> + *
> * This driver currently supports:
> * - SMMUv1 and v2 implementations
> * - Stream-matching and stream-indexing
> @@ -28,25 +35,287 @@
> * - Context fault reporting
> */
>
> -#define pr_fmt(fmt) "arm-smmu: " fmt
>
> -#include <linux/delay.h>
> -#include <linux/dma-mapping.h>
> -#include <linux/err.h>
> -#include <linux/interrupt.h>
> -#include <linux/io.h>
> -#include <linux/iommu.h>
> -#include <linux/mm.h>
> -#include <linux/module.h>
> -#include <linux/of.h>
> -#include <linux/pci.h>
> -#include <linux/platform_device.h>
> -#include <linux/slab.h>
> -#include <linux/spinlock.h>
> +#include <xen/config.h>
> +#include <xen/delay.h>
> +#include <xen/errno.h>
> +#include <xen/err.h>
> +#include <xen/irq.h>
> +#include <xen/lib.h>
> +#include <xen/list.h>
> +#include <xen/mm.h>
> +#include <xen/vmap.h>
> +#include <xen/rbtree.h>
> +#include <xen/sched.h>
> +#include <xen/sizes.h>
> +#include <asm/atomic.h>
> +#include <asm/device.h>
> +#include <asm/io.h>
> +#include <asm/platform.h>
> +
> +/* Xen: The below defines are redefined within the file. Undef it */
> +#undef SCTLR_AFE
> +#undef SCTLR_TRE
> +#undef SCTLR_M
> +#undef TTBCR_EAE
> +
> +/* Alias to Xen device tree helpers */
> +#define device_node dt_device_node
> +#define of_phandle_args dt_phandle_args
> +#define of_device_id dt_device_match
> +#define of_match_node dt_match_node
> +#define of_property_read_u32(np, pname, out) (!dt_property_read_u32(np, pname, out))
> +#define of_property_read_bool dt_property_read_bool
> +#define of_parse_phandle_with_args dt_parse_phandle_with_args
> +
> +/* Xen: Helpers to get device MMIO and IRQs */
> +struct resource
> +{
> + u64 addr;
> + u64 size;
> + unsigned int type;
> +};
> +
> +#define resource_size(res) (res)->size;
> +
> +#define platform_device dt_device_node
> +
> +#define IORESOURCE_MEM 0
> +#define IORESOURCE_IRQ 1
> +
> +static struct resource *platform_get_resource(struct platform_device *pdev,
> + unsigned int type,
> + unsigned int num)
> +{
> + /*
> + * The resource is only used between 2 calls of platform_get_resource.
> + * It's quite ugly but it's avoid to add too much code in the part
> + * imported from Linux
> + */
> + static struct resource res;
> + int ret = 0;
> +
> + res.type = type;
> +
> + switch (type) {
> + case IORESOURCE_MEM:
> + ret = dt_device_get_address(pdev, num, &res.addr, &res.size);
> +
> + return ((ret) ? NULL : &res);
> +
> + case IORESOURCE_IRQ:
> + ret = platform_get_irq(pdev, num);
> + if (ret < 0)
> + return NULL;
> +
> + res.addr = ret;
> + res.size = 1;
> +
> + return &res;
> +
> + default:
> + return NULL;
> + }
> +}
> +
> +/* Xen: Helpers for IRQ functions */
> +#define request_irq(irq, func, flags, name, dev) request_irq(irq, flags, func, name, dev)
> +#define free_irq release_irq
> +
> +enum irqreturn {
> + IRQ_NONE = (0 << 0),
> + IRQ_HANDLED = (1 << 0),
> +};
> +
> +typedef enum irqreturn irqreturn_t;
> +
> +/* Device logger functions
> + * TODO: Handle PCI
> + */
> +#define dev_print(dev, lvl, fmt, ...) \
> + printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__)
> +
> +#define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__)
> +#define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__)
> +#define dev_warn(dev, fmt, ...) dev_print(dev, XENLOG_WARNING, fmt, ## __VA_ARGS__)
> +#define dev_err(dev, fmt, ...) dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
> +
> +#define dev_err_ratelimited(dev, fmt, ...) \
> + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
> +
> +#define dev_name(dev) dt_node_full_name(dev_to_dt(dev))
> +
> +/* Alias to Xen allocation helpers */
> +#define kfree xfree
> +#define kmalloc(size, flags) _xmalloc(size, sizeof(void *))
> +#define kzalloc(size, flags) _xzalloc(size, sizeof(void *))
> +#define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void *))
> +#define kmalloc_array(size, n, flags) _xmalloc_array(size, sizeof(void *), n)
> +
> +static void __iomem *devm_ioremap_resource(struct device *dev,
> + struct resource *res)
> +{
> + void __iomem *ptr;
> +
> + if (!res || res->type != IORESOURCE_MEM) {
> + dev_err(dev, "Invalid resource\n");
> + return ERR_PTR(-EINVAL);
> + }
> +
> + ptr = ioremap_nocache(res->addr, res->size);
> + if (!ptr) {
> + dev_err(dev,
> + "ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n",
> + res->addr, res->size);
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + return ptr;
> +}
> +
> +/* Xen doesn't handle IOMMU fault */
> +#define report_iommu_fault(...) 1
> +
> +#define IOMMU_FAULT_READ 0
> +#define IOMMU_FAULT_WRITE 1
> +
> +/*
> + * Xen: PCI functions
> + * TODO: It should be implemented when PCI will be supported
> + */
> +#define to_pci_dev(dev) (NULL)
> +static inline int pci_for_each_dma_alias(struct pci_dev *pdev,
> + int (*fn) (struct pci_dev *pdev,
> + u16 alias, void *data),
> + void *data)
> +{
> + BUG();
> + return 0;
> +}
> +
> +/* Xen: misc */
> +#define PHYS_MASK_SHIFT PADDR_BITS
> +typedef paddr_t phys_addr_t;
> +
> +#ifdef CONFIG_ARM_64
> +# define CONFIG_64BIT
> +#endif
> +
> +#define VA_BITS 0 /* Only used for configuring stage-1 input size */
> +
> +/* The macro ACCESS_ONCE start to be replaced in Linux in favor of
> + * {READ, WRITE}_ONCE. Rather than introducing in the common code, keep a
> + * version here. We will have to drop it when the SMMU code in Linux will
> + * switch to {READ, WRITE}_ONCE.
> + */
> +#define __ACCESS_ONCE(x) ({ \
> + __maybe_unused typeof(x) __var = 0; \
> + (volatile typeof(x) *)&(x); })
> +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
> +
> +#define MODULE_DEVICE_TABLE(type, name)
> +#define module_param_named(name, value, type, perm)
> +#define MODULE_PARM_DESC(_parm, desc)
> +
> +/* Xen: Dummy iommu_domain */
> +struct iommu_domain
> +{
> + /* Runtime SMMU configuration for this iommu_domain */
> + struct arm_smmu_domain *priv;
> +
> + /* Used to link iommu_domain contexts for a same domain.
> + * There is at least one per-SMMU to used by the domain.
> + * */
> + struct list_head list;
> +};
> +
> +/* Xen: Describes informations required for a Xen domain */
> +struct arm_smmu_xen_domain {
> + spinlock_t lock;
> + /* List of context (i.e iommu_domain) associated to this domain */
> + struct list_head contexts;
> +};
> +
> +/*
> + * Xen: Information about each device stored in dev->archdata.iommu
> + *
> + * Initially dev->archdata.iommu only stores the iommu_domain (runtime
> + * configuration of the SMMU) but, on Xen, we also have to store the
> + * iommu_group (list of streamIDs associated to the device).
> + *
> + * This is because Linux has a field iommu_group in the struct device. On Xen,
> + * that would require to move so hackery (dummy iommu_group) in a more generic
> + * place.
> + * */
> +struct arm_smmu_xen_device {
> + struct iommu_domain *domain;
> + struct iommu_group *group;
> +};
> +
> +#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu)
> +#define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
> +#define dev_iommu_group(dev) (dev_archdata(dev)->group)
> +
> +/* Xen: Dummy iommu_group */
> +struct iommu_group
> +{
> + /* Streamids of the device */
> + struct arm_smmu_master_cfg *cfg;
> +
> + atomic_t ref;
> +};
> +
> +static struct iommu_group *iommu_group_alloc(void)
> +{
> + struct iommu_group *group = xzalloc(struct iommu_group);
> +
> + if (!group)
> + return ERR_PTR(-ENOMEM);
>
> -#include <linux/amba/bus.h>
> + atomic_set(&group->ref, 1);
>
> -#include <asm/pgalloc.h>
> + return group;
> +}
> +
> +static void iommu_group_put(struct iommu_group *group)
> +{
> + if (atomic_dec_and_test(&group->ref))
> + xfree(group);
> +}
> +
> +static void iommu_group_set_iommudata(struct iommu_group *group,
> + struct arm_smmu_master_cfg *cfg,
> + void (*releasefn)(void *))
> +{
> + /* TODO: Store the releasefn for the PCI */
> + ASSERT(releasefn == NULL);
> +
> + group->cfg = cfg;
> +}
> +
> +static int iommu_group_add_device(struct iommu_group *group,
> + struct device *dev)
> +{
> + dev_iommu_group(dev) = group;
> +
> + atomic_inc(&group->ref);
> +
> + return 0;
> +}
> +
> +static struct iommu_group *iommu_group_get(struct device *dev)
> +{
> + struct iommu_group *group = dev_iommu_group(dev);
> +
> + if (group)
> + atomic_inc(&group->ref);
> +
> + return group;
> +}
> +
> +#define iommu_group_get_iommudata(group) (group)->cfg
> +
> +/***** Start of Linux SMMU code *****/
>
> /* Maximum number of stream IDs assigned to a single device */
> #define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
> @@ -397,7 +666,9 @@ struct arm_smmu_cfg {
> u8 cbndx;
> u8 irptndx;
> u32 cbar;
> - pgd_t *pgd;
> +
> + /* Xen: Domain associated to this configuration */
> + struct domain *domain;
> };
> #define INVALID_IRPTNDX 0xff
>
> @@ -446,6 +717,7 @@ static void parse_driver_options(struct arm_smmu_device *smmu)
>
> static struct device_node *dev_get_dev_node(struct device *dev)
> {
> +#if 0 /* Xen: TODO: Add support for PCI */
> if (dev_is_pci(dev)) {
> struct pci_bus *bus = to_pci_dev(dev)->bus;
>
> @@ -453,6 +725,7 @@ static struct device_node *dev_get_dev_node(struct device *dev)
> bus = bus->parent;
> return bus->bridge->parent->of_node;
> }
> +#endif
>
> return dev->of_node;
> }
> @@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
> master->of_node = masterspec->np;
> master->cfg.num_streamids = masterspec->args_count;
>
> + /* Xen: Let Xen knows that the device is protected by an SMMU */
> + dt_device_set_protected(masterspec->np);
> +
> for (i = 0; i < master->cfg.num_streamids; ++i) {
> u16 streamid = masterspec->args[i];
>
> @@ -712,6 +988,24 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
> return IRQ_HANDLED;
> }
>
> +/* Xen: Interrupt handlers wrapper */
> +static void arm_smmu_context_fault_xen(int irq, void *dev,
> + struct cpu_user_regs *regs)
> +{
> + arm_smmu_context_fault(irq, dev);
> +}
> +
> +#define arm_smmu_context_fault arm_smmu_context_fault_xen
> +
> +static void arm_smmu_global_fault_xen(int irq, void *dev,
> + struct cpu_user_regs *regs)
> +{
> + arm_smmu_global_fault(irq, dev);
> +}
> +
> +#define arm_smmu_global_fault arm_smmu_global_fault_xen
> +
> +#if 0 /* Xen: Page tables are shared with the processor */
> static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
> size_t size)
> {
> @@ -733,6 +1027,7 @@ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
> DMA_TO_DEVICE);
> }
> }
> +#endif
>
> static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
> {
> @@ -741,6 +1036,7 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
> struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
> struct arm_smmu_device *smmu = smmu_domain->smmu;
> void __iomem *cb_base, *gr0_base, *gr1_base;
> + paddr_t p2maddr;
>
> gr0_base = ARM_SMMU_GR0(smmu);
> gr1_base = ARM_SMMU_GR1(smmu);
> @@ -824,11 +1120,16 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
> }
>
> /* TTBR0 */
> - arm_smmu_flush_pgtable(smmu, cfg->pgd,
> - PTRS_PER_PGD * sizeof(pgd_t));
> - reg = __pa(cfg->pgd);
> + /* Xen: The page table is shared with the P2M code */
> + ASSERT(smmu_domain->cfg.domain != NULL);
> + p2maddr = page_to_maddr(smmu_domain->cfg.domain->arch.p2m.root);
> +
> + dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n",
> + smmu_domain->cfg.domain->domain_id, p2maddr);
> +
> + reg = (p2maddr & ((1ULL << 32) - 1));
> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
> - reg = (phys_addr_t)__pa(cfg->pgd) >> 32;
> + reg = (p2maddr >> 32);
> if (stage1)
> reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
> @@ -873,6 +1174,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
> reg = 0;
> }
>
> + /* Xen: The attributes to walk the page table should be the same as
> + * VTCR_EL2. Currently doesn't differ from Linux ones.
> + */
> reg |= TTBCR_EAE |
> (TTBCR_SH_IS << TTBCR_SH0_SHIFT) |
> (TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) |
> @@ -1015,7 +1319,6 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain)
> static int arm_smmu_domain_init(struct iommu_domain *domain)
> {
> struct arm_smmu_domain *smmu_domain;
> - pgd_t *pgd;
>
> /*
> * Allocate the domain and initialise some of its data structures.
> @@ -1026,20 +1329,12 @@ static int arm_smmu_domain_init(struct iommu_domain *domain)
> if (!smmu_domain)
> return -ENOMEM;
>
> - pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL);
> - if (!pgd)
> - goto out_free_domain;
> - smmu_domain->cfg.pgd = pgd;
> -
> spin_lock_init(&smmu_domain->lock);
> domain->priv = smmu_domain;
> return 0;
> -
> -out_free_domain:
> - kfree(smmu_domain);
> - return -ENOMEM;
> }
>
> +#if 0 /* Xen: Page tables are shared with the processor */
> static void arm_smmu_free_ptes(pmd_t *pmd)
> {
> pgtable_t table = pmd_pgtable(*pmd);
> @@ -1102,6 +1397,7 @@ static void arm_smmu_free_pgtables(struct arm_smmu_domain *smmu_domain)
>
> kfree(pgd_base);
> }
> +#endif
>
> static void arm_smmu_domain_destroy(struct iommu_domain *domain)
> {
> @@ -1112,7 +1408,6 @@ static void arm_smmu_domain_destroy(struct iommu_domain *domain)
> * already been detached.
> */
> arm_smmu_destroy_domain_context(domain);
> - arm_smmu_free_pgtables(smmu_domain);
> kfree(smmu_domain);
> }
>
> @@ -1229,11 +1524,12 @@ static void arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain,
> /*
> * We *must* clear the S2CR first, because freeing the SMR means
> * that it can be re-allocated immediately.
> + * Xen: Unlike Linux, any access to non-configured stream will fault.
> */
> for (i = 0; i < cfg->num_streamids; ++i) {
> u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i];
>
> - writel_relaxed(S2CR_TYPE_BYPASS,
> + writel_relaxed(S2CR_TYPE_FAULT,
> gr0_base + ARM_SMMU_GR0_S2CR(idx));
> }
>
> @@ -1253,7 +1549,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
> return -ENXIO;
> }
>
> - if (dev->archdata.iommu) {
> + if (dev_iommu_domain(dev)) {
> dev_err(dev, "already attached to IOMMU domain\n");
> return -EEXIST;
> }
> @@ -1285,8 +1581,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
> return -ENODEV;
>
> ret = arm_smmu_domain_add_master(smmu_domain, cfg);
> +
> if (!ret)
> - dev->archdata.iommu = domain;
> + dev_iommu_domain(dev) = domain;
> return ret;
> }
>
> @@ -1299,10 +1596,14 @@ static void arm_smmu_detach_dev(struct iommu_domain *domain, struct device *dev)
> if (!cfg)
> return;
>
> - dev->archdata.iommu = NULL;
> + dev_iommu_domain(dev) = NULL;
> arm_smmu_domain_remove_master(smmu_domain, cfg);
> }
>
> +#if 0 /*
> + * Xen: The page table is shared with the processor, therefore
> + * helpers to implement separate is not necessary.
> + */
> static bool arm_smmu_pte_is_contiguous_range(unsigned long addr,
> unsigned long end)
> {
> @@ -1591,7 +1892,9 @@ static phys_addr_t arm_smmu_iova_to_phys(struct iommu_domain *domain,
>
> return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK);
> }
> +#endif
>
> +#if 0 /* Xen: arm_smmu_capable is not used at the moment */
> static bool arm_smmu_capable(enum iommu_cap cap)
> {
> switch (cap) {
> @@ -1609,6 +1912,7 @@ static bool arm_smmu_capable(enum iommu_cap cap)
> return false;
> }
> }
> +#endif
>
> static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data)
> {
> @@ -1676,6 +1980,7 @@ out_put_group:
> return ret;
> }
>
> +#if 0 /* Xen: We don't support remove device for now. Will be useful for PCI */
> static void arm_smmu_remove_device(struct device *dev)
> {
> iommu_group_remove_device(dev);
> @@ -1733,6 +2038,7 @@ static const struct iommu_ops arm_smmu_ops = {
> ARM_SMMU_PTE_CONT_SIZE |
> PAGE_SIZE),
> };
> +#endif
>
> static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
> {
> @@ -1748,7 +2054,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
> /* Mark all SMRn as invalid and all S2CRn as bypass */
> for (i = 0; i < smmu->num_mapping_groups; ++i) {
> writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i));
> - writel_relaxed(S2CR_TYPE_BYPASS,
> + /*
> + * Xen: Unlike Linux, any access to a non-configure stream
> + * will fault by default.
> + */
> + writel_relaxed(S2CR_TYPE_FAULT,
> gr0_base + ARM_SMMU_GR0_S2CR(i));
> }
>
> @@ -1774,6 +2084,8 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
>
> /* Enable client access, but bypass when no mapping is found */
> reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
> + /* Xen: Unlike Linux, generate a fault when no mapping is found */
> + reg |= sCR0_USFCFG;
>
> /* Disable forced broadcasting */
> reg &= ~sCR0_FB;
> @@ -1882,7 +2194,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
> }
>
> dev_notice(smmu->dev,
> - "\tstream matching with %u register groups, mask 0x%x",
> + "\tstream matching with %u register groups, mask 0x%x\n",
> smmu->num_mapping_groups, mask);
> } else {
> smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) &
> @@ -1917,12 +2229,16 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
> size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK);
> smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
>
> + /* Xen: Stage-2 input size is not restricted */
> + smmu->s2_input_size = size;
> +#if 0
> /* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */
> #ifdef CONFIG_64BIT
> smmu->s2_input_size = min_t(unsigned long, VA_BITS, size);
> #else
> smmu->s2_input_size = min(32UL, size);
> #endif
> +#endif
>
> /* The stage-2 output mask is also applied for bypass */
> size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK);
> @@ -1969,6 +2285,10 @@ static const struct of_device_id arm_smmu_of_match[] = {
> };
> MODULE_DEVICE_TABLE(of, arm_smmu_of_match);
>
> +/*
> + * Xen: We don't have refcount for allocated memory so manually free memory
> + * when an error occured.
> + */
> static int arm_smmu_device_dt_probe(struct platform_device *pdev)
> {
> const struct of_device_id *of_id;
> @@ -1991,14 +2311,17 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
>
> res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> smmu->base = devm_ioremap_resource(dev, res);
> - if (IS_ERR(smmu->base))
> - return PTR_ERR(smmu->base);
> + if (IS_ERR(smmu->base)) {
> + err = PTR_ERR(smmu->base);
> + goto out_free;
> + }
> smmu->size = resource_size(res);
>
> if (of_property_read_u32(dev->of_node, "#global-interrupts",
> &smmu->num_global_irqs)) {
> dev_err(dev, "missing #global-interrupts property\n");
> - return -ENODEV;
> + err = -ENODEV;
> + goto out_free;
> }
>
> num_irqs = 0;
> @@ -2011,14 +2334,16 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
> if (!smmu->num_context_irqs) {
> dev_err(dev, "found %d interrupts but expected at least %d\n",
> num_irqs, smmu->num_global_irqs + 1);
> - return -ENODEV;
> + err = -ENODEV;
> + goto out_free;
> }
>
> smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
> GFP_KERNEL);
> if (!smmu->irqs) {
> dev_err(dev, "failed to allocate %d irqs\n", num_irqs);
> - return -ENOMEM;
> + err = -ENOMEM;
> + goto out_free;
> }
>
> for (i = 0; i < num_irqs; ++i) {
> @@ -2026,7 +2351,8 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
>
> if (irq < 0) {
> dev_err(dev, "failed to get irq index %d\n", i);
> - return -ENODEV;
> + err = -ENODEV;
> + goto out_free;
> }
> smmu->irqs[i] = irq;
> }
> @@ -2091,12 +2417,19 @@ out_put_masters:
> for (node = rb_first(&smmu->masters); node; node = rb_next(node)) {
> struct arm_smmu_master *master
> = container_of(node, struct arm_smmu_master, node);
> - of_node_put(master->of_node);
> + kfree(master);
> }
>
> +out_free:
> + kfree(smmu->irqs);
> + if (!IS_ERR(smmu->base))
> + iounmap(smmu->base);
> + kfree(smmu);
> +
> return err;
> }
>
> +#if 0 /* Xen: We never remove SMMU */
> static int arm_smmu_device_remove(struct platform_device *pdev)
> {
> int i;
> @@ -2191,3 +2524,241 @@ module_exit(arm_smmu_exit);
> MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
> MODULE_AUTHOR("Will Deacon <will.deacon@arm.com>");
> MODULE_LICENSE("GPL v2");
> +#endif
> +
> +/***** Start of Xen specific code *****/
> +
> +/* Xen only supports stage-2 translation, so force the value to 2. */
> +static int force_stage = 2;
> +
> +static void arm_smmu_iotlb_flush_all(struct domain *d)
> +{
> + struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
> + struct iommu_domain *cfg;
> +
> + spin_lock(&smmu_domain->lock);
> + list_for_each_entry(cfg, &smmu_domain->contexts, list) {
> + /*
> + * Only invalidate the context when SMMU is present.
> + * This is because the context initialization is delayed
> + * until a master has been added.
> + */
> + if (unlikely(!ACCESS_ONCE(cfg->priv->smmu)))
> + continue;
> + arm_smmu_tlb_inv_context(cfg->priv);
> + }
> + spin_unlock(&smmu_domain->lock);
> +}
> +
> +static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn,
> + unsigned int page_count)
> +{
> + /* ARM SMMU v1 doesn't have flush by VMA and VMID */
> + arm_smmu_iotlb_flush_all(d);
> +}
> +
> +static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
> + struct device *dev)
> +{
> + struct iommu_domain *domain;
> + struct arm_smmu_xen_domain *xen_domain;
> + int ret;
> +
> + xen_domain = domain_hvm_iommu(d)->arch.priv;
> +
> + if (!dev->archdata.iommu) {
> + dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device);
> + if (!dev->archdata.iommu)
> + return -ENOMEM;
> + }
> +
> + if (!dev_iommu_group(dev)) {
> + ret = arm_smmu_add_device(dev);
> + if (ret)
> + return ret;
> + }
> +
> + /*
> + * TODO: Share the context bank (i.e iommu_domain) when the device is
> + * under the same SMMU as another device assigned to this domain.
> + * Would it useful for PCI
> + */
> + domain = xzalloc(struct iommu_domain);
> + if (!domain)
> + return -ENOMEM;
> +
> + ret = arm_smmu_domain_init(domain);
> + if (ret)
> + goto err_dom_init;
> +
> + domain->priv->cfg.domain = d;
> +
> + ret = arm_smmu_attach_dev(domain, dev);
> + if (ret)
> + goto err_attach_dev;
> +
> + spin_lock(&xen_domain->lock);
> + /* Chain the new context to the domain */
> + list_add(&domain->list, &xen_domain->contexts);
> + spin_unlock(&xen_domain->lock);
> +
> + return 0;
> +
> +err_attach_dev:
> + arm_smmu_domain_destroy(domain);
> +err_dom_init:
> + xfree(domain);
> +
> + return ret;
> +}
> +
> +static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
> +{
> + struct iommu_domain *domain = dev_iommu_domain(dev);
> + struct arm_smmu_xen_domain *xen_domain;
> +
> + xen_domain = domain_hvm_iommu(d)->arch.priv;
> +
> + if (!domain || domain->priv->cfg.domain != d) {
> + dev_err(dev, " not attached to domain %d\n", d->domain_id);
> + return -ESRCH;
> + }
> +
> + arm_smmu_detach_dev(domain, dev);
> +
> + spin_lock(&xen_domain->lock);
> + list_del(&domain->list);
> + spin_unlock(&xen_domain->lock);
> +
> + arm_smmu_domain_destroy(domain);
> + xfree(domain);
> +
> + return 0;
> +}
> +
> +static int arm_smmu_reassign_dev(struct domain *s, struct domain *t,
> + u8 devfn, struct device *dev)
> +{
> + int ret = 0;
> +
> + /* Don't allow remapping on other domain than hwdom */
> + if (t != hardware_domain)
> + return -EPERM;
> +
> + if (t == s)
> + return 0;
> +
> + ret = arm_smmu_deassign_dev(s, dev);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> +static int arm_smmu_iommu_domain_init(struct domain *d)
> +{
> + struct arm_smmu_xen_domain *xen_domain;
> +
> + xen_domain = xzalloc(struct arm_smmu_xen_domain);
> + if ( !xen_domain )
> + return -ENOMEM;
> +
> + spin_lock_init(&xen_domain->lock);
> + INIT_LIST_HEAD(&xen_domain->contexts);
> +
> + domain_hvm_iommu(d)->arch.priv = xen_domain;
> +
> + return 0;
> +}
> +
> +static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d)
> +{
> +}
> +
> +static void arm_smmu_iommu_domain_teardown(struct domain *d)
> +{
> + struct arm_smmu_xen_domain *xen_domain = domain_hvm_iommu(d)->arch.priv;
> +
> + ASSERT(list_empty(&xen_domain->contexts));
> + xfree(xen_domain);
> +}
> +
> +static int arm_smmu_map_page(struct domain *d, unsigned long gfn,
> + unsigned long mfn, unsigned int flags)
> +{
> + p2m_type_t t;
> +
> + /*
> + * Grant mappings can be used for DMA requests. The dev_bus_addr
> + * returned by the hypercall is the MFN (not the IPA). For device
> + * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain
> + * p2m to allow DMA request to work.
> + * This is only valid when the domain is directed mapped. Hence this
> + * function should only be used by gnttab code with gfn == mfn.
> + */
> + BUG_ON(!is_domain_direct_mapped(d));
> + BUG_ON(mfn != gfn);
> +
> + /* We only support readable and writable flags */
> + if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
> + return -EINVAL;
> +
> + t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro;
> +
> + /*
> + * The function guest_physmap_add_entry replaces the current mapping
> + * if there is already one...
> + */
> + return guest_physmap_add_entry(d, gfn, mfn, 0, t);
> +}
> +
> +static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn)
> +{
> + /*
> + * This function should only be used by gnttab code when the domain
> + * is direct mapped
> + */
> + if ( !is_domain_direct_mapped(d) )
> + return -EINVAL;
> +
> + guest_physmap_remove_page(d, gfn, gfn, 0);
> +
> + return 0;
> +}
> +
> +static const struct iommu_ops arm_smmu_iommu_ops = {
> + .init = arm_smmu_iommu_domain_init,
> + .hwdom_init = arm_smmu_iommu_hwdom_init,
> + .teardown = arm_smmu_iommu_domain_teardown,
> + .iotlb_flush = arm_smmu_iotlb_flush,
> + .iotlb_flush_all = arm_smmu_iotlb_flush_all,
> + .assign_device = arm_smmu_assign_dev,
> + .reassign_device = arm_smmu_reassign_dev,
> + .map_page = arm_smmu_map_page,
> + .unmap_page = arm_smmu_unmap_page,
> +};
> +
> +static __init int arm_smmu_dt_init(struct dt_device_node *dev,
> + const void *data)
> +{
> + int rc;
> +
> + /*
> + * Even if the device can't be initialized, we don't want to
> + * give the SMMU device to dom0.
> + */
> + dt_device_set_used_by(dev, DOMID_XEN);
> +
> + rc = arm_smmu_device_dt_probe(dev);
> + if (rc)
> + return rc;
> +
> + iommu_set_ops(&arm_smmu_iommu_ops);
> +
> + return 0;
> +}
> +
> +DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU)
> + .dt_match = arm_smmu_of_match,
> + .init = arm_smmu_dt_init,
> +DT_DEVICE_END
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk
2015-02-25 18:53 ` [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk Julien Grall
@ 2015-03-02 13:10 ` Ian Campbell
2015-03-02 15:23 ` Julien Grall
0 siblings, 1 reply; 19+ messages in thread
From: Ian Campbell @ 2015-03-02 13:10 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
On Wed, 2015-02-25 at 18:53 +0000, Julien Grall wrote:
> When SMMU doesn't support coherent table walk, Xen may need to clean
> updated PT (see commit 4c5f4cb "xen/arm: p2m: Clean cache PT when the
> IOMMU doesn't support coherent walk").
>
> If one SMMU of the platform doesn't support coherent table walk, the
> feature is disabled for the whole platform. This is because device is
> assigned to a domain after the page table are populated.
>
> This could impact performance on domain which doesn't use device
> passthrough. But, as the spec strongly recommends the support of this
> feature for maintstream platform, I expect server will always have SMMUs
"mainstream".
> supporting coherent table walk. If not, we may need to enable this feature
> per-domain.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
> I've just noticed that the support on the previous driver (i.e in
> Xen 4.5) may incorrectly expose this feature when all the SMMUs is
> not supporting coherent table walk.
>
> I'm not sure, if I should send a patch for it.
>
> Also I didn't squash this patch into "xen/iommu: smmu: Add Xen specific
> code to be able to use the driver" to help for review and to catch
> possible error in this patch.
>
> Changes in v4:
> - Browse the list to retrieve the last SMMU added.
> - Take the spinlock
>
> Changes in v3:
> - Patch added
> ---
> xen/drivers/passthrough/arm/smmu.c | 29 +++++++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
>
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index d01a26a..b478463 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -2531,6 +2531,13 @@ MODULE_LICENSE("GPL v2");
> /* Xen only supports stage-2 translation, so force the value to 2. */
> static int force_stage = 2;
>
> +/*
> + * Platform features. It indicates the list of features supported by all the
> + * SMMUs.
s/the//
> + * Actually we only care about coherent table walk.
> + */
> +static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK;
> +
> static void arm_smmu_iotlb_flush_all(struct domain *d)
> {
> struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
> @@ -2668,6 +2675,13 @@ static int arm_smmu_iommu_domain_init(struct domain *d)
>
> domain_hvm_iommu(d)->arch.priv = xen_domain;
>
> + /*
> + * The feature coherent walk can be enabled only when all SMMUs
> + * support it.
s/The feature c/C/
or "The coherent walk feature..." (I prefer the former since it is
briefer)
> + */
> + if (platform_features & ARM_SMMU_FEAT_COHERENT_WALK)
> + iommu_set_feature(d, IOMMU_FEAT_COHERENT_WALK);
> +
> return 0;
> }
>
> @@ -2742,6 +2756,7 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
> const void *data)
> {
> int rc;
> + struct arm_smmu_device *smmu;
>
> /*
> * Even if the device can't be initialized, we don't want to
> @@ -2755,6 +2770,20 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>
> iommu_set_ops(&arm_smmu_iommu_ops);
>
> + /* Find the last SMMU added and retrieve its features. */
This comment no longer applies, I think?
> + spin_lock(&arm_smmu_devices_lock);
> + list_for_each_entry(smmu, &arm_smmu_devices, list) {
> + if (smmu->dev == &dev->dev)
> + goto found;
Please try and avoid goto. In this case I think
{
platform_features &= smmu->features;
break;
}
within the if would be fine, combined with dropping the following BUG().
Alternatively you might prefer to provide a helper function to lookup an
smmu by &dev->dev.
> + }
> +
> + BUG();
> +
> +found:
> + platform_features &= smmu->features;
> +
> + spin_unlock(&arm_smmu_devices_lock);
> +
> return 0;
> }
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-02-25 18:52 ` [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Julien Grall
2015-03-02 7:03 ` Manish Jaggi
@ 2015-03-02 13:17 ` Ian Campbell
2015-03-02 13:45 ` Julien Grall
1 sibling, 1 reply; 19+ messages in thread
From: Ian Campbell @ 2015-03-02 13:17 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
On Wed, 2015-02-25 at 18:52 +0000, Julien Grall wrote:
> The main goal is to modify as little the Linux code to be able to port
> easily new feature added in Linux repo for the driver.
>
> To achieve that we:
> - Add helpers to Linux function not implemented on Xen
> - Add callbacks used by Xen to do our own stuff and call Linux ones
> - Only modify when required the code which comes from Linux. If so a
> comment has been added with /* Xen: ... */ explaining why it's
> necessary.
>
> The support for PCI has been commented because it's not yet supported by
> Xen ARM and therefore won't compile.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Looks good to me now, thanks. Couple of nits inline, but otherwise:
Acked-by: Ian Campbell <ian.campbell@citrix.com>
> + * This is because Linux has a field iommu_group in the struct device. On Xen,
> + * that would require to move so hackery (dummy iommu_group) in a more generic
Was "so" here supposed to be "some"?
> @@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
> master->of_node = masterspec->np;
> master->cfg.num_streamids = masterspec->args_count;
>
> + /* Xen: Let Xen knows that the device is protected by an SMMU */
s/knows/know/
Ian.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-03-02 13:17 ` Ian Campbell
@ 2015-03-02 13:45 ` Julien Grall
2015-03-02 13:54 ` Ian Campbell
0 siblings, 1 reply; 19+ messages in thread
From: Julien Grall @ 2015-03-02 13:45 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
Hi Ian,
On 02/03/15 13:17, Ian Campbell wrote:
> On Wed, 2015-02-25 at 18:52 +0000, Julien Grall wrote:
>> The main goal is to modify as little the Linux code to be able to port
>> easily new feature added in Linux repo for the driver.
>>
>> To achieve that we:
>> - Add helpers to Linux function not implemented on Xen
>> - Add callbacks used by Xen to do our own stuff and call Linux ones
>> - Only modify when required the code which comes from Linux. If so a
>> comment has been added with /* Xen: ... */ explaining why it's
>> necessary.
>>
>> The support for PCI has been commented because it's not yet supported by
>> Xen ARM and therefore won't compile.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> Looks good to me now, thanks. Couple of nits inline, but otherwise:
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
Thanks for the ack.
>> + * This is because Linux has a field iommu_group in the struct device. On Xen,
>> + * that would require to move so hackery (dummy iommu_group) in a more generic
>
> Was "so" here supposed to be "some"?
Yes.
>> @@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
>> master->of_node = masterspec->np;
>> master->cfg.num_streamids = masterspec->args_count;
>>
>> + /* Xen: Let Xen knows that the device is protected by an SMMU */
>
> s/knows/know/
Yes.
Given that the series has been acked up to this patch (#6), would it be
possible to apply the patches #1-#6)?
The patch #7 is not strictly necessary, by default Xen will flush the
cache for the PT.
If it's fine for you, I will resend the patch #7 separately.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-03-02 13:45 ` Julien Grall
@ 2015-03-02 13:54 ` Ian Campbell
0 siblings, 0 replies; 19+ messages in thread
From: Ian Campbell @ 2015-03-02 13:54 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
On Mon, 2015-03-02 at 13:45 +0000, Julien Grall wrote:
> Hi Ian,
>
> On 02/03/15 13:17, Ian Campbell wrote:
> > On Wed, 2015-02-25 at 18:52 +0000, Julien Grall wrote:
> >> The main goal is to modify as little the Linux code to be able to port
> >> easily new feature added in Linux repo for the driver.
> >>
> >> To achieve that we:
> >> - Add helpers to Linux function not implemented on Xen
> >> - Add callbacks used by Xen to do our own stuff and call Linux ones
> >> - Only modify when required the code which comes from Linux. If so a
> >> comment has been added with /* Xen: ... */ explaining why it's
> >> necessary.
> >>
> >> The support for PCI has been commented because it's not yet supported by
> >> Xen ARM and therefore won't compile.
> >>
> >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >
> > Looks good to me now, thanks. Couple of nits inline, but otherwise:
> > Acked-by: Ian Campbell <ian.campbell@citrix.com>
>
> Thanks for the ack.
>
> >> + * This is because Linux has a field iommu_group in the struct device. On Xen,
> >> + * that would require to move so hackery (dummy iommu_group) in a more generic
> >
> > Was "so" here supposed to be "some"?
>
> Yes.
>
> >> @@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
> >> master->of_node = masterspec->np;
> >> master->cfg.num_streamids = masterspec->args_count;
> >>
> >> + /* Xen: Let Xen knows that the device is protected by an SMMU */
> >
> > s/knows/know/
>
> Yes.
>
> Given that the series has been acked up to this patch (#6), would it be
> possible to apply the patches #1-#6)?
Yes, for some reason I thought only up to #6 wasn't useful because it
hadn't reinstated the driver yet, but I seem to have miscounted.
I'll apply that subset shortly (and fixup the things above).
> The patch #7 is not strictly necessary, by default Xen will flush the
> cache for the PT.
>
> If it's fine for you, I will resend the patch #7 separately.
Please do, thanks.
Ian.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
` (7 preceding siblings ...)
2015-02-25 18:53 ` [PATCH v4 8/8] DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU Julien Grall
@ 2015-03-02 14:47 ` Ian Campbell
8 siblings, 0 replies; 19+ messages in thread
From: Ian Campbell @ 2015-03-02 14:47 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
On Wed, 2015-02-25 at 18:52 +0000, Julien Grall wrote:
> xen/iommu: arm: Remove temporary the SMMU driver
> xen/arm: Introduce a generic way to describe device
> xen/iommu: Consolidate device assignment ops into a single set
> xen/arm: Describe devices supported by a driver with dt_device_match
> xen/iommu: arm: Import the SMMU driver from Linux
> xen/iommu: smmu: Add Xen specific code to be able to use the driver
Applied these.
> xen/iommu: smmu: Advertise when the SMMU support coherent table walk
> DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU
But not these. You were going to resend the first separately.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk
2015-03-02 13:10 ` Ian Campbell
@ 2015-03-02 15:23 ` Julien Grall
2015-03-02 16:24 ` Ian Campbell
0 siblings, 1 reply; 19+ messages in thread
From: Julien Grall @ 2015-03-02 15:23 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
Hi Ian,
On 02/03/15 13:10, Ian Campbell wrote:
>> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
>> index d01a26a..b478463 100644
>> --- a/xen/drivers/passthrough/arm/smmu.c
>> +++ b/xen/drivers/passthrough/arm/smmu.c
>> @@ -2531,6 +2531,13 @@ MODULE_LICENSE("GPL v2");
>> /* Xen only supports stage-2 translation, so force the value to 2. */
>> static int force_stage = 2;
>>
>> +/*
>> + * Platform features. It indicates the list of features supported by all the
>> + * SMMUs.
>
> s/the//
>
>> + * Actually we only care about coherent table walk.
>> + */
>> +static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK;
>> +
>> static void arm_smmu_iotlb_flush_all(struct domain *d)
>> {
>> struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
>> @@ -2668,6 +2675,13 @@ static int arm_smmu_iommu_domain_init(struct domain *d)
>>
>> domain_hvm_iommu(d)->arch.priv = xen_domain;
>>
>> + /*
>> + * The feature coherent walk can be enabled only when all SMMUs
>> + * support it.
>
> s/The feature c/C/
>
> or "The coherent walk feature..." (I prefer the former since it is
> briefer)
I will use the former.
>
>> + */
>> + if (platform_features & ARM_SMMU_FEAT_COHERENT_WALK)
>> + iommu_set_feature(d, IOMMU_FEAT_COHERENT_WALK);
>> +
>> return 0;
>> }
>>
>> @@ -2742,6 +2756,7 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>> const void *data)
>> {
>> int rc;
>> + struct arm_smmu_device *smmu;
>>
>> /*
>> * Even if the device can't be initialized, we don't want to
>> @@ -2755,6 +2770,20 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>>
>> iommu_set_ops(&arm_smmu_iommu_ops);
>>
>> + /* Find the last SMMU added and retrieve its features. */
>
> This comment no longer applies, I think?
I think it's useful to have a comment explaining why we are retrieving
the SMMU.
>> + spin_lock(&arm_smmu_devices_lock);
>> + list_for_each_entry(smmu, &arm_smmu_devices, list) {
>> + if (smmu->dev == &dev->dev)
>> + goto found;
>
> Please try and avoid goto. In this case I think
> {
> platform_features &= smmu->features;
> break;
> }
I though about this solution but it doesn't say if for some reason we
miss to find the SMMU.
>
> within the if would be fine, combined with dropping the following BUG().
>
> Alternatively you might prefer to provide a helper function to lookup an
> smmu by &dev->dev.
I would like to keep the BUG(). Because this would be a mistake to not
find the last SMMU added.
I will move to an helper function.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk
2015-03-02 15:23 ` Julien Grall
@ 2015-03-02 16:24 ` Ian Campbell
0 siblings, 0 replies; 19+ messages in thread
From: Ian Campbell @ 2015-03-02 16:24 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, manish.jaggi, stefano.stabellini, tim
On Mon, 2015-03-02 at 15:23 +0000, Julien Grall wrote:
> >> + /* Find the last SMMU added and retrieve its features. */
> >
> > This comment no longer applies, I think?
>
> I think it's useful to have a comment explaining why we are retrieving
> the SMMU.
Sorry, I initially misread things and thought this was processing all
smmus.
>
> >> + spin_lock(&arm_smmu_devices_lock);
> >> + list_for_each_entry(smmu, &arm_smmu_devices, list) {
> >> + if (smmu->dev == &dev->dev)
> >> + goto found;
> >
> > Please try and avoid goto. In this case I think
> > {
> > platform_features &= smmu->features;
> > break;
> > }
>
> I though about this solution but it doesn't say if for some reason we
> miss to find the SMMU.
I think that's fine in this context. If it isn't on the list at this
point it might as well not exist and/or there would have been an error
somewhere else already.
> >
> > within the if would be fine, combined with dropping the following BUG().
> >
> > Alternatively you might prefer to provide a helper function to lookup an
> > smmu by &dev->dev.
>
> I would like to keep the BUG(). Because this would be a mistake to not
> find the last SMMU added.
Sounds more like an ASSERT to me.
> I will move to an helper function.
Sure.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver
2015-03-02 7:03 ` Manish Jaggi
@ 2015-03-04 2:08 ` Manish Jaggi
0 siblings, 0 replies; 19+ messages in thread
From: Manish Jaggi @ 2015-03-04 2:08 UTC (permalink / raw)
To: Julien Grall, xen-devel
Cc: stefano.stabellini, manish.jaggi, tim, ian.campbell
On 02/03/15 12:33 pm, Manish Jaggi wrote:
>
> On Thursday 26 February 2015 12:22 AM, Julien Grall wrote:
>> The main goal is to modify as little the Linux code to be able to port
>> easily new feature added in Linux repo for the driver.
>>
>> To achieve that we:
>> - Add helpers to Linux function not implemented on Xen
>> - Add callbacks used by Xen to do our own stuff and call Linux ones
>> - Only modify when required the code which comes from Linux. If
>> so a
>> comment has been added with /* Xen: ... */ explaining why it's
>> necessary.
>>
>> The support for PCI has been commented because it's not yet supported by
>> Xen ARM and therefore won't compile.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>
>> ---
>> Changes in v4:
>> - Re-shuffle the code to rationalize the number of lines
>> changed
>> in the Linux SMMU code
>> - Use #if 0 /* Xen: ... */ rather than using 2 lines
>> - Add a bunch of definitions to avoid some #if 0
>> - Remove the "hack" for the Midway SMMU
>>
>> Changes in v2:
>> - Add the ACCESS_ONCE definition in the drivers. The patch to
>> introduce the one in common code has been dropped.
>> - The include xen/device.h has been dropped in favor of
>> asm/device.h
>> ---
>> xen/drivers/passthrough/arm/Makefile | 1 +
>> xen/drivers/passthrough/arm/smmu.c | 661
>> ++++++++++++++++++++++++++++++++---
>> 2 files changed, 617 insertions(+), 45 deletions(-)
> Hi Julien,
> Few comments (first level)
> a) What is the difference between assign / attach / add. There are
> different apis and I get lost reading the code between them.
> Either provide a description or use a better nomenclature.
>
> b) In which case a xen domain be having multiple contexts. Even though
> it is technically possible but there is no defined method to create a
> different context in non-pci or pci-passthrough.
> Could you please add some detail.
>
Julien, could you please address these points
> -mj
>>
>> diff --git a/xen/drivers/passthrough/arm/Makefile
>> b/xen/drivers/passthrough/arm/Makefile
>> index 0484b79..f4cd26e 100644
>> --- a/xen/drivers/passthrough/arm/Makefile
>> +++ b/xen/drivers/passthrough/arm/Makefile
>> @@ -1 +1,2 @@
>> obj-y += iommu.o
>> +obj-y += smmu.o
>> diff --git a/xen/drivers/passthrough/arm/smmu.c
>> b/xen/drivers/passthrough/arm/smmu.c
>> index 6cd47b7..d01a26a 100644
>> --- a/xen/drivers/passthrough/arm/smmu.c
>> +++ b/xen/drivers/passthrough/arm/smmu.c
>> @@ -18,6 +18,13 @@
>> *
>> * Author: Will Deacon <will.deacon@arm.com>
>> *
>> + * Based on Linux drivers/iommu/arm-smmu.c
>> + * => commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3
>> + *
>> + * Xen modification:
>> + * Julien Grall <julien.grall@linaro.org>
>> + * Copyright (C) 2014 Linaro Limited.
>> + *
>> * This driver currently supports:
>> * - SMMUv1 and v2 implementations
>> * - Stream-matching and stream-indexing
>> @@ -28,25 +35,287 @@
>> * - Context fault reporting
>> */
>> -#define pr_fmt(fmt) "arm-smmu: " fmt
>> -#include <linux/delay.h>
>> -#include <linux/dma-mapping.h>
>> -#include <linux/err.h>
>> -#include <linux/interrupt.h>
>> -#include <linux/io.h>
>> -#include <linux/iommu.h>
>> -#include <linux/mm.h>
>> -#include <linux/module.h>
>> -#include <linux/of.h>
>> -#include <linux/pci.h>
>> -#include <linux/platform_device.h>
>> -#include <linux/slab.h>
>> -#include <linux/spinlock.h>
>> +#include <xen/config.h>
>> +#include <xen/delay.h>
>> +#include <xen/errno.h>
>> +#include <xen/err.h>
>> +#include <xen/irq.h>
>> +#include <xen/lib.h>
>> +#include <xen/list.h>
>> +#include <xen/mm.h>
>> +#include <xen/vmap.h>
>> +#include <xen/rbtree.h>
>> +#include <xen/sched.h>
>> +#include <xen/sizes.h>
>> +#include <asm/atomic.h>
>> +#include <asm/device.h>
>> +#include <asm/io.h>
>> +#include <asm/platform.h>
>> +
>> +/* Xen: The below defines are redefined within the file. Undef it */
>> +#undef SCTLR_AFE
>> +#undef SCTLR_TRE
>> +#undef SCTLR_M
>> +#undef TTBCR_EAE
>> +
>> +/* Alias to Xen device tree helpers */
>> +#define device_node dt_device_node
>> +#define of_phandle_args dt_phandle_args
>> +#define of_device_id dt_device_match
>> +#define of_match_node dt_match_node
>> +#define of_property_read_u32(np, pname, out)
>> (!dt_property_read_u32(np, pname, out))
>> +#define of_property_read_bool dt_property_read_bool
>> +#define of_parse_phandle_with_args dt_parse_phandle_with_args
>> +
>> +/* Xen: Helpers to get device MMIO and IRQs */
>> +struct resource
>> +{
>> + u64 addr;
>> + u64 size;
>> + unsigned int type;
>> +};
>> +
>> +#define resource_size(res) (res)->size;
>> +
>> +#define platform_device dt_device_node
>> +
>> +#define IORESOURCE_MEM 0
>> +#define IORESOURCE_IRQ 1
>> +
>> +static struct resource *platform_get_resource(struct platform_device
>> *pdev,
>> + unsigned int type,
>> + unsigned int num)
>> +{
>> + /*
>> + * The resource is only used between 2 calls of
>> platform_get_resource.
>> + * It's quite ugly but it's avoid to add too much code in the part
>> + * imported from Linux
>> + */
>> + static struct resource res;
>> + int ret = 0;
>> +
>> + res.type = type;
>> +
>> + switch (type) {
>> + case IORESOURCE_MEM:
>> + ret = dt_device_get_address(pdev, num, &res.addr, &res.size);
>> +
>> + return ((ret) ? NULL : &res);
>> +
>> + case IORESOURCE_IRQ:
>> + ret = platform_get_irq(pdev, num);
>> + if (ret < 0)
>> + return NULL;
>> +
>> + res.addr = ret;
>> + res.size = 1;
>> +
>> + return &res;
>> +
>> + default:
>> + return NULL;
>> + }
>> +}
>> +
>> +/* Xen: Helpers for IRQ functions */
>> +#define request_irq(irq, func, flags, name, dev) request_irq(irq,
>> flags, func, name, dev)
>> +#define free_irq release_irq
>> +
>> +enum irqreturn {
>> + IRQ_NONE = (0 << 0),
>> + IRQ_HANDLED = (1 << 0),
>> +};
>> +
>> +typedef enum irqreturn irqreturn_t;
>> +
>> +/* Device logger functions
>> + * TODO: Handle PCI
>> + */
>> +#define dev_print(dev, lvl, fmt, ...) \
>> + printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)),
>> ## __VA_ARGS__)
>> +
>> +#define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ##
>> __VA_ARGS__)
>> +#define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt,
>> ## __VA_ARGS__)
>> +#define dev_warn(dev, fmt, ...) dev_print(dev, XENLOG_WARNING, fmt,
>> ## __VA_ARGS__)
>> +#define dev_err(dev, fmt, ...) dev_print(dev, XENLOG_ERR, fmt, ##
>> __VA_ARGS__)
>> +
>> +#define dev_err_ratelimited(dev, fmt, ...) \
>> + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__)
>> +
>> +#define dev_name(dev) dt_node_full_name(dev_to_dt(dev))
>> +
>> +/* Alias to Xen allocation helpers */
>> +#define kfree xfree
>> +#define kmalloc(size, flags) _xmalloc(size, sizeof(void *))
>> +#define kzalloc(size, flags) _xzalloc(size, sizeof(void *))
>> +#define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void
>> *))
>> +#define kmalloc_array(size, n, flags) _xmalloc_array(size,
>> sizeof(void *), n)
>> +
>> +static void __iomem *devm_ioremap_resource(struct device *dev,
>> + struct resource *res)
>> +{
>> + void __iomem *ptr;
>> +
>> + if (!res || res->type != IORESOURCE_MEM) {
>> + dev_err(dev, "Invalid resource\n");
>> + return ERR_PTR(-EINVAL);
>> + }
>> +
>> + ptr = ioremap_nocache(res->addr, res->size);
>> + if (!ptr) {
>> + dev_err(dev,
>> + "ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n",
>> + res->addr, res->size);
>> + return ERR_PTR(-ENOMEM);
>> + }
>> +
>> + return ptr;
>> +}
>> +
>> +/* Xen doesn't handle IOMMU fault */
>> +#define report_iommu_fault(...) 1
>> +
>> +#define IOMMU_FAULT_READ 0
>> +#define IOMMU_FAULT_WRITE 1
>> +
>> +/*
>> + * Xen: PCI functions
>> + * TODO: It should be implemented when PCI will be supported
>> + */
>> +#define to_pci_dev(dev) (NULL)
>> +static inline int pci_for_each_dma_alias(struct pci_dev *pdev,
>> + int (*fn) (struct pci_dev *pdev,
>> + u16 alias, void *data),
>> + void *data)
>> +{
>> + BUG();
>> + return 0;
>> +}
>> +
>> +/* Xen: misc */
>> +#define PHYS_MASK_SHIFT PADDR_BITS
>> +typedef paddr_t phys_addr_t;
>> +
>> +#ifdef CONFIG_ARM_64
>> +# define CONFIG_64BIT
>> +#endif
>> +
>> +#define VA_BITS 0 /* Only used for configuring stage-1
>> input size */
>> +
>> +/* The macro ACCESS_ONCE start to be replaced in Linux in favor of
>> + * {READ, WRITE}_ONCE. Rather than introducing in the common code,
>> keep a
>> + * version here. We will have to drop it when the SMMU code in Linux
>> will
>> + * switch to {READ, WRITE}_ONCE.
>> + */
>> +#define __ACCESS_ONCE(x) ({ \
>> + __maybe_unused typeof(x) __var = 0; \
>> + (volatile typeof(x) *)&(x); })
>> +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x))
>> +
>> +#define MODULE_DEVICE_TABLE(type, name)
>> +#define module_param_named(name, value, type, perm)
>> +#define MODULE_PARM_DESC(_parm, desc)
>> +
>> +/* Xen: Dummy iommu_domain */
>> +struct iommu_domain
>> +{
>> + /* Runtime SMMU configuration for this iommu_domain */
>> + struct arm_smmu_domain *priv;
>> +
>> + /* Used to link iommu_domain contexts for a same domain.
>> + * There is at least one per-SMMU to used by the domain.
>> + * */
>> + struct list_head list;
>> +};
>> +
>> +/* Xen: Describes informations required for a Xen domain */
>> +struct arm_smmu_xen_domain {
>> + spinlock_t lock;
>> + /* List of context (i.e iommu_domain) associated to this domain */
>> + struct list_head contexts;
>> +};
>> +
>> +/*
>> + * Xen: Information about each device stored in dev->archdata.iommu
>> + *
>> + * Initially dev->archdata.iommu only stores the iommu_domain (runtime
>> + * configuration of the SMMU) but, on Xen, we also have to store the
>> + * iommu_group (list of streamIDs associated to the device).
>> + *
>> + * This is because Linux has a field iommu_group in the struct
>> device. On Xen,
>> + * that would require to move so hackery (dummy iommu_group) in a
>> more generic
>> + * place.
>> + * */
>> +struct arm_smmu_xen_device {
>> + struct iommu_domain *domain;
>> + struct iommu_group *group;
>> +};
>> +
>> +#define dev_archdata(dev) ((struct arm_smmu_xen_device
>> *)dev->archdata.iommu)
>> +#define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
>> +#define dev_iommu_group(dev) (dev_archdata(dev)->group)
>> +
>> +/* Xen: Dummy iommu_group */
>> +struct iommu_group
>> +{
>> + /* Streamids of the device */
>> + struct arm_smmu_master_cfg *cfg;
>> +
>> + atomic_t ref;
>> +};
>> +
>> +static struct iommu_group *iommu_group_alloc(void)
>> +{
>> + struct iommu_group *group = xzalloc(struct iommu_group);
>> +
>> + if (!group)
>> + return ERR_PTR(-ENOMEM);
>> -#include <linux/amba/bus.h>
>> + atomic_set(&group->ref, 1);
>> -#include <asm/pgalloc.h>
>> + return group;
>> +}
>> +
>> +static void iommu_group_put(struct iommu_group *group)
>> +{
>> + if (atomic_dec_and_test(&group->ref))
>> + xfree(group);
>> +}
>> +
>> +static void iommu_group_set_iommudata(struct iommu_group *group,
>> + struct arm_smmu_master_cfg *cfg,
>> + void (*releasefn)(void *))
>> +{
>> + /* TODO: Store the releasefn for the PCI */
>> + ASSERT(releasefn == NULL);
>> +
>> + group->cfg = cfg;
>> +}
>> +
>> +static int iommu_group_add_device(struct iommu_group *group,
>> + struct device *dev)
>> +{
>> + dev_iommu_group(dev) = group;
>> +
>> + atomic_inc(&group->ref);
>> +
>> + return 0;
>> +}
>> +
>> +static struct iommu_group *iommu_group_get(struct device *dev)
>> +{
>> + struct iommu_group *group = dev_iommu_group(dev);
>> +
>> + if (group)
>> + atomic_inc(&group->ref);
>> +
>> + return group;
>> +}
>> +
>> +#define iommu_group_get_iommudata(group) (group)->cfg
>> +
>> +/***** Start of Linux SMMU code *****/
>> /* Maximum number of stream IDs assigned to a single device */
>> #define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
>> @@ -397,7 +666,9 @@ struct arm_smmu_cfg {
>> u8 cbndx;
>> u8 irptndx;
>> u32 cbar;
>> - pgd_t *pgd;
>> +
>> + /* Xen: Domain associated to this configuration */
>> + struct domain *domain;
>> };
>> #define INVALID_IRPTNDX 0xff
>> @@ -446,6 +717,7 @@ static void parse_driver_options(struct
>> arm_smmu_device *smmu)
>> static struct device_node *dev_get_dev_node(struct device *dev)
>> {
>> +#if 0 /* Xen: TODO: Add support for PCI */
>> if (dev_is_pci(dev)) {
>> struct pci_bus *bus = to_pci_dev(dev)->bus;
>> @@ -453,6 +725,7 @@ static struct device_node
>> *dev_get_dev_node(struct device *dev)
>> bus = bus->parent;
>> return bus->bridge->parent->of_node;
>> }
>> +#endif
>> return dev->of_node;
>> }
>> @@ -546,6 +819,9 @@ static int register_smmu_master(struct
>> arm_smmu_device *smmu,
>> master->of_node = masterspec->np;
>> master->cfg.num_streamids = masterspec->args_count;
>> + /* Xen: Let Xen knows that the device is protected by an SMMU */
>> + dt_device_set_protected(masterspec->np);
>> +
>> for (i = 0; i < master->cfg.num_streamids; ++i) {
>> u16 streamid = masterspec->args[i];
>> @@ -712,6 +988,24 @@ static irqreturn_t arm_smmu_global_fault(int
>> irq, void *dev)
>> return IRQ_HANDLED;
>> }
>> +/* Xen: Interrupt handlers wrapper */
>> +static void arm_smmu_context_fault_xen(int irq, void *dev,
>> + struct cpu_user_regs *regs)
>> +{
>> + arm_smmu_context_fault(irq, dev);
>> +}
>> +
>> +#define arm_smmu_context_fault arm_smmu_context_fault_xen
>> +
>> +static void arm_smmu_global_fault_xen(int irq, void *dev,
>> + struct cpu_user_regs *regs)
>> +{
>> + arm_smmu_global_fault(irq, dev);
>> +}
>> +
>> +#define arm_smmu_global_fault arm_smmu_global_fault_xen
>> +
>> +#if 0 /* Xen: Page tables are shared with the processor */
>> static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu,
>> void *addr,
>> size_t size)
>> {
>> @@ -733,6 +1027,7 @@ static void arm_smmu_flush_pgtable(struct
>> arm_smmu_device *smmu, void *addr,
>> DMA_TO_DEVICE);
>> }
>> }
>> +#endif
>> static void arm_smmu_init_context_bank(struct arm_smmu_domain
>> *smmu_domain)
>> {
>> @@ -741,6 +1036,7 @@ static void arm_smmu_init_context_bank(struct
>> arm_smmu_domain *smmu_domain)
>> struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
>> struct arm_smmu_device *smmu = smmu_domain->smmu;
>> void __iomem *cb_base, *gr0_base, *gr1_base;
>> + paddr_t p2maddr;
>> gr0_base = ARM_SMMU_GR0(smmu);
>> gr1_base = ARM_SMMU_GR1(smmu);
>> @@ -824,11 +1120,16 @@ static void arm_smmu_init_context_bank(struct
>> arm_smmu_domain *smmu_domain)
>> }
>> /* TTBR0 */
>> - arm_smmu_flush_pgtable(smmu, cfg->pgd,
>> - PTRS_PER_PGD * sizeof(pgd_t));
>> - reg = __pa(cfg->pgd);
>> + /* Xen: The page table is shared with the P2M code */
>> + ASSERT(smmu_domain->cfg.domain != NULL);
>> + p2maddr = page_to_maddr(smmu_domain->cfg.domain->arch.p2m.root);
>> +
>> + dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n",
>> + smmu_domain->cfg.domain->domain_id, p2maddr);
>> +
>> + reg = (p2maddr & ((1ULL << 32) - 1));
>> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
>> - reg = (phys_addr_t)__pa(cfg->pgd) >> 32;
>> + reg = (p2maddr >> 32);
>> if (stage1)
>> reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
>> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
>> @@ -873,6 +1174,9 @@ static void arm_smmu_init_context_bank(struct
>> arm_smmu_domain *smmu_domain)
>> reg = 0;
>> }
>> + /* Xen: The attributes to walk the page table should be the
>> same as
>> + * VTCR_EL2. Currently doesn't differ from Linux ones.
>> + */
>> reg |= TTBCR_EAE |
>> (TTBCR_SH_IS << TTBCR_SH0_SHIFT) |
>> (TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) |
>> @@ -1015,7 +1319,6 @@ static void
>> arm_smmu_destroy_domain_context(struct iommu_domain *domain)
>> static int arm_smmu_domain_init(struct iommu_domain *domain)
>> {
>> struct arm_smmu_domain *smmu_domain;
>> - pgd_t *pgd;
>> /*
>> * Allocate the domain and initialise some of its data structures.
>> @@ -1026,20 +1329,12 @@ static int arm_smmu_domain_init(struct
>> iommu_domain *domain)
>> if (!smmu_domain)
>> return -ENOMEM;
>> - pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL);
>> - if (!pgd)
>> - goto out_free_domain;
>> - smmu_domain->cfg.pgd = pgd;
>> -
>> spin_lock_init(&smmu_domain->lock);
>> domain->priv = smmu_domain;
>> return 0;
>> -
>> -out_free_domain:
>> - kfree(smmu_domain);
>> - return -ENOMEM;
>> }
>> +#if 0 /* Xen: Page tables are shared with the processor */
>> static void arm_smmu_free_ptes(pmd_t *pmd)
>> {
>> pgtable_t table = pmd_pgtable(*pmd);
>> @@ -1102,6 +1397,7 @@ static void arm_smmu_free_pgtables(struct
>> arm_smmu_domain *smmu_domain)
>> kfree(pgd_base);
>> }
>> +#endif
>> static void arm_smmu_domain_destroy(struct iommu_domain *domain)
>> {
>> @@ -1112,7 +1408,6 @@ static void arm_smmu_domain_destroy(struct
>> iommu_domain *domain)
>> * already been detached.
>> */
>> arm_smmu_destroy_domain_context(domain);
>> - arm_smmu_free_pgtables(smmu_domain);
>> kfree(smmu_domain);
>> }
>> @@ -1229,11 +1524,12 @@ static void
>> arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain,
>> /*
>> * We *must* clear the S2CR first, because freeing the SMR means
>> * that it can be re-allocated immediately.
>> + * Xen: Unlike Linux, any access to non-configured stream will
>> fault.
>> */
>> for (i = 0; i < cfg->num_streamids; ++i) {
>> u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i];
>> - writel_relaxed(S2CR_TYPE_BYPASS,
>> + writel_relaxed(S2CR_TYPE_FAULT,
>> gr0_base + ARM_SMMU_GR0_S2CR(idx));
>> }
>> @@ -1253,7 +1549,7 @@ static int arm_smmu_attach_dev(struct
>> iommu_domain *domain, struct device *dev)
>> return -ENXIO;
>> }
>> - if (dev->archdata.iommu) {
>> + if (dev_iommu_domain(dev)) {
>> dev_err(dev, "already attached to IOMMU domain\n");
>> return -EEXIST;
>> }
>> @@ -1285,8 +1581,9 @@ static int arm_smmu_attach_dev(struct
>> iommu_domain *domain, struct device *dev)
>> return -ENODEV;
>> ret = arm_smmu_domain_add_master(smmu_domain, cfg);
>> +
>> if (!ret)
>> - dev->archdata.iommu = domain;
>> + dev_iommu_domain(dev) = domain;
>> return ret;
>> }
>> @@ -1299,10 +1596,14 @@ static void arm_smmu_detach_dev(struct
>> iommu_domain *domain, struct device *dev)
>> if (!cfg)
>> return;
>> - dev->archdata.iommu = NULL;
>> + dev_iommu_domain(dev) = NULL;
>> arm_smmu_domain_remove_master(smmu_domain, cfg);
>> }
>> +#if 0 /*
>> + * Xen: The page table is shared with the processor, therefore
>> + * helpers to implement separate is not necessary.
>> + */
>> static bool arm_smmu_pte_is_contiguous_range(unsigned long addr,
>> unsigned long end)
>> {
>> @@ -1591,7 +1892,9 @@ static phys_addr_t arm_smmu_iova_to_phys(struct
>> iommu_domain *domain,
>> return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK);
>> }
>> +#endif
>> +#if 0 /* Xen: arm_smmu_capable is not used at the moment */
>> static bool arm_smmu_capable(enum iommu_cap cap)
>> {
>> switch (cap) {
>> @@ -1609,6 +1912,7 @@ static bool arm_smmu_capable(enum iommu_cap cap)
>> return false;
>> }
>> }
>> +#endif
>> static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16
>> alias, void *data)
>> {
>> @@ -1676,6 +1980,7 @@ out_put_group:
>> return ret;
>> }
>> +#if 0 /* Xen: We don't support remove device for now. Will be
>> useful for PCI */
>> static void arm_smmu_remove_device(struct device *dev)
>> {
>> iommu_group_remove_device(dev);
>> @@ -1733,6 +2038,7 @@ static const struct iommu_ops arm_smmu_ops = {
>> ARM_SMMU_PTE_CONT_SIZE |
>> PAGE_SIZE),
>> };
>> +#endif
>> static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
>> {
>> @@ -1748,7 +2054,11 @@ static void arm_smmu_device_reset(struct
>> arm_smmu_device *smmu)
>> /* Mark all SMRn as invalid and all S2CRn as bypass */
>> for (i = 0; i < smmu->num_mapping_groups; ++i) {
>> writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i));
>> - writel_relaxed(S2CR_TYPE_BYPASS,
>> + /*
>> + * Xen: Unlike Linux, any access to a non-configure stream
>> + * will fault by default.
>> + */
>> + writel_relaxed(S2CR_TYPE_FAULT,
>> gr0_base + ARM_SMMU_GR0_S2CR(i));
>> }
>> @@ -1774,6 +2084,8 @@ static void arm_smmu_device_reset(struct
>> arm_smmu_device *smmu)
>> /* Enable client access, but bypass when no mapping is found */
>> reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
>> + /* Xen: Unlike Linux, generate a fault when no mapping is found */
>> + reg |= sCR0_USFCFG;
>> /* Disable forced broadcasting */
>> reg &= ~sCR0_FB;
>> @@ -1882,7 +2194,7 @@ static int arm_smmu_device_cfg_probe(struct
>> arm_smmu_device *smmu)
>> }
>> dev_notice(smmu->dev,
>> - "\tstream matching with %u register groups, mask 0x%x",
>> + "\tstream matching with %u register groups, mask
>> 0x%x\n",
>> smmu->num_mapping_groups, mask);
>> } else {
>> smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) &
>> @@ -1917,12 +2229,16 @@ static int arm_smmu_device_cfg_probe(struct
>> arm_smmu_device *smmu)
>> size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) &
>> ID2_IAS_MASK);
>> smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT,
>> size);
>> + /* Xen: Stage-2 input size is not restricted */
>> + smmu->s2_input_size = size;
>> +#if 0
>> /* Stage-2 input size limited due to pgd allocation
>> (PTRS_PER_PGD) */
>> #ifdef CONFIG_64BIT
>> smmu->s2_input_size = min_t(unsigned long, VA_BITS, size);
>> #else
>> smmu->s2_input_size = min(32UL, size);
>> #endif
>> +#endif
>> /* The stage-2 output mask is also applied for bypass */
>> size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) &
>> ID2_OAS_MASK);
>> @@ -1969,6 +2285,10 @@ static const struct of_device_id
>> arm_smmu_of_match[] = {
>> };
>> MODULE_DEVICE_TABLE(of, arm_smmu_of_match);
>> +/*
>> + * Xen: We don't have refcount for allocated memory so manually free
>> memory
>> + * when an error occured.
>> + */
>> static int arm_smmu_device_dt_probe(struct platform_device *pdev)
>> {
>> const struct of_device_id *of_id;
>> @@ -1991,14 +2311,17 @@ static int arm_smmu_device_dt_probe(struct
>> platform_device *pdev)
>> res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
>> smmu->base = devm_ioremap_resource(dev, res);
>> - if (IS_ERR(smmu->base))
>> - return PTR_ERR(smmu->base);
>> + if (IS_ERR(smmu->base)) {
>> + err = PTR_ERR(smmu->base);
>> + goto out_free;
>> + }
>> smmu->size = resource_size(res);
>> if (of_property_read_u32(dev->of_node, "#global-interrupts",
>> &smmu->num_global_irqs)) {
>> dev_err(dev, "missing #global-interrupts property\n");
>> - return -ENODEV;
>> + err = -ENODEV;
>> + goto out_free;
>> }
>> num_irqs = 0;
>> @@ -2011,14 +2334,16 @@ static int arm_smmu_device_dt_probe(struct
>> platform_device *pdev)
>> if (!smmu->num_context_irqs) {
>> dev_err(dev, "found %d interrupts but expected at least %d\n",
>> num_irqs, smmu->num_global_irqs + 1);
>> - return -ENODEV;
>> + err = -ENODEV;
>> + goto out_free;
>> }
>> smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
>> GFP_KERNEL);
>> if (!smmu->irqs) {
>> dev_err(dev, "failed to allocate %d irqs\n", num_irqs);
>> - return -ENOMEM;
>> + err = -ENOMEM;
>> + goto out_free;
>> }
>> for (i = 0; i < num_irqs; ++i) {
>> @@ -2026,7 +2351,8 @@ static int arm_smmu_device_dt_probe(struct
>> platform_device *pdev)
>> if (irq < 0) {
>> dev_err(dev, "failed to get irq index %d\n", i);
>> - return -ENODEV;
>> + err = -ENODEV;
>> + goto out_free;
>> }
>> smmu->irqs[i] = irq;
>> }
>> @@ -2091,12 +2417,19 @@ out_put_masters:
>> for (node = rb_first(&smmu->masters); node; node =
>> rb_next(node)) {
>> struct arm_smmu_master *master
>> = container_of(node, struct arm_smmu_master, node);
>> - of_node_put(master->of_node);
>> + kfree(master);
>> }
>> +out_free:
>> + kfree(smmu->irqs);
>> + if (!IS_ERR(smmu->base))
>> + iounmap(smmu->base);
>> + kfree(smmu);
>> +
>> return err;
>> }
>> +#if 0 /* Xen: We never remove SMMU */
>> static int arm_smmu_device_remove(struct platform_device *pdev)
>> {
>> int i;
>> @@ -2191,3 +2524,241 @@ module_exit(arm_smmu_exit);
>> MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU
>> implementations");
>> MODULE_AUTHOR("Will Deacon <will.deacon@arm.com>");
>> MODULE_LICENSE("GPL v2");
>> +#endif
>> +
>> +/***** Start of Xen specific code *****/
>> +
>> +/* Xen only supports stage-2 translation, so force the value to 2. */
>> +static int force_stage = 2;
>> +
>> +static void arm_smmu_iotlb_flush_all(struct domain *d)
>> +{
>> + struct arm_smmu_xen_domain *smmu_domain =
>> domain_hvm_iommu(d)->arch.priv;
>> + struct iommu_domain *cfg;
>> +
>> + spin_lock(&smmu_domain->lock);
>> + list_for_each_entry(cfg, &smmu_domain->contexts, list) {
>> + /*
>> + * Only invalidate the context when SMMU is present.
>> + * This is because the context initialization is delayed
>> + * until a master has been added.
>> + */
>> + if (unlikely(!ACCESS_ONCE(cfg->priv->smmu)))
>> + continue;
>> + arm_smmu_tlb_inv_context(cfg->priv);
>> + }
>> + spin_unlock(&smmu_domain->lock);
>> +}
>> +
>> +static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn,
>> + unsigned int page_count)
>> +{
>> + /* ARM SMMU v1 doesn't have flush by VMA and VMID */
>> + arm_smmu_iotlb_flush_all(d);
>> +}
>> +
>> +static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
>> + struct device *dev)
>> +{
>> + struct iommu_domain *domain;
>> + struct arm_smmu_xen_domain *xen_domain;
>> + int ret;
>> +
>> + xen_domain = domain_hvm_iommu(d)->arch.priv;
>> +
>> + if (!dev->archdata.iommu) {
>> + dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device);
>> + if (!dev->archdata.iommu)
>> + return -ENOMEM;
>> + }
>> +
>> + if (!dev_iommu_group(dev)) {
>> + ret = arm_smmu_add_device(dev);
>> + if (ret)
>> + return ret;
>> + }
>> +
>> + /*
>> + * TODO: Share the context bank (i.e iommu_domain) when the
>> device is
>> + * under the same SMMU as another device assigned to this domain.
>> + * Would it useful for PCI
>> + */
>> + domain = xzalloc(struct iommu_domain);
>> + if (!domain)
>> + return -ENOMEM;
>> +
>> + ret = arm_smmu_domain_init(domain);
>> + if (ret)
>> + goto err_dom_init;
>> +
>> + domain->priv->cfg.domain = d;
>> +
>> + ret = arm_smmu_attach_dev(domain, dev);
>> + if (ret)
>> + goto err_attach_dev;
>> +
>> + spin_lock(&xen_domain->lock);
>> + /* Chain the new context to the domain */
>> + list_add(&domain->list, &xen_domain->contexts);
>> + spin_unlock(&xen_domain->lock);
>> +
>> + return 0;
>> +
>> +err_attach_dev:
>> + arm_smmu_domain_destroy(domain);
>> +err_dom_init:
>> + xfree(domain);
>> +
>> + return ret;
>> +}
>> +
>> +static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
>> +{
>> + struct iommu_domain *domain = dev_iommu_domain(dev);
>> + struct arm_smmu_xen_domain *xen_domain;
>> +
>> + xen_domain = domain_hvm_iommu(d)->arch.priv;
>> +
>> + if (!domain || domain->priv->cfg.domain != d) {
>> + dev_err(dev, " not attached to domain %d\n", d->domain_id);
>> + return -ESRCH;
>> + }
>> +
>> + arm_smmu_detach_dev(domain, dev);
>> +
>> + spin_lock(&xen_domain->lock);
>> + list_del(&domain->list);
>> + spin_unlock(&xen_domain->lock);
>> +
>> + arm_smmu_domain_destroy(domain);
>> + xfree(domain);
>> +
>> + return 0;
>> +}
>> +
>> +static int arm_smmu_reassign_dev(struct domain *s, struct domain *t,
>> + u8 devfn, struct device *dev)
>> +{
>> + int ret = 0;
>> +
>> + /* Don't allow remapping on other domain than hwdom */
>> + if (t != hardware_domain)
>> + return -EPERM;
>> +
>> + if (t == s)
>> + return 0;
>> +
>> + ret = arm_smmu_deassign_dev(s, dev);
>> + if (ret)
>> + return ret;
>> +
>> + return 0;
>> +}
>> +
>> +static int arm_smmu_iommu_domain_init(struct domain *d)
>> +{
>> + struct arm_smmu_xen_domain *xen_domain;
>> +
>> + xen_domain = xzalloc(struct arm_smmu_xen_domain);
>> + if ( !xen_domain )
>> + return -ENOMEM;
>> +
>> + spin_lock_init(&xen_domain->lock);
>> + INIT_LIST_HEAD(&xen_domain->contexts);
>> +
>> + domain_hvm_iommu(d)->arch.priv = xen_domain;
>> +
>> + return 0;
>> +}
>> +
>> +static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d)
>> +{
>> +}
>> +
>> +static void arm_smmu_iommu_domain_teardown(struct domain *d)
>> +{
>> + struct arm_smmu_xen_domain *xen_domain =
>> domain_hvm_iommu(d)->arch.priv;
>> +
>> + ASSERT(list_empty(&xen_domain->contexts));
>> + xfree(xen_domain);
>> +}
>> +
>> +static int arm_smmu_map_page(struct domain *d, unsigned long gfn,
>> + unsigned long mfn, unsigned int flags)
>> +{
>> + p2m_type_t t;
>> +
>> + /*
>> + * Grant mappings can be used for DMA requests. The dev_bus_addr
>> + * returned by the hypercall is the MFN (not the IPA). For device
>> + * protected by an IOMMU, Xen needs to add a 1:1 mapping in the
>> domain
>> + * p2m to allow DMA request to work.
>> + * This is only valid when the domain is directed mapped. Hence
>> this
>> + * function should only be used by gnttab code with gfn == mfn.
>> + */
>> + BUG_ON(!is_domain_direct_mapped(d));
>> + BUG_ON(mfn != gfn);
>> +
>> + /* We only support readable and writable flags */
>> + if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
>> + return -EINVAL;
>> +
>> + t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw :
>> p2m_iommu_map_ro;
>> +
>> + /*
>> + * The function guest_physmap_add_entry replaces the current
>> mapping
>> + * if there is already one...
>> + */
>> + return guest_physmap_add_entry(d, gfn, mfn, 0, t);
>> +}
>> +
>> +static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn)
>> +{
>> + /*
>> + * This function should only be used by gnttab code when the domain
>> + * is direct mapped
>> + */
>> + if ( !is_domain_direct_mapped(d) )
>> + return -EINVAL;
>> +
>> + guest_physmap_remove_page(d, gfn, gfn, 0);
>> +
>> + return 0;
>> +}
>> +
>> +static const struct iommu_ops arm_smmu_iommu_ops = {
>> + .init = arm_smmu_iommu_domain_init,
>> + .hwdom_init = arm_smmu_iommu_hwdom_init,
>> + .teardown = arm_smmu_iommu_domain_teardown,
>> + .iotlb_flush = arm_smmu_iotlb_flush,
>> + .iotlb_flush_all = arm_smmu_iotlb_flush_all,
>> + .assign_device = arm_smmu_assign_dev,
>> + .reassign_device = arm_smmu_reassign_dev,
>> + .map_page = arm_smmu_map_page,
>> + .unmap_page = arm_smmu_unmap_page,
>> +};
>> +
>> +static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>> + const void *data)
>> +{
>> + int rc;
>> +
>> + /*
>> + * Even if the device can't be initialized, we don't want to
>> + * give the SMMU device to dom0.
>> + */
>> + dt_device_set_used_by(dev, DOMID_XEN);
>> +
>> + rc = arm_smmu_device_dt_probe(dev);
>> + if (rc)
>> + return rc;
>> +
>> + iommu_set_ops(&arm_smmu_iommu_ops);
>> +
>> + return 0;
>> +}
>> +
>> +DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU)
>> + .dt_match = arm_smmu_of_match,
>> + .init = arm_smmu_dt_init,
>> +DT_DEVICE_END
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device
2015-02-25 18:52 ` [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device Julien Grall
@ 2015-03-30 9:51 ` Manish Jaggi
0 siblings, 0 replies; 19+ messages in thread
From: Manish Jaggi @ 2015-03-30 9:51 UTC (permalink / raw)
To: Julien Grall, xen-devel
Cc: Keir Fraser, ian.campbell, Stefano Stabellini, Andrew Cooper,
manish.jaggi, tim, stefano.stabellini, Jan Beulich
On Thursday 26 February 2015 12:22 AM, Julien Grall wrote:
> Currently, Xen is supporting PCI and Platform device (based on Device Tree).
>
> While Xen only supports Platform device on ARM, Xen will gain support of
> PCI soon.
>
> Some drivers, such as IOMMU drivers, may handle PCI and platform device in
> the same way. Only few lines of code differs.
>
> Rather than requesting to provide 2 set of functions (one for PCI and
> one for platform device), introduce a generic structure "device" which
> is embedded in each specialized device.
>
> As x86 only supports PCI, introduce a new type device_t which will be an
> alias to pci_dev for this architecture. It will avoid to add a new field
> for this place.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> CC: Keir Fraser <keir@xen.org>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
>
> ---
> Changes in v4:
> - Add Jan and Stefano's ack
> - Typoes
>
> Changes in v3:
> - Drop common/device.c as it wasn't used after v1
>
> Changes in v2:
> - As x86 will only support PCI, only introduce the generic
> device on ARM
> - Introduce a typedef device_t to be used in common code
> - Drop the PCI code for ARM for now. It will be reintroduced
> when PCI support will be added
> - s#asm/device.h#xen/device.h# is not anymore needed
> ---
> xen/common/device_tree.c | 3 +++
> xen/include/asm-arm/device.h | 26 ++++++++++++++++++++++++++
> xen/include/asm-x86/device.h | 25 +++++++++++++++++++++++++
> xen/include/xen/device_tree.h | 13 +++++++++++++
> xen/include/xen/iommu.h | 1 +
> xen/include/xen/pci.h | 1 +
> 6 files changed, 69 insertions(+)
> create mode 100644 xen/include/asm-x86/device.h
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 34a1b9e..d1c716f 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -1454,6 +1454,9 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
> ((char *)pp->value)[sz - 1] = 0;
> dt_dprintk("fixed up name for %s -> %s\n", pathp,
> (char *)pp->value);
> + /* Generic device initialization */
> + np->dev.type = DEV_DT;
> + np->dev.of_node = np;
> }
> }
> if ( allnextpp )
> diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h
> index b6b32bc..1cedd49 100644
> --- a/xen/include/asm-arm/device.h
> +++ b/xen/include/asm-arm/device.h
> @@ -2,8 +2,34 @@
> #define __ASM_ARM_DEVICE_H
>
> #include <xen/init.h>
> +
> +enum device_type
> +{
> + DEV_DT,
> +};
> +
> +struct dev_archdata {
> + void *iommu; /* IOMMU private data */
> +};
> +
> +/* struct device - The basic device structure */
> +struct device
> +{
> + enum device_type type;
> +#ifdef HAS_DEVICE_TREE
> + struct dt_device_node *of_node; /* Used by drivers imported from Linux */
> +#endif
> + struct dev_archdata archdata;
> +};
> +
> +typedef struct device device_t;
> +
> #include <xen/device_tree.h>
>
> +/* TODO: Correctly implement dev_is_pci when PCI is supported on ARM */
> +#define dev_is_pci(dev) ((void)(dev), 0)
> +#define dev_is_dt(dev) ((dev->type == DEV_DT)
> +
> enum device_class
> {
> DEVICE_SERIAL,
> diff --git a/xen/include/asm-x86/device.h b/xen/include/asm-x86/device.h
> new file mode 100644
> index 0000000..f2acc7e
> --- /dev/null
> +++ b/xen/include/asm-x86/device.h
> @@ -0,0 +1,25 @@
> +#ifndef __ASM_X86_DEVICE_H
> +#define __ASM_X86_DEVICE_H
> +
> +#include <xen/pci.h>
> +
> +/*
> + * x86 only supports PCI. Therefore it's possible to directly use
> + * pci_dev to avoid adding new field.
> + */
> +
> +typedef struct pci_dev device_t;
> +
> +#define dev_is_pci(dev) ((void)(dev), 1)
> +#define pci_to_dev(pci) (pci)
When HAS_PCI and HAS_PASSTHROUGH is enabled for ARM, pci_to_dev is
undefined for ARM and compilation fails.
How do you propose to fix this?
Should I add a device as a member in pci_dev, as you mentioned in the
commit "... embedded in each specialized device"
> +
> +#endif /* __ASM_X86_DEVICE_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 6502369..c8a0375 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -11,7 +11,9 @@
> #define __XEN_DEVICE_TREE_H__
>
> #include <asm/byteorder.h>
> +#include <asm/device.h>
> #include <public/xen.h>
> +#include <xen/kernel.h>
> #include <xen/init.h>
> #include <xen/string.h>
> #include <xen/types.h>
> @@ -80,8 +82,19 @@ struct dt_device_node {
> /* IOMMU specific fields */
> bool is_protected;
> struct list_head domain_list;
> +
> + struct device dev;
> };
>
> +#define dt_to_dev(dt_node) (&(dt_node)->dev)
> +
> +static inline struct dt_device_node *dev_to_dt(struct device *dev)
> +{
> + ASSERT(dev->type == DEV_DT);
> +
> + return container_of(dev, struct dt_device_node, dev);
> +}
> +
> #define MAX_PHANDLE_ARGS 16
> struct dt_phandle_args {
> struct dt_device_node *np;
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index 8eb764a..ecb2627 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -25,6 +25,7 @@
> #include <xen/pci.h>
> #include <public/hvm/ioreq.h>
> #include <public/domctl.h>
> +#include <asm/device.h>
> #include <asm/iommu.h>
>
> extern bool_t iommu_enable, iommu_enabled;
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index 5f295f3..3988ee68 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -13,6 +13,7 @@
> #include <xen/irq.h>
> #include <xen/pci_regs.h>
> #include <xen/pfn.h>
> +#include <asm/device.h>
> #include <asm/pci.h>
>
> /*
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2015-03-30 9:52 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-02-25 18:52 [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Julien Grall
2015-02-25 18:52 ` [PATCH v4 1/8] xen/iommu: arm: Remove temporary the SMMU driver Julien Grall
2015-02-25 18:52 ` [PATCH v4 2/8] xen/arm: Introduce a generic way to describe device Julien Grall
2015-03-30 9:51 ` Manish Jaggi
2015-02-25 18:52 ` [PATCH v4 3/8] xen/iommu: Consolidate device assignment ops into a single set Julien Grall
2015-02-25 18:52 ` [PATCH v4 4/8] xen/arm: Describe devices supported by a driver with dt_device_match Julien Grall
2015-02-25 18:52 ` [PATCH v4 5/8] xen/iommu: arm: Import the SMMU driver from Linux Julien Grall
2015-02-25 18:52 ` [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Julien Grall
2015-03-02 7:03 ` Manish Jaggi
2015-03-04 2:08 ` Manish Jaggi
2015-03-02 13:17 ` Ian Campbell
2015-03-02 13:45 ` Julien Grall
2015-03-02 13:54 ` Ian Campbell
2015-02-25 18:53 ` [PATCH v4 7/8] xen/iommu: smmu: Advertise when the SMMU support coherent table walk Julien Grall
2015-03-02 13:10 ` Ian Campbell
2015-03-02 15:23 ` Julien Grall
2015-03-02 16:24 ` Ian Campbell
2015-02-25 18:53 ` [PATCH v4 8/8] DO NOT APPLY xen/iommu: smmu: Changes to support Midway SMMU Julien Grall
2015-03-02 14:47 ` [PATCH v4 0/8] xen/arm: Resync the SMMU driver with the Linux one Ian Campbell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.