From mboxrd@z Thu Jan 1 00:00:00 1970 From: Manish Jaggi Subject: Re: [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver Date: Wed, 4 Mar 2015 07:38:24 +0530 Message-ID: <54F66918.9020404@caviumnetworks.com> References: <1424890381-4225-1-git-send-email-julien.grall@linaro.org> <1424890381-4225-7-git-send-email-julien.grall@linaro.org> <54F40B53.4000108@caviumnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YSykU-0002xY-Kq for xen-devel@lists.xenproject.org; Wed, 04 Mar 2015 02:09:22 +0000 In-Reply-To: <54F40B53.4000108@caviumnetworks.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Julien Grall , xen-devel@lists.xenproject.org Cc: stefano.stabellini@citrix.com, manish.jaggi@caviumnetworks.com, tim@xen.org, ian.campbell@citrix.com List-Id: xen-devel@lists.xenproject.org On 02/03/15 12:33 pm, Manish Jaggi wrote: > > On Thursday 26 February 2015 12:22 AM, Julien Grall wrote: >> The main goal is to modify as little the Linux code to be able to port >> easily new feature added in Linux repo for the driver. >> >> To achieve that we: >> - Add helpers to Linux function not implemented on Xen >> - Add callbacks used by Xen to do our own stuff and call Linux ones >> - Only modify when required the code which comes from Linux. If >> so a >> comment has been added with /* Xen: ... */ explaining why it's >> necessary. >> >> The support for PCI has been commented because it's not yet supported by >> Xen ARM and therefore won't compile. >> >> Signed-off-by: Julien Grall >> >> --- >> Changes in v4: >> - Re-shuffle the code to rationalize the number of lines >> changed >> in the Linux SMMU code >> - Use #if 0 /* Xen: ... */ rather than using 2 lines >> - Add a bunch of definitions to avoid some #if 0 >> - Remove the "hack" for the Midway SMMU >> >> Changes in v2: >> - Add the ACCESS_ONCE definition in the drivers. The patch to >> introduce the one in common code has been dropped. >> - The include xen/device.h has been dropped in favor of >> asm/device.h >> --- >> xen/drivers/passthrough/arm/Makefile | 1 + >> xen/drivers/passthrough/arm/smmu.c | 661 >> ++++++++++++++++++++++++++++++++--- >> 2 files changed, 617 insertions(+), 45 deletions(-) > Hi Julien, > Few comments (first level) > a) What is the difference between assign / attach / add. There are > different apis and I get lost reading the code between them. > Either provide a description or use a better nomenclature. > > b) In which case a xen domain be having multiple contexts. Even though > it is technically possible but there is no defined method to create a > different context in non-pci or pci-passthrough. > Could you please add some detail. > Julien, could you please address these points > -mj >> >> diff --git a/xen/drivers/passthrough/arm/Makefile >> b/xen/drivers/passthrough/arm/Makefile >> index 0484b79..f4cd26e 100644 >> --- a/xen/drivers/passthrough/arm/Makefile >> +++ b/xen/drivers/passthrough/arm/Makefile >> @@ -1 +1,2 @@ >> obj-y += iommu.o >> +obj-y += smmu.o >> diff --git a/xen/drivers/passthrough/arm/smmu.c >> b/xen/drivers/passthrough/arm/smmu.c >> index 6cd47b7..d01a26a 100644 >> --- a/xen/drivers/passthrough/arm/smmu.c >> +++ b/xen/drivers/passthrough/arm/smmu.c >> @@ -18,6 +18,13 @@ >> * >> * Author: Will Deacon >> * >> + * Based on Linux drivers/iommu/arm-smmu.c >> + * => commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3 >> + * >> + * Xen modification: >> + * Julien Grall >> + * Copyright (C) 2014 Linaro Limited. >> + * >> * This driver currently supports: >> * - SMMUv1 and v2 implementations >> * - Stream-matching and stream-indexing >> @@ -28,25 +35,287 @@ >> * - Context fault reporting >> */ >> -#define pr_fmt(fmt) "arm-smmu: " fmt >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> -#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> + >> +/* Xen: The below defines are redefined within the file. Undef it */ >> +#undef SCTLR_AFE >> +#undef SCTLR_TRE >> +#undef SCTLR_M >> +#undef TTBCR_EAE >> + >> +/* Alias to Xen device tree helpers */ >> +#define device_node dt_device_node >> +#define of_phandle_args dt_phandle_args >> +#define of_device_id dt_device_match >> +#define of_match_node dt_match_node >> +#define of_property_read_u32(np, pname, out) >> (!dt_property_read_u32(np, pname, out)) >> +#define of_property_read_bool dt_property_read_bool >> +#define of_parse_phandle_with_args dt_parse_phandle_with_args >> + >> +/* Xen: Helpers to get device MMIO and IRQs */ >> +struct resource >> +{ >> + u64 addr; >> + u64 size; >> + unsigned int type; >> +}; >> + >> +#define resource_size(res) (res)->size; >> + >> +#define platform_device dt_device_node >> + >> +#define IORESOURCE_MEM 0 >> +#define IORESOURCE_IRQ 1 >> + >> +static struct resource *platform_get_resource(struct platform_device >> *pdev, >> + unsigned int type, >> + unsigned int num) >> +{ >> + /* >> + * The resource is only used between 2 calls of >> platform_get_resource. >> + * It's quite ugly but it's avoid to add too much code in the part >> + * imported from Linux >> + */ >> + static struct resource res; >> + int ret = 0; >> + >> + res.type = type; >> + >> + switch (type) { >> + case IORESOURCE_MEM: >> + ret = dt_device_get_address(pdev, num, &res.addr, &res.size); >> + >> + return ((ret) ? NULL : &res); >> + >> + case IORESOURCE_IRQ: >> + ret = platform_get_irq(pdev, num); >> + if (ret < 0) >> + return NULL; >> + >> + res.addr = ret; >> + res.size = 1; >> + >> + return &res; >> + >> + default: >> + return NULL; >> + } >> +} >> + >> +/* Xen: Helpers for IRQ functions */ >> +#define request_irq(irq, func, flags, name, dev) request_irq(irq, >> flags, func, name, dev) >> +#define free_irq release_irq >> + >> +enum irqreturn { >> + IRQ_NONE = (0 << 0), >> + IRQ_HANDLED = (1 << 0), >> +}; >> + >> +typedef enum irqreturn irqreturn_t; >> + >> +/* Device logger functions >> + * TODO: Handle PCI >> + */ >> +#define dev_print(dev, lvl, fmt, ...) \ >> + printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), >> ## __VA_ARGS__) >> + >> +#define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## >> __VA_ARGS__) >> +#define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, >> ## __VA_ARGS__) >> +#define dev_warn(dev, fmt, ...) dev_print(dev, XENLOG_WARNING, fmt, >> ## __VA_ARGS__) >> +#define dev_err(dev, fmt, ...) dev_print(dev, XENLOG_ERR, fmt, ## >> __VA_ARGS__) >> + >> +#define dev_err_ratelimited(dev, fmt, ...) \ >> + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__) >> + >> +#define dev_name(dev) dt_node_full_name(dev_to_dt(dev)) >> + >> +/* Alias to Xen allocation helpers */ >> +#define kfree xfree >> +#define kmalloc(size, flags) _xmalloc(size, sizeof(void *)) >> +#define kzalloc(size, flags) _xzalloc(size, sizeof(void *)) >> +#define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void >> *)) >> +#define kmalloc_array(size, n, flags) _xmalloc_array(size, >> sizeof(void *), n) >> + >> +static void __iomem *devm_ioremap_resource(struct device *dev, >> + struct resource *res) >> +{ >> + void __iomem *ptr; >> + >> + if (!res || res->type != IORESOURCE_MEM) { >> + dev_err(dev, "Invalid resource\n"); >> + return ERR_PTR(-EINVAL); >> + } >> + >> + ptr = ioremap_nocache(res->addr, res->size); >> + if (!ptr) { >> + dev_err(dev, >> + "ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n", >> + res->addr, res->size); >> + return ERR_PTR(-ENOMEM); >> + } >> + >> + return ptr; >> +} >> + >> +/* Xen doesn't handle IOMMU fault */ >> +#define report_iommu_fault(...) 1 >> + >> +#define IOMMU_FAULT_READ 0 >> +#define IOMMU_FAULT_WRITE 1 >> + >> +/* >> + * Xen: PCI functions >> + * TODO: It should be implemented when PCI will be supported >> + */ >> +#define to_pci_dev(dev) (NULL) >> +static inline int pci_for_each_dma_alias(struct pci_dev *pdev, >> + int (*fn) (struct pci_dev *pdev, >> + u16 alias, void *data), >> + void *data) >> +{ >> + BUG(); >> + return 0; >> +} >> + >> +/* Xen: misc */ >> +#define PHYS_MASK_SHIFT PADDR_BITS >> +typedef paddr_t phys_addr_t; >> + >> +#ifdef CONFIG_ARM_64 >> +# define CONFIG_64BIT >> +#endif >> + >> +#define VA_BITS 0 /* Only used for configuring stage-1 >> input size */ >> + >> +/* The macro ACCESS_ONCE start to be replaced in Linux in favor of >> + * {READ, WRITE}_ONCE. Rather than introducing in the common code, >> keep a >> + * version here. We will have to drop it when the SMMU code in Linux >> will >> + * switch to {READ, WRITE}_ONCE. >> + */ >> +#define __ACCESS_ONCE(x) ({ \ >> + __maybe_unused typeof(x) __var = 0; \ >> + (volatile typeof(x) *)&(x); }) >> +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x)) >> + >> +#define MODULE_DEVICE_TABLE(type, name) >> +#define module_param_named(name, value, type, perm) >> +#define MODULE_PARM_DESC(_parm, desc) >> + >> +/* Xen: Dummy iommu_domain */ >> +struct iommu_domain >> +{ >> + /* Runtime SMMU configuration for this iommu_domain */ >> + struct arm_smmu_domain *priv; >> + >> + /* Used to link iommu_domain contexts for a same domain. >> + * There is at least one per-SMMU to used by the domain. >> + * */ >> + struct list_head list; >> +}; >> + >> +/* Xen: Describes informations required for a Xen domain */ >> +struct arm_smmu_xen_domain { >> + spinlock_t lock; >> + /* List of context (i.e iommu_domain) associated to this domain */ >> + struct list_head contexts; >> +}; >> + >> +/* >> + * Xen: Information about each device stored in dev->archdata.iommu >> + * >> + * Initially dev->archdata.iommu only stores the iommu_domain (runtime >> + * configuration of the SMMU) but, on Xen, we also have to store the >> + * iommu_group (list of streamIDs associated to the device). >> + * >> + * This is because Linux has a field iommu_group in the struct >> device. On Xen, >> + * that would require to move so hackery (dummy iommu_group) in a >> more generic >> + * place. >> + * */ >> +struct arm_smmu_xen_device { >> + struct iommu_domain *domain; >> + struct iommu_group *group; >> +}; >> + >> +#define dev_archdata(dev) ((struct arm_smmu_xen_device >> *)dev->archdata.iommu) >> +#define dev_iommu_domain(dev) (dev_archdata(dev)->domain) >> +#define dev_iommu_group(dev) (dev_archdata(dev)->group) >> + >> +/* Xen: Dummy iommu_group */ >> +struct iommu_group >> +{ >> + /* Streamids of the device */ >> + struct arm_smmu_master_cfg *cfg; >> + >> + atomic_t ref; >> +}; >> + >> +static struct iommu_group *iommu_group_alloc(void) >> +{ >> + struct iommu_group *group = xzalloc(struct iommu_group); >> + >> + if (!group) >> + return ERR_PTR(-ENOMEM); >> -#include >> + atomic_set(&group->ref, 1); >> -#include >> + return group; >> +} >> + >> +static void iommu_group_put(struct iommu_group *group) >> +{ >> + if (atomic_dec_and_test(&group->ref)) >> + xfree(group); >> +} >> + >> +static void iommu_group_set_iommudata(struct iommu_group *group, >> + struct arm_smmu_master_cfg *cfg, >> + void (*releasefn)(void *)) >> +{ >> + /* TODO: Store the releasefn for the PCI */ >> + ASSERT(releasefn == NULL); >> + >> + group->cfg = cfg; >> +} >> + >> +static int iommu_group_add_device(struct iommu_group *group, >> + struct device *dev) >> +{ >> + dev_iommu_group(dev) = group; >> + >> + atomic_inc(&group->ref); >> + >> + return 0; >> +} >> + >> +static struct iommu_group *iommu_group_get(struct device *dev) >> +{ >> + struct iommu_group *group = dev_iommu_group(dev); >> + >> + if (group) >> + atomic_inc(&group->ref); >> + >> + return group; >> +} >> + >> +#define iommu_group_get_iommudata(group) (group)->cfg >> + >> +/***** Start of Linux SMMU code *****/ >> /* Maximum number of stream IDs assigned to a single device */ >> #define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS >> @@ -397,7 +666,9 @@ struct arm_smmu_cfg { >> u8 cbndx; >> u8 irptndx; >> u32 cbar; >> - pgd_t *pgd; >> + >> + /* Xen: Domain associated to this configuration */ >> + struct domain *domain; >> }; >> #define INVALID_IRPTNDX 0xff >> @@ -446,6 +717,7 @@ static void parse_driver_options(struct >> arm_smmu_device *smmu) >> static struct device_node *dev_get_dev_node(struct device *dev) >> { >> +#if 0 /* Xen: TODO: Add support for PCI */ >> if (dev_is_pci(dev)) { >> struct pci_bus *bus = to_pci_dev(dev)->bus; >> @@ -453,6 +725,7 @@ static struct device_node >> *dev_get_dev_node(struct device *dev) >> bus = bus->parent; >> return bus->bridge->parent->of_node; >> } >> +#endif >> return dev->of_node; >> } >> @@ -546,6 +819,9 @@ static int register_smmu_master(struct >> arm_smmu_device *smmu, >> master->of_node = masterspec->np; >> master->cfg.num_streamids = masterspec->args_count; >> + /* Xen: Let Xen knows that the device is protected by an SMMU */ >> + dt_device_set_protected(masterspec->np); >> + >> for (i = 0; i < master->cfg.num_streamids; ++i) { >> u16 streamid = masterspec->args[i]; >> @@ -712,6 +988,24 @@ static irqreturn_t arm_smmu_global_fault(int >> irq, void *dev) >> return IRQ_HANDLED; >> } >> +/* Xen: Interrupt handlers wrapper */ >> +static void arm_smmu_context_fault_xen(int irq, void *dev, >> + struct cpu_user_regs *regs) >> +{ >> + arm_smmu_context_fault(irq, dev); >> +} >> + >> +#define arm_smmu_context_fault arm_smmu_context_fault_xen >> + >> +static void arm_smmu_global_fault_xen(int irq, void *dev, >> + struct cpu_user_regs *regs) >> +{ >> + arm_smmu_global_fault(irq, dev); >> +} >> + >> +#define arm_smmu_global_fault arm_smmu_global_fault_xen >> + >> +#if 0 /* Xen: Page tables are shared with the processor */ >> static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, >> void *addr, >> size_t size) >> { >> @@ -733,6 +1027,7 @@ static void arm_smmu_flush_pgtable(struct >> arm_smmu_device *smmu, void *addr, >> DMA_TO_DEVICE); >> } >> } >> +#endif >> static void arm_smmu_init_context_bank(struct arm_smmu_domain >> *smmu_domain) >> { >> @@ -741,6 +1036,7 @@ static void arm_smmu_init_context_bank(struct >> arm_smmu_domain *smmu_domain) >> struct arm_smmu_cfg *cfg = &smmu_domain->cfg; >> struct arm_smmu_device *smmu = smmu_domain->smmu; >> void __iomem *cb_base, *gr0_base, *gr1_base; >> + paddr_t p2maddr; >> gr0_base = ARM_SMMU_GR0(smmu); >> gr1_base = ARM_SMMU_GR1(smmu); >> @@ -824,11 +1120,16 @@ static void arm_smmu_init_context_bank(struct >> arm_smmu_domain *smmu_domain) >> } >> /* TTBR0 */ >> - arm_smmu_flush_pgtable(smmu, cfg->pgd, >> - PTRS_PER_PGD * sizeof(pgd_t)); >> - reg = __pa(cfg->pgd); >> + /* Xen: The page table is shared with the P2M code */ >> + ASSERT(smmu_domain->cfg.domain != NULL); >> + p2maddr = page_to_maddr(smmu_domain->cfg.domain->arch.p2m.root); >> + >> + dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n", >> + smmu_domain->cfg.domain->domain_id, p2maddr); >> + >> + reg = (p2maddr & ((1ULL << 32) - 1)); >> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO); >> - reg = (phys_addr_t)__pa(cfg->pgd) >> 32; >> + reg = (p2maddr >> 32); >> if (stage1) >> reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT; >> writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI); >> @@ -873,6 +1174,9 @@ static void arm_smmu_init_context_bank(struct >> arm_smmu_domain *smmu_domain) >> reg = 0; >> } >> + /* Xen: The attributes to walk the page table should be the >> same as >> + * VTCR_EL2. Currently doesn't differ from Linux ones. >> + */ >> reg |= TTBCR_EAE | >> (TTBCR_SH_IS << TTBCR_SH0_SHIFT) | >> (TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) | >> @@ -1015,7 +1319,6 @@ static void >> arm_smmu_destroy_domain_context(struct iommu_domain *domain) >> static int arm_smmu_domain_init(struct iommu_domain *domain) >> { >> struct arm_smmu_domain *smmu_domain; >> - pgd_t *pgd; >> /* >> * Allocate the domain and initialise some of its data structures. >> @@ -1026,20 +1329,12 @@ static int arm_smmu_domain_init(struct >> iommu_domain *domain) >> if (!smmu_domain) >> return -ENOMEM; >> - pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL); >> - if (!pgd) >> - goto out_free_domain; >> - smmu_domain->cfg.pgd = pgd; >> - >> spin_lock_init(&smmu_domain->lock); >> domain->priv = smmu_domain; >> return 0; >> - >> -out_free_domain: >> - kfree(smmu_domain); >> - return -ENOMEM; >> } >> +#if 0 /* Xen: Page tables are shared with the processor */ >> static void arm_smmu_free_ptes(pmd_t *pmd) >> { >> pgtable_t table = pmd_pgtable(*pmd); >> @@ -1102,6 +1397,7 @@ static void arm_smmu_free_pgtables(struct >> arm_smmu_domain *smmu_domain) >> kfree(pgd_base); >> } >> +#endif >> static void arm_smmu_domain_destroy(struct iommu_domain *domain) >> { >> @@ -1112,7 +1408,6 @@ static void arm_smmu_domain_destroy(struct >> iommu_domain *domain) >> * already been detached. >> */ >> arm_smmu_destroy_domain_context(domain); >> - arm_smmu_free_pgtables(smmu_domain); >> kfree(smmu_domain); >> } >> @@ -1229,11 +1524,12 @@ static void >> arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain, >> /* >> * We *must* clear the S2CR first, because freeing the SMR means >> * that it can be re-allocated immediately. >> + * Xen: Unlike Linux, any access to non-configured stream will >> fault. >> */ >> for (i = 0; i < cfg->num_streamids; ++i) { >> u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i]; >> - writel_relaxed(S2CR_TYPE_BYPASS, >> + writel_relaxed(S2CR_TYPE_FAULT, >> gr0_base + ARM_SMMU_GR0_S2CR(idx)); >> } >> @@ -1253,7 +1549,7 @@ static int arm_smmu_attach_dev(struct >> iommu_domain *domain, struct device *dev) >> return -ENXIO; >> } >> - if (dev->archdata.iommu) { >> + if (dev_iommu_domain(dev)) { >> dev_err(dev, "already attached to IOMMU domain\n"); >> return -EEXIST; >> } >> @@ -1285,8 +1581,9 @@ static int arm_smmu_attach_dev(struct >> iommu_domain *domain, struct device *dev) >> return -ENODEV; >> ret = arm_smmu_domain_add_master(smmu_domain, cfg); >> + >> if (!ret) >> - dev->archdata.iommu = domain; >> + dev_iommu_domain(dev) = domain; >> return ret; >> } >> @@ -1299,10 +1596,14 @@ static void arm_smmu_detach_dev(struct >> iommu_domain *domain, struct device *dev) >> if (!cfg) >> return; >> - dev->archdata.iommu = NULL; >> + dev_iommu_domain(dev) = NULL; >> arm_smmu_domain_remove_master(smmu_domain, cfg); >> } >> +#if 0 /* >> + * Xen: The page table is shared with the processor, therefore >> + * helpers to implement separate is not necessary. >> + */ >> static bool arm_smmu_pte_is_contiguous_range(unsigned long addr, >> unsigned long end) >> { >> @@ -1591,7 +1892,9 @@ static phys_addr_t arm_smmu_iova_to_phys(struct >> iommu_domain *domain, >> return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK); >> } >> +#endif >> +#if 0 /* Xen: arm_smmu_capable is not used at the moment */ >> static bool arm_smmu_capable(enum iommu_cap cap) >> { >> switch (cap) { >> @@ -1609,6 +1912,7 @@ static bool arm_smmu_capable(enum iommu_cap cap) >> return false; >> } >> } >> +#endif >> static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 >> alias, void *data) >> { >> @@ -1676,6 +1980,7 @@ out_put_group: >> return ret; >> } >> +#if 0 /* Xen: We don't support remove device for now. Will be >> useful for PCI */ >> static void arm_smmu_remove_device(struct device *dev) >> { >> iommu_group_remove_device(dev); >> @@ -1733,6 +2038,7 @@ static const struct iommu_ops arm_smmu_ops = { >> ARM_SMMU_PTE_CONT_SIZE | >> PAGE_SIZE), >> }; >> +#endif >> static void arm_smmu_device_reset(struct arm_smmu_device *smmu) >> { >> @@ -1748,7 +2054,11 @@ static void arm_smmu_device_reset(struct >> arm_smmu_device *smmu) >> /* Mark all SMRn as invalid and all S2CRn as bypass */ >> for (i = 0; i < smmu->num_mapping_groups; ++i) { >> writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i)); >> - writel_relaxed(S2CR_TYPE_BYPASS, >> + /* >> + * Xen: Unlike Linux, any access to a non-configure stream >> + * will fault by default. >> + */ >> + writel_relaxed(S2CR_TYPE_FAULT, >> gr0_base + ARM_SMMU_GR0_S2CR(i)); >> } >> @@ -1774,6 +2084,8 @@ static void arm_smmu_device_reset(struct >> arm_smmu_device *smmu) >> /* Enable client access, but bypass when no mapping is found */ >> reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG); >> + /* Xen: Unlike Linux, generate a fault when no mapping is found */ >> + reg |= sCR0_USFCFG; >> /* Disable forced broadcasting */ >> reg &= ~sCR0_FB; >> @@ -1882,7 +2194,7 @@ static int arm_smmu_device_cfg_probe(struct >> arm_smmu_device *smmu) >> } >> dev_notice(smmu->dev, >> - "\tstream matching with %u register groups, mask 0x%x", >> + "\tstream matching with %u register groups, mask >> 0x%x\n", >> smmu->num_mapping_groups, mask); >> } else { >> smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) & >> @@ -1917,12 +2229,16 @@ static int arm_smmu_device_cfg_probe(struct >> arm_smmu_device *smmu) >> size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & >> ID2_IAS_MASK); >> smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, >> size); >> + /* Xen: Stage-2 input size is not restricted */ >> + smmu->s2_input_size = size; >> +#if 0 >> /* Stage-2 input size limited due to pgd allocation >> (PTRS_PER_PGD) */ >> #ifdef CONFIG_64BIT >> smmu->s2_input_size = min_t(unsigned long, VA_BITS, size); >> #else >> smmu->s2_input_size = min(32UL, size); >> #endif >> +#endif >> /* The stage-2 output mask is also applied for bypass */ >> size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & >> ID2_OAS_MASK); >> @@ -1969,6 +2285,10 @@ static const struct of_device_id >> arm_smmu_of_match[] = { >> }; >> MODULE_DEVICE_TABLE(of, arm_smmu_of_match); >> +/* >> + * Xen: We don't have refcount for allocated memory so manually free >> memory >> + * when an error occured. >> + */ >> static int arm_smmu_device_dt_probe(struct platform_device *pdev) >> { >> const struct of_device_id *of_id; >> @@ -1991,14 +2311,17 @@ static int arm_smmu_device_dt_probe(struct >> platform_device *pdev) >> res = platform_get_resource(pdev, IORESOURCE_MEM, 0); >> smmu->base = devm_ioremap_resource(dev, res); >> - if (IS_ERR(smmu->base)) >> - return PTR_ERR(smmu->base); >> + if (IS_ERR(smmu->base)) { >> + err = PTR_ERR(smmu->base); >> + goto out_free; >> + } >> smmu->size = resource_size(res); >> if (of_property_read_u32(dev->of_node, "#global-interrupts", >> &smmu->num_global_irqs)) { >> dev_err(dev, "missing #global-interrupts property\n"); >> - return -ENODEV; >> + err = -ENODEV; >> + goto out_free; >> } >> num_irqs = 0; >> @@ -2011,14 +2334,16 @@ static int arm_smmu_device_dt_probe(struct >> platform_device *pdev) >> if (!smmu->num_context_irqs) { >> dev_err(dev, "found %d interrupts but expected at least %d\n", >> num_irqs, smmu->num_global_irqs + 1); >> - return -ENODEV; >> + err = -ENODEV; >> + goto out_free; >> } >> smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs, >> GFP_KERNEL); >> if (!smmu->irqs) { >> dev_err(dev, "failed to allocate %d irqs\n", num_irqs); >> - return -ENOMEM; >> + err = -ENOMEM; >> + goto out_free; >> } >> for (i = 0; i < num_irqs; ++i) { >> @@ -2026,7 +2351,8 @@ static int arm_smmu_device_dt_probe(struct >> platform_device *pdev) >> if (irq < 0) { >> dev_err(dev, "failed to get irq index %d\n", i); >> - return -ENODEV; >> + err = -ENODEV; >> + goto out_free; >> } >> smmu->irqs[i] = irq; >> } >> @@ -2091,12 +2417,19 @@ out_put_masters: >> for (node = rb_first(&smmu->masters); node; node = >> rb_next(node)) { >> struct arm_smmu_master *master >> = container_of(node, struct arm_smmu_master, node); >> - of_node_put(master->of_node); >> + kfree(master); >> } >> +out_free: >> + kfree(smmu->irqs); >> + if (!IS_ERR(smmu->base)) >> + iounmap(smmu->base); >> + kfree(smmu); >> + >> return err; >> } >> +#if 0 /* Xen: We never remove SMMU */ >> static int arm_smmu_device_remove(struct platform_device *pdev) >> { >> int i; >> @@ -2191,3 +2524,241 @@ module_exit(arm_smmu_exit); >> MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU >> implementations"); >> MODULE_AUTHOR("Will Deacon "); >> MODULE_LICENSE("GPL v2"); >> +#endif >> + >> +/***** Start of Xen specific code *****/ >> + >> +/* Xen only supports stage-2 translation, so force the value to 2. */ >> +static int force_stage = 2; >> + >> +static void arm_smmu_iotlb_flush_all(struct domain *d) >> +{ >> + struct arm_smmu_xen_domain *smmu_domain = >> domain_hvm_iommu(d)->arch.priv; >> + struct iommu_domain *cfg; >> + >> + spin_lock(&smmu_domain->lock); >> + list_for_each_entry(cfg, &smmu_domain->contexts, list) { >> + /* >> + * Only invalidate the context when SMMU is present. >> + * This is because the context initialization is delayed >> + * until a master has been added. >> + */ >> + if (unlikely(!ACCESS_ONCE(cfg->priv->smmu))) >> + continue; >> + arm_smmu_tlb_inv_context(cfg->priv); >> + } >> + spin_unlock(&smmu_domain->lock); >> +} >> + >> +static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn, >> + unsigned int page_count) >> +{ >> + /* ARM SMMU v1 doesn't have flush by VMA and VMID */ >> + arm_smmu_iotlb_flush_all(d); >> +} >> + >> +static int arm_smmu_assign_dev(struct domain *d, u8 devfn, >> + struct device *dev) >> +{ >> + struct iommu_domain *domain; >> + struct arm_smmu_xen_domain *xen_domain; >> + int ret; >> + >> + xen_domain = domain_hvm_iommu(d)->arch.priv; >> + >> + if (!dev->archdata.iommu) { >> + dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device); >> + if (!dev->archdata.iommu) >> + return -ENOMEM; >> + } >> + >> + if (!dev_iommu_group(dev)) { >> + ret = arm_smmu_add_device(dev); >> + if (ret) >> + return ret; >> + } >> + >> + /* >> + * TODO: Share the context bank (i.e iommu_domain) when the >> device is >> + * under the same SMMU as another device assigned to this domain. >> + * Would it useful for PCI >> + */ >> + domain = xzalloc(struct iommu_domain); >> + if (!domain) >> + return -ENOMEM; >> + >> + ret = arm_smmu_domain_init(domain); >> + if (ret) >> + goto err_dom_init; >> + >> + domain->priv->cfg.domain = d; >> + >> + ret = arm_smmu_attach_dev(domain, dev); >> + if (ret) >> + goto err_attach_dev; >> + >> + spin_lock(&xen_domain->lock); >> + /* Chain the new context to the domain */ >> + list_add(&domain->list, &xen_domain->contexts); >> + spin_unlock(&xen_domain->lock); >> + >> + return 0; >> + >> +err_attach_dev: >> + arm_smmu_domain_destroy(domain); >> +err_dom_init: >> + xfree(domain); >> + >> + return ret; >> +} >> + >> +static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) >> +{ >> + struct iommu_domain *domain = dev_iommu_domain(dev); >> + struct arm_smmu_xen_domain *xen_domain; >> + >> + xen_domain = domain_hvm_iommu(d)->arch.priv; >> + >> + if (!domain || domain->priv->cfg.domain != d) { >> + dev_err(dev, " not attached to domain %d\n", d->domain_id); >> + return -ESRCH; >> + } >> + >> + arm_smmu_detach_dev(domain, dev); >> + >> + spin_lock(&xen_domain->lock); >> + list_del(&domain->list); >> + spin_unlock(&xen_domain->lock); >> + >> + arm_smmu_domain_destroy(domain); >> + xfree(domain); >> + >> + return 0; >> +} >> + >> +static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, >> + u8 devfn, struct device *dev) >> +{ >> + int ret = 0; >> + >> + /* Don't allow remapping on other domain than hwdom */ >> + if (t != hardware_domain) >> + return -EPERM; >> + >> + if (t == s) >> + return 0; >> + >> + ret = arm_smmu_deassign_dev(s, dev); >> + if (ret) >> + return ret; >> + >> + return 0; >> +} >> + >> +static int arm_smmu_iommu_domain_init(struct domain *d) >> +{ >> + struct arm_smmu_xen_domain *xen_domain; >> + >> + xen_domain = xzalloc(struct arm_smmu_xen_domain); >> + if ( !xen_domain ) >> + return -ENOMEM; >> + >> + spin_lock_init(&xen_domain->lock); >> + INIT_LIST_HEAD(&xen_domain->contexts); >> + >> + domain_hvm_iommu(d)->arch.priv = xen_domain; >> + >> + return 0; >> +} >> + >> +static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d) >> +{ >> +} >> + >> +static void arm_smmu_iommu_domain_teardown(struct domain *d) >> +{ >> + struct arm_smmu_xen_domain *xen_domain = >> domain_hvm_iommu(d)->arch.priv; >> + >> + ASSERT(list_empty(&xen_domain->contexts)); >> + xfree(xen_domain); >> +} >> + >> +static int arm_smmu_map_page(struct domain *d, unsigned long gfn, >> + unsigned long mfn, unsigned int flags) >> +{ >> + p2m_type_t t; >> + >> + /* >> + * Grant mappings can be used for DMA requests. The dev_bus_addr >> + * returned by the hypercall is the MFN (not the IPA). For device >> + * protected by an IOMMU, Xen needs to add a 1:1 mapping in the >> domain >> + * p2m to allow DMA request to work. >> + * This is only valid when the domain is directed mapped. Hence >> this >> + * function should only be used by gnttab code with gfn == mfn. >> + */ >> + BUG_ON(!is_domain_direct_mapped(d)); >> + BUG_ON(mfn != gfn); >> + >> + /* We only support readable and writable flags */ >> + if (!(flags & (IOMMUF_readable | IOMMUF_writable))) >> + return -EINVAL; >> + >> + t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : >> p2m_iommu_map_ro; >> + >> + /* >> + * The function guest_physmap_add_entry replaces the current >> mapping >> + * if there is already one... >> + */ >> + return guest_physmap_add_entry(d, gfn, mfn, 0, t); >> +} >> + >> +static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn) >> +{ >> + /* >> + * This function should only be used by gnttab code when the domain >> + * is direct mapped >> + */ >> + if ( !is_domain_direct_mapped(d) ) >> + return -EINVAL; >> + >> + guest_physmap_remove_page(d, gfn, gfn, 0); >> + >> + return 0; >> +} >> + >> +static const struct iommu_ops arm_smmu_iommu_ops = { >> + .init = arm_smmu_iommu_domain_init, >> + .hwdom_init = arm_smmu_iommu_hwdom_init, >> + .teardown = arm_smmu_iommu_domain_teardown, >> + .iotlb_flush = arm_smmu_iotlb_flush, >> + .iotlb_flush_all = arm_smmu_iotlb_flush_all, >> + .assign_device = arm_smmu_assign_dev, >> + .reassign_device = arm_smmu_reassign_dev, >> + .map_page = arm_smmu_map_page, >> + .unmap_page = arm_smmu_unmap_page, >> +}; >> + >> +static __init int arm_smmu_dt_init(struct dt_device_node *dev, >> + const void *data) >> +{ >> + int rc; >> + >> + /* >> + * Even if the device can't be initialized, we don't want to >> + * give the SMMU device to dom0. >> + */ >> + dt_device_set_used_by(dev, DOMID_XEN); >> + >> + rc = arm_smmu_device_dt_probe(dev); >> + if (rc) >> + return rc; >> + >> + iommu_set_ops(&arm_smmu_iommu_ops); >> + >> + return 0; >> +} >> + >> +DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU) >> + .dt_match = arm_smmu_of_match, >> + .init = arm_smmu_dt_init, >> +DT_DEVICE_END >