[PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Shaik Ameer Basha]

The current exynos-iommu(System MMU) driver does not work autonomously
since it is lack of support for power management of peripheral blocks.
For example, MFC device driver must ensure that its System MMU is disabled
before MFC block is power-down not to invalidate IOTLB in the System MMU
when I/O memory mapping is changed. Because a System MMU resides in the
same H/W block, access to control registers of System MMU while the H/W
block is turned off must be prohibited.

This set of changes solves the above problem with setting each System MMUs
as the parent of the device which owns the System MMU to receive the
information when the device is turned off or turned on.

Another big change to the driver is the support for devicetree.
The bindings for System MMU is described in
Documentation/devicetree/bindings/arm/samsung/system-mmu.txt

In addition, this patchset also includes several bug fixes and enhancements
of the current driver.

Change log:
v12:
- Rebased to the latest 3.15-rc2 master branch
  git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master (3.15-rc2)
- Addressed v11 review comments from
	'Sachin Kamat', 'Tomasz Figa' and 'Shaik Ameer Basha'
- Uses macro names instead of magic numbers for clock description in DT
- Moved DT binding document to seperate patch
- dtsi changes are separated into multiple patches
- patch description of some patches are updated according to the review comments
- removed the macros which hides the clock operations
- review comments related to compatible strings will be fixed in followup patches

v11:
- Rebased on the latest works on clock, arm/samsung, iommu branches
- Change the property to link System MMU and its master H/W
  'iommu' in the master's node -> 'mmu-masters' in the System MMU's node
- Changed compatible string:
  "samsung,sysmmu-v1"
  "samsung,sysmmu-v2"
  "samsung,sysmmu-v3.1"
  "samsung,sysmmu-v3.2"
  "samsung,sysmmu-v3.3"
- Change the implementation of retrieving System MMU version -> simpler
- Check NULL pointer before call to clk_enable() and clk_disable()
- Allow a single master to link to multiple System MMUs.
  (fimc-is, fimd/g2d/Scaler in Exynos5420)
- Workarounds of known problems of System MMU
- Code enhancements:
  * Compilable for 64-bit
  * Enhanced error messages

v10:
- Rebased on the following branches
  git.linaro.org/git-ro/people/mturquette/linux.git/clk-next
  git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung.git/for-next
  git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git/next
  git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master (3.12-rc3)
- Set parent clock to all System MMU clocks.
- Add clock and DT descriptos for Exynos5420
- Modified error handling in exynos_iommu_init()
- Split "iommu/exynos: support for device tree" patch into the following 6 patches
  iommu/exynos: handle only one instance of System MMU
  iommu/exynos: always enable runtime PM
  iommu/exynos: always use a single clock descriptor
  iommu/exynos: remove dbgname from drvdata of a System MMU
  iommu/exynos: use managed driver helper functions
  iommu/exynos: support for device tree
- Remove 'interrupt-names' and 'status' properties from DT
- Change n:1 relationship between master:System MMU into 1:1 relationship.
- Removed custom fault handler and print the status of System MMU
  whenever System MMU fault is occurred.
- Post Antonios Motakis's commit together:
  "iommu/exynos: add devices attached to the System MMU to an IOMMU group"

v9:
- Rebased on the following branches
  git.linaro.org/git-ro/people/mturquette/linux.git/clk-next
  git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung.git/samsung-next
  git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master (3.11-rc4)
- Split "add bus notifier for registering System MMU" into 5 patches
- Call clk_prepare() that was missing in v8.
- Fixed base address of sysmmu_tv in exynos4210.dtsi
- BUG_ON() instead of return -EADDRINUSE when trying mapping on an mapped area
- Moved camif_top to 317 in drivers/clk/samsung/clk-exynos5250.c
- Removed 'iommu' property from 'codec'(mfc) node
- Does not make 'master' clock to be the parent of 'sysmmu' clock.
   'master' clock is enabled before accessing control registers of System MMU
   and disabled after the access.

v8:
- Reordered patch list: moved "change rwloc to spinlock" to the last.
- Fixed remained bug in "fix page table maintenance".
- Always return 0 from exynos_iommu_attach_device().
- Removed prefetch buffer setting when System MMU is enabled
  due to the restriction of prefetch buffers:
  A prefetch buffer must not hit from more than one DMA.
  For instance with GScalers, if a single prefetch buffer is initialized
  with 0x0 ~ 0xFFFFFFFF and a GScaler works on source buffer at 0x10000000
  and target buffer @ 0x20000000, the System MMU may be got deadlock.
  Clients must initialize prefetch buffers with custom function defined
  in exynos-iommu drivers whenever they need to enable prefetch buffers.
- The clock of System MMU has no relationship with the clock of its master H/W.
  The clock of master H/W is always enabled when exynos-iommu driver needs to
  access MMIO area and disabled as soon as the access finishes.
- Removed err_page variable used in exynos_iommu_unmap() in the previous patch
  "fix page table maintenance".
- Split a big patch "add bus notifier for registering System MMU".
   Extracted the following 2 patches: 9/12 and 10/12.
- And some additional fixes...

v7:
- Rebased on the stable 3.10
- Registered PM domains and gate clocks with DT
- Changed connection method between a System MMU and its master H/W
   'mmu-master' property in the node of System MMU
   --> 'iommu' property in the node of master H/W
- Marking device descriptor of master H/W of a System MMU with bus notifier.
- Power management (PM_RUNTIME, PM_SLEEP) of System MMUs with gpd_dev_ops
   of Generic IO Powerdomain. gpd_dev_ops are set to the master H/Ws
   before they are probed in the bus notifier.
- Removed additional debugging features like debugfs entries and
   version names.
- Removed support for advanced features of System MMU 3.2 and 3.3
   the current IOMMU API cannot handle the feature
  (A kind of L2 TLB that fetches several consequence page table entries.
   It must be initialized by the driver of master H/W whenever it works.)

v6:
- Rebased on the branch, next/iommu-exynos of
  git.kernel.org/pub/scm/linux/kernel/git/kgene/linux-samsung.git

v5:
- new bugfix: patch 01
- Reordered patches
  * patch 01 ~ 05: Bugfix and enhancements of the existing driver
  * patch 06 ~ 10: Device Tree support and callbacks for power management
  * patch 11     : System MMU 3.2 and 3.3 support
  * patch 12 ~ 14: Debugging features
- Additional code compaction

v4:
- Remove Change-Id from v3 patches
- Change the order of the third and the first patch
  Thanks to Kukjin Kim.
- Fix memory leak when allocating and assigning exynos_iommu_owner to client
  device if the client device has multiple System MMUs.
  Thanks to Rahul Sharma.

v3:
- Fix prefetch buffer flag definition for System MMU 3.3 (patch 10/12)
- Fix incorrect setting for SET_RUNTIME_PM_OPS (patch 09/12)
  Thanks to Prathyush.

v2:
- Split the patch to iommu/exynos into 9 patches
- Support for System MMU 3.3
- Some code compaction

Antonios Motakis (1):
  iommu/exynos: add devices attached to the System MMU to an IOMMU
    group

Cho KyongHo (30):
  iommu/exynos: do not include removed header
  iommu/exynos: add missing cache flush for removed page table entries
  iommu/exynos: change error handling when page table update is failed
  iommu/exynos: fix L2TLB invalidation
  iommu/exynos: remove prefetch buffer setting
  iommu/exynos: allocate lv2 page table from own slab
  iommu/exynos: always enable runtime PM
  iommu/exynos: handle one instance of sysmmu with a device descriptor
  iommu/exynos: remove dbgname from drvdata of a System MMU
  iommu/exynos: use managed device helper functions
  documentation: iommu: add binding document of Exynos System MMU
  iommu/exynos: support for device tree
  iommu/exynos: gating clocks of master H/W
  iommu/exynos: remove custom fault handler
  iommu/exynos: handle 'mmu-masters' property of DT and improve
    handling sysmmu
  iommu/exynos: turn on useful configuration options
  iommu/exynos: add support for power management subsystems.
  iommu/exynos: allow having multiple System MMUs for a master H/W
  iommu/exynos: change rwlock to spinlock
  iommu/exynos: fix address handling
  iommu/exynos: use exynos-iommu specific typedef
  iommu/exynos: use simpler function to get MMU version
  iommu/exynos: apply workaround of caching fault page table entries
  iommu/exynos: enhanced error messages
  clk: exynos: add gate clock descriptions of System MMU
  ARM: dts: add System MMU nodes of exynos4 series
  ARM: dts: add System MMU nodes of exynos4210
  ARM: dts: add System MMU nodes of exynos4x12
  ARM: dts: add System MMU nodes of exynos5250
  ARM: dts: add System MMU nodes of exynos5420

 .../devicetree/bindings/iommu/samsung,sysmmu.txt   |   79 ++
 arch/arm/Kconfig                                   |    2 +
 arch/arm/boot/dts/exynos4.dtsi                     |  107 ++
 arch/arm/boot/dts/exynos4210.dtsi                  |   23 +-
 arch/arm/boot/dts/exynos4x12.dtsi                  |   78 +-
 arch/arm/boot/dts/exynos5250.dtsi                  |  270 +++-
 arch/arm/boot/dts/exynos5420.dtsi                  |  209 ++-
 drivers/clk/samsung/clk-exynos5250.c               |   36 +
 drivers/clk/samsung/clk-exynos5420.c               |   13 +-
 drivers/iommu/Kconfig                              |    8 +-
 drivers/iommu/exynos-iommu.c                       | 1436 ++++++++++++++------
 include/dt-bindings/clock/exynos5250.h             |   17 +
 include/dt-bindings/clock/exynos5420.h             |    6 +-
 13 files changed, 1852 insertions(+), 432 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt
 mode change 100644 => 100755 drivers/iommu/exynos-iommu.c

-- 
1.7.9.5

[PATCH v12 01/31] iommu/exynos: do not include removed header

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Commit 25e9d28d92 (ARM: EXYNOS: remove system mmu initialization from
exynos tree) removed arch/arm/mach-exynos/mach/sysmmu.h header without
removing remaining use of it from exynos-iommu driver, thus causing a
compilation error.

This patch fixes the error by removing respective include line
from exynos-iommu.c.

CC: Tomasz Figa <t.figa-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |    2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 0740189..46f0ca1 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -29,8 +29,6 @@
 #include <asm/cacheflush.h>
 #include <asm/pgtable.h>
 
-#include <mach/sysmmu.h>
-
 /* We does not consider super section mapping (16MB) */
 #define SECT_ORDER 20
 #define LPAGE_ORDER 16
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 03/31] iommu/exynos: change error handling when page table update is failed

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch changes not to panic on any error when updating page table.
Instead prints error messages with callstack.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   58 ++++++++++++++++++++++++++++++++----------
 1 file changed, 44 insertions(+), 14 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 34e4273..84fc3b4 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -811,13 +811,18 @@ finish:
 static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
 					short *pgcounter)
 {
+	if (lv1ent_section(sent)) {
+		WARN(1, "Trying mapping on %#08lx mapped with 1MiB page", iova);
+		return ERR_PTR(-EADDRINUSE);
+	}
+
 	if (lv1ent_fault(sent)) {
 		unsigned long *pent;
 
 		pent = kzalloc(LV2TABLE_SIZE, GFP_ATOMIC);
 		BUG_ON((unsigned long)pent & (LV2TABLE_SIZE - 1));
 		if (!pent)
-			return NULL;
+			return ERR_PTR(-ENOMEM);
 
 		*sent = mk_lv1ent_page(__pa(pent));
 		*pgcounter = NUM_LV2ENTRIES;
@@ -828,14 +833,21 @@ static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
 	return page_entry(sent, iova);
 }
 
-static int lv1set_section(unsigned long *sent, phys_addr_t paddr, short *pgcnt)
+static int lv1set_section(unsigned long *sent, unsigned long iova,
+			  phys_addr_t paddr, short *pgcnt)
 {
-	if (lv1ent_section(sent))
+	if (lv1ent_section(sent)) {
+		WARN(1, "Trying mapping on 1MiB@%#08lx that is mapped",
+			iova);
 		return -EADDRINUSE;
+	}
 
 	if (lv1ent_page(sent)) {
-		if (*pgcnt != NUM_LV2ENTRIES)
+		if (*pgcnt != NUM_LV2ENTRIES) {
+			WARN(1, "Trying mapping on 1MiB@%#08lx that is mapped",
+				iova);
 			return -EADDRINUSE;
+		}
 
 		kfree(page_entry(sent, 0));
 
@@ -853,8 +865,10 @@ static int lv2set_page(unsigned long *pent, phys_addr_t paddr, size_t size,
 								short *pgcnt)
 {
 	if (size == SPAGE_SIZE) {
-		if (!lv2ent_fault(pent))
+		if (!lv2ent_fault(pent)) {
+			WARN(1, "Trying mapping on 4KiB where mapping exists");
 			return -EADDRINUSE;
+		}
 
 		*pent = mk_lv2ent_spage(paddr);
 		pgtable_flush(pent, pent + 1);
@@ -863,7 +877,10 @@ static int lv2set_page(unsigned long *pent, phys_addr_t paddr, size_t size,
 		int i;
 		for (i = 0; i < SPAGES_PER_LPAGE; i++, pent++) {
 			if (!lv2ent_fault(pent)) {
-				memset(pent, 0, sizeof(*pent) * i);
+				WARN(1,
+				"Trying mapping on 64KiB where mapping exists");
+				if (i > 0)
+					memset(pent - i, 0, sizeof(*pent) * i);
 				return -EADDRINUSE;
 			}
 
@@ -891,7 +908,7 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
 	entry = section_entry(priv->pgtable, iova);
 
 	if (size == SECT_SIZE) {
-		ret = lv1set_section(entry, paddr,
+		ret = lv1set_section(entry, iova, paddr,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 	} else {
 		unsigned long *pent;
@@ -899,17 +916,16 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
 		pent = alloc_lv2entry(entry, iova,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 
-		if (!pent)
-			ret = -ENOMEM;
+		if (IS_ERR(pent))
+			ret = PTR_ERR(pent);
 		else
 			ret = lv2set_page(pent, paddr, size,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 	}
 
-	if (ret) {
+	if (ret)
 		pr_debug("%s: Failed to map iova 0x%lx/0x%x bytes\n",
 							__func__, iova, size);
-	}
 
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
@@ -923,6 +939,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	struct sysmmu_drvdata *data;
 	unsigned long flags;
 	unsigned long *ent;
+	size_t err_pgsize;
 
 	BUG_ON(priv->pgtable == NULL);
 
@@ -931,7 +948,10 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	ent = section_entry(priv->pgtable, iova);
 
 	if (lv1ent_section(ent)) {
-		BUG_ON(size < SECT_SIZE);
+		if (size < SECT_SIZE) {
+			err_pgsize = SECT_SIZE;
+			goto err;
+		}
 
 		*ent = 0;
 		pgtable_flush(ent, ent + 1);
@@ -963,7 +983,10 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	}
 
 	/* lv1ent_large(ent) == true here */
-	BUG_ON(size < LPAGE_SIZE);
+	if (size < LPAGE_SIZE) {
+		err_pgsize = LPAGE_SIZE;
+		goto err;
+	}
 
 	memset(ent, 0, sizeof(*ent) * SPAGES_PER_LPAGE);
 	pgtable_flush(ent, ent + SPAGES_PER_LPAGE);
@@ -978,8 +1001,15 @@ done:
 		sysmmu_tlb_invalidate_entry(data->dev, iova);
 	spin_unlock_irqrestore(&priv->lock, flags);
 
-
 	return size;
+err:
+	spin_unlock_irqrestore(&priv->pgtablelock, flags);
+
+	WARN(1,
+	"%s: Failed due to size(%#x) @ %#08lx is smaller than page size %#x\n",
+	__func__, size, iova, err_pgsize);
+
+	return 0;
 }
 
 static phys_addr_t exynos_iommu_iova_to_phys(struct iommu_domain *domain,
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 05/31] iommu/exynos: remove prefetch buffer setting

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Prefetch buffer is a cache of System MMU 3.x and caches a block of
page table entries to make effect of larger page with small pages.
However, how to control prefetch buffers and the specifications of
prefetch buffers different from minor versions of System MMU v3.
Prefetch buffers must be controled with care because there are some
restrictions in H/W design.

The interface and implementation to initiate prefetch buffers will
be prepared later.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 7ce44a8..7556177 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -243,13 +243,6 @@ static void __sysmmu_set_ptbase(void __iomem *sfrbase,
 	__sysmmu_tlb_invalidate(sfrbase);
 }
 
-static void __sysmmu_set_prefbuf(void __iomem *sfrbase, unsigned long base,
-						unsigned long size, int idx)
-{
-	__raw_writel(base, sfrbase + REG_PB0_SADDR + idx * 8);
-	__raw_writel(size - 1 + base,  sfrbase + REG_PB0_EADDR + idx * 8);
-}
-
 static void __set_fault_handler(struct sysmmu_drvdata *data,
 					sysmmu_fault_handler_t handler)
 {
@@ -423,15 +416,6 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 
 	for (i = 0; i < data->nsfrs; i++) {
 		__sysmmu_set_ptbase(data->sfrbases[i], pgtable);
-
-		if ((readl(data->sfrbases[i] + REG_MMU_VERSION) >> 28) == 3) {
-			/* System MMU version is 3.x */
-			__raw_writel((1 << 12) | (2 << 28),
-					data->sfrbases[i] + REG_MMU_CFG);
-			__sysmmu_set_prefbuf(data->sfrbases[i], 0, -1, 0);
-			__sysmmu_set_prefbuf(data->sfrbases[i], 0, -1, 1);
-		}
-
 		__raw_writel(CTRL_ENABLE, data->sfrbases[i] + REG_MMU_CTRL);
 	}
 
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 09/31] iommu/exynos: remove dbgname from drvdata of a System MMU

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch removes dbgname member from sysmmu_drvdata structure.
Kernel message for debugging already has the name of a single
System MMU node. It also removes some compilation warnings.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   54 +++++++++++++++++++-----------------------
 1 file changed, 25 insertions(+), 29 deletions(-)
 mode change 100644 => 100755 drivers/iommu/exynos-iommu.c

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
old mode 100644
new mode 100755
index a15216f..1af2d23
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -169,7 +169,6 @@ struct sysmmu_drvdata {
 	struct list_head node; /* entry of exynos_iommu_domain.clients */
 	struct device *sysmmu;	/* System MMU's device descriptor */
 	struct device *dev;	/* Owner of system MMU */
-	char *dbgname;
 	void __iomem *sfrbase;
 	struct clk *clk;
 	int activations;
@@ -320,8 +319,8 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	if (!ret && (itype != SYSMMU_FAULT_UNKNOWN))
 		__raw_writel(1 << itype, data->sfrbase + REG_INT_CLEAR);
 	else
-		dev_dbg(data->sysmmu, "(%s) %s is not handled.\n",
-				data->dbgname, sysmmu_fault_name[itype]);
+		dev_dbg(data->sysmmu, "%s is not handled.\n",
+				sysmmu_fault_name[itype]);
 
 	if (itype != SYSMMU_FAULT_UNKNOWN)
 		sysmmu_unblock(data->sfrbase);
@@ -353,10 +352,10 @@ finish:
 	write_unlock_irqrestore(&data->lock, flags);
 
 	if (disabled)
-		dev_dbg(data->sysmmu, "(%s) Disabled\n", data->dbgname);
+		dev_dbg(data->sysmmu, "Disabled\n");
 	else
-		dev_dbg(data->sysmmu, "(%s) %d times left to be disabled\n",
-					data->dbgname, data->activations);
+		dev_dbg(data->sysmmu, "%d times left to be disabled\n",
+					data->activations);
 
 	return disabled;
 }
@@ -383,7 +382,7 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 			ret = 1;
 		}
 
-		dev_dbg(data->sysmmu, "(%s) Already enabled\n", data->dbgname);
+		dev_dbg(data->sysmmu, "Already enabled\n");
 		goto finish;
 	}
 
@@ -398,7 +397,7 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 
 	data->domain = domain;
 
-	dev_dbg(data->sysmmu, "(%s) Enabled\n", data->dbgname);
+	dev_dbg(data->sysmmu, "Enabled\n");
 finish:
 	write_unlock_irqrestore(&data->lock, flags);
 
@@ -414,16 +413,15 @@ int exynos_sysmmu_enable(struct device *dev, unsigned long pgtable)
 
 	ret = pm_runtime_get_sync(data->sysmmu);
 	if (ret < 0) {
-		dev_dbg(data->sysmmu, "(%s) Failed to enable\n", data->dbgname);
+		dev_dbg(data->sysmmu, "Failed to enable\n");
 		return ret;
 	}
 
 	ret = __exynos_sysmmu_enable(data, pgtable, NULL);
 	if (WARN_ON(ret < 0)) {
 		pm_runtime_put(data->sysmmu);
-		dev_err(data->sysmmu,
-			"(%s) Already enabled with page table %#lx\n",
-			data->dbgname, data->pgtable);
+		dev_err(data->sysmmu, "Already enabled with page table %#lx\n",
+			data->pgtable);
 	} else {
 		data->dev = dev;
 	}
@@ -473,9 +471,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 			sysmmu_unblock(data->sfrbase);
 		}
 	} else {
-		dev_dbg(data->sysmmu,
-			"(%s) Disabled. Skipping invalidating TLB.\n",
-			data->dbgname);
+		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
 	}
 
 	read_unlock_irqrestore(&data->lock, flags);
@@ -494,9 +490,7 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 			sysmmu_unblock(data->sfrbase);
 		}
 	} else {
-		dev_dbg(data->sysmmu,
-			"(%s) Disabled. Skipping invalidating TLB.\n",
-			data->dbgname);
+		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
 	}
 
 	read_unlock_irqrestore(&data->lock, flags);
@@ -559,7 +553,7 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 
 	pm_runtime_enable(dev);
 
-	dev_dbg(dev, "(%s) Initialized\n", data->dbgname);
+	dev_dbg(dev, "Initialized\n");
 	return 0;
 err_irq:
 	free_irq(platform_get_irq(pdev, 0), data);
@@ -659,6 +653,7 @@ static int exynos_iommu_attach_device(struct iommu_domain *domain,
 {
 	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
 	struct exynos_iommu_domain *priv = domain->priv;
+	phys_addr_t pagetable = virt_to_phys(priv->pgtable);
 	unsigned long flags;
 	int ret;
 
@@ -682,15 +677,15 @@ static int exynos_iommu_attach_device(struct iommu_domain *domain,
 	spin_unlock_irqrestore(&priv->lock, flags);
 
 	if (ret < 0) {
-		dev_err(dev, "%s: Failed to attach IOMMU with pgtable %#lx\n",
-				__func__, __pa(priv->pgtable));
+		dev_err(dev, "%s: Failed to attach IOMMU with pgtable %pa\n",
+					__func__, &pagetable);
 		pm_runtime_put(data->sysmmu);
 	} else if (ret > 0) {
-		dev_dbg(dev, "%s: IOMMU with pgtable 0x%lx already attached\n",
-					__func__, __pa(priv->pgtable));
+		dev_dbg(dev, "%s: IOMMU with pgtable %pa already attached\n",
+					__func__, &pagetable);
 	} else {
-		dev_dbg(dev, "%s: Attached new IOMMU with pgtable 0x%lx\n",
-					__func__, __pa(priv->pgtable));
+		dev_dbg(dev, "%s: Attached new IOMMU with pgtable %pa\n",
+					__func__, &pagetable);
 	}
 
 	return ret;
@@ -702,6 +697,7 @@ static void exynos_iommu_detach_device(struct iommu_domain *domain,
 	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
 	struct exynos_iommu_domain *priv = domain->priv;
 	struct list_head *pos;
+	phys_addr_t pagetable = virt_to_phys(priv->pgtable);
 	unsigned long flags;
 	bool found = false;
 
@@ -718,13 +714,13 @@ static void exynos_iommu_detach_device(struct iommu_domain *domain,
 		goto finish;
 
 	if (__exynos_sysmmu_disable(data)) {
-		dev_dbg(dev, "%s: Detached IOMMU with pgtable %#lx\n",
-					__func__, __pa(priv->pgtable));
+		dev_dbg(dev, "%s: Detached IOMMU with pgtable %pa\n",
+					__func__, &pagetable);
 		list_del_init(&data->node);
 
 	} else {
-		dev_dbg(dev, "%s: Detaching IOMMU with pgtable %#lx delayed",
-					__func__, __pa(priv->pgtable));
+		dev_dbg(dev, "%s: Detaching IOMMU with pgtable %pa delayed",
+					__func__, &pagetable);
 	}
 
 finish:
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 14/31] iommu/exynos: remove custom fault handler

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This commit removes custom fault handler. The device drivers that
need to register fault handler can register
with iommu_set_fault_handler().

CC: Grant Grundler <grundler-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org>
Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   80 +++++++++++++-----------------------------
 1 file changed, 24 insertions(+), 56 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index ba477c4..b07b78b 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -125,16 +125,6 @@ enum exynos_sysmmu_inttype {
 	SYSMMU_FAULTS_NUM
 };
 
-/*
- * @itype: type of fault.
- * @pgtable_base: the physical address of page table base. This is 0 if @itype
- *                is SYSMMU_BUSERROR.
- * @fault_addr: the device (virtual) address that the System MMU tried to
- *             translated. This is 0 if @itype is SYSMMU_BUSERROR.
- */
-typedef int (*sysmmu_fault_handler_t)(enum exynos_sysmmu_inttype itype,
-			unsigned long pgtable_base, unsigned long fault_addr);
-
 static unsigned short fault_reg_offset[SYSMMU_FAULTS_NUM] = {
 	REG_PAGE_FAULT_ADDR,
 	REG_AR_FAULT_ADDR,
@@ -176,7 +166,6 @@ struct sysmmu_drvdata {
 	int activations;
 	rwlock_t lock;
 	struct iommu_domain *domain;
-	sysmmu_fault_handler_t fault_handler;
 	unsigned long pgtable;
 };
 
@@ -245,34 +234,17 @@ static void __sysmmu_set_ptbase(void __iomem *sfrbase,
 	__sysmmu_tlb_invalidate(sfrbase);
 }
 
-static void __set_fault_handler(struct sysmmu_drvdata *data,
-					sysmmu_fault_handler_t handler)
-{
-	unsigned long flags;
-
-	write_lock_irqsave(&data->lock, flags);
-	data->fault_handler = handler;
-	write_unlock_irqrestore(&data->lock, flags);
-}
-
-void exynos_sysmmu_set_fault_handler(struct device *dev,
-					sysmmu_fault_handler_t handler)
-{
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
-
-	__set_fault_handler(data, handler);
-}
-
-static int default_fault_handler(enum exynos_sysmmu_inttype itype,
-		     unsigned long pgtable_base, unsigned long fault_addr)
+static void show_fault_information(const char *name,
+		enum exynos_sysmmu_inttype itype,
+		unsigned long pgtable_base, unsigned long fault_addr)
 {
 	unsigned long *ent;
 
 	if ((itype >= SYSMMU_FAULTS_NUM) || (itype < SYSMMU_PAGEFAULT))
 		itype = SYSMMU_FAULT_UNKNOWN;
 
-	pr_err("%s occurred at 0x%lx(Page table base: 0x%lx)\n",
-			sysmmu_fault_name[itype], fault_addr, pgtable_base);
+	pr_err("%s occurred at 0x%lx by %s(Page table base: 0x%lx)\n",
+		sysmmu_fault_name[itype], fault_addr, name, pgtable_base);
 
 	ent = section_entry(__va(pgtable_base), fault_addr);
 	pr_err("\tLv1 entry: 0x%lx\n", *ent);
@@ -281,12 +253,6 @@ static int default_fault_handler(enum exynos_sysmmu_inttype itype,
 		ent = page_entry(ent, fault_addr);
 		pr_err("\t Lv2 entry: 0x%lx\n", *ent);
 	}
-
-	pr_err("Generating Kernel OOPS... because it is unrecoverable.\n");
-
-	BUG();
-
-	return 0;
 }
 
 static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
@@ -310,24 +276,28 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	else
 		addr = __raw_readl(data->sfrbase + fault_reg_offset[itype]);
 
-	if (data->domain)
-		ret = report_iommu_fault(data->domain, data->dev, addr, itype);
-
-	if ((ret == -ENOSYS) && data->fault_handler) {
-		unsigned long base = data->pgtable;
-		if (itype != SYSMMU_FAULT_UNKNOWN)
-			base = __raw_readl(data->sfrbase + REG_PT_BASE_ADDR);
-		ret = data->fault_handler(itype, base, addr);
+	if (itype == SYSMMU_FAULT_UNKNOWN) {
+		pr_err("%s: Fault is not occurred by System MMU '%s'!\n",
+			__func__, dev_name(data->sysmmu));
+		pr_err("%s: Please check if IRQ is correctly configured.\n",
+			__func__);
+		BUG();
+	} else {
+		unsigned long base =
+				__raw_readl(data->sfrbase + REG_PT_BASE_ADDR);
+		show_fault_information(dev_name(data->sysmmu),
+					itype, base, addr);
+		if (data->domain)
+			ret = report_iommu_fault(data->domain,
+					data->dev, addr, itype);
 	}
 
-	if (!ret && (itype != SYSMMU_FAULT_UNKNOWN))
-		__raw_writel(1 << itype, data->sfrbase + REG_INT_CLEAR);
-	else
-		dev_dbg(data->sysmmu, "%s is not handled.\n",
-				sysmmu_fault_name[itype]);
+	/* fault is not recovered by fault handler */
+	BUG_ON(ret != 0);
 
-	if (itype != SYSMMU_FAULT_UNKNOWN)
-		sysmmu_unblock(data->sfrbase);
+	__raw_writel(1 << itype, data->sfrbase + REG_INT_CLEAR);
+
+	sysmmu_unblock(data->sfrbase);
 
 	if (!IS_ERR(data->clk_master))
 		clk_disable(data->clk_master);
@@ -576,8 +546,6 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 	rwlock_init(&data->lock);
 	INIT_LIST_HEAD(&data->node);
 
-	__set_fault_handler(data, &default_fault_handler);
-
 	platform_set_drvdata(pdev, data);
 
 	pm_runtime_enable(dev);
-- 
1.7.9.5

[PATCH v12 18/31] iommu/exynos: allow having multiple System MMUs for a master H/W

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Some master device descriptor like fimc-is which is an abstraction
of very complex H/W may have multiple System MMUs. For those devices,
the design of the link between System MMU and its master H/W is needed
to be reconsidered.

A link structure, sysmmu_list_data is introduced that provides a link
to master H/W and that has a pointer to the device descriptor of a
System MMU. Given a device descriptor of a master H/W, it is possible
to traverse all System MMUs that must be controlled along with the
master H/W.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |  545 ++++++++++++++++++++++++++----------------
 1 file changed, 335 insertions(+), 210 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index fefedec3..c2e6365 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -117,6 +117,10 @@
 #define REG_PB1_EADDR		0x058
 
 #define has_sysmmu(dev)		(dev->archdata.iommu != NULL)
+#define for_each_sysmmu_list(dev, list_data)				\
+	list_for_each_entry(list_data,					\
+		&((struct exynos_iommu_owner *)dev->archdata.iommu)->mmu_list, \
+		entry)
 
 static struct kmem_cache *lv2table_kmem_cache;
 
@@ -170,7 +174,7 @@ static char *sysmmu_fault_name[SYSMMU_FAULTS_NUM] = {
 struct exynos_iommu_owner {
 	struct list_head client; /* entry of exynos_iommu_domain.clients */
 	struct device *dev;
-	struct device *sysmmu;
+	struct list_head mmu_list;  /* list of sysmmu_list_data.entry */
 	struct iommu_domain *domain;
 	void *vmm_data;         /* IO virtual memory manager's data */
 	spinlock_t lock;        /* Lock to preserve consistency of System MMU */
@@ -184,6 +188,11 @@ struct exynos_iommu_domain {
 	spinlock_t pgtablelock; /* lock for modifying page table @ pgtable */
 };
 
+struct sysmmu_list_data {
+	struct list_head entry; /* entry of exynos_iommu_owner.mmu_list */
+	struct device *sysmmu;
+};
+
 struct sysmmu_drvdata {
 	struct device *sysmmu;	/* System MMU's device descriptor */
 	struct device *master;	/* Owner of system MMU */
@@ -194,6 +203,7 @@ struct sysmmu_drvdata {
 	rwlock_t lock;
 	struct iommu_domain *domain;
 	bool powered_on;
+	bool suspended;
 	unsigned long pgtable;
 };
 
@@ -466,28 +476,39 @@ static int __sysmmu_enable(struct sysmmu_drvdata *data,
 }
 
 /* __exynos_sysmmu_enable: Enables System MMU
- *
- * returns -error if an error occurred and System MMU is not enabled,
- * 0 if the System MMU has been just enabled and 1 if System MMU was already
- * enabled before.
- */
+*
+* returns -error if an error occurred and System MMU is not enabled,
+* 0 if the System MMU has been just enabled and 1 if System MMU was already
+* enabled before.
+*/
 static int __exynos_sysmmu_enable(struct device *dev, unsigned long pgtable,
 				  struct iommu_domain *domain)
 {
 	int ret = 0;
 	unsigned long flags;
 	struct exynos_iommu_owner *owner = dev->archdata.iommu;
-	struct sysmmu_drvdata *data;
+	struct sysmmu_list_data *list;
 
 	BUG_ON(!has_sysmmu(dev));
 
 	spin_lock_irqsave(&owner->lock, flags);
 
-	data = dev_get_drvdata(owner->sysmmu);
-
-	ret = __sysmmu_enable(data, pgtable, domain);
-	if (ret >= 0)
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
 		data->master = dev;
+		ret = __sysmmu_enable(data, pgtable, domain);
+		if (ret < 0) {
+			struct sysmmu_list_data *iter;
+			for_each_sysmmu_list(dev, iter) {
+				if (iter->sysmmu == list->sysmmu)
+					break;
+				data = dev_get_drvdata(iter->sysmmu);
+				__sysmmu_disable(data);
+				data->master = NULL;
+			}
+			break;
+		}
+	}
 
 	spin_unlock_irqrestore(&owner->lock, flags);
 
@@ -506,17 +527,19 @@ static bool exynos_sysmmu_disable(struct device *dev)
 	unsigned long flags;
 	bool disabled = true;
 	struct exynos_iommu_owner *owner = dev->archdata.iommu;
-	struct sysmmu_drvdata *data;
+	struct sysmmu_list_data *list;
 
 	BUG_ON(!has_sysmmu(dev));
 
 	spin_lock_irqsave(&owner->lock, flags);
 
-	data = dev_get_drvdata(owner->sysmmu);
-
-	disabled = __sysmmu_disable(data);
-	if (disabled)
-		data->master = NULL;
+	/* Every call to __sysmmu_disable() must return same result */
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		disabled = __sysmmu_disable(data);
+		if (disabled)
+			data->master = NULL;
+	}
 
 	spin_unlock_irqrestore(&owner->lock, flags);
 
@@ -528,188 +551,275 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 {
 	struct exynos_iommu_owner *owner = dev->archdata.iommu;
 	unsigned long flags;
-	struct sysmmu_drvdata *data;
-
-	data = dev_get_drvdata(owner->sysmmu);
-
-	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data) && data->powered_on) {
-		unsigned int num_inv = 1;
+	struct sysmmu_list_data *list;
 
-		if (!IS_ERR(data->clk_master))
-			clk_enable(data->clk_master);
+	spin_lock_irqsave(&owner->lock, flags);
 
-		/*
-		 * L2TLB invalidation required
-		 * 4KB page: 1 invalidation
-		 * 64KB page: 16 invalidation
-		 * 1MB page: 64 invalidation
-		 * because it is set-associative TLB
-		 * with 8-way and 64 sets.
-		 * 1MB page can be cached in one of all sets.
-		 * 64KB page can be one of 16 consecutive sets.
-		 */
-		if (__sysmmu_version(data, NULL) == 2) /* major version number */
-			num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		read_lock(&data->lock);
+		if (is_sysmmu_active(data) && data->powered_on) {
+			unsigned int num_inv = 1;
+
+			if (!IS_ERR(data->clk_master))
+				clk_enable(data->clk_master);
+
+			/*
+			 * L2TLB invalidation required
+			 * 4KB page: 1 invalidation
+			 * 64KB page: 16 invalidation
+			 * 1MB page: 64 invalidation
+			 * because it is set-associative TLB
+			 * with 8-way and 64 sets.
+			 * 1MB page can be cached in one of all sets.
+			 * 64KB page can be one of 16 consecutive sets.
+			 */
+			if (__sysmmu_version(data, NULL) == 2)
+				num_inv = min_t(unsigned int,
+						size / PAGE_SIZE, 64);
+
+			if (sysmmu_block(data->sfrbase)) {
+				__sysmmu_tlb_invalidate_entry(data->sfrbase,
+								iova, num_inv);
+				sysmmu_unblock(data->sfrbase);
+			}
 
-		if (sysmmu_block(data->sfrbase)) {
-			__sysmmu_tlb_invalidate_entry(data->sfrbase, iova,
-							num_inv);
-			sysmmu_unblock(data->sfrbase);
+			if (!IS_ERR(data->clk_master))
+				clk_disable(data->clk_master);
+		} else {
+			dev_dbg(dev,
+				"disabled. Skipping TLB invalidation @ %#lx\n",
+				iova);
 		}
-		if (!IS_ERR(data->clk_master))
-			clk_disable(data->clk_master);
-	} else {
-		dev_dbg(dev, "disabled. Skipping TLB invalidation @ %#lx\n",
-			iova);
+
+		read_unlock(&data->lock);
 	}
-	read_unlock_irqrestore(&data->lock, flags);
+
+	spin_unlock_irqrestore(&owner->lock, flags);
 }
 
 void exynos_sysmmu_tlb_invalidate(struct device *dev)
 {
 	struct exynos_iommu_owner *owner = dev->archdata.iommu;
 	unsigned long flags;
-	struct sysmmu_drvdata *data;
+	struct sysmmu_list_data *list;
 
-	data = dev_get_drvdata(owner->sysmmu);
+	spin_lock_irqsave(&owner->lock, flags);
 
-	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data) && data->powered_on) {
-		if (!IS_ERR(data->clk_master))
-			clk_enable(data->clk_master);
-		if (sysmmu_block(data->sfrbase)) {
-			__sysmmu_tlb_invalidate(data->sfrbase);
-			sysmmu_unblock(data->sfrbase);
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		read_lock(&data->lock);
+		if (is_sysmmu_active(data) && data->powered_on) {
+			if (!IS_ERR(data->clk_master))
+				clk_enable(data->clk_master);
+			if (sysmmu_block(data->sfrbase)) {
+				__sysmmu_tlb_invalidate(data->sfrbase);
+				sysmmu_unblock(data->sfrbase);
+			}
+			if (!IS_ERR(data->clk_master))
+				clk_disable(data->clk_master);
+		} else {
+			dev_dbg(dev, "disabled. Skipping TLB invalidation\n");
 		}
-		if (!IS_ERR(data->clk_master))
-			clk_disable(data->clk_master);
-	} else {
-		dev_dbg(dev, "disabled. Skipping TLB invalidation\n");
+		read_unlock(&data->lock);
 	}
-	read_unlock_irqrestore(&data->lock, flags);
-}
 
-static int __init exynos_iommu_prepare(void);
+	spin_unlock_irqrestore(&owner->lock, flags);
+}
 
-static int __init exynos_sysmmu_probe(struct platform_device *pdev)
+static int __init __sysmmu_init_clock(struct device *sysmmu,
+				      struct sysmmu_drvdata *data)
 {
-	int irq, ret;
-	struct device *dev = &pdev->dev;
-	struct sysmmu_drvdata *data;
-	struct resource *res;
-	struct device_node *node;
-
-	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
-	if (!data)
-		return -ENOMEM;
-
-	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	data->sfrbase = devm_ioremap_resource(dev, res);
-	if (IS_ERR(data->sfrbase))
-		return PTR_ERR(data->sfrbase);
+	int ret;
 
-	irq = platform_get_irq(pdev, 0);
-	if (irq <= 0) {
-		dev_dbg(dev, "Unable to find IRQ resource\n");
-		return irq;
+	data->clk = devm_clk_get(sysmmu, "sysmmu");
+	if (IS_ERR(data->clk)) {
+		dev_err(sysmmu, "Failed get sysmmu clock\n");
+		return PTR_ERR(data->clk);
 	}
 
-	ret = devm_request_irq(dev, irq, exynos_sysmmu_irq, 0,
-				dev_name(dev), data);
+	ret = clk_prepare(data->clk);
 	if (ret) {
-		dev_err(dev, "Unabled to register handler of irq %d\n", irq);
+		dev_err(sysmmu, "Failed to prepare sysmmu clock\n");
 		return ret;
 	}
 
-	data->clk = devm_clk_get(dev, "sysmmu");
-	if (IS_ERR(data->clk)) {
-		dev_err(dev, "Failed to get clock!\n");
-		return PTR_ERR(data->clk);
-	} else  {
-		ret = clk_prepare(data->clk);
-		if (ret) {
-			dev_err(dev, "Failed to prepare clk\n");
-			return ret;
-		}
+	data->clk_master = devm_clk_get(sysmmu, "master");
+	if (PTR_ERR(data->clk_master) == -ENOENT) {
+		return 0;
+	} else if (IS_ERR(data->clk_master)) {
+		dev_err(sysmmu, "Failed to get master clock\n");
+		clk_unprepare(data->clk);
+		return PTR_ERR(data->clk_master);
 	}
 
-	data->clk_master = devm_clk_get(dev, "master");
-	if (!IS_ERR(data->clk_master)) {
-		ret = clk_prepare(data->clk_master);
-		if (ret) {
-			clk_unprepare(data->clk);
-			dev_err(dev, "Failed to prepare master's clk\n");
-			return ret;
-		}
+	ret = clk_prepare(data->clk_master);
+	if (ret) {
+		clk_unprepare(data->clk);
+		dev_err(sysmmu, "Failed to prepare master clock\n");
+		return ret;
 	}
 
-	/* Relation between master and System MMU is 1:1. */
-	node = of_parse_phandle(dev->of_node, "mmu-masters", 0);
-	if (node) {
+	return 0;
+}
+
+static int __init __sysmmu_init_master(struct device *dev)
+{
+	int ret;
+	int i = 0;
+	struct device_node *node;
+
+	while ((node = of_parse_phandle(dev->of_node, "mmu-masters", i++))) {
 		struct platform_device *master = of_find_device_by_node(node);
+		struct exynos_iommu_owner *owner;
+		struct sysmmu_list_data *list_data;
 
 		if (!master) {
 			dev_err(dev, "%s: mmu-master '%s' not found\n",
 				__func__, node->name);
-			return -EINVAL;
+			ret = -EINVAL;
+			goto err;
 		}
 
-		if (master->dev.archdata.iommu != NULL) {
-			dev_err(dev, "%s: '%s' is master of other MMU\n",
-				__func__, node->name);
-			return -EINVAL;
+		owner = master->dev.archdata.iommu;
+		if (!owner) {
+			owner = devm_kzalloc(dev, sizeof(*owner), GFP_KERNEL);
+			if (!owner) {
+				dev_err(dev,
+				"%s: Failed to allocate owner structure\n",
+				__func__);
+				ret = -ENOMEM;
+				goto err;
+			}
+
+			INIT_LIST_HEAD(&owner->mmu_list);
+			INIT_LIST_HEAD(&owner->client);
+			owner->dev = &master->dev;
+			spin_lock_init(&owner->lock);
+
+			master->dev.archdata.iommu = owner;
+		}
+
+		list_data = devm_kzalloc(dev, sizeof(*list_data), GFP_KERNEL);
+		if (!list_data) {
+			dev_err(dev,
+				"%s: Failed to allocate sysmmu_list_data\n",
+				__func__);
+			ret = -ENOMEM;
+			goto err;
 		}
 
+		INIT_LIST_HEAD(&list_data->entry);
+		list_data->sysmmu = dev;
+
 		/*
-		 * archdata.iommu will be initialized with exynos_iommu_client
-		 * in sysmmu_hook_driver_register().
+		 * System MMUs are attached in the order of the presence
+		 * in device tree
 		 */
-		master->dev.archdata.iommu = dev;
+		list_add_tail(&list_data->entry, &owner->mmu_list);
 	}
 
-	data->sysmmu = dev;
-	rwlock_init(&data->lock);
+	return 0;
+err:
+	i = 0;
 
-	platform_set_drvdata(pdev, data);
+	while ((node = of_parse_phandle(dev->of_node, "mmu-masters", i++))) {
+		struct platform_device *master = of_find_device_by_node(node);
+		struct exynos_iommu_owner *owner;
+		struct sysmmu_list_data *list_data;
 
-	pm_runtime_enable(dev);
-	data->powered_on = !pm_runtime_enabled(dev);
+		if (!master)
+			continue;
 
-	ret = exynos_iommu_prepare();
-	if (ret)
-		return ret;
+		owner = master->dev.archdata.iommu;
+		if (!owner)
+			continue;
 
-	return 0;
+		for_each_sysmmu_list(owner->dev, list_data) {
+			if (list_data->sysmmu == dev) {
+				list_del(&list_data->entry);
+				kfree(list_data);
+				break;
+			}
+		}
+	}
+
+	return ret;
 }
 
-#ifdef CONFIG_PM_SLEEP
-static int sysmmu_suspend(struct device *dev)
+static int __init __sysmmu_setup(struct device *sysmmu,
+				 struct sysmmu_drvdata *data)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev);
-	unsigned long flags;
-	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data) &&
-		(!pm_runtime_enabled(dev) || data->powered_on))
-		__sysmmu_disable_nocount(data);
-	read_unlock_irqrestore(&data->lock, flags);
-	return 0;
+	int ret;
+
+	ret = __sysmmu_init_clock(sysmmu, data);
+	if (ret) {
+		dev_err(sysmmu, "Failed to initialize gating clocks\n");
+		return ret;
+	}
+
+	ret = __sysmmu_init_master(sysmmu);
+	if (ret) {
+		if (data->clk)
+			clk_unprepare(data->clk);
+		if (data->clk_master)
+			clk_unprepare(data->clk_master);
+		dev_err(sysmmu, "Failed to initialize master device.\n");
+	}
+
+	return ret;
 }
 
-static int sysmmu_resume(struct device *dev)
+static int __init exynos_iommu_prepare(void);
+
+static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev);
-	unsigned long flags;
-	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data) &&
-		(!pm_runtime_enabled(dev) || data->powered_on))
-		__sysmmu_enable_nocount(data);
-	read_unlock_irqrestore(&data->lock, flags);
-	return 0;
-}
-#endif
+	int irq, ret;
+	struct device *dev = &pdev->dev;
+	struct sysmmu_drvdata *data;
+	struct resource *res;
+
+	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+	if (!data) {
+		dev_err(dev, "Not enough memory for driver data\n");
+		return -ENOMEM;
+	}
+
+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	data->sfrbase = devm_request_and_ioremap(dev, res);
+	if (!data->sfrbase)
+		return -EINVAL;
+
+	irq = platform_get_irq(pdev, 0);
+	if (irq <= 0) {
+		dev_err(dev, "Unable to find IRQ resource\n");
+		return irq;
+	}
+
+	ret = devm_request_irq(dev, irq, exynos_sysmmu_irq, 0,
+				dev_name(dev), data);
+	if (ret) {
+		dev_err(dev, "Unabled to register handler of irq %d\n", irq);
+		return ret;
+	}
+
+	pm_runtime_enable(dev);
+
+	ret = exynos_iommu_prepare();
+	if (ret)
+		return ret;
+
+	ret = __sysmmu_setup(dev, data);
+	if (!ret) {
+		data->powered_on = !pm_runtime_enabled(dev);
+		data->sysmmu = dev;
+		rwlock_init(&data->lock);
 
-static SIMPLE_DEV_PM_OPS(sysmmu_pm_ops, sysmmu_suspend, sysmmu_resume);
+		platform_set_drvdata(pdev, data);
+	}
+
+	return ret;
+}
 
 static const struct of_device_id sysmmu_of_match[] __initconst = {
 	{ .compatible	= "samsung,sysmmu-v1", },
@@ -725,7 +835,6 @@ static struct platform_driver exynos_sysmmu_driver __refdata = {
 	.driver	= {
 		.owner		= THIS_MODULE,
 		.name		= "exynos-sysmmu",
-		.pm		= &sysmmu_pm_ops,
 		.of_match_table	= sysmmu_of_match,
 	}
 };
@@ -1145,52 +1254,77 @@ subsys_initcall(exynos_iommu_init);
 #ifdef CONFIG_PM_SLEEP
 static int sysmmu_pm_genpd_suspend(struct device *dev)
 {
-	struct exynos_iommu_owner *owner = dev->archdata.iommu;
+	struct sysmmu_list_data *list;
 	int ret;
 
 	ret = pm_generic_suspend(dev);
 	if (ret)
 		return ret;
 
-	return pm_generic_suspend(owner->sysmmu);
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		unsigned long flags;
+		write_lock_irqsave(&data->lock, flags);
+		if (!data->suspended && is_sysmmu_active(data) &&
+			(!pm_runtime_enabled(dev) || data->powered_on))
+			__sysmmu_disable_nocount(data);
+		data->suspended = true;
+		write_unlock_irqrestore(&data->lock, flags);
+	}
+
+	return 0;
 }
 
 static int sysmmu_pm_genpd_resume(struct device *dev)
 {
-	struct exynos_iommu_owner *owner = dev->archdata.iommu;
-	int ret;
-
-	ret = pm_generic_resume(owner->sysmmu);
-	if (ret)
-		return ret;
+	struct sysmmu_list_data *list;
+
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		unsigned long flags;
+		write_lock_irqsave(&data->lock, flags);
+		if (data->suspended && is_sysmmu_active(data) &&
+			(!pm_runtime_enabled(dev) || data->powered_on))
+			__sysmmu_enable_nocount(data);
+		data->suspended = false;
+		write_unlock_irqrestore(&data->lock, flags);
+	}
 
 	return pm_generic_resume(dev);
 }
 #endif
 
 #ifdef CONFIG_PM_RUNTIME
-static void sysmmu_restore_state(struct device *sysmmu)
+static void sysmmu_restore_state(struct device *dev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(sysmmu);
-	unsigned long flags;
+	struct sysmmu_list_data *list;
+
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		unsigned long flags;
 
-	spin_lock_irqsave(&data->lock, flags);
-	data->powered_on = true;
-	if (is_sysmmu_active(data))
-		__sysmmu_enable_nocount(data);
-	spin_unlock_irqrestore(&data->lock, flags);
+		spin_lock_irqsave(&data->lock, flags);
+		if (!data->powered_on && is_sysmmu_active(data))
+			__sysmmu_enable_nocount(data);
+		data->powered_on = true;
+		spin_unlock_irqrestore(&data->lock, flags);
+	}
 }
 
-static void sysmmu_save_state(struct device *sysmmu)
+static void sysmmu_save_state(struct device *dev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(sysmmu);
-	unsigned long flags;
+	struct sysmmu_list_data *list;
 
-	spin_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data))
-		__sysmmu_disable_nocount(data);
-	data->powered_on = false;
-	spin_unlock_irqrestore(&data->lock, flags);
+	for_each_sysmmu_list(dev, list) {
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+		unsigned long flags;
+
+		spin_lock_irqsave(&data->lock, flags);
+		if (data->powered_on && is_sysmmu_active(data))
+			__sysmmu_disable_nocount(data);
+		data->powered_on = false;
+		spin_unlock_irqrestore(&data->lock, flags);
+	}
 }
 
 static int sysmmu_pm_genpd_save_state(struct device *dev)
@@ -1215,7 +1349,7 @@ static int sysmmu_pm_genpd_save_state(struct device *dev)
 		ret = cb(dev);
 
 	if (ret == 0)
-		sysmmu_save_state(client->sysmmu);
+		sysmmu_save_state(dev);
 
 	return ret;
 }
@@ -1238,13 +1372,13 @@ static int sysmmu_pm_genpd_restore_state(struct device *dev)
 	if (!cb && dev->driver && dev->driver->pm)
 		cb = dev->driver->pm->runtime_resume;
 
-	sysmmu_restore_state(client->sysmmu);
+	sysmmu_restore_state(dev);
 
 	if (cb)
 		ret = cb(dev);
 
 	if (ret)
-		sysmmu_save_state(client->sysmmu);
+		sysmmu_restore_state(dev);
 
 	return ret;
 }
@@ -1277,58 +1411,49 @@ static int sysmmu_hook_driver_register(struct notifier_block *nb,
 	switch (val) {
 	case BUS_NOTIFY_BIND_DRIVER:
 	{
-		struct exynos_iommu_owner *owner;
-		int ret;
-
-		BUG_ON(!dev_get_drvdata(dev->archdata.iommu));
-
-		owner = devm_kzalloc(dev, sizeof(*owner), GFP_KERNEL);
-		if (!owner) {
-			dev_err(dev, "No Memory for exynos_iommu_owner\n");
-			dev->archdata.iommu = NULL;
-			return -ENOMEM;
-		}
-
-		owner->dev = dev;
-		INIT_LIST_HEAD(&owner->client);
-		owner->sysmmu = dev->archdata.iommu;
-
-		ret = pm_genpd_add_callbacks(dev, &sysmmu_devpm_ops, NULL);
-		if (ret && (ret != -ENOSYS)) {
-			dev_err(dev,
+		if (IS_ENABLED(CONFIG_PM_GENERIC_DOMAINS) && dev->pm_domain) {
+			int ret = pm_genpd_add_callbacks(
+					dev, &sysmmu_devpm_ops, NULL);
+			if (ret && (ret != -ENOSYS)) {
+				dev_err(dev,
 				"Failed to register 'dev_pm_ops' for iommu\n");
-			devm_kfree(dev, owner);
-			dev->archdata.iommu = NULL;
-			return ret;
+				return ret;
+			}
 		}
-
-		dev->archdata.iommu = owner;
 		break;
 	}
 	case BUS_NOTIFY_BOUND_DRIVER:
 	{
-		struct exynos_iommu_owner *owner = dev->archdata.iommu;
-		if (!pm_runtime_enabled(dev)) {
+		struct sysmmu_list_data *list;
+
+		/* OK if runtime PM is enabled with genpd for dev */
+		if (pm_runtime_enabled(dev) && dev->pm_domain)
+			break;
+
+		/*
+		 * System MMU will be permanently enabled if the master H/W is
+		 * neither registered to a power domain nor runtime PM enabled.
+		 */
+		for_each_sysmmu_list(dev, list) {
 			struct sysmmu_drvdata *data =
-					dev_get_drvdata(owner->sysmmu);
-			if (pm_runtime_enabled(data->sysmmu)) {
-				data->powered_on = true;
-				if (is_sysmmu_active(data))
-					__sysmmu_enable_nocount(data);
-				pm_runtime_disable(data->sysmmu);
-			}
+					dev_get_drvdata(list->sysmmu);
+			unsigned long flags;
+
+			write_lock_irqsave(&data->lock, flags);
+			if (is_sysmmu_active(data) && !data->powered_on)
+				__sysmmu_enable_nocount(data);
+			data->powered_on = true;
+			pm_runtime_disable(data->sysmmu);
+			write_unlock_irqrestore(&data->lock, flags);
 		}
 		break;
 	}
 	case BUS_NOTIFY_UNBOUND_DRIVER:
 	{
 		struct exynos_iommu_owner *owner = dev->archdata.iommu;
-		struct device *sysmmu = owner->sysmmu;
 		if (WARN_ON(!list_empty(&owner->client)))
 			iommu_detach_device(owner->domain, dev);
 		__pm_genpd_remove_callbacks(dev, false);
-		devm_kfree(dev, owner);
-		dev->archdata.iommu = sysmmu;
 		break;
 	}
 	} /* switch (val) */
-- 
1.7.9.5

Re: [PATCH v12 18/31] iommu/exynos: allow having multiple System MMUs for a master H/W

[Tushar Behera]

On 04/27/2014 01:07 PM, Shaik Ameer Basha wrote:
> From: Cho KyongHo <pullip.cho@samsung.com>
> 
> Some master device descriptor like fimc-is which is an abstraction
> of very complex H/W may have multiple System MMUs. For those devices,
> the design of the link between System MMU and its master H/W is needed
> to be reconsidered.
> 
> A link structure, sysmmu_list_data is introduced that provides a link
> to master H/W and that has a pointer to the device descriptor of a
> System MMU. Given a device descriptor of a master H/W, it is possible
> to traverse all System MMUs that must be controlled along with the
> master H/W.
> 
> Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>

Since you are posting the patches, you should also add your
Signed-of-by.

> ---
>  drivers/iommu/exynos-iommu.c |  545 ++++++++++++++++++++++++++----------------
>  1 file changed, 335 insertions(+), 210 deletions(-)
> 
> diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
> index fefedec3..c2e6365 100755
> --- a/drivers/iommu/exynos-iommu.c
> +++ b/drivers/iommu/exynos-iommu.c

[ ... ]

>  static int sysmmu_pm_genpd_save_state(struct device *dev)
> @@ -1215,7 +1349,7 @@ static int sysmmu_pm_genpd_save_state(struct device *dev)
>  		ret = cb(dev);
>  
>  	if (ret == 0)
> -		sysmmu_save_state(client->sysmmu);
> +		sysmmu_save_state(dev);
>  

client is now unused, remove the variable.

>  	return ret;
>  }
> @@ -1238,13 +1372,13 @@ static int sysmmu_pm_genpd_restore_state(struct device *dev)
>  	if (!cb && dev->driver && dev->driver->pm)
>  		cb = dev->driver->pm->runtime_resume;
>  
> -	sysmmu_restore_state(client->sysmmu);
> +	sysmmu_restore_state(dev);
>  
>  	if (cb)
>  		ret = cb(dev);
>  
>  	if (ret)
> -		sysmmu_save_state(client->sysmmu);
> +		sysmmu_restore_state(dev);
>  

client is now unused, remove the variable.


-- 
Tushar Behera

Re: [PATCH v12 18/31] iommu/exynos: allow having multiple System MMUs for a master H/W

[Cho KyongHo]

On Mon, 28 Apr 2014 16:08:14 +0530, Tushar Behera wrote:
> On 04/27/2014 01:07 PM, Shaik Ameer Basha wrote:
> > From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> > 
> > Some master device descriptor like fimc-is which is an abstraction
> > of very complex H/W may have multiple System MMUs. For those devices,
> > the design of the link between System MMU and its master H/W is needed
> > to be reconsidered.
> > 
> > A link structure, sysmmu_list_data is introduced that provides a link
> > to master H/W and that has a pointer to the device descriptor of a
> > System MMU. Given a device descriptor of a master H/W, it is possible
> > to traverse all System MMUs that must be controlled along with the
> > master H/W.
> > 
> > Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> 
> Since you are posting the patches, you should also add your
> Signed-of-by.
>
> > ---
> >  drivers/iommu/exynos-iommu.c |  545 ++++++++++++++++++++++++++----------------
> >  1 file changed, 335 insertions(+), 210 deletions(-)
> > 
> > diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
> > index fefedec3..c2e6365 100755
> > --- a/drivers/iommu/exynos-iommu.c
> > +++ b/drivers/iommu/exynos-iommu.c
> 
> [ ... ]
> 
> >  static int sysmmu_pm_genpd_save_state(struct device *dev)
> > @@ -1215,7 +1349,7 @@ static int sysmmu_pm_genpd_save_state(struct device *dev)
> >  		ret = cb(dev);
> >  
> >  	if (ret == 0)
> > -		sysmmu_save_state(client->sysmmu);
> > +		sysmmu_save_state(dev);
> >  
> 
> client is now unused, remove the variable.
> 
> >  	return ret;
> >  }
> > @@ -1238,13 +1372,13 @@ static int sysmmu_pm_genpd_restore_state(struct device *dev)
> >  	if (!cb && dev->driver && dev->driver->pm)
> >  		cb = dev->driver->pm->runtime_resume;
> >  
> > -	sysmmu_restore_state(client->sysmmu);
> > +	sysmmu_restore_state(dev);
> >  
> >  	if (cb)
> >  		ret = cb(dev);
> >  
> >  	if (ret)
> > -		sysmmu_save_state(client->sysmmu);
> > +		sysmmu_restore_state(dev);
> >  
> 
> client is now unused, remove the variable.
> 

Ok.

Thanks.

KyongHo

Re: [PATCH v12 18/31] iommu/exynos: allow having multiple System MMUs for a master H/W

[Tomasz Figa]

On 27.04.2014 09:37, Shaik Ameer Basha wrote:
> From: Cho KyongHo <pullip.cho@samsung.com>
>
> Some master device descriptor like fimc-is which is an abstraction
> of very complex H/W may have multiple System MMUs. For those devices,
> the design of the link between System MMU and its master H/W is needed
> to be reconsidered.
>
> A link structure, sysmmu_list_data is introduced that provides a link
> to master H/W and that has a pointer to the device descriptor of a
> System MMU. Given a device descriptor of a master H/W, it is possible
> to traverse all System MMUs that must be controlled along with the
> master H/W.
>
> Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
> ---
>   drivers/iommu/exynos-iommu.c |  545 ++++++++++++++++++++++++++----------------
>   1 file changed, 335 insertions(+), 210 deletions(-)
>
> diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
> index fefedec3..c2e6365 100755
> --- a/drivers/iommu/exynos-iommu.c
> +++ b/drivers/iommu/exynos-iommu.c
> @@ -117,6 +117,10 @@
>   #define REG_PB1_EADDR		0x058
>
>   #define has_sysmmu(dev)		(dev->archdata.iommu != NULL)
> +#define for_each_sysmmu_list(dev, list_data)				\
> +	list_for_each_entry(list_data,					\
> +		&((struct exynos_iommu_owner *)dev->archdata.iommu)->mmu_list, \
> +		entry)

Sorry, NAK.

Please don't add this kind of complexity and business logic to low level 
code. We want the configuration functions to be simple, easy to read, 
maintain and extend.

The proper way to do it is to let the IOMMUs be grouped together on 
IOMMU subsystem level, so that each IOMMU consumer driver would see just 
one IOMMU, but then IOMMU driver callbacks would handle just particular 
instances of the IOMMU IP blocks, without any loops, lists and other 
crazy code...

Best regards,
Tomasz

Re: [PATCH v12 18/31] iommu/exynos: allow having multiple System MMUs for a master H/W

[Cho KyongHo]

On Tue, 06 May 2014 20:05:14 +0200, Tomasz Figa wrote:
> On 27.04.2014 09:37, Shaik Ameer Basha wrote:
> > From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> >
> > Some master device descriptor like fimc-is which is an abstraction
> > of very complex H/W may have multiple System MMUs. For those devices,
> > the design of the link between System MMU and its master H/W is needed
> > to be reconsidered.
> >
> > A link structure, sysmmu_list_data is introduced that provides a link
> > to master H/W and that has a pointer to the device descriptor of a
> > System MMU. Given a device descriptor of a master H/W, it is possible
> > to traverse all System MMUs that must be controlled along with the
> > master H/W.
> >
> > Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> > ---
> >   drivers/iommu/exynos-iommu.c |  545 ++++++++++++++++++++++++++----------------
> >   1 file changed, 335 insertions(+), 210 deletions(-)
> >
> > diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
> > index fefedec3..c2e6365 100755
> > --- a/drivers/iommu/exynos-iommu.c
> > +++ b/drivers/iommu/exynos-iommu.c
> > @@ -117,6 +117,10 @@
> >   #define REG_PB1_EADDR		0x058
> >
> >   #define has_sysmmu(dev)		(dev->archdata.iommu != NULL)
> > +#define for_each_sysmmu_list(dev, list_data)				\
> > +	list_for_each_entry(list_data,					\
> > +		&((struct exynos_iommu_owner *)dev->archdata.iommu)->mmu_list, \
> > +		entry)
> 
> Sorry, NAK.
> 
> Please don't add this kind of complexity and business logic to low level 
> code. We want the configuration functions to be simple, easy to read, 
> maintain and extend.
> 
> The proper way to do it is to let the IOMMUs be grouped together on 
> IOMMU subsystem level, so that each IOMMU consumer driver would see just 
> one IOMMU, but then IOMMU driver callbacks would handle just particular 
> instances of the IOMMU IP blocks, without any loops, lists and other 
> crazy code...
> 

It is done in IOMMU driver internally.
IOMMU consumer driver(IOMMU client device driver?) sees IOMMU domain
but IOMMU itself. How to handle IOMMUs of the client device is just
in charge of IOMMU driver according to the hardwired bus topology.

If a master and IOMMU device should be 1:1 relationship as you contend,
the master device driver should be responsible for express the bus
topology in its business logic. I think it is not good as I told earlier.

IOMMU group is different from the grouping System MMUs in Exynos IOMMU driver.
IOMMU group is invented to group several IOMMUs on different bus with different
masters to assign the same IOMMU domain.
It is useful for user level virtualization but for expressing bus topology.
Exynos IOMMU driver just groups System MMUs that have the same master device.
Even though Exynos IOMMU driver implements IOMMU group which is already
implemented by 20/31 patch, such looping is still required.

However, I will consider other elegant way of interation of multiple System MMUs.

Regards,

KyongHo

[PATCH v12 20/31] iommu/exynos: add devices attached to the System MMU to an IOMMU group

[Shaik Ameer Basha]

From: Antonios Motakis <a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org>

Patch written by Antonios Motakis <a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org>:

IOMMU groups are expected by certain users of the IOMMU API,
e.g. VFIO. Since each device is behind its own System MMU, we
can allocate a new IOMMU group for each device.

Reviewd-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Antonios Motakis <a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index c9076e1..5c7f4d2 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -1208,6 +1208,32 @@ static phys_addr_t exynos_iommu_iova_to_phys(struct iommu_domain *domain,
 	return phys;
 }
 
+static int exynos_iommu_add_device(struct device *dev)
+{
+	struct iommu_group *group;
+	int ret;
+
+	group = iommu_group_get(dev);
+
+	if (!group) {
+		group = iommu_group_alloc();
+		if (IS_ERR(group)) {
+			dev_err(dev, "Failed to allocate IOMMU group\n");
+			return PTR_ERR(group);
+		}
+	}
+
+	ret = iommu_group_add_device(group, dev);
+	iommu_group_put(group);
+
+	return ret;
+}
+
+static void exynos_iommu_remove_device(struct device *dev)
+{
+	iommu_group_remove_device(dev);
+}
+
 static struct iommu_ops exynos_iommu_ops = {
 	.domain_init = &exynos_iommu_domain_init,
 	.domain_destroy = &exynos_iommu_domain_destroy,
@@ -1216,6 +1242,8 @@ static struct iommu_ops exynos_iommu_ops = {
 	.map = &exynos_iommu_map,
 	.unmap = &exynos_iommu_unmap,
 	.iova_to_phys = &exynos_iommu_iova_to_phys,
+	.add_device = &exynos_iommu_add_device,
+	.remove_device = &exynos_iommu_remove_device,
 	.pgsize_bitmap = SECT_SIZE | LPAGE_SIZE | SPAGE_SIZE,
 };
 
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 21/31] iommu/exynos: fix address handling

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Use of __pa and __va macro is changed to virt_to_phys and phys_to_virt
which are recommended in driver code. printk formatting of physical
address is also fixed to %pa.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 5c7f4d2..08a7ce0 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -131,7 +131,8 @@ static unsigned long *section_entry(unsigned long *pgtable, unsigned long iova)
 
 static unsigned long *page_entry(unsigned long *sent, unsigned long iova)
 {
-	return (unsigned long *)__va(lv2table_base(sent)) + lv2ent_offset(iova);
+	return (unsigned long *)phys_to_virt(
+				lv2table_base(sent)) + lv2ent_offset(iova);
 }
 
 enum exynos_sysmmu_inttype {
@@ -204,7 +205,7 @@ struct sysmmu_drvdata {
 	struct iommu_domain *domain;
 	bool powered_on;
 	bool suspended;
-	unsigned long pgtable;
+	phys_addr_t pgtable;
 };
 
 static bool set_sysmmu_active(struct sysmmu_drvdata *data)
@@ -296,17 +297,17 @@ static void __sysmmu_set_ptbase(void __iomem *sfrbase,
 
 static void show_fault_information(const char *name,
 		enum exynos_sysmmu_inttype itype,
-		unsigned long pgtable_base, unsigned long fault_addr)
+		phys_addr_t pgtable_base, unsigned long fault_addr)
 {
 	unsigned long *ent;
 
 	if ((itype >= SYSMMU_FAULTS_NUM) || (itype < SYSMMU_PAGEFAULT))
 		itype = SYSMMU_FAULT_UNKNOWN;
 
-	pr_err("%s occurred at 0x%lx by %s(Page table base: 0x%lx)\n",
-		sysmmu_fault_name[itype], fault_addr, name, pgtable_base);
+	pr_err("%s occurred at %#lx by %s(Page table base: %pa)\n",
+		sysmmu_fault_name[itype], fault_addr, name, &pgtable_base);
 
-	ent = section_entry(__va(pgtable_base), fault_addr);
+	ent = section_entry(phys_to_virt(pgtable_base), fault_addr);
 	pr_err("\tLv1 entry: 0x%lx\n", *ent);
 
 	if (lv1ent_page(ent)) {
@@ -909,7 +910,7 @@ static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
 	for (i = 0; i < NUM_LV1ENTRIES; i++)
 		if (lv1ent_page(priv->pgtable + i))
 			kmem_cache_free(lv2table_kmem_cache,
-					__va(lv2table_base(priv->pgtable + i)));
+				phys_to_virt(lv2table_base(priv->pgtable + i)));
 
 	free_pages((unsigned long)priv->pgtable, 2);
 	free_pages((unsigned long)priv->lv2entcnt, 1);
@@ -928,8 +929,7 @@ static int exynos_iommu_attach_device(struct iommu_domain *domain,
 
 	spin_lock_irqsave(&priv->lock, flags);
 
-
-	ret = __exynos_sysmmu_enable(dev, __pa(priv->pgtable), domain);
+	ret = __exynos_sysmmu_enable(dev, pagetable, domain);
 	if (ret == 0) {
 		list_add_tail(&owner->client, &priv->clients);
 		owner->domain = domain;
@@ -937,13 +937,14 @@ static int exynos_iommu_attach_device(struct iommu_domain *domain,
 
 	spin_unlock_irqrestore(&priv->lock, flags);
 
-	if (ret < 0)
+	if (ret < 0) {
 		dev_err(dev, "%s: Failed to attach IOMMU with pgtable %pa\n",
-					__func__, &pagetable);
-	else
-		dev_dbg(dev, "%s: Attached IOMMU with pgtable %pa%s\n",
-					__func__, &pagetable,
-					(ret == 0) ? "" : ", again");
+				__func__, &pagetable);
+		return ret;
+	}
+
+	dev_dbg(dev, "%s: Attached IOMMU with pgtable %pa %s\n",
+		__func__, &pagetable, (ret == 0) ? "" : ", again");
 
 	return ret;
 }
@@ -993,7 +994,7 @@ static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
 		if (!pent)
 			return ERR_PTR(-ENOMEM);
 
-		*sent = mk_lv1ent_page(__pa(pent));
+		*sent = mk_lv1ent_page(virt_to_phys(pent));
 		*pgcounter = NUM_LV2ENTRIES;
 		pgtable_flush(pent, pent + NUM_LV2ENTRIES);
 		pgtable_flush(sent, sent + 1);
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v12 24/31] iommu/exynos: apply workaround of caching fault page table entries

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch contains 2 workaround for the System MMU v3.x.

System MMU v3.2 and v3.3 has FLPD cache that caches first level page
table entries to reduce page table walking latency. However, the
FLPD cache is filled with a first level page table entry even though
it is not accessed by a master H/W because System MMU v3.3
speculatively prefetches page table entries that may be accessed
in the near future by the master H/W.
The prefetched FLPD cache entries are not invalidated by iommu_unmap()
because iommu_unmap() only unmaps and invalidates the page table
entries that is mapped.

Because exynos-iommu driver discards a second level page table when
it needs to be replaced with another second level page table or
a first level page table entry with 1MB mapping, It is required to
invalidate FLPD cache that may contain the first level page table
entry that points to the second level page table.

Another workaround of System MMU v3.3 is initializing the first level
page table entries with the second level page table which is filled
with all zeros. This prevents System MMU prefetches 'fault' first
level page table entry which may lead page fault on access to 16MiB
wide.

System MMU 3.x fetches consecutive page table entries by a page
table walking to maximize bus utilization and to minimize TLB miss
panelty.
Unfortunately, functional problem is raised with the fetching behavior
because it fetches 'fault' page table entries that specifies no
translation information and that a valid translation information will
be written to in the near future. The logic in the System MMU generates
page fault with the cached fault entries that is no longer coherent
with the page table which is updated.

There is another workaround that must be implemented by I/O virtual
memory manager: any two consecutive I/O virtual memory area must have
a hole between the two that is larger than or equal to 128KiB.
Also, next I/O virtual memory area must be started from the next
128KiB boundary.

0            128K           256K               384K             512K
|-------------|---------------|-----------------|----------------|
|area1---------------->|.........hole...........|<--- area2 -----

The constraint is depicted above.
The size is selected by the calculation followed:
 - System MMU can fetch consecutive 64 page table entries at once
   64 * 4KiB = 256KiB. This is the size between 128K ~ 384K of the
   above picture. This style of fetching is 'block fetch'. It fetches
   the page table entries predefined consecutive page table entries
   including the entry that is the reason of the page table walking.
 - System MMU can prefetch upto consecutive 32 page table entries.
   This is the size between 256K ~ 384K.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |  166 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 149 insertions(+), 17 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 54011e5..35b055e 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -49,8 +49,12 @@ typedef u32 sysmmu_pte_t;
 #define LPAGE_MASK (~(LPAGE_SIZE - 1))
 #define SPAGE_MASK (~(SPAGE_SIZE - 1))
 
-#define lv1ent_fault(sent) (((*(sent) & 3) == 0) || ((*(sent) & 3) == 3))
-#define lv1ent_page(sent) ((*(sent) & 3) == 1)
+#define lv1ent_fault(sent) ((*(sent) == ZERO_LV2LINK) || \
+			   ((*(sent) & 3) == 0) || ((*(sent) & 3) == 3))
+#define lv1ent_zero(sent) (*(sent) == ZERO_LV2LINK)
+#define lv1ent_page_zero(sent) ((*(sent) & 3) == 1)
+#define lv1ent_page(sent) ((*(sent) != ZERO_LV2LINK) && \
+			  ((*(sent) & 3) == 1))
 #define lv1ent_section(sent) ((*(sent) & 3) == 2)
 
 #define lv2ent_fault(pent) ((*(pent) & 3) == 0)
@@ -138,6 +142,8 @@ static u32 lv2ent_offset(sysmmu_iova_t iova)
 		entry)
 
 static struct kmem_cache *lv2table_kmem_cache;
+static sysmmu_pte_t *zero_lv2_table;
+#define ZERO_LV2LINK mk_lv1ent_page(virt_to_phys(zero_lv2_table))
 
 static sysmmu_pte_t *section_entry(sysmmu_pte_t *pgtable, sysmmu_iova_t iova)
 {
@@ -545,6 +551,35 @@ static bool exynos_sysmmu_disable(struct device *dev)
 	return disabled;
 }
 
+static void __sysmmu_tlb_invalidate_flpdcache(struct sysmmu_drvdata *data,
+					      sysmmu_iova_t iova)
+{
+	if (__raw_sysmmu_version(data) == MAKE_MMU_VER(3, 3))
+		__raw_writel(iova | 0x1, data->sfrbase + REG_MMU_FLUSH_ENTRY);
+}
+
+static void sysmmu_tlb_invalidate_flpdcache(struct device *dev,
+					    sysmmu_iova_t iova)
+{
+	struct sysmmu_list_data *list;
+
+	for_each_sysmmu_list(dev, list) {
+		unsigned long flags;
+		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
+
+		if (!IS_ERR(data->clk_master))
+			clk_enable(data->clk_master);
+
+		spin_lock_irqsave(&data->lock, flags);
+		if (is_sysmmu_active(data) && data->powered_on)
+			__sysmmu_tlb_invalidate_flpdcache(data, iova);
+		spin_unlock_irqrestore(&data->lock, flags);
+
+		if (!IS_ERR(data->clk_master))
+			clk_disable(data->clk_master);
+	}
+}
+
 static void sysmmu_tlb_invalidate_entry(struct device *dev, sysmmu_iova_t iova,
 					size_t size)
 {
@@ -848,21 +883,32 @@ static inline void pgtable_flush(void *vastart, void *vaend)
 static int exynos_iommu_domain_init(struct iommu_domain *domain)
 {
 	struct exynos_iommu_domain *priv;
+	int i;
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
-	priv->pgtable = (sysmmu_pte_t *)__get_free_pages(
-						GFP_KERNEL | __GFP_ZERO, 2);
+	priv->pgtable = (sysmmu_pte_t *)__get_free_pages(GFP_KERNEL, 2);
 	if (!priv->pgtable)
 		goto err_pgtable;
 
-	priv->lv2entcnt = (short *)__get_free_pages(
-						GFP_KERNEL | __GFP_ZERO, 1);
+	priv->lv2entcnt = (short *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 1);
 	if (!priv->lv2entcnt)
 		goto err_counter;
 
+	/* w/a of System MMU v3.3 to prevent caching 1MiB mapping */
+	for (i = 0; i < NUM_LV1ENTRIES; i += 8) {
+		priv->pgtable[i + 0] = ZERO_LV2LINK;
+		priv->pgtable[i + 1] = ZERO_LV2LINK;
+		priv->pgtable[i + 2] = ZERO_LV2LINK;
+		priv->pgtable[i + 3] = ZERO_LV2LINK;
+		priv->pgtable[i + 4] = ZERO_LV2LINK;
+		priv->pgtable[i + 5] = ZERO_LV2LINK;
+		priv->pgtable[i + 6] = ZERO_LV2LINK;
+		priv->pgtable[i + 7] = ZERO_LV2LINK;
+	}
+
 	pgtable_flush(priv->pgtable, priv->pgtable + NUM_LV1ENTRIES);
 
 	spin_lock_init(&priv->lock);
@@ -975,8 +1021,8 @@ static void exynos_iommu_detach_device(struct iommu_domain *domain,
 		dev_dbg(dev, "%s: No IOMMU is attached\n", __func__);
 }
 
-static sysmmu_pte_t *alloc_lv2entry(sysmmu_pte_t *sent, sysmmu_iova_t iova,
-					short *pgcounter)
+static sysmmu_pte_t *alloc_lv2entry(struct exynos_iommu_domain *priv,
+		sysmmu_pte_t *sent, sysmmu_iova_t iova, short *pgcounter)
 {
 	if (lv1ent_section(sent)) {
 		WARN(1, "Trying mapping on %#08x mapped with 1MiB page", iova);
@@ -985,6 +1031,7 @@ static sysmmu_pte_t *alloc_lv2entry(sysmmu_pte_t *sent, sysmmu_iova_t iova,
 
 	if (lv1ent_fault(sent)) {
 		sysmmu_pte_t *pent;
+		bool need_flush_flpd_cache = lv1ent_zero(sent);
 
 		pent = kmem_cache_zalloc(lv2table_kmem_cache, GFP_ATOMIC);
 		BUG_ON((unsigned int)pent & (LV2TABLE_SIZE - 1));
@@ -995,12 +1042,39 @@ static sysmmu_pte_t *alloc_lv2entry(sysmmu_pte_t *sent, sysmmu_iova_t iova,
 		*pgcounter = NUM_LV2ENTRIES;
 		pgtable_flush(pent, pent + NUM_LV2ENTRIES);
 		pgtable_flush(sent, sent + 1);
+
+		/*
+		 * If pretched SLPD is a fault SLPD in zero_l2_table, FLPD cache
+		 * may caches the address of zero_l2_table. This function
+		 * replaces the zero_l2_table with new L2 page table to write
+		 * valid mappings.
+		 * Accessing the valid area may cause page fault since FLPD
+		 * cache may still caches zero_l2_table for the valid area
+		 * instead of new L2 page table that have the mapping
+		 * information of the valid area
+		 * Thus any replacement of zero_l2_table with other valid L2
+		 * page table must involve FLPD cache invalidation for System
+		 * MMU v3.3.
+		 * FLPD cache invalidation is performed with TLB invalidation
+		 * by VPN without blocking. It is safe to invalidate TLB without
+		 * blocking because the target address of TLB invalidation is
+		 * not currently mapped.
+		 */
+		if (need_flush_flpd_cache) {
+			struct exynos_iommu_owner *owner;
+			spin_lock(&priv->lock);
+			list_for_each_entry(owner, &priv->clients, client)
+				sysmmu_tlb_invalidate_flpdcache(
+							owner->dev, iova);
+			spin_unlock(&priv->lock);
+		}
 	}
 
 	return page_entry(sent, iova);
 }
 
-static int lv1set_section(sysmmu_pte_t *sent, sysmmu_iova_t iova,
+static int lv1set_section(struct exynos_iommu_domain *priv,
+			  sysmmu_pte_t *sent, sysmmu_iova_t iova,
 			  phys_addr_t paddr, short *pgcnt)
 {
 	if (lv1ent_section(sent)) {
@@ -1020,6 +1094,18 @@ static int lv1set_section(sysmmu_pte_t *sent, sysmmu_iova_t iova,
 		*pgcnt = 0;
 	}
 
+	spin_lock(&priv->lock);
+	if (lv1ent_page_zero(sent)) {
+		struct exynos_iommu_owner *owner;
+		/*
+		 * Flushing FLPD cache in System MMU v3.3 that may cache a FLPD
+		 * entry by speculative prefetch of SLPD which has no mapping.
+		 */
+		list_for_each_entry(owner, &priv->clients, client)
+			sysmmu_tlb_invalidate_flpdcache(owner->dev, iova);
+	}
+	spin_unlock(&priv->lock);
+
 	*sent = mk_lv1ent_sect(paddr);
 
 	pgtable_flush(sent, sent + 1);
@@ -1059,6 +1145,32 @@ static int lv2set_page(sysmmu_pte_t *pent, phys_addr_t paddr, size_t size,
 	return 0;
 }
 
+/*
+ * *CAUTION* to the I/O virtual memory managers that support exynos-iommu:
+ *
+ * System MMU v3.x have an advanced logic to improve address translation
+ * performance with caching more page table entries by a page table walk.
+ * However, the logic has a bug that caching fault page table entries and System
+ * MMU reports page fault if the cached fault entry is hit even though the fault
+ * entry is updated to a valid entry after the entry is cached.
+ * To prevent caching fault page table entries which may be updated to valid
+ * entries later, the virtual memory manager should care about the w/a about the
+ * problem. The followings describe w/a.
+ *
+ * Any two consecutive I/O virtual address regions must have a hole of 128KiB
+ * in maximum to prevent misbehavior of System MMU 3.x. (w/a of h/w bug)
+ *
+ * Precisely, any start address of I/O virtual region must be aligned by
+ * the following sizes for System MMU v3.1 and v3.2.
+ * System MMU v3.1: 128KiB
+ * System MMU v3.2: 256KiB
+ *
+ * Because System MMU v3.3 caches page table entries more aggressively, it needs
+ * more w/a.
+ * - Any two consecutive I/O virtual regions must be have a hole of larger size
+ *   than or equal size to 128KiB.
+ * - Start address of an I/O virtual region must be aligned by 128KiB.
+ */
 static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
 			 phys_addr_t paddr, size_t size, int prot)
 {
@@ -1075,12 +1187,12 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
 	entry = section_entry(priv->pgtable, iova);
 
 	if (size == SECT_SIZE) {
-		ret = lv1set_section(entry, iova, paddr,
+		ret = lv1set_section(priv, entry, iova, paddr,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 	} else {
 		sysmmu_pte_t *pent;
 
-		pent = alloc_lv2entry(entry, iova,
+		pent = alloc_lv2entry(priv, entry, iova,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 
 		if (IS_ERR(pent))
@@ -1099,11 +1211,24 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
 	return ret;
 }
 
+static void exynos_iommu_tlb_invalidate_entry(struct exynos_iommu_domain *priv,
+						sysmmu_iova_t iova, size_t size)
+{
+	struct exynos_iommu_owner *owner;
+	unsigned long flags;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	list_for_each_entry(owner, &priv->clients, client)
+		sysmmu_tlb_invalidate_entry(owner->dev, iova, size);
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+}
+
 static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 				       unsigned long l_iova, size_t size)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
-	struct exynos_iommu_owner *owner;
 	sysmmu_iova_t iova = (sysmmu_iova_t)l_iova;
 	sysmmu_pte_t *ent;
 	size_t err_pgsize;
@@ -1121,7 +1246,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 			goto err;
 		}
 
-		*ent = 0;
+		*ent = ZERO_LV2LINK; /* w/a for h/w bug in Sysmem MMU v3.3 */
 		pgtable_flush(ent, ent + 1);
 		size = SECT_SIZE;
 		goto done;
@@ -1164,10 +1289,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 done:
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
-	spin_lock_irqsave(&priv->lock, flags);
-	list_for_each_entry(owner, &priv->clients, client)
-		sysmmu_tlb_invalidate_entry(owner->dev, iova, size);
-	spin_unlock_irqrestore(&priv->lock, flags);
+	exynos_iommu_tlb_invalidate_entry(priv, iova, size);
 
 	return size;
 err:
@@ -1264,6 +1386,14 @@ static int __init exynos_iommu_init(void)
 		goto err_reg_driver;
 	}
 
+	zero_lv2_table = kmem_cache_zalloc(lv2table_kmem_cache, GFP_KERNEL);
+	if (zero_lv2_table == NULL) {
+		pr_err("%s: Failed to allocate zero level2 page table\n",
+			__func__);
+		ret = -ENOMEM;
+		goto err_zero_lv2;
+	}
+
 	ret = bus_set_iommu(&platform_bus_type, &exynos_iommu_ops);
 	if (ret) {
 		pr_err("%s: Failed to register exynos-iommu driver.\n",
@@ -1273,6 +1403,8 @@ static int __init exynos_iommu_init(void)
 
 	return 0;
 err_set_iommu:
+	kmem_cache_free(lv2table_kmem_cache, zero_lv2_table);
+err_zero_lv2:
 	platform_driver_unregister(&exynos_sysmmu_driver);
 err_reg_driver:
 	kmem_cache_destroy(lv2table_kmem_cache);
-- 
1.7.9.5

[PATCH v12 25/31] iommu/exynos: enhanced error messages

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Some redundant error message is removed and some error messages
are changed to error level from debug level.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/iommu/exynos-iommu.c |   23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 35b055e..4009eb2 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -1018,7 +1018,7 @@ static void exynos_iommu_detach_device(struct iommu_domain *domain,
 		dev_dbg(dev, "%s: Detached IOMMU with pgtable %pa\n",
 					__func__, &pagetable);
 	else
-		dev_dbg(dev, "%s: No IOMMU is attached\n", __func__);
+		dev_err(dev, "%s: No IOMMU is attached\n", __func__);
 }
 
 static sysmmu_pte_t *alloc_lv2entry(struct exynos_iommu_domain *priv,
@@ -1117,10 +1117,8 @@ static int lv2set_page(sysmmu_pte_t *pent, phys_addr_t paddr, size_t size,
 								short *pgcnt)
 {
 	if (size == SPAGE_SIZE) {
-		if (!lv2ent_fault(pent)) {
-			WARN(1, "Trying mapping on 4KiB where mapping exists");
+		if (WARN_ON(!lv2ent_fault(pent)))
 			return -EADDRINUSE;
-		}
 
 		*pent = mk_lv2ent_spage(paddr);
 		pgtable_flush(pent, pent + 1);
@@ -1128,9 +1126,7 @@ static int lv2set_page(sysmmu_pte_t *pent, phys_addr_t paddr, size_t size,
 	} else { /* size == LPAGE_SIZE */
 		int i;
 		for (i = 0; i < SPAGES_PER_LPAGE; i++, pent++) {
-			if (!lv2ent_fault(pent)) {
-				WARN(1,
-				"Trying mapping on 64KiB where mapping exists");
+			if (WARN_ON(!lv2ent_fault(pent))) {
 				if (i > 0)
 					memset(pent - i, 0, sizeof(*pent) * i);
 				return -EADDRINUSE;
@@ -1203,8 +1199,8 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
 	}
 
 	if (ret)
-		pr_debug("%s: Failed to map iova %#x/%#zx bytes\n",
-							__func__, iova, size);
+		pr_err("%s: Failed(%d) to map %#zx bytes @ %#x\n",
+			__func__, ret, size, iova);
 
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
@@ -1241,7 +1237,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	ent = section_entry(priv->pgtable, iova);
 
 	if (lv1ent_section(ent)) {
-		if (size < SECT_SIZE) {
+		if (WARN_ON(size < SECT_SIZE)) {
 			err_pgsize = SECT_SIZE;
 			goto err;
 		}
@@ -1276,7 +1272,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	}
 
 	/* lv1ent_large(ent) == true here */
-	if (size < LPAGE_SIZE) {
+	if (WARN_ON(size < LPAGE_SIZE)) {
 		err_pgsize = LPAGE_SIZE;
 		goto err;
 	}
@@ -1295,9 +1291,8 @@ done:
 err:
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
-	WARN(1,
-	"%s: Failed due to size(%#zx) @ %#x is smaller than page size %#zx\n",
-	__func__, size, iova, err_pgsize);
+	pr_err("%s: Failed: size(%#zx) @ %#x is smaller than page size %#zx\n",
+		__func__, size, iova, err_pgsize);
 
 	return 0;
 }
-- 
1.7.9.5

[PATCH v12 26/31] clk: exynos: add gate clock descriptions of System MMU

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This adds gate clocks of all System MMUs and their master IPs
that are not apeared in clk-exynos5250.c and clk-exynos5420.c

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 drivers/clk/samsung/clk-exynos5250.c   |   36 ++++++++++++++++++++++++++++++++
 drivers/clk/samsung/clk-exynos5420.c   |   13 ++++++++++--
 include/dt-bindings/clock/exynos5250.h |   17 +++++++++++++++
 include/dt-bindings/clock/exynos5420.h |    6 +++++-
 4 files changed, 69 insertions(+), 3 deletions(-)

diff --git a/drivers/clk/samsung/clk-exynos5250.c b/drivers/clk/samsung/clk-exynos5250.c
index e7ee442..04f41ec 100644
--- a/drivers/clk/samsung/clk-exynos5250.c
+++ b/drivers/clk/samsung/clk-exynos5250.c
@@ -28,6 +28,8 @@
 #define MPLL_CON0		0x4100
 #define SRC_CORE1		0x4204
 #define GATE_IP_ACP		0x8800
+#define GATE_IP_ISP0		0xC800
+#define GATE_IP_ISP1		0xC804
 #define CPLL_LOCK		0x10020
 #define EPLL_LOCK		0x10030
 #define VPLL_LOCK		0x10040
@@ -37,6 +39,7 @@
 #define VPLL_CON0		0x10140
 #define GPLL_CON0		0x10150
 #define SRC_TOP0		0x10210
+#define SRC_TOP1		0x10214
 #define SRC_TOP2		0x10218
 #define SRC_TOP3		0x1021c
 #define SRC_GSCL		0x10220
@@ -100,6 +103,7 @@ static unsigned long exynos5250_clk_regs[] __initdata = {
 	DIV_CPU0,
 	SRC_CORE1,
 	SRC_TOP0,
+	SRC_TOP1,
 	SRC_TOP2,
 	SRC_TOP3,
 	SRC_GSCL,
@@ -141,6 +145,8 @@ static unsigned long exynos5250_clk_regs[] __initdata = {
 	PLL_DIV2_SEL,
 	GATE_IP_DISP1,
 	GATE_IP_ACP,
+	GATE_IP_ISP0,
+	GATE_IP_ISP1,
 };
 
 static int exynos5250_clk_suspend(void)
@@ -196,6 +202,7 @@ PNAME(mout_aclk200_p)	= { "mout_mpll_user", "mout_bpll_user" };
 PNAME(mout_aclk200_sub_p) = { "fin_pll", "div_aclk200" };
 PNAME(mout_aclk266_sub_p) = { "fin_pll", "div_aclk266" };
 PNAME(mout_aclk333_sub_p) = { "fin_pll", "div_aclk333" };
+PNAME(mout_aclk400_isp_sub_p) = { "fin_pll", "div_aclk400_isp" };
 PNAME(mout_hdmi_p)	= { "div_hdmi_pixel", "sclk_hdmiphy" };
 PNAME(mout_usb3_p)	= { "mout_mpll_user", "mout_cpll" };
 PNAME(mout_group1_p)	= { "fin_pll", "fin_pll", "sclk_hdmi27m",
@@ -273,6 +280,7 @@ static struct samsung_mux_clock exynos5250_mux_clks[] __initdata = {
 	MUX(0, "mout_aclk166", mout_aclk166_p, SRC_TOP0, 8, 1),
 	MUX(0, "mout_aclk200", mout_aclk200_p, SRC_TOP0, 12, 1),
 	MUX(0, "mout_aclk333", mout_aclk166_p, SRC_TOP0, 16, 1),
+	MUX(0, "mout_aclk400_isp", mout_aclk200_p, SRC_TOP1, 24, 1),
 
 	MUX(0, "mout_cpll", mout_cpll_p, SRC_TOP2, 8, 1),
 	MUX(0, "mout_epll", mout_epll_p, SRC_TOP2, 12, 1),
@@ -319,6 +327,8 @@ static struct samsung_mux_clock exynos5250_mux_clks[] __initdata = {
 	MUX(0, "mout_spi1", mout_group1_p, SRC_PERIC1, 20, 4),
 	MUX(0, "mout_spi2", mout_group1_p, SRC_PERIC1, 24, 4),
 
+	MUX(0, "mout_aclk_400_isp_sub", mout_aclk400_isp_sub_p, SRC_TOP3, 20, 1),
+	MUX(0, "mout_aclk_266_isp_sub", mout_aclk266_sub_p, SRC_TOP3, 16, 1),
 	/*
 	 * CMU_CDREX
 	 */
@@ -351,6 +361,7 @@ static struct samsung_div_clock exynos5250_div_clks[] __initdata = {
 	DIV(0, "div_aclk200", "mout_aclk200", DIV_TOP0, 12, 3),
 	DIV(0, "div_aclk266", "mout_mpll_user", DIV_TOP0, 16, 3),
 	DIV(0, "div_aclk333", "mout_aclk333", DIV_TOP0, 20, 3),
+	DIV(0, "div_aclk400_isp", "mout_aclk400_isp", DIV_TOP1, 20, 3),
 
 	DIV(0, "div_aclk66_pre", "mout_mpll_user", DIV_TOP1, 24, 3),
 
@@ -615,6 +626,31 @@ static struct samsung_gate_clock exynos5250_gate_clks[] __initdata = {
 	GATE(CLK_WDT, "wdt", "div_aclk66", GATE_IP_PERIS, 19, 0, 0),
 	GATE(CLK_RTC, "rtc", "div_aclk66", GATE_IP_PERIS, 20, 0, 0),
 	GATE(CLK_TMU, "tmu", "div_aclk66", GATE_IP_PERIS, 21, 0, 0),
+	GATE(CLK_SMMU_TV, "smmu_tv", "mout_aclk200_disp1_sub",
+		GATE_IP_DISP1, 2, 0, 0),
+	GATE(CLK_SMMU_FIMD1, "smmu_fimd1", "mout_aclk200_disp1_sub",
+		GATE_IP_DISP1, 8, 0, 0),
+	GATE(CLK_SMMU_2D, "smmu_2d", "div_aclk200", GATE_IP_ACP, 7, 0, 0),
+	GATE(CLK_SMMU_FIMC_ISP, "smmu_fimc_isp", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP0, 8, 0, 0),
+	GATE(CLK_SMMU_FIMC_DRC, "smmu_fimc_drc", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP0, 9, 0, 0),
+	GATE(CLK_SMMU_FIMC_FD, "smmu_fimc_fd", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP0, 10, 0, 0),
+	GATE(CLK_SMMU_FIMC_SCC, "smmu_fimc_scc", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP0, 11, 0, 0),
+	GATE(CLK_SMMU_FIMC_SCP, "smmu_fimc_scp", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP0, 12, 0, 0),
+	GATE(CLK_SMMU_FIMC_MCU, "smmu_fimc_mcu", "mout_aclk_400_isp_sub",
+		GATE_IP_ISP0, 13, 0, 0),
+	GATE(CLK_SMMU_FIMC_ODC, "smmu_fimc_odc", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP1, 4, 0, 0),
+	GATE(CLK_SMMU_FIMC_DIS0, "smmu_fimc_dis0", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP1, 5, 0, 0),
+	GATE(CLK_SMMU_FIMC_DIS1, "smmu_fimc_dis1", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP1, 6, 0, 0),
+	GATE(CLK_SMMU_FIMC_3DNR, "smmu_fimc_3dnr", "mout_aclk_266_isp_sub",
+		GATE_IP_ISP1, 7, 0, 0),
 };
 
 static struct samsung_pll_rate_table vpll_24mhz_tbl[] __initdata = {
diff --git a/drivers/clk/samsung/clk-exynos5420.c b/drivers/clk/samsung/clk-exynos5420.c
index 60b2681..b58e4d3 100644
--- a/drivers/clk/samsung/clk-exynos5420.c
+++ b/drivers/clk/samsung/clk-exynos5420.c
@@ -82,6 +82,7 @@
 #define GATE_BUS_PERIC1		0x10754
 #define GATE_BUS_PERIS0		0x10760
 #define GATE_BUS_PERIS1		0x10764
+#define GATE_IP_G2D		0x08800
 #define GATE_IP_GSCL0		0x10910
 #define GATE_IP_GSCL1		0x10920
 #define GATE_IP_MFC		0x1092c
@@ -707,6 +708,10 @@ static struct samsung_gate_clock exynos5420_gate_clks[] __initdata = {
 	GATE(CLK_GSCL_WB, "gscl_wb", "aclk300_gscl", GATE_IP_GSCL1, 13, 0, 0),
 	GATE(CLK_SMMU_FIMCL3, "smmu_fimcl3,", "aclk333_432_gscl",
 			GATE_IP_GSCL1, 16, 0, 0),
+	GATE(CLK_FIMC_LITE0, "fimc_lite0", "aclk333_432_gscl",
+			GATE_IP_GSCL0, 5, 0, 0),
+	GATE(CLK_FIMC_LITE1, "fimc_lite1", "aclk333_432_gscl",
+			GATE_IP_GSCL0, 6, 0, 0),
 	GATE(CLK_FIMC_LITE3, "fimc_lite3", "aclk333_432_gscl",
 			GATE_IP_GSCL1, 17, 0, 0),
 
@@ -715,8 +720,10 @@ static struct samsung_gate_clock exynos5420_gate_clks[] __initdata = {
 	GATE(CLK_DP1, "dp1", "aclk200_disp1", GATE_IP_DISP1, 4, 0, 0),
 	GATE(CLK_MIXER, "mixer", "aclk166", GATE_IP_DISP1, 5, 0, 0),
 	GATE(CLK_HDMI, "hdmi", "aclk200_disp1", GATE_IP_DISP1, 6, 0, 0),
-	GATE(CLK_SMMU_FIMD1, "smmu_fimd1", "aclk300_disp1", GATE_IP_DISP1, 8, 0,
-		0),
+	GATE(CLK_SMMU_FIMD1M0, "smmu_fimd1m0", "aclk300_disp1", GATE_IP_DISP1,
+		7, 0, 0),
+	GATE(CLK_SMMU_FIMD1M1, "smmu_fimd1m1", "aclk300_disp1", GATE_IP_DISP1,
+		8, 0, 0),
 
 	GATE(CLK_MFC, "mfc", "aclk333", GATE_IP_MFC, 0, 0, 0),
 	GATE(CLK_SMMU_MFCL, "smmu_mfcl", "aclk333", GATE_IP_MFC, 1, 0, 0),
@@ -743,6 +750,8 @@ static struct samsung_gate_clock exynos5420_gate_clks[] __initdata = {
 		0),
 	GATE(CLK_SMMU_MIXER, "smmu_mixer", "aclk200_disp1", GATE_IP_DISP1, 9, 0,
 		0),
+	GATE(CLK_G2D, "g2d", "aclk333_g2d", GATE_IP_G2D, 3, 0, 0),
+	GATE(CLK_SMMU_G2D, "smmu_g2d", "aclk333_g2d", GATE_IP_G2D, 7, 0, 0),
 };
 
 static struct samsung_pll_clock exynos5420_plls[nr_plls] __initdata = {
diff --git a/include/dt-bindings/clock/exynos5250.h b/include/dt-bindings/clock/exynos5250.h
index 922f2dc..743ec63 100644
--- a/include/dt-bindings/clock/exynos5250.h
+++ b/include/dt-bindings/clock/exynos5250.h
@@ -150,6 +150,23 @@
 #define CLK_G2D			345
 #define CLK_MDMA0		346
 #define CLK_SMMU_MDMA0		347
+#define CLK_SMMU_TV		348
+#define CLK_SMMU_FIMD1		349
+#define CLK_SMMU_2D		350
+#define CLK_SMMU_FIMC_ISP	351
+#define CLK_SMMU_FIMC_DRC	352
+#define CLK_SMMU_FIMC_SCC	353
+#define CLK_SMMU_FIMC_SCP	354
+#define CLK_SMMU_FIMC_FD	355
+#define CLK_SMMU_FIMC_MCU	356
+#define CLK_SMMU_FIMC_ODC	357
+#define CLK_SMMU_FIMC_DIS0	358
+#define CLK_SMMU_FIMC_DIS1	359
+#define CLK_SMMU_FIMC_3DNR	360
+#define CLK_SMMU_FIMC_LITE0	361
+#define CLK_SMMU_FIMC_LITE1	362
+#define CLK_CAMIF_TOP		363
+
 
 /* mux clocks */
 #define CLK_MOUT_HDMI		1024
diff --git a/include/dt-bindings/clock/exynos5420.h b/include/dt-bindings/clock/exynos5420.h
index 5eefd88..25dedca 100644
--- a/include/dt-bindings/clock/exynos5420.h
+++ b/include/dt-bindings/clock/exynos5420.h
@@ -140,7 +140,8 @@
 #define CLK_HDMI		413
 #define CLK_ACLK300_DISP1	420
 #define CLK_FIMD1		421
-#define CLK_SMMU_FIMD1		422
+#define CLK_SMMU_FIMD1M0	422
+#define CLK_SMMU_FIMD1M1	423
 #define CLK_ACLK166		430
 #define CLK_MIXER		431
 #define CLK_ACLK266		440
@@ -166,12 +167,15 @@
 #define CLK_MDMA0		473
 #define CLK_ACLK333_G2D		480
 #define CLK_G2D			481
+#define CLK_SMMU_G2D		482
 #define CLK_ACLK333_432_GSCL	490
 #define CLK_SMMU_3AA		491
 #define CLK_SMMU_FIMCL0		492
 #define CLK_SMMU_FIMCL1		493
 #define CLK_SMMU_FIMCL3		494
 #define CLK_FIMC_LITE3		495
+#define CLK_FIMC_LITE1		496
+#define CLK_FIMC_LITE0		497
 #define CLK_ACLK_G3D		500
 #define CLK_G3D			501
 #define CLK_SMMU_MIXER		502
-- 
1.7.9.5

[PATCH v12 27/31] ARM: dts: add System MMU nodes of exynos4 series

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch adds System MMU nodes that are common to exynos4 series.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/exynos4.dtsi |  107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)

diff --git a/arch/arm/boot/dts/exynos4.dtsi b/arch/arm/boot/dts/exynos4.dtsi
index 2f8bcd0..229efee 100644
--- a/arch/arm/boot/dts/exynos4.dtsi
+++ b/arch/arm/boot/dts/exynos4.dtsi
@@ -536,4 +536,111 @@
 		samsung,power-domain = <&pd_lcd0>;
 		status = "disabled";
 	};
+
+	sysmmu_mfc_l: sysmmu@13620000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x13620000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 5>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_MFCL>;
+		samsung,power-domain = <&pd_mfc>;
+		mmu-masters = <&mfc>;
+	};
+
+	sysmmu_mfc_r: sysmmu@13630000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x13630000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_MFCR>;
+		samsung,power-domain = <&pd_mfc>;
+		mmu-masters = <&mfc>;
+	};
+
+	sysmmu_tv: sysmmu@12E20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x12E20000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_TV>;
+		samsung,power-domain = <&pd_tv>;
+	};
+
+	sysmmu_fimc0: sysmmu@11A20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11A20000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 2>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC0>;
+		samsung,power-domain = <&pd_cam>;
+		mmu-masters = <&fimc_0>;
+	};
+
+	sysmmu_fimc1: sysmmu@11A30000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11A30000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 3>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC1>;
+		samsung,power-domain = <&pd_cam>;
+		mmu-masters = <&fimc_1>;
+	};
+
+	sysmmu_fimc2: sysmmu@11A40000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11A40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC2>;
+		samsung,power-domain = <&pd_cam>;
+		mmu-masters = <&fimc_2>;
+	};
+
+	sysmmu_fimc3: sysmmu@11A50000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11A50000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 5>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC3>;
+		samsung,power-domain = <&pd_cam>;
+		mmu-masters = <&fimc_3>;
+	};
+
+	sysmmu_jpeg: sysmmu@11A60000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11A60000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_JPEG>;
+		samsung,power-domain = <&pd_cam>;
+	};
+
+	sysmmu_rotator: sysmmu@12A30000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x12A30000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 0>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_ROTATOR>;
+		samsung,power-domain = <&pd_lcd0>;
+	};
+
+	sysmmu_fimd0: sysmmu@11E20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11E20000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 2>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMD0>;
+		samsung,power-domain = <&pd_lcd0>;
+		mmu-masters = <&fimd>;
+	};
 };
-- 
1.7.9.5

[PATCH v12 28/31] ARM: dts: add System MMU nodes of exynos4210

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch adds System MMUs that are specific to exynos4210.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/exynos4210.dtsi |   23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/exynos4210.dtsi b/arch/arm/boot/dts/exynos4210.dtsi
index cacf614..0b4ece1 100644
--- a/arch/arm/boot/dts/exynos4210.dtsi
+++ b/arch/arm/boot/dts/exynos4210.dtsi
@@ -114,7 +114,7 @@
 		status = "disabled";
 	};
 
-	g2d@12800000 {
+	g2d: g2d@12800000 {
 		compatible = "samsung,s5pv210-g2d";
 		reg = <0x12800000 0x1000>;
 		interrupts = <0 89 0>;
@@ -153,4 +153,25 @@
 			samsung,lcd-wb;
 		};
 	};
+
+	sysmmu_g2d: sysmmu@12A20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x12A20000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 7>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_G2D>;
+		samsung,power-domain = <&pd_lcd0>;
+		mmu-masters = <&g2d>;
+	};
+
+	sysmmu_fimd1: sysmmu@12220000 {
+		compatible = "samsung,sysmmu-v1";
+		interrupt-parent = <&combiner>;
+		reg = <0x12220000 0x1000>;
+		interrupts = <5 3>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMD1>;
+		samsung,power-domain = <&pd_lcd1>;
+	};
 };
-- 
1.7.9.5

[PATCH v12 29/31] ARM: dts: add System MMU nodes of exynos4x12

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch adds System MMU nodes that are specifict to exynos4x12
series.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/exynos4x12.dtsi |   78 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/exynos4x12.dtsi b/arch/arm/boot/dts/exynos4x12.dtsi
index c4a9306..21cb164 100644
--- a/arch/arm/boot/dts/exynos4x12.dtsi
+++ b/arch/arm/boot/dts/exynos4x12.dtsi
@@ -119,7 +119,7 @@
 		interrupts = <0 72 0>;
 	};
 
-	g2d@10800000 {
+	g2d: g2d@10800000 {
 		compatible = "samsung,exynos4212-g2d";
 		reg = <0x10800000 0x1000>;
 		interrupts = <0 89 0>;
@@ -243,4 +243,80 @@
 		clock-names = "biu", "ciu";
 		status = "disabled";
 	};
+
+	sysmmu_g2d: sysmmu@10A40000{
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x10A40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 7>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_G2D>;
+		mmu-masters = <&g2d>;
+	};
+
+	sysmmu_fimc_isp: sysmmu@12260000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x12260000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 2>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_ISP>;
+		mmu-masters = <&fimc_is>;
+	};
+
+	sysmmu_fimc_drc: sysmmu@12270000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x12270000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 3>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_DRC>;
+		mmu-masters = <&fimc_is>;
+	};
+
+	sysmmu_fimc_fd: sysmmu@122A0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x122A0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 4>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FD>;
+		mmu-masters = <&fimc_is>;
+	};
+
+	sysmmu_fimc_mcuctl: sysmmu@122B0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x122B0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 5>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_ISPCX>;
+		mmu-masters = <&fimc_is>;
+	};
+
+	sysmmu_fimc_lite0: sysmmu@123B0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x123B0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 0>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_LITE0>;
+		mmu-masters = <&fimc_lite_0>;
+	};
+
+	sysmmu_fimc_lite1: sysmmu@123C0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x123C0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <16 1>;
+		samsung,power-domain = <&pd_isp>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_LITE1>;
+		mmu-masters = <&fimc_lite_1>;
+	};
 };
-- 
1.7.9.5

[PATCH v12 30/31] ARM: dts: add System MMU nodes of exynos5250

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/exynos5250.dtsi |  270 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 267 insertions(+), 3 deletions(-)

diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
index 3742331..eebd397 100644
--- a/arch/arm/boot/dts/exynos5250.dtsi
+++ b/arch/arm/boot/dts/exynos5250.dtsi
@@ -82,6 +82,16 @@
 		reg = <0x10044040 0x20>;
 	};
 
+	pd_isp: isp-power-domain@0x10044020 {
+		compatible = "samsung,exynos4210-pd";
+		reg = <0x10044020 0x20>;
+	};
+
+	pd_disp1: disp1-power-domain@0x100440A0 {
+		compatible = "samsung,exynos4210-pd";
+		reg = <0x100440A0 0x20>;
+	};
+
 	clock: clock-controller@10010000 {
 		compatible = "samsung,exynos5250-clock";
 		reg = <0x10010000 0x30000>;
@@ -192,7 +202,7 @@
 		clock-names = "fimg2d";
 	};
 
-	codec@11000000 {
+	mfc: codec@11000000 {
 		compatible = "samsung,mfc-v6";
 		reg = <0x11000000 0x10000>;
 		interrupts = <0 96 0>;
@@ -692,7 +702,7 @@
 				"sclk_hdmiphy", "mout_hdmi";
 	};
 
-	mixer {
+	mixer: mixer {
 		compatible = "samsung,exynos5250-mixer";
 		reg = <0x14450000 0x10000>;
 		interrupts = <0 94 0>;
@@ -713,7 +723,7 @@
 		phy-names = "dp";
 	};
 
-	fimd@14400000 {
+	fimd: fimd@14400000 {
 		clocks = <&clock CLK_SCLK_FIMD1>, <&clock CLK_FIMD1>;
 		clock-names = "sclk_fimd", "fimd";
 	};
@@ -736,4 +746,258 @@
 		clocks = <&clock 348>;
 		clock-names = "secss";
 	};
+
+	sysmmu_g2d: sysmmu@10A60000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x10A60000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <24 5>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_2D>;
+	};
+
+	sysmmu_mfc_r: sysmmu@11200000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x11200000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <6 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MFCR>, <&clock CLK_MFC>;
+		mmu-masters = <&mfc>;
+		samsung,power-domain = <&pd_mfc>;
+	};
+
+	sysmmu_mfc_l: sysmmu@11210000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x11210000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <8 5>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MFCL>, <&clock CLK_MFC>;
+		mmu-masters = <&mfc>;
+		samsung,power-domain = <&pd_mfc>;
+	};
+
+	sysmmu_rotator: sysmmu@11D40000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11D40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 0>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_ROTATOR>;
+	};
+
+	sysmmu_fimc_isp: sysmmu@13260000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13260000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <10 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_ISP>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_drc: sysmmu@13270000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13270000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <11 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_DRC>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_scc: sysmmu@13280000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13280000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 2>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_SCC>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_scp: sysmmu@13290000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13290000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_SCP>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_fd: sysmmu@132A0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132A0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 0>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_FD>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_mcuctl: sysmmu@132B0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132B0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_MCU>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_odc: sysmmu@132C0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132C0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <11 0>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_ODC>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_dis0: sysmmu@132D0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132D0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <10 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_DIS0>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_dis1: sysmmu@132E0000{
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132E0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <9 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_DIS1>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_3dnr: sysmmu@132F0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x132F0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <5 6>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMC_3DNR>;
+		samsung,power-domain = <&pd_isp>;
+	};
+
+	sysmmu_fimc_lite0: sysmmu@13C40000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13C40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMC_LITE0>, <&clock CLK_CAMIF_TOP>;
+		samsung,power-domain = <&pd_gsc>;
+	};
+
+	sysmmu_fimc_lite1: sysmmu@13C50000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13C50000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <24 1>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMC_LITE1>, <&clock CLK_CAMIF_TOP>;
+		samsung,power-domain = <&pd_gsc>;
+	};
+
+	sysmmu_gsc0: sysmmu@13E80000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13E80000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL0>, <&clock CLK_GSCL0>;
+		samsung,power-domain = <&pd_gsc>;
+		mmu-masters = <&gsc_0>;
+	};
+
+	sysmmu_gsc1: sysmmu@13E90000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13E90000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL1>, <&clock CLK_GSCL1>;
+		samsung,power-domain = <&pd_gsc>;
+		mmu-masters = <&gsc_1>;
+	};
+
+	sysmmu_gsc2: sysmmu@13EA0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13EA0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL2>, <&clock CLK_GSCL2>;
+		samsung,power-domain = <&pd_gsc>;
+		mmu-masters = <&gsc_2>;
+	};
+
+	sysmmu_gsc3: sysmmu@13EB0000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13EB0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 6>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL3>, <&clock CLK_GSCL3>;
+		samsung,power-domain = <&pd_gsc>;
+		mmu-masters = <&gsc_3>;
+	};
+
+	sysmmu_fimd1: sysmmu@14640000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x14640000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 2>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_FIMD1>;
+		samsung,power-domain = <&pd_disp1>;
+		mmu-masters = <&fimd>;
+	};
+
+	sysmmu_tv: sysmmu@14650000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x14650000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <7 4>;
+		clock-names = "sysmmu";
+		clocks = <&clock CLK_SMMU_TV>;
+		samsung,power-domain = <&pd_disp1>;
+		mmu-masters = <&mixer>;
+	};
+
+	sysmmu_jpeg: sysmmu@11F20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11F20000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_JPEG>, <&clock CLK_JPEG>;
+		samsung,power-domain = <&pd_gsc>;
+	};
+
+	sysmmu_mdma0: sysmmu@10A40000{
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x10A40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <7 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MDMA0>, <&clock CLK_MDMA0>;
+	};
+
+	sysmmu_mdma1: sysmmu@11D50000{
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11D50000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <7 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MDMA1>, <&clock CLK_MDMA1>;
+	};
 };
-- 
1.7.9.5

Re: [PATCH v12 30/31] ARM: dts: add System MMU nodes of exynos5250

[Vikas Sajjan]

Hi shaik,

+Doug, Abhilash,

On Sun, Apr 27, 2014 at 1:08 PM, Shaik Ameer Basha
<shaik.ameer-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> wrote:
> From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
>
> Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> ---
>  arch/arm/boot/dts/exynos5250.dtsi |  270 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 267 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
> index 3742331..eebd397 100644
> --- a/arch/arm/boot/dts/exynos5250.dtsi
> +++ b/arch/arm/boot/dts/exynos5250.dtsi
> @@ -82,6 +82,16 @@
>                 reg = <0x10044040 0x20>;
>         };
>
> +       pd_isp: isp-power-domain@0x10044020 {
> +               compatible = "samsung,exynos4210-pd";
> +               reg = <0x10044020 0x20>;
> +       };
> +
> +       pd_disp1: disp1-power-domain@0x100440A0 {
> +               compatible = "samsung,exynos4210-pd";
> +               reg = <0x100440A0 0x20>;
> +       };
> +

As per subject "add System MMU nodes of exynos5250", it should only
add SysMMU node.
So, I think adding power domain nodes should go in a separate patch.

Adding power domain nodes can break the system, if powering ON/OFF of
the given power domain is NOT taken care well.
I can see ISP is one such case. With this series I can see S2R breaks
[1] on 5250 chromebook with current mainline kernel (same applies for
arndale and smdk5250, but I have tested on these boards yet)

 Doug , Abhilash, Tomasz any thoughts on this.

[1]: https://chromium.googlesource.com/chromiumos/third_party/kernel-next/+/2c3f79b4ed68a54da9065a7d1a6f46d33e1df204


>         clock: clock-controller@10010000 {
>                 compatible = "samsung,exynos5250-clock";
>                 reg = <0x10010000 0x30000>;
> @@ -192,7 +202,7 @@
>                 clock-names = "fimg2d";
>         };
>
> -       codec@11000000 {
> +       mfc: codec@11000000 {
>                 compatible = "samsung,mfc-v6";
>                 reg = <0x11000000 0x10000>;
>                 interrupts = <0 96 0>;
> @@ -692,7 +702,7 @@
>                                 "sclk_hdmiphy", "mout_hdmi";
>         };
>
> -       mixer {
> +       mixer: mixer {
>                 compatible = "samsung,exynos5250-mixer";
>                 reg = <0x14450000 0x10000>;
>                 interrupts = <0 94 0>;
> @@ -713,7 +723,7 @@
>                 phy-names = "dp";
>         };
>
> -       fimd@14400000 {
> +       fimd: fimd@14400000 {
>                 clocks = <&clock CLK_SCLK_FIMD1>, <&clock CLK_FIMD1>;
>                 clock-names = "sclk_fimd", "fimd";
>         };
> @@ -736,4 +746,258 @@
>                 clocks = <&clock 348>;
>                 clock-names = "secss";
>         };
> +
> +       sysmmu_g2d: sysmmu@10A60000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x10A60000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <24 5>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_2D>;
> +       };
> +
> +       sysmmu_mfc_r: sysmmu@11200000 {
> +               compatible = "samsung,sysmmu-v2";
> +               reg = <0x11200000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <6 2>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_MFCR>, <&clock CLK_MFC>;
> +               mmu-masters = <&mfc>;
> +               samsung,power-domain = <&pd_mfc>;
> +       };
> +
> +       sysmmu_mfc_l: sysmmu@11210000 {
> +               compatible = "samsung,sysmmu-v2";
> +               reg = <0x11210000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <8 5>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_MFCL>, <&clock CLK_MFC>;
> +               mmu-masters = <&mfc>;
> +               samsung,power-domain = <&pd_mfc>;
> +       };
> +
> +       sysmmu_rotator: sysmmu@11D40000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x11D40000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <4 0>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_ROTATOR>;
> +       };
> +
> +       sysmmu_fimc_isp: sysmmu@13260000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13260000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <10 6>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_ISP>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_drc: sysmmu@13270000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13270000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <11 6>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_DRC>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_scc: sysmmu@13280000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13280000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <5 2>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_SCC>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_scp: sysmmu@13290000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13290000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <3 6>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_SCP>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_fd: sysmmu@132A0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132A0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <5 0>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_FD>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_mcuctl: sysmmu@132B0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132B0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <5 4>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_MCU>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_odc: sysmmu@132C0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132C0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <11 0>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_ODC>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_dis0: sysmmu@132D0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132D0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <10 4>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_DIS0>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_dis1: sysmmu@132E0000{
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132E0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <9 4>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_DIS1>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_3dnr: sysmmu@132F0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x132F0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <5 6>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMC_3DNR>;
> +               samsung,power-domain = <&pd_isp>;
> +       };
> +
> +       sysmmu_fimc_lite0: sysmmu@13C40000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13C40000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <3 4>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_FIMC_LITE0>, <&clock CLK_CAMIF_TOP>;
> +               samsung,power-domain = <&pd_gsc>;
> +       };
> +
> +       sysmmu_fimc_lite1: sysmmu@13C50000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13C50000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <24 1>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_FIMC_LITE1>, <&clock CLK_CAMIF_TOP>;
> +               samsung,power-domain = <&pd_gsc>;
> +       };
> +
> +       sysmmu_gsc0: sysmmu@13E80000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13E80000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <2 0>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_GSCL0>, <&clock CLK_GSCL0>;
> +               samsung,power-domain = <&pd_gsc>;
> +               mmu-masters = <&gsc_0>;
> +       };
> +
> +       sysmmu_gsc1: sysmmu@13E90000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13E90000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <2 2>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_GSCL1>, <&clock CLK_GSCL1>;
> +               samsung,power-domain = <&pd_gsc>;
> +               mmu-masters = <&gsc_1>;
> +       };
> +
> +       sysmmu_gsc2: sysmmu@13EA0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13EA0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <2 4>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_GSCL2>, <&clock CLK_GSCL2>;
> +               samsung,power-domain = <&pd_gsc>;
> +               mmu-masters = <&gsc_2>;
> +       };
> +
> +       sysmmu_gsc3: sysmmu@13EB0000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x13EB0000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <2 6>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_GSCL3>, <&clock CLK_GSCL3>;
> +               samsung,power-domain = <&pd_gsc>;
> +               mmu-masters = <&gsc_3>;
> +       };
> +
> +       sysmmu_fimd1: sysmmu@14640000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x14640000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <3 2>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_FIMD1>;
> +               samsung,power-domain = <&pd_disp1>;
> +               mmu-masters = <&fimd>;
> +       };
> +
> +       sysmmu_tv: sysmmu@14650000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x14650000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <7 4>;
> +               clock-names = "sysmmu";
> +               clocks = <&clock CLK_SMMU_TV>;
> +               samsung,power-domain = <&pd_disp1>;
> +               mmu-masters = <&mixer>;
> +       };
> +
> +       sysmmu_jpeg: sysmmu@11F20000 {
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x11F20000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <4 2>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_JPEG>, <&clock CLK_JPEG>;
> +               samsung,power-domain = <&pd_gsc>;
> +       };
> +
> +       sysmmu_mdma0: sysmmu@10A40000{
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x10A40000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <7 0>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_MDMA0>, <&clock CLK_MDMA0>;
> +       };
> +
> +       sysmmu_mdma1: sysmmu@11D50000{
> +               compatible = "samsung,sysmmu-v1";
> +               reg = <0x11D50000 0x1000>;
> +               interrupt-parent = <&combiner>;
> +               interrupts = <7 2>;
> +               clock-names = "sysmmu", "master";
> +               clocks = <&clock CLK_SMMU_MDMA1>, <&clock CLK_MDMA1>;
> +       };
>  };
> --
> 1.7.9.5
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v12 30/31] ARM: dts: add System MMU nodes of exynos5250

[Doug Anderson]

Vikas,


On Sun, Apr 27, 2014 at 10:39 AM, Vikas Sajjan <sajjan.linux@gmail.com> wrote:
> Hi shaik,
>
> +Doug, Abhilash,
>
> On Sun, Apr 27, 2014 at 1:08 PM, Shaik Ameer Basha
> <shaik.ameer@samsung.com> wrote:
>> From: Cho KyongHo <pullip.cho@samsung.com>
>>
>> Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
>> ---
>>  arch/arm/boot/dts/exynos5250.dtsi |  270 ++++++++++++++++++++++++++++++++++++-
>>  1 file changed, 267 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
>> index 3742331..eebd397 100644
>> --- a/arch/arm/boot/dts/exynos5250.dtsi
>> +++ b/arch/arm/boot/dts/exynos5250.dtsi
>> @@ -82,6 +82,16 @@
>>                 reg = <0x10044040 0x20>;
>>         };
>>
>> +       pd_isp: isp-power-domain@0x10044020 {
>> +               compatible = "samsung,exynos4210-pd";
>> +               reg = <0x10044020 0x20>;
>> +       };
>> +
>> +       pd_disp1: disp1-power-domain@0x100440A0 {
>> +               compatible = "samsung,exynos4210-pd";
>> +               reg = <0x100440A0 0x20>;
>> +       };
>> +
>
> As per subject "add System MMU nodes of exynos5250", it should only
> add SysMMU node.
> So, I think adding power domain nodes should go in a separate patch.
>
> Adding power domain nodes can break the system, if powering ON/OFF of
> the given power domain is NOT taken care well.
> I can see ISP is one such case. With this series I can see S2R breaks
> [1] on 5250 chromebook with current mainline kernel (same applies for
> arndale and smdk5250, but I have tested on these boards yet)

Thanks for catching that!  Let's make sure not to break suspend/resume.

Given that these power domains are actually used as the domains for
some of the System MMU nodes I don't totally object to adding them in
the same patch, but if they're breaking things then I agree that we
could leave them out and add them in a later patch (once issues are
sorted out).

-Doug

Re: [PATCH v12 30/31] ARM: dts: add System MMU nodes of exynos5250

[Cho KyongHo]

On Mon, 28 Apr 2014 16:13:19 -0700, Doug Anderson wrote:
> Vikas,
> 
> 
> On Sun, Apr 27, 2014 at 10:39 AM, Vikas Sajjan <sajjan.linux-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Hi shaik,
> >
> > +Doug, Abhilash,
> >
> > On Sun, Apr 27, 2014 at 1:08 PM, Shaik Ameer Basha
> > <shaik.ameer-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org> wrote:
> >> From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> >>
> >> Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> >> ---
> >>  arch/arm/boot/dts/exynos5250.dtsi |  270 ++++++++++++++++++++++++++++++++++++-
> >>  1 file changed, 267 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
> >> index 3742331..eebd397 100644
> >> --- a/arch/arm/boot/dts/exynos5250.dtsi
> >> +++ b/arch/arm/boot/dts/exynos5250.dtsi
> >> @@ -82,6 +82,16 @@
> >>                 reg = <0x10044040 0x20>;
> >>         };
> >>
> >> +       pd_isp: isp-power-domain@0x10044020 {
> >> +               compatible = "samsung,exynos4210-pd";
> >> +               reg = <0x10044020 0x20>;
> >> +       };
> >> +
> >> +       pd_disp1: disp1-power-domain@0x100440A0 {
> >> +               compatible = "samsung,exynos4210-pd";
> >> +               reg = <0x100440A0 0x20>;
> >> +       };
> >> +
> >
> > As per subject "add System MMU nodes of exynos5250", it should only
> > add SysMMU node.
> > So, I think adding power domain nodes should go in a separate patch.
> >
> > Adding power domain nodes can break the system, if powering ON/OFF of
> > the given power domain is NOT taken care well.
> > I can see ISP is one such case. With this series I can see S2R breaks
> > [1] on 5250 chromebook with current mainline kernel (same applies for
> > arndale and smdk5250, but I have tested on these boards yet)
> 
> Thanks for catching that!  Let's make sure not to break suspend/resume.
> 
> Given that these power domains are actually used as the domains for
> some of the System MMU nodes I don't totally object to adding them in
> the same patch, but if they're breaking things then I agree that we
> could leave them out and add them in a later patch (once issues are
> sorted out).
> 

Thank you, Vikas.

Let me consider adding ISP power doamin after checking
power down sequence of ISP domain.

KyongHo

[PATCH v12 31/31] ARM: dts: add System MMU nodes of exynos5420

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

This patch adds System MMU nodes of exynos5420 except
System MMUs in Image Subsystem.

Signed-off-by: Cho KyongHo <pullip.cho-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Shaik Ameer Basha <shaik.ameer-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/exynos5420.dtsi |  209 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 206 insertions(+), 3 deletions(-)

diff --git a/arch/arm/boot/dts/exynos5420.dtsi b/arch/arm/boot/dts/exynos5420.dtsi
index c3a9a66..1fc0c9f 100644
--- a/arch/arm/boot/dts/exynos5420.dtsi
+++ b/arch/arm/boot/dts/exynos5420.dtsi
@@ -125,7 +125,7 @@
 		clock-names = "pll_ref", "pll_in", "sclk_audio", "sclk_pcm_in";
 	};
 
-	codec@11000000 {
+	mfc: codec@11000000 {
 		compatible = "samsung,mfc-v7";
 		reg = <0x11000000 0x10000>;
 		interrupts = <0 96 0>;
@@ -472,7 +472,7 @@
 		phy-names = "dp";
 	};
 
-	fimd@14400000 {
+	fimd: fimd@14400000 {
 		samsung,power-domain = <&disp_pd>;
 		clocks = <&clock CLK_SCLK_FIMD1>, <&clock CLK_FIMD1>;
 		clock-names = "sclk_fimd", "fimd";
@@ -644,7 +644,7 @@
 		status = "disabled";
 	};
 
-	mixer@14450000 {
+	mixer: mixer@14450000 {
 		compatible = "samsung,exynos5420-mixer";
 		reg = <0x14450000 0x10000>;
 		interrupts = <0 94 0>;
@@ -732,4 +732,207 @@
 		clock-names = "secss";
 		samsung,power-domain = <&g2d_pd>;
 	};
+
+	sysmmu_g2dr: sysmmu@10A60000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x10A60000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <24 5>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_G2D>, <&clock CLK_G2D>;
+	};
+
+	sysmmu_g2dw: sysmmu@10A70000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x10A70000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <22 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_G2D>, <&clock CLK_G2D>;
+	};
+
+	sysmmu_scaler0r: sysmmu@12880000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x12880000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <22 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL0>, <&clock CLK_MSCL0>;
+	};
+
+	sysmmu_scaler1r: sysmmu@12890000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x12890000 0x1000>;
+		interrupts = <0 186 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL1>, <&clock CLK_MSCL1>;
+	};
+
+	sysmmu_scaler2r: sysmmu@128A0000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x128A0000 0x1000>;
+		interrupts = <0 188 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL2>, <&clock CLK_MSCL2>;
+	};
+
+	sysmmu_scaler0w: sysmmu@128C0000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x128C0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <27 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL0>, <&clock CLK_MSCL0>;
+	};
+
+	sysmmu_scaler1w: sysmmu@128D0000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x128D0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <22 6>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL1>, <&clock CLK_MSCL1>;
+	};
+
+	sysmmu_scaler2w: sysmmu@128E0000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x128E0000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <19 6>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MSCL2>, <&clock CLK_MSCL2>;
+	};
+
+	sysmmu_mfc_l: sysmmu@11200000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x11200000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <8 5>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MFCL>, <&clock CLK_MFC>;
+		mmu-masters = <&mfc>;
+		samsung,power-domain = <&mfc_pd>;
+	};
+
+	sysmmu_mfc_r: sysmmu@11210000 {
+		compatible = "samsung,sysmmu-v2";
+		reg = <0x11210000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <6 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MFCR>, <&clock CLK_MFC>;
+		mmu-masters = <&mfc>;
+		samsung,power-domain = <&mfc_pd>;
+	};
+
+	sysmmu_rotator: sysmmu@11D40000 {
+		compatible = "samsung,sysmmu-v3.1";
+		reg = <0x11D40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_ROTATOR>, <&clock CLK_ROTATOR>;
+	};
+
+	sysmmu_fimc_lite0: sysmmu@13C40000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x13C40000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMCL0>, <&clock CLK_FIMC_LITE0>;
+		samsung,power-domain = <&gsc_pd>;
+	};
+
+	sysmmu_fimc_lite1: sysmmu@13C50000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x13C50000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <24 1>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMCL1>, <&clock CLK_FIMC_LITE1>;
+		samsung,power-domain = <&gsc_pd>;
+	};
+
+	sysmmu_fimc_lite3: sysmmu@13D50000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x13D50000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMCL3>, <&clock CLK_FIMC_LITE3>;
+		samsung,power-domain = <&gsc_pd>;
+	};
+
+	sysmmu_gsc0: sysmmu@13E80000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x13E80000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL0>, <&clock CLK_GSCL0>;
+		samsung,power-domain = <&gsc_pd>;
+		mmu-masters = <&gsc_0>;
+	};
+
+	sysmmu_gsc1: sysmmu@13E90000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x13E90000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL1>, <&clock CLK_GSCL1>;
+		samsung,power-domain = <&gsc_pd>;
+		mmu-masters = <&gsc_1>;
+	};
+
+	sysmmu_fimd0w04: sysmmu@14640000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x14640000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMD1M0>, <&clock CLK_FIMD1>;
+		samsung,power-domain = <&disp_pd>;
+		mmu-masters = <&fimd>;
+	};
+
+	sysmmu_fimd0w123: sysmmu@14680000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x14680000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <3 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_FIMD1M1>, <&clock CLK_FIMD1>;
+		samsung,power-domain = <&disp_pd>;
+		mmu-masters = <&fimd>;
+	};
+
+	sysmmu_tv: sysmmu@14650000 {
+		compatible = "samsung,sysmmu-v3.3";
+		reg = <0x14650000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <7 4>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_MIXER>, <&clock CLK_MIXER>;
+		samsung,power-domain = <&disp_pd>;
+		mmu-masters = <&mixer>;
+	};
+
+	sysmmu_jpeg: sysmmu@11F10000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11F10000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <4 2>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_JPEG>, <&clock CLK_JPEG>;
+	};
+
+	sysmmu_jpeg2: sysmmu@11F20000 {
+		compatible = "samsung,sysmmu-v1";
+		reg = <0x11F20000 0x1000>;
+		interrupts = <0 169 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_JPEG>, <&clock CLK_JPEG2>;
+	};
 };
-- 
1.7.9.5

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Arnd Bergmann]

On Sunday 27 April 2014 13:07:32 Shaik Ameer Basha wrote:
> The current exynos-iommu(System MMU) driver does not work autonomously
> since it is lack of support for power management of peripheral blocks.
> For example, MFC device driver must ensure that its System MMU is disabled
> before MFC block is power-down not to invalidate IOTLB in the System MMU
> when I/O memory mapping is changed. Because a System MMU resides in the
> same H/W block, access to control registers of System MMU while the H/W
> block is turned off must be prohibited.
> 
> This set of changes solves the above problem with setting each System MMUs
> as the parent of the device which owns the System MMU to receive the
> information when the device is turned off or turned on.
> 
> Another big change to the driver is the support for devicetree.
> The bindings for System MMU is described in
> Documentation/devicetree/bindings/arm/samsung/system-mmu.txt

Sorry I've been absent from the review so far. Most of the patches
seem entirely reasonable to me, but I'm worried about the DT binding
aspect. We are going to see more systems shipping with IOMMUs now,
and we are seeing an increasing number of submissions for 64-bit
systems. We really have to work out what the DT representation for
IOMMUs should look like in general before adding another ad-hod
implementation that is private to one driver.

	Arnd

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Shaik Ameer Basha]

On Mon, Apr 28, 2014 at 2:04 PM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote:
> On Sunday 27 April 2014 13:07:32 Shaik Ameer Basha wrote:
>> The current exynos-iommu(System MMU) driver does not work autonomously
>> since it is lack of support for power management of peripheral blocks.
>> For example, MFC device driver must ensure that its System MMU is disabled
>> before MFC block is power-down not to invalidate IOTLB in the System MMU
>> when I/O memory mapping is changed. Because a System MMU resides in the
>> same H/W block, access to control registers of System MMU while the H/W
>> block is turned off must be prohibited.
>>
>> This set of changes solves the above problem with setting each System MMUs
>> as the parent of the device which owns the System MMU to receive the
>> information when the device is turned off or turned on.
>>
>> Another big change to the driver is the support for devicetree.
>> The bindings for System MMU is described in
>> Documentation/devicetree/bindings/arm/samsung/system-mmu.txt
>
> Sorry I've been absent from the review so far. Most of the patches
> seem entirely reasonable to me, but I'm worried about the DT binding
> aspect. We are going to see more systems shipping with IOMMUs now,
> and we are seeing an increasing number of submissions for 64-bit
> systems. We really have to work out what the DT representation for
> IOMMUs should look like in general before adding another ad-hod
> implementation that is private to one driver.

Hi Arnd,

No issues. Its good that finally you are here :)

I am going through the possibilities for new bindings that you
mentioned in the other thread.
    -- [PATCH v12 11/31] documentation: iommu: add binding document of
Exynos System MMU

Exynos IOMMU driver is pretty simple with only one exception, "some
devices are using multiple IOMMUs".

>From starting (of this patch set), we were trying to fix three major issues.
[1] How to control the probing order of required IOMMU(s) for a given
device and a device itself.
[2] Handling multiple IOMMUs for one device.
[3] Generic DT bindings to link Device and IOMMUs.

I have gone through the implementation of Tegra SMMU driver by "Hiroshi Doyu"
    -- [PATCHv7 00/12] Unifying SMMU driver among Tegra SoCs
[https://lkml.org/lkml/2013/12/12/74]

For the first point [1],
------------------------------
Tegra implementation tries to fix this issue with these two patches
    -- iommu/of: check if dependee iommu is ready or not
[http://patchwork.ozlabs.org/patch/300560/]
    -- driver/core: populate devices in order for IOMMUs
[http://patchwork.ozlabs.org/patch/300558/]
I can follow this driver if this approach is acceptable.

For the second point [2]
----------------------------------
Currently we are handling this issue by providing same mapping for all
IOMMUs linked to the same device.
And current Exynos drivers doesn't have any special implementation to
handle this case differently.

I thought of understanding how Tegra SMMU driver is handling this case.
Frankly speaking, I didn't understand how its done there.

For the third point [3]
-------------------------------
As Tegra SMMU driver is inline with the discussion in other thread, we
can follow the same bindings, unless
the discussion takes us in the other direction.

"KyongHo Cho" is the author for this driver and hope he has more inputs.

Regards,
Shaik



>
>         Arnd
> --
> To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Shaik Ameer Basha]

On Mon, Apr 28, 2014 at 2:04 PM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote:
> On Sunday 27 April 2014 13:07:32 Shaik Ameer Basha wrote:
>> The current exynos-iommu(System MMU) driver does not work autonomously
>> since it is lack of support for power management of peripheral blocks.
>> For example, MFC device driver must ensure that its System MMU is disabled
>> before MFC block is power-down not to invalidate IOTLB in the System MMU
>> when I/O memory mapping is changed. Because a System MMU resides in the
>> same H/W block, access to control registers of System MMU while the H/W
>> block is turned off must be prohibited.
>>
>> This set of changes solves the above problem with setting each System MMUs
>> as the parent of the device which owns the System MMU to receive the
>> information when the device is turned off or turned on.
>>
>> Another big change to the driver is the support for devicetree.
>> The bindings for System MMU is described in
>> Documentation/devicetree/bindings/arm/samsung/system-mmu.txt
>
> Sorry I've been absent from the review so far. Most of the patches
> seem entirely reasonable to me, but I'm worried about the DT binding
> aspect. We are going to see more systems shipping with IOMMUs now,
> and we are seeing an increasing number of submissions for 64-bit
> systems. We really have to work out what the DT representation for
> IOMMUs should look like in general before adding another ad-hod
> implementation that is private to one driver.


I have one question.

This series is going on for quite a long time and most of the patches here
doesn't depend on dt bindings. As Exynos IOMMU h/w is introducing new versions
very frequently, maintaining and reviewing all these patches again and
again is quite a hard job.

If it is acceptable, I can post one more series with the subset of
above patches,
which doesn't depend on dt-bindings. As all the patches which doesn't depend on
DT bindings are already tested,  I hope merging these subset of patches may help
in reducing the rework and review effort every time.

Once we finalize the generic DT bindings for the IOMMU devices, the driver
can be updated with the proposed DT bindings in mind.

Regards,
Shaik


>
>         Arnd
> --
> To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Joerg Roedel]

On Wed, Apr 30, 2014 at 04:27:10PM +0530, Shaik Ameer Basha wrote:
> This series is going on for quite a long time and most of the patches here
> doesn't depend on dt bindings. As Exynos IOMMU h/w is introducing new versions
> very frequently, maintaining and reviewing all these patches again and
> again is quite a hard job.
> 
> If it is acceptable, I can post one more series with the subset of
> above patches,
> which doesn't depend on dt-bindings. As all the patches which doesn't depend on
> DT bindings are already tested,  I hope merging these subset of patches may help
> in reducing the rework and review effort every time.
> 
> Once we finalize the generic DT bindings for the IOMMU devices, the driver
> can be updated with the proposed DT bindings in mind.

Sounds reasonable to me if the patch subset is self-contained. When
Arnd is OK with that and no one else has objections I can take the
dt-independend parts.


	Joerg

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Tomasz Figa]

On 06.05.2014 19:59, Joerg Roedel wrote:
> On Wed, Apr 30, 2014 at 04:27:10PM +0530, Shaik Ameer Basha wrote:
>> This series is going on for quite a long time and most of the patches here
>> doesn't depend on dt bindings. As Exynos IOMMU h/w is introducing new versions
>> very frequently, maintaining and reviewing all these patches again and
>> again is quite a hard job.
>>
>> If it is acceptable, I can post one more series with the subset of
>> above patches,
>> which doesn't depend on dt-bindings. As all the patches which doesn't depend on
>> DT bindings are already tested,  I hope merging these subset of patches may help
>> in reducing the rework and review effort every time.
>>
>> Once we finalize the generic DT bindings for the IOMMU devices, the driver
>> can be updated with the proposed DT bindings in mind.
>
> Sounds reasonable to me if the patch subset is self-contained. When
> Arnd is OK with that and no one else has objections I can take the
> dt-independend parts.

+1.

I would also suggest dropping patch 18/31, for the reasons I mentioned 
in my reply to it.

Best regards,
Tomasz
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Cho KyongHo]

On Tue, 06 May 2014 20:08:47 +0200, Tomasz Figa wrote:
> On 06.05.2014 19:59, Joerg Roedel wrote:
> > On Wed, Apr 30, 2014 at 04:27:10PM +0530, Shaik Ameer Basha wrote:
> >> This series is going on for quite a long time and most of the patches here
> >> doesn't depend on dt bindings. As Exynos IOMMU h/w is introducing new versions
> >> very frequently, maintaining and reviewing all these patches again and
> >> again is quite a hard job.
> >>
> >> If it is acceptable, I can post one more series with the subset of
> >> above patches,
> >> which doesn't depend on dt-bindings. As all the patches which doesn't depend on
> >> DT bindings are already tested,  I hope merging these subset of patches may help
> >> in reducing the rework and review effort every time.
> >>
> >> Once we finalize the generic DT bindings for the IOMMU devices, the driver
> >> can be updated with the proposed DT bindings in mind.
> >
> > Sounds reasonable to me if the patch subset is self-contained. When
> > Arnd is OK with that and no one else has objections I can take the
> > dt-independend parts.
> 
> +1.
> 
> I would also suggest dropping patch 18/31, for the reasons I mentioned 
> in my reply to it.
> 

Great.

We will prepare the next patch series that are independent upon DT bindings soon.

Thank you.

KyongHo

Re: [PATCH v12 00/31] iommu/exynos: Fixes and Enhancements of System MMU driver with DT

[Arnd Bergmann]

On Tuesday 06 May 2014 19:59:05 Joerg Roedel wrote:
> On Wed, Apr 30, 2014 at 04:27:10PM +0530, Shaik Ameer Basha wrote:
> > This series is going on for quite a long time and most of the patches here
> > doesn't depend on dt bindings. As Exynos IOMMU h/w is introducing new versions
> > very frequently, maintaining and reviewing all these patches again and
> > again is quite a hard job.
> > 
> > If it is acceptable, I can post one more series with the subset of
> > above patches,
> > which doesn't depend on dt-bindings. As all the patches which doesn't depend on
> > DT bindings are already tested,  I hope merging these subset of patches may help
> > in reducing the rework and review effort every time.
> > 
> > Once we finalize the generic DT bindings for the IOMMU devices, the driver
> > can be updated with the proposed DT bindings in mind.
> 
> Sounds reasonable to me if the patch subset is self-contained. When
> Arnd is OK with that and no one else has objections I can take the
> dt-independend parts.

Sounds good to me.

	Arnd

[PATCH v12 02/31] iommu/exynos: add missing cache flush for removed page table entries

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This commit adds cache flush for removed small and large page entries
in exynos_iommu_unmap(). Missing cache flush of removed page table
entries can cause missing page fault interrupt when a master IP
accesses an unmapped area.

Reviewed-by: Tomasz Figa <t.figa@samsung.com>
Tested-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 46f0ca1..34e4273 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -957,6 +957,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	if (lv2ent_small(ent)) {
 		*ent = 0;
 		size = SPAGE_SIZE;
+		pgtable_flush(ent, ent + 1);
 		priv->lv2entcnt[lv1ent_offset(iova)] += 1;
 		goto done;
 	}
@@ -965,6 +966,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 	BUG_ON(size < LPAGE_SIZE);
 
 	memset(ent, 0, sizeof(*ent) * SPAGES_PER_LPAGE);
+	pgtable_flush(ent, ent + SPAGES_PER_LPAGE);
 
 	size = LPAGE_SIZE;
 	priv->lv2entcnt[lv1ent_offset(iova)] += SPAGES_PER_LPAGE;
-- 
1.7.9.5

[PATCH v12 04/31] iommu/exynos: fix L2TLB invalidation

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

L2TLB is 8-way set-associative TLB with 512 entries. The number of
sets is 64.
A single 4KB(small page) translation information is cached
only to a set whose index is the same with the lower 6 bits of the page
frame number.
A single 64KB(large page) translation information can be
cached to any 16 sets whose top two bits of their indices are the same
with the bit [5:4] of the page frame number.
A single 1MB(section) or larger translation information can be cached to
any set in the TLB.

It is required to invalidate entire sets that may cache the target
translation information to guarantee that the L2TLB has no stale data.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 84fc3b4..7ce44a8 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -224,9 +224,14 @@ static void __sysmmu_tlb_invalidate(void __iomem *sfrbase)
 }
 
 static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase,
-						unsigned long iova)
+				unsigned long iova, unsigned int num_inv)
 {
-	__raw_writel((iova & SPAGE_MASK) | 1, sfrbase + REG_MMU_FLUSH_ENTRY);
+	unsigned int i;
+	for (i = 0; i < num_inv; i++) {
+		__raw_writel((iova & SPAGE_MASK) | 1,
+				sfrbase + REG_MMU_FLUSH_ENTRY);
+		iova += SPAGE_SIZE;
+	}
 }
 
 static void __sysmmu_set_ptbase(void __iomem *sfrbase,
@@ -476,7 +481,8 @@ static bool exynos_sysmmu_disable(struct device *dev)
 	return disabled;
 }
 
-static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova)
+static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
+					size_t size)
 {
 	unsigned long flags;
 	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
@@ -486,9 +492,24 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova)
 	if (is_sysmmu_active(data)) {
 		int i;
 		for (i = 0; i < data->nsfrs; i++) {
+			unsigned int maj;
+			unsigned int num_inv = 1;
+			maj = __raw_readl(data->sfrbases[i] + REG_MMU_VERSION);
+			/*
+			 * L2TLB invalidation required
+			 * 4KB page: 1 invalidation
+			 * 64KB page: 16 invalidation
+			 * 1MB page: 64 invalidation
+			 * because it is set-associative TLB
+			 * with 8-way and 64 sets.
+			 * 1MB page can be cached in one of all sets.
+			 * 64KB page can be one of 16 consecutive sets.
+			 */
+			if ((maj >> 28) == 2) /* major version number */
+				num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
 			if (sysmmu_block(data->sfrbases[i])) {
 				__sysmmu_tlb_invalidate_entry(
-						data->sfrbases[i], iova);
+					data->sfrbases[i], iova, num_inv);
 				sysmmu_unblock(data->sfrbases[i]);
 			}
 		}
@@ -998,7 +1019,7 @@ done:
 
 	spin_lock_irqsave(&priv->lock, flags);
 	list_for_each_entry(data, &priv->clients, node)
-		sysmmu_tlb_invalidate_entry(data->dev, iova);
+		sysmmu_tlb_invalidate_entry(data->dev, iova, size);
 	spin_unlock_irqrestore(&priv->lock, flags);
 
 	return size;
-- 
1.7.9.5

[PATCH v12 06/31] iommu/exynos: allocate lv2 page table from own slab

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

Since kmalloc() does not guarantee that the allignment of 1KiB when it
allocates 1KiB, it is required to allocate lv2 page table from own
slab that guarantees alignment of 1KiB

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   34 ++++++++++++++++++++++++++++------
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 7556177..c7f831c 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -99,6 +99,8 @@
 #define REG_PB1_SADDR		0x054
 #define REG_PB1_EADDR		0x058
 
+static struct kmem_cache *lv2table_kmem_cache;
+
 static unsigned long *section_entry(unsigned long *pgtable, unsigned long iova)
 {
 	return pgtable + lv1ent_offset(iova);
@@ -725,7 +727,8 @@ static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
 
 	for (i = 0; i < NUM_LV1ENTRIES; i++)
 		if (lv1ent_page(priv->pgtable + i))
-			kfree(__va(lv2table_base(priv->pgtable + i)));
+			kmem_cache_free(lv2table_kmem_cache,
+					__va(lv2table_base(priv->pgtable + i)));
 
 	free_pages((unsigned long)priv->pgtable, 2);
 	free_pages((unsigned long)priv->lv2entcnt, 1);
@@ -824,7 +827,7 @@ static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
 	if (lv1ent_fault(sent)) {
 		unsigned long *pent;
 
-		pent = kzalloc(LV2TABLE_SIZE, GFP_ATOMIC);
+		pent = kmem_cache_zalloc(lv2table_kmem_cache, GFP_ATOMIC);
 		BUG_ON((unsigned long)pent & (LV2TABLE_SIZE - 1));
 		if (!pent)
 			return ERR_PTR(-ENOMEM);
@@ -854,8 +857,7 @@ static int lv1set_section(unsigned long *sent, unsigned long iova,
 			return -EADDRINUSE;
 		}
 
-		kfree(page_entry(sent, 0));
-
+		kmem_cache_free(lv2table_kmem_cache, page_entry(sent, 0));
 		*pgcnt = 0;
 	}
 
@@ -1060,11 +1062,31 @@ static int __init exynos_iommu_init(void)
 {
 	int ret;
 
+	lv2table_kmem_cache = kmem_cache_create("exynos-iommu-lv2table",
+				LV2TABLE_SIZE, LV2TABLE_SIZE, 0, NULL);
+	if (!lv2table_kmem_cache) {
+		pr_err("%s: Failed to create kmem cache\n", __func__);
+		return -ENOMEM;
+	}
+
 	ret = platform_driver_register(&exynos_sysmmu_driver);
+	if (ret) {
+		pr_err("%s: Failed to register driver\n", __func__);
+		goto err_reg_driver;
+	}
 
-	if (ret == 0)
-		bus_set_iommu(&platform_bus_type, &exynos_iommu_ops);
+	ret = bus_set_iommu(&platform_bus_type, &exynos_iommu_ops);
+	if (ret) {
+		pr_err("%s: Failed to register exynos-iommu driver.\n",
+								__func__);
+		goto err_set_iommu;
+	}
 
+	return 0;
+err_set_iommu:
+	platform_driver_unregister(&exynos_sysmmu_driver);
+err_reg_driver:
+	kmem_cache_destroy(lv2table_kmem_cache);
 	return ret;
 }
 subsys_initcall(exynos_iommu_init);
-- 
1.7.9.5

[PATCH v12 07/31] iommu/exynos: always enable runtime PM

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

Checking if the probing device has a parent device was just to discover
if the probing device is involved in a power domain when the power
domain controlled by Samsung's custom implementation.
Since generic IO power domain is applied, it is required to remove
the condition to see if the probing device has a parent device.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index c7f831c..d466076 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -631,8 +631,7 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 
 	__set_fault_handler(data, &default_fault_handler);
 
-	if (dev->parent)
-		pm_runtime_enable(dev);
+	pm_runtime_enable(dev);
 
 	dev_dbg(dev, "(%s) Initialized\n", data->dbgname);
 	return 0;
-- 
1.7.9.5

[PATCH v12 08/31] iommu/exynos: handle one instance of sysmmu with a device descriptor

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

System MMU driver is changed to control only a single instance of
System MMU at a time. Since a single instance of System MMU has only
a single clock descriptor for its clock gating, single address range
for control registers, there is no need to obtain two or more clock
descriptors and ioremaped region.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |  223 ++++++++++++++----------------------------
 1 file changed, 71 insertions(+), 152 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index d466076..a15216f 100644
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -170,9 +170,8 @@ struct sysmmu_drvdata {
 	struct device *sysmmu;	/* System MMU's device descriptor */
 	struct device *dev;	/* Owner of system MMU */
 	char *dbgname;
-	int nsfrs;
-	void __iomem **sfrbases;
-	struct clk *clk[2];
+	void __iomem *sfrbase;
+	struct clk *clk;
 	int activations;
 	rwlock_t lock;
 	struct iommu_domain *domain;
@@ -293,56 +292,39 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 {
 	/* SYSMMU is in blocked when interrupt occurred. */
 	struct sysmmu_drvdata *data = dev_id;
-	struct resource *irqres;
-	struct platform_device *pdev;
 	enum exynos_sysmmu_inttype itype;
 	unsigned long addr = -1;
-
-	int i, ret = -ENOSYS;
+	int ret = -ENOSYS;
 
 	read_lock(&data->lock);
 
 	WARN_ON(!is_sysmmu_active(data));
 
-	pdev = to_platform_device(data->sysmmu);
-	for (i = 0; i < (pdev->num_resources / 2); i++) {
-		irqres = platform_get_resource(pdev, IORESOURCE_IRQ, i);
-		if (irqres && ((int)irqres->start == irq))
-			break;
-	}
-
-	if (i == pdev->num_resources) {
+	itype = (enum exynos_sysmmu_inttype)
+		__ffs(__raw_readl(data->sfrbase + REG_INT_STATUS));
+	if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
 		itype = SYSMMU_FAULT_UNKNOWN;
-	} else {
-		itype = (enum exynos_sysmmu_inttype)
-			__ffs(__raw_readl(data->sfrbases[i] + REG_INT_STATUS));
-		if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
-			itype = SYSMMU_FAULT_UNKNOWN;
-		else
-			addr = __raw_readl(
-				data->sfrbases[i] + fault_reg_offset[itype]);
-	}
+	else
+		addr = __raw_readl(data->sfrbase + fault_reg_offset[itype]);
 
 	if (data->domain)
-		ret = report_iommu_fault(data->domain, data->dev,
-				addr, itype);
+		ret = report_iommu_fault(data->domain, data->dev, addr, itype);
 
 	if ((ret == -ENOSYS) && data->fault_handler) {
 		unsigned long base = data->pgtable;
 		if (itype != SYSMMU_FAULT_UNKNOWN)
-			base = __raw_readl(
-					data->sfrbases[i] + REG_PT_BASE_ADDR);
+			base = __raw_readl(data->sfrbase + REG_PT_BASE_ADDR);
 		ret = data->fault_handler(itype, base, addr);
 	}
 
 	if (!ret && (itype != SYSMMU_FAULT_UNKNOWN))
-		__raw_writel(1 << itype, data->sfrbases[i] + REG_INT_CLEAR);
+		__raw_writel(1 << itype, data->sfrbase + REG_INT_CLEAR);
 	else
 		dev_dbg(data->sysmmu, "(%s) %s is not handled.\n",
 				data->dbgname, sysmmu_fault_name[itype]);
 
 	if (itype != SYSMMU_FAULT_UNKNOWN)
-		sysmmu_unblock(data->sfrbases[i]);
+		sysmmu_unblock(data->sfrbase);
 
 	read_unlock(&data->lock);
 
@@ -353,20 +335,16 @@ static bool __exynos_sysmmu_disable(struct sysmmu_drvdata *data)
 {
 	unsigned long flags;
 	bool disabled = false;
-	int i;
 
 	write_lock_irqsave(&data->lock, flags);
 
 	if (!set_sysmmu_inactive(data))
 		goto finish;
 
-	for (i = 0; i < data->nsfrs; i++)
-		__raw_writel(CTRL_DISABLE, data->sfrbases[i] + REG_MMU_CTRL);
+	__raw_writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
 
-	if (data->clk[1])
-		clk_disable(data->clk[1]);
-	if (data->clk[0])
-		clk_disable(data->clk[0]);
+	if (!IS_ERR(data->clk))
+		clk_disable(data->clk);
 
 	disabled = true;
 	data->pgtable = 0;
@@ -392,7 +370,7 @@ finish:
 static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 			unsigned long pgtable, struct iommu_domain *domain)
 {
-	int i, ret = 0;
+	int ret = 0;
 	unsigned long flags;
 
 	write_lock_irqsave(&data->lock, flags);
@@ -409,17 +387,14 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 		goto finish;
 	}
 
-	if (data->clk[0])
-		clk_enable(data->clk[0]);
-	if (data->clk[1])
-		clk_enable(data->clk[1]);
+	if (!IS_ERR(data->clk))
+		clk_enable(data->clk);
 
 	data->pgtable = pgtable;
 
-	for (i = 0; i < data->nsfrs; i++) {
-		__sysmmu_set_ptbase(data->sfrbases[i], pgtable);
-		__raw_writel(CTRL_ENABLE, data->sfrbases[i] + REG_MMU_CTRL);
-	}
+	__sysmmu_set_ptbase(data->sfrbase, pgtable);
+
+	__raw_writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL);
 
 	data->domain = domain;
 
@@ -476,28 +451,26 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 	read_lock_irqsave(&data->lock, flags);
 
 	if (is_sysmmu_active(data)) {
-		int i;
-		for (i = 0; i < data->nsfrs; i++) {
-			unsigned int maj;
-			unsigned int num_inv = 1;
-			maj = __raw_readl(data->sfrbases[i] + REG_MMU_VERSION);
-			/*
-			 * L2TLB invalidation required
-			 * 4KB page: 1 invalidation
-			 * 64KB page: 16 invalidation
-			 * 1MB page: 64 invalidation
-			 * because it is set-associative TLB
-			 * with 8-way and 64 sets.
-			 * 1MB page can be cached in one of all sets.
-			 * 64KB page can be one of 16 consecutive sets.
-			 */
-			if ((maj >> 28) == 2) /* major version number */
-				num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
-			if (sysmmu_block(data->sfrbases[i])) {
-				__sysmmu_tlb_invalidate_entry(
-					data->sfrbases[i], iova, num_inv);
-				sysmmu_unblock(data->sfrbases[i]);
-			}
+		unsigned int maj;
+		unsigned int num_inv = 1;
+		maj = __raw_readl(data->sfrbase + REG_MMU_VERSION);
+		/*
+		 * L2TLB invalidation required
+		 * 4KB page: 1 invalidation
+		 * 64KB page: 16 invalidation
+		 * 1MB page: 64 invalidation
+		 * because it is set-associative TLB
+		 * with 8-way and 64 sets.
+		 * 1MB page can be cached in one of all sets.
+		 * 64KB page can be one of 16 consecutive sets.
+		 */
+		if ((maj >> 28) == 2) /* major version number */
+			num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
+
+		if (sysmmu_block(data->sfrbase)) {
+			__sysmmu_tlb_invalidate_entry(data->sfrbase, iova,
+							num_inv);
+			sysmmu_unblock(data->sfrbase);
 		}
 	} else {
 		dev_dbg(data->sysmmu,
@@ -516,12 +489,9 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 	read_lock_irqsave(&data->lock, flags);
 
 	if (is_sysmmu_active(data)) {
-		int i;
-		for (i = 0; i < data->nsfrs; i++) {
-			if (sysmmu_block(data->sfrbases[i])) {
-				__sysmmu_tlb_invalidate(data->sfrbases[i]);
-				sysmmu_unblock(data->sfrbases[i]);
-			}
+		if (sysmmu_block(data->sfrbase)) {
+			__sysmmu_tlb_invalidate(data->sfrbase);
+			sysmmu_unblock(data->sfrbase);
 		}
 	} else {
 		dev_dbg(data->sysmmu,
@@ -534,11 +504,10 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 
 static int exynos_sysmmu_probe(struct platform_device *pdev)
 {
-	int i, ret;
-	struct device *dev;
+	int ret;
+	struct device *dev = &pdev->dev;
 	struct sysmmu_drvdata *data;
-
-	dev = &pdev->dev;
+	struct resource *res;
 
 	data = kzalloc(sizeof(*data), GFP_KERNEL);
 	if (!data) {
@@ -547,82 +516,37 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 		goto err_alloc;
 	}
 
-	ret = dev_set_drvdata(dev, data);
-	if (ret) {
-		dev_dbg(dev, "Unabled to initialize driver data\n");
+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	if (!res) {
+		dev_dbg(dev, "Unable to find IOMEM region\n");
+		ret = -ENOENT;
 		goto err_init;
 	}
 
-	data->nsfrs = pdev->num_resources / 2;
-	data->sfrbases = kmalloc(sizeof(*data->sfrbases) * data->nsfrs,
-								GFP_KERNEL);
-	if (data->sfrbases == NULL) {
-		dev_dbg(dev, "Not enough memory\n");
-		ret = -ENOMEM;
-		goto err_init;
+	data->sfrbase = ioremap(res->start, resource_size(res));
+	if (!data->sfrbase) {
+		dev_dbg(dev, "Unable to map IOMEM @ PA:%#x\n", res->start);
+		ret = -ENOENT;
+		goto err_res;
 	}
 
-	for (i = 0; i < data->nsfrs; i++) {
-		struct resource *res;
-		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
-		if (!res) {
-			dev_dbg(dev, "Unable to find IOMEM region\n");
-			ret = -ENOENT;
-			goto err_res;
-		}
-
-		data->sfrbases[i] = ioremap(res->start, resource_size(res));
-		if (!data->sfrbases[i]) {
-			dev_dbg(dev, "Unable to map IOMEM @ PA:%#x\n",
-							res->start);
-			ret = -ENOENT;
-			goto err_res;
-		}
+	ret = platform_get_irq(pdev, 0);
+	if (ret <= 0) {
+		dev_dbg(dev, "Unable to find IRQ resource\n");
+		goto err_irq;
 	}
 
-	for (i = 0; i < data->nsfrs; i++) {
-		ret = platform_get_irq(pdev, i);
-		if (ret <= 0) {
-			dev_dbg(dev, "Unable to find IRQ resource\n");
-			goto err_irq;
-		}
-
-		ret = request_irq(ret, exynos_sysmmu_irq, 0,
-					dev_name(dev), data);
-		if (ret) {
-			dev_dbg(dev, "Unabled to register interrupt handler\n");
-			goto err_irq;
-		}
+	ret = request_irq(ret, exynos_sysmmu_irq, 0,
+				dev_name(dev), data);
+	if (ret) {
+		dev_dbg(dev, "Unabled to register interrupt handler\n");
+		goto err_irq;
 	}
 
 	if (dev_get_platdata(dev)) {
-		char *deli, *beg;
-		struct sysmmu_platform_data *platdata = dev_get_platdata(dev);
-
-		beg = platdata->clockname;
-
-		for (deli = beg; (*deli != '\0') && (*deli != ','); deli++)
-			/* NOTHING */;
-
-		if (*deli == '\0')
-			deli = NULL;
-		else
-			*deli = '\0';
-
-		data->clk[0] = clk_get(dev, beg);
-		if (IS_ERR(data->clk[0])) {
-			data->clk[0] = NULL;
+		data->clk = clk_get(dev, "sysmmu");
+		if (IS_ERR(data->clk))
 			dev_dbg(dev, "No clock descriptor registered\n");
-		}
-
-		if (data->clk[0] && deli) {
-			*deli = ',';
-			data->clk[1] = clk_get(dev, deli + 1);
-			if (IS_ERR(data->clk[1]))
-				data->clk[1] = NULL;
-		}
-
-		data->dbgname = platdata->dbgname;
 	}
 
 	data->sysmmu = dev;
@@ -631,21 +555,16 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 
 	__set_fault_handler(data, &default_fault_handler);
 
+	platform_set_drvdata(pdev, data);
+
 	pm_runtime_enable(dev);
 
 	dev_dbg(dev, "(%s) Initialized\n", data->dbgname);
 	return 0;
 err_irq:
-	while (i-- > 0) {
-		int irq;
-
-		irq = platform_get_irq(pdev, i);
-		free_irq(irq, data);
-	}
+	free_irq(platform_get_irq(pdev, 0), data);
 err_res:
-	while (data->nsfrs-- > 0)
-		iounmap(data->sfrbases[data->nsfrs]);
-	kfree(data->sfrbases);
+	iounmap(data->sfrbase);
 err_init:
 	kfree(data);
 err_alloc:
-- 
1.7.9.5

[PATCH v12 10/31] iommu/exynos: use managed device helper functions

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This patch uses managed device helper functions in the probe().

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   68 ++++++++++++++++--------------------------
 1 file changed, 25 insertions(+), 43 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 1af2d23..0f1d3f0 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -342,8 +342,7 @@ static bool __exynos_sysmmu_disable(struct sysmmu_drvdata *data)
 
 	__raw_writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
 
-	if (!IS_ERR(data->clk))
-		clk_disable(data->clk);
+	clk_disable(data->clk);
 
 	disabled = true;
 	data->pgtable = 0;
@@ -386,8 +385,7 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 		goto finish;
 	}
 
-	if (!IS_ERR(data->clk))
-		clk_enable(data->clk);
+	clk_enable(data->clk);
 
 	data->pgtable = pgtable;
 
@@ -498,49 +496,43 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 
 static int exynos_sysmmu_probe(struct platform_device *pdev)
 {
-	int ret;
+	int irq, ret;
 	struct device *dev = &pdev->dev;
 	struct sysmmu_drvdata *data;
 	struct resource *res;
 
-	data = kzalloc(sizeof(*data), GFP_KERNEL);
-	if (!data) {
-		dev_dbg(dev, "Not enough memory\n");
-		ret = -ENOMEM;
-		goto err_alloc;
-	}
+	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+	if (!data)
+		return -ENOMEM;
 
 	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	if (!res) {
-		dev_dbg(dev, "Unable to find IOMEM region\n");
-		ret = -ENOENT;
-		goto err_init;
-	}
+	data->sfrbase = devm_ioremap_resource(dev, res);
+	if (IS_ERR(data->sfrbase))
+		return PTR_ERR(data->sfrbase);
 
-	data->sfrbase = ioremap(res->start, resource_size(res));
-	if (!data->sfrbase) {
-		dev_dbg(dev, "Unable to map IOMEM @ PA:%#x\n", res->start);
-		ret = -ENOENT;
-		goto err_res;
-	}
-
-	ret = platform_get_irq(pdev, 0);
-	if (ret <= 0) {
+	irq = platform_get_irq(pdev, 0);
+	if (irq <= 0) {
 		dev_dbg(dev, "Unable to find IRQ resource\n");
-		goto err_irq;
+		return irq;
 	}
 
-	ret = request_irq(ret, exynos_sysmmu_irq, 0,
+	ret = devm_request_irq(dev, irq, exynos_sysmmu_irq, 0,
 				dev_name(dev), data);
 	if (ret) {
-		dev_dbg(dev, "Unabled to register interrupt handler\n");
-		goto err_irq;
+		dev_err(dev, "Unabled to register handler of irq %d\n", irq);
+		return ret;
 	}
 
-	if (dev_get_platdata(dev)) {
-		data->clk = clk_get(dev, "sysmmu");
-		if (IS_ERR(data->clk))
-			dev_dbg(dev, "No clock descriptor registered\n");
+	data->clk = devm_clk_get(dev, "sysmmu");
+	if (IS_ERR(data->clk)) {
+		dev_err(dev, "Failed to get clock!\n");
+		return PTR_ERR(data->clk);
+	} else  {
+		ret = clk_prepare(data->clk);
+		if (ret) {
+			dev_err(dev, "Failed to prepare clk\n");
+			return ret;
+		}
 	}
 
 	data->sysmmu = dev;
@@ -553,17 +545,7 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 
 	pm_runtime_enable(dev);
 
-	dev_dbg(dev, "Initialized\n");
 	return 0;
-err_irq:
-	free_irq(platform_get_irq(pdev, 0), data);
-err_res:
-	iounmap(data->sfrbase);
-err_init:
-	kfree(data);
-err_alloc:
-	dev_err(dev, "Failed to initialize\n");
-	return ret;
 }
 
 static struct platform_driver exynos_sysmmu_driver = {
-- 
1.7.9.5

[PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This patch adds a description of the device tree binding for the
Samsung Exynos System MMU.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 .../devicetree/bindings/iommu/samsung,sysmmu.txt   |   79 ++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt

diff --git a/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt b/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt
new file mode 100644
index 0000000..16e13a2
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt
@@ -0,0 +1,79 @@
+Samsung Exynos IOMMU H/W, System MMU (System Memory Management Unit)
+
+Samsung's Exynos architecture contains System MMUs that enables scattered
+physical memory chunks visible as a contiguous region to DMA-capable peripheral
+devices like MFC, FIMC, FIMD, GScaler, FIMC-IS and so forth.
+
+System MMU is an IOMMU and supports identical translation table format to
+ARMv7 translation tables with minimum set of page properties including access
+permissions, shareability and security protection. In addition, System MMU has
+another capabilities like L2 TLB or block-fetch buffers to minimize translation
+latency.
+
+System MMUs are in many to one relation with peripheral devices, i.e. single
+peripheral device might have multiple System MMUs (usually one for each bus
+master), but one System MMU can handle transactions from only one peripheral
+device. The relation between a System MMU and the peripheral device needs to be
+defined in device node of the peripheral device.
+
+MFC in all Exynos SoCs and FIMD, M2M Scalers and G2D in Exynos5420 has 2 System
+MMUs.
+* MFC has one System MMU on its left and right bus.
+* FIMD in Exynos5420 has one System MMU for window 0 and 4, the other system MMU
+  for window 1, 2 and 3.
+* M2M Scalers and G2D in Exynos5420 has one System MMU on the read channel and
+  the other System MMU on the write channel.
+The drivers must consider how to handle those System MMUs. One of the idea is
+to implement child devices or sub-devices which are the client devices of the
+System MMU.
+
+Required properties:
+- compatible: Should be one of:
+		"samsung,sysmmu-v1"
+		"samsung,sysmmu-v2"
+		"samsung,sysmmu-v3.1"
+		"samsung,sysmmu-v3.2"
+		"samsung,sysmmu-v3.3"
+
+- reg: A tuple of base address and size of System MMU registers.
+- interrupt-parent: The phandle of the interrupt controller of System MMU
+- interrupts: An interrupt specifier for interrupt signal of System MMU,
+	      according to the format defined by a particular interrupt
+	      controller.
+- clock-names: Should be "sysmmu" if the System MMU is needed to gate its clock.
+	       Optional "master" if the clock to the System MMU is gated by
+	       another gate clock other than "sysmmu".
+	       Exynos4 SoCs, there needs no "master" clock.
+	       Exynos5 SoCs, some System MMUs must have "master" clocks.
+- clocks: Required if the System MMU is needed to gate its clock.
+- samsung,power-domain: Required if the System MMU is needed to gate its power.
+	  Please refer to the following document:
+	  Documentation/devicetree/bindings/arm/exynos/power_domain.txt
+- mmu-masters: A phandle to device nodes representing the master for which
+               the System MMU can provide a translation. Any additional values
+	       after the phandle will be ignored because a System MMU never
+	       have two or more masters. "#stream-id-cells" specified in the
+	       master's node will be also ignored.
+	       If more than one phandle is specified, only the first phandle
+	       will be treated.
+
+Examples:
+	gsc_0: gsc@13e00000 {
+		compatible = "samsung,exynos5-gsc";
+		reg = <0x13e00000 0x1000>;
+		interrupts = <0 85 0>;
+		samsung,power-domain = <&pd_gsc>;
+		clocks = <&clock CLK_GSCL0>;
+		clock-names = "gscl";
+	};
+
+	sysmmu_gsc0: sysmmu@13E80000 {
+		compatible = "samsung,sysmmu-v3.2";
+		reg = <0x13E80000 0x1000>;
+		interrupt-parent = <&combiner>;
+		interrupts = <2 0>;
+		clock-names = "sysmmu", "master";
+		clocks = <&clock CLK_SMMU_GSCL0>, <&clock CLK_GSCL0>;
+		samsung,power-domain = <&pd_gsc>;
+		mmu-masters = <&gsc_0>;
+	};
-- 
1.7.9.5

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
> +- mmu-masters: A phandle to device nodes representing the master for which
> +               the System MMU can provide a translation. Any additional values
> +              after the phandle will be ignored because a System MMU never
> +              have two or more masters. "#stream-id-cells" specified in the
> +              master's node will be also ignored.
> +              If more than one phandle is specified, only the first phandle
> +              will be treated.

This seems completely backwards: Why would you list the masters for an IOMMU
in the IOMMU node?

The master should have a standard property pointing to the IOMMU instead.

We don't have a generic binding for IOMMUs yet it seems, but the time is
overdue to make one.

Consider this NAKed until there is a generic binding for IOMMUs that all
relevant developers have agreed to.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 2828 bytes --]

On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
> On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
> > +- mmu-masters: A phandle to device nodes representing the master for which
> > +               the System MMU can provide a translation. Any additional values
> > +              after the phandle will be ignored because a System MMU never
> > +              have two or more masters. "#stream-id-cells" specified in the
> > +              master's node will be also ignored.
> > +              If more than one phandle is specified, only the first phandle
> > +              will be treated.
> 
> This seems completely backwards: Why would you list the masters for an IOMMU
> in the IOMMU node?
> 
> The master should have a standard property pointing to the IOMMU instead.
> 
> We don't have a generic binding for IOMMUs yet it seems, but the time is
> overdue to make one.
> 
> Consider this NAKed until there is a generic binding for IOMMUs that all
> relevant developers have agreed to.

I'd like to take this opportunity and revive one of the hibernating
patch sets that we have for Tegra. The last effort to get things merged
was back in January I think. I haven't bothered to look up the reference
since it's probably good to start from scratch anyway.

The latest version of the binding that was under discussion back then I
think looked something like this:

	device@... {
		iommus = <&iommu [spec]>[, <&other_iommu [other_spec]>...];
	};

And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.

On Tegra the specifier would be used to encode a memory controller's
client ID. One discussion point back at the time was to encode the ID as
a bitmask to allow more than a single master per entry. Another solution
which I think is a little cleaner and more generic, would be to use one
entry per master and use a single cell to encode the client ID. Devices
with multiple clients to the same IOMMU could then use multiple entries
referencing the same IOMMU.

I've added Hiroshi Doyu on Cc since he knows the Tegra IOMMU best.
Hiroshi, can you summarize exactly what the proposed bindings were. If
my memory serves me well they were mostly along the lines of what Arnd
proposes here, and perhaps they are something that can also be used for
Exynos.

Will Deacon (I think) had some comments on the earlier discussion as
well, so I've added him on Cc for visibility. Sorry if I'm confusing you
with someone else, Will. In that case perhaps you know who to include in
the discussion from the ARM side.

Also adding Stephen Warren for visibility.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
> On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
> > On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
> > > +- mmu-masters: A phandle to device nodes representing the master for which
> > > +               the System MMU can provide a translation. Any additional values
> > > +              after the phandle will be ignored because a System MMU never
> > > +              have two or more masters. "#stream-id-cells" specified in the
> > > +              master's node will be also ignored.
> > > +              If more than one phandle is specified, only the first phandle
> > > +              will be treated.
> > 
> > This seems completely backwards: Why would you list the masters for an IOMMU
> > in the IOMMU node?
> > 
> > The master should have a standard property pointing to the IOMMU instead.
> > 
> > We don't have a generic binding for IOMMUs yet it seems, but the time is
> > overdue to make one.
> > 
> > Consider this NAKed until there is a generic binding for IOMMUs that all
> > relevant developers have agreed to.
> 
> I'd like to take this opportunity and revive one of the hibernating
> patch sets that we have for Tegra. The last effort to get things merged
> was back in January I think. I haven't bothered to look up the reference
> since it's probably good to start from scratch anyway.
> 
> The latest version of the binding that was under discussion back then I
> think looked something like this:
> 
> 	device@... {
> 		iommus = <&iommu [spec]>[, <&other_iommu [other_spec]>...];
> 	};
> 
> And possibly with a iommu-names property to go along with that. The idea
> being that a device can be a master on possibly multiple IOMMUs. Using
> the above it would also be possible to have one device be multiple
> masters on the same IOMMU.

Yes, that seems reasonable. Just one question: How would you represent a
device that has multiple masters, with at least one connected to an IOMMU
and another one connected to memory directly, without going to the IOMMU?

> On Tegra the specifier would be used to encode a memory controller's
> client ID. One discussion point back at the time was to encode the ID as
> a bitmask to allow more than a single master per entry. Another solution
> which I think is a little cleaner and more generic, would be to use one
> entry per master and use a single cell to encode the client ID. Devices
> with multiple clients to the same IOMMU could then use multiple entries
> referencing the same IOMMU.

I'm not completely following here. Are you talking about the generic
binding, or the part that is tegra specific for the specifier?

My first impression is that the generic binding should just allow an
arbitrary specifier with a variable #iommu-cells, and leave the format
up to the IOMMU driver. A lot of drivers probably only support one
master, so they can just set #iommu-cells=<0>, others might require
IDs that do not fit into one cell.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 4460 bytes --]

On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
> On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
> > On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
> > > On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
> > > > +- mmu-masters: A phandle to device nodes representing the master for which
> > > > +               the System MMU can provide a translation. Any additional values
> > > > +              after the phandle will be ignored because a System MMU never
> > > > +              have two or more masters. "#stream-id-cells" specified in the
> > > > +              master's node will be also ignored.
> > > > +              If more than one phandle is specified, only the first phandle
> > > > +              will be treated.
> > > 
> > > This seems completely backwards: Why would you list the masters for an IOMMU
> > > in the IOMMU node?
> > > 
> > > The master should have a standard property pointing to the IOMMU instead.
> > > 
> > > We don't have a generic binding for IOMMUs yet it seems, but the time is
> > > overdue to make one.
> > > 
> > > Consider this NAKed until there is a generic binding for IOMMUs that all
> > > relevant developers have agreed to.
> > 
> > I'd like to take this opportunity and revive one of the hibernating
> > patch sets that we have for Tegra. The last effort to get things merged
> > was back in January I think. I haven't bothered to look up the reference
> > since it's probably good to start from scratch anyway.
> > 
> > The latest version of the binding that was under discussion back then I
> > think looked something like this:
> > 
> > 	device@... {
> > 		iommus = <&iommu [spec]>[, <&other_iommu [other_spec]>...];
> > 	};
> > 
> > And possibly with a iommu-names property to go along with that. The idea
> > being that a device can be a master on possibly multiple IOMMUs. Using
> > the above it would also be possible to have one device be multiple
> > masters on the same IOMMU.
> 
> Yes, that seems reasonable. Just one question: How would you represent a
> device that has multiple masters, with at least one connected to an IOMMU
> and another one connected to memory directly, without going to the IOMMU?

Heh, I don't think I've ever thought about that use-case. I guess I was
always assuming that in the absence of an IOMMU the device would simply
access memory directly. From what I can tell that's how Tegra works at
least. If the IOMMU is not enabled for a given client, that client will
access physical memory untranslated.

I suppose if that really must be represented then a global dummy IOMMU
could be introduced to help with these cases.

> > On Tegra the specifier would be used to encode a memory controller's
> > client ID. One discussion point back at the time was to encode the ID as
> > a bitmask to allow more than a single master per entry. Another solution
> > which I think is a little cleaner and more generic, would be to use one
> > entry per master and use a single cell to encode the client ID. Devices
> > with multiple clients to the same IOMMU could then use multiple entries
> > referencing the same IOMMU.
> 
> I'm not completely following here. Are you talking about the generic
> binding, or the part that is tegra specific for the specifier?
> 
> My first impression is that the generic binding should just allow an
> arbitrary specifier with a variable #iommu-cells, and leave the format
> up to the IOMMU driver.

Yes, I was getting ahead of myself. The idea was to have #iommu-cells
and allow the specifier to be IOMMU-specific. On Tegra that would
translate to the memory controller client ID, on other devices I suspect
something similar might exist, but for the generic binding it should be
completely opaque and hence irrelevant.

Really just like any of the other bindings that have foos and #foo-cells
properties.

> A lot of drivers probably only support one
> master, so they can just set #iommu-cells=<0>, others might require
> IDs that do not fit into one cell.

You mean "#iommu-cells = <1>" for devices that only require one master?
There still has to be one cell to specify which master. Unless perhaps
if they can be arbitrarily assigned. I guess even if there's a fixed
mapping that applies to one SoC generation, it might be good to still
employ a specifier and have the mapping in DT for flexibility.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
> On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
> > On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
> > > And possibly with a iommu-names property to go along with that. The idea
> > > being that a device can be a master on possibly multiple IOMMUs. Using
> > > the above it would also be possible to have one device be multiple
> > > masters on the same IOMMU.
> > 
> > Yes, that seems reasonable. Just one question: How would you represent a
> > device that has multiple masters, with at least one connected to an IOMMU
> > and another one connected to memory directly, without going to the IOMMU?
> 
> Heh, I don't think I've ever thought about that use-case. I guess I was
> always assuming that in the absence of an IOMMU the device would simply
> access memory directly. From what I can tell that's how Tegra works at
> least. If the IOMMU is not enabled for a given client, that client will
> access physical memory untranslated.
> 
> I suppose if that really must be represented then a global dummy IOMMU
> could be introduced to help with these cases.

It's actually not too uncommon: you can have e.g. the lower 2GB mapped
directly from the device address space into the host memory, but have
an iommu that translates accesses from some range in the upper 2GB of
the 32-bit address space into full 64-bit addresses.

This use case makes no sense if you use the IOMMU for isolation
or virtualization, but it gives better performance for lowmem access
when the only reason to have the IOMMU is to map highmem addresses.

> > > On Tegra the specifier would be used to encode a memory controller's
> > > client ID. One discussion point back at the time was to encode the ID as
> > > a bitmask to allow more than a single master per entry. Another solution
> > > which I think is a little cleaner and more generic, would be to use one
> > > entry per master and use a single cell to encode the client ID. Devices
> > > with multiple clients to the same IOMMU could then use multiple entries
> > > referencing the same IOMMU.
> > 
> > I'm not completely following here. Are you talking about the generic
> > binding, or the part that is tegra specific for the specifier?
> > 
> > My first impression is that the generic binding should just allow an
> > arbitrary specifier with a variable #iommu-cells, and leave the format
> > up to the IOMMU driver.
> 
> Yes, I was getting ahead of myself. The idea was to have #iommu-cells
> and allow the specifier to be IOMMU-specific. On Tegra that would
> translate to the memory controller client ID, on other devices I suspect
> something similar might exist, but for the generic binding it should be
> completely opaque and hence irrelevant.
> 
> Really just like any of the other bindings that have foos and #foo-cells
> properties.

Ok.

> > A lot of drivers probably only support one
> > master, so they can just set #iommu-cells=<0>, others might require
> > IDs that do not fit into one cell.
> 
> You mean "#iommu-cells = <1>" for devices that only require one master?

I meant an IOMMU device that acts as the slave for exactly one device,
even if that device has multiple master ports.

> There still has to be one cell to specify which master. Unless perhaps
> if they can be arbitrarily assigned. I guess even if there's a fixed
> mapping that applies to one SoC generation, it might be good to still
> employ a specifier and have the mapping in DT for flexibility.

let me clarify by example:

	iommu@1 {
		compatible = "some,simple-iommu";
		reg = <1>;
		#iommu-cells = <0>; /* supports only one master */
	};

	iommu@2 {
		compatible = "some,other-iommu";
		reg = <3>;
		#iommu-cells = <1>; /* contains master ID */
	};

	iommu@3 {
		compatible = "some,windowed-iommu";
		reg = <2>;
		#iommu-cells = <2>; /* contains dma-window */
	};

	device@4 {
		compatible = "some,ethernet";
		iommus = <&/iommu@1>;
	};

	device@5 {
		compatible = "some,dmaengine";
		iommus = <&/iommu@2 0x40000000 0x1000000>,
			 <&/iommu@3 0x101>;
	};

The device at address 4 has a one-one relationship with iommu@1, so there
is no need for any data. device@5 has two master ports. One is connected to
an IOMMU that has a per-device aperture, device@5 can only issue transfers
to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
this device into that address. The second master port is connected to
iommu@3, which uses a master ID that gets passed along with each transfer,
so that needs to be put into the IOTLBs.

A variation would be to not use #iommu-cells at all, but provide a
#address-cells / #size-cells pair in the IOMMU, and have a translation
as we do for dma-ranges. This is probably most flexible.

One completely open question that I just noticed is how the kernel should
deal with the case of multiple IOMMUs attached to one master: the
data structures we have assume that we know exactly how to do DMA by
setting the per-device dma_map_ops (iommu or not, coherent or not),
and by setting a pointer to at most one IOMMU.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 6611 bytes --]

On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
> On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
> > On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
> > > On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
> > > > And possibly with a iommu-names property to go along with that. The idea
> > > > being that a device can be a master on possibly multiple IOMMUs. Using
> > > > the above it would also be possible to have one device be multiple
> > > > masters on the same IOMMU.
> > > 
> > > Yes, that seems reasonable. Just one question: How would you represent a
> > > device that has multiple masters, with at least one connected to an IOMMU
> > > and another one connected to memory directly, without going to the IOMMU?
> > 
> > Heh, I don't think I've ever thought about that use-case. I guess I was
> > always assuming that in the absence of an IOMMU the device would simply
> > access memory directly. From what I can tell that's how Tegra works at
> > least. If the IOMMU is not enabled for a given client, that client will
> > access physical memory untranslated.
> > 
> > I suppose if that really must be represented then a global dummy IOMMU
> > could be introduced to help with these cases.
> 
> It's actually not too uncommon: you can have e.g. the lower 2GB mapped
> directly from the device address space into the host memory, but have
> an iommu that translates accesses from some range in the upper 2GB of
> the 32-bit address space into full 64-bit addresses.
> 
> This use case makes no sense if you use the IOMMU for isolation
> or virtualization, but it gives better performance for lowmem access
> when the only reason to have the IOMMU is to map highmem addresses.

Thinking about this some more, isn't the non-IOMMU master something we
can completely ignore in the DT? Or at least it shouldn't be handled by
the IOMMU bindings because, well, it's not an IOMMU to begin with.

Perhaps it's something that should be described using dma-ranges?

> > > A lot of drivers probably only support one
> > > master, so they can just set #iommu-cells=<0>, others might require
> > > IDs that do not fit into one cell.
> > 
> > You mean "#iommu-cells = <1>" for devices that only require one master?
> 
> I meant an IOMMU device that acts as the slave for exactly one device,
> even if that device has multiple master ports.

Okay, makes sense. I guess depending on the nature of the IOMMU it might
make sense not to expose it as an IOMMU at all. For example if it lives
completely within the register space of its master device. In that case
it could be directly programmed from the device's driver.

> > There still has to be one cell to specify which master. Unless perhaps
> > if they can be arbitrarily assigned. I guess even if there's a fixed
> > mapping that applies to one SoC generation, it might be good to still
> > employ a specifier and have the mapping in DT for flexibility.
> 
> let me clarify by example:
> 
> 	iommu@1 {
> 		compatible = "some,simple-iommu";
> 		reg = <1>;
> 		#iommu-cells = <0>; /* supports only one master */
> 	};
> 
> 	iommu@2 {
> 		compatible = "some,other-iommu";
> 		reg = <3>;
> 		#iommu-cells = <1>; /* contains master ID */
> 	};
> 
> 	iommu@3 {
> 		compatible = "some,windowed-iommu";
> 		reg = <2>;
> 		#iommu-cells = <2>; /* contains dma-window */
> 	};
> 
> 	device@4 {
> 		compatible = "some,ethernet";
> 		iommus = <&/iommu@1>;
> 	};
> 
> 	device@5 {
> 		compatible = "some,dmaengine";
> 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> 			 <&/iommu@3 0x101>;
> 	};
> 
> The device at address 4 has a one-one relationship with iommu@1, so there
> is no need for any data. device@5 has two master ports. One is connected to
> an IOMMU that has a per-device aperture, device@5 can only issue transfers
> to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> this device into that address. The second master port is connected to
> iommu@3, which uses a master ID that gets passed along with each transfer,
> so that needs to be put into the IOTLBs.

The above sounds reasonable to me with the exception of the DMA window
specifier. Isn't that precisely the information that we currently
describe using the dma-ranges property?

> A variation would be to not use #iommu-cells at all, but provide a
> #address-cells / #size-cells pair in the IOMMU, and have a translation
> as we do for dma-ranges. This is probably most flexible.

I'm not sure I follow. Wouldn't that require masters to be children of
the IOMMU DT nodes for that to work out? Also how would that work for
cases where more data than the address ranges (such as the master ID) is
needed to operate the IOMMU?

> One completely open question that I just noticed is how the kernel should
> deal with the case of multiple IOMMUs attached to one master: the
> data structures we have assume that we know exactly how to do DMA by
> setting the per-device dma_map_ops (iommu or not, coherent or not),
> and by setting a pointer to at most one IOMMU.

Perhaps we need something more fine-grained than what we currently have.
I can imagine that rather than having it all abstracted away and handled
transparently (which undoubtedly has a lot of advantages), we need to
expose a client API of sorts that drivers can use.

I'm mostly brainstorming here and not thinking of any concrete use-case:

	struct iommu_master *master = iommu_get(dev, "foo");

	iommu_master_set_range(master, 0, SZ_256M);

	pages = alloc_pages(...);

	iova = iommu_master_map(master, pages);

The above would essentially obtain a handle to the "foo" IOMMUs master,
set the valid DMA range (hard-coded in the driver or obtained from a
dma-ranges property), allocates some pages of memory and then maps them
into the device's address space via the IOMMU.

The reason why I think what we have isn't going to work is that it is
assumed that either there is an IOMMU or there isn't. So memory gets
either mapped through an IOMMU or it doesn't. Furthermore IOMMU usage is
likely very use-case dependent and it's probably not easy to determine
automatically which IOMMU is to be used for individual allocations.

Allowing more fine-grained control of the IOMMU from drivers should
increase the flexibility of users since they have more context to make
the right decisions.

But then again, I have only a very sketchy idea of what an IOMMU needs
to be able to do, so maybe the above doesn't make sense at all.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Will Deacon]

Hi Arnd,

[and thanks Thierry for CCing me -- I have been tangled up with this before
:)]

On Mon, Apr 28, 2014 at 01:05:30PM +0100, Arnd Bergmann wrote:
> On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
> > There still has to be one cell to specify which master. Unless perhaps
> > if they can be arbitrarily assigned. I guess even if there's a fixed
> > mapping that applies to one SoC generation, it might be good to still
> > employ a specifier and have the mapping in DT for flexibility.
> 
> let me clarify by example:
> 
> 	iommu@1 {
> 		compatible = "some,simple-iommu";
> 		reg = <1>;
> 		#iommu-cells = <0>; /* supports only one master */
> 	};
> 
> 	iommu@2 {
> 		compatible = "some,other-iommu";
> 		reg = <3>;
> 		#iommu-cells = <1>; /* contains master ID */
> 	};
> 
> 	iommu@3 {
> 		compatible = "some,windowed-iommu";
> 		reg = <2>;
> 		#iommu-cells = <2>; /* contains dma-window */
> 	};
> 
> 	device@4 {
> 		compatible = "some,ethernet";
> 		iommus = <&/iommu@1>;
> 	};
> 
> 	device@5 {
> 		compatible = "some,dmaengine";
> 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> 			 <&/iommu@3 0x101>;
> 	};
> 
> The device at address 4 has a one-one relationship with iommu@1, so there
> is no need for any data. device@5 has two master ports. One is connected to
> an IOMMU that has a per-device aperture, device@5 can only issue transfers
> to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> this device into that address. The second master port is connected to
> iommu@3, which uses a master ID that gets passed along with each transfer,
> so that needs to be put into the IOTLBs.

I think this is definitely going in the right direction, but it's not clear
to me how the driver for device@5 knows how to configure the two ports.
We're still lacking topology information (unless that's implicit in the
ordering of the properties) to say how the mastering capabilities of the
device are actually routed and configured.

> A variation would be to not use #iommu-cells at all, but provide a
> #address-cells / #size-cells pair in the IOMMU, and have a translation
> as we do for dma-ranges. This is probably most flexible.

That would also allow us to describe ranges of master IDs, which we need for
things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
these ranges could also be described like this, although I think Dave (CC'd)
has some similar ideas in this area.

> One completely open question that I just noticed is how the kernel should
> deal with the case of multiple IOMMUs attached to one master: the
> data structures we have assume that we know exactly how to do DMA by
> setting the per-device dma_map_ops (iommu or not, coherent or not),
> and by setting a pointer to at most one IOMMU.

Agreed.

Will

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Monday 28 April 2014 20:30:56 Will Deacon wrote:
> Hi Arnd,
> 
> [and thanks Thierry for CCing me -- I have been tangled up with this before
> :)]
> 
> On Mon, Apr 28, 2014 at 01:05:30PM +0100, Arnd Bergmann wrote:
> > On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
> > > There still has to be one cell to specify which master. Unless perhaps
> > > if they can be arbitrarily assigned. I guess even if there's a fixed
> > > mapping that applies to one SoC generation, it might be good to still
> > > employ a specifier and have the mapping in DT for flexibility.
> > 
> > let me clarify by example:
> > 
> > 	iommu@1 {
> > 		compatible = "some,simple-iommu";
> > 		reg = <1>;
> > 		#iommu-cells = <0>; /* supports only one master */
> > 	};
> > 
> > 	iommu@2 {
> > 		compatible = "some,other-iommu";
> > 		reg = <3>;
> > 		#iommu-cells = <1>; /* contains master ID */
> > 	};
> > 
> > 	iommu@3 {
> > 		compatible = "some,windowed-iommu";
> > 		reg = <2>;
> > 		#iommu-cells = <2>; /* contains dma-window */
> > 	};
> > 
> > 	device@4 {
> > 		compatible = "some,ethernet";
> > 		iommus = <&/iommu@1>;
> > 	};
> > 
> > 	device@5 {
> > 		compatible = "some,dmaengine";
> > 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> > 			 <&/iommu@3 0x101>;
> > 	};
> > 
> > The device at address 4 has a one-one relationship with iommu@1, so there
> > is no need for any data. device@5 has two master ports. One is connected to
> > an IOMMU that has a per-device aperture, device@5 can only issue transfers
> > to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> > this device into that address. The second master port is connected to
> > iommu@3, which uses a master ID that gets passed along with each transfer,
> > so that needs to be put into the IOTLBs.
> 
> I think this is definitely going in the right direction, but it's not clear
> to me how the driver for device@5 knows how to configure the two ports.
> We're still lacking topology information (unless that's implicit in the
> ordering of the properties) to say how the mastering capabilities of the
> device are actually routed and configured.

It would be helpful to have a concrete example of a device that has multiple
masters. I have heard people mention this multiple times, and I can understand
how it might be wired up in hardware, but I don't know what it's good for,
or who would actually do it.

> > A variation would be to not use #iommu-cells at all, but provide a
> > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > as we do for dma-ranges. This is probably most flexible.
> 
> That would also allow us to describe ranges of master IDs, which we need for
> things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> these ranges could also be described like this, although I think Dave (CC'd)
> has some similar ideas in this area.

Good point.

	ARnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Mon, Apr 28, 2014 at 09:55:00PM +0200, Arnd Bergmann wrote:
> On Monday 28 April 2014 20:30:56 Will Deacon wrote:
> > Hi Arnd,
> > 
> > [and thanks Thierry for CCing me -- I have been tangled up with this before
> > :)]
> > 
> > On Mon, Apr 28, 2014 at 01:05:30PM +0100, Arnd Bergmann wrote:
> > > On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
> > > > There still has to be one cell to specify which master. Unless perhaps
> > > > if they can be arbitrarily assigned. I guess even if there's a fixed
> > > > mapping that applies to one SoC generation, it might be good to still
> > > > employ a specifier and have the mapping in DT for flexibility.
> > > 
> > > let me clarify by example:
> > > 
> > > 	iommu@1 {
> > > 		compatible = "some,simple-iommu";
> > > 		reg = <1>;
> > > 		#iommu-cells = <0>; /* supports only one master */
> > > 	};
> > > 
> > > 	iommu@2 {
> > > 		compatible = "some,other-iommu";
> > > 		reg = <3>;
> > > 		#iommu-cells = <1>; /* contains master ID */
> > > 	};
> > > 
> > > 	iommu@3 {
> > > 		compatible = "some,windowed-iommu";
> > > 		reg = <2>;
> > > 		#iommu-cells = <2>; /* contains dma-window */
> > > 	};

An IOMMU is really a specialised bridge, so it may be cleaner to describe
an IOMMU using a real bus node in the DT, if we also define a way to make
master/slave linkages explicit where it matters.


The problems of how to describe master/slave linkage, coherency between
masters, and how to describe sideband ID information present on the bus
are really interrelated.


If we can come up with a consistent description for these things, it
should help us to describe IOMMUs, bus mastering peripherals, MSI
controllers and complex bridges in a more uniform way, without having to
reinvent so much for each binding.  That's my hope anyway.

I've been hacking around some proposals on these areas which are a bit
different from the approach suggested here -- I'll try to summarise some
of it intelligibly and post something tomorrow so that we can discuss.


> > > 
> > > 	device@4 {
> > > 		compatible = "some,ethernet";
> > > 		iommus = <&/iommu@1>;
> > > 	};
> > > 
> > > 	device@5 {
> > > 		compatible = "some,dmaengine";
> > > 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> > > 			 <&/iommu@3 0x101>;
> > > 	};
> > > 
> > > The device at address 4 has a one-one relationship with iommu@1, so there
> > > is no need for any data. device@5 has two master ports. One is connected to
> > > an IOMMU that has a per-device aperture, device@5 can only issue transfers
> > > to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> > > this device into that address. The second master port is connected to
> > > iommu@3, which uses a master ID that gets passed along with each transfer,
> > > so that needs to be put into the IOTLBs.
> > 
> > I think this is definitely going in the right direction, but it's not clear
> > to me how the driver for device@5 knows how to configure the two ports.
> > We're still lacking topology information (unless that's implicit in the
> > ordering of the properties) to say how the mastering capabilities of the
> > device are actually routed and configured.
> 
> It would be helpful to have a concrete example of a device that has multiple
> masters. I have heard people mention this multiple times, and I can understand

You mean "a device that contains multiple independent bus masters",
right?  In particular, a device composed of multiple bus masters that
do different things or should be handled differently by the interconnect.

There has definitely been talk on the list about real devices that
use multiple stream IDs.


I'll ask around for "device-like" examples, but the most obvious
example is the IOMMU itself.

Transactions generated by the IOMMU clearly need to be handled
differently by the interconnect, compared with transactions
translated and forwarded by IOMMU on behalf of its clients.

For example, suppose devices can post MSIs to an interrupt controller
via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
generates MSIs itself in order to signal management events or faults
to a host OS.  Linux (as host) will need to configure the interrupt
controller separately for the IOMMU and for the IOMMU clients.  This
means that Linux needs to know which IDs may travel to the interrupt
controller for which purpose, and they must be distinct.

I'm not sure whether there is actually a SoC today that is MSI-capable
and contains an IOMMU, but all the components to build one are out
there today.  GICv3 is also explicitly designed to support such
systems.


In the future, it is likely that "HSA"-style GPUs and other high-
throughput virtualisable bus mastering devices will have capabilities
of this sort, but I don't think there's anything concrete yet.


> how it might be wired up in hardware, but I don't know what it's good for,
> or who would actually do it.
> 
> > > A variation would be to not use #iommu-cells at all, but provide a
> > > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > > as we do for dma-ranges. This is probably most flexible.
> > 
> > That would also allow us to describe ranges of master IDs, which we need for
> > things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> > these ranges could also be described like this, although I think Dave (CC'd)
> > has some similar ideas in this area.

Ideally, we would reuse the ePAPR "ranges" concept and describe the way
sideband ID signals propagate down the bus hierarchy in a similar way.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Grant Grundler]

On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org> wrote:
...
> An IOMMU is really a specialised bridge

Is a GART a bridge?

IOMMUs can provide three basic functions:
1) remap address space to reach phys mem ranges that the device is
otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
Phys address)

2) implement scatter-gather (page level granularity) so the device
doesn't have to

3) provide some level of system protection against "rogue" DMA by
forcing everything through a DMA mapping interface and faulting when
encountering unmapped DMA transactions.


I summarize IOMMUs as: "participate in the routing of MMIO
transactions in the system fabric."
In this sense, IOMMUs are sort of a bridge. Defining what kind of
routing they can do (coalesce transactions? remapping MMIO domains?)
and which address ranges they route would describe most of that
functionality.

This "remapping" of MMIO transaction is also usually asymmetric.
Meaning routing of "downstream" transactions *might* be completely
different than the routing + remapping of transactions heading
upstream. DMA mappings services are designed to handle only the
transactions generated (aka "mastered") by a downstream device.


>, so it may be cleaner to describe
> an IOMMU using a real bus node in the DT, if we also define a way to make
> master/slave linkages explicit where it matters.

"where it matters" is a bit vague.  Is the goal to just enable DMA
mapping services to "do the right thing" for a device that can
generate DMA?


> The problems of how to describe master/slave linkage, coherency between
> masters, and how to describe sideband ID information present on the bus
> are really interrelated.
>
> If we can come up with a consistent description for these things, it
> should help us to describe IOMMUs, bus mastering peripherals, MSI
> controllers and complex bridges in a more uniform way, without having to
> reinvent so much for each binding.  That's my hope anyway.

I don't know...these all deal with MMIO routing. The parts that are
dynamic should be detectable by platform specific SW (in general).
But where it's not (e.g. ISTR MSI transaction addressing is defined by
Intel Arch), it needs to be described by _something_  and device tree
seems to be a reasonable place to do that.

> I've been hacking around some proposals on these areas which are a bit
> different from the approach suggested here -- I'll try to summarise some
> of it intelligibly and post something tomorrow so that we can discuss.

Are you planning on consolidating Documentation/devicetree/bindings/iommu/ ?
Do you care about Documentation/Intel-IOMMU.txt?

Lots of stuff in Documentation/ touch different parts of the MMIO routing uses.

hth,
grant

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Tuesday 29 April 2014 13:07:54 Grant Grundler wrote:
> On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org> wrote:
> ...
> > An IOMMU is really a specialised bridge
> 
> Is a GART a bridge?
> 
> IOMMUs can provide three basic functions:
> 1) remap address space to reach phys mem ranges that the device is
> otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
> Phys address)
> 
> 2) implement scatter-gather (page level granularity) so the device
> doesn't have to
> 
> 3) provide some level of system protection against "rogue" DMA by
> forcing everything through a DMA mapping interface and faulting when
> encountering unmapped DMA transactions.

[ 4) provide isolation between multiple contexts, typically for purposes
     of virtualization]

> I summarize IOMMUs as: "participate in the routing of MMIO
> transactions in the system fabric."
> In this sense, IOMMUs are sort of a bridge. Defining what kind of
> routing they can do (coalesce transactions? remapping MMIO domains?)
> and which address ranges they route would describe most of that
> functionality.
> 
> This "remapping" of MMIO transaction is also usually asymmetric.
> Meaning routing of "downstream" transactions *might* be completely
> different than the routing + remapping of transactions heading
> upstream. DMA mappings services are designed to handle only the
> transactions generated (aka "mastered") by a downstream device.

For the purposes of the DT binding, we have a 'ranges' property
that defines the downstream translation (CPU-to-MMIO) and a
'dma-ranges' property for the opposite address translation
(device-to-memory).

> >, so it may be cleaner to describe
> > an IOMMU using a real bus node in the DT, if we also define a way to make
> > master/slave linkages explicit where it matters.
> 
> "where it matters" is a bit vague.  Is the goal to just enable DMA
> mapping services to "do the right thing" for a device that can
> generate DMA?

Yes. It's very complicated unfortunately, because we have to be
able to deal with arbitrary combinations of a lot of oddball cases
that can show up in random SoCs:

- device can only do DMA to a limited address range
- DMA is noncoherent and needs manual cache management
- DMA address is at an offset from physical address
- some devices have an IOMMU
- some IOMMUs are shared between devices
- some devices with IOMMU can have multiple simultaneous contexts
- a device may access some memory directly and some other memory through IOMMU
- a device may have DMA access to a bus that is invisible to the CPU
- DMA on some device is only coherent if the IOMMU is enabled
- DMA on some device is only coherent if the IOMMU is disabled
- the IOVA range to an IOMMU is device dependent

> > I've been hacking around some proposals on these areas which are a bit
> > different from the approach suggested here -- I'll try to summarise some
> > of it intelligibly and post something tomorrow so that we can discuss.
> 
> Are you planning on consolidating Documentation/devicetree/bindings/iommu/ ?
> Do you care about Documentation/Intel-IOMMU.txt?

I think we can ignore the Intel-IOMMU because that is specialized on PCI
devices, which we don't normally represent in DT. It is also a special
case because the Intel IOMMU is a single-instance device. If it's present
and enabled, it will be used by every device. The case we do need to describe
is when we don't know which IOMMU is used for which master, or how to
configure it.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Tue, Apr 29, 2014 at 11:00:29PM +0200, Arnd Bergmann wrote: > On Tuesday 29 April 2014 13:07:54 Grant Grundler wrote: > > On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin@arm.com> wrote:
> > ...
> > > An IOMMU is really a specialised bridge
> > 
> > Is a GART a bridge?

Depends what you mean by "bridge".

I would say that it can be logically viewed as a bridge through which
graphics cards master onto the bus.

In this context it's integrated into the bus, but I think the function
it performs is still in series with, and logically distinct from, the
routing done by the bus proper.

> > IOMMUs can provide three basic functions:
> > 1) remap address space to reach phys mem ranges that the device is
> > otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
> > Phys address)
> > 
> > 2) implement scatter-gather (page level granularity) so the device
> > doesn't have to
> > 
> > 3) provide some level of system protection against "rogue" DMA by
> > forcing everything through a DMA mapping interface and faulting when
> > encountering unmapped DMA transactions.
> 
> [ 4) provide isolation between multiple contexts, typically for purposes
>      of virtualization]

All of which are classic functionalities of an MMU.  What is different
is that the CPU MMU is built into the CPU and standardised, and either
symmetrically shared or private to the CPU so that there is rarely if
ever any need to describe anything in DT ... whereas an IOMMU is not
tightly coupled to the CPU, and wired up to external hardware in
sometimes arbitrary ways that may require description (especially for
SoC systems).

> > I summarize IOMMUs as: "participate in the routing of MMIO
> > transactions in the system fabric."
> > In this sense, IOMMUs are sort of a bridge. Defining what kind of
> > routing they can do (coalesce transactions? remapping MMIO domains?)
> > and which address ranges they route would describe most of that
> > functionality.

agreed

> > This "remapping" of MMIO transaction is also usually asymmetric.
> > Meaning routing of "downstream" transactions *might* be completely
> > different than the routing + remapping of transactions heading
> > upstream. DMA mappings services are designed to handle only the
> > transactions generated (aka "mastered") by a downstream device.

Also agreed.  In the SoC world, the upstream and downstream paths
may be completely separate in the topology, not following the same
path at all, so then it becomes more natural for them to have
independent characteristics.  "dma-ranges" doesn't work for these
situations.

> For the purposes of the DT binding, we have a 'ranges' property
> that defines the downstream translation (CPU-to-MMIO) and a
> 'dma-ranges' property for the opposite address translation
> (device-to-memory).
> 
> > >, so it may be cleaner to describe
> > > an IOMMU using a real bus node in the DT, if we also define a way to make
> > > master/slave linkages explicit where it matters.
> > 
> > "where it matters" is a bit vague.  Is the goal to just enable DMA
> > mapping services to "do the right thing" for a device that can
> > generate DMA?
> 
> Yes. It's very complicated unfortunately, because we have to be
> able to deal with arbitrary combinations of a lot of oddball cases
> that can show up in random SoCs:
> 
> - device can only do DMA to a limited address range
> - DMA is noncoherent and needs manual cache management
> - DMA address is at an offset from physical address
> - some devices have an IOMMU
> - some IOMMUs are shared between devices
> - some devices with IOMMU can have multiple simultaneous contexts
> - a device may access some memory directly and some other memory through IOMMU
> - a device may have DMA access to a bus that is invisible to the CPU
> - DMA on some device is only coherent if the IOMMU is enabled
> - DMA on some device is only coherent if the IOMMU is disabled
> - the IOVA range to an IOMMU is device dependent

This sounds bad on the surface, but really these are permutations of a
few key concepts:

 1) Topologies that can't be reduced to a tree, resulting from
   unidirectional buses (common in SoC architectures).

 2) Many-to-many connectivity of devices, unlike the familiar
   many-to-one parent/child relationship from PCI and similar.

 3) Interconnects made up of independently configured and
   connected components, such that they can't be described as optional
   features of some standard bus.

 4) Bridge components and devices that process, route or transform
   transactions based on more than just the destination address.
   Instead, the handling of a transaction can involve some kind of
   device ID, memory type and cacheability attributes and coherency
   domain information in addition to the address.


To address these, we need a few things:

 a) A way to describe the cross-links where the topology is not
    reducible to a tree.  This would involve a phandle to describe
    a "A masters on B" relationship analogous to what the parent/
    child relationship in DT already means.

 b) For devices with multiple distinct master roles with different
    routing or behaviour in the interconnect, a way to describe those
    roles.

 c) A way to describe mappings across bridges that builds on
    "ranges" to cover a wider range of situations.


> > > I've been hacking around some proposals on these areas which are a bit
> > > different from the approach suggested here -- I'll try to summarise some
> > > of it intelligibly and post something tomorrow so that we can discuss.
> > 
> > Are you planning on consolidating Documentation/devicetree/bindings/iommu/ ?

Not initially.  Ripping out old bindings that work perfectly well today
will just cause churn.  But if we can lay a good foundation for future
bindings that should help us in the future.

> > Do you care about Documentation/Intel-IOMMU.txt?

Probably answered by my above point.

If it ain't broke, there's no reason to fix it.

Defining DT bindings and refactoring kernel frameworks are pretty
orthogonal: we could retain compatibility with existing IOMMU bindings
within a modified framework that can also handle more diverse SoC
scenarios which may use a more uniform and sophisticated description in
the DT.

> I think we can ignore the Intel-IOMMU because that is specialized on PCI
> devices, which we don't normally represent in DT. It is also a special
> case because the Intel IOMMU is a single-instance device. If it's present
> and enabled, it will be used by every device. The case we do need to describe
> is when we don't know which IOMMU is used for which master, or how to
> configure it.

Agreed.  Because of the single-instance nature of the Intel IOMMU, and
the fact that is integrated into the interconnect in a pretty standard
way, we can get away with much less description.


I'll follow up with some more concrete details on my thinking --
hopefully that will make it a bit clearer what I mean.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Cho KyongHo]

On Tue, 29 Apr 2014 23:00:29 +0200, Arnd Bergmann wrote:
> On Tuesday 29 April 2014 13:07:54 Grant Grundler wrote:
> > On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org> wrote:
> > ...
> > > An IOMMU is really a specialised bridge
> > 
> > Is a GART a bridge?
> > 
> > IOMMUs can provide three basic functions:
> > 1) remap address space to reach phys mem ranges that the device is
> > otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
> > Phys address)
> > 
> > 2) implement scatter-gather (page level granularity) so the device
> > doesn't have to
> > 
> > 3) provide some level of system protection against "rogue" DMA by
> > forcing everything through a DMA mapping interface and faulting when
> > encountering unmapped DMA transactions.
> 
> [ 4) provide isolation between multiple contexts, typically for purposes
>      of virtualization]
> 
> > I summarize IOMMUs as: "participate in the routing of MMIO
> > transactions in the system fabric."
> > In this sense, IOMMUs are sort of a bridge. Defining what kind of
> > routing they can do (coalesce transactions? remapping MMIO domains?)
> > and which address ranges they route would describe most of that
> > functionality.
> > 
> > This "remapping" of MMIO transaction is also usually asymmetric.
> > Meaning routing of "downstream" transactions *might* be completely
> > different than the routing + remapping of transactions heading
> > upstream. DMA mappings services are designed to handle only the
> > transactions generated (aka "mastered") by a downstream device.
> 
> For the purposes of the DT binding, we have a 'ranges' property
> that defines the downstream translation (CPU-to-MMIO) and a
> 'dma-ranges' property for the opposite address translation
> (device-to-memory).
> 
> > >, so it may be cleaner to describe
> > > an IOMMU using a real bus node in the DT, if we also define a way to make
> > > master/slave linkages explicit where it matters.
> > 
> > "where it matters" is a bit vague.  Is the goal to just enable DMA
> > mapping services to "do the right thing" for a device that can
> > generate DMA?
> 
> Yes. It's very complicated unfortunately, because we have to be
> able to deal with arbitrary combinations of a lot of oddball cases
> that can show up in random SoCs:
> 
> - device can only do DMA to a limited address range
> - DMA is noncoherent and needs manual cache management
> - DMA address is at an offset from physical address
> - some devices have an IOMMU
> - some IOMMUs are shared between devices
> - some devices with IOMMU can have multiple simultaneous contexts
> - a device may access some memory directly and some other memory through IOMMU

Do we need to consider this case?
I don't think a device can have different contexts at the same time.
If there such device is in a system, its driver must handle it correctly
with different devices descriptors for the different contexts, for example.
I mean, if a device has two DMA ports that are in different contexts,
they can be treated as different devices which are handed by a driver.

I worry that abstracting everything we can think may make the problem harder.

Regards,

   KyongHo

> - a device may have DMA access to a bus that is invisible to the CPU
> - DMA on some device is only coherent if the IOMMU is enabled
> - DMA on some device is only coherent if the IOMMU is disabled
> - the IOVA range to an IOMMU is device dependent
> 
> > > I've been hacking around some proposals on these areas which are a bit
> > > different from the approach suggested here -- I'll try to summarise some
> > > of it intelligibly and post something tomorrow so that we can discuss.
> > 
> > Are you planning on consolidating Documentation/devicetree/bindings/iommu/ ?
> > Do you care about Documentation/Intel-IOMMU.txt?
> 
> I think we can ignore the Intel-IOMMU because that is specialized on PCI
> devices, which we don't normally represent in DT. It is also a special
> case because the Intel IOMMU is a single-instance device. If it's present
> and enabled, it will be used by every device. The case we do need to describe
> is when we don't know which IOMMU is used for which master, or how to
> configure it.
> 
> 	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Thursday 01 May 2014 23:02:14 Cho KyongHo wrote:
> > 
> > - device can only do DMA to a limited address range
> > - DMA is noncoherent and needs manual cache management
> > - DMA address is at an offset from physical address
> > - some devices have an IOMMU
> > - some IOMMUs are shared between devices
> > - some devices with IOMMU can have multiple simultaneous contexts
> > - a device may access some memory directly and some other memory through IOMMU
> 
> Do we need to consider this case?
> I don't think a device can have different contexts at the same time.
> If there such device is in a system, its driver must handle it correctly
> with different devices descriptors for the different contexts, for example.
> I mean, if a device has two DMA ports that are in different contexts,
> they can be treated as different devices which are handed by a driver.
> 
> I worry that abstracting everything we can think may make the problem harder.

It's the default operation for some of the simpler IOMMUs, see
arch/x86/kernel/amd_gart_64.c for instance. It's possible that AMD
will have the same thing in their ARM64 SoCs, but I don't have
specific information about that.

It can probably be handled in the iommu_map_ops() as a generalization,
at least if we only have to worry about checking whether a memory address
is below the dma_mask in order to decide whether to use the IOMMU or not.

Or we can decide not to handle it at all, and always go through the IOMMU,
which would be slightly slower but still functional.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Thu, May 01, 2014 at 11:02:14PM +0900, Cho KyongHo wrote:
> On Tue, 29 Apr 2014 23:00:29 +0200, Arnd Bergmann wrote:
> > On Tuesday 29 April 2014 13:07:54 Grant Grundler wrote:
> > > On Tue, Apr 29, 2014 at 11:16 AM, Dave Martin <Dave.Martin-5wv7dgnIgG8@public.gmane.org> wrote:
> > > ...
> > > > An IOMMU is really a specialised bridge
> > > 
> > > Is a GART a bridge?
> > > 
> > > IOMMUs can provide three basic functions:
> > > 1) remap address space to reach phys mem ranges that the device is
> > > otherwise not capable of accessing (classic 32-bit DMA to reach 64-bit
> > > Phys address)
> > > 
> > > 2) implement scatter-gather (page level granularity) so the device
> > > doesn't have to
> > > 
> > > 3) provide some level of system protection against "rogue" DMA by
> > > forcing everything through a DMA mapping interface and faulting when
> > > encountering unmapped DMA transactions.
> > 
> > [ 4) provide isolation between multiple contexts, typically for purposes
> >      of virtualization]
> > 
> > > I summarize IOMMUs as: "participate in the routing of MMIO
> > > transactions in the system fabric."
> > > In this sense, IOMMUs are sort of a bridge. Defining what kind of
> > > routing they can do (coalesce transactions? remapping MMIO domains?)
> > > and which address ranges they route would describe most of that
> > > functionality.
> > > 
> > > This "remapping" of MMIO transaction is also usually asymmetric.
> > > Meaning routing of "downstream" transactions *might* be completely
> > > different than the routing + remapping of transactions heading
> > > upstream. DMA mappings services are designed to handle only the
> > > transactions generated (aka "mastered") by a downstream device.
> > 
> > For the purposes of the DT binding, we have a 'ranges' property
> > that defines the downstream translation (CPU-to-MMIO) and a
> > 'dma-ranges' property for the opposite address translation
> > (device-to-memory).
> > 
> > > >, so it may be cleaner to describe
> > > > an IOMMU using a real bus node in the DT, if we also define a way to make
> > > > master/slave linkages explicit where it matters.
> > > 
> > > "where it matters" is a bit vague.  Is the goal to just enable DMA
> > > mapping services to "do the right thing" for a device that can
> > > generate DMA?
> > 
> > Yes. It's very complicated unfortunately, because we have to be
> > able to deal with arbitrary combinations of a lot of oddball cases
> > that can show up in random SoCs:
> > 
> > - device can only do DMA to a limited address range
> > - DMA is noncoherent and needs manual cache management
> > - DMA address is at an offset from physical address
> > - some devices have an IOMMU
> > - some IOMMUs are shared between devices
> > - some devices with IOMMU can have multiple simultaneous contexts
> > - a device may access some memory directly and some other memory through IOMMU
> 
> Do we need to consider this case?
> I don't think a device can have different contexts at the same time.
> If there such device is in a system, its driver must handle it correctly
> with different devices descriptors for the different contexts, for example.
> I mean, if a device has two DMA ports that are in different contexts,
> they can be treated as different devices which are handed by a driver.

GPUs will definitely be capable of acting on behalf of multiple contexts
at the same time, in a dynamic fashion.  This doesn't necessarily mean
that it masters through different ports or onto different buses though.

Sketching out how we would describe this in DT doesn't imply that we
need Linux to support it.  It's more about asking: if we have to support
this in the future, how badly will it screw up the current framework?

> I worry that abstracting everything we can think may make the problem harder.

That's always a risk, though pain today may be worth it if is saves a
larger amount of pain in the future.

As I say on the other branch of this thread, I'll follow up with
something a bit more concrete to illustrate the kind of thing I mean.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Stephen Warren]

On 04/29/2014 03:00 PM, Arnd Bergmann wrote:
...
> Yes. It's very complicated unfortunately, because we have to be
> able to deal with arbitrary combinations of a lot of oddball cases
> that can show up in random SoCs:
...
> - a device may have DMA access to a bus that is invisible to the CPU

The issue is slightly more general than that. It's more that the bus
structure "seen" by a device is simply /different/ than that seen by the
CPU. I don't think it's a requirement that there be CPU-invisible buses
for that to be true.

For example, I could conceive of a HW setup like:

primary CPU bus ----------------------> other devices
   |    \_________________      /
   |                      \    |
   v                      v    ^
device registers ----> some secondary bus
                          |
                          v
                       memory

Here, all the buses are visible to the CPU, yet the path that
transactions take between the buses is simply different to the CPU. More
complex situations than the above, while still maintaining that
description, are certainly possible.

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Thu, May 01, 2014 at 06:41:37PM +0100, Stephen Warren wrote:
> On 04/29/2014 03:00 PM, Arnd Bergmann wrote:
> ...
> > Yes. It's very complicated unfortunately, because we have to be
> > able to deal with arbitrary combinations of a lot of oddball cases
> > that can show up in random SoCs:
> ...
> > - a device may have DMA access to a bus that is invisible to the CPU
> 
> The issue is slightly more general than that. It's more that the bus
> structure "seen" by a device is simply /different/ than that seen by the
> CPU. I don't think it's a requirement that there be CPU-invisible buses
> for that to be true.
> 
> For example, I could conceive of a HW setup like:
> 
> primary CPU bus ----------------------> other devices
>    |    \_________________      /
>    |                      \    |
>    v                      v    ^
> device registers ----> some secondary bus
>                           |
>                           v
>                        memory
> 
> Here, all the buses are visible to the CPU, yet the path that
> transactions take between the buses is simply different to the CPU. More
> complex situations than the above, while still maintaining that
> description, are certainly possible.
> 

I tend to think in terms of links rather than buses.  A link is
effectively a 1:1 point-to-point bus that passes all transactions with
no modification.

So, although "some secondary bus" is visible to the CPUs, crucially the
link "some secondary bus" to "other devices" is not visible -- in the
sense that transactions issued by the CPUs never flow down that link.
Thus, if the link actually has remappings associated with it, then
devices mastering onto "some secondary bus" will observe those mappings
but the CPUs won't.  That's precisely what we need to know about when
configuring DMA buffers.

"invisible bus" situations are therefore a subset of "invisible link"
situations, and it is the latter which are the source of the complexity.

However, if the extra link(s) don't have any special characteristics, it
may be software-transparent with no need for description, because we
can pretend for logical purposes that there is a single bus in that case.
Effectively that what we've relied on for simpler systems up to now.

I'm assming in your example that the direct link between "primary CPU
bus" and "other devices" is always used by preference, instead of CPUs'
transactions toward "other devices" being sent to "some secondary bus"
first.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Tuesday 29 April 2014 19:16:02 Dave Martin wrote:
> On Mon, Apr 28, 2014 at 09:55:00PM +0200, Arnd Bergmann wrote:
> > On Monday 28 April 2014 20:30:56 Will Deacon wrote:
> >
> > > > 	device@4 {
> > > > 		compatible = "some,ethernet";
> > > > 		iommus = <&/iommu@1>;
> > > > 	};
> > > > 
> > > > 	device@5 {
> > > > 		compatible = "some,dmaengine";
> > > > 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> > > > 			 <&/iommu@3 0x101>;
> > > > 	};
> > > > 
> > > > The device at address 4 has a one-one relationship with iommu@1, so there
> > > > is no need for any data. device@5 has two master ports. One is connected to
> > > > an IOMMU that has a per-device aperture, device@5 can only issue transfers
> > > > to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> > > > this device into that address. The second master port is connected to
> > > > iommu@3, which uses a master ID that gets passed along with each transfer,
> > > > so that needs to be put into the IOTLBs.
> > > 
> > > I think this is definitely going in the right direction, but it's not clear
> > > to me how the driver for device@5 knows how to configure the two ports.
> > > We're still lacking topology information (unless that's implicit in the
> > > ordering of the properties) to say how the mastering capabilities of the
> > > device are actually routed and configured.
> > 
> > It would be helpful to have a concrete example of a device that has multiple
> > masters. I have heard people mention this multiple times, and I can understand
> 
> You mean "a device that contains multiple independent bus masters",
> right?  In particular, a device composed of multiple bus masters that
> do different things or should be handled differently by the interconnect.

Right.

> There has definitely been talk on the list about real devices that
> use multiple stream IDs.
> 
> 
> I'll ask around for "device-like" examples, but the most obvious
> example is the IOMMU itself.
> 
> Transactions generated by the IOMMU clearly need to be handled
> differently by the interconnect, compared with transactions
> translated and forwarded by IOMMU on behalf of its clients.
> 
> For example, suppose devices can post MSIs to an interrupt controller
> via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
> generates MSIs itself in order to signal management events or faults
> to a host OS.  Linux (as host) will need to configure the interrupt
> controller separately for the IOMMU and for the IOMMU clients.  This
> means that Linux needs to know which IDs may travel to the interrupt
> controller for which purpose, and they must be distinct.

I don't understand. An MSI controller is just an address that acts
as a DMA slave for a 4-byte inbound data packet. It has no way of
knowing who is sending data, other than by the address or the data
sent to it. Are you talking of something else?

> I'm not sure whether there is actually a SoC today that is MSI-capable
> and contains an IOMMU, but all the components to build one are out
> there today.  GICv3 is also explicitly designed to support such
> systems.

A lot of SoCs have MSI integrated into the PCI root complex, which
of course is pointless from MSI perspective, as well as implying that
the MSI won't go through the IOMMU.

We have briefly mentioned MSI in the review of the Samsung GH7 PCI
support. It's possible that this one can either use the built-in
MSI or the one in the GICv2m.

> In the future, it is likely that "HSA"-style GPUs and other high-
> throughput virtualisable bus mastering devices will have capabilities
> of this sort, but I don't think there's anything concrete yet.

Wouldn't they just have IOMMUs with multiple contexts?

> > how it might be wired up in hardware, but I don't know what it's good for,
> > or who would actually do it.
> > 
> > > > A variation would be to not use #iommu-cells at all, but provide a
> > > > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > > > as we do for dma-ranges. This is probably most flexible.
> > > 
> > > That would also allow us to describe ranges of master IDs, which we need for
> > > things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> > > these ranges could also be described like this, although I think Dave (CC'd)
> > > has some similar ideas in this area.
> 
> Ideally, we would reuse the ePAPR "ranges" concept and describe the way
> sideband ID signals propagate down the bus hierarchy in a similar way.

It would be 'dma-ranges'. Unfortunately that would imply that each DMA
master is connected to only one IOMMU, which you say is not necessarily
the case. The simpler case of a device is only a master on a single IOMMU
but can use multiple contexts would however work fine with dma-ranges.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
> On Tuesday 29 April 2014 19:16:02 Dave Martin wrote:

[...]

> > For example, suppose devices can post MSIs to an interrupt controller
> > via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
> > generates MSIs itself in order to signal management events or faults
> > to a host OS.  Linux (as host) will need to configure the interrupt
> > controller separately for the IOMMU and for the IOMMU clients.  This
> > means that Linux needs to know which IDs may travel to the interrupt
> > controller for which purpose, and they must be distinct.
> 
> I don't understand. An MSI controller is just an address that acts
> as a DMA slave for a 4-byte inbound data packet. It has no way of
> knowing who is sending data, other than by the address or the data
> sent to it. Are you talking of something else?

Oops, looks like there are a few points I failed to respond to here...


I'm not an expert on PCI -- I'm prepared to believe it works that way.

GICv3 can descriminate between different MSI senders based on ID
signals on the bus.

> 
> > I'm not sure whether there is actually a SoC today that is MSI-capable
> > and contains an IOMMU, but all the components to build one are out
> > there today.  GICv3 is also explicitly designed to support such
> > systems.
> 
> A lot of SoCs have MSI integrated into the PCI root complex, which
> of course is pointless from MSI perspective, as well as implying that
> the MSI won't go through the IOMMU.
> 
> We have briefly mentioned MSI in the review of the Samsung GH7 PCI
> support. It's possible that this one can either use the built-in
> MSI or the one in the GICv2m.

We are likely to get non-PCI MSIs in future SoC systems too, and there
are no standards governing how such systems should look.


> > In the future, it is likely that "HSA"-style GPUs and other high-
> > throughput virtualisable bus mastering devices will have capabilities
> > of this sort, but I don't think there's anything concrete yet.
> 
> Wouldn't they just have IOMMUs with multiple contexts?

Who knows?  A management component of the GPU that is under exclusive
control of the host or hypervisor might be wired up to bypass the IOMMU
completely.

I'm not saying this kind of thing definitely will happen, but I can't
say confidently that it won't.


> > > how it might be wired up in hardware, but I don't know what it's good for,
> > > or who would actually do it.
> > > 
> > > > > A variation would be to not use #iommu-cells at all, but provide a
> > > > > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > > > > as we do for dma-ranges. This is probably most flexible.
> > > > 
> > > > That would also allow us to describe ranges of master IDs, which we need for
> > > > things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> > > > these ranges could also be described like this, although I think Dave (CC'd)
> > > > has some similar ideas in this area.
> > 
> > Ideally, we would reuse the ePAPR "ranges" concept and describe the way
> > sideband ID signals propagate down the bus hierarchy in a similar way.
> 
> It would be 'dma-ranges'. Unfortunately that would imply that each DMA
> master is connected to only one IOMMU, which you say is not necessarily
> the case. The simpler case of a device is only a master on a single IOMMU
> but can use multiple contexts would however work fine with dma-ranges.

Partly, yes.  The concept embodied by "dma-ranges" is correct, but the
topological relationship is not: the assumption that a master device
always masters onto its parent node doesn't work for non-tree-like
topologies.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
> On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
> > On Tuesday 29 April 2014 19:16:02 Dave Martin wrote:
> 
> [...]
> 
> > > For example, suppose devices can post MSIs to an interrupt controller
> > > via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
> > > generates MSIs itself in order to signal management events or faults
> > > to a host OS.  Linux (as host) will need to configure the interrupt
> > > controller separately for the IOMMU and for the IOMMU clients.  This
> > > means that Linux needs to know which IDs may travel to the interrupt
> > > controller for which purpose, and they must be distinct.
> > 
> > I don't understand. An MSI controller is just an address that acts
> > as a DMA slave for a 4-byte inbound data packet. It has no way of
> > knowing who is sending data, other than by the address or the data
> > sent to it. Are you talking of something else?
> 
> Oops, looks like there are a few points I failed to respond to here...
> 
> 
> I'm not an expert on PCI -- I'm prepared to believe it works that way.
> 
> GICv3 can descriminate between different MSI senders based on ID
> signals on the bus.

Any idea what this is good for? Do we have to use it? It probably doesn't
fit very well into the way Linux handles MSIs today.

> > > I'm not sure whether there is actually a SoC today that is MSI-capable
> > > and contains an IOMMU, but all the components to build one are out
> > > there today.  GICv3 is also explicitly designed to support such
> > > systems.
> > 
> > A lot of SoCs have MSI integrated into the PCI root complex, which
> > of course is pointless from MSI perspective, as well as implying that
> > the MSI won't go through the IOMMU.
> > 
> > We have briefly mentioned MSI in the review of the Samsung GH7 PCI
> > support. It's possible that this one can either use the built-in
> > MSI or the one in the GICv2m.
> 
> We are likely to get non-PCI MSIs in future SoC systems too, and there
> are no standards governing how such systems should look.

I wouldn't call that MSI though -- using the same term in the code
can be rather confusing. There are existing SoCs that use message
based interrupt notification. We are probably better off modeling
those are regular irqchips in Linux and DT, given that they may
not be bound by the same constraints as PCI MSI.

> > > In the future, it is likely that "HSA"-style GPUs and other high-
> > > throughput virtualisable bus mastering devices will have capabilities
> > > of this sort, but I don't think there's anything concrete yet.
> > 
> > Wouldn't they just have IOMMUs with multiple contexts?
> 
> Who knows?  A management component of the GPU that is under exclusive
> control of the host or hypervisor might be wired up to bypass the IOMMU
> completely.
> 
> I'm not saying this kind of thing definitely will happen, but I can't
> say confidently that it won't.

Supporting this case in DT straight away is going to add a major burden.
If nobody can say for sure that they are actually going to do it, I'd
lean towards assuming that we won't need it and not putting the extra
complexity in.

If someone actually needs it later, let's make it their problem for
not participating in the design.

> > > > how it might be wired up in hardware, but I don't know what it's good for,
> > > > or who would actually do it.
> > > > 
> > > > > > A variation would be to not use #iommu-cells at all, but provide a
> > > > > > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > > > > > as we do for dma-ranges. This is probably most flexible.
> > > > > 
> > > > > That would also allow us to describe ranges of master IDs, which we need for
> > > > > things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> > > > > these ranges could also be described like this, although I think Dave (CC'd)
> > > > > has some similar ideas in this area.
> > > 
> > > Ideally, we would reuse the ePAPR "ranges" concept and describe the way
> > > sideband ID signals propagate down the bus hierarchy in a similar way.
> > 
> > It would be 'dma-ranges'. Unfortunately that would imply that each DMA
> > master is connected to only one IOMMU, which you say is not necessarily
> > the case. The simpler case of a device is only a master on a single IOMMU
> > but can use multiple contexts would however work fine with dma-ranges.
> 
> Partly, yes.  The concept embodied by "dma-ranges" is correct, but the
> topological relationship is not: the assumption that a master device
> always masters onto its parent node doesn't work for non-tree-like
> topologies.

In almost all cases it will fit. When it doesn't, we can work around it by
defining virtual address spaces the way that the PCI binding does. The only
major exception that we know we have to handle is IOMMUs.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Dave Martin]

On Thu, May 01, 2014 at 02:29:50PM +0100, Arnd Bergmann wrote:
> On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
> > On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
> > > On Tuesday 29 April 2014 19:16:02 Dave Martin wrote:
> > 
> > [...]
> > 
> > > > For example, suppose devices can post MSIs to an interrupt controller
> > > > via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
> > > > generates MSIs itself in order to signal management events or faults
> > > > to a host OS.  Linux (as host) will need to configure the interrupt
> > > > controller separately for the IOMMU and for the IOMMU clients.  This
> > > > means that Linux needs to know which IDs may travel to the interrupt
> > > > controller for which purpose, and they must be distinct.
> > > 
> > > I don't understand. An MSI controller is just an address that acts
> > > as a DMA slave for a 4-byte inbound data packet. It has no way of
> > > knowing who is sending data, other than by the address or the data
> > > sent to it. Are you talking of something else?
> > 
> > Oops, looks like there are a few points I failed to respond to here...
> > 
> > 
> > I'm not an expert on PCI -- I'm prepared to believe it works that way.
> > 
> > GICv3 can descriminate between different MSI senders based on ID
> > signals on the bus.
> 
> Any idea what this is good for? Do we have to use it? It probably doesn't
> fit very well into the way Linux handles MSIs today.

Marc may be better placed than me to comment on this in detail.

However, I believe it's correct to say that because the GIC is not part
of PCI, end-to-end MSI delivery inherently involves a non-PCI step from
the PCI RC to the GIC itself.

Thus this is likely to be a fundamental requirement for MSIs on ARM SoCs
using GIC, if we want to have a hope of mapping MSIs to VMs efficiently.

> > > > I'm not sure whether there is actually a SoC today that is MSI-capable
> > > > and contains an IOMMU, but all the components to build one are out
> > > > there today.  GICv3 is also explicitly designed to support such
> > > > systems.
> > > 
> > > A lot of SoCs have MSI integrated into the PCI root complex, which
> > > of course is pointless from MSI perspective, as well as implying that
> > > the MSI won't go through the IOMMU.
> > > 
> > > We have briefly mentioned MSI in the review of the Samsung GH7 PCI
> > > support. It's possible that this one can either use the built-in
> > > MSI or the one in the GICv2m.
> > 
> > We are likely to get non-PCI MSIs in future SoC systems too, and there
> > are no standards governing how such systems should look.
> 
> I wouldn't call that MSI though -- using the same term in the code
> can be rather confusing. There are existing SoCs that use message
> based interrupt notification. We are probably better off modeling
> those are regular irqchips in Linux and DT, given that they may
> not be bound by the same constraints as PCI MSI.

We can call it what we like and maybe bury the distinction in irqchip
drivers for some fixed-configuration cases, but it's logically the same
concept.  Naming and subsystem factoring are implementation decisions
for Linux.

For full dynamic assignment of pluggable devices or buses to VMs, I'm
less sure that we can model that as plain irqchips.

> > > > In the future, it is likely that "HSA"-style GPUs and other high-
> > > > throughput virtualisable bus mastering devices will have capabilities
> > > > of this sort, but I don't think there's anything concrete yet.
> > > 
> > > Wouldn't they just have IOMMUs with multiple contexts?
> > 
> > Who knows?  A management component of the GPU that is under exclusive
> > control of the host or hypervisor might be wired up to bypass the IOMMU
> > completely.
> > 
> > I'm not saying this kind of thing definitely will happen, but I can't
> > say confidently that it won't.
> 
> Supporting this case in DT straight away is going to add a major burden.
> If nobody can say for sure that they are actually going to do it, I'd
> lean towards assuming that we won't need it and not putting the extra
> complexity in.
> 
> If someone actually needs it later, let's make it their problem for
> not participating in the design.

This is a fair point, but there is a difference between the bindings and
what kind of wacky configurations a particular version of Linux actually
supports.

DT is supposed to be a description of the hardware, not a description
of how Linux subsystems are structured, though if the two are not
reasonably well aligned that will lead to pain...

The key thing is to make sure the DT bindings are extensible to
things that we can reasonably foresee.

> 
> > > > > how it might be wired up in hardware, but I don't know what it's good for,
> > > > > or who would actually do it.
> > > > > 
> > > > > > > A variation would be to not use #iommu-cells at all, but provide a
> > > > > > > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > > > > > > as we do for dma-ranges. This is probably most flexible.
> > > > > > 
> > > > > > That would also allow us to describe ranges of master IDs, which we need for
> > > > > > things like PCI RCs on the ARM SMMU. Furthermore, basic transformations of
> > > > > > these ranges could also be described like this, although I think Dave (CC'd)
> > > > > > has some similar ideas in this area.
> > > > 
> > > > Ideally, we would reuse the ePAPR "ranges" concept and describe the way
> > > > sideband ID signals propagate down the bus hierarchy in a similar way.
> > > 
> > > It would be 'dma-ranges'. Unfortunately that would imply that each DMA
> > > master is connected to only one IOMMU, which you say is not necessarily
> > > the case. The simpler case of a device is only a master on a single IOMMU
> > > but can use multiple contexts would however work fine with dma-ranges.
> > 
> > Partly, yes.  The concept embodied by "dma-ranges" is correct, but the
> > topological relationship is not: the assumption that a master device
> > always masters onto its parent node doesn't work for non-tree-like
> > topologies.
> 
> In almost all cases it will fit. When it doesn't, we can work around it by
> defining virtual address spaces the way that the PCI binding does. The only
> major exception that we know we have to handle is IOMMUs.

My concern here is that as new exceptions and oddball or complex systems
crop up, we will end up repeatedly inventing different bodges to solve
essentially the same problem.

Unlike some of the other situations we have to deal with, these are valid
hardware configurations rather than quirks or broken systems.


A more uniform approach is not necessarily a win, but it is worth discussing.
That will be easier with a bit more concrete detail -- I'll follow up with
something that I hope will focus the discussion a bit on this point.

Cheers
---Dave

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Marc Zyngier]

On 01/05/14 15:36, Dave Martin wrote:
> On Thu, May 01, 2014 at 02:29:50PM +0100, Arnd Bergmann wrote:
>> On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
>>> On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
>>>> On Tuesday 29 April 2014 19:16:02 Dave Martin wrote:
>>>
>>> [...]
>>>
>>>>> For example, suppose devices can post MSIs to an interrupt controller
>>>>> via a mailbox accessed through the IOMMU.  Suppose also that the IOMMU
>>>>> generates MSIs itself in order to signal management events or faults
>>>>> to a host OS.  Linux (as host) will need to configure the interrupt
>>>>> controller separately for the IOMMU and for the IOMMU clients.  This
>>>>> means that Linux needs to know which IDs may travel to the interrupt
>>>>> controller for which purpose, and they must be distinct.
>>>>
>>>> I don't understand. An MSI controller is just an address that acts
>>>> as a DMA slave for a 4-byte inbound data packet. It has no way of
>>>> knowing who is sending data, other than by the address or the data
>>>> sent to it. Are you talking of something else?
>>>
>>> Oops, looks like there are a few points I failed to respond to here...
>>>
>>>
>>> I'm not an expert on PCI -- I'm prepared to believe it works that way.
>>>
>>> GICv3 can descriminate between different MSI senders based on ID
>>> signals on the bus.
>>
>> Any idea what this is good for? Do we have to use it? It probably doesn't
>> fit very well into the way Linux handles MSIs today.
> 
> Marc may be better placed than me to comment on this in detail.

As to "fitting Linux", it seems to match what Linux does fairly well
(see the kvm-arm64/gicv3 branch in my tree). Not saying that it does it
in a very simple way (far from it, actually), but it works.

As to "what it is good for" (and before someone bursts into an Edwin
Starr interpretation), it mostly has to do with isolation, and the fact
that you may want to let the whole MSI programming to a guest (and yet
ensure that the guest cannot generate interrupts that would be assigned
to other devices). This is done by sampling the requester-id at the ITS
level, and use this information to index a per-device interrupt
translation table (I could talk for hours about the concept and its
variations, mostly using expletives and possibly a hammer, but I think
it is the time for my pink pill).

> However, I believe it's correct to say that because the GIC is not part
> of PCI, end-to-end MSI delivery inherently involves a non-PCI step from
> the PCI RC to the GIC itself.
> 
> Thus this is likely to be a fundamental requirement for MSIs on ARM SoCs
> using GIC, if we want to have a hope of mapping MSIs to VMs efficiently.

Indeed. GICv[34] is the ARM way of implementing MSI on SBSA compliant
systems (from level 1 onwards, if memory serves well). People are
actively building systems with this architecture, and relying on it to
provide VM isolation.

>>>>> I'm not sure whether there is actually a SoC today that is MSI-capable
>>>>> and contains an IOMMU, but all the components to build one are out
>>>>> there today.  GICv3 is also explicitly designed to support such
>>>>> systems.
>>>>
>>>> A lot of SoCs have MSI integrated into the PCI root complex, which
>>>> of course is pointless from MSI perspective, as well as implying that
>>>> the MSI won't go through the IOMMU.
>>>>
>>>> We have briefly mentioned MSI in the review of the Samsung GH7 PCI
>>>> support. It's possible that this one can either use the built-in
>>>> MSI or the one in the GICv2m.
>>>
>>> We are likely to get non-PCI MSIs in future SoC systems too, and there
>>> are no standards governing how such systems should look.
>>
>> I wouldn't call that MSI though -- using the same term in the code
>> can be rather confusing. There are existing SoCs that use message
>> based interrupt notification. We are probably better off modeling
>> those are regular irqchips in Linux and DT, given that they may
>> not be bound by the same constraints as PCI MSI.
> 
> We can call it what we like and maybe bury the distinction in irqchip
> drivers for some fixed-configuration cases, but it's logically the same
> concept.  Naming and subsystem factoring are implementation decisions
> for Linux.
> 
> For full dynamic assignment of pluggable devices or buses to VMs, I'm
> less sure that we can model that as plain irqchips.

Yeah, I've been looking at that. For some restricted cases, the irqchip
model works very well (think of wire to "MSI" translators, which are
likely to have a fixed configuration). Anything more dynamic requires a
more evolved infrastructure, but I'd hope they would also be on a
discoverable bus, removing most of the need for description in DT.

Cheers,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Thursday 01 May 2014 16:11:48 Marc Zyngier wrote:
> On 01/05/14 15:36, Dave Martin wrote:
> > On Thu, May 01, 2014 at 02:29:50PM +0100, Arnd Bergmann wrote:
> >> On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
> >>> On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
> >>>> I don't understand. An MSI controller is just an address that acts
> >>>> as a DMA slave for a 4-byte inbound data packet. It has no way of
> >>>> knowing who is sending data, other than by the address or the data
> >>>> sent to it. Are you talking of something else?
> >>>
> >>> Oops, looks like there are a few points I failed to respond to here...
> >>>
> >>>
> >>> I'm not an expert on PCI -- I'm prepared to believe it works that way.
> >>>
> >>> GICv3 can descriminate between different MSI senders based on ID
> >>> signals on the bus.
> >>
> >> Any idea what this is good for? Do we have to use it? It probably doesn't
> >> fit very well into the way Linux handles MSIs today.
> > 
> > Marc may be better placed than me to comment on this in detail.
> 
> As to "fitting Linux", it seems to match what Linux does fairly well
> (see the kvm-arm64/gicv3 branch in my tree). Not saying that it does it
> in a very simple way (far from it, actually), but it works.

ok.

> As to "what it is good for" (and before someone bursts into an Edwin
> Starr interpretation), it mostly has to do with isolation, and the fact
> that you may want to let the whole MSI programming to a guest (and yet
> ensure that the guest cannot generate interrupts that would be assigned
> to other devices). This is done by sampling the requester-id at the ITS
> level, and use this information to index a per-device interrupt
> translation table (I could talk for hours about the concept and its
> variations, mostly using expletives and possibly a hammer, but I think
> it is the time for my pink pill).

So the idea is that you want to give the guest unfiltered access to the
PCI config space of an assigned device?

Is that safe?

If the config space access goes through the hypervisor (as I think we
always do today), there should be no isse.

> > However, I believe it's correct to say that because the GIC is not part
> > of PCI, end-to-end MSI delivery inherently involves a non-PCI step from
> > the PCI RC to the GIC itself.
> > 
> > Thus this is likely to be a fundamental requirement for MSIs on ARM SoCs
> > using GIC, if we want to have a hope of mapping MSIs to VMs efficiently.
> 
> Indeed. GICv[34] is the ARM way of implementing MSI on SBSA compliant
> systems (from level 1 onwards, if memory serves well). People are
> actively building systems with this architecture, and relying on it to
> provide VM isolation.

I don't mind it being there, but if we don't need it, we shouldn't
have to use that isolation feature and be able to just allow any
initiator to send an MSI.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Marc Zyngier]

On 01/05/14 16:53, Arnd Bergmann wrote:
> On Thursday 01 May 2014 16:11:48 Marc Zyngier wrote:
>> On 01/05/14 15:36, Dave Martin wrote:
>>> On Thu, May 01, 2014 at 02:29:50PM +0100, Arnd Bergmann wrote:
>>>> On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
>>>>> On Tue, Apr 29, 2014 at 10:46:18PM +0200, Arnd Bergmann wrote:
>>>>>> I don't understand. An MSI controller is just an address that acts
>>>>>> as a DMA slave for a 4-byte inbound data packet. It has no way of
>>>>>> knowing who is sending data, other than by the address or the data
>>>>>> sent to it. Are you talking of something else?
>>>>>
>>>>> Oops, looks like there are a few points I failed to respond to here...
>>>>>
>>>>>
>>>>> I'm not an expert on PCI -- I'm prepared to believe it works that way.
>>>>>
>>>>> GICv3 can descriminate between different MSI senders based on ID
>>>>> signals on the bus.
>>>>
>>>> Any idea what this is good for? Do we have to use it? It probably doesn't
>>>> fit very well into the way Linux handles MSIs today.
>>>
>>> Marc may be better placed than me to comment on this in detail.
>>
>> As to "fitting Linux", it seems to match what Linux does fairly well
>> (see the kvm-arm64/gicv3 branch in my tree). Not saying that it does it
>> in a very simple way (far from it, actually), but it works.
> 
> ok.
> 
>> As to "what it is good for" (and before someone bursts into an Edwin
>> Starr interpretation), it mostly has to do with isolation, and the fact
>> that you may want to let the whole MSI programming to a guest (and yet
>> ensure that the guest cannot generate interrupts that would be assigned
>> to other devices). This is done by sampling the requester-id at the ITS
>> level, and use this information to index a per-device interrupt
>> translation table (I could talk for hours about the concept and its
>> variations, mostly using expletives and possibly a hammer, but I think
>> it is the time for my pink pill).
> 
> So the idea is that you want to give the guest unfiltered access to the
> PCI config space of an assigned device?
> 
> Is that safe?
> 
> If the config space access goes through the hypervisor (as I think we
> always do today), there should be no isse.

My natural tendency would be to filter everything, as we certainly don't
want the device to be reconfigured in weird and fancy ways (my PCI-foo
is a bit limited, so bear with me). But as usual, GICv3 is not PCI
specific, and is designed to cater for weird and wonderful cases, PCI
being only one possible implementation...

On a non-PCI system, you definitely could assign a MSI-like capable
device, and let the guest do its thing. I'm already seeing that kind of
design.

>>> However, I believe it's correct to say that because the GIC is not part
>>> of PCI, end-to-end MSI delivery inherently involves a non-PCI step from
>>> the PCI RC to the GIC itself.
>>>
>>> Thus this is likely to be a fundamental requirement for MSIs on ARM SoCs
>>> using GIC, if we want to have a hope of mapping MSIs to VMs efficiently.
>>
>> Indeed. GICv[34] is the ARM way of implementing MSI on SBSA compliant
>> systems (from level 1 onwards, if memory serves well). People are
>> actively building systems with this architecture, and relying on it to
>> provide VM isolation.
> 
> I don't mind it being there, but if we don't need it, we shouldn't
> have to use that isolation feature and be able to just allow any
> initiator to send an MSI.

It is not that simple. You could design things around a global ITT,
pointed to by all the possible devices in your system. But that table
would be quite big (it has to contain the routing information for all
the possible messages in your system, and must be physically
contiguous). You still need to know the devices (requester-id,
device-id, whatever), as there is no wildcard mechanism.

Also, this approach pretty much kills hotplug (you cannot easily resize
that table to add new interrupt entries). That's why I've opted for a
more dynamic configuration, where each device gets its own ITT as it
appears on the system.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Arnd Bergmann]

On Thursday 01 May 2014 15:36:54 Dave Martin wrote:
> On Thu, May 01, 2014 at 02:29:50PM +0100, Arnd Bergmann wrote:
> > On Thursday 01 May 2014 12:15:35 Dave Martin wrote:
> > > > > I'm not sure whether there is actually a SoC today that is MSI-capable
> > > > > and contains an IOMMU, but all the components to build one are out
> > > > > there today.  GICv3 is also explicitly designed to support such
> > > > > systems.
> > > > 
> > > > A lot of SoCs have MSI integrated into the PCI root complex, which
> > > > of course is pointless from MSI perspective, as well as implying that
> > > > the MSI won't go through the IOMMU.
> > > > 
> > > > We have briefly mentioned MSI in the review of the Samsung GH7 PCI
> > > > support. It's possible that this one can either use the built-in
> > > > MSI or the one in the GICv2m.
> > > 
> > > We are likely to get non-PCI MSIs in future SoC systems too, and there
> > > are no standards governing how such systems should look.
> > 
> > I wouldn't call that MSI though -- using the same term in the code
> > can be rather confusing. There are existing SoCs that use message
> > based interrupt notification. We are probably better off modeling
> > those are regular irqchips in Linux and DT, given that they may
> > not be bound by the same constraints as PCI MSI.
> 
> We can call it what we like and maybe bury the distinction in irqchip
> drivers for some fixed-configuration cases, but it's logically the same
> concept.  Naming and subsystem factoring are implementation decisions
> for Linux.
> 
> For full dynamic assignment of pluggable devices or buses to VMs, I'm
> less sure that we can model that as plain irqchips.

I definitely hope we won't have to deal with plugging non-PCI devices
into VMs. Nothing good can come out of that.

> > Supporting this case in DT straight away is going to add a major burden.
> > If nobody can say for sure that they are actually going to do it, I'd
> > lean towards assuming that we won't need it and not putting the extra
> > complexity in.
> > 
> > If someone actually needs it later, let's make it their problem for
> > not participating in the design.
> 
> This is a fair point, but there is a difference between the bindings and
> what kind of wacky configurations a particular version of Linux actually
> supports.
> 
> DT is supposed to be a description of the hardware, not a description
> of how Linux subsystems are structured, though if the two are not
> reasonably well aligned that will lead to pain...
> 
> The key thing is to make sure the DT bindings are extensible to
> things that we can reasonably foresee.

Yes, defining them in an extensible way is always a good idea, but I
think it would be better not to define the fine details until we
actually need them in this case.

> > > > It would be 'dma-ranges'. Unfortunately that would imply that each DMA
> > > > master is connected to only one IOMMU, which you say is not necessarily
> > > > the case. The simpler case of a device is only a master on a single IOMMU
> > > > but can use multiple contexts would however work fine with dma-ranges.
> > > 
> > > Partly, yes.  The concept embodied by "dma-ranges" is correct, but the
> > > topological relationship is not: the assumption that a master device
> > > always masters onto its parent node doesn't work for non-tree-like
> > > topologies.
> > 
> > In almost all cases it will fit. When it doesn't, we can work around it by
> > defining virtual address spaces the way that the PCI binding does. The only
> > major exception that we know we have to handle is IOMMUs.
> 
> My concern here is that as new exceptions and oddball or complex systems
> crop up, we will end up repeatedly inventing different bodges to solve
> essentially the same problem.
> 
> Unlike some of the other situations we have to deal with, these are valid
> hardware configurations rather than quirks or broken systems.

Can you give an example where this would be done for a good reason?
I can't come up with an example that doesn't involve the hardware
design being seriously screwed.

	Arnd

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Grant Grundler]

On Thu, May 1, 2014 at 6:29 AM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote:
...
>> GICv3 can descriminate between different MSI senders based on ID
>> signals on the bus.
>
> Any idea what this is good for? Do we have to use it? It probably doesn't
> fit very well into the way Linux handles MSIs today.

I can see this being used for diagnosing failures - e.g. hung system
would leave tracks if an interrupt was or was not provided by the
device. I can't think of a reason why Linux MSI code would need to
support this though.

...
>> We are likely to get non-PCI MSIs in future SoC systems too, and there
>> are no standards governing how such systems should look.

Why look to the future when one can look in the past? :)

PA-RISC was designed in the 1980s to use "MSI" to generate all CPU
interrupts. This is as simple as it gets.

The concept is identical to MSI: MMIO routeable address with a payload
to indicate interrupt source (or map that source to some "vector
table".) Look at arch/parisc/kernel/smp.c:ipi_send() for an example
(note "p->hpa" is "processor->host physical address").

Current and  future products do the same things but add more
"features" that make this more complicated. But the basic transaction
will be identical and it needs to be routed like any other MMIO
transaction by any bridge (including IOMMUs).

> I wouldn't call that MSI though -- using the same term in the code
> can be rather confusing. There are existing SoCs that use message
> based interrupt notification. We are probably better off modeling
> those are regular irqchips in Linux and DT, given that they may
> not be bound by the same constraints as PCI MSI.

PCI device is one "source" for MSI and PCI defines how to initialize
an MSI source.  The target is not PCI (or not even Intel). Intel
defines how MSI works on their chipsets/CPUs. Other can still do it
differently.

I'm perfectly ok with using "MSI" to refer to any "in band interrupt message".

>> Who knows?  A management component of the GPU that is under exclusive
>> control of the host or hypervisor might be wired up to bypass the IOMMU
>> completely.

Does the linux kernel need to know about a device/component that it
can't control?
Either linux kernel shouldn't be told about it OR should ignore this
device/component.

>> Partly, yes.  The concept embodied by "dma-ranges" is correct, but the
>> topological relationship is not: the assumption that a master device
>> always masters onto its parent node doesn't work for non-tree-like
>> topologies.
>
> In almost all cases it will fit. When it doesn't, we can work around it by
> defining virtual address spaces the way that the PCI binding does. The only
> major exception that we know we have to handle is IOMMUs.

MMIO routing (and thus dma-ranges) is a "graph" (very comparable to
network routing). Some simple (e.g. PCI) implementations look like a
tree - but that's not the general case.

DMA cares about more than routing though - cache coherency and
performance (BW and latency) matter too.

cheers,
grant

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Thierry Reding]

[-- Attachment #1.1: Type: text/plain, Size: 2575 bytes --]

On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
[...]
> let me clarify by example:
> 
> 	iommu@1 {
> 		compatible = "some,simple-iommu";
> 		reg = <1>;
> 		#iommu-cells = <0>; /* supports only one master */
> 	};
> 
> 	iommu@2 {
> 		compatible = "some,other-iommu";
> 		reg = <3>;
> 		#iommu-cells = <1>; /* contains master ID */
> 	};
> 
> 	iommu@3 {
> 		compatible = "some,windowed-iommu";
> 		reg = <2>;
> 		#iommu-cells = <2>; /* contains dma-window */
> 	};
> 
> 	device@4 {
> 		compatible = "some,ethernet";
> 		iommus = <&/iommu@1>;
> 	};
> 
> 	device@5 {
> 		compatible = "some,dmaengine";
> 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> 			 <&/iommu@3 0x101>;
> 	};
> 
> The device at address 4 has a one-one relationship with iommu@1, so there
> is no need for any data. device@5 has two master ports. One is connected to
> an IOMMU that has a per-device aperture, device@5 can only issue transfers
> to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> this device into that address. The second master port is connected to
> iommu@3, which uses a master ID that gets passed along with each transfer,
> so that needs to be put into the IOTLBs.

iommu@3 and the second port of device@5 seem to match what we need for
Tegra (and as I understand also Exynos). Can we settle on this for now
so that Hiroshi and Cho can go update their drivers for this binding?

> A variation would be to not use #iommu-cells at all, but provide a
> #address-cells / #size-cells pair in the IOMMU, and have a translation
> as we do for dma-ranges. This is probably most flexible.

The remainder of this discussion seems to indicate that #iommu-cells and
dma-ranges don't have to be mutually exclusive. For some IOMMUs it might
make sense to use both.

In fact perhaps we should require every IOMMU user to also specify a
dma-ranges property, even if for some cases the range would be simply
the complete physical address space. Perhaps in analogy to the ranges
property an empty dma-ranges property could be taken to mean all of the
physical address space.

I'm aware that this doesn't cover any of the more exotic cases out
there, but the fact is that we have real devices out there that ship
with some variations of these simple IOMMUs and I don't think we're
doing ourselves a favour by blocking support for these to be added on
the hope of merging the perfect solution that covers all use-cases.
Patches for Tegra have already been around for close to half a year.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Cho KyongHo]

On Thu, 15 May 2014 22:37:31 +0200, Thierry Reding wrote:
> On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
> [...]
> > let me clarify by example:
> > 
> > 	iommu@1 {
> > 		compatible = "some,simple-iommu";
> > 		reg = <1>;
> > 		#iommu-cells = <0>; /* supports only one master */
> > 	};
> > 
> > 	iommu@2 {
> > 		compatible = "some,other-iommu";
> > 		reg = <3>;
> > 		#iommu-cells = <1>; /* contains master ID */
> > 	};
> > 
> > 	iommu@3 {
> > 		compatible = "some,windowed-iommu";
> > 		reg = <2>;
> > 		#iommu-cells = <2>; /* contains dma-window */
> > 	};
> > 
> > 	device@4 {
> > 		compatible = "some,ethernet";
> > 		iommus = <&/iommu@1>;
> > 	};
> > 
> > 	device@5 {
> > 		compatible = "some,dmaengine";
> > 		iommus = <&/iommu@2 0x40000000 0x1000000>,
> > 			 <&/iommu@3 0x101>;
> > 	};
> > 
> > The device at address 4 has a one-one relationship with iommu@1, so there
> > is no need for any data. device@5 has two master ports. One is connected to
> > an IOMMU that has a per-device aperture, device@5 can only issue transfers
> > to the 256MB area at 0x40000000, and the IOMMU will have to put entries for
> > this device into that address. The second master port is connected to
> > iommu@3, which uses a master ID that gets passed along with each transfer,
> > so that needs to be put into the IOTLBs.
> 
> iommu@3 and the second port of device@5 seem to match what we need for
> Tegra (and as I understand also Exynos). Can we settle on this for now
> so that Hiroshi and Cho can go update their drivers for this binding?
> 

Currently, Exynos IOMMU is the case of iommu@1.

But in the near future, it will support multiple masters with a single context
that means all masters that shares a single System MMU also views the same
address space.

For some cases, we may need iommu@3 that supports dma-window.

So, I have no other opinion.

By the way, iommu framework should allow to process the parameters
to 'iommus' property in the master nodes by iommu driver implementations
because it is depended on implementations.

> > A variation would be to not use #iommu-cells at all, but provide a
> > #address-cells / #size-cells pair in the IOMMU, and have a translation
> > as we do for dma-ranges. This is probably most flexible.
> 
> The remainder of this discussion seems to indicate that #iommu-cells and
> dma-ranges don't have to be mutually exclusive. For some IOMMUs it might
> make sense to use both.
> 
> In fact perhaps we should require every IOMMU user to also specify a
> dma-ranges property, even if for some cases the range would be simply
> the complete physical address space. Perhaps in analogy to the ranges
> property an empty dma-ranges property could be taken to mean all of the
> physical address space.
> 
> I'm aware that this doesn't cover any of the more exotic cases out
> there, but the fact is that we have real devices out there that ship
> with some variations of these simple IOMMUs and I don't think we're
> doing ourselves a favour by blocking support for these to be added on
> the hope of merging the perfect solution that covers all use-cases.
> Patches for Tegra have already been around for close to half a year.
> 
> Thierry

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Stephen Warren]

On 04/28/2014 05:18 AM, Thierry Reding wrote:
> On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
...
>> A lot of drivers probably only support one
>> master, so they can just set #iommu-cells=<0>, others might require
>> IDs that do not fit into one cell.
> 
> You mean "#iommu-cells = <1>" for devices that only require one master?
> There still has to be one cell to specify which master. Unless perhaps
> if they can be arbitrarily assigned. I guess even if there's a fixed
> mapping that applies to one SoC generation, it might be good to still
> employ a specifier and have the mapping in DT for flexibility.

#iommu-cells doesn't include the phandle, so if you want the client
references to be:

property = <&iommu>;

then that's #iommu-cells=<0>, whereas:

property = <&iommu N>;

is #iommu-cells=<1>.

Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU

[Hiroshi Doyu]

Thierry Reding <thierry.reding@gmail.com> writes:

> * PGP Signed by an unknown key
>
> On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
>> On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
>> > +- mmu-masters: A phandle to device nodes representing the master for which
>> > +               the System MMU can provide a translation. Any additional values
>> > +              after the phandle will be ignored because a System MMU never
>> > +              have two or more masters. "#stream-id-cells" specified in the
>> > +              master's node will be also ignored.
>> > +              If more than one phandle is specified, only the first phandle
>> > +              will be treated.
>> 
>> This seems completely backwards: Why would you list the masters for an IOMMU
>> in the IOMMU node?
>> 
>> The master should have a standard property pointing to the IOMMU instead.
>> 
>> We don't have a generic binding for IOMMUs yet it seems, but the time is
>> overdue to make one.
>> 
>> Consider this NAKed until there is a generic binding for IOMMUs that all
>> relevant developers have agreed to.
>
> I'd like to take this opportunity and revive one of the hibernating
> patch sets that we have for Tegra. The last effort to get things merged
> was back in January I think. I haven't bothered to look up the reference
> since it's probably good to start from scratch anyway.
>
> The latest version of the binding that was under discussion back then I
> think looked something like this:
>
> 	device@... {
> 		iommus = <&iommu [spec]>[, <&other_iommu [other_spec]>...];
> 	};
>
> And possibly with a iommu-names property to go along with that. The idea
> being that a device can be a master on possibly multiple IOMMUs. Using
> the above it would also be possible to have one device be multiple
> masters on the same IOMMU.
>
> On Tegra the specifier would be used to encode a memory controller's
> client ID. One discussion point back at the time was to encode the ID as
> a bitmask to allow more than a single master per entry. Another solution
> which I think is a little cleaner and more generic, would be to use one
> entry per master and use a single cell to encode the client ID. Devices
> with multiple clients to the same IOMMU could then use multiple entries
> referencing the same IOMMU.
>
> I've added Hiroshi Doyu on Cc since he knows the Tegra IOMMU best.
> Hiroshi, can you summarize exactly what the proposed bindings were. If
> my memory serves me well they were mostly along the lines of what Arnd
> proposes here, and perhaps they are something that can also be used for
> Exynos.

You can find the detail from:

[PATCHv7 09/12] iommu/tegra: smmu: get swgroups from DT "iommus="
  http://lists.linuxfoundation.org/pipermail/iommu/2013-December/007212.html

You can specify any parameters which your iommu requires from a device,
and a device(master) can have multiple IOMMUs.

 	device@... {
 		iommus = <&iommu [spec]>[, <&other_iommu [other_spec]>...];
 	};

[PATCH v12 12/31] iommu/exynos: support for device tree

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This commit adds device tree support for System MMU.
It also enables iommu support for ARCH_EXYNOS.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/Kconfig             |    2 ++
 drivers/iommu/Kconfig        |    8 +++-----
 drivers/iommu/exynos-iommu.c |   19 +++++++++++++++----
 3 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index ab438cb..19689e4 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -834,6 +834,8 @@ config ARCH_EXYNOS
 	select ARCH_HAS_HOLES_MEMORYMODEL
 	select ARCH_REQUIRE_GPIOLIB
 	select ARCH_SPARSEMEM_ENABLE
+	select ARM_AMBA
+	select ARM_DMA_USE_IOMMU
 	select ARM_GIC
 	select COMMON_CLK
 	select CPU_V7
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index df56e4c..5feadb8 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -178,16 +178,14 @@ config TEGRA_IOMMU_SMMU
 
 config EXYNOS_IOMMU
 	bool "Exynos IOMMU Support"
-	depends on ARCH_EXYNOS && EXYNOS_DEV_SYSMMU
+	depends on ARCH_EXYNOS
 	select IOMMU_API
 	help
-	  Support for the IOMMU(System MMU) of Samsung Exynos application
-	  processor family. This enables H/W multimedia accellerators to see
+	  Support for the IOMMU (System MMU) of Samsung Exynos application
+	  processor family. This enables H/W multimedia accelerators to see
 	  non-linear physical memory chunks as a linear memory in their
 	  address spaces
 
-	  If unsure, say N here.
-
 config EXYNOS_IOMMU_DEBUG
 	bool "Debugging log for Exynos IOMMU"
 	depends on EXYNOS_IOMMU
diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 0f1d3f0..cbb9b67 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -25,6 +25,7 @@
 #include <linux/list.h>
 #include <linux/memblock.h>
 #include <linux/export.h>
+#include <linux/of.h>
 
 #include <asm/cacheflush.h>
 #include <asm/pgtable.h>
@@ -494,7 +495,7 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 	read_unlock_irqrestore(&data->lock, flags);
 }
 
-static int exynos_sysmmu_probe(struct platform_device *pdev)
+static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 {
 	int irq, ret;
 	struct device *dev = &pdev->dev;
@@ -548,11 +549,21 @@ static int exynos_sysmmu_probe(struct platform_device *pdev)
 	return 0;
 }
 
-static struct platform_driver exynos_sysmmu_driver = {
-	.probe		= exynos_sysmmu_probe,
-	.driver		= {
+static const struct of_device_id sysmmu_of_match[] __initconst = {
+	{ .compatible	= "samsung,sysmmu-v1", },
+	{ .compatible	= "samsung,sysmmu-v2", },
+	{ .compatible	= "samsung,sysmmu-v3.1", },
+	{ .compatible	= "samsung,sysmmu-v3.2", },
+	{ .compatible	= "samsung,sysmmu-v3.3", },
+	{ },
+};
+
+static struct platform_driver exynos_sysmmu_driver __refdata = {
+	.probe	= exynos_sysmmu_probe,
+	.driver	= {
 		.owner		= THIS_MODULE,
 		.name		= "exynos-sysmmu",
+		.of_match_table	= sysmmu_of_match,
 	}
 };
 
-- 
1.7.9.5

[PATCH v12 13/31] iommu/exynos: gating clocks of master H/W

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This patch gates clocks of master H/W as well as clocks of System MMU
if master clocks are specified.

Some Exynos SoCs (i.e. GScalers in Exynos5250) have dependencies in
the gating clocks of master H/W and its System MMU. If a H/W is the
case, accessing control registers of System MMU is prohibited unless
both of the gating clocks of System MMU and its master H/W.

CC: Tomasz Figa <t.figa@samsung.com>
Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index cbb9b67..ba477c4 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -172,6 +172,7 @@ struct sysmmu_drvdata {
 	struct device *dev;	/* Owner of system MMU */
 	void __iomem *sfrbase;
 	struct clk *clk;
+	struct clk *clk_master;
 	int activations;
 	rwlock_t lock;
 	struct iommu_domain *domain;
@@ -300,6 +301,8 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 
 	WARN_ON(!is_sysmmu_active(data));
 
+	if (!IS_ERR(data->clk_master))
+		clk_enable(data->clk_master);
 	itype = (enum exynos_sysmmu_inttype)
 		__ffs(__raw_readl(data->sfrbase + REG_INT_STATUS));
 	if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
@@ -326,6 +329,9 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	if (itype != SYSMMU_FAULT_UNKNOWN)
 		sysmmu_unblock(data->sfrbase);
 
+	if (!IS_ERR(data->clk_master))
+		clk_disable(data->clk_master);
+
 	read_unlock(&data->lock);
 
 	return IRQ_HANDLED;
@@ -341,9 +347,14 @@ static bool __exynos_sysmmu_disable(struct sysmmu_drvdata *data)
 	if (!set_sysmmu_inactive(data))
 		goto finish;
 
+	if (!IS_ERR(data->clk_master))
+		clk_enable(data->clk_master);
+
 	__raw_writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
 
 	clk_disable(data->clk);
+	if (!IS_ERR(data->clk_master))
+		clk_disable(data->clk_master);
 
 	disabled = true;
 	data->pgtable = 0;
@@ -386,14 +397,19 @@ static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
 		goto finish;
 	}
 
-	clk_enable(data->clk);
-
 	data->pgtable = pgtable;
 
+	if (!IS_ERR(data->clk_master))
+		clk_enable(data->clk_master);
+	clk_enable(data->clk);
+
 	__sysmmu_set_ptbase(data->sfrbase, pgtable);
 
 	__raw_writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL);
 
+	if (!IS_ERR(data->clk_master))
+		clk_disable(data->clk_master);
+
 	data->domain = domain;
 
 	dev_dbg(data->sysmmu, "Enabled\n");
@@ -450,6 +466,10 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 	if (is_sysmmu_active(data)) {
 		unsigned int maj;
 		unsigned int num_inv = 1;
+
+		if (!IS_ERR(data->clk_master))
+			clk_enable(data->clk_master);
+
 		maj = __raw_readl(data->sfrbase + REG_MMU_VERSION);
 		/*
 		 * L2TLB invalidation required
@@ -469,6 +489,8 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 							num_inv);
 			sysmmu_unblock(data->sfrbase);
 		}
+		if (!IS_ERR(data->clk_master))
+			clk_disable(data->clk_master);
 	} else {
 		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
 	}
@@ -484,10 +506,14 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 	read_lock_irqsave(&data->lock, flags);
 
 	if (is_sysmmu_active(data)) {
+		if (!IS_ERR(data->clk_master))
+			clk_enable(data->clk_master);
 		if (sysmmu_block(data->sfrbase)) {
 			__sysmmu_tlb_invalidate(data->sfrbase);
 			sysmmu_unblock(data->sfrbase);
 		}
+		if (!IS_ERR(data->clk_master))
+			clk_disable(data->clk_master);
 	} else {
 		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
 	}
@@ -536,6 +562,16 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 		}
 	}
 
+	data->clk_master = devm_clk_get(dev, "master");
+	if (!IS_ERR(data->clk_master)) {
+		ret = clk_prepare(data->clk_master);
+		if (ret) {
+			clk_unprepare(data->clk);
+			dev_err(dev, "Failed to prepare master's clk\n");
+			return ret;
+		}
+	}
+
 	data->sysmmu = dev;
 	rwlock_init(&data->lock);
 	INIT_LIST_HEAD(&data->node);
-- 
1.7.9.5

[PATCH v12 15/31] iommu/exynos: handle 'mmu-masters' property of DT and improve handling sysmmu

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

"iommu/exynos: support for device tree" patch just binds exynos-iommu
driver with the compatible strings of System MMUs but the important
information of the DT, 'mmu-masters' is not processed by the driver.
This patch process 'mmu-masters' so that the master device driver can
handle its system mmu.

Also, system mmu handling is improved. Previously, an IOMMU domain is
bound to a System MMU which is not correct. This patch binds an IOMMU
domain with the master device of a System MMU.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |  381 ++++++++++++++++++++++++++++--------------
 1 file changed, 253 insertions(+), 128 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index b07b78b..45c792c 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -26,6 +26,8 @@
 #include <linux/memblock.h>
 #include <linux/export.h>
 #include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/notifier.h>
 
 #include <asm/cacheflush.h>
 #include <asm/pgtable.h>
@@ -100,6 +102,8 @@
 #define REG_PB1_SADDR		0x054
 #define REG_PB1_EADDR		0x058
 
+#define has_sysmmu(dev)		(dev->archdata.iommu != NULL)
+
 static struct kmem_cache *lv2table_kmem_cache;
 
 static unsigned long *section_entry(unsigned long *pgtable, unsigned long iova)
@@ -148,6 +152,16 @@ static char *sysmmu_fault_name[SYSMMU_FAULTS_NUM] = {
 	"UNKNOWN FAULT"
 };
 
+/* attached to dev.archdata.iommu of the master device */
+struct exynos_iommu_owner {
+	struct list_head client; /* entry of exynos_iommu_domain.clients */
+	struct device *dev;
+	struct device *sysmmu;
+	struct iommu_domain *domain;
+	void *vmm_data;         /* IO virtual memory manager's data */
+	spinlock_t lock;        /* Lock to preserve consistency of System MMU */
+};
+
 struct exynos_iommu_domain {
 	struct list_head clients; /* list of sysmmu_drvdata.node */
 	unsigned long *pgtable; /* lv1 page table, 16KB */
@@ -157,9 +171,8 @@ struct exynos_iommu_domain {
 };
 
 struct sysmmu_drvdata {
-	struct list_head node; /* entry of exynos_iommu_domain.clients */
 	struct device *sysmmu;	/* System MMU's device descriptor */
-	struct device *dev;	/* Owner of system MMU */
+	struct device *master;	/* Owner of system MMU */
 	void __iomem *sfrbase;
 	struct clk *clk;
 	struct clk *clk_master;
@@ -228,7 +241,6 @@ static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase,
 static void __sysmmu_set_ptbase(void __iomem *sfrbase,
 				       unsigned long pgd)
 {
-	__raw_writel(0x1, sfrbase + REG_MMU_CFG); /* 16KB LV1, LRU */
 	__raw_writel(pgd, sfrbase + REG_PT_BASE_ADDR);
 
 	__sysmmu_tlb_invalidate(sfrbase);
@@ -289,7 +301,7 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 					itype, base, addr);
 		if (data->domain)
 			ret = report_iommu_fault(data->domain,
-					data->dev, addr, itype);
+					data->master, addr, itype);
 	}
 
 	/* fault is not recovered by fault handler */
@@ -307,120 +319,152 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	return IRQ_HANDLED;
 }
 
-static bool __exynos_sysmmu_disable(struct sysmmu_drvdata *data)
+static void __sysmmu_disable_nocount(struct sysmmu_drvdata *data)
 {
-	unsigned long flags;
-	bool disabled = false;
-
-	write_lock_irqsave(&data->lock, flags);
-
-	if (!set_sysmmu_inactive(data))
-		goto finish;
-
 	if (!IS_ERR(data->clk_master))
 		clk_enable(data->clk_master);
 
 	__raw_writel(CTRL_DISABLE, data->sfrbase + REG_MMU_CTRL);
+	__raw_writel(0, data->sfrbase + REG_MMU_CFG);
 
 	clk_disable(data->clk);
 	if (!IS_ERR(data->clk_master))
 		clk_disable(data->clk_master);
-
-	disabled = true;
-	data->pgtable = 0;
-	data->domain = NULL;
-finish:
-	write_unlock_irqrestore(&data->lock, flags);
-
-	if (disabled)
-		dev_dbg(data->sysmmu, "Disabled\n");
-	else
-		dev_dbg(data->sysmmu, "%d times left to be disabled\n",
-					data->activations);
-
-	return disabled;
 }
 
-/* __exynos_sysmmu_enable: Enables System MMU
- *
- * returns -error if an error occurred and System MMU is not enabled,
- * 0 if the System MMU has been just enabled and 1 if System MMU was already
- * enabled before.
- */
-static int __exynos_sysmmu_enable(struct sysmmu_drvdata *data,
-			unsigned long pgtable, struct iommu_domain *domain)
+static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 {
-	int ret = 0;
+	bool disabled;
 	unsigned long flags;
 
 	write_lock_irqsave(&data->lock, flags);
 
-	if (!set_sysmmu_active(data)) {
-		if (WARN_ON(pgtable != data->pgtable)) {
-			ret = -EBUSY;
-			set_sysmmu_inactive(data);
-		} else {
-			ret = 1;
-		}
+	disabled = set_sysmmu_inactive(data);
+
+	if (disabled) {
+		data->pgtable = 0;
+		data->domain = NULL;
+
+		__sysmmu_disable_nocount(data);
 
-		dev_dbg(data->sysmmu, "Already enabled\n");
-		goto finish;
+		dev_dbg(data->sysmmu, "Disabled\n");
+	} else  {
+		dev_dbg(data->sysmmu, "%d times left to disable\n",
+					data->activations);
 	}
 
-	data->pgtable = pgtable;
+	write_unlock_irqrestore(&data->lock, flags);
+
+	return disabled;
+}
+
+static void __sysmmu_init_config(struct sysmmu_drvdata *data)
+{
+	unsigned long cfg = 0;
+
+	__raw_writel(cfg, data->sfrbase + REG_MMU_CFG);
+}
 
+static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data)
+{
 	if (!IS_ERR(data->clk_master))
 		clk_enable(data->clk_master);
 	clk_enable(data->clk);
 
-	__sysmmu_set_ptbase(data->sfrbase, pgtable);
+	__raw_writel(CTRL_BLOCK, data->sfrbase + REG_MMU_CTRL);
+
+	__sysmmu_init_config(data);
+
+	__sysmmu_set_ptbase(data->sfrbase, data->pgtable);
 
 	__raw_writel(CTRL_ENABLE, data->sfrbase + REG_MMU_CTRL);
 
 	if (!IS_ERR(data->clk_master))
 		clk_disable(data->clk_master);
+}
+
+static int __sysmmu_enable(struct sysmmu_drvdata *data,
+			unsigned long pgtable, struct iommu_domain *domain)
+{
+	int ret = 0;
+	unsigned long flags;
 
-	data->domain = domain;
+	write_lock_irqsave(&data->lock, flags);
+	if (set_sysmmu_active(data)) {
+		data->pgtable = pgtable;
+		data->domain = domain;
+
+		__sysmmu_enable_nocount(data);
+
+		dev_dbg(data->sysmmu, "Enabled\n");
+	} else {
+		ret = (pgtable == data->pgtable) ? 1 : -EBUSY;
+
+		dev_dbg(data->sysmmu, "already enabled\n");
+	}
+
+	if (WARN_ON(ret < 0))
+		set_sysmmu_inactive(data); /* decrement count */
 
-	dev_dbg(data->sysmmu, "Enabled\n");
-finish:
 	write_unlock_irqrestore(&data->lock, flags);
 
 	return ret;
 }
 
-int exynos_sysmmu_enable(struct device *dev, unsigned long pgtable)
+/* __exynos_sysmmu_enable: Enables System MMU
+ *
+ * returns -error if an error occurred and System MMU is not enabled,
+ * 0 if the System MMU has been just enabled and 1 if System MMU was already
+ * enabled before.
+ */
+static int __exynos_sysmmu_enable(struct device *dev, unsigned long pgtable,
+				  struct iommu_domain *domain)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
-	int ret;
+	int ret = 0;
+	unsigned long flags;
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
+	struct sysmmu_drvdata *data;
 
-	BUG_ON(!memblock_is_memory(pgtable));
+	BUG_ON(!has_sysmmu(dev));
 
-	ret = pm_runtime_get_sync(data->sysmmu);
-	if (ret < 0) {
-		dev_dbg(data->sysmmu, "Failed to enable\n");
-		return ret;
-	}
+	spin_lock_irqsave(&owner->lock, flags);
 
-	ret = __exynos_sysmmu_enable(data, pgtable, NULL);
-	if (WARN_ON(ret < 0)) {
-		pm_runtime_put(data->sysmmu);
-		dev_err(data->sysmmu, "Already enabled with page table %#lx\n",
-			data->pgtable);
-	} else {
-		data->dev = dev;
-	}
+	data = dev_get_drvdata(owner->sysmmu);
+
+	ret = __sysmmu_enable(data, pgtable, domain);
+	if (ret >= 0)
+		data->master = dev;
+
+	spin_unlock_irqrestore(&owner->lock, flags);
 
 	return ret;
 }
 
+int exynos_sysmmu_enable(struct device *dev, unsigned long pgtable)
+{
+	BUG_ON(!memblock_is_memory(pgtable));
+
+	return __exynos_sysmmu_enable(dev, pgtable, NULL);
+}
+
 static bool exynos_sysmmu_disable(struct device *dev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
-	bool disabled;
+	unsigned long flags;
+	bool disabled = true;
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
+	struct sysmmu_drvdata *data;
+
+	BUG_ON(!has_sysmmu(dev));
 
-	disabled = __exynos_sysmmu_disable(data);
-	pm_runtime_put(data->sysmmu);
+	spin_lock_irqsave(&owner->lock, flags);
+
+	data = dev_get_drvdata(owner->sysmmu);
+
+	disabled = __sysmmu_disable(data);
+	if (disabled)
+		data->master = NULL;
+
+	spin_unlock_irqrestore(&owner->lock, flags);
 
 	return disabled;
 }
@@ -428,11 +472,13 @@ static bool exynos_sysmmu_disable(struct device *dev)
 static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 					size_t size)
 {
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
 	unsigned long flags;
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+	struct sysmmu_drvdata *data;
 
-	read_lock_irqsave(&data->lock, flags);
+	data = dev_get_drvdata(owner->sysmmu);
 
+	read_lock_irqsave(&data->lock, flags);
 	if (is_sysmmu_active(data)) {
 		unsigned int maj;
 		unsigned int num_inv = 1;
@@ -462,19 +508,21 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 		if (!IS_ERR(data->clk_master))
 			clk_disable(data->clk_master);
 	} else {
-		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
+		dev_dbg(dev, "disabled. Skipping TLB invalidation @ %#lx\n",
+			iova);
 	}
-
 	read_unlock_irqrestore(&data->lock, flags);
 }
 
 void exynos_sysmmu_tlb_invalidate(struct device *dev)
 {
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
 	unsigned long flags;
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+	struct sysmmu_drvdata *data;
 
-	read_lock_irqsave(&data->lock, flags);
+	data = dev_get_drvdata(owner->sysmmu);
 
+	read_lock_irqsave(&data->lock, flags);
 	if (is_sysmmu_active(data)) {
 		if (!IS_ERR(data->clk_master))
 			clk_enable(data->clk_master);
@@ -485,18 +533,20 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 		if (!IS_ERR(data->clk_master))
 			clk_disable(data->clk_master);
 	} else {
-		dev_dbg(data->sysmmu, "Disabled. Skipping invalidating TLB.\n");
+		dev_dbg(dev, "disabled. Skipping TLB invalidation\n");
 	}
-
 	read_unlock_irqrestore(&data->lock, flags);
 }
 
+static int __init exynos_iommu_prepare(void);
+
 static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 {
 	int irq, ret;
 	struct device *dev = &pdev->dev;
 	struct sysmmu_drvdata *data;
 	struct resource *res;
+	struct device_node *node;
 
 	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
 	if (!data)
@@ -542,14 +592,41 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 		}
 	}
 
+	/* Relation between master and System MMU is 1:1. */
+	node = of_parse_phandle(dev->of_node, "mmu-masters", 0);
+	if (node) {
+		struct platform_device *master = of_find_device_by_node(node);
+
+		if (!master) {
+			dev_err(dev, "%s: mmu-master '%s' not found\n",
+				__func__, node->name);
+			return -EINVAL;
+		}
+
+		if (master->dev.archdata.iommu != NULL) {
+			dev_err(dev, "%s: '%s' is master of other MMU\n",
+				__func__, node->name);
+			return -EINVAL;
+		}
+
+		/*
+		 * archdata.iommu will be initialized with exynos_iommu_client
+		 * in sysmmu_hook_driver_register().
+		 */
+		master->dev.archdata.iommu = dev;
+	}
+
 	data->sysmmu = dev;
 	rwlock_init(&data->lock);
-	INIT_LIST_HEAD(&data->node);
 
 	platform_set_drvdata(pdev, data);
 
 	pm_runtime_enable(dev);
 
+	ret = exynos_iommu_prepare();
+	if (ret)
+		return ret;
+
 	return 0;
 }
 
@@ -619,7 +696,7 @@ err_pgtable:
 static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
-	struct sysmmu_drvdata *data;
+	struct exynos_iommu_owner *owner;
 	unsigned long flags;
 	int i;
 
@@ -627,11 +704,14 @@ static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
 
 	spin_lock_irqsave(&priv->lock, flags);
 
-	list_for_each_entry(data, &priv->clients, node) {
-		while (!exynos_sysmmu_disable(data->dev))
+	list_for_each_entry(owner, &priv->clients, client) {
+		while (!exynos_sysmmu_disable(owner->dev))
 			; /* until System MMU is actually disabled */
 	}
 
+	while (!list_empty(&priv->clients))
+		list_del_init(priv->clients.next);
+
 	spin_unlock_irqrestore(&priv->lock, flags);
 
 	for (i = 0; i < NUM_LV1ENTRIES; i++)
@@ -648,42 +728,30 @@ static void exynos_iommu_domain_destroy(struct iommu_domain *domain)
 static int exynos_iommu_attach_device(struct iommu_domain *domain,
 				   struct device *dev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
 	struct exynos_iommu_domain *priv = domain->priv;
 	phys_addr_t pagetable = virt_to_phys(priv->pgtable);
 	unsigned long flags;
 	int ret;
 
-	ret = pm_runtime_get_sync(data->sysmmu);
-	if (ret < 0)
-		return ret;
-
-	ret = 0;
-
 	spin_lock_irqsave(&priv->lock, flags);
 
-	ret = __exynos_sysmmu_enable(data, __pa(priv->pgtable), domain);
 
+	ret = __exynos_sysmmu_enable(dev, __pa(priv->pgtable), domain);
 	if (ret == 0) {
-		/* 'data->node' must not be appeared in priv->clients */
-		BUG_ON(!list_empty(&data->node));
-		data->dev = dev;
-		list_add_tail(&data->node, &priv->clients);
+		list_add_tail(&owner->client, &priv->clients);
+		owner->domain = domain;
 	}
 
 	spin_unlock_irqrestore(&priv->lock, flags);
 
-	if (ret < 0) {
+	if (ret < 0)
 		dev_err(dev, "%s: Failed to attach IOMMU with pgtable %pa\n",
 					__func__, &pagetable);
-		pm_runtime_put(data->sysmmu);
-	} else if (ret > 0) {
-		dev_dbg(dev, "%s: IOMMU with pgtable %pa already attached\n",
-					__func__, &pagetable);
-	} else {
-		dev_dbg(dev, "%s: Attached new IOMMU with pgtable %pa\n",
-					__func__, &pagetable);
-	}
+	else
+		dev_dbg(dev, "%s: Attached IOMMU with pgtable %pa%s\n",
+					__func__, &pagetable,
+					(ret == 0) ? "" : ", again");
 
 	return ret;
 }
@@ -691,40 +759,30 @@ static int exynos_iommu_attach_device(struct iommu_domain *domain,
 static void exynos_iommu_detach_device(struct iommu_domain *domain,
 				    struct device *dev)
 {
-	struct sysmmu_drvdata *data = dev_get_drvdata(dev->archdata.iommu);
+	struct exynos_iommu_owner *owner;
 	struct exynos_iommu_domain *priv = domain->priv;
-	struct list_head *pos;
 	phys_addr_t pagetable = virt_to_phys(priv->pgtable);
 	unsigned long flags;
-	bool found = false;
 
 	spin_lock_irqsave(&priv->lock, flags);
 
-	list_for_each(pos, &priv->clients) {
-		if (list_entry(pos, struct sysmmu_drvdata, node) == data) {
-			found = true;
+	list_for_each_entry(owner, &priv->clients, client) {
+		if (owner == dev->archdata.iommu) {
+			if (exynos_sysmmu_disable(dev)) {
+				list_del_init(&owner->client);
+				owner->domain = NULL;
+			}
 			break;
 		}
 	}
 
-	if (!found)
-		goto finish;
+	spin_unlock_irqrestore(&priv->lock, flags);
 
-	if (__exynos_sysmmu_disable(data)) {
+	if (owner == dev->archdata.iommu)
 		dev_dbg(dev, "%s: Detached IOMMU with pgtable %pa\n",
 					__func__, &pagetable);
-		list_del_init(&data->node);
-
-	} else {
-		dev_dbg(dev, "%s: Detaching IOMMU with pgtable %pa delayed",
-					__func__, &pagetable);
-	}
-
-finish:
-	spin_unlock_irqrestore(&priv->lock, flags);
-
-	if (found)
-		pm_runtime_put(data->sysmmu);
+	else
+		dev_dbg(dev, "%s: No IOMMU is attached\n", __func__);
 }
 
 static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
@@ -854,7 +912,7 @@ static size_t exynos_iommu_unmap(struct iommu_domain *domain,
 					       unsigned long iova, size_t size)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
-	struct sysmmu_drvdata *data;
+	struct exynos_iommu_owner *owner;
 	unsigned long flags;
 	unsigned long *ent;
 	size_t err_pgsize;
@@ -915,8 +973,8 @@ done:
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
 	spin_lock_irqsave(&priv->lock, flags);
-	list_for_each_entry(data, &priv->clients, node)
-		sysmmu_tlb_invalidate_entry(data->dev, iova, size);
+	list_for_each_entry(owner, &priv->clients, client)
+		sysmmu_tlb_invalidate_entry(owner->dev, iova, size);
 	spin_unlock_irqrestore(&priv->lock, flags);
 
 	return size;
@@ -1001,3 +1059,70 @@ err_reg_driver:
 	return ret;
 }
 subsys_initcall(exynos_iommu_init);
+
+static int sysmmu_hook_driver_register(struct notifier_block *nb,
+					unsigned long val,
+					void *p)
+{
+	struct device *dev = p;
+
+	switch (val) {
+	case BUS_NOTIFY_BIND_DRIVER:
+	{
+		struct exynos_iommu_owner *owner;
+
+		/* No System MMU assigned. See exynos_sysmmu_probe(). */
+		if (dev->archdata.iommu == NULL)
+			break;
+
+		owner = devm_kzalloc(dev, sizeof(*owner), GFP_KERNEL);
+		if (!owner) {
+			dev_err(dev, "No Memory for exynos_iommu_owner\n");
+			return -ENOMEM;
+		}
+
+		owner->dev = dev;
+		INIT_LIST_HEAD(&owner->client);
+		owner->sysmmu = dev->archdata.iommu;
+
+		dev->archdata.iommu = owner;
+		break;
+	}
+	case BUS_NOTIFY_UNBOUND_DRIVER:
+	{
+		struct exynos_iommu_owner *owner = dev->archdata.iommu;
+		if (owner) {
+			struct device *sysmmu = owner->sysmmu;
+			/* if still attached to an iommu_domain. */
+			if (WARN_ON(!list_empty(&owner->client)))
+				iommu_detach_device(owner->domain, owner->dev);
+			devm_kfree(dev, owner);
+			dev->archdata.iommu = sysmmu;
+		}
+		break;
+	}
+	} /* switch (val) */
+
+	return 0;
+}
+
+static struct notifier_block sysmmu_notifier = {
+	.notifier_call = &sysmmu_hook_driver_register,
+};
+
+static int __init exynos_iommu_prepare(void)
+{
+	static bool registered = false;
+	int ret;
+
+	if (registered)
+		return 0;
+
+	ret = bus_register_notifier(&platform_bus_type, &sysmmu_notifier);
+	if (!ret)
+		registered = true;
+	else
+		pr_err("Failed to register sysmmu_notifier\n");
+
+	return ret;
+}
-- 
1.7.9.5

Re: [PATCH v12 15/31] iommu/exynos: handle 'mmu-masters' property of DT and improve handling sysmmu

[Arnd Bergmann]

On Sunday 27 April 2014 13:07:47 Shaik Ameer Basha wrote:
> @@ -542,14 +592,41 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
>  		}
>  	}
>  
> +	/* Relation between master and System MMU is 1:1. */
> +	node = of_parse_phandle(dev->of_node, "mmu-masters", 0);
> +	if (node) {
> +		struct platform_device *master = of_find_device_by_node(node);
> +
> +		if (!master) {
> +			dev_err(dev, "%s: mmu-master '%s' not found\n",
> +				__func__, node->name);
> +			return -EINVAL;
> +		}
> +
> +		if (master->dev.archdata.iommu != NULL) {
> +			dev_err(dev, "%s: '%s' is master of other MMU\n",
> +				__func__, node->name);
> +			return -EINVAL;
> +		}
> +
> +		/*
> +		 * archdata.iommu will be initialized with exynos_iommu_client
> +		 * in sysmmu_hook_driver_register().
> +		 */
> +		master->dev.archdata.iommu = dev;
> +	}

I think parsing of generic properties like this shouldn't be done
by the individual driver. It belongs into core iommu code and should
be done when the devices are probed.

> +
> +static struct notifier_block sysmmu_notifier = {
> +	.notifier_call = &sysmmu_hook_driver_register,
> +};
> +
> +static int __init exynos_iommu_prepare(void)
> +{
> +	static bool registered = false;
> +	int ret;
> +
> +	if (registered)
> +		return 0;
> +
> +	ret = bus_register_notifier(&platform_bus_type, &sysmmu_notifier);
> +	if (!ret)
> +		registered = true;
> +	else
> +		pr_err("Failed to register sysmmu_notifier\n");
> +
> +	return ret;
> +}

This also means we should not have notifiers. Put it into core code to
make sure all IOMMUs are handled the same way.

	Arnd

Re: [PATCH v12 15/31] iommu/exynos: handle 'mmu-masters' property of DT and improve handling sysmmu

[Cho KyongHo]

On Sun, 27 Apr 2014 20:17:48 +0200, Arnd Bergmann wrote:
> On Sunday 27 April 2014 13:07:47 Shaik Ameer Basha wrote:
> > @@ -542,14 +592,41 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
> >  		}
> >  	}
> >  
> > +	/* Relation between master and System MMU is 1:1. */
> > +	node = of_parse_phandle(dev->of_node, "mmu-masters", 0);
> > +	if (node) {
> > +		struct platform_device *master = of_find_device_by_node(node);
> > +
> > +		if (!master) {
> > +			dev_err(dev, "%s: mmu-master '%s' not found\n",
> > +				__func__, node->name);
> > +			return -EINVAL;
> > +		}
> > +
> > +		if (master->dev.archdata.iommu != NULL) {
> > +			dev_err(dev, "%s: '%s' is master of other MMU\n",
> > +				__func__, node->name);
> > +			return -EINVAL;
> > +		}
> > +
> > +		/*
> > +		 * archdata.iommu will be initialized with exynos_iommu_client
> > +		 * in sysmmu_hook_driver_register().
> > +		 */
> > +		master->dev.archdata.iommu = dev;
> > +	}
> 
> I think parsing of generic properties like this shouldn't be done
> by the individual driver. It belongs into core iommu code and should
> be done when the devices are probed.
> 
> > +
> > +static struct notifier_block sysmmu_notifier = {
> > +	.notifier_call = &sysmmu_hook_driver_register,
> > +};
> > +
> > +static int __init exynos_iommu_prepare(void)
> > +{
> > +	static bool registered = false;
> > +	int ret;
> > +
> > +	if (registered)
> > +		return 0;
> > +
> > +	ret = bus_register_notifier(&platform_bus_type, &sysmmu_notifier);
> > +	if (!ret)
> > +		registered = true;
> > +	else
> > +		pr_err("Failed to register sysmmu_notifier\n");
> > +
> > +	return ret;
> > +}
> 
> This also means we should not have notifiers. Put it into core code to
> make sure all IOMMUs are handled the same way.
> 

Ok.

Thank you for addressing that.
I am also very interested in the discussion about it in the different thread.

KyongHo

[PATCH v12 16/31] iommu/exynos: turn on useful configuration options

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This turns on FLPD_CACHE, ACGEN and SYSSEL.

FLPD_CACHE is a cache of 1st level page table entries that contains
the address of a 2nd level page table to reduce latency of page table
walking.

ACGEN is architectural clock gating that gates clocks by System MMU
itself if it is not active. Note that ACGEN is different from clock
gating by the CPU. ACGEN just gates clocks to the internal logic of
System MMU while clock gating by the CPU gates clocks to the System
MMU.

SYSSEL selects System MMU version in some Exynos SoCs. Some Exynos
SoCs have an option to select System MMU versions exclusively because
the SoCs adopts new System MMU version experimentally.

This also always selects LRU as TLB replacement policy. Selecting TLB
replacement policy is deprecated from System MMU 3.2. TLB in System
MMU 3.3 has single TLB replacement policy, LRU. The bit of MMU_CFG
selecting TLB replacement policy is remained as reserved.

QoS value of page table walking is set to 15 (highst value). System
MMU 3.3 can inherit QoS value of page table walking from its master
H/W's transaction. This new feature is enabled by default and QoS
value written to MMU_CFG is ignored.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   52 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 51 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 45c792c..810bcaf 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -81,6 +81,13 @@
 #define CTRL_BLOCK	0x7
 #define CTRL_DISABLE	0x0
 
+#define CFG_LRU		0x1
+#define CFG_QOS(n)	((n & 0xF) << 7)
+#define CFG_MASK	0x0150FFFF /* Selecting bit 0-15, 20, 22 and 24 */
+#define CFG_ACGEN	(1 << 24) /* System MMU 3.3 only */
+#define CFG_SYSSEL	(1 << 22) /* System MMU 3.2 only */
+#define CFG_FLPDCACHE	(1 << 20) /* System MMU 3.2+ only */
+
 #define REG_MMU_CTRL		0x000
 #define REG_MMU_CFG		0x004
 #define REG_MMU_STATUS		0x008
@@ -97,6 +104,12 @@
 
 #define REG_MMU_VERSION		0x034
 
+#define MMU_MAJ_VER(val)	((val) >> 7)
+#define MMU_MIN_VER(val)	((val) & 0x7F)
+#define MMU_RAW_VER(reg)	(((reg) >> 21) & ((1 << 11) - 1)) /* 11 bits */
+
+#define MAKE_MMU_VER(maj, min)	((((maj) & 0xF) << 7) | ((min) & 0x7F))
+
 #define REG_PB0_SADDR		0x04C
 #define REG_PB0_EADDR		0x050
 #define REG_PB1_SADDR		0x054
@@ -206,6 +219,29 @@ static void sysmmu_unblock(void __iomem *sfrbase)
 	__raw_writel(CTRL_ENABLE, sfrbase + REG_MMU_CTRL);
 }
 
+static unsigned int __raw_sysmmu_version(struct sysmmu_drvdata *data)
+{
+	return MMU_RAW_VER(__raw_readl(data->sfrbase + REG_MMU_VERSION));
+}
+
+static unsigned int __sysmmu_version(struct sysmmu_drvdata *data,
+				     unsigned int *minor)
+{
+	unsigned int ver = 0;
+
+	ver = __raw_sysmmu_version(data);
+	if (ver > MAKE_MMU_VER(3, 3)) {
+		dev_err(data->sysmmu, "%s: version(%d.%d) is higher than 3.3\n",
+			__func__, MMU_MAJ_VER(ver), MMU_MIN_VER(ver));
+		BUG();
+	}
+
+	if (minor)
+		*minor = MMU_MIN_VER(ver);
+
+	return MMU_MAJ_VER(ver);
+}
+
 static bool sysmmu_block(void __iomem *sfrbase)
 {
 	int i = 120;
@@ -360,7 +396,21 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 
 static void __sysmmu_init_config(struct sysmmu_drvdata *data)
 {
-	unsigned long cfg = 0;
+	unsigned long cfg = CFG_LRU | CFG_QOS(15);
+	int maj, min = 0;
+
+	maj = __sysmmu_version(data, &min);
+	if (maj == 3) {
+		if (min >= 2) {
+			cfg |= CFG_FLPDCACHE;
+			if (min == 3) {
+				cfg |= CFG_ACGEN;
+				cfg &= ~CFG_LRU;
+			} else {
+				cfg |= CFG_SYSSEL;
+			}
+		}
+	}
 
 	__raw_writel(cfg, data->sfrbase + REG_MMU_CFG);
 }
-- 
1.7.9.5

[PATCH v12 17/31] iommu/exynos: add support for power management subsystems.

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This adds support for Suspend to RAM and Runtime Power Management.

Since System MMU is located in the same local power domain of its
master H/W, System MMU must be initialized before it is working if
its power domain was ever turned off. TLB invalidation according to
unmapping on page tables must also be performed while power domain is
turned on.

This patch ensures that resume and runtime_resume(restore_state)
functions in this driver is called before the calls to resume and
runtime_resume callback functions in the drivers of master H/Ws.
Likewise, suspend and runtime_suspend(save_state) functions in this
driver is called after the calls to suspend and runtime_suspend in the
drivers of master H/Ws.

In order to get benefit of this support, the master H/W and its System
MMU must resides in the same power domain in terms of Linux kernel. If
a master H/W does not use generic I/O power domain, its driver must
call iommu_attach_device() after its local power domain is turned on,
iommu_detach_device before turned off.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |  216 ++++++++++++++++++++++++++++++++++++++----
 1 file changed, 198 insertions(+), 18 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 810bcaf..fefedec3 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -27,6 +27,7 @@
 #include <linux/export.h>
 #include <linux/of.h>
 #include <linux/of_platform.h>
+#include <linux/pm_domain.h>
 #include <linux/notifier.h>
 
 #include <asm/cacheflush.h>
@@ -192,6 +193,7 @@ struct sysmmu_drvdata {
 	int activations;
 	rwlock_t lock;
 	struct iommu_domain *domain;
+	bool powered_on;
 	unsigned long pgtable;
 };
 
@@ -381,7 +383,8 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 		data->pgtable = 0;
 		data->domain = NULL;
 
-		__sysmmu_disable_nocount(data);
+		if (data->powered_on)
+			__sysmmu_disable_nocount(data);
 
 		dev_dbg(data->sysmmu, "Disabled\n");
 	} else  {
@@ -444,7 +447,8 @@ static int __sysmmu_enable(struct sysmmu_drvdata *data,
 		data->pgtable = pgtable;
 		data->domain = domain;
 
-		__sysmmu_enable_nocount(data);
+		if (data->powered_on)
+			__sysmmu_enable_nocount(data);
 
 		dev_dbg(data->sysmmu, "Enabled\n");
 	} else {
@@ -529,14 +533,12 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 	data = dev_get_drvdata(owner->sysmmu);
 
 	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data)) {
-		unsigned int maj;
+	if (is_sysmmu_active(data) && data->powered_on) {
 		unsigned int num_inv = 1;
 
 		if (!IS_ERR(data->clk_master))
 			clk_enable(data->clk_master);
 
-		maj = __raw_readl(data->sfrbase + REG_MMU_VERSION);
 		/*
 		 * L2TLB invalidation required
 		 * 4KB page: 1 invalidation
@@ -547,7 +549,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 		 * 1MB page can be cached in one of all sets.
 		 * 64KB page can be one of 16 consecutive sets.
 		 */
-		if ((maj >> 28) == 2) /* major version number */
+		if (__sysmmu_version(data, NULL) == 2) /* major version number */
 			num_inv = min_t(unsigned int, size / PAGE_SIZE, 64);
 
 		if (sysmmu_block(data->sfrbase)) {
@@ -573,7 +575,7 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 	data = dev_get_drvdata(owner->sysmmu);
 
 	read_lock_irqsave(&data->lock, flags);
-	if (is_sysmmu_active(data)) {
+	if (is_sysmmu_active(data) && data->powered_on) {
 		if (!IS_ERR(data->clk_master))
 			clk_enable(data->clk_master);
 		if (sysmmu_block(data->sfrbase)) {
@@ -672,6 +674,7 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 	platform_set_drvdata(pdev, data);
 
 	pm_runtime_enable(dev);
+	data->powered_on = !pm_runtime_enabled(dev);
 
 	ret = exynos_iommu_prepare();
 	if (ret)
@@ -680,6 +683,34 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 	return 0;
 }
 
+#ifdef CONFIG_PM_SLEEP
+static int sysmmu_suspend(struct device *dev)
+{
+	struct sysmmu_drvdata *data = dev_get_drvdata(dev);
+	unsigned long flags;
+	read_lock_irqsave(&data->lock, flags);
+	if (is_sysmmu_active(data) &&
+		(!pm_runtime_enabled(dev) || data->powered_on))
+		__sysmmu_disable_nocount(data);
+	read_unlock_irqrestore(&data->lock, flags);
+	return 0;
+}
+
+static int sysmmu_resume(struct device *dev)
+{
+	struct sysmmu_drvdata *data = dev_get_drvdata(dev);
+	unsigned long flags;
+	read_lock_irqsave(&data->lock, flags);
+	if (is_sysmmu_active(data) &&
+		(!pm_runtime_enabled(dev) || data->powered_on))
+		__sysmmu_enable_nocount(data);
+	read_unlock_irqrestore(&data->lock, flags);
+	return 0;
+}
+#endif
+
+static SIMPLE_DEV_PM_OPS(sysmmu_pm_ops, sysmmu_suspend, sysmmu_resume);
+
 static const struct of_device_id sysmmu_of_match[] __initconst = {
 	{ .compatible	= "samsung,sysmmu-v1", },
 	{ .compatible	= "samsung,sysmmu-v2", },
@@ -694,6 +725,7 @@ static struct platform_driver exynos_sysmmu_driver __refdata = {
 	.driver	= {
 		.owner		= THIS_MODULE,
 		.name		= "exynos-sysmmu",
+		.pm		= &sysmmu_pm_ops,
 		.of_match_table	= sysmmu_of_match,
 	}
 };
@@ -1110,24 +1142,150 @@ err_reg_driver:
 }
 subsys_initcall(exynos_iommu_init);
 
+#ifdef CONFIG_PM_SLEEP
+static int sysmmu_pm_genpd_suspend(struct device *dev)
+{
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
+	int ret;
+
+	ret = pm_generic_suspend(dev);
+	if (ret)
+		return ret;
+
+	return pm_generic_suspend(owner->sysmmu);
+}
+
+static int sysmmu_pm_genpd_resume(struct device *dev)
+{
+	struct exynos_iommu_owner *owner = dev->archdata.iommu;
+	int ret;
+
+	ret = pm_generic_resume(owner->sysmmu);
+	if (ret)
+		return ret;
+
+	return pm_generic_resume(dev);
+}
+#endif
+
+#ifdef CONFIG_PM_RUNTIME
+static void sysmmu_restore_state(struct device *sysmmu)
+{
+	struct sysmmu_drvdata *data = dev_get_drvdata(sysmmu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&data->lock, flags);
+	data->powered_on = true;
+	if (is_sysmmu_active(data))
+		__sysmmu_enable_nocount(data);
+	spin_unlock_irqrestore(&data->lock, flags);
+}
+
+static void sysmmu_save_state(struct device *sysmmu)
+{
+	struct sysmmu_drvdata *data = dev_get_drvdata(sysmmu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&data->lock, flags);
+	if (is_sysmmu_active(data))
+		__sysmmu_disable_nocount(data);
+	data->powered_on = false;
+	spin_unlock_irqrestore(&data->lock, flags);
+}
+
+static int sysmmu_pm_genpd_save_state(struct device *dev)
+{
+	struct exynos_iommu_client *client = dev->archdata.iommu;
+	int (*cb)(struct device *__dev);
+	int ret = 0;
+
+	if (dev->type && dev->type->pm)
+		cb = dev->type->pm->runtime_suspend;
+	else if (dev->class && dev->class->pm)
+		cb = dev->class->pm->runtime_suspend;
+	else if (dev->bus && dev->bus->pm)
+		cb = dev->bus->pm->runtime_suspend;
+	else
+		cb = NULL;
+
+	if (!cb && dev->driver && dev->driver->pm)
+		cb = dev->driver->pm->runtime_suspend;
+
+	if (cb)
+		ret = cb(dev);
+
+	if (ret == 0)
+		sysmmu_save_state(client->sysmmu);
+
+	return ret;
+}
+
+static int sysmmu_pm_genpd_restore_state(struct device *dev)
+{
+	struct exynos_iommu_client *client = dev->archdata.iommu;
+	int (*cb)(struct device *__dev);
+	int ret = 0;
+
+	if (dev->type && dev->type->pm)
+		cb = dev->type->pm->runtime_resume;
+	else if (dev->class && dev->class->pm)
+		cb = dev->class->pm->runtime_resume;
+	else if (dev->bus && dev->bus->pm)
+		cb = dev->bus->pm->runtime_resume;
+	else
+		cb = NULL;
+
+	if (!cb && dev->driver && dev->driver->pm)
+		cb = dev->driver->pm->runtime_resume;
+
+	sysmmu_restore_state(client->sysmmu);
+
+	if (cb)
+		ret = cb(dev);
+
+	if (ret)
+		sysmmu_save_state(client->sysmmu);
+
+	return ret;
+}
+#endif
+
+struct gpd_dev_ops sysmmu_devpm_ops = {
+#ifdef CONFIG_PM_RUNTIME
+	.save_state = &sysmmu_pm_genpd_save_state,
+	.restore_state = &sysmmu_pm_genpd_restore_state,
+#endif
+#ifdef CONFIG_PM_SLEEP
+	.suspend = &sysmmu_pm_genpd_suspend,
+	.resume = &sysmmu_pm_genpd_resume,
+#endif
+};
+
 static int sysmmu_hook_driver_register(struct notifier_block *nb,
 					unsigned long val,
 					void *p)
 {
 	struct device *dev = p;
 
+	/*
+	 * No System MMU assigned even though in the initial state.
+	 * See exynos_sysmmu_probe().
+	 */
+	if (dev->archdata.iommu == NULL)
+		return 0;
+
 	switch (val) {
 	case BUS_NOTIFY_BIND_DRIVER:
 	{
 		struct exynos_iommu_owner *owner;
+		int ret;
 
-		/* No System MMU assigned. See exynos_sysmmu_probe(). */
-		if (dev->archdata.iommu == NULL)
-			break;
+		BUG_ON(!dev_get_drvdata(dev->archdata.iommu));
 
 		owner = devm_kzalloc(dev, sizeof(*owner), GFP_KERNEL);
 		if (!owner) {
 			dev_err(dev, "No Memory for exynos_iommu_owner\n");
+			dev->archdata.iommu = NULL;
 			return -ENOMEM;
 		}
 
@@ -1135,22 +1293,44 @@ static int sysmmu_hook_driver_register(struct notifier_block *nb,
 		INIT_LIST_HEAD(&owner->client);
 		owner->sysmmu = dev->archdata.iommu;
 
+		ret = pm_genpd_add_callbacks(dev, &sysmmu_devpm_ops, NULL);
+		if (ret && (ret != -ENOSYS)) {
+			dev_err(dev,
+				"Failed to register 'dev_pm_ops' for iommu\n");
+			devm_kfree(dev, owner);
+			dev->archdata.iommu = NULL;
+			return ret;
+		}
+
 		dev->archdata.iommu = owner;
 		break;
 	}
-	case BUS_NOTIFY_UNBOUND_DRIVER:
+	case BUS_NOTIFY_BOUND_DRIVER:
 	{
 		struct exynos_iommu_owner *owner = dev->archdata.iommu;
-		if (owner) {
-			struct device *sysmmu = owner->sysmmu;
-			/* if still attached to an iommu_domain. */
-			if (WARN_ON(!list_empty(&owner->client)))
-				iommu_detach_device(owner->domain, owner->dev);
-			devm_kfree(dev, owner);
-			dev->archdata.iommu = sysmmu;
+		if (!pm_runtime_enabled(dev)) {
+			struct sysmmu_drvdata *data =
+					dev_get_drvdata(owner->sysmmu);
+			if (pm_runtime_enabled(data->sysmmu)) {
+				data->powered_on = true;
+				if (is_sysmmu_active(data))
+					__sysmmu_enable_nocount(data);
+				pm_runtime_disable(data->sysmmu);
+			}
 		}
 		break;
 	}
+	case BUS_NOTIFY_UNBOUND_DRIVER:
+	{
+		struct exynos_iommu_owner *owner = dev->archdata.iommu;
+		struct device *sysmmu = owner->sysmmu;
+		if (WARN_ON(!list_empty(&owner->client)))
+			iommu_detach_device(owner->domain, dev);
+		__pm_genpd_remove_callbacks(dev, false);
+		devm_kfree(dev, owner);
+		dev->archdata.iommu = sysmmu;
+		break;
+	}
 	} /* switch (val) */
 
 	return 0;
-- 
1.7.9.5

[PATCH v12 19/31] iommu/exynos: change rwlock to spinlock

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

Since acquiring read_lock is not more frequent than write_lock, it is
not beneficial to use rwlock, this commit changes rwlock to spinlock.

Reviewed-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index c2e6365..c9076e1 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -200,7 +200,7 @@ struct sysmmu_drvdata {
 	struct clk *clk;
 	struct clk *clk_master;
 	int activations;
-	rwlock_t lock;
+	spinlock_t lock;
 	struct iommu_domain *domain;
 	bool powered_on;
 	bool suspended;
@@ -323,12 +323,13 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	unsigned long addr = -1;
 	int ret = -ENOSYS;
 
-	read_lock(&data->lock);
-
 	WARN_ON(!is_sysmmu_active(data));
 
+	spin_lock(&data->lock);
+
 	if (!IS_ERR(data->clk_master))
 		clk_enable(data->clk_master);
+
 	itype = (enum exynos_sysmmu_inttype)
 		__ffs(__raw_readl(data->sfrbase + REG_INT_STATUS));
 	if (WARN_ON(!((itype >= 0) && (itype < SYSMMU_FAULT_UNKNOWN))))
@@ -362,7 +363,7 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	if (!IS_ERR(data->clk_master))
 		clk_disable(data->clk_master);
 
-	read_unlock(&data->lock);
+	spin_unlock(&data->lock);
 
 	return IRQ_HANDLED;
 }
@@ -385,7 +386,7 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 	bool disabled;
 	unsigned long flags;
 
-	write_lock_irqsave(&data->lock, flags);
+	spin_lock_irqsave(&data->lock, flags);
 
 	disabled = set_sysmmu_inactive(data);
 
@@ -402,7 +403,7 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 					data->activations);
 	}
 
-	write_unlock_irqrestore(&data->lock, flags);
+	spin_unlock_irqrestore(&data->lock, flags);
 
 	return disabled;
 }
@@ -452,7 +453,7 @@ static int __sysmmu_enable(struct sysmmu_drvdata *data,
 	int ret = 0;
 	unsigned long flags;
 
-	write_lock_irqsave(&data->lock, flags);
+	spin_lock_irqsave(&data->lock, flags);
 	if (set_sysmmu_active(data)) {
 		data->pgtable = pgtable;
 		data->domain = domain;
@@ -470,7 +471,7 @@ static int __sysmmu_enable(struct sysmmu_drvdata *data,
 	if (WARN_ON(ret < 0))
 		set_sysmmu_inactive(data); /* decrement count */
 
-	write_unlock_irqrestore(&data->lock, flags);
+	spin_unlock_irqrestore(&data->lock, flags);
 
 	return ret;
 }
@@ -557,7 +558,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 
 	for_each_sysmmu_list(dev, list) {
 		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
-		read_lock(&data->lock);
+		spin_lock(&data->lock);
 		if (is_sysmmu_active(data) && data->powered_on) {
 			unsigned int num_inv = 1;
 
@@ -592,7 +593,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 				iova);
 		}
 
-		read_unlock(&data->lock);
+		spin_unlock(&data->lock);
 	}
 
 	spin_unlock_irqrestore(&owner->lock, flags);
@@ -608,7 +609,7 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 
 	for_each_sysmmu_list(dev, list) {
 		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
-		read_lock(&data->lock);
+		spin_lock(&data->lock);
 		if (is_sysmmu_active(data) && data->powered_on) {
 			if (!IS_ERR(data->clk_master))
 				clk_enable(data->clk_master);
@@ -621,7 +622,7 @@ void exynos_sysmmu_tlb_invalidate(struct device *dev)
 		} else {
 			dev_dbg(dev, "disabled. Skipping TLB invalidation\n");
 		}
-		read_unlock(&data->lock);
+		spin_unlock(&data->lock);
 	}
 
 	spin_unlock_irqrestore(&owner->lock, flags);
@@ -813,7 +814,7 @@ static int __init exynos_sysmmu_probe(struct platform_device *pdev)
 	if (!ret) {
 		data->powered_on = !pm_runtime_enabled(dev);
 		data->sysmmu = dev;
-		rwlock_init(&data->lock);
+		spin_lock_init(&data->lock);
 
 		platform_set_drvdata(pdev, data);
 	}
@@ -1264,12 +1265,12 @@ static int sysmmu_pm_genpd_suspend(struct device *dev)
 	for_each_sysmmu_list(dev, list) {
 		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
 		unsigned long flags;
-		write_lock_irqsave(&data->lock, flags);
+		spin_lock_irqsave(&data->lock, flags);
 		if (!data->suspended && is_sysmmu_active(data) &&
 			(!pm_runtime_enabled(dev) || data->powered_on))
 			__sysmmu_disable_nocount(data);
 		data->suspended = true;
-		write_unlock_irqrestore(&data->lock, flags);
+		spin_unlock_irqrestore(&data->lock, flags);
 	}
 
 	return 0;
@@ -1282,12 +1283,12 @@ static int sysmmu_pm_genpd_resume(struct device *dev)
 	for_each_sysmmu_list(dev, list) {
 		struct sysmmu_drvdata *data = dev_get_drvdata(list->sysmmu);
 		unsigned long flags;
-		write_lock_irqsave(&data->lock, flags);
+		spin_lock_irqsave(&data->lock, flags);
 		if (data->suspended && is_sysmmu_active(data) &&
 			(!pm_runtime_enabled(dev) || data->powered_on))
 			__sysmmu_enable_nocount(data);
 		data->suspended = false;
-		write_unlock_irqrestore(&data->lock, flags);
+		spin_unlock_irqrestore(&data->lock, flags);
 	}
 
 	return pm_generic_resume(dev);
@@ -1439,12 +1440,12 @@ static int sysmmu_hook_driver_register(struct notifier_block *nb,
 					dev_get_drvdata(list->sysmmu);
 			unsigned long flags;
 
-			write_lock_irqsave(&data->lock, flags);
+			spin_lock_irqsave(&data->lock, flags);
 			if (is_sysmmu_active(data) && !data->powered_on)
 				__sysmmu_enable_nocount(data);
 			data->powered_on = true;
 			pm_runtime_disable(data->sysmmu);
-			write_unlock_irqrestore(&data->lock, flags);
+			spin_unlock_irqrestore(&data->lock, flags);
 		}
 		break;
 	}
-- 
1.7.9.5

[PATCH v12 22/31] iommu/exynos: use exynos-iommu specific typedef

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This commit introduces sysmmu_pte_t for page table entries and
sysmmu_iova_t vor I/O virtual address that is manipulated by
exynos-iommu driver. The purpose of the typedef is to remove
dependencies to the driver code from the change of CPU architecture
from 32 bit to 64 bit.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |  117 ++++++++++++++++++++++++------------------
 1 file changed, 67 insertions(+), 50 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 08a7ce0..00915f2 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -33,6 +33,9 @@
 #include <asm/cacheflush.h>
 #include <asm/pgtable.h>
 
+typedef u32 sysmmu_iova_t;
+typedef u32 sysmmu_pte_t;
+
 /* We does not consider super section mapping (16MB) */
 #define SECT_ORDER 20
 #define LPAGE_ORDER 16
@@ -54,20 +57,32 @@
 #define lv2ent_small(pent) ((*(pent) & 2) == 2)
 #define lv2ent_large(pent) ((*(pent) & 3) == 1)
 
+static u32 sysmmu_page_offset(sysmmu_iova_t iova, u32 size)
+{
+	return iova & (size - 1);
+}
+
 #define section_phys(sent) (*(sent) & SECT_MASK)
-#define section_offs(iova) ((iova) & 0xFFFFF)
+#define section_offs(iova) sysmmu_page_offset((iova), SECT_SIZE)
 #define lpage_phys(pent) (*(pent) & LPAGE_MASK)
-#define lpage_offs(iova) ((iova) & 0xFFFF)
+#define lpage_offs(iova) sysmmu_page_offset((iova), LPAGE_SIZE)
 #define spage_phys(pent) (*(pent) & SPAGE_MASK)
-#define spage_offs(iova) ((iova) & 0xFFF)
-
-#define lv1ent_offset(iova) ((iova) >> SECT_ORDER)
-#define lv2ent_offset(iova) (((iova) & 0xFF000) >> SPAGE_ORDER)
+#define spage_offs(iova) sysmmu_page_offset((iova), SPAGE_SIZE)
 
 #define NUM_LV1ENTRIES 4096
-#define NUM_LV2ENTRIES 256
+#define NUM_LV2ENTRIES (SECT_SIZE / SPAGE_SIZE)
 
-#define LV2TABLE_SIZE (NUM_LV2ENTRIES * sizeof(long))
+static u32 lv1ent_offset(sysmmu_iova_t iova)
+{
+	return iova >> SECT_ORDER;
+}
+
+static u32 lv2ent_offset(sysmmu_iova_t iova)
+{
+	return (iova >> SPAGE_ORDER) & (NUM_LV2ENTRIES - 1);
+}
+
+#define LV2TABLE_SIZE (NUM_LV2ENTRIES * sizeof(sysmmu_pte_t))
 
 #define SPAGES_PER_LPAGE (LPAGE_SIZE / SPAGE_SIZE)
 
@@ -124,14 +139,14 @@
 
 static struct kmem_cache *lv2table_kmem_cache;
 
-static unsigned long *section_entry(unsigned long *pgtable, unsigned long iova)
+static sysmmu_pte_t *section_entry(sysmmu_pte_t *pgtable, sysmmu_iova_t iova)
 {
 	return pgtable + lv1ent_offset(iova);
 }
 
-static unsigned long *page_entry(unsigned long *sent, unsigned long iova)
+static sysmmu_pte_t *page_entry(sysmmu_pte_t *sent, sysmmu_iova_t iova)
 {
-	return (unsigned long *)phys_to_virt(
+	return (sysmmu_pte_t *)phys_to_virt(
 				lv2table_base(sent)) + lv2ent_offset(iova);
 }
 
@@ -183,7 +198,7 @@ struct exynos_iommu_owner {
 
 struct exynos_iommu_domain {
 	struct list_head clients; /* list of sysmmu_drvdata.node */
-	unsigned long *pgtable; /* lv1 page table, 16KB */
+	sysmmu_pte_t *pgtable; /* lv1 page table, 16KB */
 	short *lv2entcnt; /* free lv2 entry counter for each section */
 	spinlock_t lock; /* lock for this structure */
 	spinlock_t pgtablelock; /* lock for modifying page table @ pgtable */
@@ -277,7 +292,7 @@ static void __sysmmu_tlb_invalidate(void __iomem *sfrbase)
 }
 
 static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase,
-				unsigned long iova, unsigned int num_inv)
+				sysmmu_iova_t iova, unsigned int num_inv)
 {
 	unsigned int i;
 	for (i = 0; i < num_inv; i++) {
@@ -288,7 +303,7 @@ static void __sysmmu_tlb_invalidate_entry(void __iomem *sfrbase,
 }
 
 static void __sysmmu_set_ptbase(void __iomem *sfrbase,
-				       unsigned long pgd)
+				       phys_addr_t pgd)
 {
 	__raw_writel(pgd, sfrbase + REG_PT_BASE_ADDR);
 
@@ -297,22 +312,22 @@ static void __sysmmu_set_ptbase(void __iomem *sfrbase,
 
 static void show_fault_information(const char *name,
 		enum exynos_sysmmu_inttype itype,
-		phys_addr_t pgtable_base, unsigned long fault_addr)
+		phys_addr_t pgtable_base, sysmmu_iova_t fault_addr)
 {
-	unsigned long *ent;
+	sysmmu_pte_t *ent;
 
 	if ((itype >= SYSMMU_FAULTS_NUM) || (itype < SYSMMU_PAGEFAULT))
 		itype = SYSMMU_FAULT_UNKNOWN;
 
-	pr_err("%s occurred at %#lx by %s(Page table base: %pa)\n",
+	pr_err("%s occurred at %#x by %s(Page table base: %pa)\n",
 		sysmmu_fault_name[itype], fault_addr, name, &pgtable_base);
 
 	ent = section_entry(phys_to_virt(pgtable_base), fault_addr);
-	pr_err("\tLv1 entry: 0x%lx\n", *ent);
+	pr_err("\tLv1 entry: %#x\n", *ent);
 
 	if (lv1ent_page(ent)) {
 		ent = page_entry(ent, fault_addr);
-		pr_err("\t Lv2 entry: 0x%lx\n", *ent);
+		pr_err("\t Lv2 entry: %#x\n", *ent);
 	}
 }
 
@@ -321,7 +336,7 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 	/* SYSMMU is in blocked when interrupt occurred. */
 	struct sysmmu_drvdata *data = dev_id;
 	enum exynos_sysmmu_inttype itype;
-	unsigned long addr = -1;
+	sysmmu_iova_t addr = -1;
 	int ret = -ENOSYS;
 
 	WARN_ON(!is_sysmmu_active(data));
@@ -345,7 +360,7 @@ static irqreturn_t exynos_sysmmu_irq(int irq, void *dev_id)
 			__func__);
 		BUG();
 	} else {
-		unsigned long base =
+		unsigned int base =
 				__raw_readl(data->sfrbase + REG_PT_BASE_ADDR);
 		show_fault_information(dev_name(data->sysmmu),
 					itype, base, addr);
@@ -411,7 +426,7 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 
 static void __sysmmu_init_config(struct sysmmu_drvdata *data)
 {
-	unsigned long cfg = CFG_LRU | CFG_QOS(15);
+	unsigned int cfg = CFG_LRU | CFG_QOS(15);
 	int maj, min = 0;
 
 	maj = __sysmmu_version(data, &min);
@@ -449,7 +464,7 @@ static void __sysmmu_enable_nocount(struct sysmmu_drvdata *data)
 }
 
 static int __sysmmu_enable(struct sysmmu_drvdata *data,
-			unsigned long pgtable, struct iommu_domain *domain)
+			phys_addr_t pgtable, struct iommu_domain *domain)
 {
 	int ret = 0;
 	unsigned long flags;
@@ -478,12 +493,12 @@ static int __sysmmu_enable(struct sysmmu_drvdata *data,
 }
 
 /* __exynos_sysmmu_enable: Enables System MMU
-*
-* returns -error if an error occurred and System MMU is not enabled,
-* 0 if the System MMU has been just enabled and 1 if System MMU was already
-* enabled before.
-*/
-static int __exynos_sysmmu_enable(struct device *dev, unsigned long pgtable,
+ *
+ * returns -error if an error occurred and System MMU is not enabled,
+ * 0 if the System MMU has been just enabled and 1 if System MMU was already
+ * enabled before.
+ */
+static int __exynos_sysmmu_enable(struct device *dev, phys_addr_t pgtable,
 				  struct iommu_domain *domain)
 {
 	int ret = 0;
@@ -517,7 +532,7 @@ static int __exynos_sysmmu_enable(struct device *dev, unsigned long pgtable,
 	return ret;
 }
 
-int exynos_sysmmu_enable(struct device *dev, unsigned long pgtable)
+int exynos_sysmmu_enable(struct device *dev, phys_addr_t pgtable)
 {
 	BUG_ON(!memblock_is_memory(pgtable));
 
@@ -548,7 +563,7 @@ static bool exynos_sysmmu_disable(struct device *dev)
 	return disabled;
 }
 
-static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
+static void sysmmu_tlb_invalidate_entry(struct device *dev, sysmmu_iova_t iova,
 					size_t size)
 {
 	struct exynos_iommu_owner *owner = dev->archdata.iommu;
@@ -590,7 +605,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, unsigned long iova,
 				clk_disable(data->clk_master);
 		} else {
 			dev_dbg(dev,
-				"disabled. Skipping TLB invalidation @ %#lx\n",
+				"disabled. Skipping TLB invalidation @ %#x\n",
 				iova);
 		}
 
@@ -856,7 +871,7 @@ static int exynos_iommu_domain_init(struct iommu_domain *domain)
 	if (!priv)
 		return -ENOMEM;
 
-	priv->pgtable = (unsigned long *)__get_free_pages(
+	priv->pgtable = (sysmmu_pte_t *)__get_free_pages(
 						GFP_KERNEL | __GFP_ZERO, 2);
 	if (!priv->pgtable)
 		goto err_pgtable;
@@ -978,19 +993,19 @@ static void exynos_iommu_detach_device(struct iommu_domain *domain,
 		dev_dbg(dev, "%s: No IOMMU is attached\n", __func__);
 }
 
-static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
+static sysmmu_pte_t *alloc_lv2entry(sysmmu_pte_t *sent, sysmmu_iova_t iova,
 					short *pgcounter)
 {
 	if (lv1ent_section(sent)) {
-		WARN(1, "Trying mapping on %#08lx mapped with 1MiB page", iova);
+		WARN(1, "Trying mapping on %#08x mapped with 1MiB page", iova);
 		return ERR_PTR(-EADDRINUSE);
 	}
 
 	if (lv1ent_fault(sent)) {
-		unsigned long *pent;
+		sysmmu_pte_t *pent;
 
 		pent = kmem_cache_zalloc(lv2table_kmem_cache, GFP_ATOMIC);
-		BUG_ON((unsigned long)pent & (LV2TABLE_SIZE - 1));
+		BUG_ON((unsigned int)pent & (LV2TABLE_SIZE - 1));
 		if (!pent)
 			return ERR_PTR(-ENOMEM);
 
@@ -1003,18 +1018,18 @@ static unsigned long *alloc_lv2entry(unsigned long *sent, unsigned long iova,
 	return page_entry(sent, iova);
 }
 
-static int lv1set_section(unsigned long *sent, unsigned long iova,
+static int lv1set_section(sysmmu_pte_t *sent, sysmmu_iova_t iova,
 			  phys_addr_t paddr, short *pgcnt)
 {
 	if (lv1ent_section(sent)) {
-		WARN(1, "Trying mapping on 1MiB@%#08lx that is mapped",
+		WARN(1, "Trying mapping on 1MiB@%#08x that is mapped",
 			iova);
 		return -EADDRINUSE;
 	}
 
 	if (lv1ent_page(sent)) {
 		if (*pgcnt != NUM_LV2ENTRIES) {
-			WARN(1, "Trying mapping on 1MiB@%#08lx that is mapped",
+			WARN(1, "Trying mapping on 1MiB@%#08x that is mapped",
 				iova);
 			return -EADDRINUSE;
 		}
@@ -1030,7 +1045,7 @@ static int lv1set_section(unsigned long *sent, unsigned long iova,
 	return 0;
 }
 
-static int lv2set_page(unsigned long *pent, phys_addr_t paddr, size_t size,
+static int lv2set_page(sysmmu_pte_t *pent, phys_addr_t paddr, size_t size,
 								short *pgcnt)
 {
 	if (size == SPAGE_SIZE) {
@@ -1062,11 +1077,12 @@ static int lv2set_page(unsigned long *pent, phys_addr_t paddr, size_t size,
 	return 0;
 }
 
-static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
+static int exynos_iommu_map(struct iommu_domain *domain, unsigned long l_iova,
 			 phys_addr_t paddr, size_t size, int prot)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
-	unsigned long *entry;
+	sysmmu_pte_t *entry;
+	sysmmu_iova_t iova = (sysmmu_iova_t)l_iova;
 	unsigned long flags;
 	int ret = -ENOMEM;
 
@@ -1080,7 +1096,7 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
 		ret = lv1set_section(entry, iova, paddr,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
 	} else {
-		unsigned long *pent;
+		sysmmu_pte_t *pent;
 
 		pent = alloc_lv2entry(entry, iova,
 					&priv->lv2entcnt[lv1ent_offset(iova)]);
@@ -1093,7 +1109,7 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
 	}
 
 	if (ret)
-		pr_debug("%s: Failed to map iova 0x%lx/0x%x bytes\n",
+		pr_debug("%s: Failed to map iova %#x/%#zx bytes\n",
 							__func__, iova, size);
 
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
@@ -1102,13 +1118,14 @@ static int exynos_iommu_map(struct iommu_domain *domain, unsigned long iova,
 }
 
 static size_t exynos_iommu_unmap(struct iommu_domain *domain,
-					       unsigned long iova, size_t size)
+				       unsigned long l_iova, size_t size)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
 	struct exynos_iommu_owner *owner;
-	unsigned long flags;
-	unsigned long *ent;
+	sysmmu_iova_t iova = (sysmmu_iova_t)l_iova;
+	sysmmu_pte_t *ent;
 	size_t err_pgsize;
+	unsigned long flags;
 
 	BUG_ON(priv->pgtable == NULL);
 
@@ -1175,7 +1192,7 @@ err:
 	spin_unlock_irqrestore(&priv->pgtablelock, flags);
 
 	WARN(1,
-	"%s: Failed due to size(%#x) @ %#08lx is smaller than page size %#x\n",
+	"%s: Failed due to size(%#zx) @ %#x is smaller than page size %#zx\n",
 	__func__, size, iova, err_pgsize);
 
 	return 0;
@@ -1185,7 +1202,7 @@ static phys_addr_t exynos_iommu_iova_to_phys(struct iommu_domain *domain,
 					  dma_addr_t iova)
 {
 	struct exynos_iommu_domain *priv = domain->priv;
-	unsigned long *entry;
+	sysmmu_pte_t *entry;
 	unsigned long flags;
 	phys_addr_t phys = 0;
 
-- 
1.7.9.5

[PATCH v12 23/31] iommu/exynos: use simpler function to get MMU version

[Shaik Ameer Basha]

From: Cho KyongHo <pullip.cho@samsung.com>

This commit changes the function to get MMU version simpler.

Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>
---
 drivers/iommu/exynos-iommu.c |   30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/drivers/iommu/exynos-iommu.c b/drivers/iommu/exynos-iommu.c
index 00915f2..54011e5 100755
--- a/drivers/iommu/exynos-iommu.c
+++ b/drivers/iommu/exynos-iommu.c
@@ -252,24 +252,6 @@ static unsigned int __raw_sysmmu_version(struct sysmmu_drvdata *data)
 	return MMU_RAW_VER(__raw_readl(data->sfrbase + REG_MMU_VERSION));
 }
 
-static unsigned int __sysmmu_version(struct sysmmu_drvdata *data,
-				     unsigned int *minor)
-{
-	unsigned int ver = 0;
-
-	ver = __raw_sysmmu_version(data);
-	if (ver > MAKE_MMU_VER(3, 3)) {
-		dev_err(data->sysmmu, "%s: version(%d.%d) is higher than 3.3\n",
-			__func__, MMU_MAJ_VER(ver), MMU_MIN_VER(ver));
-		BUG();
-	}
-
-	if (minor)
-		*minor = MMU_MIN_VER(ver);
-
-	return MMU_MAJ_VER(ver);
-}
-
 static bool sysmmu_block(void __iomem *sfrbase)
 {
 	int i = 120;
@@ -427,13 +409,13 @@ static bool __sysmmu_disable(struct sysmmu_drvdata *data)
 static void __sysmmu_init_config(struct sysmmu_drvdata *data)
 {
 	unsigned int cfg = CFG_LRU | CFG_QOS(15);
-	int maj, min = 0;
+	unsigned int ver;
 
-	maj = __sysmmu_version(data, &min);
-	if (maj == 3) {
-		if (min >= 2) {
+	ver = __raw_sysmmu_version(data);
+	if (MMU_MAJ_VER(ver) == 3) {
+		if (MMU_MIN_VER(ver) >= 2) {
 			cfg |= CFG_FLPDCACHE;
-			if (min == 3) {
+			if (MMU_MIN_VER(ver) == 3) {
 				cfg |= CFG_ACGEN;
 				cfg &= ~CFG_LRU;
 			} else {
@@ -591,7 +573,7 @@ static void sysmmu_tlb_invalidate_entry(struct device *dev, sysmmu_iova_t iova,
 			 * 1MB page can be cached in one of all sets.
 			 * 64KB page can be one of 16 consecutive sets.
 			 */
-			if (__sysmmu_version(data, NULL) == 2)
+			if (MMU_MAJ_VER(__raw_sysmmu_version(data)) == 2)
 				num_inv = min_t(unsigned int,
 						size / PAGE_SIZE, 64);
 
-- 
1.7.9.5