[RFC PATCH 0/3] Add ioremap_exec and extend drivers/misc/sram.c

[RFC PATCH 0/3] Add ioremap_exec and extend drivers/misc/sram.c

[Dave Gerlach]

Hi,
There are several instances when one would want to execute out of on-chip
SRAM, such as PM code on ARM platforms, so once again revisiting this
series to allow that. Seems that having a solution for allowing SRAM to be
mapped as executable will help clean up PM code on several ARM platforms and
also open the door for others like TI AM335x and AM437x. This was first sent
here [1] but this is rebased and updated for v4.6-rc.

Many platforms have migrated to using the generic SRAM driver at
drivers/misc/sram.c but it doesn't seem like there is a clean solution
for the common problem of needing to map executable pages.

Currently I see several platforms (at-91, imx6, socfpga) taking the
address allocated by the genpool given by the SRAM driver and then calling
__arm_ioremap_exec on it again to get a new address that can be executed.
This doesn't seem like the cleanest solution, but maybe it works for code
that is under mach-xxx but what about code that migrates outside into the
drivers layer? Do any other architectures have a requirement for this?

I've converted omap3 PM code to use the generic SRAM driver which I'll send
in a series right after this one, and AM335x and AM437x can both make use of
this series as well and even go as far as to move a chunk of the PM
code into drivers (needs to be updated but sent long ago here [2]), but
that will be blocked if we don't have a generic way to ioremap memory as
exec, or at least a way to do it from drivers/.

If we don't want to go down this path, is there a better idea for how to
call ioremap_exec from drivers/ that we can all agree on?

Regards,
Dave

[1] https://lkml.org/lkml/2014/11/26/575
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-November/306534.html

Dave Gerlach (1):
  misc: SRAM: Add option to map SRAM to allow code execution

Russ Dill (2):
  asm-generic: io: Add exec versions of ioremap
  lib: devres: Add exec and exec_nocache versions of devm_ioremap

 Documentation/devicetree/bindings/sram/sram.txt |  2 +
 arch/arm/include/asm/io.h                       |  5 +++
 arch/arm/mm/ioremap.c                           | 14 ++++++
 arch/arm/mm/nommu.c                             | 14 ++++++
 drivers/misc/sram.c                             |  8 ++++
 include/asm-generic/iomap.h                     |  5 +++
 include/linux/io.h                              |  5 +++
 lib/devres.c                                    | 58 +++++++++++++++++++++++++
 8 files changed, 111 insertions(+)

-- 
2.7.3

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Dave Gerlach]

From: Russ Dill <russ.dill@ti.com>

If code is to be copied into and area (such as SRAM) and run,
it needs to be marked as exec. Currently only an ARM version
of this exists, but a generic version will be useful for drivers.

Signed-off-by: Russ Dill <Russ.Dill@ti.com>
Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 arch/arm/include/asm/io.h   |  5 +++++
 arch/arm/mm/ioremap.c       | 14 ++++++++++++++
 arch/arm/mm/nommu.c         | 14 ++++++++++++++
 include/asm-generic/iomap.h |  5 +++++
 4 files changed, 38 insertions(+)

diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
index 485982084fe9..7d07a02cb7bc 100644
--- a/arch/arm/include/asm/io.h
+++ b/arch/arm/include/asm/io.h
@@ -399,6 +399,11 @@ void __iomem *ioremap_wc(resource_size_t res_cookie, size_t size);
 #define ioremap_wc ioremap_wc
 #define ioremap_wt ioremap_wc
 
+void __iomem *ioremap_exec(resource_size_t res_cookie, size_t size);
+void __iomem *ioremap_exec_nocache(resource_size_t res_cookie, size_t size);
+#define ioremap_exec ioremap_exec
+#define ioremap_exec_nocache ioremap_exec_nocache
+
 void iounmap(volatile void __iomem *iomem_cookie);
 #define iounmap iounmap
 
diff --git a/arch/arm/mm/ioremap.c b/arch/arm/mm/ioremap.c
index 66a978d05958..c6eef3c98074 100644
--- a/arch/arm/mm/ioremap.c
+++ b/arch/arm/mm/ioremap.c
@@ -400,6 +400,20 @@ EXPORT_SYMBOL(ioremap_wc);
  * clocks that would affect normal memory for example. Please see
  * CONFIG_GENERIC_ALLOCATOR for allocating external memory.
  */
+void __iomem *ioremap_exec(resource_size_t res_cookie, size_t size)
+{
+	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX,
+				   __builtin_return_address(0));
+}
+EXPORT_SYMBOL(ioremap_exec);
+
+void __iomem *ioremap_exec_nocache(resource_size_t res_cookie, size_t size)
+{
+	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX_NONCACHED,
+				   __builtin_return_address(0));
+}
+EXPORT_SYMBOL(ioremap_exec_nocache);
+
 void __iomem *
 __arm_ioremap_exec(phys_addr_t phys_addr, size_t size, bool cached)
 {
diff --git a/arch/arm/mm/nommu.c b/arch/arm/mm/nommu.c
index 1dd10936d68d..1d1d587340e6 100644
--- a/arch/arm/mm/nommu.c
+++ b/arch/arm/mm/nommu.c
@@ -380,6 +380,20 @@ void __iomem *ioremap_wc(resource_size_t res_cookie, size_t size)
 }
 EXPORT_SYMBOL(ioremap_wc);
 
+void __iomem *ioremap_exec(resource_size_t res_cookie, size_t size)
+{
+	return __arm_ioremap_caller(res_cookie, size, 0,
+				    __builtin_return_address(0));
+}
+EXPORT_SYMBOL(ioremap_exec);
+
+void __iomem *ioremap_exec_nocache(resource_size_t res_cookie, size_t size)
+{
+	return __arm_ioremap_caller(res_cookie, size, 0,
+				    __builtin_return_address(0));
+}
+EXPORT_SYMBOL(ioremap_exec_nocache);
+
 void __iounmap(volatile void __iomem *addr)
 {
 }
diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h
index d8f8622fa044..a789aad509c3 100644
--- a/include/asm-generic/iomap.h
+++ b/include/asm-generic/iomap.h
@@ -70,6 +70,11 @@ extern void ioport_unmap(void __iomem *);
 #define ioremap_wt ioremap_nocache
 #endif
 
+#ifndef ARCH_HAS_IOREMAP_EXEC
+#define ioremap_exec ioremap
+#define ioremap_exec_nocache ioremap_nocache
+#endif
+
 #ifdef CONFIG_PCI
 /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */
 struct pci_dev;
-- 
2.7.3

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Russell King - ARM Linux]

On Mon, May 09, 2016 at 04:41:49PM -0500, Dave Gerlach wrote:
> diff --git a/arch/arm/mm/ioremap.c b/arch/arm/mm/ioremap.c
> index 66a978d05958..c6eef3c98074 100644
> --- a/arch/arm/mm/ioremap.c
> +++ b/arch/arm/mm/ioremap.c
> @@ -400,6 +400,20 @@ EXPORT_SYMBOL(ioremap_wc);
>   * clocks that would affect normal memory for example. Please see
>   * CONFIG_GENERIC_ALLOCATOR for allocating external memory.
>   */
> +void __iomem *ioremap_exec(resource_size_t res_cookie, size_t size)
> +{
> +	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX,
> +				   __builtin_return_address(0));
> +}
> +EXPORT_SYMBOL(ioremap_exec);
> +
> +void __iomem *ioremap_exec_nocache(resource_size_t res_cookie, size_t size)
> +{
> +	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX_NONCACHED,
> +				   __builtin_return_address(0));
> +}
> +EXPORT_SYMBOL(ioremap_exec_nocache);

I think these should be called memremap_exec() and similar.  Please
see the description of memremap() in kernel/memremap.c.  If you're
going to be executing code, the region must not have I/O side effects
and according to the new definition of memremap() vs ioremap(), the
memremap() interfaces fit better.

Please also get these reviewed by Dan Williams who provided the
memremap() API.

Thanks.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Dave Gerlach]

Hi,
On 05/12/2016 11:37 AM, Russell King - ARM Linux wrote:
> On Mon, May 09, 2016 at 04:41:49PM -0500, Dave Gerlach wrote:
>> diff --git a/arch/arm/mm/ioremap.c b/arch/arm/mm/ioremap.c
>> index 66a978d05958..c6eef3c98074 100644
>> --- a/arch/arm/mm/ioremap.c
>> +++ b/arch/arm/mm/ioremap.c
>> @@ -400,6 +400,20 @@ EXPORT_SYMBOL(ioremap_wc);
>>    * clocks that would affect normal memory for example. Please see
>>    * CONFIG_GENERIC_ALLOCATOR for allocating external memory.
>>    */
>> +void __iomem *ioremap_exec(resource_size_t res_cookie, size_t size)
>> +{
>> +	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX,
>> +				   __builtin_return_address(0));
>> +}
>> +EXPORT_SYMBOL(ioremap_exec);
>> +
>> +void __iomem *ioremap_exec_nocache(resource_size_t res_cookie, size_t size)
>> +{
>> +	return arch_ioremap_caller(res_cookie, size, MT_MEMORY_RWX_NONCACHED,
>> +				   __builtin_return_address(0));
>> +}
>> +EXPORT_SYMBOL(ioremap_exec_nocache);
>
> I think these should be called memremap_exec() and similar.  Please
> see the description of memremap() in kernel/memremap.c.  If you're
> going to be executing code, the region must not have I/O side effects
> and according to the new definition of memremap() vs ioremap(), the
> memremap() interfaces fit better.
>
> Please also get these reviewed by Dan Williams who provided the
> memremap() API.

Ok thank you for the pointer. I agree, the memremap API looks like a 
better fit for this. I think it likely makes the most sense to still add 
these ioremap_exec and ioremap_exec_nocache and then call them through 
the memremap API based on new flags. This will fit into the current use 
model for memremap as it currently uses all of the other ioremap calls 
internally, and doing it how I just described will let this code evolve 
along with memremap.

I will put v2 together this way and send it out.

Regards,
Dave

>
> Thanks.
>

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Russell King - ARM Linux]

On Wed, May 18, 2016 at 09:12:20AM -0500, Dave Gerlach wrote:
> Ok thank you for the pointer. I agree, the memremap API looks like a better
> fit for this. I think it likely makes the most sense to still add these
> ioremap_exec and ioremap_exec_nocache and then call them through the
> memremap API based on new flags. This will fit into the current use model
> for memremap as it currently uses all of the other ioremap calls internally,
> and doing it how I just described will let this code evolve along with
> memremap.

I would _really_ prefer not to do that.  Why?  Because IO memory does
not have the required properties to be executable.  IO memory is normally
memory which has side effects - and by side effects, I mean reading it
can provoke hardware to perform some action.  You don't want to be
executing from such memory.

So, in my mind, ioremap_exec makes absolutely no sense, and having it
gives people a new interface to abuse - and abuse they will.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Arnd Bergmann]

On Wednesday 18 May 2016 18:51:02 Russell King - ARM Linux wrote:
> On Wed, May 18, 2016 at 09:12:20AM -0500, Dave Gerlach wrote:
> > Ok thank you for the pointer. I agree, the memremap API looks like a better
> > fit for this. I think it likely makes the most sense to still add these
> > ioremap_exec and ioremap_exec_nocache and then call them through the
> > memremap API based on new flags. This will fit into the current use model
> > for memremap as it currently uses all of the other ioremap calls internally,
> > and doing it how I just described will let this code evolve along with
> > memremap.
> 
> I would _really_ prefer not to do that.  Why?  Because IO memory does
> not have the required properties to be executable.  IO memory is normally
> memory which has side effects - and by side effects, I mean reading it
> can provoke hardware to perform some action.  You don't want to be
> executing from such memory.
> 
> So, in my mind, ioremap_exec makes absolutely no sense, and having it
> gives people a new interface to abuse - and abuse they will.

Agreed, calling it ioremap when it is really memremap makes no sense.

I also see another problem in the asm-generic portion:

+#ifndef ARCH_HAS_IOREMAP_EXEC
+#define ioremap_exec ioremap
+#define ioremap_exec_nocache ioremap_nocache
+#endif

The ARM version of ioremap_exec() that gets added in this patch is cached
(like memremap()), but then the asm-generic version is not? This is
even more confusing, it should at least do roughly the same thing across
architectures.

There should also be some documentation about what the expected behavior is, e.g.:

- is memremap_exec() by default cached or not? (I assume it would
  be like memremap())
- If we have an interface that does explicit uncached executable mapping,
  what about architectures on which this is not possible? Should they
  fall back to cached or non-executable, or cause a link error?  

	Arnd

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Russell King - ARM Linux]

On Wed, May 18, 2016 at 10:25:03PM +0200, Arnd Bergmann wrote:
> The ARM version of ioremap_exec() that gets added in this patch is cached
> (like memremap()), but then the asm-generic version is not? This is
> even more confusing, it should at least do roughly the same thing across
> architectures.
> 
> There should also be some documentation about what the expected behavior is, e.g.:
> 
> - is memremap_exec() by default cached or not? (I assume it would
>   be like memremap())
> - If we have an interface that does explicit uncached executable mapping,
>   what about architectures on which this is not possible? Should they
>   fall back to cached or non-executable, or cause a link error?  

Another important point is whether atomic instructions / kernel locks
can be located within the mapped memory.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[RFC PATCH 1/3] asm-generic: io: Add exec versions of ioremap

[Dave Gerlach]

On 05/18/2016 03:57 PM, Russell King - ARM Linux wrote:
> On Wed, May 18, 2016 at 10:25:03PM +0200, Arnd Bergmann wrote:
>> The ARM version of ioremap_exec() that gets added in this patch is cached
>> (like memremap()), but then the asm-generic version is not? This is
>> even more confusing, it should at least do roughly the same thing across
>> architectures.
>>
>> There should also be some documentation about what the expected behavior is, e.g.:
>>
>> - is memremap_exec() by default cached or not? (I assume it would
>>    be like memremap())
>> - If we have an interface that does explicit uncached executable mapping,
>>    what about architectures on which this is not possible? Should they
>>    fall back to cached or non-executable, or cause a link error?

Yes by default memremap_exec is cached, I do plan to add more explicit 
documentation.

Well, I dont think that memremap_exec will be called directly but rather 
using a flag with memremap as arch_memremap_wb is now, to keep the 
memremap API unified, so a link error will prevent this. Also, the 
function may be present in code but not actually used in all cases, the 
example that comes to mind is the drivers/misc/sram.c code where other 
runtime options are perfectly valid for determining how to map memory 
even on architectures that can't memremap_exec_nocache.

I think that a remap that can't deliver what you have asked for should 
return NULL here because if you are requesting executable, noncached 
memory you presumably will try to execute from it and fail, so the 
mapping should fail as it isn't actually valid if it can't do what you want.

>
> Another important point is whether atomic instructions / kernel locks
> can be located within the mapped memory.
>

At this point I'd imagine most of the users of this would be copying 
small chunks of relocatable code (likely written in assembly) that would 
handle low-level tasks without the need for atomic instructions/locks, 
but is this something we should explicitly forbid in documentation?

Regards,
Dave

[RFC PATCH 2/3] lib: devres: Add exec and exec_nocache versions of devm_ioremap

[Dave Gerlach]

From: Russ Dill <russ.dill@ti.com>

Now that there are _exec and _exec_nocache versions of ioremap, add devm
support for them.

Signed-off-by: Russ Dill <Russ.Dill@ti.com>
Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 include/linux/io.h |  5 +++++
 lib/devres.c       | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+)

diff --git a/include/linux/io.h b/include/linux/io.h
index e2c8419278c1..1c0442e5d72c 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -79,6 +79,11 @@ void __iomem *devm_ioremap_nocache(struct device *dev, resource_size_t offset,
 				   resource_size_t size);
 void __iomem *devm_ioremap_wc(struct device *dev, resource_size_t offset,
 				   resource_size_t size);
+void __iomem *devm_ioremap_exec(struct device *dev, resource_size_t offset,
+				unsigned long size);
+void __iomem *devm_ioremap_exec_nocache(struct device *dev,
+					resource_size_t offset,
+					unsigned long size);
 void devm_iounmap(struct device *dev, void __iomem *addr);
 int check_signature(const volatile void __iomem *io_addr,
 			const unsigned char *signature, int length);
diff --git a/lib/devres.c b/lib/devres.c
index cb1464c411a2..1181a739fd49 100644
--- a/lib/devres.c
+++ b/lib/devres.c
@@ -100,6 +100,64 @@ void __iomem *devm_ioremap_wc(struct device *dev, resource_size_t offset,
 EXPORT_SYMBOL(devm_ioremap_wc);
 
 /**
+ * devm_ioremap_exec - Managed ioremap_exec()
+ * @dev: Generic device to remap IO address for
+ * @offset: BUS offset to map
+ * @size: Size of map
+ *
+ * Managed ioremap_exec().  Map is automatically unmapped on driver detach.
+ */
+void __iomem *devm_ioremap_exec(struct device *dev, resource_size_t offset,
+				unsigned long size)
+{
+	void __iomem **ptr, *addr;
+
+	ptr = devres_alloc(devm_ioremap_release, sizeof(*ptr), GFP_KERNEL);
+	if (!ptr)
+		return NULL;
+
+	addr = ioremap_exec(offset, size);
+	if (addr) {
+		*ptr = addr;
+		devres_add(dev, ptr);
+	} else
+		devres_free(ptr);
+
+	return addr;
+}
+EXPORT_SYMBOL(devm_ioremap_exec);
+
+/**
+ * devm_ioremap_exec_nocache - Managed ioremap_exec_nocache()
+ * @dev: Generic device to remap IO address for
+ * @offset: BUS offset to map
+ * @size: Size of map
+ *
+ * Managed ioremap_exec_nocache().  Map is automatically unmapped on driver
+ * detach.
+ */
+void __iomem *devm_ioremap_exec_nocache(struct device *dev,
+					resource_size_t offset,
+					unsigned long size)
+{
+	void __iomem **ptr, *addr;
+
+	ptr = devres_alloc(devm_ioremap_release, sizeof(*ptr), GFP_KERNEL);
+	if (!ptr)
+		return NULL;
+
+	addr = ioremap_exec_nocache(offset, size);
+	if (addr) {
+		*ptr = addr;
+		devres_add(dev, ptr);
+	} else
+		devres_free(ptr);
+
+	return addr;
+}
+EXPORT_SYMBOL(devm_ioremap_exec_nocache);
+
+/**
  * devm_iounmap - Managed iounmap()
  * @dev: Generic device to unmap for
  * @addr: Address to unmap
-- 
2.7.3

[RFC PATCH 3/3] misc: SRAM: Add option to map SRAM to allow code execution

[Dave Gerlach]

Allow option for mapping SRAM as executable. DT node can specify
"memory-exec" and "memory-exec-nocache" to also map it as non-cached.
This is useful for platforms using the sram driver that need to run
PM code from sram like several ARM platforms.

Signed-off-by: Russ Dill <Russ.Dill@ti.com>
Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 Documentation/devicetree/bindings/sram/sram.txt | 2 ++
 drivers/misc/sram.c                             | 8 ++++++++
 2 files changed, 10 insertions(+)

diff --git a/Documentation/devicetree/bindings/sram/sram.txt b/Documentation/devicetree/bindings/sram/sram.txt
index 227e3a341af1..1f1d0aa52703 100644
--- a/Documentation/devicetree/bindings/sram/sram.txt
+++ b/Documentation/devicetree/bindings/sram/sram.txt
@@ -29,6 +29,8 @@ Optional properties in the sram node:
 
 - no-memory-wc : the flag indicating, that SRAM memory region has not to
                  be remapped as write combining. WC is used by default.
+- memory-exec : map range to allow code execution
+- memory-exec-nocache : map range to allow code execution and also non-cached
 
 Required properties in the area nodes:
 
diff --git a/drivers/misc/sram.c b/drivers/misc/sram.c
index 69cdabea9c03..7f00ba574a83 100644
--- a/drivers/misc/sram.c
+++ b/drivers/misc/sram.c
@@ -362,6 +362,14 @@ static int sram_probe(struct platform_device *pdev)
 
 	if (of_property_read_bool(pdev->dev.of_node, "no-memory-wc"))
 		sram->virt_base = devm_ioremap(sram->dev, res->start, size);
+	else if (of_property_read_bool(pdev->dev.of_node, "memory-exec"))
+		sram->virt_base = devm_ioremap_exec(sram->dev, res->start,
+						    size);
+	else if (of_property_read_bool(pdev->dev.of_node,
+				       "memory-exec-nocache"))
+		sram->virt_base = devm_ioremap_exec_nocache(sram->dev,
+							    res->start,
+							    size);
 	else
 		sram->virt_base = devm_ioremap_wc(sram->dev, res->start, size);
 	if (IS_ERR(sram->virt_base))
-- 
2.7.3

[RFC PATCH 0/3] Add ioremap_exec and extend drivers/misc/sram.c

[Tony Lindgren]

* Dave Gerlach <d-gerlach@ti.com> [160509 14:44]:
> Hi,
> There are several instances when one would want to execute out of on-chip
> SRAM, such as PM code on ARM platforms, so once again revisiting this
> series to allow that. Seems that having a solution for allowing SRAM to be
> mapped as executable will help clean up PM code on several ARM platforms and
> also open the door for others like TI AM335x and AM437x. This was first sent
> here [1] but this is rebased and updated for v4.6-rc.
> 
> Many platforms have migrated to using the generic SRAM driver at
> drivers/misc/sram.c but it doesn't seem like there is a clean solution
> for the common problem of needing to map executable pages.
> 
> Currently I see several platforms (at-91, imx6, socfpga) taking the
> address allocated by the genpool given by the SRAM driver and then calling
> __arm_ioremap_exec on it again to get a new address that can be executed.
> This doesn't seem like the cleanest solution, but maybe it works for code
> that is under mach-xxx but what about code that migrates outside into the
> drivers layer? Do any other architectures have a requirement for this?
> 
> I've converted omap3 PM code to use the generic SRAM driver which I'll send
> in a series right after this one, and AM335x and AM437x can both make use of
> this series as well and even go as far as to move a chunk of the PM
> code into drivers (needs to be updated but sent long ago here [2]), but
> that will be blocked if we don't have a generic way to ioremap memory as
> exec, or at least a way to do it from drivers/.
> 
> If we don't want to go down this path, is there a better idea for how to
> call ioremap_exec from drivers/ that we can all agree on?

Looks to me that this should allow moving the current SRAM code
into drivers. So feel free to add:

Acked-by: Tony Lindgren <tony@atomide.com>