[PATCH v2 2/9] arm64: mm: free __init memory via the linear mapping

[PATCH v2 2/9] arm64: mm: free __init memory via the linear mapping

[Ard Biesheuvel]

The implementation of free_initmem_default() expects __init_begin
and __init_end to be covered by the linear mapping, which is no
longer the case. So open code it instead, using addresses that are
explicitly translated from kernel virtual to linear virtual.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/mm/init.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index a08153c99ffa..cebad6cceda8 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -416,7 +416,8 @@ void __init mem_init(void)
 
 void free_initmem(void)
 {
-	free_initmem_default(0);
+	free_reserved_area(__va(__pa(__init_begin)), __va(__pa(__init_end)),
+			   0, "unused kernel");
 	fixup_init();
 }
 
-- 
2.5.0

[PATCH v2 0/9] arm64: optimize virt_to_page and page_address

[Ard Biesheuvel]

Apologies for the wall of text. This is a followup to 'restrict virt_to_page
to linear region (instead of __pa)' [1], posted on the 24th of February.
This v2 series applies onto v4.6-rc1 with the series 'arm64: memstart_addr
alignment and vmemmap offset fixes' [2] applied on top.

Only minor changes since v1, primarily replacing 'x >> PAGE_SHIFT' instances
with PHYS_PFN, and rebasing onto the latest upstream.

When building the arm64 defconfig kernel, which has CONFIG_SPARSEMEM_VMEMMAP
enabled, the implementations of virt_to_page and its converse
[lowmem_]page_address resolve to

   virt_to_page
     6f8:   b6300180        tbz     x0, #38, 728 <bar+0x30>
     6fc:   90000001        adrp    x1, 0 <memstart_addr>
     700:   92409400        and     x0, x0, #0x3fffffffff
     704:   f9400021        ldr     x1, [x1]
     708:   8b000020        add     x0, x1, x0
     70c:   37000261        tbnz    w1, #0, 758 <bar+0x60>
     710:   d34cfc00        lsr     x0, x0, #12
     714:   d2dff7c2        mov     x2, #0xffbe00000000
     718:   cb813000        sub     x0, x0, x1, asr #12
     71c:   f2ffffe2        movk    x2, #0xffff, lsl #48
     720:   8b001840        add     x0, x2, x0, lsl #6
     724:   d65f03c0        ret
     728:   90000002        adrp    x2, 0 <init_pgd>
     72c:   90000001        adrp    x1, 0 <memstart_addr>
     730:   f9400042        ldr     x2, [x2]
     734:   f9400021        ldr     x1, [x1]
     738:   cb020000        sub     x0, x0, x2
     73c:   d2dff7c2        mov     x2, #0xffbe00000000
     740:   d34cfc00        lsr     x0, x0, #12
     744:   f2ffffe2        movk    x2, #0xffff, lsl #48
     748:   cb813000        sub     x0, x0, x1, asr #12
     74c:   8b001840        add     x0, x2, x0, lsl #6
     750:   d65f03c0        ret
     754:   d503201f        nop
     758:   d4210000        brk     #0x800

   page_address:
     6c0:   90000002        adrp    x2, 0 <memstart_addr>
     6c4:   d2c00841        mov     x1, #0x4200000000
     6c8:   f9400043        ldr     x3, [x2]
     6cc:   934cfc62        asr     x2, x3, #12
     6d0:   8b021800        add     x0, x0, x2, lsl #6
     6d4:   8b010001        add     x1, x0, x1
     6d8:   9346fc21        asr     x1, x1, #6
     6dc:   d374cc21        lsl     x1, x1, #12
     6e0:   37000083        tbnz    w3, #0, 6f0 <foo+0x30>
     6e4:   cb030020        sub     x0, x1, x3
     6e8:   b25a6400        orr     x0, x0, #0xffffffc000000000
     6ec:   d65f03c0        ret
     6f0:   d4210000        brk     #0x800

Disappointingly, even though this translation is independent of the physical
start of RAM since commit dfd55ad85e ("arm64: vmemmap: use virtual projection
of linear region"), the expression is evaluated in a way that does not allow
the compiler to eliminate the read of memstart_addr, presumably since it is
unaware that its value is aligned to PAGE_SIZE, and that shifting it down and
up again by PAGE_SHIFT bits produces the exact same value.

So let's give the compiler a hand here. First of all, let's reimplement
virt_to_page() (patch #6) so that it explicitly translates without taking
the physical placement into account. This results in the virt_to_page()
translation to only work correctly for addresses above PAGE_OFFSET, but
this is a reasonable restriction to impose, even if it means a couple of
incorrect uses need to be fixed (patches #1 to #4). If we also, in patch #5,
move the vmemmap region right below the linear region (which guarantees that
the region is always aligned to a power-of-2 upper bound of its size, which
means we can treat VMEMMAP_START as a bitmask rather than an offset), we end
up with

   virt_to_page
     6d0:   d34c9400        ubfx    x0, x0, #12, #26
     6d4:   d2dff7c1        mov     x1, #0xffbe00000000
     6d8:   f2ffffe1        movk    x1, #0xffff, lsl #48
     6dc:   aa001820        orr     x0, x1, x0, lsl #6
     6e0:   d65f03c0        ret

In the same way, we can get page_address to look like this

   page_address:
     6c0:   d37a7c00        ubfiz   x0, x0, #6, #32
     6c4:   b25a6400        orr     x0, x0, #0xffffffc000000000
     6c8:   d65f03c0        ret

However, in this case, we need to slightly refactor the implementation of
lowmem_page_paddress(), since it performs an explicit page-to-pa-to-va
translation, rather than going through an opaque arch-defined definition
of page_to_virt. (patches #7 to #9)

[1] http://thread.gmane.org/gmane.linux.ports.arm.kernel/481327
[2] http://thread.gmane.org/gmane.linux.ports.arm.kernel/488876

Ard Biesheuvel (9):
  arm64: vdso: avoid virt_to_page() translations on kernel symbols
  arm64: mm: free __init memory via the linear mapping
  arm64: mm: avoid virt_to_page() translation for the zero page
  arm64: insn: avoid virt_to_page() translations on core kernel symbols
  arm64: mm: move vmemmap region right below the linear region
  arm64: mm: restrict virt_to_page() to the linear mapping
  nios2: use correct void* return type for page_to_virt()
  openrisc: drop wrongly typed definition of page_to_virt()
  mm: replace open coded page to virt conversion with page_to_virt()

 arch/arm64/include/asm/memory.h  | 30 ++++++++++++++++++--
 arch/arm64/include/asm/pgtable.h | 13 +++------
 arch/arm64/kernel/insn.c         |  2 +-
 arch/arm64/kernel/vdso.c         |  4 +--
 arch/arm64/mm/dump.c             | 16 +++++------
 arch/arm64/mm/init.c             | 17 +++++++----
 arch/nios2/include/asm/io.h      |  1 -
 arch/nios2/include/asm/page.h    |  2 +-
 arch/nios2/include/asm/pgtable.h |  2 +-
 arch/openrisc/include/asm/page.h |  2 --
 include/linux/mm.h               |  6 +++-
 11 files changed, 62 insertions(+), 33 deletions(-)

-- 
2.5.0

[PATCH v2 1/9] arm64: vdso: avoid virt_to_page() translations on kernel symbols

[Ard Biesheuvel]

The translation performed by virt_to_page() is only valid for linear
addresses, and kernel symbols are no longer in the linear mapping.
So perform the __pa() translation explicitly, which does the right
thing in either case, and only then translate to a struct page offset.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/kernel/vdso.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 97bc68f4c689..64fc030be0f2 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -131,11 +131,11 @@ static int __init vdso_init(void)
 		return -ENOMEM;
 
 	/* Grab the vDSO data page. */
-	vdso_pagelist[0] = virt_to_page(vdso_data);
+	vdso_pagelist[0] = pfn_to_page(PHYS_PFN(__pa(vdso_data)));
 
 	/* Grab the vDSO code pages. */
 	for (i = 0; i < vdso_pages; i++)
-		vdso_pagelist[i + 1] = virt_to_page(&vdso_start + i * PAGE_SIZE);
+		vdso_pagelist[i + 1] = pfn_to_page(PHYS_PFN(__pa(&vdso_start)) + i);
 
 	/* Populate the special mapping structures */
 	vdso_spec[0] = (struct vm_special_mapping) {
-- 
2.5.0

[PATCH v2 2/9] arm64: mm: free __init memory via the linear mapping

[Ard Biesheuvel]

The implementation of free_initmem_default() expects __init_begin
and __init_end to be covered by the linear mapping, which is no
longer the case. So open code it instead, using addresses that are
explicitly translated from kernel virtual to linear virtual.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/mm/init.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 82ced5fa1e66..89376f3c65a3 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -452,7 +452,8 @@ void __init mem_init(void)
 
 void free_initmem(void)
 {
-	free_initmem_default(0);
+	free_reserved_area(__va(__pa(__init_begin)), __va(__pa(__init_end)),
+			   0, "unused kernel");
 	fixup_init();
 }
 
-- 
2.5.0

[PATCH v2 3/9] arm64: mm: avoid virt_to_page() translation for the zero page

[Ard Biesheuvel]

The zero page is statically allocated, so grab its struct page pointer
without using virt_to_page(), which will be restricted to the linear
mapping later.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/include/asm/pgtable.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index aa6106ac050c..377257a8d393 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -57,7 +57,7 @@ extern void __pgd_error(const char *file, int line, unsigned long val);
  * for zero-mapped memory areas etc..
  */
 extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];
-#define ZERO_PAGE(vaddr)	virt_to_page(empty_zero_page)
+#define ZERO_PAGE(vaddr)	pfn_to_page(PHYS_PFN(__pa(empty_zero_page)))
 
 #define pte_ERROR(pte)		__pte_error(__FILE__, __LINE__, pte_val(pte))
 
-- 
2.5.0

[PATCH v2 4/9] arm64: insn: avoid virt_to_page() translations on core kernel symbols

[Ard Biesheuvel]

Before restricting virt_to_page() to the linear mapping, ensure that
the text patching code does not use it to resolve references into the
core kernel text, which is mapped in the vmalloc area.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/kernel/insn.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c
index 7371455160e5..368c08290dd8 100644
--- a/arch/arm64/kernel/insn.c
+++ b/arch/arm64/kernel/insn.c
@@ -96,7 +96,7 @@ static void __kprobes *patch_map(void *addr, int fixmap)
 	if (module && IS_ENABLED(CONFIG_DEBUG_SET_MODULE_RONX))
 		page = vmalloc_to_page(addr);
 	else if (!module && IS_ENABLED(CONFIG_DEBUG_RODATA))
-		page = virt_to_page(addr);
+		page = pfn_to_page(PHYS_PFN(__pa(addr)));
 	else
 		return addr;
 
-- 
2.5.0

[PATCH v2 5/9] arm64: mm: move vmemmap region right below the linear region

[Ard Biesheuvel]

This moves the vmemmap region right below PAGE_OFFSET, aka the start
of the linear region, and redefines its size to be a power of two.
Due to the placement of PAGE_OFFSET in the middle of the address space,
whose size is a power of two as well, this guarantees that virt to
page conversions and vice versa can be implemented efficiently, by
masking and shifting rather than ordinary arithmetic.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/include/asm/memory.h  | 18 +++++++++++++++++-
 arch/arm64/include/asm/pgtable.h | 11 +++--------
 arch/arm64/mm/dump.c             | 16 ++++++++--------
 arch/arm64/mm/init.c             | 14 ++++++++++----
 4 files changed, 38 insertions(+), 21 deletions(-)

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 12f8a00fb3f1..8a2ab195ca77 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -40,6 +40,21 @@
 #define PCI_IO_SIZE		SZ_16M
 
 /*
+ * Log2 of the upper bound of the size of a struct page. Used for sizing
+ * the vmemmap region only, does not affect actual memory footprint.
+ * We don't use sizeof(struct page) directly since taking its size here
+ * requires its definition to be available at this point in the inclusion
+ * chain, and it may not be a power of 2 in the first place.
+ */
+#define STRUCT_PAGE_MAX_SHIFT	6
+
+/*
+ * VMEMMAP_SIZE - allows the whole linear region to be covered by
+ *                a struct page array
+ */
+#define VMEMMAP_SIZE (UL(1) << (VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT))
+
+/*
  * PAGE_OFFSET - the virtual address of the start of the kernel image (top
  *		 (VA_BITS - 1))
  * VA_BITS - the maximum number of bits for virtual addresses.
@@ -54,7 +69,8 @@
 #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)
 #define MODULES_VADDR		(VA_START + KASAN_SHADOW_SIZE)
 #define MODULES_VSIZE		(SZ_128M)
-#define PCI_IO_END		(PAGE_OFFSET - SZ_2M)
+#define VMEMMAP_START		(PAGE_OFFSET - VMEMMAP_SIZE)
+#define PCI_IO_END		(VMEMMAP_START - SZ_2M)
 #define PCI_IO_START		(PCI_IO_END - PCI_IO_SIZE)
 #define FIXADDR_TOP		(PCI_IO_START - SZ_2M)
 #define TASK_SIZE_64		(UL(1) << VA_BITS)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 377257a8d393..2abfa4d09e65 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -24,20 +24,15 @@
 #include <asm/pgtable-prot.h>
 
 /*
- * VMALLOC and SPARSEMEM_VMEMMAP ranges.
+ * VMALLOC range.
  *
- * VMEMAP_SIZE: allows the whole linear region to be covered by a struct page array
- *	(rounded up to PUD_SIZE).
  * VMALLOC_START: beginning of the kernel vmalloc space
- * VMALLOC_END: extends to the available space below vmmemmap, PCI I/O space,
- *	fixed mappings and modules
+ * VMALLOC_END: extends to the available space below vmmemmap, PCI I/O space
+ *	and fixed mappings
  */
-#define VMEMMAP_SIZE		ALIGN((1UL << (VA_BITS - PAGE_SHIFT - 1)) * sizeof(struct page), PUD_SIZE)
-
 #define VMALLOC_START		(MODULES_END)
 #define VMALLOC_END		(PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K)
 
-#define VMEMMAP_START		(VMALLOC_END + SZ_64K)
 #define vmemmap			((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT))
 
 #define FIRST_USER_ADDRESS	0UL
diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c
index f9271cb2f5e3..a21f47421b0c 100644
--- a/arch/arm64/mm/dump.c
+++ b/arch/arm64/mm/dump.c
@@ -37,14 +37,14 @@ enum address_markers_idx {
 	MODULES_END_NR,
 	VMALLOC_START_NR,
 	VMALLOC_END_NR,
-#ifdef CONFIG_SPARSEMEM_VMEMMAP
-	VMEMMAP_START_NR,
-	VMEMMAP_END_NR,
-#endif
 	FIXADDR_START_NR,
 	FIXADDR_END_NR,
 	PCI_START_NR,
 	PCI_END_NR,
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+	VMEMMAP_START_NR,
+	VMEMMAP_END_NR,
+#endif
 	KERNEL_SPACE_NR,
 };
 
@@ -53,14 +53,14 @@ static struct addr_marker address_markers[] = {
 	{ MODULES_END,		"Modules end" },
 	{ VMALLOC_START,	"vmalloc() Area" },
 	{ VMALLOC_END,		"vmalloc() End" },
-#ifdef CONFIG_SPARSEMEM_VMEMMAP
-	{ 0,			"vmemmap start" },
-	{ 0,			"vmemmap end" },
-#endif
 	{ FIXADDR_START,	"Fixmap start" },
 	{ FIXADDR_TOP,		"Fixmap end" },
 	{ PCI_IO_START,		"PCI I/O start" },
 	{ PCI_IO_END,		"PCI I/O end" },
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+	{ 0,			"vmemmap start" },
+	{ 0,			"vmemmap end" },
+#endif
 	{ PAGE_OFFSET,		"Linear Mapping" },
 	{ -1,			NULL },
 };
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 89376f3c65a3..d55d720dba79 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -412,6 +412,10 @@ void __init mem_init(void)
 		MLK_ROUNDUP(__start_rodata, _etext),
 		MLK_ROUNDUP(__init_begin, __init_end),
 		MLK_ROUNDUP(_sdata, _edata));
+	pr_cont("    fixed   : 0x%16lx - 0x%16lx   (%6ld KB)\n",
+		MLK(FIXADDR_START, FIXADDR_TOP));
+	pr_cont("    PCI I/O : 0x%16lx - 0x%16lx   (%6ld MB)\n",
+		MLM(PCI_IO_START, PCI_IO_END));
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 	pr_cont("    vmemmap : 0x%16lx - 0x%16lx   (%6ld GB maximum)\n"
 		"              0x%16lx - 0x%16lx   (%6ld MB actual)\n",
@@ -420,10 +424,6 @@ void __init mem_init(void)
 		MLM((unsigned long)phys_to_page(memblock_start_of_DRAM()),
 		    (unsigned long)virt_to_page(high_memory)));
 #endif
-	pr_cont("    fixed   : 0x%16lx - 0x%16lx   (%6ld KB)\n",
-		MLK(FIXADDR_START, FIXADDR_TOP));
-	pr_cont("    PCI I/O : 0x%16lx - 0x%16lx   (%6ld MB)\n",
-		MLM(PCI_IO_START, PCI_IO_END));
 	pr_cont("    memory  : 0x%16lx - 0x%16lx   (%6ld MB)\n",
 		MLM(__phys_to_virt(memblock_start_of_DRAM()),
 		    (unsigned long)high_memory));
@@ -440,6 +440,12 @@ void __init mem_init(void)
 	BUILD_BUG_ON(TASK_SIZE_32			> TASK_SIZE_64);
 #endif
 
+	/*
+	 * Make sure we chose the upper bound of sizeof(struct page)
+	 * correctly.
+	 */
+	BUILD_BUG_ON(sizeof(struct page) > (1 << STRUCT_PAGE_MAX_SHIFT));
+
 	if (PAGE_SIZE >= 16384 && get_num_physpages() <= 128) {
 		extern int sysctl_overcommit_memory;
 		/*
-- 
2.5.0

[PATCH v2 6/9] arm64: mm: restrict virt_to_page() to the linear mapping

[Ard Biesheuvel]

Now that the vmemmap region has been redefined to cover the linear region
rather than the entire physical address space, we no longer need to
perform a virtual-to-physical translation in the implementaion of
virt_to_page(). This restricts virt_to_page() translations to the linear
region, so redefine virt_addr_valid() as well.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm64/include/asm/memory.h | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 8a2ab195ca77..f412f502ccdd 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -208,9 +208,19 @@ static inline void *phys_to_virt(phys_addr_t x)
  */
 #define ARCH_PFN_OFFSET		((unsigned long)PHYS_PFN_OFFSET)
 
+#ifndef CONFIG_SPARSEMEM_VMEMMAP
 #define virt_to_page(kaddr)	pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
-#define	virt_addr_valid(kaddr)	pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
+#define virt_addr_valid(kaddr)	pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
+#else
+#define __virt_to_pgoff(kaddr)	(((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page))
+#define __page_to_voff(kaddr)	(((u64)(page) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page))
+
+#define page_to_virt(page)	((void *)((__page_to_voff(page)) | PAGE_OFFSET))
+#define virt_to_page(vaddr)	((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START))
 
+#define virt_addr_valid(kaddr)	pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \
+					   + PHYS_OFFSET) >> PAGE_SHIFT)
+#endif
 #endif
 
 #include <asm-generic/memory_model.h>
-- 
2.5.0

[PATCH v2 7/9] nios2: use correct void* return type for page_to_virt()

[Ard Biesheuvel]

To align with other architectures, the expression procuded by expanding
the macro page_to_virt() should be of type void*, since it returns a
virtual address. Fix that, and also fix up an instance where page_to_virt
was expected to return 'unsigned long', and drop another instance that was
entirely unused (page_to_bus)

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/nios2/include/asm/io.h      | 1 -
 arch/nios2/include/asm/page.h    | 2 +-
 arch/nios2/include/asm/pgtable.h | 2 +-
 3 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/nios2/include/asm/io.h b/arch/nios2/include/asm/io.h
index c5a62da22cd2..ce072ba0f8dd 100644
--- a/arch/nios2/include/asm/io.h
+++ b/arch/nios2/include/asm/io.h
@@ -50,7 +50,6 @@ static inline void iounmap(void __iomem *addr)
 
 /* Pages to physical address... */
 #define page_to_phys(page)	virt_to_phys(page_to_virt(page))
-#define page_to_bus(page)	page_to_virt(page)
 
 /* Macros used for converting between virtual and physical mappings. */
 #define phys_to_virt(vaddr)	\
diff --git a/arch/nios2/include/asm/page.h b/arch/nios2/include/asm/page.h
index 4b32d6fd9d98..c1683f51ad0f 100644
--- a/arch/nios2/include/asm/page.h
+++ b/arch/nios2/include/asm/page.h
@@ -84,7 +84,7 @@ extern struct page *mem_map;
 	((void *)((unsigned long)(x) + PAGE_OFFSET - PHYS_OFFSET))
 
 #define page_to_virt(page)	\
-	((((page) - mem_map) << PAGE_SHIFT) + PAGE_OFFSET)
+	((void *)(((page) - mem_map) << PAGE_SHIFT) + PAGE_OFFSET)
 
 # define pfn_to_kaddr(pfn)	__va((pfn) << PAGE_SHIFT)
 # define pfn_valid(pfn)		((pfn) >= ARCH_PFN_OFFSET &&	\
diff --git a/arch/nios2/include/asm/pgtable.h b/arch/nios2/include/asm/pgtable.h
index a213e8c9aad0..298393c3cb42 100644
--- a/arch/nios2/include/asm/pgtable.h
+++ b/arch/nios2/include/asm/pgtable.h
@@ -209,7 +209,7 @@ static inline void set_pte(pte_t *ptep, pte_t pteval)
 static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
 			      pte_t *ptep, pte_t pteval)
 {
-	unsigned long paddr = page_to_virt(pte_page(pteval));
+	unsigned long paddr = (unsigned long)page_to_virt(pte_page(pteval));
 
 	flush_dcache_range(paddr, paddr + PAGE_SIZE);
 	set_pte(ptep, pteval);
-- 
2.5.0

[PATCH v2 8/9] openrisc: drop wrongly typed definition of page_to_virt()

[Ard Biesheuvel]

To align with generic code and other architectures that expect the macro
page_to_virt to produce an expression whose type is 'void*', drop the
arch specific definition, which is never referenced anyway.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/openrisc/include/asm/page.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/openrisc/include/asm/page.h b/arch/openrisc/include/asm/page.h
index e613d3673034..35bcb7cd2cde 100644
--- a/arch/openrisc/include/asm/page.h
+++ b/arch/openrisc/include/asm/page.h
@@ -81,8 +81,6 @@ typedef struct page *pgtable_t;
 
 #define virt_to_page(addr) \
 	(mem_map + (((unsigned long)(addr)-PAGE_OFFSET) >> PAGE_SHIFT))
-#define page_to_virt(page) \
-	((((page) - mem_map) << PAGE_SHIFT) + PAGE_OFFSET)
 
 #define page_to_phys(page)      ((dma_addr_t)page_to_pfn(page) << PAGE_SHIFT)
 
-- 
2.5.0

[PATCH v2 9/9] mm: replace open coded page to virt conversion with page_to_virt()

[Ard Biesheuvel]

The open coded conversion from struct page address to virtual address in
lowmem_page_address() involves an intermediate conversion step to pfn
number/physical address. Since the placement of the struct page array
relative to the linear mapping may be completely independent from the
placement of physical RAM (as is that case for arm64 after commit
dfd55ad85e 'arm64: vmemmap: use virtual projection of linear region'),
the conversion to physical address and back again should factor out of
the equation, but unfortunately, the shifting and pointer arithmetic
involved prevent this from happening, and the resulting calculation
essentially subtracts the address of the start of physical memory and
adds it back again, in a way that prevents the compiler from optimizing
it away.

Since the start of physical memory is not a build time constant on arm64,
the resulting conversion involves an unnecessary memory access, which
we would like to get rid of. So replace the open coded conversion with
a call to page_to_virt(), and use the open coded conversion as its
default definition, to be overriden by the architecture, if desired.
The existing arch specific definitions of page_to_virt are all equivalent
to this default definition, so by itself this patch is a no-op.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 include/linux/mm.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index ed6407d1b7b5..474c4625756e 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -72,6 +72,10 @@ extern int mmap_rnd_compat_bits __read_mostly;
 #define __pa_symbol(x)  __pa(RELOC_HIDE((unsigned long)(x), 0))
 #endif
 
+#ifndef page_to_virt
+#define page_to_virt(x)	__va(PFN_PHYS(page_to_pfn(x)))
+#endif
+
 /*
  * To prevent common memory management code establishing
  * a zero page mapping on a read fault.
@@ -948,7 +952,7 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
 
 static __always_inline void *lowmem_page_address(const struct page *page)
 {
-	return __va(PFN_PHYS(page_to_pfn(page)));
+	return page_to_virt(page);
 }
 
 #if defined(CONFIG_HIGHMEM) && !defined(WANT_PAGE_VIRTUAL)
-- 
2.5.0

[PATCH v2 9/9] mm: replace open coded page to virt conversion with page_to_virt()

[Will Deacon]

On Wed, Mar 30, 2016 at 04:46:04PM +0200, Ard Biesheuvel wrote:
> The open coded conversion from struct page address to virtual address in
> lowmem_page_address() involves an intermediate conversion step to pfn
> number/physical address. Since the placement of the struct page array
> relative to the linear mapping may be completely independent from the
> placement of physical RAM (as is that case for arm64 after commit
> dfd55ad85e 'arm64: vmemmap: use virtual projection of linear region'),
> the conversion to physical address and back again should factor out of
> the equation, but unfortunately, the shifting and pointer arithmetic
> involved prevent this from happening, and the resulting calculation
> essentially subtracts the address of the start of physical memory and
> adds it back again, in a way that prevents the compiler from optimizing
> it away.
> 
> Since the start of physical memory is not a build time constant on arm64,
> the resulting conversion involves an unnecessary memory access, which
> we would like to get rid of. So replace the open coded conversion with
> a call to page_to_virt(), and use the open coded conversion as its
> default definition, to be overriden by the architecture, if desired.
> The existing arch specific definitions of page_to_virt are all equivalent
> to this default definition, so by itself this patch is a no-op.
> 
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Acked-by: Will Deacon <will.deacon@arm.com>

I assume you'll post this patch (and the nios2/openrisc) patches as
individual patches targetting the relevant trees?

Will

> ---
>  include/linux/mm.h | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index ed6407d1b7b5..474c4625756e 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -72,6 +72,10 @@ extern int mmap_rnd_compat_bits __read_mostly;
>  #define __pa_symbol(x)  __pa(RELOC_HIDE((unsigned long)(x), 0))
>  #endif
>  
> +#ifndef page_to_virt
> +#define page_to_virt(x)	__va(PFN_PHYS(page_to_pfn(x)))
> +#endif
> +
>  /*
>   * To prevent common memory management code establishing
>   * a zero page mapping on a read fault.
> @@ -948,7 +952,7 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
>  
>  static __always_inline void *lowmem_page_address(const struct page *page)
>  {
> -	return __va(PFN_PHYS(page_to_pfn(page)));
> +	return page_to_virt(page);
>  }
>  
>  #if defined(CONFIG_HIGHMEM) && !defined(WANT_PAGE_VIRTUAL)
> -- 
> 2.5.0
> 

[PATCH v2 9/9] mm: replace open coded page to virt conversion with page_to_virt()

[Ard Biesheuvel]

On 14 April 2016 at 17:25, Will Deacon <will.deacon@arm.com> wrote:
> On Wed, Mar 30, 2016 at 04:46:04PM +0200, Ard Biesheuvel wrote:
>> The open coded conversion from struct page address to virtual address in
>> lowmem_page_address() involves an intermediate conversion step to pfn
>> number/physical address. Since the placement of the struct page array
>> relative to the linear mapping may be completely independent from the
>> placement of physical RAM (as is that case for arm64 after commit
>> dfd55ad85e 'arm64: vmemmap: use virtual projection of linear region'),
>> the conversion to physical address and back again should factor out of
>> the equation, but unfortunately, the shifting and pointer arithmetic
>> involved prevent this from happening, and the resulting calculation
>> essentially subtracts the address of the start of physical memory and
>> adds it back again, in a way that prevents the compiler from optimizing
>> it away.
>>
>> Since the start of physical memory is not a build time constant on arm64,
>> the resulting conversion involves an unnecessary memory access, which
>> we would like to get rid of. So replace the open coded conversion with
>> a call to page_to_virt(), and use the open coded conversion as its
>> default definition, to be overriden by the architecture, if desired.
>> The existing arch specific definitions of page_to_virt are all equivalent
>> to this default definition, so by itself this patch is a no-op.
>>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>
> Acked-by: Will Deacon <will.deacon@arm.com>
>
> I assume you'll post this patch (and the nios2/openrisc) patches as
> individual patches targetting the relevant trees?
>

Sure, as they are completely independent from the rest of the series.