* [PATCH v2 resend 1/6] extable: add support for relative extables to search and sort routines
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 2/6] alpha/extable: use generic " Ard Biesheuvel
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
This adds support to the generic search_extable() and sort_extable()
implementations for dealing with exception table entries whose fields
contain relative offsets rather than absolute addresses.
Acked-by: Helge Deller <deller@gmx.de>
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Acked-by: Tony Luck <tony.luck@intel.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
lib/extable.c | 50 ++++++++++++++++----
1 file changed, 41 insertions(+), 9 deletions(-)
diff --git a/lib/extable.c b/lib/extable.c
index 4cac81ec225e..0be02ad561e9 100644
--- a/lib/extable.c
+++ b/lib/extable.c
@@ -14,7 +14,37 @@
#include <linux/sort.h>
#include <asm/uaccess.h>
+#ifndef ARCH_HAS_RELATIVE_EXTABLE
+#define ex_to_insn(x) ((x)->insn)
+#else
+static inline unsigned long ex_to_insn(const struct exception_table_entry *x)
+{
+ return (unsigned long)&x->insn + x->insn;
+}
+#endif
+
#ifndef ARCH_HAS_SORT_EXTABLE
+#ifndef ARCH_HAS_RELATIVE_EXTABLE
+#define swap_ex NULL
+#else
+static void swap_ex(void *a, void *b, int size)
+{
+ struct exception_table_entry *x = a, *y = b, tmp;
+ int delta = b - a;
+
+ tmp = *x;
+ x->insn = y->insn + delta;
+ y->insn = tmp.insn - delta;
+
+#ifdef swap_ex_entry_fixup
+ swap_ex_entry_fixup(x, y, tmp, delta);
+#else
+ x->fixup = y->fixup + delta;
+ y->fixup = tmp.fixup - delta;
+#endif
+}
+#endif /* ARCH_HAS_RELATIVE_EXTABLE */
+
/*
* The exception table needs to be sorted so that the binary
* search that we use to find entries in it works properly.
@@ -26,9 +56,9 @@ static int cmp_ex(const void *a, const void *b)
const struct exception_table_entry *x = a, *y = b;
/* avoid overflow */
- if (x->insn > y->insn)
+ if (ex_to_insn(x) > ex_to_insn(y))
return 1;
- if (x->insn < y->insn)
+ if (ex_to_insn(x) < ex_to_insn(y))
return -1;
return 0;
}
@@ -37,7 +67,7 @@ void sort_extable(struct exception_table_entry *start,
struct exception_table_entry *finish)
{
sort(start, finish - start, sizeof(struct exception_table_entry),
- cmp_ex, NULL);
+ cmp_ex, swap_ex);
}
#ifdef CONFIG_MODULES
@@ -48,13 +78,15 @@ void sort_extable(struct exception_table_entry *start,
void trim_init_extable(struct module *m)
{
/*trim the beginning*/
- while (m->num_exentries && within_module_init(m->extable[0].insn, m)) {
+ while (m->num_exentries &&
+ within_module_init(ex_to_insn(&m->extable[0]), m)) {
m->extable++;
m->num_exentries--;
}
/*trim the end*/
while (m->num_exentries &&
- within_module_init(m->extable[m->num_exentries-1].insn, m))
+ within_module_init(ex_to_insn(&m->extable[m->num_exentries - 1]),
+ m))
m->num_exentries--;
}
#endif /* CONFIG_MODULES */
@@ -81,13 +113,13 @@ search_extable(const struct exception_table_entry *first,
* careful, the distance between value and insn
* can be larger than MAX_LONG:
*/
- if (mid->insn < value)
+ if (ex_to_insn(mid) < value)
first = mid + 1;
- else if (mid->insn > value)
+ else if (ex_to_insn(mid) > value)
last = mid - 1;
else
return mid;
- }
- return NULL;
+ }
+ return NULL;
}
#endif
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 resend 2/6] alpha/extable: use generic search and sort routines
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 1/6] extable: add support for relative extables to search and sort routines Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 3/6] s390/extable: " Ard Biesheuvel
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
Replace the arch specific versions of search_extable() and sort_extable()
with calls to the generic ones, which now support relative exception
tables as well.
Acked-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/alpha/include/asm/uaccess.h | 10 ++-
arch/alpha/mm/Makefile | 2 +-
arch/alpha/mm/extable.c | 92 --------------------
3 files changed, 9 insertions(+), 95 deletions(-)
diff --git a/arch/alpha/include/asm/uaccess.h b/arch/alpha/include/asm/uaccess.h
index 9b0d40093c9a..c419b43c461d 100644
--- a/arch/alpha/include/asm/uaccess.h
+++ b/arch/alpha/include/asm/uaccess.h
@@ -483,7 +483,13 @@ struct exception_table_entry
(pc) + (_fixup)->fixup.bits.nextinsn; \
})
-#define ARCH_HAS_SORT_EXTABLE
-#define ARCH_HAS_SEARCH_EXTABLE
+#define ARCH_HAS_RELATIVE_EXTABLE
+
+#define swap_ex_entry_fixup(a, b, tmp, delta) \
+ do { \
+ (a)->fixup.unit = (b)->fixup.unit; \
+ (b)->fixup.unit = (tmp).fixup.unit; \
+ } while (0)
+
#endif /* __ALPHA_UACCESS_H */
diff --git a/arch/alpha/mm/Makefile b/arch/alpha/mm/Makefile
index c993d3f93cf6..5a9807936411 100644
--- a/arch/alpha/mm/Makefile
+++ b/arch/alpha/mm/Makefile
@@ -4,6 +4,6 @@
ccflags-y := -Werror
-obj-y := init.o fault.o extable.o
+obj-y := init.o fault.o
obj-$(CONFIG_DISCONTIGMEM) += numa.o
diff --git a/arch/alpha/mm/extable.c b/arch/alpha/mm/extable.c
deleted file mode 100644
index 813c9b63c0e1..000000000000
--- a/arch/alpha/mm/extable.c
+++ /dev/null
@@ -1,92 +0,0 @@
-/*
- * linux/arch/alpha/mm/extable.c
- */
-
-#include <linux/module.h>
-#include <linux/sort.h>
-#include <asm/uaccess.h>
-
-static inline unsigned long ex_to_addr(const struct exception_table_entry *x)
-{
- return (unsigned long)&x->insn + x->insn;
-}
-
-static void swap_ex(void *a, void *b, int size)
-{
- struct exception_table_entry *ex_a = a, *ex_b = b;
- unsigned long addr_a = ex_to_addr(ex_a), addr_b = ex_to_addr(ex_b);
- unsigned int t = ex_a->fixup.unit;
-
- ex_a->fixup.unit = ex_b->fixup.unit;
- ex_b->fixup.unit = t;
- ex_a->insn = (int)(addr_b - (unsigned long)&ex_a->insn);
- ex_b->insn = (int)(addr_a - (unsigned long)&ex_b->insn);
-}
-
-/*
- * The exception table needs to be sorted so that the binary
- * search that we use to find entries in it works properly.
- * This is used both for the kernel exception table and for
- * the exception tables of modules that get loaded.
- */
-static int cmp_ex(const void *a, const void *b)
-{
- const struct exception_table_entry *x = a, *y = b;
-
- /* avoid overflow */
- if (ex_to_addr(x) > ex_to_addr(y))
- return 1;
- if (ex_to_addr(x) < ex_to_addr(y))
- return -1;
- return 0;
-}
-
-void sort_extable(struct exception_table_entry *start,
- struct exception_table_entry *finish)
-{
- sort(start, finish - start, sizeof(struct exception_table_entry),
- cmp_ex, swap_ex);
-}
-
-#ifdef CONFIG_MODULES
-/*
- * Any entry referring to the module init will be at the beginning or
- * the end.
- */
-void trim_init_extable(struct module *m)
-{
- /*trim the beginning*/
- while (m->num_exentries &&
- within_module_init(ex_to_addr(&m->extable[0]), m)) {
- m->extable++;
- m->num_exentries--;
- }
- /*trim the end*/
- while (m->num_exentries &&
- within_module_init(ex_to_addr(&m->extable[m->num_exentries-1]),
- m))
- m->num_exentries--;
-}
-#endif /* CONFIG_MODULES */
-
-const struct exception_table_entry *
-search_extable(const struct exception_table_entry *first,
- const struct exception_table_entry *last,
- unsigned long value)
-{
- while (first <= last) {
- const struct exception_table_entry *mid;
- unsigned long mid_value;
-
- mid = (last - first) / 2 + first;
- mid_value = ex_to_addr(mid);
- if (mid_value == value)
- return mid;
- else if (mid_value < value)
- first = mid+1;
- else
- last = mid-1;
- }
-
- return NULL;
-}
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 resend 3/6] s390/extable: use generic search and sort routines
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 1/6] extable: add support for relative extables to search and sort routines Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 2/6] alpha/extable: use generic " Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 4/6] x86/extable: " Ard Biesheuvel
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
Replace the arch specific versions of search_extable() and sort_extable()
with calls to the generic ones, which now support relative exception
tables as well.
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/s390/include/asm/uaccess.h | 8 +-
arch/s390/mm/Makefile | 2 +-
arch/s390/mm/extable.c | 85 --------------------
3 files changed, 2 insertions(+), 93 deletions(-)
diff --git a/arch/s390/include/asm/uaccess.h b/arch/s390/include/asm/uaccess.h
index 9dd4cc47ddc7..e0900ddf91dd 100644
--- a/arch/s390/include/asm/uaccess.h
+++ b/arch/s390/include/asm/uaccess.h
@@ -79,18 +79,12 @@ struct exception_table_entry
int insn, fixup;
};
-static inline unsigned long extable_insn(const struct exception_table_entry *x)
-{
- return (unsigned long)&x->insn + x->insn;
-}
-
static inline unsigned long extable_fixup(const struct exception_table_entry *x)
{
return (unsigned long)&x->fixup + x->fixup;
}
-#define ARCH_HAS_SORT_EXTABLE
-#define ARCH_HAS_SEARCH_EXTABLE
+#define ARCH_HAS_RELATIVE_EXTABLE
/**
* __copy_from_user: - Copy a block of data from user space, with less checking.
diff --git a/arch/s390/mm/Makefile b/arch/s390/mm/Makefile
index 839592ca265c..479550dae80e 100644
--- a/arch/s390/mm/Makefile
+++ b/arch/s390/mm/Makefile
@@ -3,7 +3,7 @@
#
obj-y := init.o fault.o extmem.o mmap.o vmem.o pgtable.o maccess.o
-obj-y += page-states.o gup.o extable.o pageattr.o mem_detect.o
+obj-y += page-states.o gup.o pageattr.o mem_detect.o
obj-$(CONFIG_CMM) += cmm.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
diff --git a/arch/s390/mm/extable.c b/arch/s390/mm/extable.c
deleted file mode 100644
index 18c8b819b0aa..000000000000
--- a/arch/s390/mm/extable.c
+++ /dev/null
@@ -1,85 +0,0 @@
-#include <linux/module.h>
-#include <linux/sort.h>
-#include <asm/uaccess.h>
-
-/*
- * Search one exception table for an entry corresponding to the
- * given instruction address, and return the address of the entry,
- * or NULL if none is found.
- * We use a binary search, and thus we assume that the table is
- * already sorted.
- */
-const struct exception_table_entry *
-search_extable(const struct exception_table_entry *first,
- const struct exception_table_entry *last,
- unsigned long value)
-{
- const struct exception_table_entry *mid;
- unsigned long addr;
-
- while (first <= last) {
- mid = ((last - first) >> 1) + first;
- addr = extable_insn(mid);
- if (addr < value)
- first = mid + 1;
- else if (addr > value)
- last = mid - 1;
- else
- return mid;
- }
- return NULL;
-}
-
-/*
- * The exception table needs to be sorted so that the binary
- * search that we use to find entries in it works properly.
- * This is used both for the kernel exception table and for
- * the exception tables of modules that get loaded.
- *
- */
-static int cmp_ex(const void *a, const void *b)
-{
- const struct exception_table_entry *x = a, *y = b;
-
- /* This compare is only valid after normalization. */
- return x->insn - y->insn;
-}
-
-void sort_extable(struct exception_table_entry *start,
- struct exception_table_entry *finish)
-{
- struct exception_table_entry *p;
- int i;
-
- /* Normalize entries to being relative to the start of the section */
- for (p = start, i = 0; p < finish; p++, i += 8) {
- p->insn += i;
- p->fixup += i + 4;
- }
- sort(start, finish - start, sizeof(*start), cmp_ex, NULL);
- /* Denormalize all entries */
- for (p = start, i = 0; p < finish; p++, i += 8) {
- p->insn -= i;
- p->fixup -= i + 4;
- }
-}
-
-#ifdef CONFIG_MODULES
-/*
- * If the exception table is sorted, any referring to the module init
- * will be at the beginning or the end.
- */
-void trim_init_extable(struct module *m)
-{
- /* Trim the beginning */
- while (m->num_exentries &&
- within_module_init(extable_insn(&m->extable[0]), m)) {
- m->extable++;
- m->num_exentries--;
- }
- /* Trim the end */
- while (m->num_exentries &&
- within_module_init(extable_insn(&m->extable[m->num_exentries-1]), m))
- m->num_exentries--;
-}
-#endif /* CONFIG_MODULES */
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 resend 4/6] x86/extable: use generic search and sort routines
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
` (2 preceding siblings ...)
2016-01-27 10:55 ` [PATCH v2 resend 3/6] s390/extable: " Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 5/6] ia64/extable: " Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 6/6] arm64: switch to relative exception tables Ard Biesheuvel
5 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
Replace the arch specific versions of search_extable() and sort_extable()
with calls to the generic ones, which now support relative exception
tables as well.
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/x86/include/asm/uaccess.h | 5 +-
arch/x86/mm/extable.c | 106 +-------------------
2 files changed, 4 insertions(+), 107 deletions(-)
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index a4a30e4b2d34..5664bf927264 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -106,9 +106,8 @@ static inline bool __chk_range_not_ok(unsigned long addr, unsigned long size, un
struct exception_table_entry {
int insn, fixup;
};
-/* This is not the generic standard exception_table_entry format */
-#define ARCH_HAS_SORT_EXTABLE
-#define ARCH_HAS_SEARCH_EXTABLE
+
+#define ARCH_HAS_RELATIVE_EXTABLE
extern int fixup_exception(struct pt_regs *regs);
extern int early_fixup_exception(unsigned long *ip);
diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c
index 903ec1e9c326..0b172b656f43 100644
--- a/arch/x86/mm/extable.c
+++ b/arch/x86/mm/extable.c
@@ -1,14 +1,9 @@
+
#include <linux/module.h>
-#include <linux/spinlock.h>
-#include <linux/sort.h>
+
#include <asm/uaccess.h>
static inline unsigned long
-ex_insn_addr(const struct exception_table_entry *x)
-{
- return (unsigned long)&x->insn + x->insn;
-}
-static inline unsigned long
ex_fixup_addr(const struct exception_table_entry *x)
{
return (unsigned long)&x->fixup + x->fixup;
@@ -70,100 +65,3 @@ int __init early_fixup_exception(unsigned long *ip)
return 0;
}
-
-/*
- * Search one exception table for an entry corresponding to the
- * given instruction address, and return the address of the entry,
- * or NULL if none is found.
- * We use a binary search, and thus we assume that the table is
- * already sorted.
- */
-const struct exception_table_entry *
-search_extable(const struct exception_table_entry *first,
- const struct exception_table_entry *last,
- unsigned long value)
-{
- while (first <= last) {
- const struct exception_table_entry *mid;
- unsigned long addr;
-
- mid = ((last - first) >> 1) + first;
- addr = ex_insn_addr(mid);
- if (addr < value)
- first = mid + 1;
- else if (addr > value)
- last = mid - 1;
- else
- return mid;
- }
- return NULL;
-}
-
-/*
- * The exception table needs to be sorted so that the binary
- * search that we use to find entries in it works properly.
- * This is used both for the kernel exception table and for
- * the exception tables of modules that get loaded.
- *
- */
-static int cmp_ex(const void *a, const void *b)
-{
- const struct exception_table_entry *x = a, *y = b;
-
- /*
- * This value will always end up fittin in an int, because on
- * both i386 and x86-64 the kernel symbol-reachable address
- * space is < 2 GiB.
- *
- * This compare is only valid after normalization.
- */
- return x->insn - y->insn;
-}
-
-void sort_extable(struct exception_table_entry *start,
- struct exception_table_entry *finish)
-{
- struct exception_table_entry *p;
- int i;
-
- /* Convert all entries to being relative to the start of the section */
- i = 0;
- for (p = start; p < finish; p++) {
- p->insn += i;
- i += 4;
- p->fixup += i;
- i += 4;
- }
-
- sort(start, finish - start, sizeof(struct exception_table_entry),
- cmp_ex, NULL);
-
- /* Denormalize all entries */
- i = 0;
- for (p = start; p < finish; p++) {
- p->insn -= i;
- i += 4;
- p->fixup -= i;
- i += 4;
- }
-}
-
-#ifdef CONFIG_MODULES
-/*
- * If the exception table is sorted, any referring to the module init
- * will be at the beginning or the end.
- */
-void trim_init_extable(struct module *m)
-{
- /*trim the beginning*/
- while (m->num_exentries &&
- within_module_init(ex_insn_addr(&m->extable[0]), m)) {
- m->extable++;
- m->num_exentries--;
- }
- /*trim the end*/
- while (m->num_exentries &&
- within_module_init(ex_insn_addr(&m->extable[m->num_exentries-1]), m))
- m->num_exentries--;
-}
-#endif /* CONFIG_MODULES */
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 resend 5/6] ia64/extable: use generic search and sort routines
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
` (3 preceding siblings ...)
2016-01-27 10:55 ` [PATCH v2 resend 4/6] x86/extable: " Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 10:55 ` [PATCH v2 resend 6/6] arm64: switch to relative exception tables Ard Biesheuvel
5 siblings, 0 replies; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
Replace the arch specific versions of search_extable() and sort_extable()
with calls to the generic ones, which now support relative exception
tables as well.
Acked-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/ia64/include/asm/uaccess.h | 8 +-
arch/ia64/mm/extable.c | 97 +-------------------
2 files changed, 4 insertions(+), 101 deletions(-)
diff --git a/arch/ia64/include/asm/uaccess.h b/arch/ia64/include/asm/uaccess.h
index 4f3fb6ccbf21..2189d5ddc1ee 100644
--- a/arch/ia64/include/asm/uaccess.h
+++ b/arch/ia64/include/asm/uaccess.h
@@ -341,13 +341,11 @@ extern unsigned long __strnlen_user (const char __user *, long);
__su_ret; \
})
-/* Generic code can't deal with the location-relative format that we use for compactness. */
-#define ARCH_HAS_SORT_EXTABLE
-#define ARCH_HAS_SEARCH_EXTABLE
+#define ARCH_HAS_RELATIVE_EXTABLE
struct exception_table_entry {
- int addr; /* location-relative address of insn this fixup is for */
- int cont; /* location-relative continuation addr.; if bit 2 is set, r9 is set to 0 */
+ int insn; /* location-relative address of insn this fixup is for */
+ int fixup; /* location-relative continuation addr.; if bit 2 is set, r9 is set to 0 */
};
extern void ia64_handle_exception (struct pt_regs *regs, const struct exception_table_entry *e);
diff --git a/arch/ia64/mm/extable.c b/arch/ia64/mm/extable.c
index c99a41e29fe8..8f70bb2d0c37 100644
--- a/arch/ia64/mm/extable.c
+++ b/arch/ia64/mm/extable.c
@@ -5,107 +5,12 @@
* David Mosberger-Tang <davidm@hpl.hp.com>
*/
-#include <linux/sort.h>
-
#include <asm/uaccess.h>
-#include <linux/module.h>
-
-static int cmp_ex(const void *a, const void *b)
-{
- const struct exception_table_entry *l = a, *r = b;
- u64 lip = (u64) &l->addr + l->addr;
- u64 rip = (u64) &r->addr + r->addr;
-
- /* avoid overflow */
- if (lip > rip)
- return 1;
- if (lip < rip)
- return -1;
- return 0;
-}
-
-static void swap_ex(void *a, void *b, int size)
-{
- struct exception_table_entry *l = a, *r = b, tmp;
- u64 delta = (u64) r - (u64) l;
-
- tmp = *l;
- l->addr = r->addr + delta;
- l->cont = r->cont + delta;
- r->addr = tmp.addr - delta;
- r->cont = tmp.cont - delta;
-}
-
-/*
- * Sort the exception table. It's usually already sorted, but there
- * may be unordered entries due to multiple text sections (such as the
- * .init text section). Note that the exception-table-entries contain
- * location-relative addresses, which requires a bit of care during
- * sorting to avoid overflows in the offset members (e.g., it would
- * not be safe to make a temporary copy of an exception-table entry on
- * the stack, because the stack may be more than 2GB away from the
- * exception-table).
- */
-void sort_extable (struct exception_table_entry *start,
- struct exception_table_entry *finish)
-{
- sort(start, finish - start, sizeof(struct exception_table_entry),
- cmp_ex, swap_ex);
-}
-
-static inline unsigned long ex_to_addr(const struct exception_table_entry *x)
-{
- return (unsigned long)&x->addr + x->addr;
-}
-
-#ifdef CONFIG_MODULES
-/*
- * Any entry referring to the module init will be at the beginning or
- * the end.
- */
-void trim_init_extable(struct module *m)
-{
- /*trim the beginning*/
- while (m->num_exentries &&
- within_module_init(ex_to_addr(&m->extable[0]), m)) {
- m->extable++;
- m->num_exentries--;
- }
- /*trim the end*/
- while (m->num_exentries &&
- within_module_init(ex_to_addr(&m->extable[m->num_exentries-1]),
- m))
- m->num_exentries--;
-}
-#endif /* CONFIG_MODULES */
-
-const struct exception_table_entry *
-search_extable (const struct exception_table_entry *first,
- const struct exception_table_entry *last,
- unsigned long ip)
-{
- const struct exception_table_entry *mid;
- unsigned long mid_ip;
- long diff;
-
- while (first <= last) {
- mid = &first[(last - first)/2];
- mid_ip = (u64) &mid->addr + mid->addr;
- diff = mid_ip - ip;
- if (diff == 0)
- return mid;
- else if (diff < 0)
- first = mid + 1;
- else
- last = mid - 1;
- }
- return NULL;
-}
void
ia64_handle_exception (struct pt_regs *regs, const struct exception_table_entry *e)
{
- long fix = (u64) &e->cont + e->cont;
+ long fix = (u64) &e->fixup + e->fixup;
regs->r8 = -EFAULT;
if (fix & 4)
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 resend 6/6] arm64: switch to relative exception tables
2016-01-27 10:55 [PATCH v2 resend 0/6] generic relative extable support Ard Biesheuvel
` (4 preceding siblings ...)
2016-01-27 10:55 ` [PATCH v2 resend 5/6] ia64/extable: " Ard Biesheuvel
@ 2016-01-27 10:55 ` Ard Biesheuvel
2016-01-27 19:35 ` Andrew Morton
5 siblings, 1 reply; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 10:55 UTC (permalink / raw)
To: akpm, linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon
Cc: deller, Ard Biesheuvel
Instead of using absolute addresses for both the exception location
and the fixup, use offsets relative to the exception table entry values.
Not only does this cut the size of the exception table in half, it is
also a prerequisite for KASLR, since absolute exception table entries
are subject to dynamic relocation, which is incompatible with the sorting
of the exception table that occurs at build time.
This patch also introduces the _ASM_EXTABLE preprocessor macro (which
exists on x86 as well) and its _asm_extable assembly counterpart, as
shorthands to emit exception table entries.
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/include/asm/assembler.h | 15 +++++++---
arch/arm64/include/asm/futex.h | 12 +++-----
arch/arm64/include/asm/uaccess.h | 30 +++++++++++---------
arch/arm64/include/asm/word-at-a-time.h | 7 ++---
arch/arm64/kernel/armv8_deprecated.c | 7 ++---
arch/arm64/mm/extable.c | 2 +-
scripts/sortextable.c | 2 +-
7 files changed, 38 insertions(+), 37 deletions(-)
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index bb7b72734c24..d8bfcc1ce923 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -94,12 +94,19 @@
dmb \opt
.endm
+/*
+ * Emit an entry into the exception table
+ */
+ .macro _asm_extable, from, to
+ .pushsection __ex_table, "a"
+ .align 3
+ .long (\from - .), (\to - .)
+ .popsection
+ .endm
+
#define USER(l, x...) \
9999: x; \
- .section __ex_table,"a"; \
- .align 3; \
- .quad 9999b,l; \
- .previous
+ _asm_extable 9999b, l
/*
* Register aliases.
diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h
index 007a69fc4f40..1ab15a3b5a0e 100644
--- a/arch/arm64/include/asm/futex.h
+++ b/arch/arm64/include/asm/futex.h
@@ -42,10 +42,8 @@
"4: mov %w0, %w5\n" \
" b 3b\n" \
" .popsection\n" \
-" .pushsection __ex_table,\"a\"\n" \
-" .align 3\n" \
-" .quad 1b, 4b, 2b, 4b\n" \
-" .popsection\n" \
+ _ASM_EXTABLE(1b, 4b) \
+ _ASM_EXTABLE(2b, 4b) \
ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN) \
: "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \
@@ -133,10 +131,8 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
"4: mov %w0, %w6\n"
" b 3b\n"
" .popsection\n"
-" .pushsection __ex_table,\"a\"\n"
-" .align 3\n"
-" .quad 1b, 4b, 2b, 4b\n"
-" .popsection\n"
+ _ASM_EXTABLE(1b, 4b)
+ _ASM_EXTABLE(2b, 4b)
: "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp)
: "r" (oldval), "r" (newval), "Ir" (-EFAULT)
: "memory");
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index b2ede967fe7d..dc11577fab7e 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -36,11 +36,11 @@
#define VERIFY_WRITE 1
/*
- * The exception table consists of pairs of addresses: the first is the
- * address of an instruction that is allowed to fault, and the second is
- * the address at which the program should continue. No registers are
- * modified, so it is entirely up to the continuation code to figure out
- * what to do.
+ * The exception table consists of pairs of relative offsets: the first
+ * is the relative offset to an instruction that is allowed to fault,
+ * and the second is the relative offset at which the program should
+ * continue. No registers are modified, so it is entirely up to the
+ * continuation code to figure out what to do.
*
* All the routines below use bits of fixup code that are out of line
* with the main instruction path. This means when everything is well,
@@ -50,9 +50,11 @@
struct exception_table_entry
{
- unsigned long insn, fixup;
+ int insn, fixup;
};
+#define ARCH_HAS_RELATIVE_EXTABLE
+
extern int fixup_exception(struct pt_regs *regs);
#define KERNEL_DS (-1UL)
@@ -105,6 +107,12 @@ static inline void set_fs(mm_segment_t fs)
#define access_ok(type, addr, size) __range_ok(addr, size)
#define user_addr_max get_fs
+#define _ASM_EXTABLE(from, to) \
+ " .pushsection __ex_table, \"a\"\n" \
+ " .align 3\n" \
+ " .long (" #from " - .), (" #to " - .)\n" \
+ " .popsection\n"
+
/*
* The "__xxx" versions of the user access functions do not verify the address
* space - it must have been done previously with a separate "access_ok()"
@@ -123,10 +131,7 @@ static inline void set_fs(mm_segment_t fs)
" mov %1, #0\n" \
" b 2b\n" \
" .previous\n" \
- " .section __ex_table,\"a\"\n" \
- " .align 3\n" \
- " .quad 1b, 3b\n" \
- " .previous" \
+ _ASM_EXTABLE(1b, 3b) \
: "+r" (err), "=&r" (x) \
: "r" (addr), "i" (-EFAULT))
@@ -190,10 +195,7 @@ do { \
"3: mov %w0, %3\n" \
" b 2b\n" \
" .previous\n" \
- " .section __ex_table,\"a\"\n" \
- " .align 3\n" \
- " .quad 1b, 3b\n" \
- " .previous" \
+ _ASM_EXTABLE(1b, 3b) \
: "+r" (err) \
: "r" (x), "r" (addr), "i" (-EFAULT))
diff --git a/arch/arm64/include/asm/word-at-a-time.h b/arch/arm64/include/asm/word-at-a-time.h
index aab5bf09e9d9..2b79b8a89457 100644
--- a/arch/arm64/include/asm/word-at-a-time.h
+++ b/arch/arm64/include/asm/word-at-a-time.h
@@ -16,6 +16,8 @@
#ifndef __ASM_WORD_AT_A_TIME_H
#define __ASM_WORD_AT_A_TIME_H
+#include <asm/uaccess.h>
+
#ifndef __AARCH64EB__
#include <linux/kernel.h>
@@ -81,10 +83,7 @@ static inline unsigned long load_unaligned_zeropad(const void *addr)
#endif
" b 2b\n"
" .popsection\n"
- " .pushsection __ex_table,\"a\"\n"
- " .align 3\n"
- " .quad 1b, 3b\n"
- " .popsection"
+ _ASM_EXTABLE(1b, 3b)
: "=&r" (ret), "=&r" (offset)
: "r" (addr), "Q" (*(unsigned long *)addr));
diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c
index 3e01207917b1..c37202c0c838 100644
--- a/arch/arm64/kernel/armv8_deprecated.c
+++ b/arch/arm64/kernel/armv8_deprecated.c
@@ -297,11 +297,8 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table)
"4: mov %w0, %w5\n" \
" b 3b\n" \
" .popsection" \
- " .pushsection __ex_table,\"a\"\n" \
- " .align 3\n" \
- " .quad 0b, 4b\n" \
- " .quad 1b, 4b\n" \
- " .popsection\n" \
+ _ASM_EXTABLE(0b, 4b) \
+ _ASM_EXTABLE(1b, 4b) \
ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \
CONFIG_ARM64_PAN) \
: "=&r" (res), "+r" (data), "=&r" (temp) \
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 79444279ba8c..81acd4706878 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -11,7 +11,7 @@ int fixup_exception(struct pt_regs *regs)
fixup = search_exception_tables(instruction_pointer(regs));
if (fixup)
- regs->pc = fixup->fixup;
+ regs->pc = (unsigned long)&fixup->fixup + fixup->fixup;
return fixup != NULL;
}
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index c2423d913b46..af247c70fb66 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -282,12 +282,12 @@ do_file(char const *const fname)
case EM_386:
case EM_X86_64:
case EM_S390:
+ case EM_AARCH64:
custom_sort = sort_relative_table;
break;
case EM_ARCOMPACT:
case EM_ARCV2:
case EM_ARM:
- case EM_AARCH64:
case EM_MICROBLAZE:
case EM_MIPS:
case EM_XTENSA:
--
2.5.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 resend 6/6] arm64: switch to relative exception tables
2016-01-27 10:55 ` [PATCH v2 resend 6/6] arm64: switch to relative exception tables Ard Biesheuvel
@ 2016-01-27 19:35 ` Andrew Morton
2016-01-27 19:39 ` Ard Biesheuvel
0 siblings, 1 reply; 10+ messages in thread
From: Andrew Morton @ 2016-01-27 19:35 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-alpha, linux-kernel, linux-arm-kernel, linux-ia64,
linux-s390, x86, hpa, heiko.carstens, rth, tony.luck, will.deacon,
deller
On Wed, 27 Jan 2016 11:55:23 +0100 Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> Instead of using absolute addresses for both the exception location
> and the fixup, use offsets relative to the exception table entry values.
> Not only does this cut the size of the exception table in half, it is
> also a prerequisite for KASLR, since absolute exception table entries
> are subject to dynamic relocation, which is incompatible with the sorting
> of the exception table that occurs at build time.
>
> This patch also introduces the _ASM_EXTABLE preprocessor macro (which
> exists on x86 as well) and its _asm_extable assembly counterpart, as
> shorthands to emit exception table entries.
checkpatch speaketh truth:
ERROR: #define of 'ARCH_HAS_RELATIVE_EXTABLE' is wrong - use Kconfig variables or standard guards instead
#113: FILE: arch/arm64/include/asm/uaccess.h:56:
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 resend 6/6] arm64: switch to relative exception tables
2016-01-27 19:35 ` Andrew Morton
@ 2016-01-27 19:39 ` Ard Biesheuvel
2016-01-27 19:46 ` Andrew Morton
0 siblings, 1 reply; 10+ messages in thread
From: Ard Biesheuvel @ 2016-01-27 19:39 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-alpha, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org,
linux-s390, x86@kernel.org, hpa@zytor.com, Heiko Carstens,
Richard Henderson, Tony Luck, Will Deacon, Helge Deller
On 27 January 2016 at 20:35, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 27 Jan 2016 11:55:23 +0100 Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>
>> Instead of using absolute addresses for both the exception location
>> and the fixup, use offsets relative to the exception table entry values.
>> Not only does this cut the size of the exception table in half, it is
>> also a prerequisite for KASLR, since absolute exception table entries
>> are subject to dynamic relocation, which is incompatible with the sorting
>> of the exception table that occurs at build time.
>>
>> This patch also introduces the _ASM_EXTABLE preprocessor macro (which
>> exists on x86 as well) and its _asm_extable assembly counterpart, as
>> shorthands to emit exception table entries.
>
> checkpatch speaketh truth:
>
> ERROR: #define of 'ARCH_HAS_RELATIVE_EXTABLE' is wrong - use Kconfig variables or standard guards instead
> #113: FILE: arch/arm64/include/asm/uaccess.h:56:
That may be true, but it extends an established pattern of #defines in
the various uaccess.h versions, i.e., ARCH_HAS_SORT_EXTABLE, and
ARCH_HAS_SEARCH_EXTABLE
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 resend 6/6] arm64: switch to relative exception tables
2016-01-27 19:39 ` Ard Biesheuvel
@ 2016-01-27 19:46 ` Andrew Morton
0 siblings, 0 replies; 10+ messages in thread
From: Andrew Morton @ 2016-01-27 19:46 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-alpha, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org,
linux-s390, x86@kernel.org, hpa@zytor.com, Heiko Carstens,
Richard Henderson, Tony Luck, Will Deacon, Helge Deller
On Wed, 27 Jan 2016 20:39:23 +0100 Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 27 January 2016 at 20:35, Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Wed, 27 Jan 2016 11:55:23 +0100 Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> >> Instead of using absolute addresses for both the exception location
> >> and the fixup, use offsets relative to the exception table entry values.
> >> Not only does this cut the size of the exception table in half, it is
> >> also a prerequisite for KASLR, since absolute exception table entries
> >> are subject to dynamic relocation, which is incompatible with the sorting
> >> of the exception table that occurs at build time.
> >>
> >> This patch also introduces the _ASM_EXTABLE preprocessor macro (which
> >> exists on x86 as well) and its _asm_extable assembly counterpart, as
> >> shorthands to emit exception table entries.
> >
> > checkpatch speaketh truth:
> >
> > ERROR: #define of 'ARCH_HAS_RELATIVE_EXTABLE' is wrong - use Kconfig variables or standard guards instead
> > #113: FILE: arch/arm64/include/asm/uaccess.h:56:
>
> That may be true, but it extends an established pattern of #defines in
> the various uaccess.h versions, i.e., ARCH_HAS_SORT_EXTABLE, and
> ARCH_HAS_SEARCH_EXTABLE
True. But now we have two things to fix, not one ;)
Whatever. Later.
^ permalink raw reply [flat|nested] 10+ messages in thread