* [PATCH v2 01/70] xen/domain: Improve pirq handling
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
` (70 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
free_pirq_struct() has no external users, so shouldn't be exposed. Making it
static necessistates moving the function as domain_destroy() uses it.
Rework pirq_get_info() to have easier-to-follow logic. The one functional
change is to the insertion failure path; we should not be using a full
call_rcu() chain to free an otherwise local structure we failed to insert into
the radix tree to begin with.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v2:
* Drop likely/unlikey
---
xen/common/domain.c | 62 ++++++++++++++++++++++++++----------------------
xen/include/xen/domain.h | 1 -
2 files changed, 33 insertions(+), 30 deletions(-)
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 2048ebad86ff..571ef4fe615e 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -365,6 +365,39 @@ static int __init parse_extra_guest_irqs(const char *s)
}
custom_param("extra_guest_irqs", parse_extra_guest_irqs);
+static void _free_pirq_struct(struct rcu_head *head)
+{
+ xfree(container_of(head, struct pirq, rcu_head));
+}
+
+static void free_pirq_struct(void *ptr)
+{
+ struct pirq *pirq = ptr;
+
+ call_rcu(&pirq->rcu_head, _free_pirq_struct);
+}
+
+struct pirq *pirq_get_info(struct domain *d, int pirq)
+{
+ struct pirq *info = pirq_info(d, pirq);
+
+ if ( info )
+ return info;
+
+ info = alloc_pirq_struct(d);
+ if ( !info )
+ return NULL;
+
+ info->pirq = pirq;
+ if ( radix_tree_insert(&d->pirq_tree, pirq, info) == 0 )
+ return info; /* Success. */
+
+ /* Don't use call_rcu() to free a struct we failed to insert. */
+ _free_pirq_struct(&info->rcu_head);
+
+ return NULL;
+}
+
/*
* Release resources held by a domain. There may or may not be live
* references to the domain, and it may or may not be fully constructed.
@@ -1780,35 +1813,6 @@ long do_vm_assist(unsigned int cmd, unsigned int type)
}
#endif
-struct pirq *pirq_get_info(struct domain *d, int pirq)
-{
- struct pirq *info = pirq_info(d, pirq);
-
- if ( !info && (info = alloc_pirq_struct(d)) != NULL )
- {
- info->pirq = pirq;
- if ( radix_tree_insert(&d->pirq_tree, pirq, info) )
- {
- free_pirq_struct(info);
- info = NULL;
- }
- }
-
- return info;
-}
-
-static void _free_pirq_struct(struct rcu_head *head)
-{
- xfree(container_of(head, struct pirq, rcu_head));
-}
-
-void free_pirq_struct(void *ptr)
-{
- struct pirq *pirq = ptr;
-
- call_rcu(&pirq->rcu_head, _free_pirq_struct);
-}
-
struct migrate_info {
long (*func)(void *data);
void *data;
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 160c8dbdab33..b4d202fda9fd 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -44,7 +44,6 @@ void free_vcpu_struct(struct vcpu *v);
#ifndef alloc_pirq_struct
struct pirq *alloc_pirq_struct(struct domain *);
#endif
-void free_pirq_struct(void *);
/*
* Initialise/destroy arch-specific details of a VCPU.
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 01/70] xen/domain: Improve pirq handling Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 13:13 ` Bertrand Marquis
2022-02-14 13:17 ` Julien Grall
2022-02-14 12:50 ` [PATCH v2 03/70] xen/xsm: Move {do,compat}_flask_op() declarations into a header Andrew Cooper
` (69 subsequent siblings)
71 siblings, 2 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel
Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu,
Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Bertrand Marquis
There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
loop like this are problematic for performance, especially with Spectre v2
protections, which is why extern inline is used commonly by libraries.
Both ARM callers pass in NULL for the swap function, and while this might seem
like an attractive option at first, it causes generic_swap() to be used, which
forced a byte-wise copy. Provide real swap functions so the compiler can
optimise properly, which is very important for ARM downstreams where
milliseconds until the system is up matters.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
v2:
* Adjust commit message
---
xen/arch/arm/bootfdt.c | 9 +++++-
xen/arch/arm/io.c | 9 +++++-
xen/include/xen/sort.h | 55 +++++++++++++++++++++++++++++++++-
xen/lib/sort.c | 80 ++------------------------------------------------
4 files changed, 72 insertions(+), 81 deletions(-)
diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index afaa0e249b71..e318ef960386 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -448,6 +448,13 @@ static int __init cmp_memory_node(const void *key, const void *elem)
return 0;
}
+static void __init swap_memory_node(void *_a, void *_b, size_t size)
+{
+ struct membank *a = _a, *b = _b;
+
+ SWAP(*a, *b);
+}
+
/**
* boot_fdt_info - initialize bootinfo from a DTB
* @fdt: flattened device tree binary
@@ -472,7 +479,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
* the banks sorted in ascending order. So sort them through.
*/
sort(bootinfo.mem.bank, bootinfo.mem.nr_banks, sizeof(struct membank),
- cmp_memory_node, NULL);
+ cmp_memory_node, swap_memory_node);
early_print_info();
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index 729287e37c59..1a066f9ae502 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -80,6 +80,13 @@ static int cmp_mmio_handler(const void *key, const void *elem)
return 0;
}
+static void swap_mmio_handler(void *_a, void *_b, size_t size)
+{
+ struct mmio_handler *a = _a, *b = _b;
+
+ SWAP(*a, *b);
+}
+
static const struct mmio_handler *find_mmio_handler(struct domain *d,
paddr_t gpa)
{
@@ -170,7 +177,7 @@ void register_mmio_handler(struct domain *d,
/* Sort mmio handlers in ascending order based on base address */
sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
- cmp_mmio_handler, NULL);
+ cmp_mmio_handler, swap_mmio_handler);
write_unlock(&vmmio->lock);
}
diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
index a403652948e7..01479ea44606 100644
--- a/xen/include/xen/sort.h
+++ b/xen/include/xen/sort.h
@@ -3,8 +3,61 @@
#include <xen/types.h>
+/*
+ * sort - sort an array of elements
+ * @base: pointer to data to sort
+ * @num: number of elements
+ * @size: size of each element
+ * @cmp: pointer to comparison function
+ * @swap: pointer to swap function or NULL
+ *
+ * This function does a heapsort on the given array. You may provide a
+ * swap function optimized to your element type.
+ *
+ * Sorting time is O(n log n) both on average and worst-case. While
+ * qsort is about 20% faster on average, it suffers from exploitable
+ * O(n*n) worst-case behavior and extra memory requirements that make
+ * it less suitable for kernel use.
+ */
+#ifndef SORT_IMPLEMENTATION
+extern gnu_inline
+#endif
void sort(void *base, size_t num, size_t size,
int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t));
+ void (*swap)(void *, void *, size_t))
+{
+ /* pre-scale counters for performance */
+ size_t i = (num / 2) * size, n = num * size, c, r;
+
+ /* heapify */
+ while ( i > 0 )
+ {
+ for ( r = i -= size; r * 2 + size < n; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+
+ /* sort */
+ for ( i = n; i > 0; )
+ {
+ i -= size;
+ swap(base, base + i, size);
+ for ( r = 0; r * 2 + size < i; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+}
#endif /* __XEN_SORT_H__ */
diff --git a/xen/lib/sort.c b/xen/lib/sort.c
index 35ce0d7abdec..b7e78cc0e8d2 100644
--- a/xen/lib/sort.c
+++ b/xen/lib/sort.c
@@ -4,81 +4,5 @@
* Jan 23 2005 Matt Mackall <mpm@selenic.com>
*/
-#include <xen/types.h>
-
-static void u32_swap(void *a, void *b, size_t size)
-{
- uint32_t t = *(uint32_t *)a;
-
- *(uint32_t *)a = *(uint32_t *)b;
- *(uint32_t *)b = t;
-}
-
-static void generic_swap(void *a, void *b, size_t size)
-{
- char t;
-
- do {
- t = *(char *)a;
- *(char *)a++ = *(char *)b;
- *(char *)b++ = t;
- } while ( --size > 0 );
-}
-
-/*
- * sort - sort an array of elements
- * @base: pointer to data to sort
- * @num: number of elements
- * @size: size of each element
- * @cmp: pointer to comparison function
- * @swap: pointer to swap function or NULL
- *
- * This function does a heapsort on the given array. You may provide a
- * swap function optimized to your element type.
- *
- * Sorting time is O(n log n) both on average and worst-case. While
- * qsort is about 20% faster on average, it suffers from exploitable
- * O(n*n) worst-case behavior and extra memory requirements that make
- * it less suitable for kernel use.
- */
-
-void sort(void *base, size_t num, size_t size,
- int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t size))
-{
- /* pre-scale counters for performance */
- size_t i = (num / 2) * size, n = num * size, c, r;
-
- if ( !swap )
- swap = (size == 4 ? u32_swap : generic_swap);
-
- /* heapify */
- while ( i > 0 )
- {
- for ( r = i -= size; r * 2 + size < n; r = c )
- {
- c = r * 2 + size;
- if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-
- /* sort */
- for ( i = n; i > 0; )
- {
- i -= size;
- swap(base, base + i, size);
- for ( r = 0; r * 2 + size < i; r = c )
- {
- c = r * 2 + size;
- if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-}
+#define SORT_IMPLEMENTATION
+#include <xen/sort.h>
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-14 12:50 ` [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
@ 2022-02-14 13:13 ` Bertrand Marquis
2022-02-14 18:30 ` Andrew Cooper
2022-02-14 13:17 ` Julien Grall
1 sibling, 1 reply; 123+ messages in thread
From: Bertrand Marquis @ 2022-02-14 13:13 UTC (permalink / raw)
To: Andrew Cooper
Cc: Xen-devel, Jan Beulich, Roger Pau Monné, Wei Liu,
Stefano Stabellini, Julien Grall, Volodymyr Babchuk
Hi Andrew,
> On 14 Feb 2022, at 12:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
> loop like this are problematic for performance, especially with Spectre v2
> protections, which is why extern inline is used commonly by libraries.
>
> Both ARM callers pass in NULL for the swap function, and while this might seem
> like an attractive option at first, it causes generic_swap() to be used, which
> forced a byte-wise copy. Provide real swap functions so the compiler can
> optimise properly, which is very important for ARM downstreams where
> milliseconds until the system is up matters.
>
> No functional change.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Just one comment fix after, with it fixed for the arm part:
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
>
> v2:
> * Adjust commit message
> ---
> xen/arch/arm/bootfdt.c | 9 +++++-
> xen/arch/arm/io.c | 9 +++++-
> xen/include/xen/sort.h | 55 +++++++++++++++++++++++++++++++++-
> xen/lib/sort.c | 80 ++------------------------------------------------
> 4 files changed, 72 insertions(+), 81 deletions(-)
>
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index afaa0e249b71..e318ef960386 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -448,6 +448,13 @@ static int __init cmp_memory_node(const void *key, const void *elem)
> return 0;
> }
>
> +static void __init swap_memory_node(void *_a, void *_b, size_t size)
> +{
> + struct membank *a = _a, *b = _b;
> +
> + SWAP(*a, *b);
> +}
> +
> /**
> * boot_fdt_info - initialize bootinfo from a DTB
> * @fdt: flattened device tree binary
> @@ -472,7 +479,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
> * the banks sorted in ascending order. So sort them through.
> */
> sort(bootinfo.mem.bank, bootinfo.mem.nr_banks, sizeof(struct membank),
> - cmp_memory_node, NULL);
> + cmp_memory_node, swap_memory_node);
>
> early_print_info();
>
> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> index 729287e37c59..1a066f9ae502 100644
> --- a/xen/arch/arm/io.c
> +++ b/xen/arch/arm/io.c
> @@ -80,6 +80,13 @@ static int cmp_mmio_handler(const void *key, const void *elem)
> return 0;
> }
>
> +static void swap_mmio_handler(void *_a, void *_b, size_t size)
> +{
> + struct mmio_handler *a = _a, *b = _b;
> +
> + SWAP(*a, *b);
> +}
> +
> static const struct mmio_handler *find_mmio_handler(struct domain *d,
> paddr_t gpa)
> {
> @@ -170,7 +177,7 @@ void register_mmio_handler(struct domain *d,
>
> /* Sort mmio handlers in ascending order based on base address */
> sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
> - cmp_mmio_handler, NULL);
> + cmp_mmio_handler, swap_mmio_handler);
>
> write_unlock(&vmmio->lock);
> }
> diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
> index a403652948e7..01479ea44606 100644
> --- a/xen/include/xen/sort.h
> +++ b/xen/include/xen/sort.h
> @@ -3,8 +3,61 @@
>
> #include <xen/types.h>
>
> +/*
> + * sort - sort an array of elements
> + * @base: pointer to data to sort
> + * @num: number of elements
> + * @size: size of each element
> + * @cmp: pointer to comparison function
> + * @swap: pointer to swap function or NULL
The function is not accepting anymore to have NULL as parameter.
The comment should be fixed here.
Bertrand
> + *
> + * This function does a heapsort on the given array. You may provide a
> + * swap function optimized to your element type.
> + *
> + * Sorting time is O(n log n) both on average and worst-case. While
> + * qsort is about 20% faster on average, it suffers from exploitable
> + * O(n*n) worst-case behavior and extra memory requirements that make
> + * it less suitable for kernel use.
> + */
> +#ifndef SORT_IMPLEMENTATION
> +extern gnu_inline
> +#endif
> void sort(void *base, size_t num, size_t size,
> int (*cmp)(const void *, const void *),
> - void (*swap)(void *, void *, size_t));
> + void (*swap)(void *, void *, size_t))
> +{
> + /* pre-scale counters for performance */
> + size_t i = (num / 2) * size, n = num * size, c, r;
> +
> + /* heapify */
> + while ( i > 0 )
> + {
> + for ( r = i -= size; r * 2 + size < n; r = c )
> + {
> + c = r * 2 + size;
> + if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
> + c += size;
> + if ( cmp(base + r, base + c) >= 0 )
> + break;
> + swap(base + r, base + c, size);
> + }
> + }
> +
> + /* sort */
> + for ( i = n; i > 0; )
> + {
> + i -= size;
> + swap(base, base + i, size);
> + for ( r = 0; r * 2 + size < i; r = c )
> + {
> + c = r * 2 + size;
> + if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
> + c += size;
> + if ( cmp(base + r, base + c) >= 0 )
> + break;
> + swap(base + r, base + c, size);
> + }
> + }
> +}
>
> #endif /* __XEN_SORT_H__ */
> diff --git a/xen/lib/sort.c b/xen/lib/sort.c
> index 35ce0d7abdec..b7e78cc0e8d2 100644
> --- a/xen/lib/sort.c
> +++ b/xen/lib/sort.c
> @@ -4,81 +4,5 @@
> * Jan 23 2005 Matt Mackall <mpm@selenic.com>
> */
>
> -#include <xen/types.h>
> -
> -static void u32_swap(void *a, void *b, size_t size)
> -{
> - uint32_t t = *(uint32_t *)a;
> -
> - *(uint32_t *)a = *(uint32_t *)b;
> - *(uint32_t *)b = t;
> -}
> -
> -static void generic_swap(void *a, void *b, size_t size)
> -{
> - char t;
> -
> - do {
> - t = *(char *)a;
> - *(char *)a++ = *(char *)b;
> - *(char *)b++ = t;
> - } while ( --size > 0 );
> -}
> -
> -/*
> - * sort - sort an array of elements
> - * @base: pointer to data to sort
> - * @num: number of elements
> - * @size: size of each element
> - * @cmp: pointer to comparison function
> - * @swap: pointer to swap function or NULL
> - *
> - * This function does a heapsort on the given array. You may provide a
> - * swap function optimized to your element type.
> - *
> - * Sorting time is O(n log n) both on average and worst-case. While
> - * qsort is about 20% faster on average, it suffers from exploitable
> - * O(n*n) worst-case behavior and extra memory requirements that make
> - * it less suitable for kernel use.
> - */
> -
> -void sort(void *base, size_t num, size_t size,
> - int (*cmp)(const void *, const void *),
> - void (*swap)(void *, void *, size_t size))
> -{
> - /* pre-scale counters for performance */
> - size_t i = (num / 2) * size, n = num * size, c, r;
> -
> - if ( !swap )
> - swap = (size == 4 ? u32_swap : generic_swap);
> -
> - /* heapify */
> - while ( i > 0 )
> - {
> - for ( r = i -= size; r * 2 + size < n; r = c )
> - {
> - c = r * 2 + size;
> - if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
> - c += size;
> - if ( cmp(base + r, base + c) >= 0 )
> - break;
> - swap(base + r, base + c, size);
> - }
> - }
> -
> - /* sort */
> - for ( i = n; i > 0; )
> - {
> - i -= size;
> - swap(base, base + i, size);
> - for ( r = 0; r * 2 + size < i; r = c )
> - {
> - c = r * 2 + size;
> - if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
> - c += size;
> - if ( cmp(base + r, base + c) >= 0 )
> - break;
> - swap(base + r, base + c, size);
> - }
> - }
> -}
> +#define SORT_IMPLEMENTATION
> +#include <xen/sort.h>
> --
> 2.11.0
>
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-14 13:13 ` Bertrand Marquis
@ 2022-02-14 18:30 ` Andrew Cooper
0 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 18:30 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Xen-devel, Jan Beulich, Roger Pau Monne, Wei Liu,
Stefano Stabellini, Julien Grall, Volodymyr Babchuk
On 14/02/2022 13:13, Bertrand Marquis wrote:
> Hi Andrew,
>
>> On 14 Feb 2022, at 12:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>
>> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
>> loop like this are problematic for performance, especially with Spectre v2
>> protections, which is why extern inline is used commonly by libraries.
>>
>> Both ARM callers pass in NULL for the swap function, and while this might seem
>> like an attractive option at first, it causes generic_swap() to be used, which
>> forced a byte-wise copy. Provide real swap functions so the compiler can
>> optimise properly, which is very important for ARM downstreams where
>> milliseconds until the system is up matters.
>>
>> No functional change.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> Just one comment fix after, with it fixed for the arm part:
>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Thanks.
>> diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
>> index a403652948e7..01479ea44606 100644
>> --- a/xen/include/xen/sort.h
>> +++ b/xen/include/xen/sort.h
>> @@ -3,8 +3,61 @@
>>
>> #include <xen/types.h>
>>
>> +/*
>> + * sort - sort an array of elements
>> + * @base: pointer to data to sort
>> + * @num: number of elements
>> + * @size: size of each element
>> + * @cmp: pointer to comparison function
>> + * @swap: pointer to swap function or NULL
> The function is not accepting anymore to have NULL as parameter.
> The comment should be fixed here.
Will fix.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-14 12:50 ` [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
2022-02-14 13:13 ` Bertrand Marquis
@ 2022-02-14 13:17 ` Julien Grall
2022-02-16 3:46 ` Stefano Stabellini
1 sibling, 1 reply; 123+ messages in thread
From: Julien Grall @ 2022-02-14 13:17 UTC (permalink / raw)
To: Andrew Cooper, Xen-devel
Cc: Jan Beulich, Roger Pau Monné, Wei Liu, Stefano Stabellini,
Volodymyr Babchuk, Bertrand Marquis
Hi,
On 14/02/2022 12:50, Andrew Cooper wrote:
> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
> loop like this are problematic for performance, especially with Spectre v2
> protections, which is why extern inline is used commonly by libraries.
>
> Both ARM callers pass in NULL for the swap function, and while this might seem
> like an attractive option at first, it causes generic_swap() to be used, which
> forced a byte-wise copy. Provide real swap functions so the compiler can
> optimise properly, which is very important for ARM downstreams where
> milliseconds until the system is up matters.
Did you actually benchmark it? Both those lists will have < 128 elements
in them. So I would be extremely surprised if you save more than a few
hundreds microseconds with this approach.
So, my opinion on this approach hasn't changed. On v1, we discussed an
approach that would suit both Stefano and I. Jan seemed to confirm that
would also suit x86.
Therefore, for this approach:
Nacked-by: Julien Grall <jgrall@amazon.com>
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-14 13:17 ` Julien Grall
@ 2022-02-16 3:46 ` Stefano Stabellini
2022-02-16 9:29 ` Bertrand Marquis
2022-02-16 10:44 ` Andrew Cooper
0 siblings, 2 replies; 123+ messages in thread
From: Stefano Stabellini @ 2022-02-16 3:46 UTC (permalink / raw)
To: Julien Grall
Cc: Andrew Cooper, Xen-devel, Jan Beulich, Roger Pau Monné,
Wei Liu, Stefano Stabellini, Volodymyr Babchuk, Bertrand Marquis
On Mon, 14 Feb 2022, Julien Grall wrote:
> On 14/02/2022 12:50, Andrew Cooper wrote:
> > There are exactly 3 callers of sort() in the hypervisor. Callbacks in a
> > tight
> > loop like this are problematic for performance, especially with Spectre v2
> > protections, which is why extern inline is used commonly by libraries.
> >
> > Both ARM callers pass in NULL for the swap function, and while this might
> > seem
> > like an attractive option at first, it causes generic_swap() to be used,
> > which
> > forced a byte-wise copy. Provide real swap functions so the compiler can
> > optimise properly, which is very important for ARM downstreams where
> > milliseconds until the system is up matters.
>
> Did you actually benchmark it? Both those lists will have < 128 elements in
> them. So I would be extremely surprised if you save more than a few hundreds
> microseconds with this approach.
>
> So, my opinion on this approach hasn't changed. On v1, we discussed an
> approach that would suit both Stefano and I. Jan seemed to confirm that would
> also suit x86.
This patch series has become 70 patches and for the sake of helping
Andrew move forward in the quickest and most painless way possible, I
append the following using generic_swap as static inline.
Julien, Bertrand, is that acceptable to you?
Andrew, I know this is not your favorite approach but you have quite a
lot of changes to handle -- probably not worth focussing on one detail
which is pretty minor?
---
xen/sort: Switch to an extern inline implementation
There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
loop like this are problematic for performance, especially with Spectre v2
protections, which is why extern inline is used commonly by libraries.
Make generic_swap() a static inline and used it at the two ARM call
sites.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/arm/bootfdt.c | 2 +-
xen/arch/arm/io.c | 2 +-
xen/include/xen/sort.h | 67 ++++++++++++++++++++++++++++++++++-
xen/lib/sort.c | 80 ++----------------------------------------
4 files changed, 70 insertions(+), 81 deletions(-)
diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index afaa0e249b..0d62945d56 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -472,7 +472,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
* the banks sorted in ascending order. So sort them through.
*/
sort(bootinfo.mem.bank, bootinfo.mem.nr_banks, sizeof(struct membank),
- cmp_memory_node, NULL);
+ cmp_memory_node, generic_swap);
early_print_info();
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index 729287e37c..1f35aaeea6 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -170,7 +170,7 @@ void register_mmio_handler(struct domain *d,
/* Sort mmio handlers in ascending order based on base address */
sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
- cmp_mmio_handler, NULL);
+ cmp_mmio_handler, generic_swap);
write_unlock(&vmmio->lock);
}
diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
index a403652948..f6065eda58 100644
--- a/xen/include/xen/sort.h
+++ b/xen/include/xen/sort.h
@@ -3,8 +3,73 @@
#include <xen/types.h>
+extern gnu_inline
+void generic_swap(void *a, void *b, size_t size)
+{
+ char t;
+
+ do {
+ t = *(char *)a;
+ *(char *)a++ = *(char *)b;
+ *(char *)b++ = t;
+ } while ( --size > 0 );
+}
+
+/*
+ * sort - sort an array of elements
+ * @base: pointer to data to sort
+ * @num: number of elements
+ * @size: size of each element
+ * @cmp: pointer to comparison function
+ * @swap: pointer to swap function or NULL
+ *
+ * This function does a heapsort on the given array. You may provide a
+ * swap function optimized to your element type.
+ *
+ * Sorting time is O(n log n) both on average and worst-case. While
+ * qsort is about 20% faster on average, it suffers from exploitable
+ * O(n*n) worst-case behavior and extra memory requirements that make
+ * it less suitable for kernel use.
+ */
+#ifndef SORT_IMPLEMENTATION
+extern gnu_inline
+#endif
void sort(void *base, size_t num, size_t size,
int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t));
+ void (*swap)(void *, void *, size_t))
+{
+ /* pre-scale counters for performance */
+ size_t i = (num / 2) * size, n = num * size, c, r;
+
+ /* heapify */
+ while ( i > 0 )
+ {
+ for ( r = i -= size; r * 2 + size < n; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+
+ /* sort */
+ for ( i = n; i > 0; )
+ {
+ i -= size;
+ swap(base, base + i, size);
+ for ( r = 0; r * 2 + size < i; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+}
#endif /* __XEN_SORT_H__ */
diff --git a/xen/lib/sort.c b/xen/lib/sort.c
index 35ce0d7abd..b7e78cc0e8 100644
--- a/xen/lib/sort.c
+++ b/xen/lib/sort.c
@@ -4,81 +4,5 @@
* Jan 23 2005 Matt Mackall <mpm@selenic.com>
*/
-#include <xen/types.h>
-
-static void u32_swap(void *a, void *b, size_t size)
-{
- uint32_t t = *(uint32_t *)a;
-
- *(uint32_t *)a = *(uint32_t *)b;
- *(uint32_t *)b = t;
-}
-
-static void generic_swap(void *a, void *b, size_t size)
-{
- char t;
-
- do {
- t = *(char *)a;
- *(char *)a++ = *(char *)b;
- *(char *)b++ = t;
- } while ( --size > 0 );
-}
-
-/*
- * sort - sort an array of elements
- * @base: pointer to data to sort
- * @num: number of elements
- * @size: size of each element
- * @cmp: pointer to comparison function
- * @swap: pointer to swap function or NULL
- *
- * This function does a heapsort on the given array. You may provide a
- * swap function optimized to your element type.
- *
- * Sorting time is O(n log n) both on average and worst-case. While
- * qsort is about 20% faster on average, it suffers from exploitable
- * O(n*n) worst-case behavior and extra memory requirements that make
- * it less suitable for kernel use.
- */
-
-void sort(void *base, size_t num, size_t size,
- int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t size))
-{
- /* pre-scale counters for performance */
- size_t i = (num / 2) * size, n = num * size, c, r;
-
- if ( !swap )
- swap = (size == 4 ? u32_swap : generic_swap);
-
- /* heapify */
- while ( i > 0 )
- {
- for ( r = i -= size; r * 2 + size < n; r = c )
- {
- c = r * 2 + size;
- if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-
- /* sort */
- for ( i = n; i > 0; )
- {
- i -= size;
- swap(base, base + i, size);
- for ( r = 0; r * 2 + size < i; r = c )
- {
- c = r * 2 + size;
- if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-}
+#define SORT_IMPLEMENTATION
+#include <xen/sort.h>
--
2.25.1
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-16 3:46 ` Stefano Stabellini
@ 2022-02-16 9:29 ` Bertrand Marquis
2022-02-16 10:44 ` Andrew Cooper
1 sibling, 0 replies; 123+ messages in thread
From: Bertrand Marquis @ 2022-02-16 9:29 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Julien Grall, Andrew Cooper, Xen-devel, Jan Beulich,
Roger Pau Monné, Wei Liu, Volodymyr Babchuk
Hi Stefano,
> On 16 Feb 2022, at 03:46, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Mon, 14 Feb 2022, Julien Grall wrote:
>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a
>>> tight
>>> loop like this are problematic for performance, especially with Spectre v2
>>> protections, which is why extern inline is used commonly by libraries.
>>>
>>> Both ARM callers pass in NULL for the swap function, and while this might
>>> seem
>>> like an attractive option at first, it causes generic_swap() to be used,
>>> which
>>> forced a byte-wise copy. Provide real swap functions so the compiler can
>>> optimise properly, which is very important for ARM downstreams where
>>> milliseconds until the system is up matters.
>>
>> Did you actually benchmark it? Both those lists will have < 128 elements in
>> them. So I would be extremely surprised if you save more than a few hundreds
>> microseconds with this approach.
>>
>> So, my opinion on this approach hasn't changed. On v1, we discussed an
>> approach that would suit both Stefano and I. Jan seemed to confirm that would
>> also suit x86.
>
>
> This patch series has become 70 patches and for the sake of helping
> Andrew move forward in the quickest and most painless way possible, I
> append the following using generic_swap as static inline.
>
> Julien, Bertrand, is that acceptable to you?
Any reason why we cannot in this case keep the NULL parameter in the
existing code and do the if swap == NULL handling in the sort code ?
Other then that this is acceptable for me but I will let Julien say if he is
ok or not as I had no objections before.
Regards
Bertrand
>
> Andrew, I know this is not your favorite approach but you have quite a
> lot of changes to handle -- probably not worth focussing on one detail
> which is pretty minor?
>
>
> ---
> xen/sort: Switch to an extern inline implementation
>
> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
> loop like this are problematic for performance, especially with Spectre v2
> protections, which is why extern inline is used commonly by libraries.
>
> Make generic_swap() a static inline and used it at the two ARM call
> sites.
>
> No functional change.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> xen/arch/arm/bootfdt.c | 2 +-
> xen/arch/arm/io.c | 2 +-
> xen/include/xen/sort.h | 67 ++++++++++++++++++++++++++++++++++-
> xen/lib/sort.c | 80 ++----------------------------------------
> 4 files changed, 70 insertions(+), 81 deletions(-)
>
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index afaa0e249b..0d62945d56 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -472,7 +472,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
> * the banks sorted in ascending order. So sort them through.
> */
> sort(bootinfo.mem.bank, bootinfo.mem.nr_banks, sizeof(struct membank),
> - cmp_memory_node, NULL);
> + cmp_memory_node, generic_swap);
>
> early_print_info();
>
> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> index 729287e37c..1f35aaeea6 100644
> --- a/xen/arch/arm/io.c
> +++ b/xen/arch/arm/io.c
> @@ -170,7 +170,7 @@ void register_mmio_handler(struct domain *d,
>
> /* Sort mmio handlers in ascending order based on base address */
> sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
> - cmp_mmio_handler, NULL);
> + cmp_mmio_handler, generic_swap);
>
> write_unlock(&vmmio->lock);
> }
> diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
> index a403652948..f6065eda58 100644
> --- a/xen/include/xen/sort.h
> +++ b/xen/include/xen/sort.h
> @@ -3,8 +3,73 @@
>
> #include <xen/types.h>
>
> +extern gnu_inline
> +void generic_swap(void *a, void *b, size_t size)
> +{
> + char t;
> +
> + do {
> + t = *(char *)a;
> + *(char *)a++ = *(char *)b;
> + *(char *)b++ = t;
> + } while ( --size > 0 );
> +}
> +
> +/*
> + * sort - sort an array of elements
> + * @base: pointer to data to sort
> + * @num: number of elements
> + * @size: size of each element
> + * @cmp: pointer to comparison function
> + * @swap: pointer to swap function or NULL
> + *
> + * This function does a heapsort on the given array. You may provide a
> + * swap function optimized to your element type.
> + *
> + * Sorting time is O(n log n) both on average and worst-case. While
> + * qsort is about 20% faster on average, it suffers from exploitable
> + * O(n*n) worst-case behavior and extra memory requirements that make
> + * it less suitable for kernel use.
> + */
> +#ifndef SORT_IMPLEMENTATION
> +extern gnu_inline
> +#endif
> void sort(void *base, size_t num, size_t size,
> int (*cmp)(const void *, const void *),
> - void (*swap)(void *, void *, size_t));
> + void (*swap)(void *, void *, size_t))
> +{
> + /* pre-scale counters for performance */
> + size_t i = (num / 2) * size, n = num * size, c, r;
> +
> + /* heapify */
> + while ( i > 0 )
> + {
> + for ( r = i -= size; r * 2 + size < n; r = c )
> + {
> + c = r * 2 + size;
> + if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
> + c += size;
> + if ( cmp(base + r, base + c) >= 0 )
> + break;
> + swap(base + r, base + c, size);
> + }
> + }
> +
> + /* sort */
> + for ( i = n; i > 0; )
> + {
> + i -= size;
> + swap(base, base + i, size);
> + for ( r = 0; r * 2 + size < i; r = c )
> + {
> + c = r * 2 + size;
> + if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
> + c += size;
> + if ( cmp(base + r, base + c) >= 0 )
> + break;
> + swap(base + r, base + c, size);
> + }
> + }
> +}
>
> #endif /* __XEN_SORT_H__ */
> diff --git a/xen/lib/sort.c b/xen/lib/sort.c
> index 35ce0d7abd..b7e78cc0e8 100644
> --- a/xen/lib/sort.c
> +++ b/xen/lib/sort.c
> @@ -4,81 +4,5 @@
> * Jan 23 2005 Matt Mackall <mpm@selenic.com>
> */
>
> -#include <xen/types.h>
> -
> -static void u32_swap(void *a, void *b, size_t size)
> -{
> - uint32_t t = *(uint32_t *)a;
> -
> - *(uint32_t *)a = *(uint32_t *)b;
> - *(uint32_t *)b = t;
> -}
> -
> -static void generic_swap(void *a, void *b, size_t size)
> -{
> - char t;
> -
> - do {
> - t = *(char *)a;
> - *(char *)a++ = *(char *)b;
> - *(char *)b++ = t;
> - } while ( --size > 0 );
> -}
> -
> -/*
> - * sort - sort an array of elements
> - * @base: pointer to data to sort
> - * @num: number of elements
> - * @size: size of each element
> - * @cmp: pointer to comparison function
> - * @swap: pointer to swap function or NULL
> - *
> - * This function does a heapsort on the given array. You may provide a
> - * swap function optimized to your element type.
> - *
> - * Sorting time is O(n log n) both on average and worst-case. While
> - * qsort is about 20% faster on average, it suffers from exploitable
> - * O(n*n) worst-case behavior and extra memory requirements that make
> - * it less suitable for kernel use.
> - */
> -
> -void sort(void *base, size_t num, size_t size,
> - int (*cmp)(const void *, const void *),
> - void (*swap)(void *, void *, size_t size))
> -{
> - /* pre-scale counters for performance */
> - size_t i = (num / 2) * size, n = num * size, c, r;
> -
> - if ( !swap )
> - swap = (size == 4 ? u32_swap : generic_swap);
> -
> - /* heapify */
> - while ( i > 0 )
> - {
> - for ( r = i -= size; r * 2 + size < n; r = c )
> - {
> - c = r * 2 + size;
> - if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
> - c += size;
> - if ( cmp(base + r, base + c) >= 0 )
> - break;
> - swap(base + r, base + c, size);
> - }
> - }
> -
> - /* sort */
> - for ( i = n; i > 0; )
> - {
> - i -= size;
> - swap(base, base + i, size);
> - for ( r = 0; r * 2 + size < i; r = c )
> - {
> - c = r * 2 + size;
> - if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
> - c += size;
> - if ( cmp(base + r, base + c) >= 0 )
> - break;
> - swap(base + r, base + c, size);
> - }
> - }
> -}
> +#define SORT_IMPLEMENTATION
> +#include <xen/sort.h>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-16 3:46 ` Stefano Stabellini
2022-02-16 9:29 ` Bertrand Marquis
@ 2022-02-16 10:44 ` Andrew Cooper
2022-02-16 11:46 ` Julien Grall
1 sibling, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 10:44 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall
Cc: Xen-devel, Jan Beulich, Roger Pau Monne, Wei Liu,
Volodymyr Babchuk, Bertrand Marquis
On 16/02/2022 03:46, Stefano Stabellini wrote:
> On Mon, 14 Feb 2022, Julien Grall wrote:
>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a
>>> tight
>>> loop like this are problematic for performance, especially with Spectre v2
>>> protections, which is why extern inline is used commonly by libraries.
>>>
>>> Both ARM callers pass in NULL for the swap function, and while this might
>>> seem
>>> like an attractive option at first, it causes generic_swap() to be used,
>>> which
>>> forced a byte-wise copy. Provide real swap functions so the compiler can
>>> optimise properly, which is very important for ARM downstreams where
>>> milliseconds until the system is up matters.
>> Did you actually benchmark it? Both those lists will have < 128 elements in
>> them. So I would be extremely surprised if you save more than a few hundreds
>> microseconds with this approach.
>>
>> So, my opinion on this approach hasn't changed. On v1, we discussed an
>> approach that would suit both Stefano and I. Jan seemed to confirm that would
>> also suit x86.
> This patch series has become 70 patches and for the sake of helping
> Andrew move forward in the quickest and most painless way possible, I
> append the following using generic_swap as static inline.
>
> Julien, Bertrand, is that acceptable to you?
>
> Andrew, I know this is not your favorite approach but you have quite a
> lot of changes to handle -- probably not worth focussing on one detail
> which is pretty minor?
It's not pretty minor. My version really is the best thing for ARM.
The perf improvement alone, marginal as it may be in practice, is
justification alone for the patch, and Bertrand's R-by is testament to this.
But the reasons why getting rid the swap functions is important for
CET-IBT on x86 are exactly the same as why getting rid of them on ARM
will be important for BTI support. A tagged function doing an arbitrary
bytewise swap from two parameters controlled by the third is far more
valuable to an attackers gadget library than a typical function.
i.e. this proposed intermediary, if it compiles, is just busywork which
someone else is going to have to revert in the future, along with having
this argument again.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-16 10:44 ` Andrew Cooper
@ 2022-02-16 11:46 ` Julien Grall
2022-02-16 11:55 ` Bertrand Marquis
0 siblings, 1 reply; 123+ messages in thread
From: Julien Grall @ 2022-02-16 11:46 UTC (permalink / raw)
To: Andrew Cooper, Stefano Stabellini
Cc: Xen-devel, Jan Beulich, Roger Pau Monne, Wei Liu,
Volodymyr Babchuk, Bertrand Marquis
Hi,
On 16/02/2022 10:44, Andrew Cooper wrote:
> On 16/02/2022 03:46, Stefano Stabellini wrote:
>> On Mon, 14 Feb 2022, Julien Grall wrote:
>>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>>> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a
>>>> tight
>>>> loop like this are problematic for performance, especially with Spectre v2
>>>> protections, which is why extern inline is used commonly by libraries.
>>>>
>>>> Both ARM callers pass in NULL for the swap function, and while this might
>>>> seem
>>>> like an attractive option at first, it causes generic_swap() to be used,
>>>> which
>>>> forced a byte-wise copy. Provide real swap functions so the compiler can
>>>> optimise properly, which is very important for ARM downstreams where
>>>> milliseconds until the system is up matters.
>>> Did you actually benchmark it? Both those lists will have < 128 elements in
>>> them. So I would be extremely surprised if you save more than a few hundreds
>>> microseconds with this approach.
>>>
>>> So, my opinion on this approach hasn't changed. On v1, we discussed an
>>> approach that would suit both Stefano and I. Jan seemed to confirm that would
>>> also suit x86.
>> This patch series has become 70 patches and for the sake of helping
>> Andrew move forward in the quickest and most painless way possible, I
>> append the following using generic_swap as static inline.
>>
>> Julien, Bertrand, is that acceptable to you?
>>
>> Andrew, I know this is not your favorite approach but you have quite a
>> lot of changes to handle -- probably not worth focussing on one detail
>> which is pretty minor?
>
> It's not pretty minor. My version really is the best thing for ARM.
>
> The perf improvement alone, marginal as it may be in practice, is
Our take on Arm has always been to avoid micro-optimization when the
resulting code is more difficult to maintain.
> justification alone for the patch, and Bertrand's R-by is testament to this.
I am not sure why calling out that Bertrand agreed means that everyone
else should accept your approach.
This reminds me other series that have been blocked for a long time by
you. Yet you made no effort to compromise... How ironic.
>
> But the reasons why getting rid the swap functions is important for
> CET-IBT on x86 are exactly the same as why getting rid of them on ARM
> will be important for BTI support. A tagged function doing an arbitrary
> bytewise swap from two parameters controlled by the third is far more
> valuable to an attackers gadget library than a typical function.
This is a more compelling reason. However, I am a bit puzzled why it
took you so long to come up with this reason.
> i.e. this proposed intermediary, if it compiles, is just busywork which
> someone else is going to have to revert in the future, along with having
> this argument again.
Well, this argument would have never happened if your commit message
contained information about BTI.
Instead you decided to just mention the performance part despite me
objecting it and requesting for a different reason in v1 (see [1]).
Anyway, I will wait for a reword of the commit message before lifting my
Nacked-by.
Cheers,
[1]
https://lore.kernel.org/xen-devel/f7bb7a08-4721-f2a8-8602-0cd1baf1f083@xen.org/
--
Julien Grall
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation
2022-02-16 11:46 ` Julien Grall
@ 2022-02-16 11:55 ` Bertrand Marquis
0 siblings, 0 replies; 123+ messages in thread
From: Bertrand Marquis @ 2022-02-16 11:55 UTC (permalink / raw)
To: Julien Grall
Cc: Andrew Cooper, Stefano Stabellini, Xen-devel, Jan Beulich,
Roger Pau Monne, Wei Liu, Volodymyr Babchuk
Hi,
> On 16 Feb 2022, at 11:46, Julien Grall <julien@xen.org> wrote:
>
> Hi,
>
> On 16/02/2022 10:44, Andrew Cooper wrote:
>> On 16/02/2022 03:46, Stefano Stabellini wrote:
>>> On Mon, 14 Feb 2022, Julien Grall wrote:
>>>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>>>> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a
>>>>> tight
>>>>> loop like this are problematic for performance, especially with Spectre v2
>>>>> protections, which is why extern inline is used commonly by libraries.
>>>>>
>>>>> Both ARM callers pass in NULL for the swap function, and while this might
>>>>> seem
>>>>> like an attractive option at first, it causes generic_swap() to be used,
>>>>> which
>>>>> forced a byte-wise copy. Provide real swap functions so the compiler can
>>>>> optimise properly, which is very important for ARM downstreams where
>>>>> milliseconds until the system is up matters.
>>>> Did you actually benchmark it? Both those lists will have < 128 elements in
>>>> them. So I would be extremely surprised if you save more than a few hundreds
>>>> microseconds with this approach.
>>>>
>>>> So, my opinion on this approach hasn't changed. On v1, we discussed an
>>>> approach that would suit both Stefano and I. Jan seemed to confirm that would
>>>> also suit x86.
>>> This patch series has become 70 patches and for the sake of helping
>>> Andrew move forward in the quickest and most painless way possible, I
>>> append the following using generic_swap as static inline.
>>>
>>> Julien, Bertrand, is that acceptable to you?
>>>
>>> Andrew, I know this is not your favorite approach but you have quite a
>>> lot of changes to handle -- probably not worth focussing on one detail
>>> which is pretty minor?
>> It's not pretty minor. My version really is the best thing for ARM.
> >
>> The perf improvement alone, marginal as it may be in practice, is
>
> Our take on Arm has always been to avoid micro-optimization when the resulting code is more difficult to maintain.
>
>> justification alone for the patch, and Bertrand's R-by is testament to this.
>
> I am not sure why calling out that Bertrand agreed means that everyone else should accept your approach.
>
> This reminds me other series that have been blocked for a long time by you. Yet you made no effort to compromise... How ironic.
>
>> But the reasons why getting rid the swap functions is important for
>> CET-IBT on x86 are exactly the same as why getting rid of them on ARM
>> will be important for BTI support. A tagged function doing an arbitrary
>> bytewise swap from two parameters controlled by the third is far more
>> valuable to an attackers gadget library than a typical function.
>
> This is a more compelling reason. However, I am a bit puzzled why it took you so long to come up with this reason.
>
>> i.e. this proposed intermediary, if it compiles, is just busywork which
>> someone else is going to have to revert in the future, along with having
>> this argument again.
>
> Well, this argument would have never happened if your commit message contained information about BTI.
I agree that this would be nice to have in the commit message as a justification for the change.
Cheers
Bertrand
>
> Instead you decided to just mention the performance part despite me objecting it and requesting for a different reason in v1 (see [1]).
>
> Anyway, I will wait for a reword of the commit message before lifting my Nacked-by.
>
> Cheers,
>
> [1] https://lore.kernel.org/xen-devel/f7bb7a08-4721-f2a8-8602-0cd1baf1f083@xen.org/
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 03/70] xen/xsm: Move {do,compat}_flask_op() declarations into a header
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 01/70] xen/domain: Improve pirq handling Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 02/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 14:36 ` Daniel P. Smith
2022-02-14 12:50 ` [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
` (68 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Daniel De Graaf, Daniel Smith
Declaring sideways like this is unsafe, because the compiler can't check that
the implementaton in flask_op.c still has the same type.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: Daniel Smith <dpsmith@apertussolutions.com>
v2:
* Rework in the face of no useful progress on the better fix.
---
xen/xsm/flask/flask_op.c | 1 +
xen/xsm/flask/hooks.c | 4 +---
xen/xsm/flask/private.h | 9 +++++++++
3 files changed, 11 insertions(+), 3 deletions(-)
create mode 100644 xen/xsm/flask/private.h
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 221ff00fd3cc..bb3bebc30e01 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -21,6 +21,7 @@
#include <avc_ss.h>
#include <objsec.h>
#include <conditional.h>
+#include "private.h"
#define ret_t long
#define _copy_to_guest copy_to_guest
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 3b29f7fde372..6ff1be28e4a4 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -36,6 +36,7 @@
#include <avc_ss.h>
#include <objsec.h>
#include <conditional.h>
+#include "private.h"
static u32 domain_sid(const struct domain *dom)
{
@@ -1742,9 +1743,6 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
#endif
-long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
-int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
-
static const struct xsm_ops __initconstrel flask_ops = {
.security_domaininfo = flask_security_domaininfo,
.domain_create = flask_domain_create,
diff --git a/xen/xsm/flask/private.h b/xen/xsm/flask/private.h
new file mode 100644
index 000000000000..73b0de87245a
--- /dev/null
+++ b/xen/xsm/flask/private.h
@@ -0,0 +1,9 @@
+#ifndef XSM_FLASK_PRIVATE
+#define XSM_FLASK_PRIVATE
+
+#include <public/xen.h>
+
+long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+
+#endif /* XSM_FLASK_PRIVATE */
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 03/70] xen/xsm: Move {do,compat}_flask_op() declarations into a header
2022-02-14 12:50 ` [PATCH v2 03/70] xen/xsm: Move {do,compat}_flask_op() declarations into a header Andrew Cooper
@ 2022-02-14 14:36 ` Daniel P. Smith
0 siblings, 0 replies; 123+ messages in thread
From: Daniel P. Smith @ 2022-02-14 14:36 UTC (permalink / raw)
To: Andrew Cooper, Xen-devel; +Cc: Daniel De Graaf
On 2/14/22 07:50, Andrew Cooper wrote:
> Declaring sideways like this is unsafe, because the compiler can't check that
> the implementaton in flask_op.c still has the same type.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> CC: Daniel Smith <dpsmith@apertussolutions.com>
>
> v2:
> * Rework in the face of no useful progress on the better fix.
> ---
> xen/xsm/flask/flask_op.c | 1 +
> xen/xsm/flask/hooks.c | 4 +---
> xen/xsm/flask/private.h | 9 +++++++++
> 3 files changed, 11 insertions(+), 3 deletions(-)
> create mode 100644 xen/xsm/flask/private.h
>
> diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
> index 221ff00fd3cc..bb3bebc30e01 100644
> --- a/xen/xsm/flask/flask_op.c
> +++ b/xen/xsm/flask/flask_op.c
> @@ -21,6 +21,7 @@
> #include <avc_ss.h>
> #include <objsec.h>
> #include <conditional.h>
> +#include "private.h"
>
> #define ret_t long
> #define _copy_to_guest copy_to_guest
> diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
> index 3b29f7fde372..6ff1be28e4a4 100644
> --- a/xen/xsm/flask/hooks.c
> +++ b/xen/xsm/flask/hooks.c
> @@ -36,6 +36,7 @@
> #include <avc_ss.h>
> #include <objsec.h>
> #include <conditional.h>
> +#include "private.h"
>
> static u32 domain_sid(const struct domain *dom)
> {
> @@ -1742,9 +1743,6 @@ static int flask_argo_send(const struct domain *d, const struct domain *t)
>
> #endif
>
> -long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
> -int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
> -
> static const struct xsm_ops __initconstrel flask_ops = {
> .security_domaininfo = flask_security_domaininfo,
> .domain_create = flask_domain_create,
> diff --git a/xen/xsm/flask/private.h b/xen/xsm/flask/private.h
> new file mode 100644
> index 000000000000..73b0de87245a
> --- /dev/null
> +++ b/xen/xsm/flask/private.h
> @@ -0,0 +1,9 @@
> +#ifndef XSM_FLASK_PRIVATE
> +#define XSM_FLASK_PRIVATE
> +
> +#include <public/xen.h>
> +
> +long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
> +int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
> +
> +#endif /* XSM_FLASK_PRIVATE */
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (2 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 03/70] xen/xsm: Move {do,compat}_flask_op() declarations into a header Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 13:33 ` Jan Beulich
2022-02-14 12:50 ` [PATCH v2 05/70] x86: Don't use the hypercall table for calling compat hypercalls Andrew Cooper
` (67 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Juergen Gross, Andrew Cooper
From: Juergen Gross <jgross@suse.com>
When running as pv-shim the hypercall is modified today in order to
replace the functions for __HYPERVISOR_event_channel_op and
__HYPERVISOR_grant_table_op hypercalls.
Change this to call the related functions from the normal handlers
instead when running as shim. The performance implications are not
really relevant, as a normal production hypervisor will not be
configured to support shim mode, so the related calls will be dropped
due to optimization of the compiler.
Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
wrapper do_grant_table_op() needed, as in this case grant_table.c
isn't being built.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/include/asm/hypercall.h | 4 ++-
xen/arch/x86/include/asm/pv/shim.h | 3 ++
xen/arch/x86/pv/hypercall.c | 2 +-
xen/arch/x86/pv/shim.c | 54 ++++++++++++++++----------------
xen/arch/x86/x86_64/platform_hypercall.c | 2 +-
xen/common/compat/multicall.c | 3 +-
xen/common/event_channel.c | 9 ++++++
xen/common/grant_table.c | 9 ++++++
8 files changed, 54 insertions(+), 32 deletions(-)
diff --git a/xen/arch/x86/include/asm/hypercall.h b/xen/arch/x86/include/asm/hypercall.h
index 5d394d492318..f004824f16b6 100644
--- a/xen/arch/x86/include/asm/hypercall.h
+++ b/xen/arch/x86/include/asm/hypercall.h
@@ -145,6 +145,7 @@ do_set_segment_base(
#include <compat/arch-x86/xen.h>
#include <compat/physdev.h>
+#include <compat/platform.h>
extern int
compat_physdev_op(
@@ -161,8 +162,9 @@ extern int compat_mmuext_op(
XEN_GUEST_HANDLE_PARAM(uint) pdone,
unsigned int foreigndom);
+DEFINE_XEN_GUEST_HANDLE(compat_platform_op_t);
extern int compat_platform_op(
- XEN_GUEST_HANDLE_PARAM(void) u_xenpf_op);
+ XEN_GUEST_HANDLE_PARAM(compat_platform_op_t) u_xenpf_op);
extern long compat_callback_op(
int cmd, XEN_GUEST_HANDLE(void) arg);
diff --git a/xen/arch/x86/include/asm/pv/shim.h b/xen/arch/x86/include/asm/pv/shim.h
index 8a91f4f9dfbf..6415f8068e5c 100644
--- a/xen/arch/x86/include/asm/pv/shim.h
+++ b/xen/arch/x86/include/asm/pv/shim.h
@@ -19,6 +19,7 @@
#ifndef __X86_PV_SHIM_H__
#define __X86_PV_SHIM_H__
+#include <xen/hypercall.h>
#include <xen/types.h>
#if defined(CONFIG_PV_SHIM_EXCLUSIVE)
@@ -45,6 +46,8 @@ domid_t get_initial_domain_id(void);
uint64_t pv_shim_mem(uint64_t avail);
void pv_shim_fixup_e820(struct e820map *e820);
const struct platform_bad_page *pv_shim_reserved_pages(unsigned int *size);
+typeof(do_event_channel_op) pv_shim_event_channel_op;
+typeof(do_grant_table_op) pv_shim_grant_table_op;
#else
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index ecdd58deea69..50cd219c18fc 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -64,7 +64,7 @@ const pv_hypercall_table_t pv_hypercall_table[] = {
COMPAT_CALL(xen_version),
HYPERCALL(console_io),
COMPAT_CALL(physdev_op_compat),
-#ifdef CONFIG_GRANT_TABLE
+#if defined(CONFIG_GRANT_TABLE) || defined(CONFIG_PV_SHIM)
COMPAT_CALL(grant_table_op),
#endif
HYPERCALL(vm_assist),
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index d9704121a739..7e891fe2f7a4 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -56,11 +56,6 @@ static DEFINE_SPINLOCK(balloon_lock);
static struct platform_bad_page __initdata reserved_pages[2];
-static long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-static long pv_shim_grant_table_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count);
-
/*
* By default give the shim 1MB of free memory slack. Some users may wish to
* tune this constants for better memory utilization. This can be achieved
@@ -203,7 +198,6 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
start_info_t *si)
{
bool compat = is_pv_32bit_domain(d);
- pv_hypercall_table_t *rw_pv_hypercall_table;
uint64_t param = 0;
long rc;
@@ -249,23 +243,6 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
consoled_set_ring_addr(page);
}
- /*
- * Locate pv_hypercall_table[] (usually .rodata) in the directmap (which
- * is writeable) and insert some shim-specific hypercall handlers.
- */
- rw_pv_hypercall_table = __va(__pa(pv_hypercall_table));
- rw_pv_hypercall_table[__HYPERVISOR_event_channel_op].native =
- (hypercall_fn_t *)pv_shim_event_channel_op;
- rw_pv_hypercall_table[__HYPERVISOR_grant_table_op].native =
- (hypercall_fn_t *)pv_shim_grant_table_op;
-
-#ifdef CONFIG_PV32
- rw_pv_hypercall_table[__HYPERVISOR_event_channel_op].compat =
- (hypercall_fn_t *)pv_shim_event_channel_op;
- rw_pv_hypercall_table[__HYPERVISOR_grant_table_op].compat =
- (hypercall_fn_t *)pv_shim_grant_table_op;
-#endif
-
guest = d;
/*
@@ -435,7 +412,7 @@ int pv_shim_shutdown(uint8_t reason)
return 0;
}
-static long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct domain *d = current->domain;
struct evtchn_close close;
@@ -683,9 +660,9 @@ void pv_shim_inject_evtchn(unsigned int port)
# define compat_handle_okay guest_handle_okay
#endif
-static long pv_shim_grant_table_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count)
+long pv_shim_grant_table_op(unsigned int cmd,
+ XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
{
struct domain *d = current->domain;
long rc = 0;
@@ -845,6 +822,29 @@ static long pv_shim_grant_table_op(unsigned int cmd,
return rc;
}
+#ifndef CONFIG_GRANT_TABLE
+/* Thin wrapper(s) needed. */
+long do_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
+{
+ if ( !pv_shim )
+ return -ENOSYS;
+
+ return pv_shim_grant_table_op(cmd, uop, count);
+}
+
+#ifdef CONFIG_PV32
+int compat_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
+{
+ if ( !pv_shim )
+ return -ENOSYS;
+
+ return pv_shim_grant_table_op(cmd, uop, count);
+}
+#endif
+#endif
+
long pv_shim_cpu_up(void *data)
{
struct vcpu *v = data;
diff --git a/xen/arch/x86/x86_64/platform_hypercall.c b/xen/arch/x86/x86_64/platform_hypercall.c
index fbba893a47cb..966fd27b5f22 100644
--- a/xen/arch/x86/x86_64/platform_hypercall.c
+++ b/xen/arch/x86/x86_64/platform_hypercall.c
@@ -6,8 +6,8 @@ EMIT_FILE;
#include <xen/lib.h>
#include <compat/platform.h>
+#include <xen/hypercall.h>
-DEFINE_XEN_GUEST_HANDLE(compat_platform_op_t);
#define xen_platform_op compat_platform_op
#define xen_platform_op_t compat_platform_op_t
#define do_platform_op(x) compat_platform_op(_##x)
diff --git a/xen/common/compat/multicall.c b/xen/common/compat/multicall.c
index a0e9918f4805..b17739d21829 100644
--- a/xen/common/compat/multicall.c
+++ b/xen/common/compat/multicall.c
@@ -5,7 +5,7 @@
EMIT_FILE;
#include <xen/types.h>
-#include <xen/multicall.h>
+#include <xen/hypercall.h>
#include <xen/trace.h>
#define COMPAT
@@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
mcs->compat_call.args[i] = mcs->call.args[i];
}
-DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
#define multicall_entry compat_multicall_entry
#define multicall_entry_t multicall_entry_compat_t
#define do_multicall_call compat_multicall_call
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index da88ad141a69..c9912122d1e5 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -31,6 +31,10 @@
#include <public/event_channel.h>
#include <xsm/xsm.h>
+#ifdef CONFIG_PV_SHIM
+#include <asm/guest.h>
+#endif
+
#define ERROR_EXIT(_errno) \
do { \
gdprintk(XENLOG_WARNING, \
@@ -1189,6 +1193,11 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int rc;
+#ifdef CONFIG_PV_SHIM
+ if ( unlikely(pv_shim) )
+ return pv_shim_event_channel_op(cmd, arg);
+#endif
+
switch ( cmd )
{
case EVTCHNOP_alloc_unbound: {
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 3d92fee59285..925ed7d6bee2 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -44,6 +44,10 @@
#include <asm/flushtlb.h>
#include <asm/guest_atomics.h>
+#ifdef CONFIG_PV_SHIM
+#include <asm/guest.h>
+#endif
+
/* Per-domain grant information. */
struct grant_table {
/*
@@ -3561,6 +3565,11 @@ do_grant_table_op(
long rc;
unsigned int opaque_in = cmd & GNTTABOP_ARG_MASK, opaque_out = 0;
+#ifdef CONFIG_PV_SHIM
+ if ( unlikely(pv_shim) )
+ return pv_shim_grant_table_op(cmd, uop, count);
+#endif
+
if ( (int)count < 0 )
return -EINVAL;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-14 12:50 ` [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
@ 2022-02-14 13:33 ` Jan Beulich
2022-02-14 13:50 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-14 13:33 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 14.02.2022 13:50, Andrew Cooper wrote:
> From: Juergen Gross <jgross@suse.com>
>
> When running as pv-shim the hypercall is modified today in order to
> replace the functions for __HYPERVISOR_event_channel_op and
> __HYPERVISOR_grant_table_op hypercalls.
>
> Change this to call the related functions from the normal handlers
> instead when running as shim. The performance implications are not
> really relevant, as a normal production hypervisor will not be
> configured to support shim mode, so the related calls will be dropped
> due to optimization of the compiler.
>
> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
> wrapper do_grant_table_op() needed, as in this case grant_table.c
> isn't being built.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
I don't think you sync-ed this with Jürgen's v3. There were only minor
changes but having a stale version sent two months later isn't very
nice.
> --- a/xen/common/compat/multicall.c
> +++ b/xen/common/compat/multicall.c
> @@ -5,7 +5,7 @@
> EMIT_FILE;
>
> #include <xen/types.h>
> -#include <xen/multicall.h>
> +#include <xen/hypercall.h>
> #include <xen/trace.h>
>
> #define COMPAT
> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
> mcs->compat_call.args[i] = mcs->call.args[i];
> }
>
> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
> #define multicall_entry compat_multicall_entry
> #define multicall_entry_t multicall_entry_compat_t
> #define do_multicall_call compat_multicall_call
Jürgen's patch doesn't have any change to this file, and I'm afraid I
also don't see how these adjustments are related here. The commit
message sadly also doesn't help ...
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-14 13:33 ` Jan Beulich
@ 2022-02-14 13:50 ` Andrew Cooper
2022-02-14 13:56 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 13:50 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 14/02/2022 13:33, Jan Beulich wrote:
> On 14.02.2022 13:50, Andrew Cooper wrote:
>> From: Juergen Gross <jgross@suse.com>
>>
>> When running as pv-shim the hypercall is modified today in order to
>> replace the functions for __HYPERVISOR_event_channel_op and
>> __HYPERVISOR_grant_table_op hypercalls.
>>
>> Change this to call the related functions from the normal handlers
>> instead when running as shim. The performance implications are not
>> really relevant, as a normal production hypervisor will not be
>> configured to support shim mode, so the related calls will be dropped
>> due to optimization of the compiler.
>>
>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>> isn't being built.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> I don't think you sync-ed this with Jürgen's v3. There were only minor
> changes but having a stale version sent two months later isn't very
> nice.
I did resync. What do you think is missing?
>
>> --- a/xen/common/compat/multicall.c
>> +++ b/xen/common/compat/multicall.c
>> @@ -5,7 +5,7 @@
>> EMIT_FILE;
>>
>> #include <xen/types.h>
>> -#include <xen/multicall.h>
>> +#include <xen/hypercall.h>
>> #include <xen/trace.h>
>>
>> #define COMPAT
>> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
>> mcs->compat_call.args[i] = mcs->call.args[i];
>> }
>>
>> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
>> #define multicall_entry compat_multicall_entry
>> #define multicall_entry_t multicall_entry_compat_t
>> #define do_multicall_call compat_multicall_call
> Jürgen's patch doesn't have any change to this file, and I'm afraid I
> also don't see how these adjustments are related here. The commit
> message sadly also doesn't help ...
The changes are very necessary to split it out of Juergen's series.
Without the adjustment, the correction of compat_platform_op()'s guest
handle type from void to compat_platform_op_t doesn't compile.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-14 13:50 ` Andrew Cooper
@ 2022-02-14 13:56 ` Jan Beulich
2022-02-16 22:17 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-14 13:56 UTC (permalink / raw)
To: Andrew Cooper, Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 14.02.2022 14:50, Andrew Cooper wrote:
> On 14/02/2022 13:33, Jan Beulich wrote:
>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>> From: Juergen Gross <jgross@suse.com>
>>>
>>> When running as pv-shim the hypercall is modified today in order to
>>> replace the functions for __HYPERVISOR_event_channel_op and
>>> __HYPERVISOR_grant_table_op hypercalls.
>>>
>>> Change this to call the related functions from the normal handlers
>>> instead when running as shim. The performance implications are not
>>> really relevant, as a normal production hypervisor will not be
>>> configured to support shim mode, so the related calls will be dropped
>>> due to optimization of the compiler.
>>>
>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>> isn't being built.
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>> changes but having a stale version sent two months later isn't very
>> nice.
>
> I did resync. What do you think is missing?
A few likely() / unlikely() as far as I could see.
>>> --- a/xen/common/compat/multicall.c
>>> +++ b/xen/common/compat/multicall.c
>>> @@ -5,7 +5,7 @@
>>> EMIT_FILE;
>>>
>>> #include <xen/types.h>
>>> -#include <xen/multicall.h>
>>> +#include <xen/hypercall.h>
>>> #include <xen/trace.h>
>>>
>>> #define COMPAT
>>> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
>>> mcs->compat_call.args[i] = mcs->call.args[i];
>>> }
>>>
>>> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
>>> #define multicall_entry compat_multicall_entry
>>> #define multicall_entry_t multicall_entry_compat_t
>>> #define do_multicall_call compat_multicall_call
>> Jürgen's patch doesn't have any change to this file, and I'm afraid I
>> also don't see how these adjustments are related here. The commit
>> message sadly also doesn't help ...
>
> The changes are very necessary to split it out of Juergen's series.
>
> Without the adjustment, the correction of compat_platform_op()'s guest
> handle type from void to compat_platform_op_t doesn't compile.
Interesting. That's quite far from obvious in this context, so clarifying
the purpose in the description would seem helpful.
Coming back to the syncing with v3: Was this change the reason then why
you did drop my R-b?
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-14 13:56 ` Jan Beulich
@ 2022-02-16 22:17 ` Andrew Cooper
2022-02-17 10:20 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 22:17 UTC (permalink / raw)
To: Jan Beulich; +Cc: Juergen Gross, Xen-devel
On 14/02/2022 13:56, Jan Beulich wrote:
> On 14.02.2022 14:50, Andrew Cooper wrote:
>> On 14/02/2022 13:33, Jan Beulich wrote:
>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>> From: Juergen Gross <jgross@suse.com>
>>>>
>>>> When running as pv-shim the hypercall is modified today in order to
>>>> replace the functions for __HYPERVISOR_event_channel_op and
>>>> __HYPERVISOR_grant_table_op hypercalls.
>>>>
>>>> Change this to call the related functions from the normal handlers
>>>> instead when running as shim. The performance implications are not
>>>> really relevant, as a normal production hypervisor will not be
>>>> configured to support shim mode, so the related calls will be dropped
>>>> due to optimization of the compiler.
>>>>
>>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>>> isn't being built.
>>>>
>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>>> changes but having a stale version sent two months later isn't very
>>> nice.
>> I did resync. What do you think is missing?
> A few likely() / unlikely() as far as I could see.
Oh those two. I appear to have forgot to email.
They're wrong - observe they're in an ifndef block, not an ifdef block.
Obligatory citation of the recommendation for humans not to try annotating.
>>>> --- a/xen/common/compat/multicall.c
>>>> +++ b/xen/common/compat/multicall.c
>>>> @@ -5,7 +5,7 @@
>>>> EMIT_FILE;
>>>>
>>>> #include <xen/types.h>
>>>> -#include <xen/multicall.h>
>>>> +#include <xen/hypercall.h>
>>>> #include <xen/trace.h>
>>>>
>>>> #define COMPAT
>>>> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
>>>> mcs->compat_call.args[i] = mcs->call.args[i];
>>>> }
>>>>
>>>> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
>>>> #define multicall_entry compat_multicall_entry
>>>> #define multicall_entry_t multicall_entry_compat_t
>>>> #define do_multicall_call compat_multicall_call
>>> Jürgen's patch doesn't have any change to this file, and I'm afraid I
>>> also don't see how these adjustments are related here. The commit
>>> message sadly also doesn't help ...
>> The changes are very necessary to split it out of Juergen's series.
>>
>> Without the adjustment, the correction of compat_platform_op()'s guest
>> handle type from void to compat_platform_op_t doesn't compile.
> Interesting. That's quite far from obvious in this context, so clarifying
> the purpose in the description would seem helpful.
>
> Coming back to the syncing with v3: Was this change the reason then why
> you did drop my R-b?
My porting of this patch is a non-trivial modification from Juergen's
version, and not eligible to retain any tags.
I thought I'd discussed this, but I appear to have missed it from both
versions of the series. Sorry.
Either way. It's exactly the same purpose as before, but modified to
compile in isolation.
Juergen's second patch, subsequent to this one, is unmodified which is
why I retained your R-by there.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-16 22:17 ` Andrew Cooper
@ 2022-02-17 10:20 ` Jan Beulich
2022-02-17 10:34 ` Juergen Gross
2022-02-21 19:21 ` Andrew Cooper
0 siblings, 2 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 10:20 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 16.02.2022 23:17, Andrew Cooper wrote:
> On 14/02/2022 13:56, Jan Beulich wrote:
>> On 14.02.2022 14:50, Andrew Cooper wrote:
>>> On 14/02/2022 13:33, Jan Beulich wrote:
>>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>
>>>>> When running as pv-shim the hypercall is modified today in order to
>>>>> replace the functions for __HYPERVISOR_event_channel_op and
>>>>> __HYPERVISOR_grant_table_op hypercalls.
>>>>>
>>>>> Change this to call the related functions from the normal handlers
>>>>> instead when running as shim. The performance implications are not
>>>>> really relevant, as a normal production hypervisor will not be
>>>>> configured to support shim mode, so the related calls will be dropped
>>>>> due to optimization of the compiler.
>>>>>
>>>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>>>> isn't being built.
>>>>>
>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>>>> changes but having a stale version sent two months later isn't very
>>>> nice.
>>> I did resync. What do you think is missing?
>> A few likely() / unlikely() as far as I could see.
>
> Oh those two. I appear to have forgot to email.
>
> They're wrong - observe they're in an ifndef block, not an ifdef block.
I don't see how the (unrelated) #ifndef matters here: The #ifndef
is about grant table availability. The two likely() are about
running as shim. I'm of the firm opinion that a binary built
without PV_SHIM_EXCLUSIVE is far more likely to be used as a bare
metal hypervisor. And for a PV_SHIM_EXCLUSIVE hypervisor the
conditions are constant anyway, and hence the unlikely() has no
effect.
And if your way should really be followed, why did you deem the two
unlikely() in do_event_channel_op() and do_grant_table_op() okay?
>>>>> --- a/xen/common/compat/multicall.c
>>>>> +++ b/xen/common/compat/multicall.c
>>>>> @@ -5,7 +5,7 @@
>>>>> EMIT_FILE;
>>>>>
>>>>> #include <xen/types.h>
>>>>> -#include <xen/multicall.h>
>>>>> +#include <xen/hypercall.h>
>>>>> #include <xen/trace.h>
>>>>>
>>>>> #define COMPAT
>>>>> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
>>>>> mcs->compat_call.args[i] = mcs->call.args[i];
>>>>> }
>>>>>
>>>>> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
>>>>> #define multicall_entry compat_multicall_entry
>>>>> #define multicall_entry_t multicall_entry_compat_t
>>>>> #define do_multicall_call compat_multicall_call
>>>> Jürgen's patch doesn't have any change to this file, and I'm afraid I
>>>> also don't see how these adjustments are related here. The commit
>>>> message sadly also doesn't help ...
>>> The changes are very necessary to split it out of Juergen's series.
>>>
>>> Without the adjustment, the correction of compat_platform_op()'s guest
>>> handle type from void to compat_platform_op_t doesn't compile.
>> Interesting. That's quite far from obvious in this context, so clarifying
>> the purpose in the description would seem helpful.
>>
>> Coming back to the syncing with v3: Was this change the reason then why
>> you did drop my R-b?
>
> My porting of this patch is a non-trivial modification from Juergen's
> version, and not eligible to retain any tags.
>
> I thought I'd discussed this, but I appear to have missed it from both
> versions of the series. Sorry.
>
> Either way. It's exactly the same purpose as before, but modified to
> compile in isolation.
I see. I'm under the impression though that parts were effectively
present elsewhere in Jürgen's series. Perhaps it would have been easier
if his series (at least up to the point to which you need it here)
would (long) have gone in already. What it looks to be blocked on are
two or three Arm acks and an x86 ack on patch 1 (which I've expressed
I'm not entirely happy about, and hence I'm not going to either ack or
nack it).
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-17 10:20 ` Jan Beulich
@ 2022-02-17 10:34 ` Juergen Gross
2022-02-21 19:21 ` Andrew Cooper
1 sibling, 0 replies; 123+ messages in thread
From: Juergen Gross @ 2022-02-17 10:34 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper; +Cc: Xen-devel
[-- Attachment #1.1.1: Type: text/plain, Size: 4696 bytes --]
On 17.02.22 11:20, Jan Beulich wrote:
> On 16.02.2022 23:17, Andrew Cooper wrote:
>> On 14/02/2022 13:56, Jan Beulich wrote:
>>> On 14.02.2022 14:50, Andrew Cooper wrote:
>>>> On 14/02/2022 13:33, Jan Beulich wrote:
>>>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>>
>>>>>> When running as pv-shim the hypercall is modified today in order to
>>>>>> replace the functions for __HYPERVISOR_event_channel_op and
>>>>>> __HYPERVISOR_grant_table_op hypercalls.
>>>>>>
>>>>>> Change this to call the related functions from the normal handlers
>>>>>> instead when running as shim. The performance implications are not
>>>>>> really relevant, as a normal production hypervisor will not be
>>>>>> configured to support shim mode, so the related calls will be dropped
>>>>>> due to optimization of the compiler.
>>>>>>
>>>>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>>>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>>>>> isn't being built.
>>>>>>
>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>>>>> changes but having a stale version sent two months later isn't very
>>>>> nice.
>>>> I did resync. What do you think is missing?
>>> A few likely() / unlikely() as far as I could see.
>>
>> Oh those two. I appear to have forgot to email.
>>
>> They're wrong - observe they're in an ifndef block, not an ifdef block.
>
> I don't see how the (unrelated) #ifndef matters here: The #ifndef
> is about grant table availability. The two likely() are about
> running as shim. I'm of the firm opinion that a binary built
> without PV_SHIM_EXCLUSIVE is far more likely to be used as a bare
> metal hypervisor. And for a PV_SHIM_EXCLUSIVE hypervisor the
> conditions are constant anyway, and hence the unlikely() has no
> effect.
>
> And if your way should really be followed, why did you deem the two
> unlikely() in do_event_channel_op() and do_grant_table_op() okay?
>
>>>>>> --- a/xen/common/compat/multicall.c
>>>>>> +++ b/xen/common/compat/multicall.c
>>>>>> @@ -5,7 +5,7 @@
>>>>>> EMIT_FILE;
>>>>>>
>>>>>> #include <xen/types.h>
>>>>>> -#include <xen/multicall.h>
>>>>>> +#include <xen/hypercall.h>
>>>>>> #include <xen/trace.h>
>>>>>>
>>>>>> #define COMPAT
>>>>>> @@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
>>>>>> mcs->compat_call.args[i] = mcs->call.args[i];
>>>>>> }
>>>>>>
>>>>>> -DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
>>>>>> #define multicall_entry compat_multicall_entry
>>>>>> #define multicall_entry_t multicall_entry_compat_t
>>>>>> #define do_multicall_call compat_multicall_call
>>>>> Jürgen's patch doesn't have any change to this file, and I'm afraid I
>>>>> also don't see how these adjustments are related here. The commit
>>>>> message sadly also doesn't help ...
>>>> The changes are very necessary to split it out of Juergen's series.
>>>>
>>>> Without the adjustment, the correction of compat_platform_op()'s guest
>>>> handle type from void to compat_platform_op_t doesn't compile.
>>> Interesting. That's quite far from obvious in this context, so clarifying
>>> the purpose in the description would seem helpful.
>>>
>>> Coming back to the syncing with v3: Was this change the reason then why
>>> you did drop my R-b?
>>
>> My porting of this patch is a non-trivial modification from Juergen's
>> version, and not eligible to retain any tags.
>>
>> I thought I'd discussed this, but I appear to have missed it from both
>> versions of the series. Sorry.
>>
>> Either way. It's exactly the same purpose as before, but modified to
>> compile in isolation.
>
> I see. I'm under the impression though that parts were effectively
> present elsewhere in Jürgen's series. Perhaps it would have been easier
> if his series (at least up to the point to which you need it here)
> would (long) have gone in already. What it looks to be blocked on are
> two or three Arm acks and an x86 ack on patch 1 (which I've expressed
> I'm not entirely happy about, and hence I'm not going to either ack or
> nack it).
The main blocking point currently is that Julien would like me to let
all hypercalls return an int (apart from the ones which really need
a long). This will affect lot of common code and I need to have more
time for that endeavor.
An alternative to that would be to not rework the Arm side of the
hypercall logic.
Juergen
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3149 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-17 10:20 ` Jan Beulich
2022-02-17 10:34 ` Juergen Gross
@ 2022-02-21 19:21 ` Andrew Cooper
2022-02-22 8:41 ` Jan Beulich
1 sibling, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-21 19:21 UTC (permalink / raw)
To: Jan Beulich; +Cc: Juergen Gross, Xen-devel
On 17/02/2022 10:20, Jan Beulich wrote:
> On 16.02.2022 23:17, Andrew Cooper wrote:
>> On 14/02/2022 13:56, Jan Beulich wrote:
>>> On 14.02.2022 14:50, Andrew Cooper wrote:
>>>> On 14/02/2022 13:33, Jan Beulich wrote:
>>>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>>
>>>>>> When running as pv-shim the hypercall is modified today in order to
>>>>>> replace the functions for __HYPERVISOR_event_channel_op and
>>>>>> __HYPERVISOR_grant_table_op hypercalls.
>>>>>>
>>>>>> Change this to call the related functions from the normal handlers
>>>>>> instead when running as shim. The performance implications are not
>>>>>> really relevant, as a normal production hypervisor will not be
>>>>>> configured to support shim mode, so the related calls will be dropped
>>>>>> due to optimization of the compiler.
>>>>>>
>>>>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>>>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>>>>> isn't being built.
>>>>>>
>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>>>>> changes but having a stale version sent two months later isn't very
>>>>> nice.
>>>> I did resync. What do you think is missing?
>>> A few likely() / unlikely() as far as I could see.
>> Oh those two. I appear to have forgot to email.
>>
>> They're wrong - observe they're in an ifndef block, not an ifdef block.
> I don't see how the (unrelated) #ifndef matters here: The #ifndef
> is about grant table availability. The two likely() are about
> running as shim. I'm of the firm opinion that a binary built
> without PV_SHIM_EXCLUSIVE is far more likely to be used as a bare
> metal hypervisor. And for a PV_SHIM_EXCLUSIVE hypervisor the
> conditions are constant anyway, and hence the unlikely() has no
> effect.
>
> And if your way should really be followed, why did you deem the two
> unlikely() in do_event_channel_op() and do_grant_table_op() okay?
Because those are at least not incorrect. (I still think we have far
too many annotations, and I doubt they're all helpful.)
The gnttab stubs in the !GNTTAB case exist strictly for compile tests
(there's no such thing as a production build of Xen without grant
tables) and PV_SHIM_EXCLUSIVE builds.
Code layout only matters for cases where we're executing code, which is
the PV Shim case, at which point the condition is constant and doesn't
generate a branch.
A compiler ought to raise a warning on finding that __builtin_expect()
has a constant parameter, because it's a nop in one case, and
demonstrably false in the other.
As for the function in question, the compiled result is an unconditional
tailcall to pv_shim_grant_table_op.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table
2022-02-21 19:21 ` Andrew Cooper
@ 2022-02-22 8:41 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-22 8:41 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 21.02.2022 20:21, Andrew Cooper wrote:
> On 17/02/2022 10:20, Jan Beulich wrote:
>> On 16.02.2022 23:17, Andrew Cooper wrote:
>>> On 14/02/2022 13:56, Jan Beulich wrote:
>>>> On 14.02.2022 14:50, Andrew Cooper wrote:
>>>>> On 14/02/2022 13:33, Jan Beulich wrote:
>>>>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>>>>> From: Juergen Gross <jgross@suse.com>
>>>>>>>
>>>>>>> When running as pv-shim the hypercall is modified today in order to
>>>>>>> replace the functions for __HYPERVISOR_event_channel_op and
>>>>>>> __HYPERVISOR_grant_table_op hypercalls.
>>>>>>>
>>>>>>> Change this to call the related functions from the normal handlers
>>>>>>> instead when running as shim. The performance implications are not
>>>>>>> really relevant, as a normal production hypervisor will not be
>>>>>>> configured to support shim mode, so the related calls will be dropped
>>>>>>> due to optimization of the compiler.
>>>>>>>
>>>>>>> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
>>>>>>> wrapper do_grant_table_op() needed, as in this case grant_table.c
>>>>>>> isn't being built.
>>>>>>>
>>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>>>> I don't think you sync-ed this with Jürgen's v3. There were only minor
>>>>>> changes but having a stale version sent two months later isn't very
>>>>>> nice.
>>>>> I did resync. What do you think is missing?
>>>> A few likely() / unlikely() as far as I could see.
>>> Oh those two. I appear to have forgot to email.
>>>
>>> They're wrong - observe they're in an ifndef block, not an ifdef block.
>> I don't see how the (unrelated) #ifndef matters here: The #ifndef
>> is about grant table availability. The two likely() are about
>> running as shim. I'm of the firm opinion that a binary built
>> without PV_SHIM_EXCLUSIVE is far more likely to be used as a bare
>> metal hypervisor. And for a PV_SHIM_EXCLUSIVE hypervisor the
>> conditions are constant anyway, and hence the unlikely() has no
>> effect.
>>
>> And if your way should really be followed, why did you deem the two
>> unlikely() in do_event_channel_op() and do_grant_table_op() okay?
>
> Because those are at least not incorrect. (I still think we have far
> too many annotations, and I doubt they're all helpful.)
I'm afraid I'm completely lost then as to the (consistent) model you
want to see used. When putting them side by side:
@@ -3543,6 +3547,11 @@ do_grant_table_op(
long rc;
unsigned int opaque_in = cmd & GNTTABOP_ARG_MASK, opaque_out = 0;
+#ifdef CONFIG_PV_SHIM
+ if ( unlikely(pv_shim) )
+ return pv_shim_grant_table_op(cmd, uop, count);
+#endif
+
if ( (int)count < 0 )
return -EINVAL;
and
long do_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
unsigned int count)
{
if ( likely(!pv_shim) )
return -ENOSYS;
return pv_shim_grant_table_op(cmd, uop, count);
}
it is (to me at least) quite obvious that the unlikely() and likely()
both express _exactly_ the same thing.
> The gnttab stubs in the !GNTTAB case exist strictly for compile tests
> (there's no such thing as a production build of Xen without grant
> tables) and PV_SHIM_EXCLUSIVE builds.
If certain options (or combinations thereof) are not supposed to be
used in practice, why would we allow them in the first place? Sadly
the commit introducing the GRANT_TABLE option supplies no justification
at all as to _why_ this control is/was wanted.
> Code layout only matters for cases where we're executing code, which is
> the PV Shim case, at which point the condition is constant and doesn't
> generate a branch.
>
> A compiler ought to raise a warning on finding that __builtin_expect()
> has a constant parameter, because it's a nop in one case, and
> demonstrably false in the other.
Such a warning would imo be as appropriate (or not) as one for e.g.
"if ( 1 )".
> As for the function in question, the compiled result is an unconditional
> tailcall to pv_shim_grant_table_op.
For the "#define pv_shim true" case, I suppose. And then yes, as expected
for this particular case.
Anyway - once again in the interest of not blocking progress of the full
series, and once again contrary to my intention to not ever again do so
in situations like this one:
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(with at least half a sentence said on the seemingly unrelated changes)
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 05/70] x86: Don't use the hypercall table for calling compat hypercalls
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (3 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 04/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 06/70] x86: Introduce support for CET-IBT Andrew Cooper
` (66 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Juergen Gross
From: Juergen Gross <jgross@suse.com>
Today the *_op_compat hypercalls call the modern handler functions by
using the entries from the hypercall table. This is resulting in a
not needed indirect function call which can be avoided by using the
correct handler function directly. This is basically a revert of
commit 1252e282311734 ("86/pv: Export pv_hypercall_table[] rather
than working around it in several ways"), which reasoning no longer
applies, as shim no longer modifies the hypercall table.
The hypercall table can now be made static as there is no external
reference to it any longer.
Commit 834cb8761051f7 ("x86/PV32: fix physdev_op_compat handling")
can be reverted, too, as using the direct call of the correct handler
is already handled fine without that patch.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/compat.c | 14 ++++----------
xen/arch/x86/include/asm/hypercall.h | 8 --------
xen/arch/x86/pv/hypercall.c | 9 ++++++++-
xen/arch/x86/x86_64/compat.c | 1 -
4 files changed, 12 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/compat.c b/xen/arch/x86/compat.c
index 58b202f701d5..939b449dec29 100644
--- a/xen/arch/x86/compat.c
+++ b/xen/arch/x86/compat.c
@@ -17,14 +17,12 @@ typedef long ret_t;
/* Legacy hypercall (as of 0x00030202). */
ret_t do_physdev_op_compat(XEN_GUEST_HANDLE_PARAM(physdev_op_t) uop)
{
- typeof(do_physdev_op) *fn =
- (void *)pv_hypercall_table[__HYPERVISOR_physdev_op].native;
struct physdev_op op;
if ( unlikely(copy_from_guest(&op, uop, 1) != 0) )
return -EFAULT;
- return fn(op.cmd, guest_handle_from_ptr(&uop.p->u, void));
+ return do_physdev_op(op.cmd, guest_handle_from_ptr(&uop.p->u, void));
}
#ifndef COMPAT
@@ -32,14 +30,11 @@ ret_t do_physdev_op_compat(XEN_GUEST_HANDLE_PARAM(physdev_op_t) uop)
/* Legacy hypercall (as of 0x00030101). */
long do_sched_op_compat(int cmd, unsigned long arg)
{
- typeof(do_sched_op) *fn =
- (void *)pv_hypercall_table[__HYPERVISOR_sched_op].native;
-
switch ( cmd )
{
case SCHEDOP_yield:
case SCHEDOP_block:
- return fn(cmd, guest_handle_from_ptr(NULL, void));
+ return do_sched_op(cmd, guest_handle_from_ptr(NULL, void));
case SCHEDOP_shutdown:
TRACE_3D(TRC_SCHED_SHUTDOWN,
@@ -57,8 +52,6 @@ long do_sched_op_compat(int cmd, unsigned long arg)
/* Legacy hypercall (as of 0x00030202). */
long do_event_channel_op_compat(XEN_GUEST_HANDLE_PARAM(evtchn_op_t) uop)
{
- typeof(do_event_channel_op) *fn =
- (void *)pv_hypercall_table[__HYPERVISOR_event_channel_op].native;
struct evtchn_op op;
if ( unlikely(copy_from_guest(&op, uop, 1) != 0) )
@@ -76,7 +69,8 @@ long do_event_channel_op_compat(XEN_GUEST_HANDLE_PARAM(evtchn_op_t) uop)
case EVTCHNOP_bind_ipi:
case EVTCHNOP_bind_vcpu:
case EVTCHNOP_unmask:
- return fn(op.cmd, guest_handle_from_ptr(&uop.p->u, void));
+ return do_event_channel_op(op.cmd,
+ guest_handle_from_ptr(&uop.p->u, void));
default:
return -ENOSYS;
diff --git a/xen/arch/x86/include/asm/hypercall.h b/xen/arch/x86/include/asm/hypercall.h
index f004824f16b6..eb3aed3bf70e 100644
--- a/xen/arch/x86/include/asm/hypercall.h
+++ b/xen/arch/x86/include/asm/hypercall.h
@@ -16,13 +16,6 @@ typedef unsigned long hypercall_fn_t(
unsigned long, unsigned long);
typedef struct {
- hypercall_fn_t *native;
-#ifdef CONFIG_PV32
- hypercall_fn_t *compat;
-#endif
-} pv_hypercall_table_t;
-
-typedef struct {
uint8_t native;
#ifdef CONFIG_COMPAT
uint8_t compat;
@@ -32,7 +25,6 @@ typedef struct {
extern const hypercall_args_t hypercall_args_table[NR_hypercalls];
#ifdef CONFIG_PV
-extern const pv_hypercall_table_t pv_hypercall_table[];
void pv_hypercall(struct cpu_user_regs *regs);
#endif
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index 50cd219c18fc..e8fbee7bbbc0 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -27,6 +27,13 @@
#include <asm/multicall.h>
#include <irq_vectors.h>
+typedef struct {
+ hypercall_fn_t *native;
+#ifdef CONFIG_PV32
+ hypercall_fn_t *compat;
+#endif
+} pv_hypercall_table_t;
+
#ifdef CONFIG_PV32
#define HYPERCALL(x) \
[ __HYPERVISOR_ ## x ] = { (hypercall_fn_t *) do_ ## x, \
@@ -42,7 +49,7 @@
#define do_arch_1 paging_domctl_continuation
-const pv_hypercall_table_t pv_hypercall_table[] = {
+static const pv_hypercall_table_t pv_hypercall_table[] = {
COMPAT_CALL(set_trap_table),
HYPERCALL(mmu_update),
COMPAT_CALL(set_gdt),
diff --git a/xen/arch/x86/x86_64/compat.c b/xen/arch/x86/x86_64/compat.c
index fcbc1cc0d780..0e4c71f2aa17 100644
--- a/xen/arch/x86/x86_64/compat.c
+++ b/xen/arch/x86/x86_64/compat.c
@@ -12,7 +12,6 @@ EMIT_FILE;
#define physdev_op_t physdev_op_compat_t
#define do_physdev_op compat_physdev_op
#define do_physdev_op_compat(x) compat_physdev_op_compat(_##x)
-#define native compat
#define COMPAT
#define _XEN_GUEST_HANDLE(t) XEN_GUEST_HANDLE(t)
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 06/70] x86: Introduce support for CET-IBT
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (4 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 05/70] x86: Don't use the hypercall table for calling compat hypercalls Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-15 14:01 ` Jan Beulich
2022-02-14 12:50 ` [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
` (65 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
CET Indirect Branch Tracking is a hardware feature designed to provide
forward-edge control flow integrity, protecting against jump/call oriented
programming.
IBT requires the placement of ENDBR{32,64} instructions at the target of every
indirect call/jmp, and every entrypoint.
However, the default -fcf-protection=branch places an ENDBR on every function
which far more than necessary, and reduces the quantity of protection
afforded. Therefore, we use manual placement using the cf_check attribute.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
Clang/LLVM support for -mmanual-endbr is in progress:
https://reviews.llvm.org/D118355
v2:
* Correct CONFIG_HAS_CC_CET_IBT to CONFIG_XEN_IBT in some places
* Move cf_check compatibility into tools/tests/x86_emulator/x86-emulate.h
---
Config.mk | 1 -
tools/firmware/Makefile | 2 ++
tools/libs/guest/xg_dom_decompress_unsafe.h | 2 ++
tools/tests/x86_emulator/x86-emulate.h | 2 ++
xen/arch/x86/Kconfig | 17 +++++++++++++++++
xen/arch/x86/arch.mk | 6 ++++++
xen/arch/x86/include/asm/asm-defns.h | 6 ++++++
xen/arch/x86/include/asm/cpufeature.h | 1 +
xen/arch/x86/include/asm/cpufeatures.h | 1 +
xen/include/xen/compiler.h | 6 ++++++
10 files changed, 43 insertions(+), 1 deletion(-)
diff --git a/Config.mk b/Config.mk
index 95c053212ec3..f56f7dc33468 100644
--- a/Config.mk
+++ b/Config.mk
@@ -190,7 +190,6 @@ APPEND_CFLAGS += $(foreach i, $(APPEND_INCLUDES), -I$(i))
EMBEDDED_EXTRA_CFLAGS := -nopie -fno-stack-protector -fno-stack-protector-all
EMBEDDED_EXTRA_CFLAGS += -fno-exceptions -fno-asynchronous-unwind-tables
-EMBEDDED_EXTRA_CFLAGS += -fcf-protection=none
XEN_EXTFILES_URL ?= http://xenbits.xen.org/xen-extfiles
# All the files at that location were downloaded from elsewhere on
diff --git a/tools/firmware/Makefile b/tools/firmware/Makefile
index 345037b93b7f..53ed4f161edb 100644
--- a/tools/firmware/Makefile
+++ b/tools/firmware/Makefile
@@ -6,6 +6,8 @@ TARGET := hvmloader/hvmloader
INST_DIR := $(DESTDIR)$(XENFIRMWAREDIR)
DEBG_DIR := $(DESTDIR)$(DEBUG_DIR)$(XENFIRMWAREDIR)
+EMBEDDED_EXTRA_CFLAGS += -fcf-protection=none
+
SUBDIRS-y :=
SUBDIRS-$(CONFIG_OVMF) += ovmf-dir
SUBDIRS-$(CONFIG_SEABIOS) += seabios-dir
diff --git a/tools/libs/guest/xg_dom_decompress_unsafe.h b/tools/libs/guest/xg_dom_decompress_unsafe.h
index 4e0bf23aa587..3bce0cfefb88 100644
--- a/tools/libs/guest/xg_dom_decompress_unsafe.h
+++ b/tools/libs/guest/xg_dom_decompress_unsafe.h
@@ -8,6 +8,8 @@ typedef int decompress_fn(unsigned char *inbuf, unsigned int len,
void (*error)(const char *x));
#endif
+#define cf_check
+
int xc_dom_decompress_unsafe(
decompress_fn fn, struct xc_dom_image *dom, void **blob, size_t *size)
__attribute__((visibility("internal")));
diff --git a/tools/tests/x86_emulator/x86-emulate.h b/tools/tests/x86_emulator/x86-emulate.h
index 7f60ef9e89ba..c6819a417d05 100644
--- a/tools/tests/x86_emulator/x86-emulate.h
+++ b/tools/tests/x86_emulator/x86-emulate.h
@@ -54,6 +54,8 @@
#define likely(x) __builtin_expect(!!(x), true)
#define unlikely(x) __builtin_expect(!!(x), false)
+#define cf_check
+
#define container_of(ptr, type, member) ({ \
typeof(((type *)0)->member) *mptr__ = (ptr); \
(type *)((char *)mptr__ - offsetof(type, member)); \
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index b4abfca46f6a..8b7ad0145b29 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -39,6 +39,11 @@ config HAS_AS_CET_SS
# binutils >= 2.29 or LLVM >= 6
def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
+config HAS_CC_CET_IBT
+ # GCC >= 9 and binutils >= 2.29
+ # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
+ def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
+
menu "Architecture Features"
source "arch/Kconfig"
@@ -124,6 +129,18 @@ config XEN_SHSTK
When CET-SS is active, 32bit PV guests cannot be used. Backwards
compatiblity can be provided via the PV Shim mechanism.
+config XEN_IBT
+ bool "Supervisor Indirect Branch Tracking"
+ depends on HAS_CC_CET_IBT
+ default y
+ help
+ Control-flow Enforcement Technology (CET) is a set of features in
+ hardware designed to combat Return-oriented Programming (ROP, also
+ call/jump COP/JOP) attacks. Indirect Branch Tracking is one CET
+ feature designed to provide function pointer protection.
+
+ This option arranges for Xen to use CET-IBT for its own protection.
+
config SHADOW_PAGING
bool "Shadow Paging"
default !PV_SHIM_EXCLUSIVE
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index fa7cf3844362..8b88f0240e85 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -47,6 +47,12 @@ CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch=thunk-extern
CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch-register
CFLAGS-$(CONFIG_INDIRECT_THUNK) += -fno-jump-tables
+ifdef CONFIG_XEN_IBT
+CFLAGS += -fcf-protection=branch -mmanual-endbr
+else
+$(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
+endif
+
# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
# this to be overridden elsewhere.
$(call cc-option-add,CFLAGS_stack_boundary,CC,-mpreferred-stack-boundary=3)
diff --git a/xen/arch/x86/include/asm/asm-defns.h b/xen/arch/x86/include/asm/asm-defns.h
index 505f39ad5f76..8bd9007731d5 100644
--- a/xen/arch/x86/include/asm/asm-defns.h
+++ b/xen/arch/x86/include/asm/asm-defns.h
@@ -57,6 +57,12 @@
INDIRECT_BRANCH jmp \arg
.endm
+#ifdef CONFIG_XEN_IBT
+# define ENDBR64 endbr64
+#else
+# define ENDBR64
+#endif
+
.macro guest_access_mask_ptr ptr:req, scratch1:req, scratch2:req
#if defined(CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS)
/*
diff --git a/xen/arch/x86/include/asm/cpufeature.h b/xen/arch/x86/include/asm/cpufeature.h
index a0ab6d7d78ea..f2c6f255ace9 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -152,6 +152,7 @@
#define cpu_has_nscb boot_cpu_has(X86_FEATURE_NSCB)
#define cpu_has_xen_lbr boot_cpu_has(X86_FEATURE_XEN_LBR)
#define cpu_has_xen_shstk boot_cpu_has(X86_FEATURE_XEN_SHSTK)
+#define cpu_has_xen_ibt boot_cpu_has(X86_FEATURE_XEN_IBT)
#define cpu_has_msr_tsc_aux (cpu_has_rdtscp || cpu_has_rdpid)
diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/asm/cpufeatures.h
index b10154fc44bb..7413febd7ad8 100644
--- a/xen/arch/x86/include/asm/cpufeatures.h
+++ b/xen/arch/x86/include/asm/cpufeatures.h
@@ -39,6 +39,7 @@ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */
XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */
XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */
XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */
+XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect Branch Tracking */
/* Bug words follow the synthetic words. */
#define X86_NR_BUG 1
diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
index 696c7eb89e4c..933aec09a92d 100644
--- a/xen/include/xen/compiler.h
+++ b/xen/include/xen/compiler.h
@@ -37,6 +37,12 @@
# define nocall
#endif
+#ifdef CONFIG_XEN_IBT
+# define cf_check __attribute__((__cf_check__))
+#else
+# define cf_check
+#endif
+
#if (!defined(__clang__) && (__GNUC__ == 4) && (__GNUC_MINOR__ < 5))
#define unreachable() do {} while (1)
#else
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 06/70] x86: Introduce support for CET-IBT
2022-02-14 12:50 ` [PATCH v2 06/70] x86: Introduce support for CET-IBT Andrew Cooper
@ 2022-02-15 14:01 ` Jan Beulich
2022-02-16 21:54 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 14:01 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 14.02.2022 13:50, Andrew Cooper wrote:
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -39,6 +39,11 @@ config HAS_AS_CET_SS
> # binutils >= 2.29 or LLVM >= 6
> def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
>
> +config HAS_CC_CET_IBT
> + # GCC >= 9 and binutils >= 2.29
> + # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
> + def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
At the top of asm-defns.h we have a number of similarly operand-less
instructions expressed via .macro expanding to .byte. I don't see why
we couldn't do so here as well, eliminating the need for the
$(as-instr ...). In fact ...
> --- a/xen/arch/x86/include/asm/asm-defns.h
> +++ b/xen/arch/x86/include/asm/asm-defns.h
> @@ -57,6 +57,12 @@
> INDIRECT_BRANCH jmp \arg
> .endm
>
> +#ifdef CONFIG_XEN_IBT
> +# define ENDBR64 endbr64
> +#else
> +# define ENDBR64
> +#endif
... it could also be this macro which ends up conditionally empty,
but would then want expressing as an assembler macro. Albeit no, the
lower case form would probably still be needed to deal with compiler
emitted insns, as the compiler doesn't appear to make recognition of
the command line option dependent on the underlying assembler's
capabilities.
> --- a/xen/arch/x86/include/asm/cpufeatures.h
> +++ b/xen/arch/x86/include/asm/cpufeatures.h
> @@ -39,6 +39,7 @@ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */
> XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */
> XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */
> XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */
> +XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect Branch Tracking */
Is a feature flag actually warranted here, rather than a single
global boolean? You don't key any alternatives patching to this
bit, unlike was the case for XEN_SHSTK. And the only consumer is
cpu_has_xen_ibt, expanding to the boot CPU's instance of the bit.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 06/70] x86: Introduce support for CET-IBT
2022-02-15 14:01 ` Jan Beulich
@ 2022-02-16 21:54 ` Andrew Cooper
2022-02-17 11:32 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 21:54 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 15/02/2022 14:01, Jan Beulich wrote:
> On 14.02.2022 13:50, Andrew Cooper wrote:
>> --- a/xen/arch/x86/Kconfig
>> +++ b/xen/arch/x86/Kconfig
>> @@ -39,6 +39,11 @@ config HAS_AS_CET_SS
>> # binutils >= 2.29 or LLVM >= 6
>> def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
>>
>> +config HAS_CC_CET_IBT
>> + # GCC >= 9 and binutils >= 2.29
>> + # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
>> + def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
> At the top of asm-defns.h we have a number of similarly operand-less
> instructions expressed via .macro expanding to .byte. I don't see why
> we couldn't do so here as well, eliminating the need for the
> $(as-instr ...). In fact ...
>
>> --- a/xen/arch/x86/include/asm/asm-defns.h
>> +++ b/xen/arch/x86/include/asm/asm-defns.h
>> @@ -57,6 +57,12 @@
>> INDIRECT_BRANCH jmp \arg
>> .endm
>>
>> +#ifdef CONFIG_XEN_IBT
>> +# define ENDBR64 endbr64
>> +#else
>> +# define ENDBR64
>> +#endif
> ... it could also be this macro which ends up conditionally empty,
> but would then want expressing as an assembler macro. Albeit no, the
> lower case form would probably still be needed to deal with compiler
> emitted insns, as the compiler doesn't appear to make recognition of
> the command line option dependent on the underlying assembler's
> capabilities.
$(as-instr) isn't only for endbr64. It also for the notrack prefix,
which GCC does emit for any function pointer call laundered through void
* even when everything was otherwise cf_check.
It's another area where treating the cf_check-ness as type-checking
falls down, and created some very weird build failures until I figured
out that Juergen's "Don't use the hypercall table for calling compat
hypercalls" really did need to be a prerequisite.
CET-IBT toolchain support is 3 years old already, and I don't think
there is any value attempting to support a developer mixing a new GCC
and ancient binutils.
>> --- a/xen/arch/x86/include/asm/cpufeatures.h
>> +++ b/xen/arch/x86/include/asm/cpufeatures.h
>> @@ -39,6 +39,7 @@ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */
>> XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */
>> XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */
>> XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */
>> +XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect Branch Tracking */
> Is a feature flag actually warranted here, rather than a single
> global boolean? You don't key any alternatives patching to this
> bit, unlike was the case for XEN_SHSTK. And the only consumer is
> cpu_has_xen_ibt, expanding to the boot CPU's instance of the bit.
These are just bits. They long predate alternatives finding a
convenient use for the form, and are 8 times more compact than a global
boolean, with better locality of reference too.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 06/70] x86: Introduce support for CET-IBT
2022-02-16 21:54 ` Andrew Cooper
@ 2022-02-17 11:32 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 11:32 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 16.02.2022 22:54, Andrew Cooper wrote:
> On 15/02/2022 14:01, Jan Beulich wrote:
>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/Kconfig
>>> +++ b/xen/arch/x86/Kconfig
>>> @@ -39,6 +39,11 @@ config HAS_AS_CET_SS
>>> # binutils >= 2.29 or LLVM >= 6
>>> def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
>>>
>>> +config HAS_CC_CET_IBT
>>> + # GCC >= 9 and binutils >= 2.29
>>> + # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
>>> + def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
>> At the top of asm-defns.h we have a number of similarly operand-less
>> instructions expressed via .macro expanding to .byte. I don't see why
>> we couldn't do so here as well, eliminating the need for the
>> $(as-instr ...). In fact ...
>>
>>> --- a/xen/arch/x86/include/asm/asm-defns.h
>>> +++ b/xen/arch/x86/include/asm/asm-defns.h
>>> @@ -57,6 +57,12 @@
>>> INDIRECT_BRANCH jmp \arg
>>> .endm
>>>
>>> +#ifdef CONFIG_XEN_IBT
>>> +# define ENDBR64 endbr64
>>> +#else
>>> +# define ENDBR64
>>> +#endif
>> ... it could also be this macro which ends up conditionally empty,
>> but would then want expressing as an assembler macro. Albeit no, the
>> lower case form would probably still be needed to deal with compiler
>> emitted insns, as the compiler doesn't appear to make recognition of
>> the command line option dependent on the underlying assembler's
>> capabilities.
>
> $(as-instr) isn't only for endbr64. It also for the notrack prefix,
> which GCC does emit for any function pointer call laundered through void
> * even when everything was otherwise cf_check.
>
> It's another area where treating the cf_check-ness as type-checking
> falls down, and created some very weird build failures until I figured
> out that Juergen's "Don't use the hypercall table for calling compat
> hypercalls" really did need to be a prerequisite.
Oh, I see. I can certainly accept this as a reason, but half a sentence
mentioning this would be nice in the description.
>>> --- a/xen/arch/x86/include/asm/cpufeatures.h
>>> +++ b/xen/arch/x86/include/asm/cpufeatures.h
>>> @@ -39,6 +39,7 @@ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */
>>> XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */
>>> XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */
>>> XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */
>>> +XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect Branch Tracking */
>> Is a feature flag actually warranted here, rather than a single
>> global boolean? You don't key any alternatives patching to this
>> bit, unlike was the case for XEN_SHSTK. And the only consumer is
>> cpu_has_xen_ibt, expanding to the boot CPU's instance of the bit.
>
> These are just bits. They long predate alternatives finding a
> convenient use for the form, and are 8 times more compact than a global
> boolean, with better locality of reference too.
Well, I disagree (and we were here before, so I think you could have
predicted such a comment coming back). We should never have cloned this
directly from Linux. It's only bits, but with enough CPUs it sums up.
We shouldn't duplicate data when we need only a single instance (and
when no other infrastructure, like alternative patching, depends on it).
Last time you put me in a situation like this one, I told myself to not
ack such changes anymore, but here I am again - in the interest of not
being blamed for blocking this series:
Acked-by: Jan Beulich <jbeulich@suse.com>
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (5 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 06/70] x86: Introduce support for CET-IBT Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-15 15:12 ` Jan Beulich
2022-02-14 12:50 ` [PATCH v2 08/70] xen: CFI hardening for x86 hypercalls Andrew Cooper
` (64 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Marek Marczykowski-Górecki, Andrew Cooper
From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Embedded endbr64 instructions mark legal indirect branches as far as the CPU
is concerned, which aren't legal as far as the logic is concerned.
When CET-IBT is active, check for embedded byte sequences. Example failures
look like:
Fail: Found 2 embedded endbr64 instructions
0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v2:
* New
---
xen/arch/x86/Makefile | 3 ++
xen/tools/check-endbr.sh | 76 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 79 insertions(+)
create mode 100755 xen/tools/check-endbr.sh
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 9fc884813cb5..f15a984aacc2 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -155,6 +155,9 @@ $(TARGET)-syms: prelink.o xen.lds
$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
$(@D)/.$(@F).1.o -o $@
+ifeq ($(CONFIG_XEN_IBT),y)
+ $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
+endif
$(NM) -pa --format=sysv $(@D)/$(@F) \
| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
>$(@D)/$(@F).map
diff --git a/xen/tools/check-endbr.sh b/xen/tools/check-endbr.sh
new file mode 100755
index 000000000000..3d96e02bdf93
--- /dev/null
+++ b/xen/tools/check-endbr.sh
@@ -0,0 +1,76 @@
+#!/bin/sh
+
+#
+# Usage ./$0 xen-syms
+#
+
+set -e
+
+OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
+OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
+
+D=$(mktemp -d)
+trap "rm -rf $D" EXIT
+
+TEXT_BIN=$D/xen-syms.text
+VALID=$D/valid-addrs
+ALL=$D/all-addrs
+BAD=$D/bad-addrs
+
+#
+# First, look for all the valid endbr64 instructions.
+# A worst-case disassembly, viewed through cat -A, may look like:
+#
+# ffff82d040337bd4 <endbr64>:$
+# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
+# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
+# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
+#
+# Want to grab the address of endbr64 instructions only, ignoring function
+# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
+# number of trailing spaces before the end of the line.
+#
+${OBJDUMP} -d | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
+
+#
+# Second, look for any endbr64 byte sequence
+# This has a couple of complications:
+#
+# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
+# the grep offset to be from the start of .text.
+#
+# 2) AWK can't add 64bit integers, because internally all numbers are doubles.
+# When the upper bits are set, the exponents worth of precision is lost in
+# the lower bits, rounding integers to the nearest 4k.
+#
+# Instead, use the fact that Xen's .text is within a 1G aligned region, and
+# split the VMA in half so AWK's numeric addition is only working on 32 bit
+# numbers, which don't lose precision.
+#
+eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
+
+${OBJCOPY} -O binary $TEXT_BIN
+grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
+ awk -F':' '{printf "%s%x\n", "'$vma_hi'", strtonum(0x'$vma_lo') + $1}' > $ALL
+
+# Wait for $VALID to become complete
+wait
+
+# Sanity check $VALID and $ALL, in case the string parsing bitrots
+val_sz=$(stat -c '%s' $VALID)
+all_sz=$(stat -c '%s' $ALL)
+[ "$val_sz" -eq 0 ] && { echo "Error: Empty valid-addrs" >&2; exit 1; }
+[ "$all_sz" -eq 0 ] && { echo "Error: Empty all-addrs" >&2; exit 1; }
+[ "$all_sz" -lt "$val_sz" ] && { echo "Error: More valid-addrs than all-addrs" >&2; exit 1; }
+
+# $BAD = $ALL - $VALID
+join -v 2 $VALID $ALL > $BAD
+nr_bad=$(wc -l < $BAD)
+
+# Success
+[ "$nr_bad" -eq 0 ] && exit 0
+
+# Failure
+echo "Fail: Found ${nr_bad} embedded endbr64 instructions" >&2
+addr2line -afip -e $1 < $BAD >&2
+exit 1
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions
2022-02-14 12:50 ` [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
@ 2022-02-15 15:12 ` Jan Beulich
2022-02-15 17:52 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 15:12 UTC (permalink / raw)
To: Andrew Cooper, Marek Marczykowski-Górecki; +Cc: Xen-devel
On 14.02.2022 13:50, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>
> Embedded endbr64 instructions mark legal indirect branches as far as the CPU
> is concerned, which aren't legal as far as the logic is concerned.
I think it would help if it was clarified what "embedded" actually means
here.
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -155,6 +155,9 @@ $(TARGET)-syms: prelink.o xen.lds
> $(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
> $(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
> $(@D)/.$(@F).1.o -o $@
> +ifeq ($(CONFIG_XEN_IBT),y)
> + $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
> +endif
> $(NM) -pa --format=sysv $(@D)/$(@F) \
> | $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
> >$(@D)/$(@F).map
The same wants doing on xen.efi, I guess?
> --- /dev/null
> +++ b/xen/tools/check-endbr.sh
> @@ -0,0 +1,76 @@
> +#!/bin/sh
> +
> +#
> +# Usage ./$0 xen-syms
> +#
> +
> +set -e
> +
> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
> +
> +D=$(mktemp -d)
> +trap "rm -rf $D" EXIT
> +
> +TEXT_BIN=$D/xen-syms.text
> +VALID=$D/valid-addrs
> +ALL=$D/all-addrs
> +BAD=$D/bad-addrs
> +
> +#
> +# First, look for all the valid endbr64 instructions.
> +# A worst-case disassembly, viewed through cat -A, may look like:
> +#
> +# ffff82d040337bd4 <endbr64>:$
> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
> +#
> +# Want to grab the address of endbr64 instructions only, ignoring function
> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
> +# number of trailing spaces before the end of the line.
> +#
> +${OBJDUMP} -d | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
Since you look at only .text the risk of the disassembler coming
out of sync with the actual instruction stream is lower than when
32- and 16-bit code was also part of what is disassembled, but it's
not zero. Any zero-padding inserted anywhere by the linker can
result in an immediately following ENDBR to be missed (because
sequences of zeros resemble 2-byte insns). While this risk may be
acceptable, I think it wants mentioning at least in the description,
maybe even at the top of the script (where one would likely look
first after it spitting out an error).
Do you perhaps want to also pass -w to objdump, to eliminate the
risk of getting confused by split lines?
> +#
> +# Second, look for any endbr64 byte sequence
> +# This has a couple of complications:
> +#
> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
> +# the grep offset to be from the start of .text.
> +#
> +# 2) AWK can't add 64bit integers, because internally all numbers are doubles.
> +# When the upper bits are set, the exponents worth of precision is lost in
> +# the lower bits, rounding integers to the nearest 4k.
> +#
> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
> +# numbers, which don't lose precision.
> +#
> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
> +
> +${OBJCOPY} -O binary $TEXT_BIN
> +grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
> + awk -F':' '{printf "%s%x\n", "'$vma_hi'", strtonum(0x'$vma_lo') + $1}' > $ALL
None of the three options passed to grep look to be standardized.
Is this going to cause problems on non-Linux systems? Should this
checking perhaps be put behind a separate Kconfig option?
> +# Wait for $VALID to become complete
> +wait
> +
> +# Sanity check $VALID and $ALL, in case the string parsing bitrots
> +val_sz=$(stat -c '%s' $VALID)
> +all_sz=$(stat -c '%s' $ALL)
> +[ "$val_sz" -eq 0 ] && { echo "Error: Empty valid-addrs" >&2; exit 1; }
> +[ "$all_sz" -eq 0 ] && { echo "Error: Empty all-addrs" >&2; exit 1; }
> +[ "$all_sz" -lt "$val_sz" ] && { echo "Error: More valid-addrs than all-addrs" >&2; exit 1; }
> +
> +# $BAD = $ALL - $VALID
> +join -v 2 $VALID $ALL > $BAD
> +nr_bad=$(wc -l < $BAD)
> +
> +# Success
> +[ "$nr_bad" -eq 0 ] && exit 0
> +
> +# Failure
> +echo "Fail: Found ${nr_bad} embedded endbr64 instructions" >&2
> +addr2line -afip -e $1 < $BAD >&2
There probably also wants to be an ADDR2LINE variable then. If
one overrides objdump and objcopy, one would likely want/need to
override this one as well.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions
2022-02-15 15:12 ` Jan Beulich
@ 2022-02-15 17:52 ` Andrew Cooper
2022-02-16 8:41 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-15 17:52 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper, Marek Marczykowski-Górecki; +Cc: Xen-devel
On 15/02/2022 15:12, Jan Beulich wrote:
> On 14.02.2022 13:50, Andrew Cooper wrote:
>> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>
>> Embedded endbr64 instructions mark legal indirect branches as far as the CPU
>> is concerned, which aren't legal as far as the logic is concerned.
> I think it would help if it was clarified what "embedded" actually means
> here.
Oh yeah, that's lost a bit of context now I've split it out of the patch
introducing endbr.h
>
>> --- a/xen/arch/x86/Makefile
>> +++ b/xen/arch/x86/Makefile
>> @@ -155,6 +155,9 @@ $(TARGET)-syms: prelink.o xen.lds
>> $(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
>> $(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
>> $(@D)/.$(@F).1.o -o $@
>> +ifeq ($(CONFIG_XEN_IBT),y)
>> + $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
>> +endif
>> $(NM) -pa --format=sysv $(@D)/$(@F) \
>> | $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
>> >$(@D)/$(@F).map
> The same wants doing on xen.efi, I guess?
Probably.
>
>> --- /dev/null
>> +++ b/xen/tools/check-endbr.sh
>> @@ -0,0 +1,76 @@
>> +#!/bin/sh
>> +
>> +#
>> +# Usage ./$0 xen-syms
>> +#
>> +
>> +set -e
>> +
>> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
>> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
>> +
>> +D=$(mktemp -d)
>> +trap "rm -rf $D" EXIT
>> +
>> +TEXT_BIN=$D/xen-syms.text
>> +VALID=$D/valid-addrs
>> +ALL=$D/all-addrs
>> +BAD=$D/bad-addrs
>> +
>> +#
>> +# First, look for all the valid endbr64 instructions.
>> +# A worst-case disassembly, viewed through cat -A, may look like:
>> +#
>> +# ffff82d040337bd4 <endbr64>:$
>> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
>> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
>> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
>> +#
>> +# Want to grab the address of endbr64 instructions only, ignoring function
>> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
>> +# number of trailing spaces before the end of the line.
>> +#
>> +${OBJDUMP} -d | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
> Since you look at only .text the risk of the disassembler coming
> out of sync with the actual instruction stream is lower than when
> 32- and 16-bit code was also part of what is disassembled, but it's
> not zero.
I'm not sure that we have any interesting non-64bit code at all in .text.
_start is technically 32bit but is mode-invariant as far as decoding goes.
The kexec trampoline is here too, but when I dust off my cleanup patch,
there will no longer be data or mode-dependent things to disassemble.
Everything else I can think of is in .init.text.
> Any zero-padding inserted anywhere by the linker can
> result in an immediately following ENDBR to be missed (because
> sequences of zeros resemble 2-byte insns).
I'm not sure this is a problem. This pass is looking for everything
that objdump thinks is a legal endbr64 instruction, and it splits at labels.
Only the hand-written stubs can legitimately have an endbr64 without a
symbol pointing at it.
We also don't have any 0 padding. It's specified as 0x90 in the linker
file, although I've been debating switching this to 0xcc for a while now
already.
> While this risk may be
> acceptable, I think it wants mentioning at least in the description,
> maybe even at the top of the script (where one would likely look
> first after it spitting out an error).
>
> Do you perhaps want to also pass -w to objdump, to eliminate the
> risk of getting confused by split lines?
I think that's probably a good move irrespective. This particular pipe
is the longest single task in the script which is why I backgrounded it
while the second scan occurs. -w means fewer lines so hopefully a minor
speedup.
>> +#
>> +# Second, look for any endbr64 byte sequence
>> +# This has a couple of complications:
>> +#
>> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
>> +# the grep offset to be from the start of .text.
>> +#
>> +# 2) AWK can't add 64bit integers, because internally all numbers are doubles.
>> +# When the upper bits are set, the exponents worth of precision is lost in
>> +# the lower bits, rounding integers to the nearest 4k.
>> +#
>> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
>> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
>> +# numbers, which don't lose precision.
>> +#
>> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
>> +
>> +${OBJCOPY} -O binary $TEXT_BIN
>> +grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
>> + awk -F':' '{printf "%s%x\n", "'$vma_hi'", strtonum(0x'$vma_lo') + $1}' > $ALL
> None of the three options passed to grep look to be standardized.
> Is this going to cause problems on non-Linux systems? Should this
> checking perhaps be put behind a separate Kconfig option?
CI says that FreeBSD is entirely happy, while Alpine Linux isn't. This
is because Alpine has busybox's grep unless you install the GNU grep
package, and I'm doing a fix to our container.
My plan to fix this is to just declare a "grep capable of binary
searching" a conditional build requirement for Xen. I don't think this
is onerous, and there no other plausible alternatives here.
The other option is to detect the absence of support an skip the check.
It is after all a defence in depth scheme, and anything liable to cause
a problem would be caught in CI anyway.
>> +# Wait for $VALID to become complete
>> +wait
>> +
>> +# Sanity check $VALID and $ALL, in case the string parsing bitrots
>> +val_sz=$(stat -c '%s' $VALID)
>> +all_sz=$(stat -c '%s' $ALL)
>> +[ "$val_sz" -eq 0 ] && { echo "Error: Empty valid-addrs" >&2; exit 1; }
>> +[ "$all_sz" -eq 0 ] && { echo "Error: Empty all-addrs" >&2; exit 1; }
>> +[ "$all_sz" -lt "$val_sz" ] && { echo "Error: More valid-addrs than all-addrs" >&2; exit 1; }
>> +
>> +# $BAD = $ALL - $VALID
>> +join -v 2 $VALID $ALL > $BAD
>> +nr_bad=$(wc -l < $BAD)
>> +
>> +# Success
>> +[ "$nr_bad" -eq 0 ] && exit 0
>> +
>> +# Failure
>> +echo "Fail: Found ${nr_bad} embedded endbr64 instructions" >&2
>> +addr2line -afip -e $1 < $BAD >&2
> There probably also wants to be an ADDR2LINE variable then. If
> one overrides objdump and objcopy, one would likely want/need to
> override this one as well.
Ah yes. Will fix.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions
2022-02-15 17:52 ` Andrew Cooper
@ 2022-02-16 8:41 ` Jan Beulich
2022-02-16 11:55 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-16 8:41 UTC (permalink / raw)
To: Andrew Cooper, Andrew Cooper; +Cc: Xen-devel, Marek Marczykowski-Górecki
On 15.02.2022 18:52, Andrew Cooper wrote:
> On 15/02/2022 15:12, Jan Beulich wrote:
>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>> --- /dev/null
>>> +++ b/xen/tools/check-endbr.sh
>>> @@ -0,0 +1,76 @@
>>> +#!/bin/sh
>>> +
>>> +#
>>> +# Usage ./$0 xen-syms
>>> +#
>>> +
>>> +set -e
>>> +
>>> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
>>> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
>>> +
>>> +D=$(mktemp -d)
>>> +trap "rm -rf $D" EXIT
>>> +
>>> +TEXT_BIN=$D/xen-syms.text
>>> +VALID=$D/valid-addrs
>>> +ALL=$D/all-addrs
>>> +BAD=$D/bad-addrs
>>> +
>>> +#
>>> +# First, look for all the valid endbr64 instructions.
>>> +# A worst-case disassembly, viewed through cat -A, may look like:
>>> +#
>>> +# ffff82d040337bd4 <endbr64>:$
>>> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
>>> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
>>> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
>>> +#
>>> +# Want to grab the address of endbr64 instructions only, ignoring function
>>> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
>>> +# number of trailing spaces before the end of the line.
>>> +#
>>> +${OBJDUMP} -d | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
>> Since you look at only .text the risk of the disassembler coming
>> out of sync with the actual instruction stream is lower than when
>> 32- and 16-bit code was also part of what is disassembled, but it's
>> not zero.
>
> I'm not sure that we have any interesting non-64bit code at all in .text.
>
> _start is technically 32bit but is mode-invariant as far as decoding goes.
>
> The kexec trampoline is here too, but when I dust off my cleanup patch,
> there will no longer be data or mode-dependent things to disassemble.
>
> Everything else I can think of is in .init.text.
>
>> Any zero-padding inserted anywhere by the linker can
>> result in an immediately following ENDBR to be missed (because
>> sequences of zeros resemble 2-byte insns).
>
> I'm not sure this is a problem. This pass is looking for everything
> that objdump thinks is a legal endbr64 instruction, and it splits at labels.
Oh, right - I did miss the splitting at labels aspect. Hopefully
objdump is really consistent with this.
> Only the hand-written stubs can legitimately have an endbr64 without a
> symbol pointing at it.
>
> We also don't have any 0 padding. It's specified as 0x90 in the linker
> file, although I've been debating switching this to 0xcc for a while now
> already.
The linker script comes into play only in the final linking step.
Prior "ld -r" could easily have inserted other padding.
>>> +#
>>> +# Second, look for any endbr64 byte sequence
>>> +# This has a couple of complications:
>>> +#
>>> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
>>> +# the grep offset to be from the start of .text.
>>> +#
>>> +# 2) AWK can't add 64bit integers, because internally all numbers are doubles.
>>> +# When the upper bits are set, the exponents worth of precision is lost in
>>> +# the lower bits, rounding integers to the nearest 4k.
>>> +#
>>> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
>>> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
>>> +# numbers, which don't lose precision.
>>> +#
>>> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
>>> +
>>> +${OBJCOPY} -O binary $TEXT_BIN
>>> +grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
>>> + awk -F':' '{printf "%s%x\n", "'$vma_hi'", strtonum(0x'$vma_lo') + $1}' > $ALL
>> None of the three options passed to grep look to be standardized.
>> Is this going to cause problems on non-Linux systems? Should this
>> checking perhaps be put behind a separate Kconfig option?
>
> CI says that FreeBSD is entirely happy, while Alpine Linux isn't. This
> is because Alpine has busybox's grep unless you install the GNU grep
> package, and I'm doing a fix to our container.
>
> My plan to fix this is to just declare a "grep capable of binary
> searching" a conditional build requirement for Xen. I don't think this
> is onerous, and there no other plausible alternatives here.
>
> The other option is to detect the absence of support an skip the check.
> It is after all a defence in depth scheme, and anything liable to cause
> a problem would be caught in CI anyway.
I'd favor the latter approach (but I wouldn't mind the conditional build
requirement, if you and others deem that better), with a warning issued
when the check can't be performed. I have to admit that I didn't expect
there would be no simple and standardized binary search tool on Unix-es.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions
2022-02-16 8:41 ` Jan Beulich
@ 2022-02-16 11:55 ` Andrew Cooper
0 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 11:55 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper; +Cc: Xen-devel, Marek Marczykowski-Górecki
On 16/02/2022 08:41, Jan Beulich wrote:
>>> Any zero-padding inserted anywhere by the linker can
>>> result in an immediately following ENDBR to be missed (because
>>> sequences of zeros resemble 2-byte insns).
>> I'm not sure this is a problem. This pass is looking for everything
>> that objdump thinks is a legal endbr64 instruction, and it splits at labels.
> Oh, right - I did miss the splitting at labels aspect. Hopefully
> objdump is really consistent with this.
Certainly appears to be in my experience.
>>>> +#
>>>> +# Second, look for any endbr64 byte sequence
>>>> +# This has a couple of complications:
>>>> +#
>>>> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
>>>> +# the grep offset to be from the start of .text.
>>>> +#
>>>> +# 2) AWK can't add 64bit integers, because internally all numbers are doubles.
>>>> +# When the upper bits are set, the exponents worth of precision is lost in
>>>> +# the lower bits, rounding integers to the nearest 4k.
>>>> +#
>>>> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
>>>> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
>>>> +# numbers, which don't lose precision.
>>>> +#
>>>> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
>>>> +
>>>> +${OBJCOPY} -O binary $TEXT_BIN
>>>> +grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
>>>> + awk -F':' '{printf "%s%x\n", "'$vma_hi'", strtonum(0x'$vma_lo') + $1}' > $ALL
>>> None of the three options passed to grep look to be standardized.
>>> Is this going to cause problems on non-Linux systems? Should this
>>> checking perhaps be put behind a separate Kconfig option?
>> CI says that FreeBSD is entirely happy, while Alpine Linux isn't. This
>> is because Alpine has busybox's grep unless you install the GNU grep
>> package, and I'm doing a fix to our container.
>>
>> My plan to fix this is to just declare a "grep capable of binary
>> searching" a conditional build requirement for Xen. I don't think this
>> is onerous, and there no other plausible alternatives here.
>>
>> The other option is to detect the absence of support an skip the check.
>> It is after all a defence in depth scheme, and anything liable to cause
>> a problem would be caught in CI anyway.
> I'd favor the latter approach (but I wouldn't mind the conditional build
> requirement, if you and others deem that better), with a warning issued
> when the check can't be performed. I have to admit that I didn't expect
> there would be no simple and standardized binary search tool on Unix-es.
Ok, so lets do this:
1) This script gets a check for $(grep -aob) and emits a warning to
stderr but exits 0. This lets people using IBT know that something was
missing.
2) Optional build dependency of `grep -aob` for Xen. (just a tweak to
README)
3) Update the alpine containers to not miss out.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 08/70] xen: CFI hardening for x86 hypercalls
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (6 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 07/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 09/70] xen: CFI hardening for custom_param() Andrew Cooper
` (63 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/compat.c | 7 ++--
xen/arch/x86/cpu/mcheck/mce.c | 2 +-
xen/arch/x86/cpu/vpmu.c | 3 +-
xen/arch/x86/hvm/dm.c | 5 ++-
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/hvm/hypercall.c | 5 +--
xen/arch/x86/include/asm/hypercall.h | 69 +++++++++++++++++++-----------------
xen/arch/x86/include/asm/paging.h | 2 +-
xen/arch/x86/mm.c | 23 ++++++------
xen/arch/x86/mm/paging.c | 3 +-
xen/arch/x86/physdev.c | 2 +-
xen/arch/x86/platform_hypercall.c | 3 +-
xen/arch/x86/pv/callback.c | 25 +++++++------
xen/arch/x86/pv/descriptor-tables.c | 14 ++++----
xen/arch/x86/pv/iret.c | 4 +--
xen/arch/x86/pv/misc-hypercalls.c | 10 +++---
xen/arch/x86/pv/shim.c | 8 ++---
xen/arch/x86/x86_64/compat/mm.c | 7 ++--
xen/common/argo.c | 4 +--
xen/common/compat/domain.c | 3 +-
xen/common/compat/grant_table.c | 5 ++-
xen/common/compat/kernel.c | 2 +-
xen/common/compat/memory.c | 3 +-
xen/common/dm.c | 6 ++--
xen/common/domain.c | 5 +--
xen/common/domctl.c | 2 +-
xen/common/event_channel.c | 2 +-
xen/common/grant_table.c | 2 +-
xen/common/hypfs.c | 6 ++--
xen/common/kernel.c | 2 +-
xen/common/kexec.c | 4 +--
xen/common/memory.c | 2 +-
xen/common/multicall.c | 2 +-
xen/common/sched/compat.c | 2 +-
xen/common/sched/core.c | 4 +--
xen/common/sysctl.c | 2 +-
xen/common/xenoprof.c | 2 +-
xen/drivers/char/console.c | 4 +--
xen/include/xen/hypercall.h | 69 ++++++++++++++++++------------------
xen/xsm/xsm_core.c | 4 +--
40 files changed, 169 insertions(+), 162 deletions(-)
diff --git a/xen/arch/x86/compat.c b/xen/arch/x86/compat.c
index 939b449dec29..28281a262a3c 100644
--- a/xen/arch/x86/compat.c
+++ b/xen/arch/x86/compat.c
@@ -15,7 +15,7 @@ typedef long ret_t;
#endif
/* Legacy hypercall (as of 0x00030202). */
-ret_t do_physdev_op_compat(XEN_GUEST_HANDLE_PARAM(physdev_op_t) uop)
+ret_t cf_check do_physdev_op_compat(XEN_GUEST_HANDLE_PARAM(physdev_op_t) uop)
{
struct physdev_op op;
@@ -28,7 +28,7 @@ ret_t do_physdev_op_compat(XEN_GUEST_HANDLE_PARAM(physdev_op_t) uop)
#ifndef COMPAT
/* Legacy hypercall (as of 0x00030101). */
-long do_sched_op_compat(int cmd, unsigned long arg)
+long cf_check do_sched_op_compat(int cmd, unsigned long arg)
{
switch ( cmd )
{
@@ -50,7 +50,8 @@ long do_sched_op_compat(int cmd, unsigned long arg)
}
/* Legacy hypercall (as of 0x00030202). */
-long do_event_channel_op_compat(XEN_GUEST_HANDLE_PARAM(evtchn_op_t) uop)
+long cf_check do_event_channel_op_compat(
+ XEN_GUEST_HANDLE_PARAM(evtchn_op_t) uop)
{
struct evtchn_op op;
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index 7f433343bca3..eae08caa07cd 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -1351,7 +1351,7 @@ CHECK_mcinfo_recovery;
# endif /* CONFIG_COMPAT */
/* Machine Check Architecture Hypercall */
-long do_mca(XEN_GUEST_HANDLE_PARAM(xen_mc_t) u_xen_mc)
+long cf_check do_mca(XEN_GUEST_HANDLE_PARAM(xen_mc_t) u_xen_mc)
{
long ret = 0;
struct xen_mc curop, *op = &curop;
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index 8ec4547bedf4..598291f4ece9 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -671,7 +671,8 @@ void vpmu_dump(struct vcpu *v)
alternative_vcall(vpmu_ops.arch_vpmu_dump, v);
}
-long do_xenpmu_op(unsigned int op, XEN_GUEST_HANDLE_PARAM(xen_pmu_params_t) arg)
+long cf_check do_xenpmu_op(
+ unsigned int op, XEN_GUEST_HANDLE_PARAM(xen_pmu_params_t) arg)
{
int ret;
struct vcpu *curr;
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index b60b9f3364a8..d80975efcf45 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -654,9 +654,8 @@ CHECK_dm_op_relocate_memory;
CHECK_dm_op_pin_memory_cacheattr;
CHECK_dm_op_nr_vcpus;
-int compat_dm_op(domid_t domid,
- unsigned int nr_bufs,
- XEN_GUEST_HANDLE_PARAM(void) bufs)
+int cf_check compat_dm_op(
+ domid_t domid, unsigned int nr_bufs, XEN_GUEST_HANDLE_PARAM(void) bufs)
{
struct dmop_args args;
unsigned int i;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index c4ddb8607d9c..4e685c1b0c1b 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5015,7 +5015,7 @@ static int hvmop_get_mem_type(
return rc;
}
-long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
+long cf_check do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
{
long rc = 0;
diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 384724ec41a3..030243810e48 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -31,7 +31,8 @@
#include <public/hvm/hvm_op.h>
#include <public/hvm/params.h>
-static long hvm_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+static long cf_check hvm_memory_op(
+ unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
long rc;
@@ -51,7 +52,7 @@ static long hvm_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
}
#ifdef CONFIG_GRANT_TABLE
-static long hvm_grant_table_op(
+static long cf_check hvm_grant_table_op(
unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count)
{
switch ( cmd )
diff --git a/xen/arch/x86/include/asm/hypercall.h b/xen/arch/x86/include/asm/hypercall.h
index eb3aed3bf70e..16d8418b5774 100644
--- a/xen/arch/x86/include/asm/hypercall.h
+++ b/xen/arch/x86/include/asm/hypercall.h
@@ -38,97 +38,97 @@ void pv_ring3_init_hypercall_page(void *ptr);
*/
#define MMU_UPDATE_PREEMPTED (~(~0U>>1))
-extern long
+extern long cf_check
do_event_channel_op_compat(
XEN_GUEST_HANDLE_PARAM(evtchn_op_t) uop);
/* Legacy hypercall (as of 0x00030202). */
-extern long do_physdev_op_compat(
+extern long cf_check do_physdev_op_compat(
XEN_GUEST_HANDLE(physdev_op_t) uop);
/* Legacy hypercall (as of 0x00030101). */
-extern long do_sched_op_compat(
+extern long cf_check do_sched_op_compat(
int cmd, unsigned long arg);
-extern long
+extern long cf_check
do_set_trap_table(
XEN_GUEST_HANDLE_PARAM(const_trap_info_t) traps);
-extern long
+extern long cf_check
do_mmu_update(
XEN_GUEST_HANDLE_PARAM(mmu_update_t) ureqs,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(uint) pdone,
unsigned int foreigndom);
-extern long
+extern long cf_check
do_set_gdt(
XEN_GUEST_HANDLE_PARAM(xen_ulong_t) frame_list,
unsigned int entries);
-extern long
+extern long cf_check
do_stack_switch(
unsigned long ss,
unsigned long esp);
-extern long
+extern long cf_check
do_fpu_taskswitch(
int set);
-extern long
+extern long cf_check
do_set_debugreg(
int reg,
unsigned long value);
-extern unsigned long
+extern unsigned long cf_check
do_get_debugreg(
int reg);
-extern long
+extern long cf_check
do_update_descriptor(
uint64_t gaddr, seg_desc_t desc);
-extern long
+extern long cf_check
do_mca(XEN_GUEST_HANDLE_PARAM(xen_mc_t) u_xen_mc);
-extern long
+extern long cf_check
do_update_va_mapping(
unsigned long va,
u64 val64,
unsigned long flags);
-extern long
+extern long cf_check
do_physdev_op(
int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_update_va_mapping_otherdomain(
unsigned long va,
u64 val64,
unsigned long flags,
domid_t domid);
-extern long
+extern long cf_check
do_mmuext_op(
XEN_GUEST_HANDLE_PARAM(mmuext_op_t) uops,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(uint) pdone,
unsigned int foreigndom);
-extern long do_callback_op(
+extern long cf_check do_callback_op(
int cmd, XEN_GUEST_HANDLE_PARAM(const_void) arg);
-extern unsigned long
+extern unsigned long cf_check
do_iret(
void);
-extern long
+extern long cf_check
do_set_callbacks(
unsigned long event_address,
unsigned long failsafe_address,
unsigned long syscall_address);
-extern long
+extern long cf_check
do_set_segment_base(
unsigned int which,
unsigned long base);
@@ -139,7 +139,7 @@ do_set_segment_base(
#include <compat/physdev.h>
#include <compat/platform.h>
-extern int
+extern int cf_check
compat_physdev_op(
int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
@@ -148,44 +148,47 @@ extern int
arch_compat_vcpu_op(
int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int compat_mmuext_op(
+extern int cf_check compat_mmuext_op(
XEN_GUEST_HANDLE_PARAM(void) arg,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(uint) pdone,
unsigned int foreigndom);
DEFINE_XEN_GUEST_HANDLE(compat_platform_op_t);
-extern int compat_platform_op(
+extern int cf_check compat_platform_op(
XEN_GUEST_HANDLE_PARAM(compat_platform_op_t) u_xenpf_op);
-extern long compat_callback_op(
+extern long cf_check compat_callback_op(
int cmd, XEN_GUEST_HANDLE(void) arg);
-extern int compat_update_va_mapping(
+extern int cf_check compat_update_va_mapping(
unsigned int va, u32 lo, u32 hi, unsigned int flags);
-extern int compat_update_va_mapping_otherdomain(
+extern int cf_check compat_update_va_mapping_otherdomain(
unsigned int va, u32 lo, u32 hi, unsigned int flags, domid_t domid);
DEFINE_XEN_GUEST_HANDLE(trap_info_compat_t);
-extern int compat_set_trap_table(XEN_GUEST_HANDLE(trap_info_compat_t) traps);
+extern int cf_check compat_set_trap_table(
+ XEN_GUEST_HANDLE(trap_info_compat_t) traps);
-extern int compat_set_gdt(
+extern int cf_check compat_set_gdt(
XEN_GUEST_HANDLE_PARAM(uint) frame_list, unsigned int entries);
-extern int compat_update_descriptor(
+extern int cf_check compat_update_descriptor(
u32 pa_lo, u32 pa_hi, u32 desc_lo, u32 desc_hi);
-extern unsigned int compat_iret(void);
+extern unsigned int cf_check compat_iret(void);
-extern int compat_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
+extern int cf_check compat_nmi_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long compat_set_callbacks(
+extern long cf_check compat_set_callbacks(
unsigned long event_selector, unsigned long event_address,
unsigned long failsafe_selector, unsigned long failsafe_address);
DEFINE_XEN_GUEST_HANDLE(physdev_op_compat_t);
-extern int compat_physdev_op_compat(XEN_GUEST_HANDLE(physdev_op_compat_t) uop);
+extern int cf_check compat_physdev_op_compat(
+ XEN_GUEST_HANDLE(physdev_op_compat_t) uop);
#endif /* CONFIG_COMPAT */
diff --git a/xen/arch/x86/include/asm/paging.h b/xen/arch/x86/include/asm/paging.h
index 2ddcfb022c7b..f0b4efc66e2d 100644
--- a/xen/arch/x86/include/asm/paging.h
+++ b/xen/arch/x86/include/asm/paging.h
@@ -235,7 +235,7 @@ int paging_domctl(struct domain *d, struct xen_domctl_shadow_op *sc,
bool_t resuming);
/* Helper hypercall for dealing with continuations. */
-long paging_domctl_continuation(XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
+long cf_check paging_domctl_continuation(XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
/* Call when destroying a vcpu/domain */
void paging_vcpu_teardown(struct vcpu *v);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index b80e4ab9c564..6484c3b97bbc 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -3377,7 +3377,7 @@ static int vcpumask_to_pcpumask(
}
}
-long do_mmuext_op(
+long cf_check do_mmuext_op(
XEN_GUEST_HANDLE_PARAM(mmuext_op_t) uops,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(uint) pdone,
@@ -3916,7 +3916,7 @@ long do_mmuext_op(
return rc;
}
-long do_mmu_update(
+long cf_check do_mmu_update(
XEN_GUEST_HANDLE_PARAM(mmu_update_t) ureqs,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(uint) pdone,
@@ -4501,8 +4501,8 @@ static int __do_update_va_mapping(
return rc;
}
-long do_update_va_mapping(unsigned long va, u64 val64,
- unsigned long flags)
+long cf_check do_update_va_mapping(
+ unsigned long va, u64 val64, unsigned long flags)
{
int rc = __do_update_va_mapping(va, val64, flags, current->domain);
@@ -4513,9 +4513,8 @@ long do_update_va_mapping(unsigned long va, u64 val64,
return rc;
}
-long do_update_va_mapping_otherdomain(unsigned long va, u64 val64,
- unsigned long flags,
- domid_t domid)
+long cf_check do_update_va_mapping_otherdomain(
+ unsigned long va, u64 val64, unsigned long flags, domid_t domid)
{
struct domain *pg_owner;
int rc;
@@ -4537,8 +4536,8 @@ long do_update_va_mapping_otherdomain(unsigned long va, u64 val64,
#endif /* CONFIG_PV */
#ifdef CONFIG_PV32
-int compat_update_va_mapping(unsigned int va, uint32_t lo, uint32_t hi,
- unsigned int flags)
+int cf_check compat_update_va_mapping(
+ unsigned int va, uint32_t lo, uint32_t hi, unsigned int flags)
{
int rc = __do_update_va_mapping(va, ((uint64_t)hi << 32) | lo,
flags, current->domain);
@@ -4550,9 +4549,9 @@ int compat_update_va_mapping(unsigned int va, uint32_t lo, uint32_t hi,
return rc;
}
-int compat_update_va_mapping_otherdomain(unsigned int va,
- uint32_t lo, uint32_t hi,
- unsigned int flags, domid_t domid)
+int cf_check compat_update_va_mapping_otherdomain(
+ unsigned int va, uint32_t lo, uint32_t hi, unsigned int flags,
+ domid_t domid)
{
struct domain *pg_owner;
int rc;
diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c
index dc3a4a0b4bb1..1f0b94ad212a 100644
--- a/xen/arch/x86/mm/paging.c
+++ b/xen/arch/x86/mm/paging.c
@@ -759,7 +759,8 @@ int paging_domctl(struct domain *d, struct xen_domctl_shadow_op *sc,
return shadow_domctl(d, sc, u_domctl);
}
-long paging_domctl_continuation(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+long cf_check paging_domctl_continuation(
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
{
struct xen_domctl op;
struct domain *d;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index ea38be8b797a..2ddcf44f33a4 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -174,7 +174,7 @@ int physdev_unmap_pirq(domid_t domid, int pirq)
}
#endif /* COMPAT */
-ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+ret_t cf_check do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int irq;
ret_t ret;
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index bf4090c94201..84566bbfaa3d 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -213,7 +213,8 @@ void resource_access(void *info)
}
#endif
-ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
+ret_t cf_check do_platform_op(
+ XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
{
ret_t ret;
struct xen_platform_op curop, *op = &curop;
diff --git a/xen/arch/x86/pv/callback.c b/xen/arch/x86/pv/callback.c
index 42a6aa0831b7..55148c7f9e50 100644
--- a/xen/arch/x86/pv/callback.c
+++ b/xen/arch/x86/pv/callback.c
@@ -140,7 +140,7 @@ static long unregister_guest_callback(struct callback_unregister *unreg)
return ret;
}
-long do_callback_op(int cmd, XEN_GUEST_HANDLE_PARAM(const_void) arg)
+long cf_check do_callback_op(int cmd, XEN_GUEST_HANDLE_PARAM(const_void) arg)
{
long ret;
@@ -178,9 +178,9 @@ long do_callback_op(int cmd, XEN_GUEST_HANDLE_PARAM(const_void) arg)
return ret;
}
-long do_set_callbacks(unsigned long event_address,
- unsigned long failsafe_address,
- unsigned long syscall_address)
+long cf_check do_set_callbacks(
+ unsigned long event_address, unsigned long failsafe_address,
+ unsigned long syscall_address)
{
struct callback_register event = {
.type = CALLBACKTYPE_event,
@@ -283,7 +283,7 @@ static long compat_unregister_guest_callback(
return ret;
}
-long compat_callback_op(int cmd, XEN_GUEST_HANDLE(void) arg)
+long cf_check compat_callback_op(int cmd, XEN_GUEST_HANDLE(void) arg)
{
long ret;
@@ -321,10 +321,9 @@ long compat_callback_op(int cmd, XEN_GUEST_HANDLE(void) arg)
return ret;
}
-long compat_set_callbacks(unsigned long event_selector,
- unsigned long event_address,
- unsigned long failsafe_selector,
- unsigned long failsafe_address)
+long cf_check compat_set_callbacks(
+ unsigned long event_selector, unsigned long event_address,
+ unsigned long failsafe_selector, unsigned long failsafe_address)
{
struct compat_callback_register event = {
.type = CALLBACKTYPE_event,
@@ -349,7 +348,7 @@ long compat_set_callbacks(unsigned long event_selector,
#endif /* CONFIG_PV32 */
-long do_set_trap_table(XEN_GUEST_HANDLE_PARAM(const_trap_info_t) traps)
+long cf_check do_set_trap_table(XEN_GUEST_HANDLE_PARAM(const_trap_info_t) traps)
{
struct trap_info cur;
struct vcpu *curr = current;
@@ -395,7 +394,7 @@ long do_set_trap_table(XEN_GUEST_HANDLE_PARAM(const_trap_info_t) traps)
}
#ifdef CONFIG_PV32
-int compat_set_trap_table(XEN_GUEST_HANDLE(trap_info_compat_t) traps)
+int cf_check compat_set_trap_table(XEN_GUEST_HANDLE(trap_info_compat_t) traps)
{
struct vcpu *curr = current;
struct compat_trap_info cur;
@@ -438,7 +437,7 @@ int compat_set_trap_table(XEN_GUEST_HANDLE(trap_info_compat_t) traps)
}
#endif
-long do_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long cf_check do_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct xennmi_callback cb;
long rc = 0;
@@ -464,7 +463,7 @@ long do_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
}
#ifdef CONFIG_PV32
-int compat_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+int cf_check compat_nmi_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct compat_nmi_callback cb;
int rc = 0;
diff --git a/xen/arch/x86/pv/descriptor-tables.c b/xen/arch/x86/pv/descriptor-tables.c
index 5e847044009b..653a61d0b518 100644
--- a/xen/arch/x86/pv/descriptor-tables.c
+++ b/xen/arch/x86/pv/descriptor-tables.c
@@ -124,8 +124,8 @@ int pv_set_gdt(struct vcpu *v, const unsigned long frames[],
return -EINVAL;
}
-long do_set_gdt(XEN_GUEST_HANDLE_PARAM(xen_ulong_t) frame_list,
- unsigned int entries)
+long cf_check do_set_gdt(
+ XEN_GUEST_HANDLE_PARAM(xen_ulong_t) frame_list, unsigned int entries)
{
unsigned int nr_frames = DIV_ROUND_UP(entries, 512);
unsigned long frames[16];
@@ -151,8 +151,8 @@ long do_set_gdt(XEN_GUEST_HANDLE_PARAM(xen_ulong_t) frame_list,
#ifdef CONFIG_PV32
-int compat_set_gdt(XEN_GUEST_HANDLE_PARAM(uint) frame_list,
- unsigned int entries)
+int cf_check compat_set_gdt(
+ XEN_GUEST_HANDLE_PARAM(uint) frame_list, unsigned int entries)
{
struct vcpu *curr = current;
unsigned int i, nr_frames = DIV_ROUND_UP(entries, 512);
@@ -187,8 +187,8 @@ int compat_set_gdt(XEN_GUEST_HANDLE_PARAM(uint) frame_list,
return ret;
}
-int compat_update_descriptor(uint32_t pa_lo, uint32_t pa_hi,
- uint32_t desc_lo, uint32_t desc_hi)
+int cf_check compat_update_descriptor(
+ uint32_t pa_lo, uint32_t pa_hi, uint32_t desc_lo, uint32_t desc_hi)
{
seg_desc_t d;
@@ -299,7 +299,7 @@ int validate_segdesc_page(struct page_info *page)
return i == 512 ? 0 : -EINVAL;
}
-long do_update_descriptor(uint64_t gaddr, seg_desc_t d)
+long cf_check do_update_descriptor(uint64_t gaddr, seg_desc_t d)
{
struct domain *currd = current->domain;
gfn_t gfn = gaddr_to_gfn(gaddr);
diff --git a/xen/arch/x86/pv/iret.c b/xen/arch/x86/pv/iret.c
index 29a2f7cc452b..dd2965d8f08d 100644
--- a/xen/arch/x86/pv/iret.c
+++ b/xen/arch/x86/pv/iret.c
@@ -48,7 +48,7 @@ static void async_exception_cleanup(struct vcpu *curr)
curr->arch.async_exception_state(trap).old_mask;
}
-unsigned long do_iret(void)
+unsigned long cf_check do_iret(void)
{
struct cpu_user_regs *regs = guest_cpu_user_regs();
struct iret_context iret_saved;
@@ -105,7 +105,7 @@ unsigned long do_iret(void)
}
#ifdef CONFIG_PV32
-unsigned int compat_iret(void)
+unsigned int cf_check compat_iret(void)
{
struct cpu_user_regs *regs = guest_cpu_user_regs();
struct vcpu *v = current;
diff --git a/xen/arch/x86/pv/misc-hypercalls.c b/xen/arch/x86/pv/misc-hypercalls.c
index 5dade2472687..5649aaab4436 100644
--- a/xen/arch/x86/pv/misc-hypercalls.c
+++ b/xen/arch/x86/pv/misc-hypercalls.c
@@ -23,12 +23,12 @@
#include <asm/debugreg.h>
-long do_set_debugreg(int reg, unsigned long value)
+long cf_check do_set_debugreg(int reg, unsigned long value)
{
return set_debugreg(current, reg, value);
}
-unsigned long do_get_debugreg(int reg)
+unsigned long cf_check do_get_debugreg(int reg)
{
unsigned long val;
int res = x86emul_read_dr(reg, &val, NULL);
@@ -36,7 +36,7 @@ unsigned long do_get_debugreg(int reg)
return res == X86EMUL_OKAY ? val : -ENODEV;
}
-long do_fpu_taskswitch(int set)
+long cf_check do_fpu_taskswitch(int set)
{
struct vcpu *v = current;
@@ -171,7 +171,7 @@ long set_debugreg(struct vcpu *v, unsigned int reg, unsigned long value)
return 0;
}
-long do_stack_switch(unsigned long ss, unsigned long esp)
+long cf_check do_stack_switch(unsigned long ss, unsigned long esp)
{
fixup_guest_stack_selector(current->domain, ss);
current->arch.pv.kernel_ss = ss;
@@ -180,7 +180,7 @@ long do_stack_switch(unsigned long ss, unsigned long esp)
return 0;
}
-long do_set_segment_base(unsigned int which, unsigned long base)
+long cf_check do_set_segment_base(unsigned int which, unsigned long base)
{
struct vcpu *v = current;
long ret = 0;
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index 7e891fe2f7a4..4c710ad8913f 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -824,8 +824,8 @@ long pv_shim_grant_table_op(unsigned int cmd,
#ifndef CONFIG_GRANT_TABLE
/* Thin wrapper(s) needed. */
-long do_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count)
+long cf_check do_grant_table_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count)
{
if ( !pv_shim )
return -ENOSYS;
@@ -834,8 +834,8 @@ long do_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
}
#ifdef CONFIG_PV32
-int compat_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count)
+int cf_check compat_grant_table_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count)
{
if ( !pv_shim )
return -ENOSYS;
diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c
index 215e96aba032..b3da8fafbb0c 100644
--- a/xen/arch/x86/x86_64/compat/mm.c
+++ b/xen/arch/x86/x86_64/compat/mm.c
@@ -176,10 +176,9 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
#ifdef CONFIG_PV
DEFINE_XEN_GUEST_HANDLE(mmuext_op_compat_t);
-int compat_mmuext_op(XEN_GUEST_HANDLE_PARAM(void) arg,
- unsigned int count,
- XEN_GUEST_HANDLE_PARAM(uint) pdone,
- unsigned int foreigndom)
+int cf_check compat_mmuext_op(
+ XEN_GUEST_HANDLE_PARAM(void) arg, unsigned int count,
+ XEN_GUEST_HANDLE_PARAM(uint) pdone, unsigned int foreigndom)
{
unsigned int i, preempt_mask;
int rc = 0;
diff --git a/xen/common/argo.c b/xen/common/argo.c
index eaea7ba8885a..1448faf65731 100644
--- a/xen/common/argo.c
+++ b/xen/common/argo.c
@@ -2069,7 +2069,7 @@ sendv(struct domain *src_d, xen_argo_addr_t *src_addr,
return ( ret < 0 ) ? ret : len;
}
-long
+long cf_check
do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long raw_arg3,
unsigned long raw_arg4)
@@ -2207,7 +2207,7 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
}
#ifdef CONFIG_COMPAT
-long
+long cf_check
compat_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,
XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
unsigned long arg4)
diff --git a/xen/common/compat/domain.c b/xen/common/compat/domain.c
index 98b8c15cea7f..afae27eeba02 100644
--- a/xen/common/compat/domain.c
+++ b/xen/common/compat/domain.c
@@ -38,7 +38,8 @@ CHECK_vcpu_hvm_context;
#endif
-int compat_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
+int cf_check compat_vcpu_op(
+ int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct domain *d = current->domain;
struct vcpu *v;
diff --git a/xen/common/compat/grant_table.c b/xen/common/compat/grant_table.c
index ff1d678f01c7..c6199e89188d 100644
--- a/xen/common/compat/grant_table.c
+++ b/xen/common/compat/grant_table.c
@@ -55,9 +55,8 @@ CHECK_gnttab_swap_grant_ref;
CHECK_gnttab_cache_flush;
#undef xen_gnttab_cache_flush
-int compat_grant_table_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(void) cmp_uop,
- unsigned int count)
+int cf_check compat_grant_table_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) cmp_uop, unsigned int count)
{
int rc = 0;
unsigned int i, cmd_op;
diff --git a/xen/common/compat/kernel.c b/xen/common/compat/kernel.c
index 804b919bdc72..8e8c413bf1d0 100644
--- a/xen/common/compat/kernel.c
+++ b/xen/common/compat/kernel.c
@@ -37,7 +37,7 @@ CHECK_TYPE(capabilities_info);
CHECK_TYPE(domain_handle);
-#define DO(fn) int compat_##fn
+#define DO(fn) int cf_check compat_##fn
#define COMPAT
#include "../kernel.c"
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index c43fa97cf15f..ec8ba54bb66e 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -53,7 +53,8 @@ static int get_reserved_device_memory(xen_pfn_t start, xen_ulong_t nr,
}
#endif
-int compat_memory_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat)
+int cf_check compat_memory_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat)
{
struct vcpu *curr = current;
struct domain *currd = curr->domain;
diff --git a/xen/common/dm.c b/xen/common/dm.c
index 2d1d98ca583e..fcb3a1aa0557 100644
--- a/xen/common/dm.c
+++ b/xen/common/dm.c
@@ -19,9 +19,9 @@
#include <xen/hypercall.h>
#include <xen/nospec.h>
-long do_dm_op(domid_t domid,
- unsigned int nr_bufs,
- XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
+long cf_check do_dm_op(
+ domid_t domid, unsigned int nr_bufs,
+ XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
{
struct dmop_args args;
int rc;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 571ef4fe615e..e58c0bd4ddc4 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -1603,7 +1603,8 @@ int default_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
return rc;
}
-long do_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
+long cf_check do_vcpu_op(
+ int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct domain *d = current->domain;
struct vcpu *v;
@@ -1790,7 +1791,7 @@ long do_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
}
#ifdef arch_vm_assist_valid_mask
-long do_vm_assist(unsigned int cmd, unsigned int type)
+long cf_check do_vm_assist(unsigned int cmd, unsigned int type)
{
struct domain *currd = current->domain;
const unsigned long valid = arch_vm_assist_valid_mask(currd);
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 879a2adcbeb0..9606fa4f1aed 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -274,7 +274,7 @@ static struct vnuma_info *vnuma_init(const struct xen_domctl_vnuma *uinfo,
return ERR_PTR(ret);
}
-long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+long cf_check do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
{
long ret = 0;
bool_t copyback = 0;
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index c9912122d1e5..a5ee8b8ebf76 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -1189,7 +1189,7 @@ static int evtchn_set_priority(const struct evtchn_set_priority *set_priority)
return ret;
}
-long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long cf_check do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int rc;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 925ed7d6bee2..153332b7bfbe 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3558,7 +3558,7 @@ gnttab_cache_flush(XEN_GUEST_HANDLE_PARAM(gnttab_cache_flush_t) uop,
return 0;
}
-long
+long cf_check
do_grant_table_op(
unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count)
{
diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index e71f7df47958..1526bcc52810 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -671,9 +671,9 @@ static int hypfs_write(struct hypfs_entry *entry,
return entry->funcs->write(l, uaddr, ulen);
}
-long do_hypfs_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(const_char) arg1, unsigned long arg2,
- XEN_GUEST_HANDLE_PARAM(void) arg3, unsigned long arg4)
+long cf_check do_hypfs_op(
+ unsigned int cmd, XEN_GUEST_HANDLE_PARAM(const_char) arg1,
+ unsigned long arg2, XEN_GUEST_HANDLE_PARAM(void) arg3, unsigned long arg4)
{
int ret;
struct hypfs_entry *entry;
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index e119e5401f9d..752c2e0dae44 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -451,7 +451,7 @@ static int __init param_init(void)
__initcall(param_init);
#endif
-# define DO(fn) long do_##fn
+# define DO(fn) long cf_check do_##fn
#endif
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index c63db618a707..8471590aeea2 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -1265,13 +1265,13 @@ static int do_kexec_op_internal(unsigned long op,
return ret;
}
-long do_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg)
+long cf_check do_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg)
{
return do_kexec_op_internal(op, uarg, 0);
}
#ifdef CONFIG_COMPAT
-int compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg)
+int cf_check compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg)
{
return do_kexec_op_internal(op, uarg, 1);
}
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 0d7c413df820..38732dde6fd7 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1367,7 +1367,7 @@ static int acquire_resource(
return rc;
}
-long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long cf_check do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct domain *d, *curr_d = current->domain;
long rc;
diff --git a/xen/common/multicall.c b/xen/common/multicall.c
index 794638392b40..e48f46dbe011 100644
--- a/xen/common/multicall.c
+++ b/xen/common/multicall.c
@@ -32,7 +32,7 @@ static void trace_multicall_call(multicall_entry_t *call)
__trace_multicall_call(call);
}
-ret_t
+ret_t cf_check
do_multicall(
XEN_GUEST_HANDLE_PARAM(multicall_entry_t) call_list, uint32_t nr_calls)
{
diff --git a/xen/common/sched/compat.c b/xen/common/sched/compat.c
index 040b4caca2c4..66ba0fe88f2e 100644
--- a/xen/common/sched/compat.c
+++ b/xen/common/sched/compat.c
@@ -39,7 +39,7 @@ static int compat_poll(struct compat_sched_poll *compat)
#include "core.c"
-int compat_set_timer_op(u32 lo, s32 hi)
+int cf_check compat_set_timer_op(u32 lo, s32 hi)
{
return do_set_timer_op(((s64)hi << 32) | lo);
}
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 8f4b1ca10d1c..0f527024ba2e 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1862,7 +1862,7 @@ typedef long ret_t;
#endif /* !COMPAT */
-ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+ret_t cf_check do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
ret_t ret = 0;
@@ -1999,7 +1999,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
#ifndef COMPAT
/* Per-vcpu oneshot-timer hypercall. */
-long do_set_timer_op(s_time_t timeout)
+long cf_check do_set_timer_op(s_time_t timeout)
{
struct vcpu *v = current;
s_time_t offset = timeout - NOW();
diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c
index 1ad3c29351db..fc4a0b31d6f0 100644
--- a/xen/common/sysctl.c
+++ b/xen/common/sysctl.c
@@ -29,7 +29,7 @@
#include <xen/livepatch.h>
#include <xen/coverage.h>
-long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+long cf_check do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
{
long ret = 0;
int copyback = -1;
diff --git a/xen/common/xenoprof.c b/xen/common/xenoprof.c
index 1926a92fe481..af617f1d0b83 100644
--- a/xen/common/xenoprof.c
+++ b/xen/common/xenoprof.c
@@ -721,7 +721,7 @@ static int xenoprof_op_get_buffer(XEN_GUEST_HANDLE_PARAM(void) arg)
|| (op == XENOPROF_disable_virq) \
|| (op == XENOPROF_get_buffer))
-ret_t do_xenoprof_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+ret_t cf_check do_xenoprof_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int ret = 0;
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 7d0a603d0311..a043e9521afd 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -675,8 +675,8 @@ static long guest_console_write(XEN_GUEST_HANDLE_PARAM(char) buffer,
return 0;
}
-long do_console_io(unsigned int cmd, unsigned int count,
- XEN_GUEST_HANDLE_PARAM(char) buffer)
+long cf_check do_console_io(
+ unsigned int cmd, unsigned int count, XEN_GUEST_HANDLE_PARAM(char) buffer)
{
long rc;
unsigned int idx, len;
diff --git a/xen/include/xen/hypercall.h b/xen/include/xen/hypercall.h
index 07b10ec2303b..4dfd64cf716c 100644
--- a/xen/include/xen/hypercall.h
+++ b/xen/include/xen/hypercall.h
@@ -18,12 +18,12 @@
#include <asm/hypercall.h>
#include <xsm/xsm.h>
-extern long
+extern long cf_check
do_sched_op(
int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_domctl(
XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
@@ -32,7 +32,7 @@ arch_do_domctl(
struct xen_domctl *domctl, struct domain *d,
XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl);
-extern long
+extern long cf_check
do_sysctl(
XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl);
@@ -41,7 +41,7 @@ arch_do_sysctl(
struct xen_sysctl *sysctl,
XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl);
-extern long
+extern long cf_check
do_platform_op(
XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op);
@@ -62,47 +62,47 @@ pci_physdev_op(
#define MEMOP_EXTENT_SHIFT 6 /* cmd[:6] == start_extent */
#define MEMOP_CMD_MASK ((1 << MEMOP_EXTENT_SHIFT) - 1)
-extern long
+extern long cf_check
do_memory_op(
unsigned long cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_multicall(
XEN_GUEST_HANDLE_PARAM(multicall_entry_t) call_list,
unsigned int nr_calls);
-extern long
+extern long cf_check
do_set_timer_op(
s_time_t timeout);
-extern long
+extern long cf_check
do_event_channel_op(
int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_xen_version(
int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_console_io(
unsigned int cmd,
unsigned int count,
XEN_GUEST_HANDLE_PARAM(char) buffer);
-extern long
+extern long cf_check
do_grant_table_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) uop,
unsigned int count);
-extern long
+extern long cf_check
do_vm_assist(
unsigned int cmd,
unsigned int type);
-extern long
+extern long cf_check
do_vcpu_op(
int cmd,
unsigned int vcpuid,
@@ -114,27 +114,27 @@ arch_do_vcpu_op(int cmd,
struct vcpu *v,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_nmi_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_hvm_op(
unsigned long op,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_kexec_op(
unsigned long op,
XEN_GUEST_HANDLE_PARAM(void) uarg);
-extern long
+extern long cf_check
do_xsm_op(
XEN_GUEST_HANDLE_PARAM(void) u_xsm_op);
#ifdef CONFIG_ARGO
-extern long do_argo_op(
+extern long cf_check do_argo_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg1,
XEN_GUEST_HANDLE_PARAM(void) arg2,
@@ -142,20 +142,20 @@ extern long do_argo_op(
unsigned long arg4);
#endif
-extern long
+extern long cf_check
do_xenoprof_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern long
+extern long cf_check
do_xenpmu_op(unsigned int op, XEN_GUEST_HANDLE_PARAM(xen_pmu_params_t) arg);
-extern long
+extern long cf_check
do_dm_op(
domid_t domid,
unsigned int nr_bufs,
XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs);
#ifdef CONFIG_HYPFS
-extern long
+extern long cf_check
do_hypfs_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(const_char) arg1,
@@ -166,53 +166,54 @@ do_hypfs_op(
#ifdef CONFIG_COMPAT
-extern int
+extern int cf_check
compat_memory_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int
+extern int cf_check
compat_grant_table_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) uop,
unsigned int count);
-extern int
+extern int cf_check
compat_vcpu_op(
int cmd,
unsigned int vcpuid,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int
+extern int cf_check
compat_xenoprof_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int
+extern int cf_check
compat_xen_version(
int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int
+extern int cf_check
compat_sched_op(
int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg);
-extern int
+extern int cf_check
compat_set_timer_op(
u32 lo,
s32 hi);
-extern int compat_xsm_op(
+extern int cf_check compat_xsm_op(
XEN_GUEST_HANDLE_PARAM(void) op);
-extern int compat_kexec_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg);
+extern int cf_check compat_kexec_op(
+ unsigned long op, XEN_GUEST_HANDLE_PARAM(void) uarg);
DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
-extern int compat_multicall(
+extern int cf_check compat_multicall(
XEN_GUEST_HANDLE_PARAM(multicall_entry_compat_t) call_list,
uint32_t nr_calls);
#ifdef CONFIG_ARGO
-extern long compat_argo_op(
+extern long cf_check compat_argo_op(
unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void) arg1,
XEN_GUEST_HANDLE_PARAM(void) arg2,
@@ -220,7 +221,7 @@ extern long compat_argo_op(
unsigned long arg4);
#endif
-extern int
+extern int cf_check
compat_dm_op(
domid_t domid,
unsigned int nr_bufs,
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 14d98f1f725f..5fc3a5f75478 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -219,13 +219,13 @@ bool __init has_xsm_magic(paddr_t start)
#endif
-long do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+long cf_check do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
{
return xsm_do_xsm_op(op);
}
#ifdef CONFIG_COMPAT
-int compat_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+int cf_check compat_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
{
return xsm_do_compat_op(op);
}
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 09/70] xen: CFI hardening for custom_param()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (7 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 08/70] xen: CFI hardening for x86 hypercalls Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 10/70] xen: CFI hardening for __initcall() Andrew Cooper
` (62 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The "watchdog_timeout" and "cpu_type" handlers were missing __init.
The "numa", "acpi", "irq_vector_map" and "flask" handlers can skip forward
declarations by altering the custom_param() position.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 2 +-
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/apic.c | 4 ++--
xen/arch/x86/cpu/mcheck/mce.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/cpu/vpmu.c | 2 +-
xen/arch/x86/cpuid.c | 8 ++++----
xen/arch/x86/dom0_build.c | 8 ++++----
xen/arch/x86/genapic/probe.c | 2 +-
xen/arch/x86/hpet.c | 2 +-
xen/arch/x86/hvm/viridian/viridian.c | 2 +-
xen/arch/x86/hvm/vmx/vmcs.c | 8 ++++----
xen/arch/x86/io_apic.c | 2 +-
xen/arch/x86/irq.c | 6 ++----
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/nmi.c | 4 ++--
xen/arch/x86/numa.c | 6 ++----
xen/arch/x86/oprofile/nmi_int.c | 2 +-
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/pv/domain.c | 8 ++++----
xen/arch/x86/pv/shim.c | 2 +-
xen/arch/x86/setup.c | 11 +++++------
xen/arch/x86/shutdown.c | 2 +-
xen/arch/x86/spec_ctrl.c | 6 +++---
xen/arch/x86/time.c | 2 +-
xen/arch/x86/tsx.c | 2 +-
xen/arch/x86/x86_64/mmconfig-shared.c | 2 +-
xen/common/argo.c | 2 +-
xen/common/core_parking.c | 2 +-
xen/common/debugtrace.c | 2 +-
xen/common/domain.c | 2 +-
xen/common/efi/boot.c | 2 +-
xen/common/grant_table.c | 14 +++++++-------
xen/common/kexec.c | 6 +++---
xen/common/memory.c | 2 +-
xen/common/page_alloc.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
xen/common/sched/credit2.c | 2 +-
xen/drivers/acpi/tables.c | 2 +-
xen/drivers/char/console.c | 18 +++++++++---------
xen/drivers/cpufreq/cpufreq.c | 2 +-
xen/drivers/passthrough/amd/iommu_acpi.c | 6 +++---
xen/drivers/passthrough/iommu.c | 4 ++--
xen/drivers/passthrough/pci.c | 4 ++--
xen/drivers/passthrough/vtd/dmar.c | 2 +-
xen/drivers/passthrough/vtd/quirks.c | 2 +-
xen/drivers/video/vesa.c | 2 +-
xen/xsm/flask/flask_op.c | 5 ++---
xen/xsm/xsm_core.c | 2 +-
49 files changed, 92 insertions(+), 98 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index d788c8bffc84..5d73eb5917af 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -106,7 +106,7 @@ void (*__read_mostly pm_idle_save)(void);
unsigned int max_cstate __read_mostly = UINT_MAX;
unsigned int max_csubstate __read_mostly = UINT_MAX;
-static int __init parse_cstate(const char *s)
+static int __init cf_check parse_cstate(const char *s)
{
max_cstate = simple_strtoul(s, &s, 0);
if ( *s == ',' )
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 5eaa77f66a28..912d4c4d62f4 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -35,7 +35,7 @@
uint32_t system_reset_counter = 1;
-static int __init parse_acpi_sleep(const char *s)
+static int __init cf_check parse_acpi_sleep(const char *s)
{
const char *ss;
unsigned int flag = 0;
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 583656158532..68e4d870c749 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -775,7 +775,7 @@ int lapic_resume(void)
* Original code written by Keir Fraser.
*/
-static int __init lapic_disable(const char *str)
+static int __init cf_check lapic_disable(const char *str)
{
enable_local_apic = -1;
setup_clear_cpu_cap(X86_FEATURE_APIC);
@@ -784,7 +784,7 @@ static int __init lapic_disable(const char *str)
custom_param("nolapic", lapic_disable);
boolean_param("lapic", enable_local_apic);
-static int __init apic_set_verbosity(const char *str)
+static int __init cf_check apic_set_verbosity(const char *str)
{
if (strcmp("debug", str) == 0)
apic_verbosity = APIC_DEBUG;
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index eae08caa07cd..ea86d84481b2 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -63,7 +63,7 @@ struct mca_banks *mca_allbanks;
#endif
int mce_verbosity;
-static int __init mce_set_verbosity(const char *str)
+static int __init cf_check mce_set_verbosity(const char *str)
{
if ( strcmp("verbose", str) == 0 )
mce_verbosity = MCE_VERBOSE;
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index ac3ceb567c41..95d35ca0f3f7 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -111,7 +111,7 @@ void __init microcode_set_module(unsigned int idx)
* optional. If the EFI has forced which of the multiboot payloads is to be
* used, only nmi=<bool> is parsed.
*/
-static int __init parse_ucode(const char *s)
+static int __init cf_check parse_ucode(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index 598291f4ece9..b10d6e2eb458 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -56,7 +56,7 @@ static unsigned vpmu_count;
static DEFINE_PER_CPU(struct vcpu *, last_vcpu);
-static int __init parse_vpmu_params(const char *s)
+static int __init cf_check parse_vpmu_params(const char *s)
{
const char *ss;
int rc = 0, val;
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index e24dd283e761..7632b8344f87 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -101,7 +101,7 @@ static int __init always_inline parse_cpuid(
return rc;
}
-static void __init _parse_xen_cpuid(unsigned int feat, bool val)
+static void __init cf_check _parse_xen_cpuid(unsigned int feat, bool val)
{
if ( !val )
setup_clear_cpu_cap(feat);
@@ -110,7 +110,7 @@ static void __init _parse_xen_cpuid(unsigned int feat, bool val)
setup_force_cpu_cap(X86_FEATURE_RDRAND);
}
-static int __init parse_xen_cpuid(const char *s)
+static int __init cf_check parse_xen_cpuid(const char *s)
{
return parse_cpuid(s, _parse_xen_cpuid);
}
@@ -120,13 +120,13 @@ static bool __initdata dom0_cpuid_cmdline;
static uint32_t __initdata dom0_enable_feat[FSCAPINTS];
static uint32_t __initdata dom0_disable_feat[FSCAPINTS];
-static void __init _parse_dom0_cpuid(unsigned int feat, bool val)
+static void __init cf_check _parse_dom0_cpuid(unsigned int feat, bool val)
{
__set_bit (feat, val ? dom0_enable_feat : dom0_disable_feat);
__clear_bit(feat, val ? dom0_disable_feat : dom0_enable_feat );
}
-static int __init parse_dom0_cpuid(const char *s)
+static int __init cf_check parse_dom0_cpuid(const char *s)
{
dom0_cpuid_cmdline = true;
diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
index a7fec05956c1..4d1c5c60e407 100644
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -115,7 +115,7 @@ static int __init parse_amt(const char *s, const char **ps, struct memsize *sz)
return 0;
}
-static int __init parse_dom0_mem(const char *s)
+static int __init cf_check parse_dom0_mem(const char *s)
{
int ret;
@@ -144,7 +144,7 @@ custom_param("dom0_mem", parse_dom0_mem);
static unsigned int __initdata opt_dom0_max_vcpus_min = 1;
static unsigned int __initdata opt_dom0_max_vcpus_max = UINT_MAX;
-static int __init parse_dom0_max_vcpus(const char *s)
+static int __init cf_check parse_dom0_max_vcpus(const char *s)
{
if ( *s == '-' ) /* -M */
opt_dom0_max_vcpus_max = simple_strtoul(s + 1, &s, 0);
@@ -168,7 +168,7 @@ static __initdata unsigned int dom0_pxms[MAX_NUMNODES] =
{ [0 ... MAX_NUMNODES - 1] = ~0 };
bool __initdata dom0_affinity_relaxed;
-static int __init parse_dom0_nodes(const char *s)
+static int __init cf_check parse_dom0_nodes(const char *s)
{
const char *ss;
int rc = 0;
@@ -266,7 +266,7 @@ bool __initdata opt_dom0_pvh = !IS_ENABLED(CONFIG_PV);
bool __initdata opt_dom0_verbose = IS_ENABLED(CONFIG_VERBOSE_DEBUG);
bool __initdata opt_dom0_msr_relaxed;
-static int __init parse_dom0_param(const char *s)
+static int __init cf_check parse_dom0_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/arch/x86/genapic/probe.c b/xen/arch/x86/genapic/probe.c
index 66bc5ce072dc..ad57912f506b 100644
--- a/xen/arch/x86/genapic/probe.c
+++ b/xen/arch/x86/genapic/probe.c
@@ -43,7 +43,7 @@ void __init generic_bigsmp_probe(void)
}
}
-static int __init genapic_apic_force(const char *str)
+static int __init cf_check genapic_apic_force(const char *str)
{
int i, rc = -EINVAL;
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index afe104dc93c2..7b009a930498 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -65,7 +65,7 @@ u8 __initdata hpet_flags;
static bool __initdata force_hpet_broadcast;
boolean_param("hpetbroadcast", force_hpet_broadcast);
-static int __init parse_hpet_param(const char *s)
+static int __init cf_check parse_hpet_param(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
index 8986b8e03c2e..7ebcaa1c899f 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -1186,7 +1186,7 @@ static int viridian_load_vcpu_ctxt(struct domain *d,
HVM_REGISTER_SAVE_RESTORE(VIRIDIAN_VCPU, viridian_save_vcpu_ctxt,
viridian_load_vcpu_ctxt, 1, HVMSR_PER_VCPU);
-static int __init parse_viridian_version(const char *arg)
+static int __init cf_check parse_viridian_version(const char *arg)
{
const char *t;
unsigned int n[3];
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 7ab15e07a0b2..f72a7db0453d 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -71,7 +71,7 @@ static bool __read_mostly opt_ept_pml = true;
static s8 __read_mostly opt_ept_ad = -1;
int8_t __read_mostly opt_ept_exec_sp = -1;
-static int __init parse_ept_param(const char *s)
+static int __init cf_check parse_ept_param(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -107,16 +107,16 @@ static void update_ept_param(void)
opt_ept_exec_sp);
}
-static void __init init_ept_param(struct param_hypfs *par)
+static void __init cf_check init_ept_param(struct param_hypfs *par)
{
update_ept_param();
custom_runtime_set_var(par, opt_ept_setting);
}
-static int parse_ept_param_runtime(const char *s);
+static int cf_check parse_ept_param_runtime(const char *s);
custom_runtime_only_param("ept", parse_ept_param_runtime, init_ept_param);
-static int parse_ept_param_runtime(const char *s)
+static int cf_check parse_ept_param_runtime(const char *s)
{
struct domain *d;
int val;
diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index 1c49a0fe1478..4135a9c06052 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -1601,7 +1601,7 @@ static unsigned int startup_level_ioapic_irq(struct irq_desc *desc)
return 0; /* don't check for pending */
}
-static int __init setup_ioapic_ack(const char *s)
+static int __init cf_check setup_ioapic_ack(const char *s)
{
if ( !strcmp(s, "old") )
{
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 67cbf6b979dc..84b174d0f51f 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -28,8 +28,6 @@
#include <irq_vectors.h>
#include <public/physdev.h>
-static int parse_irq_vector_map_param(const char *s);
-
/* opt_noirqbalance: If true, software IRQ balancing/affinity is disabled. */
bool __read_mostly opt_noirqbalance;
boolean_param("noirqbalance", opt_noirqbalance);
@@ -40,7 +38,6 @@ integer_param("nr_irqs", nr_irqs);
/* This default may be changed by the AMD IOMMU code */
int __read_mostly opt_irq_vector_map = OPT_IRQ_VECTOR_MAP_DEFAULT;
-custom_param("irq_vector_map", parse_irq_vector_map_param);
/* Max number of guests IRQ could be shared with */
static unsigned char __read_mostly irq_max_guests;
@@ -66,7 +63,7 @@ static struct timer irq_ratelimit_timer;
static unsigned int __read_mostly irq_ratelimit_threshold = 10000;
integer_param("irq_ratelimit", irq_ratelimit_threshold);
-static int __init parse_irq_vector_map_param(const char *s)
+static int __init cf_check parse_irq_vector_map_param(const char *s)
{
const char *ss;
int rc = 0;
@@ -90,6 +87,7 @@ static int __init parse_irq_vector_map_param(const char *s)
return rc;
}
+custom_param("irq_vector_map", parse_irq_vector_map_param);
/* Must be called when irq disabled */
void lock_vector_lock(void)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 6484c3b97bbc..3b8bc3dda977 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -189,7 +189,7 @@ static uint32_t base_disallow_mask;
static s8 __read_mostly opt_mmio_relax;
-static int __init parse_mmio_relax(const char *s)
+static int __init cf_check parse_mmio_relax(const char *s)
{
if ( !*s )
opt_mmio_relax = 1;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index ab94a96c4d03..1a225d499295 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -48,7 +48,7 @@ bool __initdata opt_watchdog;
/* watchdog_force: If true, process unknown NMIs when running the watchdog. */
bool watchdog_force;
-static int __init parse_watchdog(const char *s)
+static int __init cf_check parse_watchdog(const char *s)
{
if ( !*s )
{
@@ -78,7 +78,7 @@ custom_param("watchdog", parse_watchdog);
/* opt_watchdog_timeout: Number of seconds to wait before panic. */
static unsigned int opt_watchdog_timeout = 5;
-static int parse_watchdog_timeout(const char *s)
+static int __init cf_check parse_watchdog_timeout(const char *s)
{
const char *q;
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index ce79ee44cefe..6be5a0c93322 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -19,9 +19,6 @@
#include <xen/sched.h>
#include <xen/softirq.h>
-static int numa_setup(const char *s);
-custom_param("numa", numa_setup);
-
#ifndef Dprintk
#define Dprintk(x...)
#endif
@@ -294,7 +291,7 @@ void numa_set_node(int cpu, nodeid_t node)
}
/* [numa=off] */
-static __init int numa_setup(const char *opt)
+static int __init cf_check numa_setup(const char *opt)
{
if ( !strncmp(opt,"off",3) )
numa_off = true;
@@ -321,6 +318,7 @@ static __init int numa_setup(const char *opt)
return 0;
}
+custom_param("numa", numa_setup);
/*
* Setup early cpu_to_node.
diff --git a/xen/arch/x86/oprofile/nmi_int.c b/xen/arch/x86/oprofile/nmi_int.c
index a13bd82915ac..7842d95b95ea 100644
--- a/xen/arch/x86/oprofile/nmi_int.c
+++ b/xen/arch/x86/oprofile/nmi_int.c
@@ -340,7 +340,7 @@ static int __init p4_init(char ** cpu_type)
static int force_arch_perfmon;
-static int force_cpu_type(const char *str)
+static int __init cf_check force_cpu_type(const char *str)
{
if (!strcmp(str, "arch_perfmon")) {
force_arch_perfmon = 1;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index d805b85dc60b..56916344cb1d 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -573,7 +573,7 @@ static bool __init parse_psr_bool(const char *s, const char *delim,
return false;
}
-static int __init parse_psr_param(const char *s)
+static int __init cf_check parse_psr_param(const char *s)
{
const char *ss, *val_delim;
const char *q;
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 6ad533183bcd..125c4561a7ea 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -20,7 +20,7 @@
int8_t __read_mostly opt_pv32 = -1;
#endif
-static __init int parse_pv(const char *s)
+static int __init cf_check parse_pv(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -63,16 +63,16 @@ static const char opt_pcid_2_string[][7] = {
[PCID_NOXPTI] = "noxpti",
};
-static void __init opt_pcid_init(struct param_hypfs *par)
+static void __init cf_check opt_pcid_init(struct param_hypfs *par)
{
custom_runtime_set_var(par, opt_pcid_2_string[opt_pcid]);
}
#endif
-static int parse_pcid(const char *s);
+static int cf_check parse_pcid(const char *s);
custom_runtime_param("pcid", parse_pcid, opt_pcid_init);
-static int parse_pcid(const char *s)
+static int cf_check parse_pcid(const char *s)
{
int rc = 0;
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index 4c710ad8913f..ae4d8913faa1 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -73,7 +73,7 @@ static uint64_t __initdata shim_nrpages;
static uint64_t __initdata shim_min_nrpages;
static uint64_t __initdata shim_max_nrpages;
-static int __init parse_shim_mem(const char *s)
+static int __init cf_check parse_shim_mem(const char *s)
{
do {
if ( !strncmp(s, "min:", 4) )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 115f8f651734..eceff0a4e2b4 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -81,8 +81,6 @@ unsigned long __read_mostly cr4_pv32_mask;
/* "acpi=ht": Limit ACPI just to boot-time to enable HT. */
/* "acpi=noirq": Disables ACPI interrupt routing. */
/* "acpi=verbose": Enables more verbose ACPI boot time logging. */
-static int parse_acpi_param(const char *s);
-custom_param("acpi", parse_acpi_param);
/* **** Linux config option: propagated to domain0. */
/* noapic: Disable IOAPIC setup. */
@@ -104,7 +102,7 @@ static bool __initdata opt_xen_shstk = true;
#define opt_xen_shstk false
#endif
-static int __init parse_cet(const char *s)
+static int __init cf_check parse_cet(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -159,7 +157,7 @@ static s8 __initdata opt_smep = -1;
*/
static struct domain *__initdata dom0;
-static int __init parse_smep_param(const char *s)
+static int __init cf_check parse_smep_param(const char *s)
{
if ( !*s )
{
@@ -190,7 +188,7 @@ custom_param("smep", parse_smep_param);
#define SMAP_HVM_ONLY (-2)
static s8 __initdata opt_smap = -1;
-static int __init parse_smap_param(const char *s)
+static int __init cf_check parse_smap_param(const char *s)
{
if ( !*s )
{
@@ -221,7 +219,7 @@ bool __read_mostly acpi_disabled;
bool __initdata acpi_force;
static char __initdata acpi_param[10] = "";
-static int __init parse_acpi_param(const char *s)
+static int __init cf_check parse_acpi_param(const char *s)
{
/* Interpret the parameter for use within Xen. */
if ( !parse_bool(s, NULL) )
@@ -257,6 +255,7 @@ static int __init parse_acpi_param(const char *s)
return 0;
}
+custom_param("acpi", parse_acpi_param);
static const module_t *__initdata initial_images;
static unsigned int __initdata nr_initial_images;
diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index acef03314372..a01354d93319 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -56,7 +56,7 @@ static int reboot_mode;
*/
static enum reboot_type reboot_type = BOOT_INVALID;
-static int __init set_reboot_type(const char *str)
+static int __init cf_check set_reboot_type(const char *str)
{
int rc = 0;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index cbeeb199037e..d45841043c58 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -68,7 +68,7 @@ static bool __initdata cpu_has_bug_mds; /* Any other M{LP,SB,FB}DS combination.
static int8_t __initdata opt_srb_lock = -1;
-static int __init parse_spec_ctrl(const char *s)
+static int __init cf_check parse_spec_ctrl(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -218,7 +218,7 @@ static __init void xpti_init_default(uint64_t caps)
}
}
-static __init int parse_xpti(const char *s)
+static int __init cf_check parse_xpti(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -264,7 +264,7 @@ custom_param("xpti", parse_xpti);
int8_t __read_mostly opt_pv_l1tf_hwdom = -1;
int8_t __read_mostly opt_pv_l1tf_domu = -1;
-static __init int parse_pv_l1tf(const char *s)
+static int __init cf_check parse_pv_l1tf(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 17f64a6ccdd1..2da9c7f5c8dd 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -2356,7 +2356,7 @@ int hwdom_pit_access(struct ioreq *ioreq)
* tsc=skewed: Assume TSCs are individually reliable, but skewed across CPUs.
* tsc=stable:socket: Assume TSCs are reliable across sockets.
*/
-static int __init tsc_parse(const char *s)
+static int __init cf_check tsc_parse(const char *s)
{
if ( !strcmp(s, "unstable") )
{
diff --git a/xen/arch/x86/tsx.c b/xen/arch/x86/tsx.c
index be89741a2f6d..b156844cdec1 100644
--- a/xen/arch/x86/tsx.c
+++ b/xen/arch/x86/tsx.c
@@ -22,7 +22,7 @@ int8_t __read_mostly opt_tsx = -1;
int8_t __read_mostly cpu_has_tsx_ctrl = -1;
bool __read_mostly rtm_disabled;
-static int __init parse_tsx(const char *s)
+static int __init cf_check parse_tsx(const char *s)
{
int rc = 0, val = parse_bool(s, NULL);
diff --git a/xen/arch/x86/x86_64/mmconfig-shared.c b/xen/arch/x86/x86_64/mmconfig-shared.c
index 7c3ed64b4c6c..2fa7f3f0bc4b 100644
--- a/xen/arch/x86/x86_64/mmconfig-shared.c
+++ b/xen/arch/x86/x86_64/mmconfig-shared.c
@@ -29,7 +29,7 @@
unsigned int pci_probe = PCI_PROBE_CONF1 | PCI_PROBE_MMCONF;
-static int __init parse_mmcfg(const char *s)
+static int __init cf_check parse_mmcfg(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/common/argo.c b/xen/common/argo.c
index 1448faf65731..297f6d11f04d 100644
--- a/xen/common/argo.c
+++ b/xen/common/argo.c
@@ -78,7 +78,7 @@ DEFINE_COMPAT_HANDLE(compat_argo_iov_t);
static bool __read_mostly opt_argo;
static bool __read_mostly opt_argo_mac_permissive;
-static int __init parse_argo(const char *s)
+static int __init cf_check parse_argo(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index 411106c675c9..aa432ed2f57b 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -40,7 +40,7 @@ static enum core_parking_controller {
PERFORMANCE_FIRST
} core_parking_controller __initdata = POWER_FIRST;
-static int __init setup_core_parking_option(const char *str)
+static int __init cf_check setup_core_parking_option(const char *str)
{
if ( !strcmp(str, "power") )
core_parking_controller = POWER_FIRST;
diff --git a/xen/common/debugtrace.c b/xen/common/debugtrace.c
index f3794b945376..29b11239f5a5 100644
--- a/xen/common/debugtrace.c
+++ b/xen/common/debugtrace.c
@@ -38,7 +38,7 @@ static bool debugtrace_buf_empty = true;
static bool debugtrace_used;
static DEFINE_SPINLOCK(debugtrace_lock);
-static int __init debugtrace_parse_param(const char *s)
+static int __init cf_check debugtrace_parse_param(const char *s)
{
unsigned long bytes;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index e58c0bd4ddc4..5df0d167537b 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -354,7 +354,7 @@ static int late_hwdom_init(struct domain *d)
static unsigned int __read_mostly extra_hwdom_irqs;
static unsigned int __read_mostly extra_domU_irqs = 32;
-static int __init parse_extra_guest_irqs(const char *s)
+static int __init cf_check parse_extra_guest_irqs(const char *s)
{
if ( isdigit(*s) )
extra_domU_irqs = simple_strtoul(s, &s, 0);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 12fd0844bd55..f31f68fd4cd1 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1417,7 +1417,7 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
static bool __initdata efi_map_uc;
-static int __init parse_efi_param(const char *s)
+static int __init cf_check parse_efi_param(const char *s)
{
const char *ss;
int rc = 0, val;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 153332b7bfbe..b663845d9c6c 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -117,12 +117,12 @@ static void update_gnttab_par(unsigned int val, struct param_hypfs *par,
custom_runtime_set_var_sz(par, parval, GRANT_CUSTOM_VAL_SZ);
}
-static void __init gnttab_max_frames_init(struct param_hypfs *par)
+static void __init cf_check gnttab_max_frames_init(struct param_hypfs *par)
{
update_gnttab_par(opt_max_grant_frames, par, opt_max_grant_frames_val);
}
-static void __init max_maptrack_frames_init(struct param_hypfs *par)
+static void __init cf_check max_maptrack_frames_init(struct param_hypfs *par)
{
update_gnttab_par(opt_max_maptrack_frames, par,
opt_max_maptrack_frames_val);
@@ -156,23 +156,23 @@ static int parse_gnttab_limit(const char *arg, unsigned int *valp,
return 0;
}
-static int parse_gnttab_max_frames(const char *arg);
+static int cf_check parse_gnttab_max_frames(const char *arg);
custom_runtime_param("gnttab_max_frames", parse_gnttab_max_frames,
gnttab_max_frames_init);
-static int parse_gnttab_max_frames(const char *arg)
+static int cf_check parse_gnttab_max_frames(const char *arg)
{
return parse_gnttab_limit(arg, &opt_max_grant_frames,
param_2_parfs(parse_gnttab_max_frames),
opt_max_grant_frames_val);
}
-static int parse_gnttab_max_maptrack_frames(const char *arg);
+static int cf_check parse_gnttab_max_maptrack_frames(const char *arg);
custom_runtime_param("gnttab_max_maptrack_frames",
parse_gnttab_max_maptrack_frames,
max_maptrack_frames_init);
-static int parse_gnttab_max_maptrack_frames(const char *arg)
+static int cf_check parse_gnttab_max_maptrack_frames(const char *arg)
{
return parse_gnttab_limit(arg, &opt_max_maptrack_frames,
param_2_parfs(parse_gnttab_max_maptrack_frames),
@@ -191,7 +191,7 @@ static bool __ro_after_init opt_grant_transfer = true;
#define opt_grant_transfer false
#endif
-static int __init parse_gnttab(const char *s)
+static int __init cf_check parse_gnttab(const char *s)
{
const char *ss, *e;
int val, rc = 0;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 8471590aeea2..6286c0bbf08b 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -104,7 +104,7 @@ static void *crash_heap_current = NULL, *crash_heap_end = NULL;
* < and below are synonyomous, the latter being useful for grub2 systems
* which would otherwise require escaping of the < option
*/
-static int __init parse_crashkernel(const char *str)
+static int __init cf_check parse_crashkernel(const char *str)
{
const char *cur;
int rc = 0;
@@ -201,7 +201,7 @@ custom_param("crashkernel", parse_crashkernel);
* - all will allocate additional structures such as domain and vcpu structs
* low so the crash kernel can perform an extended analysis of state.
*/
-static int __init parse_low_crashinfo(const char *str)
+static int __init cf_check parse_low_crashinfo(const char *str)
{
if ( !strlen(str) )
@@ -230,7 +230,7 @@ custom_param("low_crashinfo", parse_low_crashinfo);
*
* <addr> will be rounded down to the nearest power of two. Defaults to 64G
*/
-static int __init parse_crashinfo_maxaddr(const char *str)
+static int __init cf_check parse_crashinfo_maxaddr(const char *str)
{
u64 addr;
const char *q;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 38732dde6fd7..ede45c4af9db 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -62,7 +62,7 @@ static unsigned int __read_mostly hwdom_max_order = CONFIG_HWDOM_MAX_ORDER;
static unsigned int __read_mostly ptdom_max_order = CONFIG_PTDOM_MAX_ORDER;
#endif
-static int __init parse_max_order(const char *s)
+static int __init cf_check parse_max_order(const char *s)
{
if ( *s != ',' )
domu_max_order = simple_strtoul(s, &s, 0);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index f8749b0787a6..ad06655158d2 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -179,7 +179,7 @@ enum bootscrub_mode {
* https://bugs.llvm.org/show_bug.cgi?id=39707
*/
static enum bootscrub_mode __read_mostly opt_bootscrub = BOOTSCRUB_IDLE;
-static int __init parse_bootscrub_param(const char *s)
+static int __init cf_check parse_bootscrub_param(const char *s)
{
/* Interpret 'bootscrub' alone in its positive boolean form */
if ( *s == '\0' )
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 8c6e6eb9ccd5..f0dd626054a6 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -93,7 +93,7 @@ static int sched_gran_get(const char *str, enum sched_gran *mode)
return -EINVAL;
}
-static int __init sched_select_granularity(const char *str)
+static int __init cf_check sched_select_granularity(const char *str)
{
return sched_gran_get(str, &opt_sched_granularity);
}
diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index 6396b38e044c..a5f073cda51e 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -456,7 +456,7 @@ static const char *const opt_runqueue_str[] = {
};
static int __read_mostly opt_runqueue = OPT_RUNQUEUE_SOCKET;
-static int __init parse_credit2_runqueue(const char *s)
+static int __init cf_check parse_credit2_runqueue(const char *s)
{
unsigned int i;
diff --git a/xen/drivers/acpi/tables.c b/xen/drivers/acpi/tables.c
index f39cd5eaac89..96ff96b84c66 100644
--- a/xen/drivers/acpi/tables.c
+++ b/xen/drivers/acpi/tables.c
@@ -472,7 +472,7 @@ int __init acpi_table_init(void)
return 0;
}
-static int __init acpi_parse_apic_instance(const char *str)
+static int __init cf_check acpi_parse_apic_instance(const char *str)
{
const char *q;
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index a043e9521afd..4694be83db45 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -88,7 +88,7 @@ static const char con_timestamp_mode_2_string[][7] = {
[TSM_RAW] = "raw",
};
-static void con_timestamp_mode_upd(struct param_hypfs *par)
+static void cf_check con_timestamp_mode_upd(struct param_hypfs *par)
{
const char *val = con_timestamp_mode_2_string[opt_con_timestamp_mode];
@@ -98,7 +98,7 @@ static void con_timestamp_mode_upd(struct param_hypfs *par)
#define con_timestamp_mode_upd(par)
#endif
-static int parse_console_timestamps(const char *s);
+static int cf_check parse_console_timestamps(const char *s);
custom_runtime_param("console_timestamps", parse_console_timestamps,
con_timestamp_mode_upd);
@@ -160,8 +160,8 @@ static int __read_mostly xenlog_guest_upper_thresh =
static int __read_mostly xenlog_guest_lower_thresh =
XENLOG_GUEST_LOWER_THRESHOLD;
-static int parse_loglvl(const char *s);
-static int parse_guest_loglvl(const char *s);
+static int cf_check parse_loglvl(const char *s);
+static int cf_check parse_guest_loglvl(const char *s);
#ifdef CONFIG_HYPFS
#define LOGLVL_VAL_SZ 16
@@ -176,13 +176,13 @@ static void xenlog_update_val(int lower, int upper, char *val)
snprintf(val, LOGLVL_VAL_SZ, "%s/%s", lvl2opt[lower], lvl2opt[upper]);
}
-static void __init xenlog_init(struct param_hypfs *par)
+static void __init cf_check xenlog_init(struct param_hypfs *par)
{
xenlog_update_val(xenlog_lower_thresh, xenlog_upper_thresh, xenlog_val);
custom_runtime_set_var(par, xenlog_val);
}
-static void __init xenlog_guest_init(struct param_hypfs *par)
+static void __init cf_check xenlog_guest_init(struct param_hypfs *par)
{
xenlog_update_val(xenlog_guest_lower_thresh, xenlog_guest_upper_thresh,
xenlog_guest_val);
@@ -240,7 +240,7 @@ static int _parse_loglvl(const char *s, int *lower, int *upper, char *val)
return *s ? -EINVAL : 0;
}
-static int parse_loglvl(const char *s)
+static int cf_check parse_loglvl(const char *s)
{
int ret;
@@ -251,7 +251,7 @@ static int parse_loglvl(const char *s)
return ret;
}
-static int parse_guest_loglvl(const char *s)
+static int cf_check parse_guest_loglvl(const char *s)
{
int ret;
@@ -793,7 +793,7 @@ static int printk_prefix_check(char *p, char **pp)
((loglvl < upper_thresh) && printk_ratelimit()));
}
-static int parse_console_timestamps(const char *s)
+static int cf_check parse_console_timestamps(const char *s)
{
switch ( parse_bool(s, NULL) )
{
diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
index 419aae83eea6..36b079296235 100644
--- a/xen/drivers/cpufreq/cpufreq.c
+++ b/xen/drivers/cpufreq/cpufreq.c
@@ -65,7 +65,7 @@ enum cpufreq_controller cpufreq_controller = FREQCTL_xen;
static int __init cpufreq_cmdline_parse(const char *s);
-static int __init setup_cpufreq_option(const char *str)
+static int __init cf_check setup_cpufreq_option(const char *str)
{
const char *arg = strpbrk(str, ",:");
int choice;
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index b07fa4c40124..5ea227732821 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -704,7 +704,7 @@ static u16 __init parse_ivhd_device_extended_range(
return dev_length;
}
-static int __init parse_ivrs_ioapic(const char *str)
+static int __init cf_check parse_ivrs_ioapic(const char *str)
{
const char *s = str;
unsigned long id;
@@ -742,7 +742,7 @@ static int __init parse_ivrs_ioapic(const char *str)
}
custom_param("ivrs_ioapic[", parse_ivrs_ioapic);
-static int __init parse_ivrs_hpet(const char *str)
+static int __init cf_check parse_ivrs_hpet(const char *str)
{
const char *s = str;
unsigned long id;
@@ -1369,7 +1369,7 @@ int __init amd_iommu_get_supported_ivhd_type(void)
* Format:
* ivmd=<start>[-<end>][=<bdf1>[-<bdf1>'][,<bdf2>[-<bdf2>'][,...]]][;<start>...]
*/
-static int __init parse_ivmd_param(const char *s)
+static int __init cf_check parse_ivmd_param(const char *s)
{
do {
unsigned long start, end;
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index fc18f63bd4ac..6ee267d2bfd4 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -64,7 +64,7 @@ bool_t __read_mostly amd_iommu_perdev_intremap = 1;
DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
-static int __init parse_iommu_param(const char *s)
+static int __init cf_check parse_iommu_param(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -135,7 +135,7 @@ static int __init parse_iommu_param(const char *s)
}
custom_param("iommu", parse_iommu_param);
-static int __init parse_dom0_iommu_param(const char *s)
+static int __init cf_check parse_dom0_iommu_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index e8b09d77d880..20cb246598b1 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -146,7 +146,7 @@ static struct phantom_dev {
} phantom_devs[8];
static unsigned int nr_phantom_devs;
-static int __init parse_phantom_dev(const char *str)
+static int __init cf_check parse_phantom_dev(const char *str)
{
const char *s;
unsigned int seg, bus, slot;
@@ -182,7 +182,7 @@ custom_param("pci-phantom", parse_phantom_dev);
static u16 __read_mostly command_mask;
static u16 __read_mostly bridge_ctl_mask;
-static int __init parse_pci_param(const char *s)
+static int __init cf_check parse_pci_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index 33a12b2ae976..b152f3da916b 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -1084,7 +1084,7 @@ int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
* If a segment is specified for other than the first device, and it does not
* match the one specified for the first one, an error will be reported.
*/
-static int __init parse_rmrr_param(const char *str)
+static int __init cf_check parse_rmrr_param(const char *str)
{
const char *s = str, *cur, *stmp;
unsigned int seg, bus, dev, func, dev_count;
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 52b47dd89325..0590ddeea7c4 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -308,7 +308,7 @@ void vtd_ops_postamble_quirk(struct vtd_iommu *iommu)
}
}
-static int __init parse_snb_timeout(const char *s)
+static int __init cf_check parse_snb_timeout(const char *s)
{
int t;
const char *q = NULL;
diff --git a/xen/drivers/video/vesa.c b/xen/drivers/video/vesa.c
index 2c1bbd927806..cb0e443be4dd 100644
--- a/xen/drivers/video/vesa.c
+++ b/xen/drivers/video/vesa.c
@@ -30,7 +30,7 @@ static unsigned int vram_remap;
integer_param("vesa-map", vram_remap);
static int font_height;
-static int __init parse_font_height(const char *s)
+static int __init cf_check parse_font_height(const char *s)
{
if ( simple_strtoul(s, &s, 10) == 8 && (*s++ == 'x') )
font_height = simple_strtoul(s, &s, 10);
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index bb3bebc30e01..2d7ca3abaecd 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -28,8 +28,6 @@
#define _copy_from_guest copy_from_guest
enum flask_bootparam_t __read_mostly flask_bootparam = FLASK_BOOTPARAM_ENFORCING;
-static int parse_flask_param(const char *s);
-custom_param("flask", parse_flask_param);
bool __read_mostly flask_enforcing = true;
@@ -60,7 +58,7 @@ static int flask_security_make_bools(void);
extern int ss_initialized;
-static int __init parse_flask_param(const char *s)
+static int __init cf_check parse_flask_param(const char *s)
{
if ( !strcmp(s, "enforcing") )
flask_bootparam = FLASK_BOOTPARAM_ENFORCING;
@@ -75,6 +73,7 @@ static int __init parse_flask_param(const char *s)
return (flask_bootparam == FLASK_BOOTPARAM_INVALID) ? -EINVAL : 0;
}
+custom_param("flask", parse_flask_param);
static int domain_has_security(struct domain *d, u32 perms)
{
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 5fc3a5f75478..2286a502e3e8 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -55,7 +55,7 @@ static enum xsm_bootparam __initdata xsm_bootparam =
XSM_BOOTPARAM_DUMMY;
#endif
-static int __init parse_xsm_param(const char *s)
+static int __init cf_check parse_xsm_param(const char *s)
{
int rc = 0;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 10/70] xen: CFI hardening for __initcall()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (8 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 09/70] xen: CFI hardening for custom_param() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 11/70] xen: CFI hardening for notifier callbacks Andrew Cooper
` (61 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 4 ++--
xen/arch/x86/acpi/cpufreq/cpufreq.c | 2 +-
xen/arch/x86/cpu/mcheck/non-fatal.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/cpu/mtrr/main.c | 2 +-
xen/arch/x86/cpu/vpmu.c | 2 +-
xen/arch/x86/domain.c | 2 +-
xen/arch/x86/extable.c | 2 +-
xen/arch/x86/hvm/hvm.c | 4 ++--
xen/arch/x86/hvm/irq.c | 2 +-
xen/arch/x86/hvm/mtrr.c | 2 +-
xen/arch/x86/hvm/nestedhvm.c | 3 +--
xen/arch/x86/hvm/quirks.c | 2 +-
xen/arch/x86/include/asm/hvm/save.h | 2 +-
xen/arch/x86/ioport_emulate.c | 4 ++--
xen/arch/x86/irq.c | 4 ++--
xen/arch/x86/mm/shadow/common.c | 4 ++--
xen/arch/x86/msi.c | 2 +-
xen/arch/x86/nmi.c | 2 +-
xen/arch/x86/numa.c | 2 +-
xen/arch/x86/oprofile/nmi_int.c | 2 +-
xen/arch/x86/percpu.c | 2 +-
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/pv/domain.c | 2 +-
xen/arch/x86/shutdown.c | 2 +-
xen/arch/x86/time.c | 8 ++++----
xen/common/core_parking.c | 2 +-
xen/common/debugtrace.c | 2 +-
xen/common/event_channel.c | 2 +-
xen/common/gdbstub.c | 2 +-
xen/common/grant_table.c | 2 +-
xen/common/kernel.c | 4 ++--
xen/common/kexec.c | 2 +-
xen/common/livepatch.c | 2 +-
xen/common/page_alloc.c | 4 ++--
xen/common/radix-tree.c | 2 +-
xen/common/random.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
xen/common/spinlock.c | 2 +-
xen/common/stop_machine.c | 2 +-
xen/drivers/cpufreq/cpufreq.c | 2 +-
xen/drivers/cpufreq/cpufreq_misc_governors.c | 6 +++---
xen/drivers/cpufreq/cpufreq_ondemand.c | 2 +-
xen/drivers/passthrough/amd/iommu.h | 2 +-
xen/drivers/passthrough/amd/iommu_init.c | 2 +-
xen/drivers/passthrough/pci.c | 2 +-
xen/drivers/passthrough/vtd/iommu.c | 2 +-
xen/drivers/passthrough/x86/hvm.c | 2 +-
48 files changed, 60 insertions(+), 61 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index 5d73eb5917af..7902ccce6b98 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -410,7 +410,7 @@ static void dump_cx(unsigned char key)
}
}
-static int __init cpu_idle_key_init(void)
+static int __init cf_check cpu_idle_key_init(void)
{
register_keyhandler('c', dump_cx, "dump ACPI Cx structures", 1);
return 0;
@@ -1655,7 +1655,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback
};
-static int __init cpuidle_presmp_init(void)
+static int __init cf_check cpuidle_presmp_init(void)
{
void *cpu = (void *)(long)smp_processor_id();
diff --git a/xen/arch/x86/acpi/cpufreq/cpufreq.c b/xen/arch/x86/acpi/cpufreq/cpufreq.c
index 029c9398c42a..9510f05340aa 100644
--- a/xen/arch/x86/acpi/cpufreq/cpufreq.c
+++ b/xen/arch/x86/acpi/cpufreq/cpufreq.c
@@ -630,7 +630,7 @@ static const struct cpufreq_driver __initconstrel acpi_cpufreq_driver = {
.exit = acpi_cpufreq_cpu_exit,
};
-static int __init cpufreq_driver_init(void)
+static int __init cf_check cpufreq_driver_init(void)
{
int ret = 0;
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c
index ec52d37c96e1..2679c220a8a2 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -86,7 +86,7 @@ static void mce_work_fn(void *data)
adjust = 0;
}
-static int __init init_nonfatal_mce_checker(void)
+static int __init cf_check init_nonfatal_mce_checker(void)
{
struct cpuinfo_x86 *c = &boot_cpu_data;
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 95d35ca0f3f7..46f55fe7f191 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -696,7 +696,7 @@ int microcode_update(XEN_GUEST_HANDLE(const_void) buf, unsigned long len)
microcode_update_helper, buffer);
}
-static int __init microcode_init(void)
+static int __init cf_check microcode_init(void)
{
/*
* At this point, all CPUs should have updated their microcode
diff --git a/xen/arch/x86/cpu/mtrr/main.c b/xen/arch/x86/cpu/mtrr/main.c
index e9df53f00d61..428133100d46 100644
--- a/xen/arch/x86/cpu/mtrr/main.c
+++ b/xen/arch/x86/cpu/mtrr/main.c
@@ -632,7 +632,7 @@ void mtrr_bp_restore(void)
mtrr_if->set_all();
}
-static int __init mtrr_init_finialize(void)
+static int __init cf_check mtrr_init_finialize(void)
{
if (!mtrr_if)
return 0;
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index b10d6e2eb458..9fc897dc8403 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -844,7 +844,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback
};
-static int __init vpmu_init(void)
+static int __init cf_check vpmu_init(void)
{
int vendor = current_cpu_data.x86_vendor;
const struct arch_vpmu_ops *ops = NULL;
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ef1812dc1402..f943283b2a88 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -2532,7 +2532,7 @@ static void vcpu_kick_softirq(void)
*/
}
-static int __init init_vcpu_kick_softirq(void)
+static int __init cf_check init_vcpu_kick_softirq(void)
{
open_softirq(VCPU_KICK_SOFTIRQ, vcpu_kick_softirq);
return 0;
diff --git a/xen/arch/x86/extable.c b/xen/arch/x86/extable.c
index 78d672722580..51ef863d786c 100644
--- a/xen/arch/x86/extable.c
+++ b/xen/arch/x86/extable.c
@@ -126,7 +126,7 @@ search_exception_table(const struct cpu_user_regs *regs)
#ifndef NDEBUG
#include <asm/traps.h>
-static int __init stub_selftest(void)
+static int __init cf_check stub_selftest(void)
{
static const struct {
uint8_t opc[4];
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4e685c1b0c1b..2ae1685d0aff 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -164,7 +164,7 @@ static bool __init hap_supported(struct hvm_function_table *fns)
return true;
}
-static int __init hvm_enable(void)
+static int __init cf_check hvm_enable(void)
{
const struct hvm_function_table *fns = NULL;
@@ -1506,7 +1506,7 @@ static int hvm_load_cpu_msrs(struct domain *d, hvm_domain_context_t *h)
/* We need variable length data chunks for XSAVE area and MSRs, hence
* a custom declaration rather than HVM_REGISTER_SAVE_RESTORE.
*/
-static int __init hvm_register_CPU_save_and_restore(void)
+static int __init cf_check hvm_register_CPU_save_and_restore(void)
{
hvm_register_savevm(CPU_XSAVE_CODE,
"CPU_XSAVE",
diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index 52aae4565f0c..6045c9149bad 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -650,7 +650,7 @@ static void dump_irq_info(unsigned char key)
rcu_read_unlock(&domlist_read_lock);
}
-static int __init dump_irq_info_key_init(void)
+static int __init cf_check dump_irq_info_key_init(void)
{
register_keyhandler('I', dump_irq_info, "dump HVM irq info", 1);
return 0;
diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
index 4a9f3177edd4..b3ef1bf54133 100644
--- a/xen/arch/x86/hvm/mtrr.c
+++ b/xen/arch/x86/hvm/mtrr.c
@@ -75,7 +75,7 @@ static uint8_t __read_mostly mtrr_epat_tbl[MTRR_NUM_TYPES][MEMORY_NUM_TYPES] =
static uint8_t __read_mostly pat_entry_tbl[PAT_TYPE_NUMS] =
{ [0 ... PAT_TYPE_NUMS-1] = INVALID_MEM_TYPE };
-static int __init hvm_mtrr_pat_init(void)
+static int __init cf_check hvm_mtrr_pat_init(void)
{
unsigned int i, j;
diff --git a/xen/arch/x86/hvm/nestedhvm.c b/xen/arch/x86/hvm/nestedhvm.c
index 5021da667a47..23516884482b 100644
--- a/xen/arch/x86/hvm/nestedhvm.c
+++ b/xen/arch/x86/hvm/nestedhvm.c
@@ -125,8 +125,7 @@ nestedhvm_vmcx_flushtlb(struct p2m_domain *p2m)
* iomap[2] set set
*/
-static int __init
-nestedhvm_setup(void)
+static int __init cf_check nestedhvm_setup(void)
{
/* Same format and size as hvm_io_bitmap (Intel needs only 2 pages). */
unsigned nr = cpu_has_vmx ? 2 : 3;
diff --git a/xen/arch/x86/hvm/quirks.c b/xen/arch/x86/hvm/quirks.c
index 54cc66c382b6..917356b1312c 100644
--- a/xen/arch/x86/hvm/quirks.c
+++ b/xen/arch/x86/hvm/quirks.c
@@ -36,7 +36,7 @@ static int __init dmi_hvm_deny_port80(const struct dmi_system_id *id)
return 0;
}
-static int __init check_port80(void)
+static int __init cf_check check_port80(void)
{
/*
* Quirk table for systems that misbehave (lock up, etc.) if port
diff --git a/xen/arch/x86/include/asm/hvm/save.h b/xen/arch/x86/include/asm/hvm/save.h
index 4efc53505500..e975011ddb71 100644
--- a/xen/arch/x86/include/asm/hvm/save.h
+++ b/xen/arch/x86/include/asm/hvm/save.h
@@ -115,7 +115,7 @@ void hvm_register_savevm(uint16_t typecode,
/* Syntactic sugar around that function: specify the max number of
* saves, and this calculates the size of buffer needed */
#define HVM_REGISTER_SAVE_RESTORE(_x, _save, _load, _num, _k) \
-static int __init __hvm_register_##_x##_save_and_restore(void) \
+static int __init cf_check __hvm_register_##_x##_save_and_restore(void) \
{ \
hvm_register_savevm(HVM_SAVE_CODE(_x), \
#_x, \
diff --git a/xen/arch/x86/ioport_emulate.c b/xen/arch/x86/ioport_emulate.c
index cf1f3f922959..6caeb3d470ce 100644
--- a/xen/arch/x86/ioport_emulate.c
+++ b/xen/arch/x86/ioport_emulate.c
@@ -11,7 +11,7 @@
unsigned int (*__read_mostly ioemul_handle_quirk)(
uint8_t opcode, char *io_emul_stub, struct cpu_user_regs *regs);
-static unsigned int ioemul_handle_proliant_quirk(
+static unsigned int cf_check ioemul_handle_proliant_quirk(
u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs)
{
static const char stub[] = {
@@ -100,7 +100,7 @@ static const struct dmi_system_id __initconstrel ioport_quirks_tbl[] = {
{ }
};
-static int __init ioport_quirks_init(void)
+static int __init cf_check ioport_quirks_init(void)
{
if ( dmi_check_system(ioport_quirks_tbl) )
ioemul_handle_quirk = ioemul_handle_proliant_quirk;
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 84b174d0f51f..bcf46cd54d16 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -954,7 +954,7 @@ static void irq_ratelimit_timer_fn(void *data)
spin_unlock_irqrestore(&irq_ratelimit_lock, flags);
}
-static int __init irq_ratelimit_init(void)
+static int __init cf_check irq_ratelimit_init(void)
{
if ( irq_ratelimit_threshold )
init_timer(&irq_ratelimit_timer, irq_ratelimit_timer_fn, NULL, 0);
@@ -2504,7 +2504,7 @@ static void dump_irqs(unsigned char key)
dump_ioapic_irq_info();
}
-static int __init setup_dump_irqs(void)
+static int __init cf_check setup_dump_irqs(void)
{
/* In lieu of being able to live in init_irq_data(). */
BUILD_BUG_ON(sizeof(irq_max_guests) >
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 6221630fc2dc..b8730a9441ce 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -105,7 +105,7 @@ static void shadow_audit_key(unsigned char key)
__func__, shadow_audit_enable);
}
-static int __init shadow_audit_key_init(void)
+static int __init cf_check shadow_audit_key_init(void)
{
register_keyhandler('O', shadow_audit_key, "toggle shadow audits", 0);
return 0;
@@ -1057,7 +1057,7 @@ static void shadow_blow_all_tables(unsigned char c)
}
/* Register this function in the Xen console keypress table */
-static __init int shadow_blow_tables_keyhandler_init(void)
+static int __init cf_check shadow_blow_tables_keyhandler_init(void)
{
register_keyhandler('S', shadow_blow_all_tables, "reset shadow pagetables", 1);
return 0;
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 5febc0ea4b7c..d1497254b188 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -1485,7 +1485,7 @@ static void dump_msi(unsigned char key)
vpci_dump_msi();
}
-static int __init msi_setup_keyhandler(void)
+static int __init cf_check msi_setup_keyhandler(void)
{
register_keyhandler('M', dump_msi, "dump MSI state", 1);
return 0;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index 1a225d499295..21e947a46f24 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -606,7 +606,7 @@ static void do_nmi_stats(unsigned char key)
printk("%pv: NMI neither pending nor masked\n", v);
}
-static __init int register_nmi_trigger(void)
+static int __init cf_check register_nmi_trigger(void)
{
register_keyhandler('N', do_nmi_trigger, "trigger an NMI", 0);
register_keyhandler('n', do_nmi_stats, "NMI statistics", 1);
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 6be5a0c93322..5de9db4e9943 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -504,7 +504,7 @@ static void dump_numa(unsigned char key)
rcu_read_unlock(&domlist_read_lock);
}
-static __init int register_numa_trigger(void)
+static int __init cf_check register_numa_trigger(void)
{
register_keyhandler('u', dump_numa, "dump NUMA info", 1);
return 0;
diff --git a/xen/arch/x86/oprofile/nmi_int.c b/xen/arch/x86/oprofile/nmi_int.c
index 7842d95b95ea..ba9c4b9804ca 100644
--- a/xen/arch/x86/oprofile/nmi_int.c
+++ b/xen/arch/x86/oprofile/nmi_int.c
@@ -388,7 +388,7 @@ static int __init arch_perfmon_init(char **cpu_type)
return 1;
}
-static int __init nmi_init(void)
+static int __init cf_check nmi_init(void)
{
__u8 vendor = current_cpu_data.x86_vendor;
__u8 family = current_cpu_data.x86;
diff --git a/xen/arch/x86/percpu.c b/xen/arch/x86/percpu.c
index 5ea14b6ec312..0e0b6577ca45 100644
--- a/xen/arch/x86/percpu.c
+++ b/xen/arch/x86/percpu.c
@@ -94,7 +94,7 @@ static struct notifier_block cpu_percpu_nfb = {
.priority = 100 /* highest priority */
};
-static int __init percpu_presmp_init(void)
+static int __init cf_check percpu_presmp_init(void)
{
register_cpu_notifier(&cpu_percpu_nfb);
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 56916344cb1d..9a3670afc341 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -1675,7 +1675,7 @@ static struct notifier_block cpu_nfb = {
.priority = -1
};
-static int __init psr_presmp_init(void)
+static int __init cf_check psr_presmp_init(void)
{
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 125c4561a7ea..55146c15c853 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -167,7 +167,7 @@ unsigned long pv_fixup_guest_cr4(const struct vcpu *v, unsigned long cr4)
static int8_t __read_mostly opt_global_pages = -1;
boolean_runtime_param("global-pages", opt_global_pages);
-static int __init pge_init(void)
+static int __init cf_check pge_init(void)
{
if ( opt_global_pages == -1 )
opt_global_pages = !cpu_has_hypervisor ||
diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index a01354d93319..ad3e3a76916f 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -533,7 +533,7 @@ static const struct dmi_system_id __initconstrel reboot_dmi_table[] = {
{ }
};
-static int __init reboot_init(void)
+static int __init cf_check reboot_init(void)
{
/*
* Only do the DMI check if reboot_type hasn't been overridden
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 2da9c7f5c8dd..14f7d3fd635e 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -537,7 +537,7 @@ static struct platform_timesource __initdata plt_pmtimer =
static struct time_scale __read_mostly pmt_scale;
-static __init int init_pmtmr_scale(void)
+static __init int cf_check init_pmtmr_scale(void)
{
set_time_scale(&pmt_scale, ACPI_PM_FREQUENCY);
return 0;
@@ -2052,7 +2052,7 @@ static void __init try_platform_timer_tail(void)
}
/* Late init function, after all cpus have booted */
-static int __init verify_tsc_reliability(void)
+static int __init cf_check verify_tsc_reliability(void)
{
if ( boot_cpu_has(X86_FEATURE_TSC_RELIABLE) )
{
@@ -2223,7 +2223,7 @@ static int _disable_pit_irq(void(*hpet_broadcast_setup)(void))
return ret;
}
-static int __init disable_pit_irq(void)
+static int __init cf_check disable_pit_irq(void)
{
if ( !_disable_pit_irq(hpet_broadcast_init) )
{
@@ -2586,7 +2586,7 @@ static void dump_softtsc(unsigned char key)
printk("No domains have emulated TSC\n");
}
-static int __init setup_dump_softtsc(void)
+static int __init cf_check setup_dump_softtsc(void)
{
register_keyhandler('s', dump_softtsc, "dump softtsc stats", 1);
return 0;
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index aa432ed2f57b..44a907abfd7f 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -258,7 +258,7 @@ static int __init register_core_parking_policy(const struct cp_policy *policy)
return 0;
}
-static int __init core_parking_init(void)
+static int __init cf_check core_parking_init(void)
{
int ret = 0;
diff --git a/xen/common/debugtrace.c b/xen/common/debugtrace.c
index 29b11239f5a5..f3c0fd8aa17b 100644
--- a/xen/common/debugtrace.c
+++ b/xen/common/debugtrace.c
@@ -279,7 +279,7 @@ static struct notifier_block debugtrace_nfb = {
.notifier_call = debugtrace_cpu_callback
};
-static int __init debugtrace_init(void)
+static int __init cf_check debugtrace_init(void)
{
unsigned int cpu;
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index a5ee8b8ebf76..2026bc30dc95 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -1642,7 +1642,7 @@ static void dump_evtchn_info(unsigned char key)
rcu_read_unlock(&domlist_read_lock);
}
-static int __init dump_evtchn_info_key_init(void)
+static int __init cf_check dump_evtchn_info_key_init(void)
{
register_keyhandler('e', dump_evtchn_info, "dump evtchn info", 1);
return 0;
diff --git a/xen/common/gdbstub.c b/xen/common/gdbstub.c
index 848c1f4327e8..99bfd9a654c9 100644
--- a/xen/common/gdbstub.c
+++ b/xen/common/gdbstub.c
@@ -640,7 +640,7 @@ __trap_to_gdb(struct cpu_user_regs *regs, unsigned long cookie)
return rc;
}
-static int __init initialise_gdb(void)
+static int __init cf_check initialise_gdb(void)
{
if ( *opt_gdb == '\0' )
return 0;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index b663845d9c6c..1078e3e16cda 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -4291,7 +4291,7 @@ static void gnttab_usage_print_all(unsigned char key)
printk("%s ] done\n", __func__);
}
-static int __init gnttab_usage_init(void)
+static int __init cf_check gnttab_usage_init(void)
{
register_keyhandler('g', gnttab_usage_print_all,
"print grant table usage", 1);
diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index 752c2e0dae44..adff2d2c77f3 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -393,7 +393,7 @@ static HYPFS_STRING_INIT(extra, "extra");
static HYPFS_STRING_INIT(config, "config");
#endif
-static int __init buildinfo_init(void)
+static int __init cf_check buildinfo_init(void)
{
hypfs_add_dir(&hypfs_root, &buildinfo, true);
@@ -431,7 +431,7 @@ __initcall(buildinfo_init);
static HYPFS_DIR_INIT(params, "params");
-static int __init param_init(void)
+static int __init cf_check param_init(void)
{
struct param_hypfs *param;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 6286c0bbf08b..36384f782db3 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -570,7 +570,7 @@ void __init kexec_early_calculations(void)
crashinfo_maxaddr_bits = fls64(crashinfo_maxaddr) - 1;
}
-static int __init kexec_init(void)
+static int __init cf_check kexec_init(void)
{
void *cpu = (void *)(unsigned long)smp_processor_id();
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 7118551b27e2..33708b4e2388 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -2139,7 +2139,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback
};
-static int __init livepatch_init(void)
+static int __init cf_check livepatch_init(void)
{
unsigned int cpu;
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index ad06655158d2..561e238d2d6a 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2549,7 +2549,7 @@ static void pagealloc_info(unsigned char key)
printk(" Dom heap: %lukB free\n", total << (PAGE_SHIFT-10));
}
-static __init int pagealloc_keyhandler_init(void)
+static __init int cf_check pagealloc_keyhandler_init(void)
{
register_keyhandler('m', pagealloc_info, "memory info", 1);
return 0;
@@ -2597,7 +2597,7 @@ static void dump_heap(unsigned char key)
}
}
-static __init int register_heap_trigger(void)
+static __init int cf_check register_heap_trigger(void)
{
register_keyhandler('H', dump_heap, "dump heap info", 1);
return 0;
diff --git a/xen/common/radix-tree.c b/xen/common/radix-tree.c
index 2384655a2e90..628a7e06988f 100644
--- a/xen/common/radix-tree.c
+++ b/xen/common/radix-tree.c
@@ -744,7 +744,7 @@ static __init unsigned long __maxindex(unsigned int height)
return ~0UL >> shift;
}
-static __init int radix_tree_init_maxindex(void)
+static int __init cf_check radix_tree_init_maxindex(void)
{
unsigned int i;
diff --git a/xen/common/random.c b/xen/common/random.c
index fb805b0ecd95..a29f2fcb991a 100644
--- a/xen/common/random.c
+++ b/xen/common/random.c
@@ -31,7 +31,7 @@ unsigned int get_random(void)
return val;
}
-static int __init init_boot_random(void)
+static int __init cf_check init_boot_random(void)
{
boot_random = get_random();
return 0;
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index f0dd626054a6..f26c7f289539 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -1218,7 +1218,7 @@ static void cpupool_hypfs_init(void)
#endif /* CONFIG_HYPFS */
-static int __init cpupool_init(void)
+static int __init cf_check cpupool_init(void)
{
unsigned int cpu;
diff --git a/xen/common/spinlock.c b/xen/common/spinlock.c
index b90981bb271e..5ce7e3363863 100644
--- a/xen/common/spinlock.c
+++ b/xen/common/spinlock.c
@@ -508,7 +508,7 @@ void _lock_profile_deregister_struct(
spin_unlock(&lock_profile_lock);
}
-static int __init lock_prof_init(void)
+static int __init cf_check lock_prof_init(void)
{
struct lock_profile **q;
diff --git a/xen/common/stop_machine.c b/xen/common/stop_machine.c
index 2d5f6aef61ed..8979d553d677 100644
--- a/xen/common/stop_machine.c
+++ b/xen/common/stop_machine.c
@@ -198,7 +198,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback
};
-static int __init cpu_stopmachine_init(void)
+static int __init cf_check cpu_stopmachine_init(void)
{
unsigned int cpu;
for_each_online_cpu ( cpu )
diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
index 36b079296235..e55e202d5a18 100644
--- a/xen/drivers/cpufreq/cpufreq.c
+++ b/xen/drivers/cpufreq/cpufreq.c
@@ -657,7 +657,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback
};
-static int __init cpufreq_presmp_init(void)
+static int __init cf_check cpufreq_presmp_init(void)
{
register_cpu_notifier(&cpu_nfb);
return 0;
diff --git a/xen/drivers/cpufreq/cpufreq_misc_governors.c b/xen/drivers/cpufreq/cpufreq_misc_governors.c
index 746bbcd5ff36..8343f491da87 100644
--- a/xen/drivers/cpufreq/cpufreq_misc_governors.c
+++ b/xen/drivers/cpufreq/cpufreq_misc_governors.c
@@ -116,7 +116,7 @@ struct cpufreq_governor cpufreq_gov_userspace = {
.handle_option = cpufreq_userspace_handle_option
};
-static int __init cpufreq_gov_userspace_init(void)
+static int __init cf_check cpufreq_gov_userspace_init(void)
{
unsigned int cpu;
@@ -160,7 +160,7 @@ struct cpufreq_governor cpufreq_gov_performance = {
.governor = cpufreq_governor_performance,
};
-static int __init cpufreq_gov_performance_init(void)
+static int __init cf_check cpufreq_gov_performance_init(void)
{
return cpufreq_register_governor(&cpufreq_gov_performance);
}
@@ -199,7 +199,7 @@ struct cpufreq_governor cpufreq_gov_powersave = {
.governor = cpufreq_governor_powersave,
};
-static int __init cpufreq_gov_powersave_init(void)
+static int __init cf_check cpufreq_gov_powersave_init(void)
{
return cpufreq_register_governor(&cpufreq_gov_powersave);
}
diff --git a/xen/drivers/cpufreq/cpufreq_ondemand.c b/xen/drivers/cpufreq/cpufreq_ondemand.c
index 6b905d7cfca8..cabd9ffa8886 100644
--- a/xen/drivers/cpufreq/cpufreq_ondemand.c
+++ b/xen/drivers/cpufreq/cpufreq_ondemand.c
@@ -356,7 +356,7 @@ struct cpufreq_governor cpufreq_gov_dbs = {
.handle_option = cpufreq_dbs_handle_option
};
-static int __init cpufreq_gov_dbs_init(void)
+static int __init cf_check cpufreq_gov_dbs_init(void)
{
return cpufreq_register_governor(&cpufreq_gov_dbs);
}
diff --git a/xen/drivers/passthrough/amd/iommu.h b/xen/drivers/passthrough/amd/iommu.h
index 93243424e85d..04517c1a024c 100644
--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -234,7 +234,7 @@ int amd_iommu_prepare(bool xt);
int amd_iommu_init(bool xt);
int amd_iommu_init_late(void);
int amd_iommu_update_ivrs_mapping_acpi(void);
-int iov_adjust_irq_affinities(void);
+int cf_check iov_adjust_irq_affinities(void);
int amd_iommu_quarantine_init(struct domain *d);
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 559a734bdaa5..f1ed75558227 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -808,7 +808,7 @@ static bool_t __init set_iommu_interrupt_handler(struct amd_iommu *iommu)
return 1;
}
-int iov_adjust_irq_affinities(void)
+int cf_check iov_adjust_irq_affinities(void)
{
const struct amd_iommu *iommu;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 20cb246598b1..ed459f6409a8 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1358,7 +1358,7 @@ static void dump_pci_devices(unsigned char ch)
pcidevs_unlock();
}
-static int __init setup_dump_pcidevs(void)
+static int __init cf_check setup_dump_pcidevs(void)
{
register_keyhandler('Q', dump_pci_devices, "dump PCI devices", 1);
return 0;
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 34ea5f485de7..9e85016830bf 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2154,7 +2154,7 @@ static void adjust_irq_affinity(struct acpi_drhd_unit *drhd)
spin_unlock_irqrestore(&desc->lock, flags);
}
-static int adjust_vtd_irq_affinities(void)
+static int cf_check adjust_vtd_irq_affinities(void)
{
struct acpi_drhd_unit *drhd;
diff --git a/xen/drivers/passthrough/x86/hvm.c b/xen/drivers/passthrough/x86/hvm.c
index 0b37cd145b60..e5a2c5830348 100644
--- a/xen/drivers/passthrough/x86/hvm.c
+++ b/xen/drivers/passthrough/x86/hvm.c
@@ -1076,7 +1076,7 @@ static struct notifier_block cpu_nfb = {
.notifier_call = cpu_callback,
};
-static int __init setup_dpci_softirq(void)
+static int __init cf_check setup_dpci_softirq(void)
{
unsigned int cpu;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 11/70] xen: CFI hardening for notifier callbacks
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (9 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 10/70] xen: CFI hardening for __initcall() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 12/70] xen: CFI hardening for acpi_table_parse() Andrew Cooper
` (60 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 2 +-
xen/arch/x86/cpu/mcheck/mce.c | 2 +-
xen/arch/x86/cpu/mcheck/mce_intel.c | 2 +-
xen/arch/x86/cpu/mwait-idle.c | 4 ++--
xen/arch/x86/cpu/vpmu.c | 2 +-
xen/arch/x86/genapic/x2apic.c | 2 +-
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/nmi.c | 2 +-
xen/arch/x86/percpu.c | 2 +-
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/smpboot.c | 2 +-
xen/common/debugtrace.c | 4 ++--
xen/common/kexec.c | 2 +-
xen/common/livepatch.c | 2 +-
xen/common/rcupdate.c | 2 +-
xen/common/sched/core.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
xen/common/stop_machine.c | 2 +-
xen/common/tasklet.c | 2 +-
xen/common/timer.c | 2 +-
xen/common/trace.c | 2 +-
xen/drivers/cpufreq/cpufreq.c | 2 +-
xen/drivers/cpufreq/cpufreq_misc_governors.c | 2 +-
xen/drivers/passthrough/x86/hvm.c | 2 +-
24 files changed, 26 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index 7902ccce6b98..fb47eb9ad68e 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -1622,7 +1622,7 @@ bool cpuidle_using_deep_cstate(void)
: ACPI_STATE_C1);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index ea86d84481b2..a449fa0424ce 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -733,7 +733,7 @@ static int cpu_bank_alloc(unsigned int cpu)
return 0;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c
index d63d6083d3a7..b6da8262e662 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -931,7 +931,7 @@ static int cpu_mcabank_alloc(unsigned int cpu)
return -ENOMEM;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index 24d073d315ac..fe1b7af25f01 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -1275,8 +1275,8 @@ static int __init mwait_idle_probe(void)
return 0;
}
-static int mwait_idle_cpu_init(struct notifier_block *nfb,
- unsigned long action, void *hcpu)
+static int cf_check mwait_idle_cpu_init(
+ struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu, cstate;
struct acpi_processor_power *dev = processor_powers[cpu];
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index 9fc897dc8403..df3c9201b2c1 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -817,7 +817,7 @@ long cf_check do_xenpmu_op(
return ret;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/genapic/x2apic.c b/xen/arch/x86/genapic/x2apic.c
index 9064a0ca4696..bd44bb753995 100644
--- a/xen/arch/x86/genapic/x2apic.c
+++ b/xen/arch/x86/genapic/x2apic.c
@@ -187,7 +187,7 @@ static const struct genapic __initconstrel apic_x2apic_cluster = {
.send_IPI_self = send_IPI_self_x2apic
};
-static int update_clusterinfo(
+static int cf_check update_clusterinfo(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 2ae1685d0aff..0a193530680a 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -117,7 +117,7 @@ static const char __initconst warning_hvm_fep[] =
static bool_t __initdata opt_altp2m_enabled = 0;
boolean_param("altp2m", opt_altp2m_enabled);
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index 21e947a46f24..5c101a9f97b3 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -428,7 +428,7 @@ void setup_apic_nmi_watchdog(void)
nmi_active = 1;
}
-static int cpu_nmi_callback(
+static int cf_check cpu_nmi_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/percpu.c b/xen/arch/x86/percpu.c
index 0e0b6577ca45..eb3ba7bc8874 100644
--- a/xen/arch/x86/percpu.c
+++ b/xen/arch/x86/percpu.c
@@ -63,7 +63,7 @@ static void free_percpu_area(unsigned int cpu)
call_rcu(&info->rcu, _free_percpu_area);
}
-static int cpu_percpu_callback(
+static int cf_check cpu_percpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 9a3670afc341..5b9991bd5b12 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -1642,7 +1642,7 @@ static void psr_cpu_fini(unsigned int cpu)
free_socket_resources(socket);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
int rc = 0;
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 335129a0104d..709704d71ada 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -1112,7 +1112,7 @@ static int cpu_smpboot_alloc(unsigned int cpu)
return rc;
}
-static int cpu_smpboot_callback(
+static int cf_check cpu_smpboot_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/debugtrace.c b/xen/common/debugtrace.c
index f3c0fd8aa17b..160d00b79607 100644
--- a/xen/common/debugtrace.c
+++ b/xen/common/debugtrace.c
@@ -263,8 +263,8 @@ static void debugtrace_alloc_buffer(struct debugtrace_data **ptr,
*ptr = data;
}
-static int debugtrace_cpu_callback(struct notifier_block *nfb,
- unsigned long action, void *hcpu)
+static int cf_check debugtrace_cpu_callback(
+ struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 36384f782db3..3b223cd03d75 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -531,7 +531,7 @@ static int kexec_init_cpu_notes(const unsigned long cpu)
return ret;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned long cpu = (unsigned long)hcpu;
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 33708b4e2388..701efd87a173 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -2124,7 +2124,7 @@ static void livepatch_printall(unsigned char key)
spin_unlock(&payload_lock);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
index a5a27af3def0..2ec5606de5dd 100644
--- a/xen/common/rcupdate.c
+++ b/xen/common/rcupdate.c
@@ -641,7 +641,7 @@ static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp,
init_timer(&rdp->idle_timer, rcu_idle_timer_handler, rdp, cpu);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 0f527024ba2e..4a79971a1d45 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -2839,7 +2839,7 @@ void sched_rm_cpu(unsigned int cpu)
cpu_schedule_down(cpu);
}
-static int cpu_schedule_callback(
+static int cf_check cpu_schedule_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index f26c7f289539..e5cfb03b857e 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -985,7 +985,7 @@ void dump_runq(unsigned char key)
spin_unlock(&cpupool_lock);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/stop_machine.c b/xen/common/stop_machine.c
index 8979d553d677..a122bd4afe09 100644
--- a/xen/common/stop_machine.c
+++ b/xen/common/stop_machine.c
@@ -182,7 +182,7 @@ static void stopmachine_action(void *data)
local_irq_enable();
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/tasklet.c b/xen/common/tasklet.c
index ac89511a0955..1b16bbcdeb0b 100644
--- a/xen/common/tasklet.c
+++ b/xen/common/tasklet.c
@@ -214,7 +214,7 @@ void softirq_tasklet_init(struct tasklet *t, void (*func)(void *), void *data)
t->is_softirq = 1;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/timer.c b/xen/common/timer.c
index 1bb265ceea0e..b788050ea1d8 100644
--- a/xen/common/timer.c
+++ b/xen/common/timer.c
@@ -637,7 +637,7 @@ static void free_percpu_timers(unsigned int cpu)
ASSERT(ts->heap == dummy_heap);
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/common/trace.c b/xen/common/trace.c
index 61fecc2b2bd2..b5358508f804 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -79,7 +79,7 @@ static u32 tb_event_mask = TRC_ALL;
* i.e., sizeof(_type) * ans >= _x. */
#define fit_to_type(_type, _x) (((_x)+sizeof(_type)-1) / sizeof(_type))
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
index e55e202d5a18..a94520ee57ac 100644
--- a/xen/drivers/cpufreq/cpufreq.c
+++ b/xen/drivers/cpufreq/cpufreq.c
@@ -632,7 +632,7 @@ static int __init cpufreq_cmdline_parse(const char *s)
return rc;
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/drivers/cpufreq/cpufreq_misc_governors.c b/xen/drivers/cpufreq/cpufreq_misc_governors.c
index 8343f491da87..ad79d0f5d246 100644
--- a/xen/drivers/cpufreq/cpufreq_misc_governors.c
+++ b/xen/drivers/cpufreq/cpufreq_misc_governors.c
@@ -91,7 +91,7 @@ cpufreq_userspace_handle_option(const char *name, const char *val)
return 0;
}
-static int cpufreq_userspace_cpu_callback(
+static int cf_check cpufreq_userspace_cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
diff --git a/xen/drivers/passthrough/x86/hvm.c b/xen/drivers/passthrough/x86/hvm.c
index e5a2c5830348..527bd6a56d83 100644
--- a/xen/drivers/passthrough/x86/hvm.c
+++ b/xen/drivers/passthrough/x86/hvm.c
@@ -1046,7 +1046,7 @@ static void dpci_softirq(void)
}
}
-static int cpu_callback(
+static int cf_check cpu_callback(
struct notifier_block *nfb, unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 12/70] xen: CFI hardening for acpi_table_parse()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (10 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 11/70] xen: CFI hardening for notifier callbacks Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 13/70] xen: CFI hardening for continue_hypercall_on_cpu() Andrew Cooper
` (59 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/boot.c | 24 ++++++++++++------------
xen/arch/x86/hvm/dom0_build.c | 16 ++++++++--------
xen/arch/x86/include/asm/tboot.h | 2 +-
xen/arch/x86/srat.c | 4 ++--
xen/arch/x86/tboot.c | 2 +-
xen/arch/x86/x86_64/acpi_mmcfg.c | 2 +-
xen/arch/x86/x86_64/mmconfig.h | 2 +-
xen/drivers/acpi/apei/hest.c | 4 ++--
xen/drivers/acpi/numa.c | 10 +++++-----
xen/drivers/passthrough/amd/iommu_acpi.c | 9 +++++----
xen/drivers/passthrough/pci.c | 3 ++-
xen/drivers/passthrough/vtd/dmar.c | 2 +-
xen/include/xen/acpi.h | 2 +-
13 files changed, 42 insertions(+), 40 deletions(-)
diff --git a/xen/arch/x86/acpi/boot.c b/xen/arch/x86/acpi/boot.c
index cc4bbc0284fa..54b72d716bed 100644
--- a/xen/arch/x86/acpi/boot.c
+++ b/xen/arch/x86/acpi/boot.c
@@ -60,7 +60,7 @@ static u64 acpi_lapic_addr __initdata = APIC_DEFAULT_PHYS_BASE;
Boot-time Configuration
-------------------------------------------------------------------------- */
-static int __init acpi_parse_madt(struct acpi_table_header *table)
+static int __init cf_check acpi_parse_madt(struct acpi_table_header *table)
{
struct acpi_table_madt *madt =
container_of(table, struct acpi_table_madt, header);
@@ -77,7 +77,7 @@ static int __init acpi_parse_madt(struct acpi_table_header *table)
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
{
struct acpi_madt_local_x2apic *processor =
@@ -133,7 +133,7 @@ acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_lapic(struct acpi_subtable_header * header, const unsigned long end)
{
struct acpi_madt_local_apic *processor =
@@ -171,7 +171,7 @@ acpi_parse_lapic(struct acpi_subtable_header * header, const unsigned long end)
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_lapic_addr_ovr(struct acpi_subtable_header * header,
const unsigned long end)
{
@@ -187,7 +187,7 @@ acpi_parse_lapic_addr_ovr(struct acpi_subtable_header * header,
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_x2apic_nmi(struct acpi_subtable_header *header,
const unsigned long end)
{
@@ -206,7 +206,7 @@ acpi_parse_x2apic_nmi(struct acpi_subtable_header *header,
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_lapic_nmi(struct acpi_subtable_header * header, const unsigned long end)
{
struct acpi_madt_local_apic_nmi *lapic_nmi =
@@ -223,7 +223,7 @@ acpi_parse_lapic_nmi(struct acpi_subtable_header * header, const unsigned long e
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_ioapic(struct acpi_subtable_header * header, const unsigned long end)
{
struct acpi_madt_io_apic *ioapic =
@@ -240,7 +240,7 @@ acpi_parse_ioapic(struct acpi_subtable_header * header, const unsigned long end)
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_int_src_ovr(struct acpi_subtable_header * header,
const unsigned long end)
{
@@ -267,7 +267,7 @@ acpi_parse_int_src_ovr(struct acpi_subtable_header * header,
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_nmi_src(struct acpi_subtable_header * header, const unsigned long end)
{
struct acpi_madt_nmi_source *nmi_src =
@@ -283,7 +283,7 @@ acpi_parse_nmi_src(struct acpi_subtable_header * header, const unsigned long end
return 0;
}
-static int __init acpi_parse_hpet(struct acpi_table_header *table)
+static int __init cf_check acpi_parse_hpet(struct acpi_table_header *table)
{
const struct acpi_table_hpet *hpet_tbl =
container_of(table, const struct acpi_table_hpet, header);
@@ -319,7 +319,7 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
return 0;
}
-static int __init acpi_invalidate_bgrt(struct acpi_table_header *table)
+static int __init cf_check acpi_invalidate_bgrt(struct acpi_table_header *table)
{
struct acpi_table_bgrt *bgrt_tbl =
container_of(table, struct acpi_table_bgrt, header);
@@ -472,7 +472,7 @@ acpi_fadt_parse_sleep_info(const struct acpi_table_fadt *fadt)
acpi_sinfo.wakeup_vector, acpi_sinfo.vector_width);
}
-static int __init acpi_parse_fadt(struct acpi_table_header *table)
+static int __init cf_check acpi_parse_fadt(struct acpi_table_header *table)
{
const struct acpi_table_fadt *fadt =
container_of(table, const struct acpi_table_fadt, header);
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index f9e17249dcda..25fb05a389df 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -745,15 +745,15 @@ static int __init pvh_setup_cpus(struct domain *d, paddr_t entry,
return 0;
}
-static int __init acpi_count_intr_ovr(struct acpi_subtable_header *header,
- const unsigned long end)
+static int __init cf_check acpi_count_intr_ovr(
+ struct acpi_subtable_header *header, const unsigned long end)
{
acpi_intr_overrides++;
return 0;
}
-static int __init acpi_set_intr_ovr(struct acpi_subtable_header *header,
- const unsigned long end)
+static int __init cf_check acpi_set_intr_ovr(
+ struct acpi_subtable_header *header, const unsigned long end)
{
const struct acpi_madt_interrupt_override *intr =
container_of(header, struct acpi_madt_interrupt_override, header);
@@ -764,15 +764,15 @@ static int __init acpi_set_intr_ovr(struct acpi_subtable_header *header,
return 0;
}
-static int __init acpi_count_nmi_src(struct acpi_subtable_header *header,
- const unsigned long end)
+static int __init cf_check acpi_count_nmi_src(
+ struct acpi_subtable_header *header, const unsigned long end)
{
acpi_nmi_sources++;
return 0;
}
-static int __init acpi_set_nmi_src(struct acpi_subtable_header *header,
- const unsigned long end)
+static int __init cf_check acpi_set_nmi_src(
+ struct acpi_subtable_header *header, const unsigned long end)
{
const struct acpi_madt_nmi_source *src =
container_of(header, struct acpi_madt_nmi_source, header);
diff --git a/xen/arch/x86/include/asm/tboot.h b/xen/arch/x86/include/asm/tboot.h
index bfeed1542fa3..818d5fa45132 100644
--- a/xen/arch/x86/include/asm/tboot.h
+++ b/xen/arch/x86/include/asm/tboot.h
@@ -124,7 +124,7 @@ void tboot_probe(void);
void tboot_shutdown(uint32_t shutdown_type);
int tboot_in_measured_env(void);
int tboot_protect_mem_regions(void);
-int tboot_parse_dmar_table(acpi_table_handler dmar_handler);
+int cf_check tboot_parse_dmar_table(acpi_table_handler dmar_handler);
int tboot_s3_resume(void);
void tboot_s3_error(int error);
int tboot_wake_ap(int apicid, unsigned long sipi_vec);
diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c
index 6b77b9820195..cfe24c7e781c 100644
--- a/xen/arch/x86/srat.c
+++ b/xen/arch/x86/srat.c
@@ -407,8 +407,8 @@ void __init acpi_numa_arch_fixup(void) {}
static uint64_t __initdata srat_region_mask;
-static int __init srat_parse_region(struct acpi_subtable_header *header,
- const unsigned long end)
+static int __init cf_check srat_parse_region(
+ struct acpi_subtable_header *header, const unsigned long end)
{
struct acpi_srat_mem_affinity *ma;
diff --git a/xen/arch/x86/tboot.c b/xen/arch/x86/tboot.c
index 529367ed8167..fe1abfdf08ff 100644
--- a/xen/arch/x86/tboot.c
+++ b/xen/arch/x86/tboot.c
@@ -450,7 +450,7 @@ int __init tboot_protect_mem_regions(void)
return 1;
}
-int __init tboot_parse_dmar_table(acpi_table_handler dmar_handler)
+int __init cf_check tboot_parse_dmar_table(acpi_table_handler dmar_handler)
{
int rc;
uint64_t size;
diff --git a/xen/arch/x86/x86_64/acpi_mmcfg.c b/xen/arch/x86/x86_64/acpi_mmcfg.c
index 0db8f57abbed..2159c68189e4 100644
--- a/xen/arch/x86/x86_64/acpi_mmcfg.c
+++ b/xen/arch/x86/x86_64/acpi_mmcfg.c
@@ -68,7 +68,7 @@ static int __init acpi_mcfg_check_entry(struct acpi_table_mcfg *mcfg,
return -EINVAL;
}
-int __init acpi_parse_mcfg(struct acpi_table_header *header)
+int __init cf_check acpi_parse_mcfg(struct acpi_table_header *header)
{
struct acpi_table_mcfg *mcfg;
unsigned long i;
diff --git a/xen/arch/x86/x86_64/mmconfig.h b/xen/arch/x86/x86_64/mmconfig.h
index 4d3b9fcbdd3c..433046be663a 100644
--- a/xen/arch/x86/x86_64/mmconfig.h
+++ b/xen/arch/x86/x86_64/mmconfig.h
@@ -76,7 +76,7 @@ static inline void mmio_config_writel(void __iomem *pos, u32 val)
/* function prototypes */
struct acpi_table_header;
-int acpi_parse_mcfg(struct acpi_table_header *header);
+int cf_check acpi_parse_mcfg(struct acpi_table_header *header);
int pci_mmcfg_reserved(uint64_t address, unsigned int segment,
unsigned int start_bus, unsigned int end_bus,
unsigned int flags);
diff --git a/xen/drivers/acpi/apei/hest.c b/xen/drivers/acpi/apei/hest.c
index c5f3aaab7c4e..5881275d2f37 100644
--- a/xen/drivers/acpi/apei/hest.c
+++ b/xen/drivers/acpi/apei/hest.c
@@ -128,8 +128,8 @@ int apei_hest_parse(apei_hest_func_t func, void *data)
* Check if firmware advertises firmware first mode. We need FF bit to be set
* along with a set of MC banks which work in FF mode.
*/
-static int __init hest_parse_cmc(const struct acpi_hest_header *hest_hdr,
- void *data)
+static int __init cf_check hest_parse_cmc(
+ const struct acpi_hest_header *hest_hdr, void *data)
{
#ifdef CONFIG_X86_MCE
unsigned int i;
diff --git a/xen/drivers/acpi/numa.c b/xen/drivers/acpi/numa.c
index 85f891757c21..bc6e888234e4 100644
--- a/xen/drivers/acpi/numa.c
+++ b/xen/drivers/acpi/numa.c
@@ -112,14 +112,14 @@ void __init acpi_table_print_srat_entry(struct acpi_subtable_header * header)
}
}
-static int __init acpi_parse_slit(struct acpi_table_header *table)
+static int __init cf_check acpi_parse_slit(struct acpi_table_header *table)
{
acpi_numa_slit_init((struct acpi_table_slit *)table);
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
const unsigned long end)
{
@@ -138,7 +138,7 @@ acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_processor_affinity(struct acpi_subtable_header *header,
const unsigned long end)
{
@@ -156,7 +156,7 @@ acpi_parse_processor_affinity(struct acpi_subtable_header *header,
return 0;
}
-static int __init
+static int __init cf_check
acpi_parse_memory_affinity(struct acpi_subtable_header *header,
const unsigned long end)
{
@@ -174,7 +174,7 @@ acpi_parse_memory_affinity(struct acpi_subtable_header *header,
return 0;
}
-int __init acpi_parse_srat(struct acpi_table_header *table)
+int __init cf_check acpi_parse_srat(struct acpi_table_header *table)
{
if (!table)
return -EINVAL;
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index 5ea227732821..3a7931458944 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -1078,7 +1078,7 @@ static inline bool_t is_ivmd_block(u8 type)
type == ACPI_IVRS_TYPE_MEMORY_IOMMU);
}
-static int __init parse_ivrs_table(struct acpi_table_header *table)
+static int __init cf_check parse_ivrs_table(struct acpi_table_header *table)
{
const struct acpi_ivrs_header *ivrs_block;
unsigned long length;
@@ -1170,7 +1170,7 @@ static int __init parse_ivrs_table(struct acpi_table_header *table)
return error;
}
-static int __init detect_iommu_acpi(struct acpi_table_header *table)
+static int __init cf_check detect_iommu_acpi(struct acpi_table_header *table)
{
const struct acpi_ivrs_header *ivrs_block;
unsigned long length = sizeof(struct acpi_table_ivrs);
@@ -1264,7 +1264,8 @@ static int __init get_last_bdf_ivhd(
return last_bdf;
}
-static int __init get_last_bdf_acpi(struct acpi_table_header *table)
+static int __init cf_check cf_check get_last_bdf_acpi(
+ struct acpi_table_header *table)
{
const struct acpi_ivrs_header *ivrs_block;
unsigned long length = sizeof(struct acpi_table_ivrs);
@@ -1306,7 +1307,7 @@ int __init amd_iommu_update_ivrs_mapping_acpi(void)
return acpi_table_parse(ACPI_SIG_IVRS, parse_ivrs_table);
}
-static int __init
+static int __init cf_check
get_supported_ivhd_type(struct acpi_table_header *table)
{
size_t length = sizeof(struct acpi_table_ivrs);
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index ed459f6409a8..c170dd06ba51 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1287,7 +1287,8 @@ static bool_t hest_source_is_pcie_aer(const struct acpi_hest_header *hest_hdr)
return 0;
}
-static int aer_hest_parse(const struct acpi_hest_header *hest_hdr, void *data)
+static int cf_check aer_hest_parse(
+ const struct acpi_hest_header *hest_hdr, void *data)
{
struct aer_hest_parse_info *info = data;
const struct acpi_hest_aer_common *p;
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index b152f3da916b..b8e91f5be1ae 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -767,7 +767,7 @@ acpi_parse_one_rhsa(struct acpi_dmar_header *header)
return ret;
}
-static int __init acpi_parse_dmar(struct acpi_table_header *table)
+static int __init cf_check acpi_parse_dmar(struct acpi_table_header *table)
{
struct acpi_table_dmar *dmar;
struct acpi_dmar_header *entry_header;
diff --git a/xen/include/xen/acpi.h b/xen/include/xen/acpi.h
index 08834f140266..39d51fcd01dd 100644
--- a/xen/include/xen/acpi.h
+++ b/xen/include/xen/acpi.h
@@ -91,7 +91,7 @@ struct acpi_subtable_header *acpi_table_get_entry_madt(enum acpi_madt_type id,
int acpi_table_parse_madt(enum acpi_madt_type id, acpi_table_entry_handler handler, unsigned int max_entries);
int acpi_table_parse_srat(int id, acpi_madt_entry_handler handler,
unsigned int max_entries);
-int acpi_parse_srat(struct acpi_table_header *);
+int cf_check acpi_parse_srat(struct acpi_table_header *);
void acpi_table_print (struct acpi_table_header *header, unsigned long phys_addr);
void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
void acpi_table_print_srat_entry (struct acpi_subtable_header *srat);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 13/70] xen: CFI hardening for continue_hypercall_on_cpu()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (11 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 12/70] xen: CFI hardening for acpi_table_parse() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 14/70] xen: CFI hardening for init_timer() Andrew Cooper
` (58 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/include/asm/pv/shim.h | 4 ++--
xen/arch/x86/include/asm/smp.h | 6 +++---
xen/arch/x86/platform_hypercall.c | 4 ++--
xen/arch/x86/pv/shim.c | 4 ++--
xen/arch/x86/smp.c | 4 ++--
xen/arch/x86/sysctl.c | 2 +-
xen/common/core_parking.c | 2 +-
xen/common/kexec.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
11 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 912d4c4d62f4..c4e7e8698920 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -326,7 +326,7 @@ static int enter_state(u32 state)
return error;
}
-static long enter_state_helper(void *data)
+static long cf_check enter_state_helper(void *data)
{
struct acpi_sleep_info *sinfo = (struct acpi_sleep_info *)data;
return enter_state(sinfo->sleep_state);
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 46f55fe7f191..9631042190ab 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -558,7 +558,7 @@ struct ucode_buf {
char buffer[];
};
-static long microcode_update_helper(void *data)
+static long cf_check microcode_update_helper(void *data)
{
int ret;
struct ucode_buf *buffer = data;
diff --git a/xen/arch/x86/include/asm/pv/shim.h b/xen/arch/x86/include/asm/pv/shim.h
index 6415f8068e5c..be4ae4e60bb8 100644
--- a/xen/arch/x86/include/asm/pv/shim.h
+++ b/xen/arch/x86/include/asm/pv/shim.h
@@ -38,8 +38,8 @@ void pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
start_info_t *si);
int pv_shim_shutdown(uint8_t reason);
void pv_shim_inject_evtchn(unsigned int port);
-long pv_shim_cpu_up(void *data);
-long pv_shim_cpu_down(void *data);
+long cf_check pv_shim_cpu_up(void *data);
+long cf_check pv_shim_cpu_down(void *data);
void pv_shim_online_memory(unsigned int nr, unsigned int order);
void pv_shim_offline_memory(unsigned int nr, unsigned int order);
domid_t get_initial_domain_id(void);
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index f7485f602efa..1747772d232e 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -57,10 +57,10 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint32_t pxm);
void __stop_this_cpu(void);
-long cpu_up_helper(void *data);
-long cpu_down_helper(void *data);
+long cf_check cpu_up_helper(void *data);
+long cf_check cpu_down_helper(void *data);
-long core_parking_helper(void *data);
+long cf_check core_parking_helper(void *data);
bool core_parking_remove(unsigned int cpu);
uint32_t get_cur_idle_nums(void);
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index 84566bbfaa3d..f5d7adc1e802 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -45,7 +45,7 @@ struct resource_access {
xenpf_resource_entry_t *entries;
};
-long cpu_frequency_change_helper(void *);
+long cf_check cpu_frequency_change_helper(void *);
void check_resource_access(struct resource_access *);
void resource_access(void *);
@@ -59,7 +59,7 @@ DEFINE_SPINLOCK(xenpf_lock);
# undef guest_from_compat_handle
# define guest_from_compat_handle(x,y) ((x)=(y))
-long cpu_frequency_change_helper(void *data)
+long cf_check cpu_frequency_change_helper(void *data)
{
return cpu_frequency_change((uint64_t)data);
}
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index ae4d8913faa1..2ee290a3920d 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -845,7 +845,7 @@ int cf_check compat_grant_table_op(
#endif
#endif
-long pv_shim_cpu_up(void *data)
+long cf_check pv_shim_cpu_up(void *data)
{
struct vcpu *v = data;
struct domain *d = v->domain;
@@ -883,7 +883,7 @@ long pv_shim_cpu_up(void *data)
return 0;
}
-long pv_shim_cpu_down(void *data)
+long cf_check pv_shim_cpu_down(void *data)
{
struct vcpu *v = data;
long rc;
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index eef0f9c6cbf4..f4952a6bf9a5 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -399,7 +399,7 @@ void call_function_interrupt(struct cpu_user_regs *regs)
smp_call_function_interrupt();
}
-long cpu_up_helper(void *data)
+long cf_check cpu_up_helper(void *data)
{
unsigned int cpu = (unsigned long)data;
int ret = cpu_up(cpu);
@@ -422,7 +422,7 @@ long cpu_up_helper(void *data)
return ret;
}
-long cpu_down_helper(void *data)
+long cf_check cpu_down_helper(void *data)
{
int cpu = (unsigned long)data;
int ret = cpu_down(cpu);
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a13f373..1772f51f8f6e 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -79,7 +79,7 @@ static void l3_cache_get(void *arg)
l3_info->size = info.size / 1024; /* in KB unit */
}
-static long smt_up_down_helper(void *data)
+static long cf_check smt_up_down_helper(void *data)
{
bool up = (bool)data;
unsigned int cpu, sibling_mask = boot_cpu_data.x86_num_siblings - 1;
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index 44a907abfd7f..4afad04f2f68 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -169,7 +169,7 @@ static unsigned int core_parking_power(unsigned int event)
return cpu;
}
-long core_parking_helper(void *data)
+long cf_check core_parking_helper(void *data)
{
uint32_t idle_nums = (unsigned long)data;
unsigned int cpu;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 3b223cd03d75..b222a5fd782e 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -395,7 +395,7 @@ void kexec_crash(enum crash_reason reason)
BUG();
}
-static long kexec_reboot(void *_image)
+static long cf_check kexec_reboot(void *_image)
{
struct kexec_image *image = _image;
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index e5cfb03b857e..b9d4babd0d8a 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -544,7 +544,7 @@ static int cpupool_unassign_cpu_start(struct cpupool *c, unsigned int cpu)
return ret;
}
-static long cpupool_unassign_cpu_helper(void *info)
+static long cf_check cpupool_unassign_cpu_helper(void *info)
{
struct cpupool *c = info;
long ret;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 14/70] xen: CFI hardening for init_timer()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (12 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 13/70] xen: CFI hardening for continue_hypercall_on_cpu() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 15/70] xen: CFI hardening for call_rcu() Andrew Cooper
` (57 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 2 +-
xen/arch/x86/cpu/mcheck/non-fatal.c | 2 +-
xen/arch/x86/hvm/pmtimer.c | 2 +-
xen/arch/x86/hvm/rtc.c | 6 +++---
xen/arch/x86/hvm/viridian/time.c | 2 +-
xen/arch/x86/hvm/vpt.c | 2 +-
xen/arch/x86/irq.c | 4 ++--
xen/arch/x86/nmi.c | 2 +-
xen/arch/x86/time.c | 4 ++--
xen/common/rcupdate.c | 2 +-
xen/common/sched/core.c | 18 +++++++++---------
xen/common/sched/credit.c | 10 ++++------
xen/common/sched/credit2.c | 2 +-
xen/common/sched/rt.c | 5 +++--
xen/drivers/char/ehci-dbgp.c | 2 +-
xen/drivers/char/ns16550.c | 6 +++---
xen/drivers/cpufreq/cpufreq_ondemand.c | 2 +-
17 files changed, 36 insertions(+), 37 deletions(-)
diff --git a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
index 6e8901530a69..da0bf85f0223 100644
--- a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
+++ b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
@@ -127,7 +127,7 @@ static void mce_amd_checkregs(void *info)
* multiple correctable errors between two polls. In that case,
* increase polling frequency higher than normal.
*/
-static void mce_amd_work_fn(void *data)
+static void cf_check mce_amd_work_fn(void *data)
{
on_each_cpu(mce_amd_checkregs, data, 1);
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c
index 2679c220a8a2..f7e411c0870e 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -67,7 +67,7 @@ static void mce_checkregs (void *info)
}
}
-static void mce_work_fn(void *data)
+static void cf_check mce_work_fn(void *data)
{
on_each_cpu(mce_checkregs, NULL, 1);
diff --git a/xen/arch/x86/hvm/pmtimer.c b/xen/arch/x86/hvm/pmtimer.c
index 97b9e41712fa..808819d1de91 100644
--- a/xen/arch/x86/hvm/pmtimer.c
+++ b/xen/arch/x86/hvm/pmtimer.c
@@ -124,7 +124,7 @@ static void pmt_update_time(PMTState *s)
/* This function should be called soon after each time the MSB of the
* pmtimer register rolls over, to make sure we update the status
* registers and SCI at least once per rollover */
-static void pmt_timer_callback(void *opaque)
+static void cf_check pmt_timer_callback(void *opaque)
{
PMTState *s = opaque;
uint32_t pmt_cycles_until_flip;
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index 3150f5f1479b..09d3501276bc 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -217,7 +217,7 @@ static void check_update_timer(RTCState *s)
s->use_timer = 0;
}
-static void rtc_update_timer(void *opaque)
+static void cf_check rtc_update_timer(void *opaque)
{
RTCState *s = opaque;
@@ -230,7 +230,7 @@ static void rtc_update_timer(void *opaque)
spin_unlock(&s->lock);
}
-static void rtc_update_timer2(void *opaque)
+static void cf_check rtc_update_timer2(void *opaque)
{
RTCState *s = opaque;
@@ -421,7 +421,7 @@ static void alarm_timer_update(RTCState *s)
}
}
-static void rtc_alarm_cb(void *opaque)
+static void cf_check rtc_alarm_cb(void *opaque)
{
RTCState *s = opaque;
diff --git a/xen/arch/x86/hvm/viridian/time.c b/xen/arch/x86/hvm/viridian/time.c
index 24ff117edb20..b56fd6766292 100644
--- a/xen/arch/x86/hvm/viridian/time.c
+++ b/xen/arch/x86/hvm/viridian/time.c
@@ -126,7 +126,7 @@ static void stop_stimer(struct viridian_stimer *vs)
vs->started = false;
}
-static void stimer_expire(void *data)
+static void cf_check stimer_expire(void *data)
{
struct viridian_stimer *vs = data;
struct vcpu *v = vs->v;
diff --git a/xen/arch/x86/hvm/vpt.c b/xen/arch/x86/hvm/vpt.c
index 6fdc3e19fe8c..cb1d81bf9e82 100644
--- a/xen/arch/x86/hvm/vpt.c
+++ b/xen/arch/x86/hvm/vpt.c
@@ -271,7 +271,7 @@ void pt_restore_timer(struct vcpu *v)
pt_vcpu_unlock(v);
}
-static void pt_timer_fn(void *data)
+static void cf_check pt_timer_fn(void *data)
{
struct periodic_time *pt = data;
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index bcf46cd54d16..f9c808455535 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -935,7 +935,7 @@ void alloc_direct_apic_vector(
spin_unlock(&lock);
}
-static void irq_ratelimit_timer_fn(void *data)
+static void cf_check irq_ratelimit_timer_fn(void *data)
{
struct irq_desc *desc, *tmp;
unsigned long flags;
@@ -1129,7 +1129,7 @@ static inline void clear_pirq_eoi(struct domain *d, unsigned int irq)
static void set_eoi_ready(void *data);
-static void irq_guest_eoi_timer_fn(void *data)
+static void cf_check irq_guest_eoi_timer_fn(void *data)
{
struct irq_desc *desc = data;
unsigned int i, irq = desc - irq_desc;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index 5c101a9f97b3..c8ae4a5d7460 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -211,7 +211,7 @@ void __init check_nmi_watchdog(void)
return;
}
-static void nmi_timer_fn(void *unused)
+static void cf_check nmi_timer_fn(void *unused)
{
this_cpu(nmi_timer_ticks)++;
set_timer(&this_cpu(nmi_timer), NOW() + MILLISECS(1000));
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 14f7d3fd635e..fac97023bf10 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -815,7 +815,7 @@ static s_time_t __read_platform_stime(u64 platform_time)
return (stime_platform_stamp + scale_delta(diff, &plt_scale));
}
-static void plt_overflow(void *unused)
+static void cf_check plt_overflow(void *unused)
{
int i;
u64 count;
@@ -1855,7 +1855,7 @@ static void time_calibration_nop_rendezvous(void *rv)
static void (*time_calibration_rendezvous_fn)(void *) =
time_calibration_std_rendezvous;
-static void time_calibration(void *unused)
+static void cf_check time_calibration(void *unused)
{
struct calibration_rendezvous r = {
.semaphore = ATOMIC_INIT(0)
diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
index 2ec5606de5dd..f9dd2584a8b7 100644
--- a/xen/common/rcupdate.c
+++ b/xen/common/rcupdate.c
@@ -575,7 +575,7 @@ static void rcu_idle_timer_stop(void)
stop_timer(&rdp->idle_timer);
}
-static void rcu_idle_timer_handler(void* data)
+static void cf_check rcu_idle_timer_handler(void* data)
{
perfc_incr(rcu_idle_timer);
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 4a79971a1d45..cf1ba01b4d87 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -71,10 +71,10 @@ cpumask_t sched_res_mask;
static DEFINE_SPINLOCK(sched_free_cpu_lock);
/* Various timer handlers. */
-static void s_timer_fn(void *unused);
-static void vcpu_periodic_timer_fn(void *data);
-static void vcpu_singleshot_timer_fn(void *data);
-static void poll_timer_fn(void *data);
+static void cf_check s_timer_fn(void *unused);
+static void cf_check vcpu_periodic_timer_fn(void *data);
+static void cf_check vcpu_singleshot_timer_fn(void *data);
+static void cf_check poll_timer_fn(void *data);
/* This is global for now so that private implementations can reach it */
DEFINE_PER_CPU_READ_MOSTLY(struct sched_resource *, sched_res);
@@ -1535,7 +1535,7 @@ long vcpu_yield(void)
return 0;
}
-static void domain_watchdog_timeout(void *data)
+static void cf_check domain_watchdog_timeout(void *data)
{
struct domain *d = data;
@@ -2697,28 +2697,28 @@ static void schedule(void)
}
/* The scheduler timer: force a run through the scheduler */
-static void s_timer_fn(void *unused)
+static void cf_check s_timer_fn(void *unused)
{
raise_softirq(SCHEDULE_SOFTIRQ);
SCHED_STAT_CRANK(sched_irq);
}
/* Per-VCPU periodic timer function: sends a virtual timer interrupt. */
-static void vcpu_periodic_timer_fn(void *data)
+static void cf_check vcpu_periodic_timer_fn(void *data)
{
struct vcpu *v = data;
vcpu_periodic_timer_work(v);
}
/* Per-VCPU single-shot timer function: sends a virtual timer interrupt. */
-static void vcpu_singleshot_timer_fn(void *data)
+static void cf_check vcpu_singleshot_timer_fn(void *data)
{
struct vcpu *v = data;
send_timer_event(v);
}
/* SCHEDOP_poll timeout callback. */
-static void poll_timer_fn(void *data)
+static void cf_check poll_timer_fn(void *data)
{
struct vcpu *v = data;
diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c
index d0aa017c643e..5635271f6fea 100644
--- a/xen/common/sched/credit.c
+++ b/xen/common/sched/credit.c
@@ -230,8 +230,8 @@ struct csched_private {
struct timer master_ticker;
};
-static void csched_tick(void *_cpu);
-static void csched_acct(void *dummy);
+static void cf_check csched_tick(void *_cpu);
+static void cf_check csched_acct(void *dummy);
static inline int
__unit_on_runq(const struct csched_unit *svc)
@@ -1356,8 +1356,7 @@ csched_runq_sort(struct csched_private *prv, unsigned int cpu)
pcpu_schedule_unlock_irqrestore(lock, flags, cpu);
}
-static void
-csched_acct(void* dummy)
+static void cf_check csched_acct(void* dummy)
{
struct csched_private *prv = dummy;
unsigned long flags;
@@ -1563,8 +1562,7 @@ csched_acct(void* dummy)
set_timer( &prv->master_ticker, NOW() + prv->tslice);
}
-static void
-csched_tick(void *_cpu)
+static void cf_check csched_tick(void *_cpu)
{
unsigned int cpu = (unsigned long)_cpu;
const struct sched_resource *sr = get_sched_res(cpu);
diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index a5f073cda51e..d96e2749ddfb 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -2072,7 +2072,7 @@ static inline void do_replenish(struct csched2_dom *sdom)
sdom->budget += sdom->tot_budget;
}
-static void replenish_domain_budget(void* data)
+static void cf_check replenish_domain_budget(void *data)
{
struct csched2_dom *sdom = data;
unsigned long flags;
diff --git a/xen/common/sched/rt.c b/xen/common/sched/rt.c
index c24cd2ac3200..5ea6f01f263c 100644
--- a/xen/common/sched/rt.c
+++ b/xen/common/sched/rt.c
@@ -173,7 +173,7 @@
#define TRC_RTDS_SCHED_TASKLET TRC_SCHED_CLASS_EVT(RTDS, 5)
#define TRC_RTDS_SCHEDULE TRC_SCHED_CLASS_EVT(RTDS, 6)
-static void repl_timer_handler(void *data);
+static void cf_check repl_timer_handler(void *data);
/*
* System-wide private data, include global RunQueue/DepletedQ
@@ -1452,7 +1452,8 @@ rt_dom_cntl(
* The replenishment timer handler picks units
* from the replq and does the actual replenishment.
*/
-static void repl_timer_handler(void *data){
+static void cf_check repl_timer_handler(void *data)
+{
s_time_t now;
const struct scheduler *ops = data;
struct rt_private *prv = rt_priv(ops);
diff --git a/xen/drivers/char/ehci-dbgp.c b/xen/drivers/char/ehci-dbgp.c
index c893d246defa..a6b57fdf2d19 100644
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1289,7 +1289,7 @@ static void _ehci_dbgp_poll(struct cpu_user_regs *regs)
set_timer(&dbgp->timer, NOW() + timeout);
}
-static void ehci_dbgp_poll(void *data)
+static void cf_check ehci_dbgp_poll(void *data)
{
poll_port = data;
#ifdef run_in_exception_handler
diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index 30596d60d4ed..990cad39fe85 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -111,7 +111,7 @@ struct ns16550_config_param {
static void enable_exar_enhanced_bits(const struct ns16550 *uart);
#endif
-static void ns16550_delayed_resume(void *data);
+static void cf_check ns16550_delayed_resume(void *data);
static u8 ns_read_reg(const struct ns16550 *uart, unsigned int reg)
{
@@ -229,7 +229,7 @@ static void __ns16550_poll(struct cpu_user_regs *regs)
set_timer(&uart->timer, NOW() + MILLISECS(uart->timeout_ms));
}
-static void ns16550_poll(void *data)
+static void cf_check ns16550_poll(void *data)
{
this_cpu(poll_port) = data;
#ifdef run_in_exception_handler
@@ -532,7 +532,7 @@ static void _ns16550_resume(struct serial_port *port)
}
static int delayed_resume_tries;
-static void ns16550_delayed_resume(void *data)
+static void cf_check ns16550_delayed_resume(void *data)
{
struct serial_port *port = data;
struct ns16550 *uart = port->uart;
diff --git a/xen/drivers/cpufreq/cpufreq_ondemand.c b/xen/drivers/cpufreq/cpufreq_ondemand.c
index cabd9ffa8886..ba03eaa2336d 100644
--- a/xen/drivers/cpufreq/cpufreq_ondemand.c
+++ b/xen/drivers/cpufreq/cpufreq_ondemand.c
@@ -172,7 +172,7 @@ static void dbs_check_cpu(struct cpu_dbs_info_s *this_dbs_info)
}
}
-static void do_dbs_timer(void *dbs)
+static void cf_check do_dbs_timer(void *dbs)
{
struct cpu_dbs_info_s *dbs_info = (struct cpu_dbs_info_s *)dbs;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 15/70] xen: CFI hardening for call_rcu()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (13 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 14/70] xen: CFI hardening for init_timer() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 16/70] xen: CFI hardening for IPIs Andrew Cooper
` (56 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/hvm/mtrr.c | 2 +-
xen/arch/x86/hvm/vmsi.c | 2 +-
xen/arch/x86/mm/mem_sharing.c | 2 +-
xen/arch/x86/percpu.c | 2 +-
xen/common/domain.c | 4 ++--
xen/common/radix-tree.c | 2 +-
xen/common/rcupdate.c | 2 +-
xen/common/sched/core.c | 2 +-
xen/xsm/flask/avc.c | 2 +-
9 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
index b3ef1bf54133..42f3d8319296 100644
--- a/xen/arch/x86/hvm/mtrr.c
+++ b/xen/arch/x86/hvm/mtrr.c
@@ -586,7 +586,7 @@ int hvm_get_mem_pinned_cacheattr(struct domain *d, gfn_t gfn,
return rc;
}
-static void free_pinned_cacheattr_entry(struct rcu_head *rcu)
+static void cf_check free_pinned_cacheattr_entry(struct rcu_head *rcu)
{
xfree(container_of(rcu, struct hvm_mem_pinned_cacheattr_range, rcu));
}
diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 13e2a190b439..2889575a2035 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -441,7 +441,7 @@ static void add_msixtbl_entry(struct domain *d,
list_add_rcu(&entry->list, &d->arch.hvm.msixtbl_list);
}
-static void free_msixtbl_entry(struct rcu_head *rcu)
+static void cf_check free_msixtbl_entry(struct rcu_head *rcu)
{
struct msixtbl_entry *entry;
diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 74d2869c0e6f..15e6a7ed814b 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -75,7 +75,7 @@ static DEFINE_SPINLOCK(shr_audit_lock);
static DEFINE_RCU_READ_LOCK(shr_audit_read_lock);
/* RCU delayed free of audit list entry */
-static void _free_pg_shared_info(struct rcu_head *head)
+static void cf_check _free_pg_shared_info(struct rcu_head *head)
{
xfree(container_of(head, struct page_sharing_info, rcu_head));
}
diff --git a/xen/arch/x86/percpu.c b/xen/arch/x86/percpu.c
index eb3ba7bc8874..46460689b73d 100644
--- a/xen/arch/x86/percpu.c
+++ b/xen/arch/x86/percpu.c
@@ -45,7 +45,7 @@ struct free_info {
};
static DEFINE_PER_CPU(struct free_info, free_info);
-static void _free_percpu_area(struct rcu_head *head)
+static void cf_check _free_percpu_area(struct rcu_head *head)
{
struct free_info *info = container_of(head, struct free_info, rcu);
unsigned int cpu = info->cpu;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 5df0d167537b..32ec156e6f6a 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -365,7 +365,7 @@ static int __init cf_check parse_extra_guest_irqs(const char *s)
}
custom_param("extra_guest_irqs", parse_extra_guest_irqs);
-static void _free_pirq_struct(struct rcu_head *head)
+static void cf_check _free_pirq_struct(struct rcu_head *head)
{
xfree(container_of(head, struct pirq, rcu_head));
}
@@ -1108,7 +1108,7 @@ void vcpu_end_shutdown_deferral(struct vcpu *v)
}
/* Complete domain destroy after RCU readers are not holding old references. */
-static void complete_domain_destroy(struct rcu_head *head)
+static void cf_check complete_domain_destroy(struct rcu_head *head)
{
struct domain *d = container_of(head, struct domain, rcu);
struct vcpu *v;
diff --git a/xen/common/radix-tree.c b/xen/common/radix-tree.c
index 628a7e06988f..33b47748ae49 100644
--- a/xen/common/radix-tree.c
+++ b/xen/common/radix-tree.c
@@ -58,7 +58,7 @@ static struct radix_tree_node *rcu_node_alloc(void *arg)
return rcu_node ? &rcu_node->node : NULL;
}
-static void _rcu_node_free(struct rcu_head *head)
+static void cf_check _rcu_node_free(struct rcu_head *head)
{
struct rcu_node *rcu_node =
container_of(head, struct rcu_node, rcu_head);
diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
index f9dd2584a8b7..423d6b1d6d02 100644
--- a/xen/common/rcupdate.c
+++ b/xen/common/rcupdate.c
@@ -167,7 +167,7 @@ static int rsinterval = 1000;
static atomic_t cpu_count = ATOMIC_INIT(0);
static atomic_t pending_count = ATOMIC_INIT(0);
-static void rcu_barrier_callback(struct rcu_head *head)
+static void cf_check rcu_barrier_callback(struct rcu_head *head)
{
/*
* We need a barrier making all previous writes visible to other cpus
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index cf1ba01b4d87..285de9ee2a19 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -2798,7 +2798,7 @@ static int cpu_schedule_up(unsigned int cpu)
return 0;
}
-static void sched_res_free(struct rcu_head *head)
+static void cf_check sched_res_free(struct rcu_head *head)
{
struct sched_resource *sr = container_of(head, struct sched_resource, rcu);
diff --git a/xen/xsm/flask/avc.c b/xen/xsm/flask/avc.c
index 87ea38b7a0d0..e20c16504213 100644
--- a/xen/xsm/flask/avc.c
+++ b/xen/xsm/flask/avc.c
@@ -276,7 +276,7 @@ int avc_get_hash_stats(struct xen_flask_hash_stats *arg)
return 0;
}
-static void avc_node_free(struct rcu_head *rhead)
+static void cf_check avc_node_free(struct rcu_head *rhead)
{
struct avc_node *node = container_of(rhead, struct avc_node, rhead);
xfree(node);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 16/70] xen: CFI hardening for IPIs
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (14 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 15/70] xen: CFI hardening for call_rcu() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 17/70] xen: CFI hardening for open_softirq() Andrew Cooper
` (55 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 2 +-
xen/arch/x86/acpi/cpufreq/cpufreq.c | 8 ++++----
xen/arch/x86/acpi/cpufreq/powernow.c | 6 +++---
xen/arch/x86/acpi/lib.c | 2 +-
xen/arch/x86/cpu/amd.c | 2 +-
xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 2 +-
xen/arch/x86/cpu/mcheck/mce.c | 6 +++---
xen/arch/x86/cpu/mcheck/mce_intel.c | 2 +-
xen/arch/x86/cpu/mcheck/non-fatal.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/cpu/mtrr/generic.c | 2 +-
xen/arch/x86/cpu/mtrr/main.c | 2 +-
xen/arch/x86/cpu/mwait-idle.c | 6 +++---
xen/arch/x86/cpu/vpmu.c | 4 ++--
xen/arch/x86/guest/xen/xen.c | 2 +-
xen/arch/x86/hvm/nestedhvm.c | 3 +--
xen/arch/x86/hvm/vmx/vmcs.c | 2 +-
xen/arch/x86/include/asm/mtrr.h | 2 +-
xen/arch/x86/irq.c | 4 ++--
xen/arch/x86/nmi.c | 2 +-
xen/arch/x86/oprofile/nmi_int.c | 10 +++++-----
xen/arch/x86/oprofile/op_model_athlon.c | 2 +-
xen/arch/x86/platform_hypercall.c | 4 ++--
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/shutdown.c | 4 ++--
xen/arch/x86/smp.c | 2 +-
xen/arch/x86/sysctl.c | 2 +-
xen/arch/x86/time.c | 8 ++++----
xen/common/cpu.c | 4 ++--
xen/common/gdbstub.c | 2 +-
xen/common/keyhandler.c | 2 +-
xen/common/page_alloc.c | 2 +-
32 files changed, 53 insertions(+), 54 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index fb47eb9ad68e..22c8bb0c2d94 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -145,7 +145,7 @@ struct hw_residencies
uint64_t cc7;
};
-static void do_get_hw_residencies(void *arg)
+static void cf_check do_get_hw_residencies(void *arg)
{
struct cpuinfo_x86 *c = ¤t_cpu_data;
struct hw_residencies *hw_res = arg;
diff --git a/xen/arch/x86/acpi/cpufreq/cpufreq.c b/xen/arch/x86/acpi/cpufreq/cpufreq.c
index 9510f05340aa..8133c2dd958c 100644
--- a/xen/arch/x86/acpi/cpufreq/cpufreq.c
+++ b/xen/arch/x86/acpi/cpufreq/cpufreq.c
@@ -129,7 +129,7 @@ struct drv_cmd {
u32 val;
};
-static void do_drv_read(void *drvcmd)
+static void cf_check do_drv_read(void *drvcmd)
{
struct drv_cmd *cmd;
@@ -148,7 +148,7 @@ static void do_drv_read(void *drvcmd)
}
}
-static void do_drv_write(void *drvcmd)
+static void cf_check do_drv_write(void *drvcmd)
{
struct drv_cmd *cmd;
uint64_t msr_content;
@@ -244,7 +244,7 @@ struct perf_pair {
static DEFINE_PER_CPU(struct perf_pair, gov_perf_pair);
static DEFINE_PER_CPU(struct perf_pair, usr_perf_pair);
-static void read_measured_perf_ctrs(void *_readin)
+static void cf_check read_measured_perf_ctrs(void *_readin)
{
struct perf_pair *readin = _readin;
@@ -340,7 +340,7 @@ static unsigned int get_cur_freq_on_cpu(unsigned int cpu)
return extract_freq(get_cur_val(cpumask_of(cpu)), data);
}
-static void feature_detect(void *info)
+static void cf_check feature_detect(void *info)
{
struct cpufreq_policy *policy = info;
unsigned int eax;
diff --git a/xen/arch/x86/acpi/cpufreq/powernow.c b/xen/arch/x86/acpi/cpufreq/powernow.c
index da8fc40b9a6f..ca71ecf72d67 100644
--- a/xen/arch/x86/acpi/cpufreq/powernow.c
+++ b/xen/arch/x86/acpi/cpufreq/powernow.c
@@ -44,12 +44,12 @@
#define ARCH_CPU_FLAG_RESUME 1
-static void transition_pstate(void *pstate)
+static void cf_check transition_pstate(void *pstate)
{
wrmsrl(MSR_PSTATE_CTRL, *(unsigned int *)pstate);
}
-static void update_cpb(void *data)
+static void cf_check update_cpb(void *data)
{
struct cpufreq_policy *policy = data;
@@ -165,7 +165,7 @@ struct amd_cpu_data {
u32 max_hw_pstate;
};
-static void get_cpu_data(void *arg)
+static void cf_check get_cpu_data(void *arg)
{
struct amd_cpu_data *data = arg;
struct processor_performance *perf = data->perf;
diff --git a/xen/arch/x86/acpi/lib.c b/xen/arch/x86/acpi/lib.c
index b66e7338e74d..43831b92d132 100644
--- a/xen/arch/x86/acpi/lib.c
+++ b/xen/arch/x86/acpi/lib.c
@@ -99,7 +99,7 @@ unsigned int acpi_get_processor_id(unsigned int cpu)
return INVALID_ACPIID;
}
-static void get_mwait_ecx(void *info)
+static void cf_check get_mwait_ecx(void *info)
{
*(u32 *)info = cpuid_ecx(CPUID_MWAIT_LEAF);
}
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index a8e37dbb1f5c..2d18223f20ef 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -430,7 +430,7 @@ static void disable_c1_ramping(void)
}
}
-static void disable_c1e(void *unused)
+static void cf_check disable_c1e(void *unused)
{
uint64_t msr_content;
diff --git a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
index da0bf85f0223..efb45c931e1f 100644
--- a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
+++ b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c
@@ -79,7 +79,7 @@ static int variable_period = 1;
* Collects information of correctable errors and notifies
* Dom0 via an event.
*/
-static void mce_amd_checkregs(void *info)
+static void cf_check mce_amd_checkregs(void *info)
{
mctelem_cookie_t mctc;
struct mca_summary bs;
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index a449fa0424ce..43f6c8471a90 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -961,7 +961,7 @@ void x86_mcinfo_dump(struct mc_info *mi)
} while ( 1 );
}
-static void do_mc_get_cpu_info(void *v)
+static void cf_check do_mc_get_cpu_info(void *v)
{
int cpu = smp_processor_id();
int cindex, cpn;
@@ -1242,7 +1242,7 @@ static void x86_mc_hwcr_wren_restore(uint64_t hwcr)
wrmsrl(MSR_K8_HWCR, hwcr);
}
-static void x86_mc_msrinject(void *data)
+static void cf_check x86_mc_msrinject(void *data)
{
struct xen_mc_msrinject *mci = data;
struct mcinfo_msr *msr;
@@ -1274,7 +1274,7 @@ static void x86_mc_msrinject(void *data)
}
/*ARGSUSED*/
-static void x86_mc_mceinject(void *data)
+static void cf_check x86_mc_mceinject(void *data)
{
printk("Simulating #MC on cpu %d\n", smp_processor_id());
__asm__ __volatile__("int $0x12");
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c
index b6da8262e662..a691e10bdcd6 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -599,7 +599,7 @@ static void mce_set_owner(void)
cmci_discover();
}
-static void __cpu_mcheck_distribute_cmci(void *unused)
+static void cf_check __cpu_mcheck_distribute_cmci(void *unused)
{
cmci_discover();
}
diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c
index f7e411c0870e..1c0c32ba0821 100644
--- a/xen/arch/x86/cpu/mcheck/non-fatal.c
+++ b/xen/arch/x86/cpu/mcheck/non-fatal.c
@@ -32,7 +32,7 @@ static uint64_t period = MCE_PERIOD;
static int adjust = 0;
static int variable_period = 1;
-static void mce_checkregs (void *info)
+static void cf_check mce_checkregs(void *info)
{
mctelem_cookie_t mctc;
struct mca_summary bs;
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 9631042190ab..841364208053 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -533,7 +533,7 @@ static int control_thread_fn(const struct microcode_patch *patch)
return ret;
}
-static int do_microcode_update(void *patch)
+static int cf_check do_microcode_update(void *patch)
{
unsigned int cpu = smp_processor_id();
int ret;
diff --git a/xen/arch/x86/cpu/mtrr/generic.c b/xen/arch/x86/cpu/mtrr/generic.c
index 883e3398ff71..7cf4cd01f3f6 100644
--- a/xen/arch/x86/cpu/mtrr/generic.c
+++ b/xen/arch/x86/cpu/mtrr/generic.c
@@ -84,7 +84,7 @@ bool is_var_mtrr_overlapped(const struct mtrr_state *m)
return false;
}
-void mtrr_save_fixed_ranges(void *info)
+void cf_check mtrr_save_fixed_ranges(void *info)
{
get_fixed_ranges(mtrr_state.fixed_ranges);
}
diff --git a/xen/arch/x86/cpu/mtrr/main.c b/xen/arch/x86/cpu/mtrr/main.c
index 428133100d46..4e01c8d6f9df 100644
--- a/xen/arch/x86/cpu/mtrr/main.c
+++ b/xen/arch/x86/cpu/mtrr/main.c
@@ -131,7 +131,7 @@ struct set_mtrr_data {
*/
int hold_mtrr_updates_on_aps;
-static void ipi_handler(void *info)
+static void cf_check ipi_handler(void *info)
/* [SUMMARY] Synchronisation handler. Executed by "other" CPUs.
[RETURNS] Nothing.
*/
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index fe1b7af25f01..927ce1b67aa5 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -877,7 +877,7 @@ static void mwait_idle(void)
cpuidle_current_governor->reflect(power);
}
-static void auto_demotion_disable(void *dummy)
+static void cf_check auto_demotion_disable(void *dummy)
{
u64 msr_bits;
@@ -886,13 +886,13 @@ static void auto_demotion_disable(void *dummy)
wrmsrl(MSR_PKG_CST_CONFIG_CONTROL, msr_bits);
}
-static void byt_auto_demotion_disable(void *dummy)
+static void cf_check byt_auto_demotion_disable(void *dummy)
{
wrmsrl(MSR_CC6_DEMOTION_POLICY_CONFIG, 0);
wrmsrl(MSR_MC6_DEMOTION_POLICY_CONFIG, 0);
}
-static void c1e_promotion_disable(void *dummy)
+static void cf_check c1e_promotion_disable(void *dummy)
{
u64 msr_bits;
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index df3c9201b2c1..4fedc7c57012 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -335,7 +335,7 @@ void vpmu_do_interrupt(struct cpu_user_regs *regs)
#endif
}
-static void vpmu_save_force(void *arg)
+static void cf_check vpmu_save_force(void *arg)
{
struct vcpu *v = arg;
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -528,7 +528,7 @@ void vpmu_initialise(struct vcpu *v)
put_vpmu(v);
}
-static void vpmu_clear_last(void *arg)
+static void cf_check vpmu_clear_last(void *arg)
{
if ( this_cpu(last_vcpu) == arg )
this_cpu(last_vcpu) = NULL;
diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c
index 2ff63d370a8a..b2aa3a009b4a 100644
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -289,7 +289,7 @@ int xg_free_unused_page(mfn_t mfn)
return rangeset_remove_range(mem, mfn_x(mfn), mfn_x(mfn));
}
-static void ap_resume(void *unused)
+static void cf_check ap_resume(void *unused)
{
BUG_ON(map_vcpuinfo());
BUG_ON(init_evtchn());
diff --git a/xen/arch/x86/hvm/nestedhvm.c b/xen/arch/x86/hvm/nestedhvm.c
index 23516884482b..58370190ffc9 100644
--- a/xen/arch/x86/hvm/nestedhvm.c
+++ b/xen/arch/x86/hvm/nestedhvm.c
@@ -82,8 +82,7 @@ nestedhvm_vcpu_destroy(struct vcpu *v)
alternative_vcall(hvm_funcs.nhvm_vcpu_destroy, v);
}
-static void
-nestedhvm_flushtlb_ipi(void *info)
+static void cf_check nestedhvm_flushtlb_ipi(void *info)
{
struct vcpu *v = current;
struct domain *d = info;
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index f72a7db0453d..2b6bafe9d542 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -589,7 +589,7 @@ static void vmx_free_vmcs(paddr_t pa)
free_domheap_page(maddr_to_page(pa));
}
-static void __vmx_clear_vmcs(void *info)
+static void cf_check __vmx_clear_vmcs(void *info)
{
struct vcpu *v = info;
struct vmx_vcpu *vmx = &v->arch.hvm.vmx;
diff --git a/xen/arch/x86/include/asm/mtrr.h b/xen/arch/x86/include/asm/mtrr.h
index e0fd1005ce72..7733800b798f 100644
--- a/xen/arch/x86/include/asm/mtrr.h
+++ b/xen/arch/x86/include/asm/mtrr.h
@@ -64,7 +64,7 @@ struct mtrr_state {
};
extern struct mtrr_state mtrr_state;
-extern void mtrr_save_fixed_ranges(void *);
+extern void cf_check mtrr_save_fixed_ranges(void *);
extern void mtrr_save_state(void);
extern int mtrr_add(unsigned long base, unsigned long size,
unsigned int type, char increment);
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index f9c808455535..d9bd355113d7 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1127,7 +1127,7 @@ static inline void clear_pirq_eoi(struct domain *d, unsigned int irq)
}
}
-static void set_eoi_ready(void *data);
+static void cf_check set_eoi_ready(void *data);
static void cf_check irq_guest_eoi_timer_fn(void *data)
{
@@ -1398,7 +1398,7 @@ static void __set_eoi_ready(const struct irq_desc *desc)
}
/* Mark specified IRQ as ready-for-EOI (if it really is) and attempt to EOI. */
-static void set_eoi_ready(void *data)
+static void cf_check set_eoi_ready(void *data)
{
struct irq_desc *desc = data;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index c8ae4a5d7460..e20d1393c881 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -149,7 +149,7 @@ int nmi_active;
(P4_CCCR_OVF_PMI0|P4_CCCR_THRESHOLD(15)|P4_CCCR_COMPLEMENT| \
P4_CCCR_COMPARE|P4_CCCR_REQUIRED|P4_CCCR_ESCR_SELECT(4)|P4_CCCR_ENABLE)
-static void __init wait_for_nmis(void *p)
+static void __init cf_check wait_for_nmis(void *p)
{
unsigned int start_count = this_cpu(nmi_count);
unsigned long ticks = 10 * 1000 * cpu_khz / nmi_hz;
diff --git a/xen/arch/x86/oprofile/nmi_int.c b/xen/arch/x86/oprofile/nmi_int.c
index ba9c4b9804ca..6ebe20bd1d3e 100644
--- a/xen/arch/x86/oprofile/nmi_int.c
+++ b/xen/arch/x86/oprofile/nmi_int.c
@@ -131,7 +131,7 @@ static void nmi_cpu_save_registers(struct op_msrs *msrs)
}
-static void nmi_save_registers(void * dummy)
+static void cf_check nmi_save_registers(void *dummy)
{
int cpu = smp_processor_id();
struct op_msrs * msrs = &cpu_msrs[cpu];
@@ -179,7 +179,7 @@ static int allocate_msrs(void)
}
-static void nmi_cpu_setup(void * dummy)
+static void cf_check nmi_cpu_setup(void *dummy)
{
int cpu = smp_processor_id();
struct op_msrs * msrs = &cpu_msrs[cpu];
@@ -245,7 +245,7 @@ static void nmi_restore_registers(struct op_msrs * msrs)
}
-static void nmi_cpu_shutdown(void * dummy)
+static void cf_check nmi_cpu_shutdown(void *dummy)
{
int cpu = smp_processor_id();
struct op_msrs * msrs = &cpu_msrs[cpu];
@@ -261,7 +261,7 @@ void nmi_release_counters(void)
}
-static void nmi_cpu_start(void * dummy)
+static void cf_check nmi_cpu_start(void *dummy)
{
int cpu = smp_processor_id();
struct op_msrs const * msrs = &cpu_msrs[cpu];
@@ -278,7 +278,7 @@ int nmi_start(void)
}
-static void nmi_cpu_stop(void * dummy)
+static void cf_check nmi_cpu_stop(void *dummy)
{
unsigned int v;
int cpu = smp_processor_id();
diff --git a/xen/arch/x86/oprofile/op_model_athlon.c b/xen/arch/x86/oprofile/op_model_athlon.c
index ee6eb0ecae79..2177f02946e2 100644
--- a/xen/arch/x86/oprofile/op_model_athlon.c
+++ b/xen/arch/x86/oprofile/op_model_athlon.c
@@ -436,7 +436,7 @@ static void athlon_stop(struct op_msrs const * const msrs)
#define APIC_EILVT_MSG_NMI 0x4
#define APIC_EILVT_LVTOFF_IBS 1
#define APIC_EILVTn(n) (0x500 + 0x10 * n)
-static inline void __init init_ibs_nmi_per_cpu(void *arg)
+static inline void __init cf_check init_ibs_nmi_per_cpu(void *arg)
{
unsigned long reg;
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index f5d7adc1e802..b91ccff58900 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -47,7 +47,7 @@ struct resource_access {
long cf_check cpu_frequency_change_helper(void *);
void check_resource_access(struct resource_access *);
-void resource_access(void *);
+void cf_check resource_access(void *);
#ifndef COMPAT
typedef long ret_t;
@@ -149,7 +149,7 @@ void check_resource_access(struct resource_access *ra)
ra->nr_done = i;
}
-void resource_access(void *info)
+void cf_check resource_access(void *info)
{
struct resource_access *ra = info;
unsigned int i;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 5b9991bd5b12..6c9cabf3843d 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -1247,7 +1247,7 @@ struct cos_write_info
const uint32_t *val;
};
-static void do_write_psr_msrs(void *data)
+static void cf_check do_write_psr_msrs(void *data)
{
const struct cos_write_info *info = data;
unsigned int i, index, cos = info->cos;
diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index ad3e3a76916f..30985d36a612 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -118,7 +118,7 @@ static inline void kb_wait(void)
break;
}
-static void noreturn __machine_halt(void *unused)
+static void noreturn cf_check __machine_halt(void *unused)
{
local_irq_disable();
@@ -548,7 +548,7 @@ static int __init cf_check reboot_init(void)
}
__initcall(reboot_init);
-static void noreturn __machine_restart(void *pdelay)
+static void cf_check noreturn __machine_restart(void *pdelay)
{
machine_restart(*(unsigned int *)pdelay);
}
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index f4952a6bf9a5..f6fd7f95df58 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -339,7 +339,7 @@ void __stop_this_cpu(void)
cpumask_clear_cpu(smp_processor_id(), &cpu_online_map);
}
-static void stop_this_cpu(void *dummy)
+static void cf_check stop_this_cpu(void *dummy)
{
__stop_this_cpu();
for ( ; ; )
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 1772f51f8f6e..f82abc248885 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -69,7 +69,7 @@ struct l3_cache_info {
unsigned long size;
};
-static void l3_cache_get(void *arg)
+static void cf_check l3_cache_get(void *arg)
{
struct cpuid4_info info;
struct l3_cache_info *l3_info = arg;
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index fac97023bf10..b444d6344e79 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1661,7 +1661,7 @@ static void check_tsc_warp(unsigned long tsc_khz, unsigned long *max_warp)
static unsigned long tsc_max_warp, tsc_check_count;
static cpumask_t tsc_check_cpumask;
-static void tsc_check_slave(void *unused)
+static void cf_check tsc_check_slave(void *unused)
{
unsigned int cpu = smp_processor_id();
local_irq_disable();
@@ -1809,7 +1809,7 @@ static void time_calibration_tsc_rendezvous(void *_r)
}
/* Ordinary rendezvous function which does not modify TSC values. */
-static void time_calibration_std_rendezvous(void *_r)
+static void cf_check time_calibration_std_rendezvous(void *_r)
{
struct calibration_rendezvous *r = _r;
unsigned int total_cpus = cpumask_weight(&r->cpu_calibration_map);
@@ -1840,7 +1840,7 @@ static void time_calibration_std_rendezvous(void *_r)
* Rendezvous function used when clocksource is TSC and
* no CPU hotplug will be performed.
*/
-static void time_calibration_nop_rendezvous(void *rv)
+static void cf_check time_calibration_nop_rendezvous(void *rv)
{
const struct calibration_rendezvous *r = rv;
struct cpu_time_stamp *c = &this_cpu(cpu_calibration);
@@ -2032,7 +2032,7 @@ static void __init tsc_check_writability(void)
disable_tsc_sync = true;
}
-static void __init reset_percpu_time(void *unused)
+static void __init cf_check reset_percpu_time(void *unused)
{
struct cpu_time *t = &this_cpu(cpu_time);
diff --git a/xen/common/cpu.c b/xen/common/cpu.c
index 1f976db0a51f..b0b63cdb3638 100644
--- a/xen/common/cpu.c
+++ b/xen/common/cpu.c
@@ -84,13 +84,13 @@ static int cpu_notifier_call_chain(unsigned int cpu, unsigned long action,
return ret;
}
-static void _take_cpu_down(void *unused)
+static void cf_check _take_cpu_down(void *unused)
{
cpu_notifier_call_chain(smp_processor_id(), CPU_DYING, NULL, true);
__cpu_disable();
}
-static int take_cpu_down(void *arg)
+static int cf_check take_cpu_down(void *arg)
{
_take_cpu_down(arg);
return 0;
diff --git a/xen/common/gdbstub.c b/xen/common/gdbstub.c
index 99bfd9a654c9..079c3ca9616a 100644
--- a/xen/common/gdbstub.c
+++ b/xen/common/gdbstub.c
@@ -660,7 +660,7 @@ static int __init cf_check initialise_gdb(void)
}
presmp_initcall(initialise_gdb);
-static void gdb_pause_this_cpu(void *unused)
+static void cf_check gdb_pause_this_cpu(void *unused)
{
unsigned long flags;
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 8b9f37837162..2c916d528ab2 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -360,7 +360,7 @@ static cpumask_t read_clocks_cpumask;
static DEFINE_PER_CPU(s_time_t, read_clocks_time);
static DEFINE_PER_CPU(u64, read_cycles_time);
-static void read_clocks_slave(void *unused)
+static void cf_check read_clocks_slave(void *unused)
{
unsigned int cpu = smp_processor_id();
local_irq_disable();
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 561e238d2d6a..827617502e35 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1898,7 +1898,7 @@ void __init end_boot_allocator(void)
printk("\n");
}
-static void __init smp_scrub_heap_pages(void *data)
+static void __init cf_check smp_scrub_heap_pages(void *data)
{
unsigned long mfn, start, end;
struct page_info *pg;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 17/70] xen: CFI hardening for open_softirq()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (15 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 16/70] xen: CFI hardening for IPIs Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 18/70] xsm/flask/ss: CFI hardening Andrew Cooper
` (54 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/mcheck/mce.c | 2 +-
xen/arch/x86/domain.c | 2 +-
xen/arch/x86/include/asm/flushtlb.h | 2 +-
xen/arch/x86/pv/traps.c | 2 +-
xen/arch/x86/smp.c | 2 +-
xen/arch/x86/time.c | 2 +-
xen/common/rcupdate.c | 2 +-
xen/common/sched/core.c | 6 +++---
xen/common/tasklet.c | 2 +-
xen/common/timer.c | 2 +-
xen/drivers/passthrough/x86/hvm.c | 2 +-
11 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index 43f6c8471a90..3467e0f1a315 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -1837,7 +1837,7 @@ static int mce_delayed_action(mctelem_cookie_t mctc)
}
/* Softirq Handler for this MCE# processing */
-static void mce_softirq(void)
+static void cf_check mce_softirq(void)
{
static DEFINE_MCE_BARRIER(mce_inside_bar);
static DEFINE_MCE_BARRIER(mce_severity_bar);
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index f943283b2a88..1c3a1ec2a080 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -2523,7 +2523,7 @@ void vcpu_mark_events_pending(struct vcpu *v)
vcpu_kick(v);
}
-static void vcpu_kick_softirq(void)
+static void cf_check vcpu_kick_softirq(void)
{
/*
* Nothing to do here: we merely prevent notifiers from racing with checks
diff --git a/xen/arch/x86/include/asm/flushtlb.h b/xen/arch/x86/include/asm/flushtlb.h
index 0be2273387ed..18777f1d4c00 100644
--- a/xen/arch/x86/include/asm/flushtlb.h
+++ b/xen/arch/x86/include/asm/flushtlb.h
@@ -87,7 +87,7 @@ static inline void tlbflush_filter(cpumask_t *mask, uint32_t page_timestamp)
__cpumask_clear_cpu(cpu, mask);
}
-void new_tlbflush_clock_period(void);
+void cf_check new_tlbflush_clock_period(void);
/* Read pagetable base. */
static inline unsigned long read_cr3(void)
diff --git a/xen/arch/x86/pv/traps.c b/xen/arch/x86/pv/traps.c
index 170e1030982b..97fe54b5ee5a 100644
--- a/xen/arch/x86/pv/traps.c
+++ b/xen/arch/x86/pv/traps.c
@@ -130,7 +130,7 @@ bool set_guest_nmi_trapbounce(void)
static DEFINE_PER_CPU(struct vcpu *, softirq_nmi_vcpu);
-static void nmi_softirq(void)
+static void cf_check nmi_softirq(void)
{
struct vcpu **v_ptr = &this_cpu(softirq_nmi_vcpu);
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index f6fd7f95df58..b9a696f61963 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -290,7 +290,7 @@ void flush_area_mask(const cpumask_t *mask, const void *va, unsigned int flags)
}
/* Call with no locks held and interrupts enabled (e.g., softirq context). */
-void new_tlbflush_clock_period(void)
+void cf_check new_tlbflush_clock_period(void)
{
cpumask_t allbutself;
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index b444d6344e79..5a72b66800e4 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1455,7 +1455,7 @@ int cpu_frequency_change(u64 freq)
static DEFINE_PER_CPU(struct cpu_time_stamp, cpu_calibration);
/* Softirq handler for per-CPU time calibration. */
-static void local_time_calibration(void)
+static void cf_check local_time_calibration(void)
{
struct cpu_time *t = &this_cpu(cpu_time);
const struct cpu_time_stamp *c = &this_cpu(cpu_calibration);
diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
index 423d6b1d6d02..212a99acd8c8 100644
--- a/xen/common/rcupdate.c
+++ b/xen/common/rcupdate.c
@@ -466,7 +466,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp,
rcu_do_batch(rdp);
}
-static void rcu_process_callbacks(void)
+static void cf_check rcu_process_callbacks(void)
{
struct rcu_data *rdp = &this_cpu(rcu_data);
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index 285de9ee2a19..b1836b591c0a 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -2568,7 +2568,7 @@ static struct sched_unit *sched_wait_rendezvous_in(struct sched_unit *prev,
return prev->next_task;
}
-static void sched_slave(void)
+static void cf_check sched_slave(void)
{
struct vcpu *v, *vprev = current;
struct sched_unit *prev = vprev->sched_unit, *next;
@@ -2632,7 +2632,7 @@ static void sched_slave(void)
* - deschedule the current domain (scheduler independent).
* - pick a new domain (scheduler dependent).
*/
-static void schedule(void)
+static void cf_check schedule(void)
{
struct vcpu *vnext, *vprev = current;
struct sched_unit *prev = vprev->sched_unit, *next = NULL;
@@ -2928,7 +2928,7 @@ const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu)
return mask;
}
-static void schedule_dummy(void)
+static void cf_check schedule_dummy(void)
{
sched_tasklet_check_cpu(smp_processor_id());
}
diff --git a/xen/common/tasklet.c b/xen/common/tasklet.c
index 1b16bbcdeb0b..3ad67b5c2493 100644
--- a/xen/common/tasklet.c
+++ b/xen/common/tasklet.c
@@ -135,7 +135,7 @@ void do_tasklet(void)
}
/* Softirq context work */
-static void tasklet_softirq_action(void)
+static void cf_check tasklet_softirq_action(void)
{
unsigned int cpu = smp_processor_id();
struct list_head *list = &per_cpu(softirq_tasklet_list, cpu);
diff --git a/xen/common/timer.c b/xen/common/timer.c
index b788050ea1d8..700f191a70b4 100644
--- a/xen/common/timer.c
+++ b/xen/common/timer.c
@@ -450,7 +450,7 @@ static void execute_timer(struct timers *ts, struct timer *t)
}
-static void timer_softirq_action(void)
+static void cf_check timer_softirq_action(void)
{
struct timer *t, **heap, *next;
struct timers *ts;
diff --git a/xen/drivers/passthrough/x86/hvm.c b/xen/drivers/passthrough/x86/hvm.c
index 527bd6a56d83..0e3c0f6aeed3 100644
--- a/xen/drivers/passthrough/x86/hvm.c
+++ b/xen/drivers/passthrough/x86/hvm.c
@@ -1003,7 +1003,7 @@ int arch_pci_clean_pirqs(struct domain *d)
* Note: 'pt_pirq_softirq_reset' can clear the STATE_SCHED before we get to
* doing it. If that is the case we let 'pt_pirq_softirq_reset' do ref-counting.
*/
-static void dpci_softirq(void)
+static void cf_check dpci_softirq(void)
{
unsigned int cpu = smp_processor_id();
LIST_HEAD(our_list);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 18/70] xsm/flask/ss: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (16 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 17/70] xen: CFI hardening for open_softirq() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 19/70] xsm: " Andrew Cooper
` (53 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
xen/xsm/flask/ss/avtab.c | 4 ++--
xen/xsm/flask/ss/conditional.c | 10 ++++----
xen/xsm/flask/ss/conditional.h | 6 ++---
xen/xsm/flask/ss/policydb.c | 53 ++++++++++++++++++++++--------------------
xen/xsm/flask/ss/services.c | 6 ++---
xen/xsm/flask/ss/symtab.c | 5 ++--
6 files changed, 44 insertions(+), 40 deletions(-)
diff --git a/xen/xsm/flask/ss/avtab.c b/xen/xsm/flask/ss/avtab.c
index bfc91c8b0c2c..55c2b4d8a46d 100644
--- a/xen/xsm/flask/ss/avtab.c
+++ b/xen/xsm/flask/ss/avtab.c
@@ -482,8 +482,8 @@ int avtab_read_item(struct avtab *a, void *fp, struct policydb *pol,
return insertf(a, &key, &datum, p);
}
-static int avtab_insertf(struct avtab *a, struct avtab_key *k,
- struct avtab_datum *d, void *p)
+static int cf_check avtab_insertf(
+ struct avtab *a, struct avtab_key *k, struct avtab_datum *d, void *p)
{
return avtab_insert(a, k, d);
}
diff --git a/xen/xsm/flask/ss/conditional.c b/xen/xsm/flask/ss/conditional.c
index 3e58aea55147..b4b116666cf7 100644
--- a/xen/xsm/flask/ss/conditional.c
+++ b/xen/xsm/flask/ss/conditional.c
@@ -189,14 +189,14 @@ int cond_init_bool_indexes(struct policydb *p)
return 0;
}
-int cond_destroy_bool(void *key, void *datum, void *p)
+int cf_check cond_destroy_bool(void *key, void *datum, void *p)
{
xfree(key);
xfree(datum);
return 0;
}
-int cond_index_bool(void *key, void *datum, void *datap)
+int cf_check cond_index_bool(void *key, void *datum, void *datap)
{
struct policydb *p;
struct cond_bool_datum *booldatum;
@@ -220,7 +220,7 @@ static int bool_isvalid(struct cond_bool_datum *b)
return 1;
}
-int cond_read_bool(struct policydb *p, struct hashtab *h, void *fp)
+int cf_check cond_read_bool(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct cond_bool_datum *booldatum;
@@ -268,8 +268,8 @@ struct cond_insertf_data
struct cond_av_list *tail;
};
-static int cond_insertf(struct avtab *a, struct avtab_key *k,
- struct avtab_datum *d, void *ptr)
+static int cf_check cond_insertf(
+ struct avtab *a, struct avtab_key *k, struct avtab_datum *d, void *ptr)
{
struct cond_insertf_data *data = ptr;
struct policydb *p = data->p;
diff --git a/xen/xsm/flask/ss/conditional.h b/xen/xsm/flask/ss/conditional.h
index 59ac6b4b57c1..500fe4305a03 100644
--- a/xen/xsm/flask/ss/conditional.h
+++ b/xen/xsm/flask/ss/conditional.h
@@ -63,11 +63,11 @@ int cond_policydb_init(struct policydb* p);
void cond_policydb_destroy(struct policydb* p);
int cond_init_bool_indexes(struct policydb* p);
-int cond_destroy_bool(void *key, void *datum, void *p);
+int cf_check cond_destroy_bool(void *key, void *datum, void *p);
-int cond_index_bool(void *key, void *datum, void *datap);
+int cf_check cond_index_bool(void *key, void *datum, void *datap);
-int cond_read_bool(struct policydb *p, struct hashtab *h, void *fp);
+int cf_check cond_read_bool(struct policydb *p, struct hashtab *h, void *fp);
int cond_read_list(struct policydb *p, void *fp);
void cond_compute_av(struct avtab *ctab, struct avtab_key *key, struct av_decision *avd);
diff --git a/xen/xsm/flask/ss/policydb.c b/xen/xsm/flask/ss/policydb.c
index 94261643532a..ff2103c63eab 100644
--- a/xen/xsm/flask/ss/policydb.c
+++ b/xen/xsm/flask/ss/policydb.c
@@ -257,12 +257,12 @@ static int policydb_init(struct policydb *p)
* of a class, role, or user are needed.
*/
-static int common_index(void *key, void *datum, void *datap)
+static int cf_check common_index(void *key, void *datum, void *datap)
{
return 0;
}
-static int class_index(void *key, void *datum, void *datap)
+static int cf_check class_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct class_datum *cladatum;
@@ -276,7 +276,7 @@ static int class_index(void *key, void *datum, void *datap)
return 0;
}
-static int role_index(void *key, void *datum, void *datap)
+static int cf_check role_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct role_datum *role;
@@ -292,7 +292,7 @@ static int role_index(void *key, void *datum, void *datap)
return 0;
}
-static int type_index(void *key, void *datum, void *datap)
+static int cf_check type_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct type_datum *typdatum;
@@ -313,7 +313,7 @@ static int type_index(void *key, void *datum, void *datap)
return 0;
}
-static int user_index(void *key, void *datum, void *datap)
+static int cf_check user_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct user_datum *usrdatum;
@@ -329,7 +329,7 @@ static int user_index(void *key, void *datum, void *datap)
return 0;
}
-static int sens_index(void *key, void *datum, void *datap)
+static int cf_check sens_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct level_datum *levdatum;
@@ -348,7 +348,7 @@ static int sens_index(void *key, void *datum, void *datap)
return 0;
}
-static int cat_index(void *key, void *datum, void *datap)
+static int cf_check cat_index(void *key, void *datum, void *datap)
{
struct policydb *p;
struct cat_datum *catdatum;
@@ -506,14 +506,14 @@ static int policydb_index_others(struct policydb *p)
* symbol data in the policy database.
*/
-static int perm_destroy(void *key, void *datum, void *p)
+static int cf_check perm_destroy(void *key, void *datum, void *p)
{
xfree(key);
xfree(datum);
return 0;
}
-static int common_destroy(void *key, void *datum, void *p)
+static int cf_check common_destroy(void *key, void *datum, void *p)
{
struct common_datum *comdatum;
@@ -525,7 +525,7 @@ static int common_destroy(void *key, void *datum, void *p)
return 0;
}
-static int class_destroy(void *key, void *datum, void *p)
+static int cf_check class_destroy(void *key, void *datum, void *p)
{
struct class_datum *cladatum;
struct constraint_node *constraint, *ctemp;
@@ -572,7 +572,7 @@ static int class_destroy(void *key, void *datum, void *p)
return 0;
}
-static int role_destroy(void *key, void *datum, void *p)
+static int cf_check role_destroy(void *key, void *datum, void *p)
{
struct role_datum *role;
@@ -584,14 +584,14 @@ static int role_destroy(void *key, void *datum, void *p)
return 0;
}
-static int type_destroy(void *key, void *datum, void *p)
+static int cf_check type_destroy(void *key, void *datum, void *p)
{
xfree(key);
xfree(datum);
return 0;
}
-static int user_destroy(void *key, void *datum, void *p)
+static int cf_check user_destroy(void *key, void *datum, void *p)
{
struct user_datum *usrdatum;
@@ -605,7 +605,7 @@ static int user_destroy(void *key, void *datum, void *p)
return 0;
}
-static int sens_destroy(void *key, void *datum, void *p)
+static int cf_check sens_destroy(void *key, void *datum, void *p)
{
struct level_datum *levdatum;
@@ -617,7 +617,7 @@ static int sens_destroy(void *key, void *datum, void *p)
return 0;
}
-static int cat_destroy(void *key, void *datum, void *p)
+static int cf_check cat_destroy(void *key, void *datum, void *p)
{
xfree(key);
xfree(datum);
@@ -989,7 +989,7 @@ static int perm_read(struct policydb *p, struct hashtab *h, void *fp)
goto out;
}
-static int common_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check common_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct common_datum *comdatum;
@@ -1151,7 +1151,7 @@ static int read_cons_helper(struct policydb *p, struct constraint_node **nodep,
return 0;
}
-static int class_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check class_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct class_datum *cladatum;
@@ -1250,7 +1250,7 @@ static int class_read(struct policydb *p, struct hashtab *h, void *fp)
goto out;
}
-static int role_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check role_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct role_datum *role;
@@ -1321,7 +1321,7 @@ static int role_read(struct policydb *p, struct hashtab *h, void *fp)
goto out;
}
-static int type_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check type_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct type_datum *typdatum;
@@ -1415,7 +1415,7 @@ static int mls_read_level(struct mls_level *lp, void *fp)
return -EINVAL;
}
-static int user_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check user_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct user_datum *usrdatum;
@@ -1479,7 +1479,7 @@ static int user_read(struct policydb *p, struct hashtab *h, void *fp)
goto out;
}
-static int sens_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check sens_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct level_datum *levdatum;
@@ -1534,7 +1534,7 @@ static int sens_read(struct policydb *p, struct hashtab *h, void *fp)
goto out;
}
-static int cat_read(struct policydb *p, struct hashtab *h, void *fp)
+static int cf_check cat_read(struct policydb *p, struct hashtab *h, void *fp)
{
char *key = NULL;
struct cat_datum *catdatum;
@@ -1591,7 +1591,8 @@ static int (*read_f[SYM_NUM]) (struct policydb *p, struct hashtab *h, void *fp)
cat_read,
};
-static int user_bounds_sanity_check(void *key, void *datum, void *datap)
+static int cf_check user_bounds_sanity_check(
+ void *key, void *datum, void *datap)
{
struct user_datum *upper, *user;
struct policydb *p = datap;
@@ -1631,7 +1632,8 @@ static int user_bounds_sanity_check(void *key, void *datum, void *datap)
return 0;
}
-static int role_bounds_sanity_check(void *key, void *datum, void *datap)
+static int cf_check role_bounds_sanity_check(
+ void *key, void *datum, void *datap)
{
struct role_datum *upper, *role;
struct policydb *p = datap;
@@ -1671,7 +1673,8 @@ static int role_bounds_sanity_check(void *key, void *datum, void *datap)
return 0;
}
-static int type_bounds_sanity_check(void *key, void *datum, void *datap)
+static int cf_check type_bounds_sanity_check(
+ void *key, void *datum, void *datap)
{
struct type_datum *upper, *type;
struct policydb *p = datap;
diff --git a/xen/xsm/flask/ss/services.c b/xen/xsm/flask/ss/services.c
index 42686535f2c4..2f6d3d350d05 100644
--- a/xen/xsm/flask/ss/services.c
+++ b/xen/xsm/flask/ss/services.c
@@ -283,7 +283,7 @@ static int constraint_expr_eval(struct context *scontext,
* security_dump_masked_av - dumps masked permissions during
* security_compute_av due to RBAC, MLS/Constraint and Type bounds.
*/
-static int dump_masked_av_helper(void *k, void *d, void *args)
+static int cf_check dump_masked_av_helper(void *k, void *d, void *args)
{
struct perm_datum *pdatum = d;
char **permission_names = args;
@@ -1240,7 +1240,7 @@ static int validate_classes(struct policydb *p)
}
/* Clone the SID into the new SID table. */
-static int clone_sid(u32 sid, struct context *context, void *arg)
+static int cf_check clone_sid(u32 sid, struct context *context, void *arg)
{
struct sidtab *s = arg;
@@ -1277,7 +1277,7 @@ struct convert_context_args {
* in the policy `p->newp'. Verify that the
* context is valid under the new policy.
*/
-static int convert_context(u32 key, struct context *c, void *p)
+static int cf_check convert_context(u32 key, struct context *c, void *p)
{
struct convert_context_args *args;
struct context oldc;
diff --git a/xen/xsm/flask/ss/symtab.c b/xen/xsm/flask/ss/symtab.c
index d98c116d5b97..0ce7e08c24ea 100644
--- a/xen/xsm/flask/ss/symtab.c
+++ b/xen/xsm/flask/ss/symtab.c
@@ -12,7 +12,7 @@
#include <xen/errno.h>
#include "symtab.h"
-static unsigned int symhash(struct hashtab *h, const void *key)
+static unsigned int cf_check symhash(struct hashtab *h, const void *key)
{
const char *p, *keyp;
unsigned int size;
@@ -26,7 +26,8 @@ static unsigned int symhash(struct hashtab *h, const void *key)
return val & (h->size - 1);
}
-static int symcmp(struct hashtab *h, const void *key1, const void *key2)
+static int cf_check symcmp(
+ struct hashtab *h, const void *key1, const void *key2)
{
const char *keyp1, *keyp2;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 19/70] xsm: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (17 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 18/70] xsm/flask/ss: CFI hardening Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 20/70] xen/sched: " Andrew Cooper
` (52 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
xen/include/xsm/dummy.h | 211 ++++++++++++++++++++++--------------------
xen/xsm/flask/flask_op.c | 2 +-
xen/xsm/flask/hooks.c | 232 ++++++++++++++++++++++++++---------------------
xen/xsm/flask/private.h | 4 +-
xen/xsm/silo.c | 24 ++---
5 files changed, 257 insertions(+), 216 deletions(-)
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index b024119896e6..58afc1d58973 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -101,46 +101,48 @@ static always_inline int xsm_default_action(
}
}
-static XSM_INLINE void xsm_security_domaininfo(
+static XSM_INLINE void cf_check xsm_security_domaininfo(
struct domain *d, struct xen_domctl_getdomaininfo *info)
{
return;
}
-static XSM_INLINE int xsm_domain_create(
+static XSM_INLINE int cf_check xsm_domain_create(
XSM_DEFAULT_ARG struct domain *d, uint32_t ssidref)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_getdomaininfo(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_domctl_scheduler_op(
+static XSM_INLINE int cf_check xsm_domctl_scheduler_op(
XSM_DEFAULT_ARG struct domain *d, int cmd)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
+static XSM_INLINE int cf_check xsm_sysctl_scheduler_op(XSM_DEFAULT_ARG int cmd)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_set_target(
+static XSM_INLINE int cf_check xsm_set_target(
XSM_DEFAULT_ARG struct domain *d, struct domain *e)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_domctl(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int cf_check xsm_domctl(
+ XSM_DEFAULT_ARG struct domain *d, int cmd)
{
XSM_ASSERT_ACTION(XSM_OTHER);
switch ( cmd )
@@ -157,91 +159,93 @@ static XSM_INLINE int xsm_domctl(XSM_DEFAULT_ARG struct domain *d, int cmd)
}
}
-static XSM_INLINE int xsm_sysctl(XSM_DEFAULT_ARG int cmd)
+static XSM_INLINE int cf_check xsm_sysctl(XSM_DEFAULT_ARG int cmd)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG uint32_t clear)
+static XSM_INLINE int cf_check xsm_readconsole(XSM_DEFAULT_ARG uint32_t clear)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_alloc_security_domain(struct domain *d)
+static XSM_INLINE int cf_check xsm_alloc_security_domain(struct domain *d)
{
return 0;
}
-static XSM_INLINE void xsm_free_security_domain(struct domain *d)
+static XSM_INLINE void cf_check xsm_free_security_domain(struct domain *d)
{
return;
}
-static XSM_INLINE int xsm_grant_mapref(
+static XSM_INLINE int cf_check xsm_grant_mapref(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2, uint32_t flags)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_grant_unmapref(
+static XSM_INLINE int cf_check xsm_grant_unmapref(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_grant_setup(
+static XSM_INLINE int cf_check xsm_grant_setup(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_grant_transfer(
+static XSM_INLINE int cf_check xsm_grant_transfer(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_grant_copy(
+static XSM_INLINE int cf_check xsm_grant_copy(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_grant_query_size(
+static XSM_INLINE int cf_check xsm_grant_query_size(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_memory_exchange(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_memory_exchange(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_memory_adjust_reservation(
+static XSM_INLINE int cf_check xsm_memory_adjust_reservation(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_memory_stat_reservation(
+static XSM_INLINE int cf_check xsm_memory_stat_reservation(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_console_io(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int cf_check xsm_console_io(
+ XSM_DEFAULT_ARG struct domain *d, int cmd)
{
XSM_ASSERT_ACTION(XSM_OTHER);
if ( d->is_console )
@@ -253,26 +257,27 @@ static XSM_INLINE int xsm_console_io(XSM_DEFAULT_ARG struct domain *d, int cmd)
return xsm_default_action(XSM_PRIV, d, NULL);
}
-static XSM_INLINE int xsm_profile(XSM_DEFAULT_ARG struct domain *d, int op)
+static XSM_INLINE int cf_check xsm_profile(
+ XSM_DEFAULT_ARG struct domain *d, int op)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d, NULL);
}
-static XSM_INLINE int xsm_kexec(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_kexec(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_schedop_shutdown(
+static XSM_INLINE int cf_check xsm_schedop_shutdown(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_memory_pin_page(
+static XSM_INLINE int cf_check xsm_memory_pin_page(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2,
struct page_info *page)
{
@@ -280,20 +285,20 @@ static XSM_INLINE int xsm_memory_pin_page(
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_claim_pages(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_evtchn_unbound(
+static XSM_INLINE int cf_check xsm_evtchn_unbound(
XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn, domid_t id2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_evtchn_interdomain(
+static XSM_INLINE int cf_check xsm_evtchn_interdomain(
XSM_DEFAULT_ARG struct domain *d1, struct evtchn *chan1, struct domain *d2,
struct evtchn *chan2)
{
@@ -301,89 +306,94 @@ static XSM_INLINE int xsm_evtchn_interdomain(
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE void xsm_evtchn_close_post(struct evtchn *chn)
+static XSM_INLINE void cf_check xsm_evtchn_close_post(struct evtchn *chn)
{
return;
}
-static XSM_INLINE int xsm_evtchn_send(
+static XSM_INLINE int cf_check xsm_evtchn_send(
XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, d, NULL);
}
-static XSM_INLINE int xsm_evtchn_status(
+static XSM_INLINE int cf_check xsm_evtchn_status(
XSM_DEFAULT_ARG struct domain *d, struct evtchn *chn)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_evtchn_reset(
+static XSM_INLINE int cf_check xsm_evtchn_reset(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_alloc_security_evtchns(
+static XSM_INLINE int cf_check xsm_alloc_security_evtchns(
struct evtchn chn[], unsigned int nr)
{
return 0;
}
-static XSM_INLINE void xsm_free_security_evtchns(
+static XSM_INLINE void cf_check xsm_free_security_evtchns(
struct evtchn chn[], unsigned int nr)
{
return;
}
-static XSM_INLINE char *xsm_show_security_evtchn(
+static XSM_INLINE char *cf_check xsm_show_security_evtchn(
struct domain *d, const struct evtchn *chn)
{
return NULL;
}
-static XSM_INLINE int xsm_init_hardware_domain(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_init_hardware_domain(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_get_pod_target(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_get_pod_target(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_set_pod_target(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_set_pod_target(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_get_vnumainfo(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_get_vnumainfo(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static XSM_INLINE int xsm_get_device_group(XSM_DEFAULT_ARG uint32_t machine_bdf)
+static XSM_INLINE int cf_check xsm_get_device_group(
+ XSM_DEFAULT_ARG uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_assign_device(
+static XSM_INLINE int cf_check xsm_assign_device(
XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_deassign_device(
+static XSM_INLINE int cf_check xsm_deassign_device(
XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_HOOK);
@@ -393,14 +403,14 @@ static XSM_INLINE int xsm_deassign_device(
#endif /* HAS_PASSTHROUGH && HAS_PCI */
#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_DEVICE_TREE)
-static XSM_INLINE int xsm_assign_dtdevice(
+static XSM_INLINE int cf_check xsm_assign_dtdevice(
XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_deassign_dtdevice(
+static XSM_INLINE int cf_check xsm_deassign_dtdevice(
XSM_DEFAULT_ARG struct domain *d, const char *dtpath)
{
XSM_ASSERT_ACTION(XSM_HOOK);
@@ -409,142 +419,144 @@ static XSM_INLINE int xsm_deassign_dtdevice(
#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
-static XSM_INLINE int xsm_resource_plug_core(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_resource_plug_core(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_unplug_core(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_resource_unplug_core(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_plug_pci(
+static XSM_INLINE int cf_check xsm_resource_plug_pci(
XSM_DEFAULT_ARG uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_unplug_pci(
+static XSM_INLINE int cf_check xsm_resource_unplug_pci(
XSM_DEFAULT_ARG uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_setup_pci(
+static XSM_INLINE int cf_check xsm_resource_setup_pci(
XSM_DEFAULT_ARG uint32_t machine_bdf)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_setup_gsi(XSM_DEFAULT_ARG int gsi)
+static XSM_INLINE int cf_check xsm_resource_setup_gsi(XSM_DEFAULT_ARG int gsi)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_resource_setup_misc(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_resource_setup_misc(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_page_offline(XSM_DEFAULT_ARG uint32_t cmd)
+static XSM_INLINE int cf_check xsm_page_offline(XSM_DEFAULT_ARG uint32_t cmd)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_hypfs_op(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_hypfs_op(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE long xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
+static XSM_INLINE long cf_check xsm_do_xsm_op(XEN_GUEST_HANDLE_PARAM(void) op)
{
return -ENOSYS;
}
#ifdef CONFIG_COMPAT
-static XSM_INLINE int xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
+static XSM_INLINE int cf_check xsm_do_compat_op(XEN_GUEST_HANDLE_PARAM(void) op)
{
return -ENOSYS;
}
#endif
-static XSM_INLINE char *xsm_show_irq_sid(int irq)
+static XSM_INLINE char *cf_check xsm_show_irq_sid(int irq)
{
return NULL;
}
-static XSM_INLINE int xsm_map_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_map_domain_pirq(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_map_domain_irq(
+static XSM_INLINE int cf_check xsm_map_domain_irq(
XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_unmap_domain_pirq(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_unmap_domain_pirq(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_bind_pt_irq(
+static XSM_INLINE int cf_check xsm_bind_pt_irq(
XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_unbind_pt_irq(
+static XSM_INLINE int cf_check xsm_unbind_pt_irq(
XSM_DEFAULT_ARG struct domain *d, struct xen_domctl_bind_pt_irq *bind)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_unmap_domain_irq(
+static XSM_INLINE int cf_check xsm_unmap_domain_irq(
XSM_DEFAULT_ARG struct domain *d, int irq, const void *data)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_irq_permission(
+static XSM_INLINE int cf_check xsm_irq_permission(
XSM_DEFAULT_ARG struct domain *d, int pirq, uint8_t allow)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_iomem_permission(
+static XSM_INLINE int cf_check xsm_iomem_permission(
XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_iomem_mapping(
+static XSM_INLINE int cf_check xsm_iomem_mapping(
XSM_DEFAULT_ARG struct domain *d, uint64_t s, uint64_t e, uint8_t allow)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_pci_config_permission(
+static XSM_INLINE int cf_check xsm_pci_config_permission(
XSM_DEFAULT_ARG struct domain *d, uint32_t machine_bdf, uint16_t start,
uint16_t end, uint8_t access)
{
@@ -552,41 +564,42 @@ static XSM_INLINE int xsm_pci_config_permission(
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_add_to_physmap(
+static XSM_INLINE int cf_check xsm_add_to_physmap(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_remove_from_physmap(
+static XSM_INLINE int cf_check xsm_remove_from_physmap(
XSM_DEFAULT_ARG struct domain *d1, struct domain *d2)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d1, d2);
}
-static XSM_INLINE int xsm_map_gmfn_foreign(
+static XSM_INLINE int cf_check xsm_map_gmfn_foreign(
XSM_DEFAULT_ARG struct domain *d, struct domain *t)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d, t);
}
-static XSM_INLINE int xsm_hvm_param(
+static XSM_INLINE int cf_check xsm_hvm_param(
XSM_DEFAULT_ARG struct domain *d, unsigned long op)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_hvm_param_altp2mhvm(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_hvm_param_altp2mhvm(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_hvm_altp2mhvm_op(
+static XSM_INLINE int cf_check xsm_hvm_altp2mhvm_op(
XSM_DEFAULT_ARG struct domain *d, uint64_t mode, uint32_t op)
{
XSM_ASSERT_ACTION(XSM_OTHER);
@@ -606,7 +619,7 @@ static XSM_INLINE int xsm_hvm_altp2mhvm_op(
}
}
-static XSM_INLINE int xsm_vm_event_control(
+static XSM_INLINE int cf_check xsm_vm_event_control(
XSM_DEFAULT_ARG struct domain *d, int mode, int op)
{
XSM_ASSERT_ACTION(XSM_PRIV);
@@ -614,7 +627,7 @@ static XSM_INLINE int xsm_vm_event_control(
}
#ifdef CONFIG_MEM_ACCESS
-static XSM_INLINE int xsm_mem_access(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_mem_access(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
@@ -622,7 +635,7 @@ static XSM_INLINE int xsm_mem_access(XSM_DEFAULT_ARG struct domain *d)
#endif
#ifdef CONFIG_MEM_PAGING
-static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
@@ -630,59 +643,61 @@ static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d)
#endif
#ifdef CONFIG_MEM_SHARING
-static XSM_INLINE int xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
#endif
-static XSM_INLINE int xsm_platform_op(XSM_DEFAULT_ARG uint32_t op)
+static XSM_INLINE int cf_check xsm_platform_op(XSM_DEFAULT_ARG uint32_t op)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
#ifdef CONFIG_X86
-static XSM_INLINE int xsm_do_mca(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_do_mca(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_shadow_control(
+static XSM_INLINE int cf_check xsm_shadow_control(
XSM_DEFAULT_ARG struct domain *d, uint32_t op)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_mem_sharing_op(
+static XSM_INLINE int cf_check xsm_mem_sharing_op(
XSM_DEFAULT_ARG struct domain *d, struct domain *cd, int op)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, cd);
}
-static XSM_INLINE int xsm_apic(XSM_DEFAULT_ARG struct domain *d, int cmd)
+static XSM_INLINE int cf_check xsm_apic(
+ XSM_DEFAULT_ARG struct domain *d, int cmd)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, d, NULL);
}
-static XSM_INLINE int xsm_machine_memory_map(XSM_DEFAULT_VOID)
+static XSM_INLINE int cf_check xsm_machine_memory_map(XSM_DEFAULT_VOID)
{
XSM_ASSERT_ACTION(XSM_PRIV);
return xsm_default_action(action, current->domain, NULL);
}
-static XSM_INLINE int xsm_domain_memory_map(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_domain_memory_map(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_mmu_update(
+static XSM_INLINE int cf_check xsm_mmu_update(
XSM_DEFAULT_ARG struct domain *d, struct domain *t, struct domain *f,
uint32_t flags)
{
@@ -695,42 +710,42 @@ static XSM_INLINE int xsm_mmu_update(
return rc;
}
-static XSM_INLINE int xsm_mmuext_op(
+static XSM_INLINE int cf_check xsm_mmuext_op(
XSM_DEFAULT_ARG struct domain *d, struct domain *f)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d, f);
}
-static XSM_INLINE int xsm_update_va_mapping(
+static XSM_INLINE int cf_check xsm_update_va_mapping(
XSM_DEFAULT_ARG struct domain *d, struct domain *f, l1_pgentry_t pte)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d, f);
}
-static XSM_INLINE int xsm_priv_mapping(
+static XSM_INLINE int cf_check xsm_priv_mapping(
XSM_DEFAULT_ARG struct domain *d, struct domain *t)
{
XSM_ASSERT_ACTION(XSM_TARGET);
return xsm_default_action(action, d, t);
}
-static XSM_INLINE int xsm_ioport_permission(
+static XSM_INLINE int cf_check xsm_ioport_permission(
XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_ioport_mapping(
+static XSM_INLINE int cf_check xsm_ioport_mapping(
XSM_DEFAULT_ARG struct domain *d, uint32_t s, uint32_t e, uint8_t allow)
{
XSM_ASSERT_ACTION(XSM_HOOK);
return xsm_default_action(action, current->domain, d);
}
-static XSM_INLINE int xsm_pmu_op(
+static XSM_INLINE int cf_check xsm_pmu_op(
XSM_DEFAULT_ARG struct domain *d, unsigned int op)
{
XSM_ASSERT_ACTION(XSM_OTHER);
@@ -748,30 +763,31 @@ static XSM_INLINE int xsm_pmu_op(
#endif /* CONFIG_X86 */
-static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
#ifdef CONFIG_ARGO
-static XSM_INLINE int xsm_argo_enable(const struct domain *d)
+static XSM_INLINE int cf_check xsm_argo_enable(const struct domain *d)
{
return 0;
}
-static XSM_INLINE int xsm_argo_register_single_source(
+static XSM_INLINE int cf_check xsm_argo_register_single_source(
const struct domain *d, const struct domain *t)
{
return 0;
}
-static XSM_INLINE int xsm_argo_register_any_source(const struct domain *d)
+static XSM_INLINE int cf_check xsm_argo_register_any_source(
+ const struct domain *d)
{
return 0;
}
-static XSM_INLINE int xsm_argo_send(
+static XSM_INLINE int cf_check xsm_argo_send(
const struct domain *d, const struct domain *t)
{
return 0;
@@ -780,7 +796,7 @@ static XSM_INLINE int xsm_argo_send(
#endif /* CONFIG_ARGO */
#include <public/version.h>
-static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
+static XSM_INLINE int cf_check xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
{
XSM_ASSERT_ACTION(XSM_OTHER);
switch ( op )
@@ -804,7 +820,8 @@ static XSM_INLINE int xsm_xen_version(XSM_DEFAULT_ARG uint32_t op)
}
}
-static XSM_INLINE int xsm_domain_resource_map(XSM_DEFAULT_ARG struct domain *d)
+static XSM_INLINE int cf_check xsm_domain_resource_map(
+ XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 2d7ca3abaecd..707be72a3b96 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -607,7 +607,7 @@ static int flask_relabel_domain(struct xen_flask_relabel *arg)
#endif /* !COMPAT */
-ret_t do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op)
+ret_t cf_check do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op)
{
xen_flask_op_t op;
int rv;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 6ff1be28e4a4..63484e323c09 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -157,7 +157,7 @@ static int avc_unknown_permission(const char *name, int id)
return rc;
}
-static int flask_domain_alloc_security(struct domain *d)
+static int cf_check flask_domain_alloc_security(struct domain *d)
{
struct domain_security_struct *dsec;
@@ -186,7 +186,7 @@ static int flask_domain_alloc_security(struct domain *d)
return 0;
}
-static void flask_domain_free_security(struct domain *d)
+static void cf_check flask_domain_free_security(struct domain *d)
{
struct domain_security_struct *dsec = d->ssid;
@@ -197,8 +197,8 @@ static void flask_domain_free_security(struct domain *d)
xfree(dsec);
}
-static int flask_evtchn_unbound(struct domain *d1, struct evtchn *chn,
- domid_t id2)
+static int cf_check flask_evtchn_unbound(
+ struct domain *d1, struct evtchn *chn, domid_t id2)
{
u32 sid1, sid2, newsid;
int rc;
@@ -230,8 +230,9 @@ static int flask_evtchn_unbound(struct domain *d1, struct evtchn *chn,
return rc;
}
-static int flask_evtchn_interdomain(struct domain *d1, struct evtchn *chn1,
- struct domain *d2, struct evtchn *chn2)
+static int cf_check flask_evtchn_interdomain(
+ struct domain *d1, struct evtchn *chn1,
+ struct domain *d2, struct evtchn *chn2)
{
u32 sid1, sid2, newsid, reverse_sid;
int rc;
@@ -273,12 +274,12 @@ static int flask_evtchn_interdomain(struct domain *d1, struct evtchn *chn1,
return rc;
}
-static void flask_evtchn_close_post(struct evtchn *chn)
+static void cf_check flask_evtchn_close_post(struct evtchn *chn)
{
chn->ssid.flask_sid = SECINITSID_UNLABELED;
}
-static int flask_evtchn_send(struct domain *d, struct evtchn *chn)
+static int cf_check flask_evtchn_send(struct domain *d, struct evtchn *chn)
{
int rc;
@@ -298,17 +299,18 @@ static int flask_evtchn_send(struct domain *d, struct evtchn *chn)
return rc;
}
-static int flask_evtchn_status(struct domain *d, struct evtchn *chn)
+static int cf_check flask_evtchn_status(struct domain *d, struct evtchn *chn)
{
return domain_has_evtchn(d, chn, EVENT__STATUS);
}
-static int flask_evtchn_reset(struct domain *d1, struct domain *d2)
+static int cf_check flask_evtchn_reset(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_EVENT, EVENT__RESET);
}
-static int flask_alloc_security_evtchns(struct evtchn chn[], unsigned int nr)
+static int cf_check flask_alloc_security_evtchns(
+ struct evtchn chn[], unsigned int nr)
{
unsigned int i;
@@ -318,7 +320,8 @@ static int flask_alloc_security_evtchns(struct evtchn chn[], unsigned int nr)
return 0;
}
-static void flask_free_security_evtchns(struct evtchn chn[], unsigned int nr)
+static void cf_check flask_free_security_evtchns(
+ struct evtchn chn[], unsigned int nr)
{
unsigned int i;
@@ -329,7 +332,8 @@ static void flask_free_security_evtchns(struct evtchn chn[], unsigned int nr)
chn[i].ssid.flask_sid = SECINITSID_UNLABELED;
}
-static char *flask_show_security_evtchn(struct domain *d, const struct evtchn *chn)
+static char *cf_check flask_show_security_evtchn(
+ struct domain *d, const struct evtchn *chn)
{
int irq;
u32 sid = 0;
@@ -355,13 +359,13 @@ static char *flask_show_security_evtchn(struct domain *d, const struct evtchn *c
return ctx;
}
-static int flask_init_hardware_domain(struct domain *d)
+static int cf_check flask_init_hardware_domain(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__CREATE_HARDWARE_DOMAIN);
}
-static int flask_grant_mapref(struct domain *d1, struct domain *d2,
- uint32_t flags)
+static int cf_check flask_grant_mapref(
+ struct domain *d1, struct domain *d2, uint32_t flags)
{
u32 perms = GRANT__MAP_READ;
@@ -371,73 +375,75 @@ static int flask_grant_mapref(struct domain *d1, struct domain *d2,
return domain_has_perm(d1, d2, SECCLASS_GRANT, perms);
}
-static int flask_grant_unmapref(struct domain *d1, struct domain *d2)
+static int cf_check flask_grant_unmapref(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_GRANT, GRANT__UNMAP);
}
-static int flask_grant_setup(struct domain *d1, struct domain *d2)
+static int cf_check flask_grant_setup(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_GRANT, GRANT__SETUP);
}
-static int flask_grant_transfer(struct domain *d1, struct domain *d2)
+static int cf_check flask_grant_transfer(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_GRANT, GRANT__TRANSFER);
}
-static int flask_grant_copy(struct domain *d1, struct domain *d2)
+static int cf_check flask_grant_copy(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_GRANT, GRANT__COPY);
}
-static int flask_grant_query_size(struct domain *d1, struct domain *d2)
+static int cf_check flask_grant_query_size(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_GRANT, GRANT__QUERY);
}
-static int flask_get_pod_target(struct domain *d)
+static int cf_check flask_get_pod_target(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN, DOMAIN__GETPODTARGET);
}
-static int flask_set_pod_target(struct domain *d)
+static int cf_check flask_set_pod_target(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN, DOMAIN__SETPODTARGET);
}
-static int flask_memory_exchange(struct domain *d)
+static int cf_check flask_memory_exchange(struct domain *d)
{
return current_has_perm(d, SECCLASS_MMU, MMU__EXCHANGE);
}
-static int flask_memory_adjust_reservation(struct domain *d1, struct domain *d2)
+static int cf_check flask_memory_adjust_reservation(
+ struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__ADJUST);
}
-static int flask_memory_stat_reservation(struct domain *d1, struct domain *d2)
+static int cf_check flask_memory_stat_reservation(
+ struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__STAT);
}
-static int flask_memory_pin_page(struct domain *d1, struct domain *d2,
- struct page_info *page)
+static int cf_check flask_memory_pin_page(
+ struct domain *d1, struct domain *d2, struct page_info *page)
{
return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PINPAGE);
}
-static int flask_claim_pages(struct domain *d)
+static int cf_check flask_claim_pages(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SETCLAIM);
}
-static int flask_get_vnumainfo(struct domain *d)
+static int cf_check flask_get_vnumainfo(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__GET_VNUMAINFO);
}
-static int flask_console_io(struct domain *d, int cmd)
+static int cf_check flask_console_io(struct domain *d, int cmd)
{
u32 perm;
@@ -456,7 +462,7 @@ static int flask_console_io(struct domain *d, int cmd)
return domain_has_xen(d, perm);
}
-static int flask_profile(struct domain *d, int op)
+static int cf_check flask_profile(struct domain *d, int op)
{
u32 perm;
@@ -488,23 +494,23 @@ static int flask_profile(struct domain *d, int op)
return domain_has_xen(d, perm);
}
-static int flask_kexec(void)
+static int cf_check flask_kexec(void)
{
return domain_has_xen(current->domain, XEN__KEXEC);
}
-static int flask_schedop_shutdown(struct domain *d1, struct domain *d2)
+static int cf_check flask_schedop_shutdown(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_DOMAIN, DOMAIN__SHUTDOWN);
}
-static void flask_security_domaininfo(struct domain *d,
- struct xen_domctl_getdomaininfo *info)
+static void cf_check flask_security_domaininfo(
+ struct domain *d, struct xen_domctl_getdomaininfo *info)
{
info->ssidref = domain_sid(d);
}
-static int flask_domain_create(struct domain *d, u32 ssidref)
+static int cf_check flask_domain_create(struct domain *d, u32 ssidref)
{
int rc;
struct domain_security_struct *dsec = d->ssid;
@@ -532,12 +538,12 @@ static int flask_domain_create(struct domain *d, u32 ssidref)
return rc;
}
-static int flask_getdomaininfo(struct domain *d)
+static int cf_check flask_getdomaininfo(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN, DOMAIN__GETDOMAININFO);
}
-static int flask_domctl_scheduler_op(struct domain *d, int op)
+static int cf_check flask_domctl_scheduler_op(struct domain *d, int op)
{
switch ( op )
{
@@ -554,7 +560,7 @@ static int flask_domctl_scheduler_op(struct domain *d, int op)
}
}
-static int flask_sysctl_scheduler_op(int op)
+static int cf_check flask_sysctl_scheduler_op(int op)
{
switch ( op )
{
@@ -569,7 +575,7 @@ static int flask_sysctl_scheduler_op(int op)
}
}
-static int flask_set_target(struct domain *d, struct domain *t)
+static int cf_check flask_set_target(struct domain *d, struct domain *t)
{
int rc;
struct domain_security_struct *dsec, *tsec;
@@ -593,7 +599,7 @@ static int flask_set_target(struct domain *d, struct domain *t)
return rc;
}
-static int flask_domctl(struct domain *d, int cmd)
+static int cf_check flask_domctl(struct domain *d, int cmd)
{
switch ( cmd )
{
@@ -757,7 +763,7 @@ static int flask_domctl(struct domain *d, int cmd)
}
}
-static int flask_sysctl(int cmd)
+static int cf_check flask_sysctl(int cmd)
{
switch ( cmd )
{
@@ -835,7 +841,7 @@ static int flask_sysctl(int cmd)
}
}
-static int flask_readconsole(uint32_t clear)
+static int cf_check flask_readconsole(uint32_t clear)
{
u32 perms = XEN__READCONSOLE;
@@ -853,7 +859,7 @@ static inline u32 resource_to_perm(uint8_t access)
return RESOURCE__REMOVE;
}
-static char *flask_show_irq_sid (int irq)
+static char *cf_check flask_show_irq_sid(int irq)
{
u32 sid, ctx_len;
char *ctx;
@@ -867,7 +873,7 @@ static char *flask_show_irq_sid (int irq)
return ctx;
}
-static int flask_map_domain_pirq (struct domain *d)
+static int cf_check flask_map_domain_pirq(struct domain *d)
{
return current_has_perm(d, SECCLASS_RESOURCE, RESOURCE__ADD);
}
@@ -907,7 +913,7 @@ static u32 flask_iommu_resource_use_perm(const struct domain *d)
return perm;
}
-static int flask_map_domain_irq (struct domain *d, int irq, const void *data)
+static int cf_check flask_map_domain_irq(struct domain *d, int irq, const void *data)
{
u32 sid, dsid;
int rc = -EPERM;
@@ -933,7 +939,7 @@ static int flask_map_domain_irq (struct domain *d, int irq, const void *data)
return rc;
}
-static int flask_unmap_domain_pirq (struct domain *d)
+static int cf_check flask_unmap_domain_pirq(struct domain *d)
{
return current_has_perm(d, SECCLASS_RESOURCE, RESOURCE__REMOVE);
}
@@ -954,7 +960,8 @@ static int flask_unmap_domain_msi (struct domain *d, int irq, const void *data,
#endif
}
-static int flask_unmap_domain_irq (struct domain *d, int irq, const void *data)
+static int cf_check flask_unmap_domain_irq(
+ struct domain *d, int irq, const void *data)
{
u32 sid;
int rc = -EPERM;
@@ -972,7 +979,8 @@ static int flask_unmap_domain_irq (struct domain *d, int irq, const void *data)
return rc;
}
-static int flask_bind_pt_irq (struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static int cf_check flask_bind_pt_irq(
+ struct domain *d, struct xen_domctl_bind_pt_irq *bind)
{
u32 dsid, rsid;
int rc = -EPERM;
@@ -998,12 +1006,14 @@ static int flask_bind_pt_irq (struct domain *d, struct xen_domctl_bind_pt_irq *b
return avc_has_perm(dsid, rsid, SECCLASS_RESOURCE, dperm, &ad);
}
-static int flask_unbind_pt_irq (struct domain *d, struct xen_domctl_bind_pt_irq *bind)
+static int cf_check flask_unbind_pt_irq(
+ struct domain *d, struct xen_domctl_bind_pt_irq *bind)
{
return current_has_perm(d, SECCLASS_RESOURCE, RESOURCE__REMOVE);
}
-static int flask_irq_permission (struct domain *d, int pirq, uint8_t access)
+static int cf_check flask_irq_permission(
+ struct domain *d, int pirq, uint8_t access)
{
/* the PIRQ number is not useful; real IRQ is checked during mapping */
return current_has_perm(d, SECCLASS_RESOURCE, resource_to_perm(access));
@@ -1016,7 +1026,8 @@ struct iomem_has_perm_data {
u32 use_perm;
};
-static int _iomem_has_perm(void *v, u32 sid, unsigned long start, unsigned long end)
+static int cf_check _iomem_has_perm(
+ void *v, u32 sid, unsigned long start, unsigned long end)
{
struct iomem_has_perm_data *data = v;
struct avc_audit_data ad;
@@ -1034,7 +1045,8 @@ static int _iomem_has_perm(void *v, u32 sid, unsigned long start, unsigned long
return avc_has_perm(data->dsid, sid, SECCLASS_RESOURCE, data->use_perm, &ad);
}
-static int flask_iomem_permission(struct domain *d, uint64_t start, uint64_t end, uint8_t access)
+static int cf_check flask_iomem_permission(
+ struct domain *d, uint64_t start, uint64_t end, uint8_t access)
{
struct iomem_has_perm_data data;
int rc;
@@ -1056,12 +1068,14 @@ static int flask_iomem_permission(struct domain *d, uint64_t start, uint64_t end
return security_iterate_iomem_sids(start, end, _iomem_has_perm, &data);
}
-static int flask_iomem_mapping(struct domain *d, uint64_t start, uint64_t end, uint8_t access)
+static int cf_check flask_iomem_mapping(struct domain *d, uint64_t start, uint64_t end, uint8_t access)
{
return flask_iomem_permission(d, start, end, access);
}
-static int flask_pci_config_permission(struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end, uint8_t access)
+static int cf_check flask_pci_config_permission(
+ struct domain *d, uint32_t machine_bdf, uint16_t start, uint16_t end,
+ uint8_t access)
{
u32 dsid, rsid;
int rc = -EPERM;
@@ -1085,12 +1099,12 @@ static int flask_pci_config_permission(struct domain *d, uint32_t machine_bdf, u
}
-static int flask_resource_plug_core(void)
+static int cf_check flask_resource_plug_core(void)
{
return avc_current_has_perm(SECINITSID_DOMXEN, SECCLASS_RESOURCE, RESOURCE__PLUG, NULL);
}
-static int flask_resource_unplug_core(void)
+static int cf_check flask_resource_unplug_core(void)
{
return avc_current_has_perm(SECINITSID_DOMXEN, SECCLASS_RESOURCE, RESOURCE__UNPLUG, NULL);
}
@@ -1100,7 +1114,7 @@ static int flask_resource_use_core(void)
return avc_current_has_perm(SECINITSID_DOMXEN, SECCLASS_RESOURCE, RESOURCE__USE, NULL);
}
-static int flask_resource_plug_pci(uint32_t machine_bdf)
+static int cf_check flask_resource_plug_pci(uint32_t machine_bdf)
{
u32 rsid;
int rc = -EPERM;
@@ -1115,7 +1129,7 @@ static int flask_resource_plug_pci(uint32_t machine_bdf)
return avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__PLUG, &ad);
}
-static int flask_resource_unplug_pci(uint32_t machine_bdf)
+static int cf_check flask_resource_unplug_pci(uint32_t machine_bdf)
{
u32 rsid;
int rc = -EPERM;
@@ -1130,7 +1144,7 @@ static int flask_resource_unplug_pci(uint32_t machine_bdf)
return avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__UNPLUG, &ad);
}
-static int flask_resource_setup_pci(uint32_t machine_bdf)
+static int cf_check flask_resource_setup_pci(uint32_t machine_bdf)
{
u32 rsid;
int rc = -EPERM;
@@ -1145,7 +1159,7 @@ static int flask_resource_setup_pci(uint32_t machine_bdf)
return avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__SETUP, &ad);
}
-static int flask_resource_setup_gsi(int gsi)
+static int cf_check flask_resource_setup_gsi(int gsi)
{
u32 rsid;
int rc = -EPERM;
@@ -1158,12 +1172,12 @@ static int flask_resource_setup_gsi(int gsi)
return avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__SETUP, &ad);
}
-static int flask_resource_setup_misc(void)
+static int cf_check flask_resource_setup_misc(void)
{
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_RESOURCE, RESOURCE__SETUP, NULL);
}
-static inline int flask_page_offline(uint32_t cmd)
+static inline int cf_check flask_page_offline(uint32_t cmd)
{
switch (cmd) {
case sysctl_page_offline:
@@ -1177,27 +1191,28 @@ static inline int flask_page_offline(uint32_t cmd)
}
}
-static inline int flask_hypfs_op(void)
+static inline int cf_check flask_hypfs_op(void)
{
return domain_has_xen(current->domain, XEN__HYPFS_OP);
}
-static int flask_add_to_physmap(struct domain *d1, struct domain *d2)
+static int cf_check flask_add_to_physmap(struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP);
}
-static int flask_remove_from_physmap(struct domain *d1, struct domain *d2)
+static int cf_check flask_remove_from_physmap(
+ struct domain *d1, struct domain *d2)
{
return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP);
}
-static int flask_map_gmfn_foreign(struct domain *d, struct domain *t)
+static int cf_check flask_map_gmfn_foreign(struct domain *d, struct domain *t)
{
return domain_has_perm(d, t, SECCLASS_MMU, MMU__MAP_READ | MMU__MAP_WRITE);
}
-static int flask_hvm_param(struct domain *d, unsigned long op)
+static int cf_check flask_hvm_param(struct domain *d, unsigned long op)
{
u32 perm;
@@ -1216,12 +1231,12 @@ static int flask_hvm_param(struct domain *d, unsigned long op)
return current_has_perm(d, SECCLASS_HVM, perm);
}
-static int flask_hvm_param_altp2mhvm(struct domain *d)
+static int cf_check flask_hvm_param_altp2mhvm(struct domain *d)
{
return current_has_perm(d, SECCLASS_HVM, HVM__ALTP2MHVM);
}
-static int flask_hvm_altp2mhvm_op(struct domain *d, uint64_t mode, uint32_t op)
+static int cf_check flask_hvm_altp2mhvm_op(struct domain *d, uint64_t mode, uint32_t op)
{
/*
* Require both mode and XSM to allow the operation. Assume XSM rules
@@ -1245,34 +1260,34 @@ static int flask_hvm_altp2mhvm_op(struct domain *d, uint64_t mode, uint32_t op)
return current_has_perm(d, SECCLASS_HVM, HVM__ALTP2MHVM_OP);
}
-static int flask_vm_event_control(struct domain *d, int mode, int op)
+static int cf_check flask_vm_event_control(struct domain *d, int mode, int op)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__VM_EVENT);
}
#ifdef CONFIG_MEM_ACCESS
-static int flask_mem_access(struct domain *d)
+static int cf_check flask_mem_access(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__MEM_ACCESS);
}
#endif
#ifdef CONFIG_MEM_PAGING
-static int flask_mem_paging(struct domain *d)
+static int cf_check flask_mem_paging(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__MEM_PAGING);
}
#endif
#ifdef CONFIG_MEM_SHARING
-static int flask_mem_sharing(struct domain *d)
+static int cf_check flask_mem_sharing(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__MEM_SHARING);
}
#endif
#if defined(CONFIG_HAS_PASSTHROUGH) && defined(CONFIG_HAS_PCI)
-static int flask_get_device_group(uint32_t machine_bdf)
+static int cf_check flask_get_device_group(uint32_t machine_bdf)
{
u32 rsid;
int rc = -EPERM;
@@ -1296,7 +1311,7 @@ static int flask_test_assign_device(uint32_t machine_bdf)
return avc_current_has_perm(rsid, SECCLASS_RESOURCE, RESOURCE__STAT_DEVICE, NULL);
}
-static int flask_assign_device(struct domain *d, uint32_t machine_bdf)
+static int cf_check flask_assign_device(struct domain *d, uint32_t machine_bdf)
{
u32 dsid, rsid;
int rc = -EPERM;
@@ -1326,7 +1341,8 @@ static int flask_assign_device(struct domain *d, uint32_t machine_bdf)
return avc_has_perm(dsid, rsid, SECCLASS_RESOURCE, dperm, &ad);
}
-static int flask_deassign_device(struct domain *d, uint32_t machine_bdf)
+static int cf_check flask_deassign_device(
+ struct domain *d, uint32_t machine_bdf)
{
u32 rsid;
int rc = -EPERM;
@@ -1357,7 +1373,7 @@ static int flask_test_assign_dtdevice(const char *dtpath)
NULL);
}
-static int flask_assign_dtdevice(struct domain *d, const char *dtpath)
+static int cf_check flask_assign_dtdevice(struct domain *d, const char *dtpath)
{
u32 dsid, rsid;
int rc = -EPERM;
@@ -1387,7 +1403,8 @@ static int flask_assign_dtdevice(struct domain *d, const char *dtpath)
return avc_has_perm(dsid, rsid, SECCLASS_RESOURCE, dperm, &ad);
}
-static int flask_deassign_dtdevice(struct domain *d, const char *dtpath)
+static int cf_check flask_deassign_dtdevice(
+ struct domain *d, const char *dtpath)
{
u32 rsid;
int rc = -EPERM;
@@ -1405,7 +1422,7 @@ static int flask_deassign_dtdevice(struct domain *d, const char *dtpath)
}
#endif /* HAS_PASSTHROUGH && HAS_DEVICE_TREE */
-static int flask_platform_op(uint32_t op)
+static int cf_check flask_platform_op(uint32_t op)
{
switch ( op )
{
@@ -1474,12 +1491,12 @@ static int flask_platform_op(uint32_t op)
}
#ifdef CONFIG_X86
-static int flask_do_mca(void)
+static int cf_check flask_do_mca(void)
{
return domain_has_xen(current->domain, XEN__MCA_OP);
}
-static int flask_shadow_control(struct domain *d, uint32_t op)
+static int cf_check flask_shadow_control(struct domain *d, uint32_t op)
{
u32 perm;
@@ -1513,7 +1530,8 @@ struct ioport_has_perm_data {
u32 use_perm;
};
-static int _ioport_has_perm(void *v, u32 sid, unsigned long start, unsigned long end)
+static int cf_check _ioport_has_perm(
+ void *v, u32 sid, unsigned long start, unsigned long end)
{
struct ioport_has_perm_data *data = v;
struct avc_audit_data ad;
@@ -1531,7 +1549,8 @@ static int _ioport_has_perm(void *v, u32 sid, unsigned long start, unsigned long
return avc_has_perm(data->dsid, sid, SECCLASS_RESOURCE, data->use_perm, &ad);
}
-static int flask_ioport_permission(struct domain *d, uint32_t start, uint32_t end, uint8_t access)
+static int cf_check flask_ioport_permission(
+ struct domain *d, uint32_t start, uint32_t end, uint8_t access)
{
int rc;
struct ioport_has_perm_data data;
@@ -1554,12 +1573,14 @@ static int flask_ioport_permission(struct domain *d, uint32_t start, uint32_t en
return security_iterate_ioport_sids(start, end, _ioport_has_perm, &data);
}
-static int flask_ioport_mapping(struct domain *d, uint32_t start, uint32_t end, uint8_t access)
+static int cf_check flask_ioport_mapping(
+ struct domain *d, uint32_t start, uint32_t end, uint8_t access)
{
return flask_ioport_permission(d, start, end, access);
}
-static int flask_mem_sharing_op(struct domain *d, struct domain *cd, int op)
+static int cf_check flask_mem_sharing_op(
+ struct domain *d, struct domain *cd, int op)
{
int rc = current_has_perm(cd, SECCLASS_HVM, HVM__MEM_SHARING);
if ( rc )
@@ -1567,7 +1588,7 @@ static int flask_mem_sharing_op(struct domain *d, struct domain *cd, int op)
return domain_has_perm(d, cd, SECCLASS_HVM, HVM__SHARE_MEM);
}
-static int flask_apic(struct domain *d, int cmd)
+static int cf_check flask_apic(struct domain *d, int cmd)
{
u32 perm;
@@ -1587,18 +1608,18 @@ static int flask_apic(struct domain *d, int cmd)
return domain_has_xen(d, perm);
}
-static int flask_machine_memory_map(void)
+static int cf_check flask_machine_memory_map(void)
{
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_MMU, MMU__MEMORYMAP, NULL);
}
-static int flask_domain_memory_map(struct domain *d)
+static int cf_check flask_domain_memory_map(struct domain *d)
{
return current_has_perm(d, SECCLASS_MMU, MMU__MEMORYMAP);
}
-static int flask_mmu_update(struct domain *d, struct domain *t,
- struct domain *f, uint32_t flags)
+static int cf_check flask_mmu_update(
+ struct domain *d, struct domain *t, struct domain *f, uint32_t flags)
{
int rc = 0;
u32 map_perms = 0;
@@ -1620,13 +1641,13 @@ static int flask_mmu_update(struct domain *d, struct domain *t,
return rc;
}
-static int flask_mmuext_op(struct domain *d, struct domain *f)
+static int cf_check flask_mmuext_op(struct domain *d, struct domain *f)
{
return domain_has_perm(d, f, SECCLASS_MMU, MMU__MMUEXT_OP);
}
-static int flask_update_va_mapping(struct domain *d, struct domain *f,
- l1_pgentry_t pte)
+static int cf_check flask_update_va_mapping(
+ struct domain *d, struct domain *f, l1_pgentry_t pte)
{
u32 map_perms = MMU__MAP_READ;
if ( !(l1e_get_flags(pte) & _PAGE_PRESENT) )
@@ -1637,12 +1658,12 @@ static int flask_update_va_mapping(struct domain *d, struct domain *f,
return domain_has_perm(d, f, SECCLASS_MMU, map_perms);
}
-static int flask_priv_mapping(struct domain *d, struct domain *t)
+static int cf_check flask_priv_mapping(struct domain *d, struct domain *t)
{
return domain_has_perm(d, t, SECCLASS_MMU, MMU__TARGET_HACK);
}
-static int flask_pmu_op (struct domain *d, unsigned int op)
+static int cf_check flask_pmu_op(struct domain *d, unsigned int op)
{
u32 dsid = domain_sid(d);
@@ -1666,12 +1687,12 @@ static int flask_pmu_op (struct domain *d, unsigned int op)
}
#endif /* CONFIG_X86 */
-static int flask_dm_op(struct domain *d)
+static int cf_check flask_dm_op(struct domain *d)
{
return current_has_perm(d, SECCLASS_HVM, HVM__DM);
}
-static int flask_xen_version (uint32_t op)
+static int cf_check flask_xen_version(uint32_t op)
{
u32 dsid = domain_sid(current->domain);
@@ -1711,32 +1732,33 @@ static int flask_xen_version (uint32_t op)
}
}
-static int flask_domain_resource_map(struct domain *d)
+static int cf_check flask_domain_resource_map(struct domain *d)
{
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__RESOURCE_MAP);
}
#ifdef CONFIG_ARGO
-static int flask_argo_enable(const struct domain *d)
+static int cf_check flask_argo_enable(const struct domain *d)
{
return avc_has_perm(domain_sid(d), SECINITSID_XEN, SECCLASS_ARGO,
ARGO__ENABLE, NULL);
}
-static int flask_argo_register_single_source(const struct domain *d,
- const struct domain *t)
+static int cf_check flask_argo_register_single_source(
+ const struct domain *d, const struct domain *t)
{
return domain_has_perm(d, t, SECCLASS_ARGO,
ARGO__REGISTER_SINGLE_SOURCE);
}
-static int flask_argo_register_any_source(const struct domain *d)
+static int cf_check flask_argo_register_any_source(const struct domain *d)
{
return avc_has_perm(domain_sid(d), SECINITSID_XEN, SECCLASS_ARGO,
ARGO__REGISTER_ANY_SOURCE, NULL);
}
-static int flask_argo_send(const struct domain *d, const struct domain *t)
+static int cf_check flask_argo_send(
+ const struct domain *d, const struct domain *t)
{
return domain_has_perm(d, t, SECCLASS_ARGO, ARGO__SEND);
}
diff --git a/xen/xsm/flask/private.h b/xen/xsm/flask/private.h
index 73b0de87245a..429f213cce74 100644
--- a/xen/xsm/flask/private.h
+++ b/xen/xsm/flask/private.h
@@ -3,7 +3,7 @@
#include <public/xen.h>
-long do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
-int compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+long cf_check do_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
+int cf_check compat_flask_op(XEN_GUEST_HANDLE_PARAM(void) u_flask_op);
#endif /* XSM_FLASK_PRIVATE */
diff --git a/xen/xsm/silo.c b/xen/xsm/silo.c
index 3550dded7b4e..4d5fc98e7e54 100644
--- a/xen/xsm/silo.c
+++ b/xen/xsm/silo.c
@@ -33,8 +33,8 @@ static bool silo_mode_dom_check(const struct domain *ldom,
is_control_domain(rdom) || ldom == rdom);
}
-static int silo_evtchn_unbound(struct domain *d1, struct evtchn *chn,
- domid_t id2)
+static int cf_check silo_evtchn_unbound(
+ struct domain *d1, struct evtchn *chn, domid_t id2)
{
int rc = -EPERM;
struct domain *d2 = rcu_lock_domain_by_any_id(id2);
@@ -51,30 +51,31 @@ static int silo_evtchn_unbound(struct domain *d1, struct evtchn *chn,
return rc;
}
-static int silo_evtchn_interdomain(struct domain *d1, struct evtchn *chan1,
- struct domain *d2, struct evtchn *chan2)
+static int cf_check silo_evtchn_interdomain(
+ struct domain *d1, struct evtchn *chan1,
+ struct domain *d2, struct evtchn *chan2)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_evtchn_interdomain(d1, chan1, d2, chan2);
return -EPERM;
}
-static int silo_grant_mapref(struct domain *d1, struct domain *d2,
- uint32_t flags)
+static int cf_check silo_grant_mapref(
+ struct domain *d1, struct domain *d2, uint32_t flags)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_grant_mapref(d1, d2, flags);
return -EPERM;
}
-static int silo_grant_transfer(struct domain *d1, struct domain *d2)
+static int cf_check silo_grant_transfer(struct domain *d1, struct domain *d2)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_grant_transfer(d1, d2);
return -EPERM;
}
-static int silo_grant_copy(struct domain *d1, struct domain *d2)
+static int cf_check silo_grant_copy(struct domain *d1, struct domain *d2)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_grant_copy(d1, d2);
@@ -83,15 +84,16 @@ static int silo_grant_copy(struct domain *d1, struct domain *d2)
#ifdef CONFIG_ARGO
-static int silo_argo_register_single_source(const struct domain *d1,
- const struct domain *d2)
+static int cf_check silo_argo_register_single_source(
+ const struct domain *d1, const struct domain *d2)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_argo_register_single_source(d1, d2);
return -EPERM;
}
-static int silo_argo_send(const struct domain *d1, const struct domain *d2)
+static int cf_check silo_argo_send(
+ const struct domain *d1, const struct domain *d2)
{
if ( silo_mode_dom_check(d1, d2) )
return xsm_argo_send(d1, d2);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 20/70] xen/sched: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (18 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 19/70] xsm: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 21/70] xen/evtchn: " Andrew Cooper
` (51 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Dario Faggioli <dfaggioli@suse.com>
---
xen/common/sched/arinc653.c | 20 +++++++--------
xen/common/sched/core.c | 8 +++---
xen/common/sched/credit.c | 49 ++++++++++++++++++------------------
xen/common/sched/credit2.c | 51 +++++++++++++++++++-------------------
xen/common/sched/null.c | 60 +++++++++++++++++++++++----------------------
xen/common/sched/rt.c | 42 +++++++++++++++----------------
6 files changed, 115 insertions(+), 115 deletions(-)
diff --git a/xen/common/sched/arinc653.c b/xen/common/sched/arinc653.c
index 542191822192..a82c0d7314a1 100644
--- a/xen/common/sched/arinc653.c
+++ b/xen/common/sched/arinc653.c
@@ -343,7 +343,7 @@ arinc653_sched_get(
* <li> !0 = error
* </ul>
*/
-static int
+static int cf_check
a653sched_init(struct scheduler *ops)
{
a653sched_priv_t *prv;
@@ -366,7 +366,7 @@ a653sched_init(struct scheduler *ops)
*
* @param ops Pointer to this instance of the scheduler structure
*/
-static void
+static void cf_check
a653sched_deinit(struct scheduler *ops)
{
xfree(SCHED_PRIV(ops));
@@ -381,7 +381,7 @@ a653sched_deinit(struct scheduler *ops)
*
* @return Pointer to the allocated data
*/
-static void *
+static void *cf_check
a653sched_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
void *dd)
{
@@ -442,7 +442,7 @@ a653sched_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
*
* @param ops Pointer to this instance of the scheduler structure
*/
-static void
+static void cf_check
a653sched_free_udata(const struct scheduler *ops, void *priv)
{
a653sched_priv_t *sched_priv = SCHED_PRIV(ops);
@@ -469,7 +469,7 @@ a653sched_free_udata(const struct scheduler *ops, void *priv)
* @param ops Pointer to this instance of the scheduler structure
* @param unit Pointer to struct sched_unit
*/
-static void
+static void cf_check
a653sched_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
{
if ( AUNIT(unit) != NULL )
@@ -489,7 +489,7 @@ a653sched_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
* @param ops Pointer to this instance of the scheduler structure
* @param unit Pointer to struct sched_unit
*/
-static void
+static void cf_check
a653sched_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
{
if ( AUNIT(unit) != NULL )
@@ -505,7 +505,7 @@ a653sched_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
* @param ops Pointer to this instance of the scheduler structure
* @param now Current time
*/
-static void
+static void cf_check
a653sched_do_schedule(
const struct scheduler *ops,
struct sched_unit *prev,
@@ -604,7 +604,7 @@ a653sched_do_schedule(
*
* @return Scheduler resource to run on
*/
-static struct sched_resource *
+static struct sched_resource *cf_check
a653sched_pick_resource(const struct scheduler *ops,
const struct sched_unit *unit)
{
@@ -634,7 +634,7 @@ a653sched_pick_resource(const struct scheduler *ops,
* @param pdata scheduler specific PCPU data (we don't have any)
* @param vdata scheduler specific UNIT data of the idle unit
*/
-static spinlock_t *
+static spinlock_t *cf_check
a653_switch_sched(struct scheduler *new_ops, unsigned int cpu,
void *pdata, void *vdata)
{
@@ -656,7 +656,7 @@ a653_switch_sched(struct scheduler *new_ops, unsigned int cpu,
* @param ops Pointer to this instance of the scheduler structure
* @param sc Pointer to the scheduler operation specified by Domain 0
*/
-static int
+static int cf_check
a653sched_adjust_global(const struct scheduler *ops,
struct xen_sysctl_scheduler_op *sc)
{
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index b1836b591c0a..9e09d9befa23 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -98,13 +98,13 @@ static bool scheduler_active;
static void sched_set_affinity(
struct sched_unit *unit, const cpumask_t *hard, const cpumask_t *soft);
-static struct sched_resource *
+static struct sched_resource *cf_check
sched_idle_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
{
return unit->res;
}
-static void *
+static void *cf_check
sched_idle_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
void *dd)
{
@@ -112,12 +112,12 @@ sched_idle_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
return ZERO_BLOCK_PTR;
}
-static void
+static void cf_check
sched_idle_free_udata(const struct scheduler *ops, void *priv)
{
}
-static void sched_idle_schedule(
+static void cf_check sched_idle_schedule(
const struct scheduler *ops, struct sched_unit *unit, s_time_t now,
bool tasklet_work_scheduled)
{
diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c
index 5635271f6fea..4d3bd8cba6fc 100644
--- a/xen/common/sched/credit.c
+++ b/xen/common/sched/credit.c
@@ -507,7 +507,7 @@ static inline void __runq_tickle(const struct csched_unit *new)
SCHED_STAT_CRANK(tickled_no_cpu);
}
-static void
+static void cf_check
csched_free_pdata(const struct scheduler *ops, void *pcpu, int cpu)
{
const struct csched_private *prv = CSCHED_PRIV(ops);
@@ -524,7 +524,7 @@ csched_free_pdata(const struct scheduler *ops, void *pcpu, int cpu)
xfree(pcpu);
}
-static void
+static void cf_check
csched_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
{
struct csched_private *prv = CSCHED_PRIV(ops);
@@ -566,7 +566,7 @@ csched_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
spin_unlock_irqrestore(&prv->lock, flags);
}
-static void *
+static void *cf_check
csched_alloc_pdata(const struct scheduler *ops, int cpu)
{
struct csched_pcpu *spc;
@@ -615,7 +615,7 @@ init_pdata(struct csched_private *prv, struct csched_pcpu *spc, int cpu)
}
/* Change the scheduler of cpu to us (Credit). */
-static spinlock_t *
+static spinlock_t *cf_check
csched_switch_sched(struct scheduler *new_ops, unsigned int cpu,
void *pdata, void *vdata)
{
@@ -848,7 +848,7 @@ _csched_cpu_pick(const struct scheduler *ops, const struct sched_unit *unit,
return cpu;
}
-static struct sched_resource *
+static struct sched_resource *cf_check
csched_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
{
struct csched_unit *svc = CSCHED_UNIT(unit);
@@ -985,9 +985,8 @@ csched_unit_acct(struct csched_private *prv, unsigned int cpu)
}
}
-static void *
-csched_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
- void *dd)
+static void *cf_check csched_alloc_udata(
+ const struct scheduler *ops, struct sched_unit *unit, void *dd)
{
struct csched_unit *svc;
@@ -1007,7 +1006,7 @@ csched_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
return svc;
}
-static void
+static void cf_check
csched_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched_unit *svc = unit->priv;
@@ -1032,7 +1031,7 @@ csched_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
SCHED_STAT_CRANK(unit_insert);
}
-static void
+static void cf_check
csched_free_udata(const struct scheduler *ops, void *priv)
{
struct csched_unit *svc = priv;
@@ -1042,7 +1041,7 @@ csched_free_udata(const struct scheduler *ops, void *priv)
xfree(svc);
}
-static void
+static void cf_check
csched_unit_remove(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched_private *prv = CSCHED_PRIV(ops);
@@ -1069,7 +1068,7 @@ csched_unit_remove(const struct scheduler *ops, struct sched_unit *unit)
BUG_ON( sdom == NULL );
}
-static void
+static void cf_check
csched_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched_unit * const svc = CSCHED_UNIT(unit);
@@ -1094,7 +1093,7 @@ csched_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
runq_remove(svc);
}
-static void
+static void cf_check
csched_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched_unit * const svc = CSCHED_UNIT(unit);
@@ -1156,7 +1155,7 @@ csched_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
__runq_tickle(svc);
}
-static void
+static void cf_check
csched_unit_yield(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched_unit * const svc = CSCHED_UNIT(unit);
@@ -1165,7 +1164,7 @@ csched_unit_yield(const struct scheduler *ops, struct sched_unit *unit)
set_bit(CSCHED_FLAG_UNIT_YIELD, &svc->flags);
}
-static int
+static int cf_check
csched_dom_cntl(
const struct scheduler *ops,
struct domain *d,
@@ -1210,7 +1209,7 @@ csched_dom_cntl(
return rc;
}
-static void
+static void cf_check
csched_aff_cntl(const struct scheduler *ops, struct sched_unit *unit,
const cpumask_t *hard, const cpumask_t *soft)
{
@@ -1238,7 +1237,7 @@ __csched_set_tslice(struct csched_private *prv, unsigned int timeslice_ms)
prv->credit = prv->credits_per_tslice * prv->ncpus;
}
-static int
+static int cf_check
csched_sys_cntl(const struct scheduler *ops,
struct xen_sysctl_scheduler_op *sc)
{
@@ -1281,7 +1280,7 @@ csched_sys_cntl(const struct scheduler *ops,
return rc;
}
-static void *
+static void *cf_check
csched_alloc_domdata(const struct scheduler *ops, struct domain *dom)
{
struct csched_dom *sdom;
@@ -1299,7 +1298,7 @@ csched_alloc_domdata(const struct scheduler *ops, struct domain *dom)
return sdom;
}
-static void
+static void cf_check
csched_free_domdata(const struct scheduler *ops, void *data)
{
xfree(data);
@@ -1809,7 +1808,7 @@ csched_load_balance(struct csched_private *prv, int cpu,
* This function is in the critical path. It is designed to be simple and
* fast for the common case.
*/
-static void csched_schedule(
+static void cf_check csched_schedule(
const struct scheduler *ops, struct sched_unit *unit, s_time_t now,
bool tasklet_work_scheduled)
{
@@ -2026,7 +2025,7 @@ csched_dump_unit(const struct csched_unit *svc)
printk("\n");
}
-static void
+static void cf_check
csched_dump_pcpu(const struct scheduler *ops, int cpu)
{
const struct list_head *runq;
@@ -2079,7 +2078,7 @@ csched_dump_pcpu(const struct scheduler *ops, int cpu)
spin_unlock_irqrestore(&prv->lock, flags);
}
-static void
+static void cf_check
csched_dump(const struct scheduler *ops)
{
struct list_head *iter_sdom, *iter_svc;
@@ -2143,7 +2142,7 @@ csched_dump(const struct scheduler *ops)
spin_unlock_irqrestore(&prv->lock, flags);
}
-static int __init
+static int __init cf_check
csched_global_init(void)
{
if ( sched_credit_tslice_ms > XEN_SYSCTL_CSCHED_TSLICE_MAX ||
@@ -2173,7 +2172,7 @@ csched_global_init(void)
return 0;
}
-static int
+static int cf_check
csched_init(struct scheduler *ops)
{
struct csched_private *prv;
@@ -2215,7 +2214,7 @@ csched_init(struct scheduler *ops)
return 0;
}
-static void
+static void cf_check
csched_deinit(struct scheduler *ops)
{
struct csched_private *prv;
diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index d96e2749ddfb..0e3f89e5378e 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -2164,7 +2164,7 @@ csched2_unit_check(const struct sched_unit *unit)
#define CSCHED2_UNIT_CHECK(unit)
#endif
-static void *
+static void *cf_check
csched2_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
void *dd)
{
@@ -2208,7 +2208,7 @@ csched2_alloc_udata(const struct scheduler *ops, struct sched_unit *unit,
return svc;
}
-static void
+static void cf_check
csched2_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -2230,7 +2230,7 @@ csched2_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
__clear_bit(__CSFLAG_delayed_runq_add, &svc->flags);
}
-static void
+static void cf_check
csched2_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -2285,7 +2285,7 @@ csched2_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
return;
}
-static void
+static void cf_check
csched2_unit_yield(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -2293,7 +2293,7 @@ csched2_unit_yield(const struct scheduler *ops, struct sched_unit *unit)
__set_bit(__CSFLAG_unit_yield, &svc->flags);
}
-static void
+static void cf_check
csched2_context_saved(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -2335,7 +2335,7 @@ csched2_context_saved(const struct scheduler *ops, struct sched_unit *unit)
}
#define MAX_LOAD (STIME_MAX)
-static struct sched_resource *
+static struct sched_resource *cf_check
csched2_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
{
struct csched2_private *prv = csched2_priv(ops);
@@ -2867,8 +2867,7 @@ static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
return;
}
-static void
-csched2_unit_migrate(
+static void cf_check csched2_unit_migrate(
const struct scheduler *ops, struct sched_unit *unit, unsigned int new_cpu)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -2894,7 +2893,7 @@ csched2_unit_migrate(
sched_set_res(unit, get_sched_res(new_cpu));
}
-static int
+static int cf_check
csched2_dom_cntl(
const struct scheduler *ops,
struct domain *d,
@@ -3100,7 +3099,7 @@ csched2_dom_cntl(
return rc;
}
-static void
+static void cf_check
csched2_aff_cntl(const struct scheduler *ops, struct sched_unit *unit,
const cpumask_t *hard, const cpumask_t *soft)
{
@@ -3116,8 +3115,8 @@ csched2_aff_cntl(const struct scheduler *ops, struct sched_unit *unit,
__clear_bit(__CSFLAG_pinned, &svc->flags);
}
-static int csched2_sys_cntl(const struct scheduler *ops,
- struct xen_sysctl_scheduler_op *sc)
+static int cf_check csched2_sys_cntl(
+ const struct scheduler *ops, struct xen_sysctl_scheduler_op *sc)
{
struct xen_sysctl_credit2_schedule *params = &sc->u.sched_credit2;
struct csched2_private *prv = csched2_priv(ops);
@@ -3148,7 +3147,7 @@ static int csched2_sys_cntl(const struct scheduler *ops,
return 0;
}
-static void *
+static void *cf_check
csched2_alloc_domdata(const struct scheduler *ops, struct domain *dom)
{
struct csched2_private *prv = csched2_priv(ops);
@@ -3180,7 +3179,7 @@ csched2_alloc_domdata(const struct scheduler *ops, struct domain *dom)
return sdom;
}
-static void
+static void cf_check
csched2_free_domdata(const struct scheduler *ops, void *data)
{
struct csched2_dom *sdom = data;
@@ -3200,7 +3199,7 @@ csched2_free_domdata(const struct scheduler *ops, void *data)
}
}
-static void
+static void cf_check
csched2_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
{
const struct csched2_unit *svc = unit->priv;
@@ -3231,7 +3230,7 @@ csched2_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
CSCHED2_UNIT_CHECK(unit);
}
-static void
+static void cf_check
csched2_free_udata(const struct scheduler *ops, void *priv)
{
struct csched2_unit *svc = priv;
@@ -3239,7 +3238,7 @@ csched2_free_udata(const struct scheduler *ops, void *priv)
xfree(svc);
}
-static void
+static void cf_check
csched2_unit_remove(const struct scheduler *ops, struct sched_unit *unit)
{
struct csched2_unit * const svc = csched2_unit(unit);
@@ -3558,7 +3557,7 @@ runq_candidate(struct csched2_runqueue_data *rqd,
* This function is in the critical path. It is designed to be simple and
* fast for the common case.
*/
-static void csched2_schedule(
+static void cf_check csched2_schedule(
const struct scheduler *ops, struct sched_unit *currunit, s_time_t now,
bool tasklet_work_scheduled)
{
@@ -3790,7 +3789,7 @@ dump_pcpu(const struct scheduler *ops, int cpu)
}
}
-static void
+static void cf_check
csched2_dump(const struct scheduler *ops)
{
struct list_head *iter_sdom;
@@ -3898,7 +3897,7 @@ csched2_dump(const struct scheduler *ops)
read_unlock_irqrestore(&prv->lock, flags);
}
-static void *
+static void *cf_check
csched2_alloc_pdata(const struct scheduler *ops, int cpu)
{
struct csched2_pcpu *spc;
@@ -3988,7 +3987,7 @@ init_pdata(struct csched2_private *prv, struct csched2_pcpu *spc,
}
/* Change the scheduler of cpu to us (Credit2). */
-static spinlock_t *
+static spinlock_t *cf_check
csched2_switch_sched(struct scheduler *new_ops, unsigned int cpu,
void *pdata, void *vdata)
{
@@ -4026,7 +4025,7 @@ csched2_switch_sched(struct scheduler *new_ops, unsigned int cpu,
return &rqd->lock;
}
-static void
+static void cf_check
csched2_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
{
unsigned long flags;
@@ -4086,7 +4085,7 @@ csched2_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
return;
}
-static void
+static void cf_check
csched2_free_pdata(const struct scheduler *ops, void *pcpu, int cpu)
{
struct csched2_private *prv = csched2_priv(ops);
@@ -4115,7 +4114,7 @@ csched2_free_pdata(const struct scheduler *ops, void *pcpu, int cpu)
xfree(pcpu);
}
-static int __init
+static int __init cf_check
csched2_global_init(void)
{
if ( opt_load_precision_shift < LOADAVG_PRECISION_SHIFT_MIN )
@@ -4142,7 +4141,7 @@ csched2_global_init(void)
return 0;
}
-static int
+static int cf_check
csched2_init(struct scheduler *ops)
{
struct csched2_private *prv;
@@ -4190,7 +4189,7 @@ csched2_init(struct scheduler *ops)
return 0;
}
-static void
+static void cf_check
csched2_deinit(struct scheduler *ops)
{
struct csched2_private *prv;
diff --git a/xen/common/sched/null.c b/xen/common/sched/null.c
index 82d5d1baab85..65a0a6c5312d 100644
--- a/xen/common/sched/null.c
+++ b/xen/common/sched/null.c
@@ -130,7 +130,7 @@ static inline bool unit_check_affinity(struct sched_unit *unit,
return cpumask_test_cpu(cpu, cpumask_scratch_cpu(cpu));
}
-static int null_init(struct scheduler *ops)
+static int cf_check null_init(struct scheduler *ops)
{
struct null_private *prv;
@@ -152,7 +152,7 @@ static int null_init(struct scheduler *ops)
return 0;
}
-static void null_deinit(struct scheduler *ops)
+static void cf_check null_deinit(struct scheduler *ops)
{
xfree(ops->sched_data);
ops->sched_data = NULL;
@@ -166,7 +166,8 @@ static void init_pdata(struct null_private *prv, struct null_pcpu *npc,
npc->unit = NULL;
}
-static void null_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
+static void cf_check null_deinit_pdata(
+ const struct scheduler *ops, void *pcpu, int cpu)
{
struct null_private *prv = null_priv(ops);
struct null_pcpu *npc = pcpu;
@@ -177,7 +178,7 @@ static void null_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
npc->unit = NULL;
}
-static void *null_alloc_pdata(const struct scheduler *ops, int cpu)
+static void *cf_check null_alloc_pdata(const struct scheduler *ops, int cpu)
{
struct null_pcpu *npc;
@@ -188,13 +189,14 @@ static void *null_alloc_pdata(const struct scheduler *ops, int cpu)
return npc;
}
-static void null_free_pdata(const struct scheduler *ops, void *pcpu, int cpu)
+static void cf_check null_free_pdata(
+ const struct scheduler *ops, void *pcpu, int cpu)
{
xfree(pcpu);
}
-static void *null_alloc_udata(const struct scheduler *ops,
- struct sched_unit *unit, void *dd)
+static void *cf_check null_alloc_udata(
+ const struct scheduler *ops, struct sched_unit *unit, void *dd)
{
struct null_unit *nvc;
@@ -210,15 +212,15 @@ static void *null_alloc_udata(const struct scheduler *ops,
return nvc;
}
-static void null_free_udata(const struct scheduler *ops, void *priv)
+static void cf_check null_free_udata(const struct scheduler *ops, void *priv)
{
struct null_unit *nvc = priv;
xfree(nvc);
}
-static void * null_alloc_domdata(const struct scheduler *ops,
- struct domain *d)
+static void *cf_check null_alloc_domdata(
+ const struct scheduler *ops, struct domain *d)
{
struct null_private *prv = null_priv(ops);
struct null_dom *ndom;
@@ -237,7 +239,7 @@ static void * null_alloc_domdata(const struct scheduler *ops,
return ndom;
}
-static void null_free_domdata(const struct scheduler *ops, void *data)
+static void cf_check null_free_domdata(const struct scheduler *ops, void *data)
{
struct null_dom *ndom = data;
struct null_private *prv = null_priv(ops);
@@ -426,9 +428,8 @@ static bool unit_deassign(struct null_private *prv, const struct sched_unit *uni
}
/* Change the scheduler of cpu to us (null). */
-static spinlock_t *null_switch_sched(struct scheduler *new_ops,
- unsigned int cpu,
- void *pdata, void *vdata)
+static spinlock_t *cf_check null_switch_sched(
+ struct scheduler *new_ops, unsigned int cpu, void *pdata, void *vdata)
{
struct sched_resource *sr = get_sched_res(cpu);
struct null_private *prv = null_priv(new_ops);
@@ -450,8 +451,8 @@ static spinlock_t *null_switch_sched(struct scheduler *new_ops,
return &sr->_lock;
}
-static void null_unit_insert(const struct scheduler *ops,
- struct sched_unit *unit)
+static void cf_check null_unit_insert(
+ const struct scheduler *ops, struct sched_unit *unit)
{
struct null_private *prv = null_priv(ops);
struct null_unit *nvc = null_unit(unit);
@@ -516,8 +517,8 @@ static void null_unit_insert(const struct scheduler *ops,
SCHED_STAT_CRANK(unit_insert);
}
-static void null_unit_remove(const struct scheduler *ops,
- struct sched_unit *unit)
+static void cf_check null_unit_remove(
+ const struct scheduler *ops, struct sched_unit *unit)
{
struct null_private *prv = null_priv(ops);
struct null_unit *nvc = null_unit(unit);
@@ -556,8 +557,8 @@ static void null_unit_remove(const struct scheduler *ops,
SCHED_STAT_CRANK(unit_remove);
}
-static void null_unit_wake(const struct scheduler *ops,
- struct sched_unit *unit)
+static void cf_check null_unit_wake(
+ const struct scheduler *ops, struct sched_unit *unit)
{
struct null_private *prv = null_priv(ops);
struct null_unit *nvc = null_unit(unit);
@@ -632,8 +633,8 @@ static void null_unit_wake(const struct scheduler *ops,
cpumask_raise_softirq(cpumask_scratch_cpu(cpu), SCHEDULE_SOFTIRQ);
}
-static void null_unit_sleep(const struct scheduler *ops,
- struct sched_unit *unit)
+static void cf_check null_unit_sleep(
+ const struct scheduler *ops, struct sched_unit *unit)
{
struct null_private *prv = null_priv(ops);
unsigned int cpu = sched_unit_master(unit);
@@ -667,15 +668,15 @@ static void null_unit_sleep(const struct scheduler *ops,
SCHED_STAT_CRANK(unit_sleep);
}
-static struct sched_resource *
+static struct sched_resource *cf_check
null_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
{
ASSERT(!is_idle_unit(unit));
return pick_res(null_priv(ops), unit);
}
-static void null_unit_migrate(const struct scheduler *ops,
- struct sched_unit *unit, unsigned int new_cpu)
+static void cf_check null_unit_migrate(
+ const struct scheduler *ops, struct sched_unit *unit, unsigned int new_cpu)
{
struct null_private *prv = null_priv(ops);
struct null_unit *nvc = null_unit(unit);
@@ -801,8 +802,9 @@ static inline void null_unit_check(struct sched_unit *unit)
* - the unit assigned to the pCPU, if there's one and it can run;
* - the idle unit, otherwise.
*/
-static void null_schedule(const struct scheduler *ops, struct sched_unit *prev,
- s_time_t now, bool tasklet_work_scheduled)
+static void cf_check null_schedule(
+ const struct scheduler *ops, struct sched_unit *prev, s_time_t now,
+ bool tasklet_work_scheduled)
{
unsigned int bs;
const unsigned int cur_cpu = smp_processor_id();
@@ -939,7 +941,7 @@ static inline void dump_unit(const struct null_private *prv,
sched_unit_master(nvc->unit) : -1);
}
-static void null_dump_pcpu(const struct scheduler *ops, int cpu)
+static void cf_check null_dump_pcpu(const struct scheduler *ops, int cpu)
{
struct null_private *prv = null_priv(ops);
const struct null_pcpu *npc = get_sched_res(cpu)->sched_priv;
@@ -968,7 +970,7 @@ static void null_dump_pcpu(const struct scheduler *ops, int cpu)
pcpu_schedule_unlock_irqrestore(lock, flags, cpu);
}
-static void null_dump(const struct scheduler *ops)
+static void cf_check null_dump(const struct scheduler *ops)
{
struct null_private *prv = null_priv(ops);
struct list_head *iter;
diff --git a/xen/common/sched/rt.c b/xen/common/sched/rt.c
index 5ea6f01f263c..d6de25531b3c 100644
--- a/xen/common/sched/rt.c
+++ b/xen/common/sched/rt.c
@@ -269,13 +269,13 @@ unit_on_q(const struct rt_unit *svc)
return !list_empty(&svc->q_elem);
}
-static struct rt_unit *
+static struct rt_unit *cf_check
q_elem(struct list_head *elem)
{
return list_entry(elem, struct rt_unit, q_elem);
}
-static struct rt_unit *
+static struct rt_unit *cf_check
replq_elem(struct list_head *elem)
{
return list_entry(elem, struct rt_unit, replq_elem);
@@ -348,7 +348,7 @@ rt_dump_unit(const struct scheduler *ops, const struct rt_unit *svc)
svc->flags, CPUMASK_PR(mask));
}
-static void
+static void cf_check
rt_dump_pcpu(const struct scheduler *ops, int cpu)
{
struct rt_private *prv = rt_priv(ops);
@@ -366,7 +366,7 @@ rt_dump_pcpu(const struct scheduler *ops, int cpu)
spin_unlock_irqrestore(&prv->lock, flags);
}
-static void
+static void cf_check
rt_dump(const struct scheduler *ops)
{
struct list_head *runq, *depletedq, *replq, *iter;
@@ -636,7 +636,7 @@ replq_reinsert(const struct scheduler *ops, struct rt_unit *svc)
* Valid resource of an unit is intesection of unit's affinity
* and available resources
*/
-static struct sched_resource *
+static struct sched_resource *cf_check
rt_res_pick_locked(const struct sched_unit *unit, unsigned int locked_cpu)
{
cpumask_t *cpus = cpumask_scratch_cpu(locked_cpu);
@@ -659,7 +659,7 @@ rt_res_pick_locked(const struct sched_unit *unit, unsigned int locked_cpu)
* Valid resource of an unit is intesection of unit's affinity
* and available resources
*/
-static struct sched_resource *
+static struct sched_resource *cf_check
rt_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
{
struct sched_resource *res;
@@ -672,7 +672,7 @@ rt_res_pick(const struct scheduler *ops, const struct sched_unit *unit)
/*
* Init/Free related code
*/
-static int
+static int cf_check
rt_init(struct scheduler *ops)
{
int rc = -ENOMEM;
@@ -701,7 +701,7 @@ rt_init(struct scheduler *ops)
return rc;
}
-static void
+static void cf_check
rt_deinit(struct scheduler *ops)
{
struct rt_private *prv = rt_priv(ops);
@@ -714,7 +714,7 @@ rt_deinit(struct scheduler *ops)
}
/* Change the scheduler of cpu to us (RTDS). */
-static spinlock_t *
+static spinlock_t *cf_check
rt_switch_sched(struct scheduler *new_ops, unsigned int cpu,
void *pdata, void *vdata)
{
@@ -750,7 +750,7 @@ rt_switch_sched(struct scheduler *new_ops, unsigned int cpu,
return &prv->lock;
}
-static void
+static void cf_check
rt_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
{
unsigned long flags;
@@ -782,7 +782,7 @@ rt_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu)
spin_unlock_irqrestore(&prv->lock, flags);
}
-static void *
+static void *cf_check
rt_alloc_domdata(const struct scheduler *ops, struct domain *dom)
{
unsigned long flags;
@@ -804,7 +804,7 @@ rt_alloc_domdata(const struct scheduler *ops, struct domain *dom)
return sdom;
}
-static void
+static void cf_check
rt_free_domdata(const struct scheduler *ops, void *data)
{
struct rt_dom *sdom = data;
@@ -822,7 +822,7 @@ rt_free_domdata(const struct scheduler *ops, void *data)
}
}
-static void *
+static void * cf_check
rt_alloc_udata(const struct scheduler *ops, struct sched_unit *unit, void *dd)
{
struct rt_unit *svc;
@@ -850,7 +850,7 @@ rt_alloc_udata(const struct scheduler *ops, struct sched_unit *unit, void *dd)
return svc;
}
-static void
+static void cf_check
rt_free_udata(const struct scheduler *ops, void *priv)
{
struct rt_unit *svc = priv;
@@ -865,7 +865,7 @@ rt_free_udata(const struct scheduler *ops, void *priv)
* It inserts units of moving domain to the scheduler's RunQ in
* dest. cpupool.
*/
-static void
+static void cf_check
rt_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
{
struct rt_unit *svc = rt_unit(unit);
@@ -901,7 +901,7 @@ rt_unit_insert(const struct scheduler *ops, struct sched_unit *unit)
/*
* Remove rt_unit svc from the old scheduler in source cpupool.
*/
-static void
+static void cf_check
rt_unit_remove(const struct scheduler *ops, struct sched_unit *unit)
{
struct rt_unit * const svc = rt_unit(unit);
@@ -1042,7 +1042,7 @@ runq_pick(const struct scheduler *ops, const cpumask_t *mask, unsigned int cpu)
* schedule function for rt scheduler.
* The lock is already grabbed in schedule.c, no need to lock here
*/
-static void
+static void cf_check
rt_schedule(const struct scheduler *ops, struct sched_unit *currunit,
s_time_t now, bool tasklet_work_scheduled)
{
@@ -1129,7 +1129,7 @@ rt_schedule(const struct scheduler *ops, struct sched_unit *currunit,
* Remove UNIT from RunQ
* The lock is already grabbed in schedule.c, no need to lock here
*/
-static void
+static void cf_check
rt_unit_sleep(const struct scheduler *ops, struct sched_unit *unit)
{
struct rt_unit * const svc = rt_unit(unit);
@@ -1244,7 +1244,7 @@ runq_tickle(const struct scheduler *ops, const struct rt_unit *new)
* The lock is already grabbed in schedule.c, no need to lock here
* TODO: what if these two units belongs to the same domain?
*/
-static void
+static void cf_check
rt_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
{
struct rt_unit * const svc = rt_unit(unit);
@@ -1314,7 +1314,7 @@ rt_unit_wake(const struct scheduler *ops, struct sched_unit *unit)
* scurr has finished context switch, insert it back to the RunQ,
* and then pick the highest priority unit from runq to run
*/
-static void
+static void cf_check
rt_context_saved(const struct scheduler *ops, struct sched_unit *unit)
{
struct rt_unit *svc = rt_unit(unit);
@@ -1341,7 +1341,7 @@ rt_context_saved(const struct scheduler *ops, struct sched_unit *unit)
/*
* set/get each unit info of each domain
*/
-static int
+static int cf_check
rt_dom_cntl(
const struct scheduler *ops,
struct domain *d,
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 21/70] xen/evtchn: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (19 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 20/70] xen/sched: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 16:53 ` David Vrabel
2022-02-14 12:50 ` [PATCH v2 22/70] xen/hypfs: " Andrew Cooper
` (50 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/common/event_2l.c | 21 ++++++++++++---------
xen/common/event_channel.c | 3 ++-
xen/common/event_fifo.c | 30 ++++++++++++++++--------------
3 files changed, 30 insertions(+), 24 deletions(-)
diff --git a/xen/common/event_2l.c b/xen/common/event_2l.c
index 7424320e525a..d40dd51ab555 100644
--- a/xen/common/event_2l.c
+++ b/xen/common/event_2l.c
@@ -16,7 +16,8 @@
#include <asm/guest_atomics.h>
-static void evtchn_2l_set_pending(struct vcpu *v, struct evtchn *evtchn)
+static void cf_check evtchn_2l_set_pending(
+ struct vcpu *v, struct evtchn *evtchn)
{
struct domain *d = v->domain;
unsigned int port = evtchn->port;
@@ -41,12 +42,14 @@ static void evtchn_2l_set_pending(struct vcpu *v, struct evtchn *evtchn)
evtchn_check_pollers(d, port);
}
-static void evtchn_2l_clear_pending(struct domain *d, struct evtchn *evtchn)
+static void cf_check evtchn_2l_clear_pending(
+ struct domain *d, struct evtchn *evtchn)
{
guest_clear_bit(d, evtchn->port, &shared_info(d, evtchn_pending));
}
-static void evtchn_2l_unmask(struct domain *d, struct evtchn *evtchn)
+static void cf_check evtchn_2l_unmask(
+ struct domain *d, struct evtchn *evtchn)
{
struct vcpu *v = d->vcpu[evtchn->notify_vcpu_id];
unsigned int port = evtchn->port;
@@ -64,8 +67,8 @@ static void evtchn_2l_unmask(struct domain *d, struct evtchn *evtchn)
}
}
-static bool evtchn_2l_is_pending(const struct domain *d,
- const struct evtchn *evtchn)
+static bool cf_check evtchn_2l_is_pending(
+ const struct domain *d, const struct evtchn *evtchn)
{
evtchn_port_t port = evtchn->port;
unsigned int max_ports = BITS_PER_EVTCHN_WORD(d) * BITS_PER_EVTCHN_WORD(d);
@@ -75,8 +78,8 @@ static bool evtchn_2l_is_pending(const struct domain *d,
guest_test_bit(d, port, &shared_info(d, evtchn_pending)));
}
-static bool evtchn_2l_is_masked(const struct domain *d,
- const struct evtchn *evtchn)
+static bool cf_check evtchn_2l_is_masked(
+ const struct domain *d, const struct evtchn *evtchn)
{
evtchn_port_t port = evtchn->port;
unsigned int max_ports = BITS_PER_EVTCHN_WORD(d) * BITS_PER_EVTCHN_WORD(d);
@@ -86,8 +89,8 @@ static bool evtchn_2l_is_masked(const struct domain *d,
guest_test_bit(d, port, &shared_info(d, evtchn_mask)));
}
-static void evtchn_2l_print_state(struct domain *d,
- const struct evtchn *evtchn)
+static void cf_check evtchn_2l_print_state(
+ struct domain *d, const struct evtchn *evtchn)
{
struct vcpu *v = d->vcpu[evtchn->notify_vcpu_id];
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index 2026bc30dc95..183e78ac17f1 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -99,7 +99,8 @@ static xen_event_channel_notification_t __read_mostly
xen_consumers[NR_XEN_CONSUMERS];
/* Default notification action: wake up from wait_on_xen_event_channel(). */
-static void default_xen_notification_fn(struct vcpu *v, unsigned int port)
+static void cf_check default_xen_notification_fn(
+ struct vcpu *v, unsigned int port)
{
/* Consumer needs notification only if blocked. */
if ( test_and_clear_bit(_VPF_blocked_in_xen, &v->pause_flags) )
diff --git a/xen/common/event_fifo.c b/xen/common/event_fifo.c
index 2fb01b82db84..ed4d3beb10f3 100644
--- a/xen/common/event_fifo.c
+++ b/xen/common/event_fifo.c
@@ -78,7 +78,7 @@ static inline event_word_t *evtchn_fifo_word_from_port(const struct domain *d,
return d->evtchn_fifo->event_array[p] + w;
}
-static void evtchn_fifo_init(struct domain *d, struct evtchn *evtchn)
+static void cf_check evtchn_fifo_init(struct domain *d, struct evtchn *evtchn)
{
event_word_t *word;
@@ -158,7 +158,8 @@ static bool_t evtchn_fifo_set_link(struct domain *d, event_word_t *word,
return 1;
}
-static void evtchn_fifo_set_pending(struct vcpu *v, struct evtchn *evtchn)
+static void cf_check evtchn_fifo_set_pending(
+ struct vcpu *v, struct evtchn *evtchn)
{
struct domain *d = v->domain;
unsigned int port;
@@ -317,7 +318,8 @@ static void evtchn_fifo_set_pending(struct vcpu *v, struct evtchn *evtchn)
evtchn_check_pollers(d, port);
}
-static void evtchn_fifo_clear_pending(struct domain *d, struct evtchn *evtchn)
+static void cf_check evtchn_fifo_clear_pending(
+ struct domain *d, struct evtchn *evtchn)
{
event_word_t *word;
@@ -334,7 +336,7 @@ static void evtchn_fifo_clear_pending(struct domain *d, struct evtchn *evtchn)
guest_clear_bit(d, EVTCHN_FIFO_PENDING, word);
}
-static void evtchn_fifo_unmask(struct domain *d, struct evtchn *evtchn)
+static void cf_check evtchn_fifo_unmask(struct domain *d, struct evtchn *evtchn)
{
struct vcpu *v = d->vcpu[evtchn->notify_vcpu_id];
event_word_t *word;
@@ -350,32 +352,32 @@ static void evtchn_fifo_unmask(struct domain *d, struct evtchn *evtchn)
evtchn_fifo_set_pending(v, evtchn);
}
-static bool evtchn_fifo_is_pending(const struct domain *d,
- const struct evtchn *evtchn)
+static bool cf_check evtchn_fifo_is_pending(
+ const struct domain *d, const struct evtchn *evtchn)
{
const event_word_t *word = evtchn_fifo_word_from_port(d, evtchn->port);
return word && guest_test_bit(d, EVTCHN_FIFO_PENDING, word);
}
-static bool_t evtchn_fifo_is_masked(const struct domain *d,
- const struct evtchn *evtchn)
+static bool cf_check evtchn_fifo_is_masked(
+ const struct domain *d, const struct evtchn *evtchn)
{
const event_word_t *word = evtchn_fifo_word_from_port(d, evtchn->port);
return !word || guest_test_bit(d, EVTCHN_FIFO_MASKED, word);
}
-static bool_t evtchn_fifo_is_busy(const struct domain *d,
- const struct evtchn *evtchn)
+static bool cf_check evtchn_fifo_is_busy(
+ const struct domain *d, const struct evtchn *evtchn)
{
const event_word_t *word = evtchn_fifo_word_from_port(d, evtchn->port);
return word && guest_test_bit(d, EVTCHN_FIFO_LINKED, word);
}
-static int evtchn_fifo_set_priority(struct domain *d, struct evtchn *evtchn,
- unsigned int priority)
+static int cf_check evtchn_fifo_set_priority(
+ struct domain *d, struct evtchn *evtchn, unsigned int priority)
{
if ( priority > EVTCHN_FIFO_PRIORITY_MIN )
return -EINVAL;
@@ -390,8 +392,8 @@ static int evtchn_fifo_set_priority(struct domain *d, struct evtchn *evtchn,
return 0;
}
-static void evtchn_fifo_print_state(struct domain *d,
- const struct evtchn *evtchn)
+static void cf_check evtchn_fifo_print_state(
+ struct domain *d, const struct evtchn *evtchn)
{
event_word_t *word;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 21/70] xen/evtchn: CFI hardening
2022-02-14 12:50 ` [PATCH v2 21/70] xen/evtchn: " Andrew Cooper
@ 2022-02-14 16:53 ` David Vrabel
2022-02-14 16:59 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: David Vrabel @ 2022-02-14 16:53 UTC (permalink / raw)
To: Andrew Cooper, Xen-devel
On 14/02/2022 12:50, Andrew Cooper wrote:
> Control Flow Integrity schemes use toolchain and optionally hardware support
> to help protect against call/jump/return oriented programming attacks.
>
> Use cf_check to annotate function pointer targets for the toolchain.
[...]
> -static void evtchn_2l_set_pending(struct vcpu *v, struct evtchn *evtchn)
> +static void cf_check evtchn_2l_set_pending(
> + struct vcpu *v, struct evtchn *evtchn)
Why manually annotate functions instead of getting the compiler to
automatically work it out?
David
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 21/70] xen/evtchn: CFI hardening
2022-02-14 16:53 ` David Vrabel
@ 2022-02-14 16:59 ` Andrew Cooper
0 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 16:59 UTC (permalink / raw)
To: David Vrabel, Andrew Cooper, Xen-devel
On 14/02/2022 16:53, David Vrabel wrote:
> On 14/02/2022 12:50, Andrew Cooper wrote:
>> Control Flow Integrity schemes use toolchain and optionally hardware
>> support
>> to help protect against call/jump/return oriented programming attacks.
>>
>> Use cf_check to annotate function pointer targets for the toolchain.
> [...]
>> -static void evtchn_2l_set_pending(struct vcpu *v, struct evtchn
>> *evtchn)
>> +static void cf_check evtchn_2l_set_pending(
>> + struct vcpu *v, struct evtchn *evtchn)
>
> Why manually annotate functions instead of getting the compiler to
> automatically work it out?
Because the compilers are not currently capable of working it out
automatically.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 22/70] xen/hypfs: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (20 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 21/70] xen/evtchn: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 23/70] xen/tasklet: " Andrew Cooper
` (49 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Juergen Gross <jgross@suse.com>
---
xen/common/hypfs.c | 57 +++++++++++++++++++++++-----------------------
xen/common/sched/cpupool.c | 25 ++++++++++----------
xen/include/xen/hypfs.h | 49 +++++++++++++++++++--------------------
3 files changed, 65 insertions(+), 66 deletions(-)
diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index 1526bcc52810..0d22396f5dd7 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -113,12 +113,13 @@ static void hypfs_unlock(void)
}
}
-const struct hypfs_entry *hypfs_node_enter(const struct hypfs_entry *entry)
+const struct hypfs_entry *cf_check hypfs_node_enter(
+ const struct hypfs_entry *entry)
{
return entry;
}
-void hypfs_node_exit(const struct hypfs_entry *entry)
+void cf_check hypfs_node_exit(const struct hypfs_entry *entry)
{
}
@@ -289,16 +290,14 @@ static int hypfs_get_path_user(char *buf,
return 0;
}
-struct hypfs_entry *hypfs_leaf_findentry(const struct hypfs_entry_dir *dir,
- const char *name,
- unsigned int name_len)
+struct hypfs_entry *cf_check hypfs_leaf_findentry(
+ const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len)
{
return ERR_PTR(-ENOTDIR);
}
-struct hypfs_entry *hypfs_dir_findentry(const struct hypfs_entry_dir *dir,
- const char *name,
- unsigned int name_len)
+struct hypfs_entry *cf_check hypfs_dir_findentry(
+ const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len)
{
struct hypfs_entry *entry;
@@ -360,7 +359,7 @@ static struct hypfs_entry *hypfs_get_entry(const char *path)
return hypfs_get_entry_rel(&hypfs_root, path + 1);
}
-unsigned int hypfs_getsize(const struct hypfs_entry *entry)
+unsigned int cf_check hypfs_getsize(const struct hypfs_entry *entry)
{
return entry->size;
}
@@ -396,7 +395,7 @@ int hypfs_read_dyndir_id_entry(const struct hypfs_entry_dir *template,
return 0;
}
-static const struct hypfs_entry *hypfs_dyndir_enter(
+static const struct hypfs_entry *cf_check hypfs_dyndir_enter(
const struct hypfs_entry *entry)
{
const struct hypfs_dyndir_id *data;
@@ -407,7 +406,7 @@ static const struct hypfs_entry *hypfs_dyndir_enter(
return data->template->e.funcs->enter(&data->template->e);
}
-static struct hypfs_entry *hypfs_dyndir_findentry(
+static struct hypfs_entry *cf_check hypfs_dyndir_findentry(
const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len)
{
const struct hypfs_dyndir_id *data;
@@ -418,8 +417,8 @@ static struct hypfs_entry *hypfs_dyndir_findentry(
return data->template->e.funcs->findentry(data->template, name, name_len);
}
-static int hypfs_read_dyndir(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr)
+static int cf_check hypfs_read_dyndir(
+ const struct hypfs_entry *entry, XEN_GUEST_HANDLE_PARAM(void) uaddr)
{
const struct hypfs_dyndir_id *data;
@@ -463,8 +462,8 @@ unsigned int hypfs_dynid_entry_size(const struct hypfs_entry *template,
return DIRENTRY_SIZE(snprintf(NULL, 0, template->name, id));
}
-int hypfs_read_dir(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr)
+int cf_check hypfs_read_dir(const struct hypfs_entry *entry,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr)
{
const struct hypfs_entry_dir *d;
const struct hypfs_entry *e;
@@ -510,8 +509,8 @@ int hypfs_read_dir(const struct hypfs_entry *entry,
return 0;
}
-int hypfs_read_leaf(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr)
+int cf_check hypfs_read_leaf(
+ const struct hypfs_entry *entry, XEN_GUEST_HANDLE_PARAM(void) uaddr)
{
const struct hypfs_entry_leaf *l;
unsigned int size = entry->funcs->getsize(entry);
@@ -555,9 +554,9 @@ static int hypfs_read(const struct hypfs_entry *entry,
return ret;
}
-int hypfs_write_leaf(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen)
+int cf_check hypfs_write_leaf(
+ struct hypfs_entry_leaf *leaf, XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen)
{
char *buf;
int ret;
@@ -596,9 +595,9 @@ int hypfs_write_leaf(struct hypfs_entry_leaf *leaf,
return ret;
}
-int hypfs_write_bool(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen)
+int cf_check hypfs_write_bool(
+ struct hypfs_entry_leaf *leaf, XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen)
{
bool buf;
@@ -618,9 +617,9 @@ int hypfs_write_bool(struct hypfs_entry_leaf *leaf,
return 0;
}
-int hypfs_write_custom(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen)
+int cf_check hypfs_write_custom(
+ struct hypfs_entry_leaf *leaf, XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen)
{
struct param_hypfs *p;
char *buf;
@@ -653,9 +652,9 @@ int hypfs_write_custom(struct hypfs_entry_leaf *leaf,
return ret;
}
-int hypfs_write_deny(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen)
+int cf_check hypfs_write_deny(
+ struct hypfs_entry_leaf *leaf, XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen)
{
return -EACCES;
}
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index b9d4babd0d8a..07f984a659cd 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -1026,8 +1026,8 @@ static struct notifier_block cpu_nfb = {
static HYPFS_DIR_INIT(cpupool_pooldir, "%u");
-static int cpupool_dir_read(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr)
+static int cf_check cpupool_dir_read(
+ const struct hypfs_entry *entry, XEN_GUEST_HANDLE_PARAM(void) uaddr)
{
int ret = 0;
struct cpupool *c;
@@ -1050,7 +1050,8 @@ static int cpupool_dir_read(const struct hypfs_entry *entry,
return ret;
}
-static unsigned int cpupool_dir_getsize(const struct hypfs_entry *entry)
+static unsigned int cf_check cpupool_dir_getsize(
+ const struct hypfs_entry *entry)
{
const struct cpupool *c;
unsigned int size = 0;
@@ -1061,7 +1062,7 @@ static unsigned int cpupool_dir_getsize(const struct hypfs_entry *entry)
return size;
}
-static const struct hypfs_entry *cpupool_dir_enter(
+static const struct hypfs_entry *cf_check cpupool_dir_enter(
const struct hypfs_entry *entry)
{
struct hypfs_dyndir_id *data;
@@ -1076,14 +1077,14 @@ static const struct hypfs_entry *cpupool_dir_enter(
return entry;
}
-static void cpupool_dir_exit(const struct hypfs_entry *entry)
+static void cf_check cpupool_dir_exit(const struct hypfs_entry *entry)
{
spin_unlock(&cpupool_lock);
hypfs_free_dyndata();
}
-static struct hypfs_entry *cpupool_dir_findentry(
+static struct hypfs_entry *cf_check cpupool_dir_findentry(
const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len)
{
unsigned long id;
@@ -1102,8 +1103,8 @@ static struct hypfs_entry *cpupool_dir_findentry(
return hypfs_gen_dyndir_id_entry(&cpupool_pooldir, id, cpupool);
}
-static int cpupool_gran_read(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr)
+static int cf_check cpupool_gran_read(
+ const struct hypfs_entry *entry, XEN_GUEST_HANDLE_PARAM(void) uaddr)
{
const struct hypfs_dyndir_id *data;
const struct cpupool *cpupool;
@@ -1121,7 +1122,7 @@ static int cpupool_gran_read(const struct hypfs_entry *entry,
return copy_to_guest(uaddr, gran, strlen(gran) + 1) ? -EFAULT : 0;
}
-static unsigned int hypfs_gran_getsize(const struct hypfs_entry *entry)
+static unsigned int cf_check hypfs_gran_getsize(const struct hypfs_entry *entry)
{
const struct hypfs_dyndir_id *data;
const struct cpupool *cpupool;
@@ -1136,9 +1137,9 @@ static unsigned int hypfs_gran_getsize(const struct hypfs_entry *entry)
return strlen(gran) + 1;
}
-static int cpupool_gran_write(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen)
+static int cf_check cpupool_gran_write(
+ struct hypfs_entry_leaf *leaf, XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen)
{
const struct hypfs_dyndir_id *data;
struct cpupool *cpupool;
diff --git a/xen/include/xen/hypfs.h b/xen/include/xen/hypfs.h
index e9d4c2555bd7..1b65a9188c6c 100644
--- a/xen/include/xen/hypfs.h
+++ b/xen/include/xen/hypfs.h
@@ -168,31 +168,30 @@ void hypfs_add_dyndir(struct hypfs_entry_dir *parent,
struct hypfs_entry_dir *template);
int hypfs_add_leaf(struct hypfs_entry_dir *parent,
struct hypfs_entry_leaf *leaf, bool nofault);
-const struct hypfs_entry *hypfs_node_enter(const struct hypfs_entry *entry);
-void hypfs_node_exit(const struct hypfs_entry *entry);
-int hypfs_read_dir(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr);
-int hypfs_read_leaf(const struct hypfs_entry *entry,
- XEN_GUEST_HANDLE_PARAM(void) uaddr);
-int hypfs_write_deny(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen);
-int hypfs_write_leaf(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen);
-int hypfs_write_bool(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen);
-int hypfs_write_custom(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
- unsigned int ulen);
-unsigned int hypfs_getsize(const struct hypfs_entry *entry);
-struct hypfs_entry *hypfs_leaf_findentry(const struct hypfs_entry_dir *dir,
- const char *name,
- unsigned int name_len);
-struct hypfs_entry *hypfs_dir_findentry(const struct hypfs_entry_dir *dir,
- const char *name,
- unsigned int name_len);
+const struct hypfs_entry *cf_check hypfs_node_enter(
+ const struct hypfs_entry *entry);
+void cf_check hypfs_node_exit(const struct hypfs_entry *entry);
+int cf_check hypfs_read_dir(const struct hypfs_entry *entry,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr);
+int cf_check hypfs_read_leaf(const struct hypfs_entry *entry,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr);
+int cf_check hypfs_write_deny(struct hypfs_entry_leaf *leaf,
+ XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen);
+int cf_check hypfs_write_leaf(struct hypfs_entry_leaf *leaf,
+ XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen);
+int cf_check hypfs_write_bool(struct hypfs_entry_leaf *leaf,
+ XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen);
+int cf_check hypfs_write_custom(struct hypfs_entry_leaf *leaf,
+ XEN_GUEST_HANDLE_PARAM(const_void) uaddr,
+ unsigned int ulen);
+unsigned int cf_check hypfs_getsize(const struct hypfs_entry *entry);
+struct hypfs_entry *cf_check hypfs_leaf_findentry(
+ const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len);
+struct hypfs_entry *cf_check hypfs_dir_findentry(
+ const struct hypfs_entry_dir *dir, const char *name, unsigned int name_len);
void *hypfs_alloc_dyndata(unsigned long size);
#define hypfs_alloc_dyndata(type) ((type *)hypfs_alloc_dyndata(sizeof(type)))
void *hypfs_get_dyndata(void);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 23/70] xen/tasklet: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (21 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 22/70] xen/hypfs: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 24/70] xen/keyhandler: " Andrew Cooper
` (48 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The function pointer cast in hvm_vcpu_initialise() is undefined behaviour.
While it happens to function correctly, it is not compatible with control flow
typechecking, so introduce a new hvm_assert_evtchn_irq_tasklet() to handle the
parameter type conversion in a legal way.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v2:
* Fix typo in commit message.
* Drop changes to arm-only driver.
---
xen/arch/x86/hvm/hvm.c | 7 ++++++-
xen/arch/x86/hvm/vlapic.c | 2 +-
xen/arch/x86/include/asm/shadow.h | 2 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/domain.c | 2 +-
xen/common/keyhandler.c | 6 +++---
xen/common/livepatch.c | 2 +-
xen/common/stop_machine.c | 2 +-
xen/common/trace.c | 2 +-
xen/drivers/char/console.c | 2 +-
xen/drivers/passthrough/amd/iommu_guest.c | 2 +-
xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
xen/drivers/passthrough/vtd/iommu.c | 2 +-
13 files changed, 21 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 0a193530680a..5ec10f30803e 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1528,6 +1528,11 @@ static int __init cf_check hvm_register_CPU_save_and_restore(void)
}
__initcall(hvm_register_CPU_save_and_restore);
+static void cf_check hvm_assert_evtchn_irq_tasklet(void *v)
+{
+ hvm_assert_evtchn_irq(v);
+}
+
int hvm_vcpu_initialise(struct vcpu *v)
{
int rc;
@@ -1552,7 +1557,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
goto fail3;
softirq_tasklet_init(&v->arch.hvm.assert_evtchn_irq_tasklet,
- (void (*)(void *))hvm_assert_evtchn_irq, v);
+ hvm_assert_evtchn_irq_tasklet, v);
v->arch.hvm.inject_event.vector = HVM_EVENT_VECTOR_UNSET;
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index b8c84458ffdc..fe375912bef1 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -312,7 +312,7 @@ static void vlapic_init_sipi_one(struct vcpu *target, uint32_t icr)
vcpu_unpause(target);
}
-static void vlapic_init_sipi_action(void *data)
+static void cf_check vlapic_init_sipi_action(void *data)
{
struct vcpu *origin = data;
uint32_t icr = vcpu_vlapic(origin)->init_sipi.icr;
diff --git a/xen/arch/x86/include/asm/shadow.h b/xen/arch/x86/include/asm/shadow.h
index e25f9604d896..7ef76cc0635b 100644
--- a/xen/arch/x86/include/asm/shadow.h
+++ b/xen/arch/x86/include/asm/shadow.h
@@ -233,7 +233,7 @@ static inline bool pv_l1tf_check_l4e(struct domain *d, l4_pgentry_t l4e)
return pv_l1tf_check_pte(d, 4, l4e.l4);
}
-void pv_l1tf_tasklet(void *data);
+void cf_check pv_l1tf_tasklet(void *data);
static inline void pv_l1tf_domain_init(struct domain *d)
{
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index b8730a9441ce..fb370af90b5c 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -3242,7 +3242,7 @@ void shadow_audit_tables(struct vcpu *v)
#ifdef CONFIG_PV
-void pv_l1tf_tasklet(void *data)
+void cf_check pv_l1tf_tasklet(void *data)
{
struct domain *d = data;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 32ec156e6f6a..a49c26064601 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -1824,7 +1824,7 @@ struct migrate_info {
static DEFINE_PER_CPU(struct migrate_info *, continue_info);
-static void continue_hypercall_tasklet_handler(void *data)
+static void cf_check continue_hypercall_tasklet_handler(void *data)
{
struct migrate_info *info = data;
struct vcpu *v = info->vcpu;
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 2c916d528ab2..b568484ca62c 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -73,7 +73,7 @@ static struct keyhandler {
#undef KEYHANDLER
};
-static void keypress_action(void *unused)
+static void cf_check keypress_action(void *unused)
{
handle_keypress(keypress_key, NULL);
}
@@ -206,7 +206,7 @@ static void dump_registers(unsigned char key, struct cpu_user_regs *regs)
static DECLARE_TASKLET(dump_hwdom_tasklet, NULL, NULL);
-static void dump_hwdom_action(void *data)
+static void cf_check dump_hwdom_action(void *data)
{
struct vcpu *v = data;
@@ -437,7 +437,7 @@ static void read_clocks(unsigned char key)
maxdif_cycles, sumdif_cycles/count, count, dif_cycles);
}
-static void run_all_nonirq_keyhandlers(void *unused)
+static void cf_check run_all_nonirq_keyhandlers(void *unused)
{
/* Fire all the non-IRQ-context diagnostic keyhandlers */
struct keyhandler *h;
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 701efd87a173..4e1c29ab7879 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -1591,7 +1591,7 @@ static int schedule_work(struct payload *data, uint32_t cmd, uint32_t timeout)
return 0;
}
-static void tasklet_fn(void *unused)
+static void cf_check tasklet_fn(void *unused)
{
this_cpu(work_to_do) = 1;
}
diff --git a/xen/common/stop_machine.c b/xen/common/stop_machine.c
index a122bd4afe09..3adbe380de96 100644
--- a/xen/common/stop_machine.c
+++ b/xen/common/stop_machine.c
@@ -141,7 +141,7 @@ int stop_machine_run(int (*fn)(void *), void *data, unsigned int cpu)
return ret;
}
-static void stopmachine_action(void *data)
+static void cf_check stopmachine_action(void *data)
{
unsigned int cpu = (unsigned long)data;
enum stopmachine_state state = STOPMACHINE_START;
diff --git a/xen/common/trace.c b/xen/common/trace.c
index b5358508f804..a7c092fcbbe9 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -663,7 +663,7 @@ static inline void insert_lost_records(struct t_buf *buf)
* Notification is performed in qtasklet to avoid deadlocks with contexts
* which __trace_var() may be called from (e.g., scheduler critical regions).
*/
-static void trace_notify_dom0(void *unused)
+static void cf_check trace_notify_dom0(void *unused)
{
send_global_virq(VIRQ_TBUF);
}
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 4694be83db45..d74b65f6bf23 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -574,7 +574,7 @@ static void serial_rx(char c, struct cpu_user_regs *regs)
__serial_rx(c, regs);
}
-static void notify_dom0_con_ring(void *unused)
+static void cf_check notify_dom0_con_ring(void *unused)
{
send_global_virq(VIRQ_CON_RING);
}
diff --git a/xen/drivers/passthrough/amd/iommu_guest.c b/xen/drivers/passthrough/amd/iommu_guest.c
index 85828490ffee..361ff864d846 100644
--- a/xen/drivers/passthrough/amd/iommu_guest.c
+++ b/xen/drivers/passthrough/amd/iommu_guest.c
@@ -456,7 +456,7 @@ static int do_invalidate_dte(struct domain *d, cmd_entry_t *cmd)
return 0;
}
-static void guest_iommu_process_command(void *data)
+static void cf_check guest_iommu_process_command(void *data)
{
unsigned long opcode, tail, head, cmd_mfn;
cmd_entry_t *cmd;
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index f1ed75558227..34a9e49f1c5a 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -26,7 +26,7 @@
static int __initdata nr_amd_iommus;
static bool __initdata pci_init;
-static void do_amd_iommu_irq(void *data);
+static void cf_check do_amd_iommu_irq(void *data);
static DECLARE_SOFTIRQ_TASKLET(amd_iommu_irq_tasklet, do_amd_iommu_irq, NULL);
unsigned int __read_mostly amd_iommu_acpi_info;
@@ -692,7 +692,7 @@ static void iommu_check_ppr_log(struct amd_iommu *iommu)
spin_unlock_irqrestore(&iommu->lock, flags);
}
-static void do_amd_iommu_irq(void *unused)
+static void cf_check do_amd_iommu_irq(void *unused)
{
struct amd_iommu *iommu;
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 9e85016830bf..7a9c391bec70 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1097,7 +1097,7 @@ static void __do_iommu_page_fault(struct vtd_iommu *iommu)
}
}
-static void do_iommu_page_fault(void *unused)
+static void cf_check do_iommu_page_fault(void *unused)
{
struct acpi_drhd_unit *drhd;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 24/70] xen/keyhandler: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (22 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 23/70] xen/tasklet: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 25/70] xen/vpci: " Andrew Cooper
` (47 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Tweak {IRQ_,}KEYHANDLER() to use a named initialiser instead of requiring a
pointer cast to compile in the IRQ case.
Reposition iommu_dump_page_tables() to avoid a forward declaration.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 2 +-
xen/arch/x86/hvm/irq.c | 2 +-
xen/arch/x86/hvm/svm/vmcb.c | 2 +-
xen/arch/x86/hvm/vmx/vmcs.c | 2 +-
xen/arch/x86/io_apic.c | 2 +-
xen/arch/x86/irq.c | 2 +-
xen/arch/x86/mm/p2m-ept.c | 2 +-
xen/arch/x86/mm/shadow/common.c | 4 +--
xen/arch/x86/msi.c | 2 +-
xen/arch/x86/nmi.c | 4 +--
xen/arch/x86/numa.c | 2 +-
xen/arch/x86/time.c | 2 +-
xen/common/debugtrace.c | 2 +-
xen/common/event_channel.c | 2 +-
xen/common/grant_table.c | 2 +-
xen/common/kexec.c | 2 +-
xen/common/keyhandler.c | 35 ++++++++++++---------
xen/common/livepatch.c | 2 +-
xen/common/page_alloc.c | 4 +--
xen/common/perfc.c | 4 +--
xen/common/sched/cpupool.c | 2 +-
xen/common/spinlock.c | 4 +--
xen/common/timer.c | 2 +-
xen/drivers/char/console.c | 8 ++---
xen/drivers/passthrough/amd/iommu.h | 2 +-
xen/drivers/passthrough/amd/iommu_intr.c | 2 +-
xen/drivers/passthrough/iommu.c | 52 +++++++++++++++-----------------
xen/drivers/passthrough/pci.c | 2 +-
xen/drivers/passthrough/vtd/extern.h | 2 +-
xen/drivers/passthrough/vtd/utils.c | 2 +-
xen/include/xen/perfc.h | 4 +--
xen/include/xen/sched.h | 2 +-
xen/include/xen/spinlock.h | 4 +--
33 files changed, 86 insertions(+), 83 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index 22c8bb0c2d94..0142671bb836 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -377,7 +377,7 @@ static void print_acpi_power(uint32_t cpu, struct acpi_processor_power *power)
print_hw_residencies(cpu);
}
-static void dump_cx(unsigned char key)
+static void cf_check dump_cx(unsigned char key)
{
unsigned int cpu;
diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index 6045c9149bad..a7f8991a7b84 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -635,7 +635,7 @@ static void irq_dump(struct domain *d)
hvm_irq->callback_via_asserted ? "" : " not");
}
-static void dump_irq_info(unsigned char key)
+static void cf_check dump_irq_info(unsigned char key)
{
struct domain *d;
diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c
index efa085032bb5..958309657799 100644
--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -226,7 +226,7 @@ void svm_destroy_vmcb(struct vcpu *v)
svm->vmcb = NULL;
}
-static void vmcb_dump(unsigned char ch)
+static void cf_check vmcb_dump(unsigned char ch)
{
struct domain *d;
struct vcpu *v;
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 2b6bafe9d542..d2cafd8ca1c5 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -2117,7 +2117,7 @@ void vmcs_dump_vcpu(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static void vmcs_dump(unsigned char ch)
+static void cf_check vmcs_dump(unsigned char ch)
{
struct domain *d;
struct vcpu *v;
diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index 4135a9c06052..4c5eaef86273 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -1268,7 +1268,7 @@ static void __init print_IO_APIC(void)
__print_IO_APIC(1);
}
-static void _print_IO_APIC_keyhandler(unsigned char key)
+static void cf_check _print_IO_APIC_keyhandler(unsigned char key)
{
__print_IO_APIC(0);
}
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index d9bd355113d7..f43b926ed26b 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2424,7 +2424,7 @@ void free_domain_pirqs(struct domain *d)
pcidevs_unlock();
}
-static void dump_irqs(unsigned char key)
+static void cf_check dump_irqs(unsigned char key)
{
int i, irq, pirq;
struct irq_desc *desc;
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index b7ee441d4573..a8a6ad629528 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1433,7 +1433,7 @@ static const char *memory_type_to_str(unsigned int x)
return memory_types[x][0] ? memory_types[x] : "?";
}
-static void ept_dump_p2m_table(unsigned char key)
+static void cf_check ept_dump_p2m_table(unsigned char key)
{
struct domain *d;
ept_entry_t *table, *ept_entry;
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index fb370af90b5c..83dedc8870aa 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -98,7 +98,7 @@ void shadow_vcpu_init(struct vcpu *v)
#if SHADOW_AUDIT
int shadow_audit_enable = 0;
-static void shadow_audit_key(unsigned char key)
+static void cf_check shadow_audit_key(unsigned char key)
{
shadow_audit_enable = !shadow_audit_enable;
printk("%s shadow_audit_enable=%d\n",
@@ -1046,7 +1046,7 @@ void shadow_blow_tables_per_domain(struct domain *d)
/* Blow all shadows of all shadowed domains: this can be used to cause the
* guest's pagetables to be re-shadowed if we suspect that the shadows
* have somehow got out of sync */
-static void shadow_blow_all_tables(unsigned char c)
+static void cf_check shadow_blow_all_tables(unsigned char c)
{
struct domain *d;
printk("'%c' pressed -> blowing all shadow tables\n", c);
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index d1497254b188..77a4fbf13f9f 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -1411,7 +1411,7 @@ void __init early_msi_init(void)
return;
}
-static void dump_msi(unsigned char key)
+static void cf_check dump_msi(unsigned char key)
{
unsigned int irq;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index e20d1393c881..5c26e8785dfc 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -578,13 +578,13 @@ void self_nmi(void)
local_irq_restore(flags);
}
-static void do_nmi_trigger(unsigned char key)
+static void cf_check do_nmi_trigger(unsigned char key)
{
printk("Triggering NMI on APIC ID %x\n", get_apic_id());
self_nmi();
}
-static void do_nmi_stats(unsigned char key)
+static void cf_check do_nmi_stats(unsigned char key)
{
const struct vcpu *v;
unsigned int cpu;
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 5de9db4e9943..680b7d9002c5 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -369,7 +369,7 @@ unsigned int __init arch_get_dma_bitsize(void)
+ PAGE_SHIFT, 32);
}
-static void dump_numa(unsigned char key)
+static void cf_check dump_numa(unsigned char key)
{
s_time_t now = NOW();
unsigned int i, j, n;
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 5a72b66800e4..3d767d70f61f 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -2541,7 +2541,7 @@ int tsc_set_info(struct domain *d,
}
/* vtsc may incur measurable performance degradation, diagnose with this */
-static void dump_softtsc(unsigned char key)
+static void cf_check dump_softtsc(unsigned char key)
{
struct domain *d;
int domcnt = 0;
diff --git a/xen/common/debugtrace.c b/xen/common/debugtrace.c
index 160d00b79607..a272e5e43761 100644
--- a/xen/common/debugtrace.c
+++ b/xen/common/debugtrace.c
@@ -233,7 +233,7 @@ void debugtrace_printk(const char *fmt, ...)
spin_unlock_irqrestore(&debugtrace_lock, flags);
}
-static void debugtrace_key(unsigned char key)
+static void cf_check debugtrace_key(unsigned char key)
{
debugtrace_toggle();
}
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index 183e78ac17f1..ffb042a241a6 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -1629,7 +1629,7 @@ static void domain_dump_evtchn_info(struct domain *d)
spin_unlock(&d->event_lock);
}
-static void dump_evtchn_info(unsigned char key)
+static void cf_check dump_evtchn_info(unsigned char key)
{
struct domain *d;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 1078e3e16cda..1e0762b064d7 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -4275,7 +4275,7 @@ static void gnttab_usage_print(struct domain *rd)
printk("no active grant table entries\n");
}
-static void gnttab_usage_print_all(unsigned char key)
+static void cf_check gnttab_usage_print_all(unsigned char key)
{
struct domain *d;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index b222a5fd782e..a2ffb6530cff 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -408,7 +408,7 @@ static long cf_check kexec_reboot(void *_image)
return 0;
}
-static void do_crashdump_trigger(unsigned char key)
+static void cf_check do_crashdump_trigger(unsigned char key)
{
printk("'%c' pressed -> triggering crashdump\n", key);
kexec_crash(CRASHREASON_DEBUGKEY);
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index b568484ca62c..5dc650a37c5c 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -26,10 +26,11 @@
static unsigned char keypress_key;
static bool_t alt_key_handling;
-static keyhandler_fn_t show_handlers, dump_hwdom_registers,
- dump_domains, read_clocks;
-static irq_keyhandler_fn_t do_toggle_alt_key, dump_registers,
- reboot_machine, run_all_keyhandlers, do_debug_key;
+static keyhandler_fn_t cf_check show_handlers, cf_check dump_hwdom_registers,
+ cf_check dump_domains, cf_check read_clocks;
+static irq_keyhandler_fn_t cf_check do_toggle_alt_key, cf_check dump_registers,
+ cf_check reboot_machine, cf_check run_all_keyhandlers,
+ cf_check do_debug_key;
static struct keyhandler {
union {
@@ -43,10 +44,10 @@ static struct keyhandler {
} key_table[128] __read_mostly =
{
#define KEYHANDLER(k, f, desc, diag) \
- [k] = { { (f) }, desc, 0, diag }
+ [k] = { { .fn = (f) }, desc, 0, diag }
#define IRQ_KEYHANDLER(k, f, desc, diag) \
- [k] = { { (keyhandler_fn_t *)(f) }, desc, 1, diag }
+ [k] = { { .irq_fn = (f) }, desc, 1, diag }
IRQ_KEYHANDLER('A', do_toggle_alt_key, "toggle alternative key handling", 0),
IRQ_KEYHANDLER('d', dump_registers, "dump registers", 1),
@@ -124,7 +125,7 @@ void register_irq_keyhandler(unsigned char key, irq_keyhandler_fn_t fn,
key_table[key].diagnostic = diagnostic;
}
-static void show_handlers(unsigned char key)
+static void cf_check show_handlers(unsigned char key)
{
unsigned int i;
@@ -170,7 +171,8 @@ void dump_execstate(struct cpu_user_regs *regs)
watchdog_enable();
}
-static void dump_registers(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check dump_registers(
+ unsigned char key, struct cpu_user_regs *regs)
{
unsigned int cpu;
@@ -224,7 +226,7 @@ static void cf_check dump_hwdom_action(void *data)
}
}
-static void dump_hwdom_registers(unsigned char key)
+static void cf_check dump_hwdom_registers(unsigned char key)
{
struct vcpu *v;
@@ -246,13 +248,14 @@ static void dump_hwdom_registers(unsigned char key)
}
}
-static void reboot_machine(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check reboot_machine(
+ unsigned char key, struct cpu_user_regs *regs)
{
printk("'%c' pressed -> rebooting machine\n", key);
machine_restart(0);
}
-static void dump_domains(unsigned char key)
+static void cf_check dump_domains(unsigned char key)
{
struct domain *d;
const struct sched_unit *unit;
@@ -372,7 +375,7 @@ static void cf_check read_clocks_slave(void *unused)
local_irq_enable();
}
-static void read_clocks(unsigned char key)
+static void cf_check read_clocks(unsigned char key)
{
unsigned int cpu = smp_processor_id(), min_stime_cpu, max_stime_cpu;
unsigned int min_cycles_cpu, max_cycles_cpu;
@@ -461,7 +464,8 @@ static void cf_check run_all_nonirq_keyhandlers(void *unused)
static DECLARE_TASKLET(run_all_keyhandlers_tasklet,
run_all_nonirq_keyhandlers, NULL);
-static void run_all_keyhandlers(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check run_all_keyhandlers(
+ unsigned char key, struct cpu_user_regs *regs)
{
struct keyhandler *h;
unsigned int k;
@@ -494,7 +498,7 @@ static void do_debugger_trap_fatal(struct cpu_user_regs *regs)
barrier();
}
-static void do_debug_key(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_debug_key(unsigned char key, struct cpu_user_regs *regs)
{
printk("'%c' pressed -> trapping into debugger\n", key);
if ( regs )
@@ -503,7 +507,8 @@ static void do_debug_key(unsigned char key, struct cpu_user_regs *regs)
run_in_exception_handler(do_debugger_trap_fatal);
}
-static void do_toggle_alt_key(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_alt_key(
+ unsigned char key, struct cpu_user_regs *regs)
{
alt_key_handling = !alt_key_handling;
printk("'%c' pressed -> using %s key handling\n", key,
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 4e1c29ab7879..e8714920dc8f 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -2068,7 +2068,7 @@ static const char *state2str(unsigned int state)
return names[state];
}
-static void livepatch_printall(unsigned char key)
+static void cf_check livepatch_printall(unsigned char key)
{
struct payload *data;
const void *binary_id = NULL;
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 827617502e35..3caf5c954b24 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2522,7 +2522,7 @@ unsigned long avail_node_heap_pages(unsigned int nodeid)
}
-static void pagealloc_info(unsigned char key)
+static void cf_check pagealloc_info(unsigned char key)
{
unsigned int zone = MEMZONE_XEN;
unsigned long n, total = 0;
@@ -2572,7 +2572,7 @@ void scrub_one_page(struct page_info *pg)
#endif
}
-static void dump_heap(unsigned char key)
+static void cf_check dump_heap(unsigned char key)
{
s_time_t now = NOW();
int i, j;
diff --git a/xen/common/perfc.c b/xen/common/perfc.c
index 3abe35892a83..7400667bf0c4 100644
--- a/xen/common/perfc.c
+++ b/xen/common/perfc.c
@@ -28,7 +28,7 @@ static const struct {
DEFINE_PER_CPU(perfc_t[NUM_PERFCOUNTERS], perfcounters);
-void perfc_printall(unsigned char key)
+void cf_check perfc_printall(unsigned char key)
{
unsigned int i, j;
s_time_t now = NOW();
@@ -115,7 +115,7 @@ void perfc_printall(unsigned char key)
}
}
-void perfc_reset(unsigned char key)
+void cf_check perfc_reset(unsigned char key)
{
unsigned int i, j;
s_time_t now = NOW();
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 07f984a659cd..a6da4970506a 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -956,7 +956,7 @@ const cpumask_t *cpupool_valid_cpus(const struct cpupool *pool)
return pool->cpu_valid;
}
-void dump_runq(unsigned char key)
+void cf_check dump_runq(unsigned char key)
{
s_time_t now = NOW();
struct cpupool *c;
diff --git a/xen/common/spinlock.c b/xen/common/spinlock.c
index 5ce7e3363863..25bfbf3c47f7 100644
--- a/xen/common/spinlock.c
+++ b/xen/common/spinlock.c
@@ -393,7 +393,7 @@ static void spinlock_profile_print_elem(struct lock_profile *data,
data->lock_cnt, data->time_hold, data->block_cnt, data->time_block);
}
-void spinlock_profile_printall(unsigned char key)
+void cf_check spinlock_profile_printall(unsigned char key)
{
s_time_t now = NOW();
s_time_t diff;
@@ -413,7 +413,7 @@ static void spinlock_profile_reset_elem(struct lock_profile *data,
data->time_block = 0;
}
-void spinlock_profile_reset(unsigned char key)
+void cf_check spinlock_profile_reset(unsigned char key)
{
s_time_t now = NOW();
diff --git a/xen/common/timer.c b/xen/common/timer.c
index 700f191a70b4..9b5016d5ed82 100644
--- a/xen/common/timer.c
+++ b/xen/common/timer.c
@@ -546,7 +546,7 @@ static void dump_timer(struct timer *t, s_time_t now)
(t->expires - now) / 1000, t, t->function, t->data);
}
-static void dump_timerq(unsigned char key)
+static void cf_check dump_timerq(unsigned char key)
{
struct timer *t;
struct timers *ts;
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index d74b65f6bf23..380765ab02fd 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -280,7 +280,7 @@ static int *__read_mostly upper_thresh_adj = &xenlog_upper_thresh;
static int *__read_mostly lower_thresh_adj = &xenlog_lower_thresh;
static const char *__read_mostly thresh_adj = "standard";
-static void do_toggle_guest(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_guest(unsigned char key, struct cpu_user_regs *regs)
{
if ( upper_thresh_adj == &xenlog_upper_thresh )
{
@@ -307,13 +307,13 @@ static void do_adj_thresh(unsigned char key)
loglvl_str(*upper_thresh_adj));
}
-static void do_inc_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_inc_thresh(unsigned char key, struct cpu_user_regs *regs)
{
++*lower_thresh_adj;
do_adj_thresh(key);
}
-static void do_dec_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_dec_thresh(unsigned char key, struct cpu_user_regs *regs)
{
if ( *lower_thresh_adj )
--*lower_thresh_adj;
@@ -424,7 +424,7 @@ void console_serial_puts(const char *s, size_t nr)
pv_console_puts(s, nr);
}
-static void dump_console_ring_key(unsigned char key)
+static void cf_check dump_console_ring_key(unsigned char key)
{
uint32_t idx, len, sofar, c;
unsigned int order;
diff --git a/xen/drivers/passthrough/amd/iommu.h b/xen/drivers/passthrough/amd/iommu.h
index 04517c1a024c..99be9aafcc53 100644
--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -297,7 +297,7 @@ unsigned int amd_iommu_read_ioapic_from_ire(
int amd_iommu_msi_msg_update_ire(
struct msi_desc *msi_desc, struct msi_msg *msg);
int amd_setup_hpet_msi(struct msi_desc *msi_desc);
-void amd_iommu_dump_intremap_tables(unsigned char key);
+void cf_check amd_iommu_dump_intremap_tables(unsigned char key);
extern struct ioapic_sbdf {
u16 bdf, seg;
diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthrough/amd/iommu_intr.c
index b166a0466654..e7804413c7f4 100644
--- a/xen/drivers/passthrough/amd/iommu_intr.c
+++ b/xen/drivers/passthrough/amd/iommu_intr.c
@@ -844,7 +844,7 @@ static int dump_intremap_mapping(const struct amd_iommu *iommu,
return 0;
}
-void amd_iommu_dump_intremap_tables(unsigned char key)
+void cf_check amd_iommu_dump_intremap_tables(unsigned char key)
{
if ( !shared_intremap_table )
{
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 6ee267d2bfd4..e220fea72c2f 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -22,8 +22,6 @@
#include <xen/keyhandler.h>
#include <xsm/xsm.h>
-static void iommu_dump_page_tables(unsigned char key);
-
unsigned int __read_mostly iommu_dev_iotlb_timeout = 1000;
integer_param("iommu_dev_iotlb_timeout", iommu_dev_iotlb_timeout);
@@ -224,6 +222,31 @@ int iommu_domain_init(struct domain *d, unsigned int opts)
return 0;
}
+static void cf_check iommu_dump_page_tables(unsigned char key)
+{
+ struct domain *d;
+
+ ASSERT(iommu_enabled);
+
+ rcu_read_lock(&domlist_read_lock);
+
+ for_each_domain(d)
+ {
+ if ( is_hardware_domain(d) || !is_iommu_enabled(d) )
+ continue;
+
+ if ( iommu_use_hap_pt(d) )
+ {
+ printk("%pd sharing page tables\n", d);
+ continue;
+ }
+
+ iommu_vcall(dom_iommu(d)->platform_ops, dump_page_tables, d);
+ }
+
+ rcu_read_unlock(&domlist_read_lock);
+}
+
void __hwdom_init iommu_hwdom_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -584,31 +607,6 @@ bool_t iommu_has_feature(struct domain *d, enum iommu_feature feature)
return is_iommu_enabled(d) && test_bit(feature, dom_iommu(d)->features);
}
-static void iommu_dump_page_tables(unsigned char key)
-{
- struct domain *d;
-
- ASSERT(iommu_enabled);
-
- rcu_read_lock(&domlist_read_lock);
-
- for_each_domain(d)
- {
- if ( is_hardware_domain(d) || !is_iommu_enabled(d) )
- continue;
-
- if ( iommu_use_hap_pt(d) )
- {
- printk("%pd sharing page tables\n", d);
- continue;
- }
-
- iommu_vcall(dom_iommu(d)->platform_ops, dump_page_tables, d);
- }
-
- rcu_read_unlock(&domlist_read_lock);
-}
-
/*
* Local variables:
* mode: C
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index c170dd06ba51..79f49f0e6007 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1351,7 +1351,7 @@ static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
return 0;
}
-static void dump_pci_devices(unsigned char ch)
+static void cf_check dump_pci_devices(unsigned char ch)
{
printk("==== PCI devices ====\n");
pcidevs_lock();
diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h
index f97883a780a2..4595b65e5bce 100644
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -31,7 +31,7 @@ extern const struct iommu_init_ops intel_iommu_init_ops;
void print_iommu_regs(struct acpi_drhd_unit *drhd);
void print_vtd_entries(struct vtd_iommu *iommu, int bus, int devfn, u64 gmfn);
-keyhandler_fn_t vtd_dump_iommu_info;
+keyhandler_fn_t cf_check vtd_dump_iommu_info;
bool intel_iommu_supports_eim(void);
int intel_iommu_enable_eim(void);
diff --git a/xen/drivers/passthrough/vtd/utils.c b/xen/drivers/passthrough/vtd/utils.c
index 56dfdff9bd0d..47922dc8e813 100644
--- a/xen/drivers/passthrough/vtd/utils.c
+++ b/xen/drivers/passthrough/vtd/utils.c
@@ -154,7 +154,7 @@ void print_vtd_entries(struct vtd_iommu *iommu, int bus, int devfn, u64 gmfn)
} while ( --level );
}
-void vtd_dump_iommu_info(unsigned char key)
+void cf_check vtd_dump_iommu_info(unsigned char key)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
diff --git a/xen/include/xen/perfc.h b/xen/include/xen/perfc.h
index 6846e7119fa3..bb010b0aae93 100644
--- a/xen/include/xen/perfc.h
+++ b/xen/include/xen/perfc.h
@@ -96,8 +96,8 @@ DECLARE_PER_CPU(perfc_t[NUM_PERFCOUNTERS], perfcounters);
struct xen_sysctl_perfc_op;
int perfc_control(struct xen_sysctl_perfc_op *);
-extern void perfc_printall(unsigned char key);
-extern void perfc_reset(unsigned char key);
+extern void cf_check perfc_printall(unsigned char key);
+extern void cf_check perfc_reset(unsigned char key);
#else /* CONFIG_PERF_COUNTERS */
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 37f78cc4c4c9..67b0f6b86853 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -1145,7 +1145,7 @@ int cpupool_move_domain(struct domain *d, struct cpupool *c);
int cpupool_do_sysctl(struct xen_sysctl_cpupool_op *op);
unsigned int cpupool_get_id(const struct domain *d);
const cpumask_t *cpupool_valid_cpus(const struct cpupool *pool);
-extern void dump_runq(unsigned char key);
+extern void cf_check dump_runq(unsigned char key);
void arch_do_physinfo(struct xen_sysctl_physinfo *pi);
diff --git a/xen/include/xen/spinlock.h b/xen/include/xen/spinlock.h
index 9fa4e600c1f7..961891bea4d5 100644
--- a/xen/include/xen/spinlock.h
+++ b/xen/include/xen/spinlock.h
@@ -129,8 +129,8 @@ void _lock_profile_deregister_struct(int32_t, struct lock_profile_qhead *);
_lock_profile_deregister_struct(type, &((ptr)->profile_head))
extern int spinlock_profile_control(struct xen_sysctl_lockprof_op *pc);
-extern void spinlock_profile_printall(unsigned char key);
-extern void spinlock_profile_reset(unsigned char key);
+extern void cf_check spinlock_profile_printall(unsigned char key);
+extern void cf_check spinlock_profile_reset(unsigned char key);
#else
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 25/70] xen/vpci: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (23 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 24/70] xen/keyhandler: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 26/70] xen/decompress: " Andrew Cooper
` (46 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/drivers/vpci/header.c | 18 +++++++++---------
xen/drivers/vpci/msi.c | 42 +++++++++++++++++++++---------------------
xen/drivers/vpci/msix.c | 20 ++++++++++----------
xen/drivers/vpci/vpci.c | 16 ++++++++--------
xen/include/xen/vpci.h | 8 ++++----
5 files changed, 52 insertions(+), 52 deletions(-)
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 40ff79c33f8f..a1c928a0d26f 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -33,8 +33,8 @@ struct map_data {
bool map;
};
-static int map_range(unsigned long s, unsigned long e, void *data,
- unsigned long *c)
+static int cf_check map_range(
+ unsigned long s, unsigned long e, void *data, unsigned long *c)
{
const struct map_data *map = data;
int rc;
@@ -332,8 +332,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
return 0;
}
-static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t cmd, void *data)
+static void cf_check cmd_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t cmd, void *data)
{
uint16_t current_cmd = pci_conf_read16(pdev->sbdf, reg);
@@ -353,8 +353,8 @@ static void cmd_write(const struct pci_dev *pdev, unsigned int reg,
pci_conf_write16(pdev->sbdf, reg, cmd);
}
-static void bar_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check bar_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_bar *bar = data;
bool hi = false;
@@ -397,8 +397,8 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg,
pci_conf_write32(pdev->sbdf, reg, val);
}
-static void rom_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check rom_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_header *header = &pdev->vpci->header;
struct vpci_bar *rom = data;
@@ -445,7 +445,7 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg,
rom->addr = val & PCI_ROM_ADDRESS_MASK;
}
-static int init_bars(struct pci_dev *pdev)
+static int cf_check init_bars(struct pci_dev *pdev)
{
uint16_t cmd;
uint64_t addr, size;
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index 5757a7aed20f..8f2b59e61aa4 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -22,8 +22,8 @@
#include <asm/msi.h>
-static uint32_t control_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check control_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msi *msi = data;
@@ -34,8 +34,8 @@ static uint32_t control_read(const struct pci_dev *pdev, unsigned int reg,
(msi->address64 ? PCI_MSI_FLAGS_64BIT : 0);
}
-static void control_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check control_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msi *msi = data;
unsigned int vectors = min_t(uint8_t,
@@ -89,16 +89,16 @@ static void update_msi(const struct pci_dev *pdev, struct vpci_msi *msi)
}
/* Handlers for the address field (32bit or low part of a 64bit address). */
-static uint32_t address_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check address_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msi *msi = data;
return msi->address;
}
-static void address_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check address_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msi *msi = data;
@@ -110,16 +110,16 @@ static void address_write(const struct pci_dev *pdev, unsigned int reg,
}
/* Handlers for the high part of a 64bit address field. */
-static uint32_t address_hi_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check address_hi_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msi *msi = data;
return msi->address >> 32;
}
-static void address_hi_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check address_hi_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msi *msi = data;
@@ -131,16 +131,16 @@ static void address_hi_write(const struct pci_dev *pdev, unsigned int reg,
}
/* Handlers for the data field. */
-static uint32_t data_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check data_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msi *msi = data;
return msi->data;
}
-static void data_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check data_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msi *msi = data;
@@ -150,16 +150,16 @@ static void data_write(const struct pci_dev *pdev, unsigned int reg,
}
/* Handlers for the MSI mask bits. */
-static uint32_t mask_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check mask_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msi *msi = data;
return msi->mask;
}
-static void mask_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check mask_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msi *msi = data;
uint32_t dmask = msi->mask ^ val;
@@ -182,7 +182,7 @@ static void mask_write(const struct pci_dev *pdev, unsigned int reg,
msi->mask = val;
}
-static int init_msi(struct pci_dev *pdev)
+static int cf_check init_msi(struct pci_dev *pdev)
{
uint8_t slot = PCI_SLOT(pdev->devfn), func = PCI_FUNC(pdev->devfn);
unsigned int pos = pci_find_cap_offset(pdev->seg, pdev->bus, slot, func,
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index 846f1b8d7038..2ab4079412f2 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -27,8 +27,8 @@
((addr) >= vmsix_table_addr(vpci, nr) && \
(addr) < vmsix_table_addr(vpci, nr) + vmsix_table_size(vpci, nr))
-static uint32_t control_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check control_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
const struct vpci_msix *msix = data;
@@ -65,8 +65,8 @@ static void update_entry(struct vpci_msix_entry *entry,
entry->updated = false;
}
-static void control_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check control_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
struct vpci_msix *msix = data;
bool new_masked = val & PCI_MSIX_FLAGS_MASKALL;
@@ -156,7 +156,7 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
return NULL;
}
-static int msix_accept(struct vcpu *v, unsigned long addr)
+static int cf_check msix_accept(struct vcpu *v, unsigned long addr)
{
return !!msix_find(v->domain, addr);
}
@@ -182,8 +182,8 @@ static struct vpci_msix_entry *get_entry(struct vpci_msix *msix,
return &msix->entries[(addr - start) / PCI_MSIX_ENTRY_SIZE];
}
-static int msix_read(struct vcpu *v, unsigned long addr, unsigned int len,
- unsigned long *data)
+static int cf_check msix_read(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
{
const struct domain *d = v->domain;
struct vpci_msix *msix = msix_find(d, addr);
@@ -259,8 +259,8 @@ static int msix_read(struct vcpu *v, unsigned long addr, unsigned int len,
return X86EMUL_OKAY;
}
-static int msix_write(struct vcpu *v, unsigned long addr, unsigned int len,
- unsigned long data)
+static int cf_check msix_write(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
{
const struct domain *d = v->domain;
struct vpci_msix *msix = msix_find(d, addr);
@@ -428,7 +428,7 @@ int vpci_make_msix_hole(const struct pci_dev *pdev)
return 0;
}
-static int init_msix(struct pci_dev *pdev)
+static int cf_check init_msix(struct pci_dev *pdev)
{
struct domain *d = pdev->domain;
uint8_t slot = PCI_SLOT(pdev->devfn), func = PCI_FUNC(pdev->devfn);
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index fb0947179b79..f3b32d66cb35 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -106,25 +106,25 @@ static int vpci_register_cmp(const struct vpci_register *r1,
}
/* Dummy hooks, writes are ignored, reads return 1's */
-static uint32_t vpci_ignored_read(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+static uint32_t cf_check vpci_ignored_read(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
return ~(uint32_t)0;
}
-static void vpci_ignored_write(const struct pci_dev *pdev, unsigned int reg,
- uint32_t val, void *data)
+static void cf_check vpci_ignored_write(
+ const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
{
}
-uint32_t vpci_hw_read16(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+uint32_t cf_check vpci_hw_read16(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
return pci_conf_read16(pdev->sbdf, reg);
}
-uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg,
- void *data)
+uint32_t cf_check vpci_hw_read32(
+ const struct pci_dev *pdev, unsigned int reg, void *data)
{
return pci_conf_read32(pdev->sbdf, reg);
}
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index e8ac1eb39513..bcad1516aee1 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -46,10 +46,10 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
uint32_t data);
/* Passthrough handlers. */
-uint32_t vpci_hw_read16(const struct pci_dev *pdev, unsigned int reg,
- void *data);
-uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg,
- void *data);
+uint32_t cf_check vpci_hw_read16(
+ const struct pci_dev *pdev, unsigned int reg, void *data);
+uint32_t cf_check vpci_hw_read32(
+ const struct pci_dev *pdev, unsigned int reg, void *data);
/*
* Check for pending vPCI operations on this vcpu. Returns true if the vcpu
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 26/70] xen/decompress: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (24 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 25/70] xen/vpci: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 27/70] xen/iommu: " Andrew Cooper
` (45 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/common/bunzip2.c | 2 +-
xen/common/decompress.c | 2 +-
xen/common/unlzma.c | 2 +-
xen/common/zstd/zstd_common.c | 4 ++--
xen/common/zstd/zstd_internal.h | 4 ++--
5 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/xen/common/bunzip2.c b/xen/common/bunzip2.c
index 2087cfbbedc8..782b589a8b01 100644
--- a/xen/common/bunzip2.c
+++ b/xen/common/bunzip2.c
@@ -607,7 +607,7 @@ static int __init read_bunzip(struct bunzip_data *bd, unsigned char *outbuf, int
goto decode_next_byte;
}
-static int __init nofill(void *buf, unsigned int len)
+static int __init cf_check nofill(void *buf, unsigned int len)
{
return -1;
}
diff --git a/xen/common/decompress.c b/xen/common/decompress.c
index 79e60f4802d5..989336983f61 100644
--- a/xen/common/decompress.c
+++ b/xen/common/decompress.c
@@ -3,7 +3,7 @@
#include <xen/string.h>
#include <xen/decompress.h>
-static void __init error(const char *msg)
+static void __init cf_check error(const char *msg)
{
printk("%s\n", msg);
}
diff --git a/xen/common/unlzma.c b/xen/common/unlzma.c
index d0ef78eef0d1..6cd99023adf6 100644
--- a/xen/common/unlzma.c
+++ b/xen/common/unlzma.c
@@ -76,7 +76,7 @@ struct rc {
#define RC_MODEL_TOTAL_BITS 11
-static int __init nofill(void *buffer, unsigned int len)
+static int __init cf_check nofill(void *buffer, unsigned int len)
{
return -1;
}
diff --git a/xen/common/zstd/zstd_common.c b/xen/common/zstd/zstd_common.c
index 9a85e938cdd6..5c44e5db7671 100644
--- a/xen/common/zstd/zstd_common.c
+++ b/xen/common/zstd/zstd_common.c
@@ -54,12 +54,12 @@ void *__init ZSTD_stackAllocAll(void *opaque, size_t *size)
return stack_push(stack, *size);
}
-void *__init ZSTD_stackAlloc(void *opaque, size_t size)
+void *__init cf_check ZSTD_stackAlloc(void *opaque, size_t size)
{
ZSTD_stack *stack = (ZSTD_stack *)opaque;
return stack_push(stack, size);
}
-void __init ZSTD_stackFree(void *opaque, void *address)
+void __init cf_check ZSTD_stackFree(void *opaque, void *address)
{
(void)opaque;
(void)address;
diff --git a/xen/common/zstd/zstd_internal.h b/xen/common/zstd/zstd_internal.h
index b7dd14f6ce79..94f8c586220e 100644
--- a/xen/common/zstd/zstd_internal.h
+++ b/xen/common/zstd/zstd_internal.h
@@ -351,8 +351,8 @@ typedef struct {
ZSTD_customMem ZSTD_initStack(void *workspace, size_t workspaceSize);
void *ZSTD_stackAllocAll(void *opaque, size_t *size);
-void *ZSTD_stackAlloc(void *opaque, size_t size);
-void ZSTD_stackFree(void *opaque, void *address);
+void *cf_check ZSTD_stackAlloc(void *opaque, size_t size);
+void cf_check ZSTD_stackFree(void *opaque, void *address);
/*====== common function ======*/
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 27/70] xen/iommu: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (25 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 26/70] xen/decompress: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 28/70] xen/video: " Andrew Cooper
` (44 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
AMD's parse_ppr_log_entry() has no external callers, so becomes static.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/common/compat/memory.c | 4 +-
xen/drivers/passthrough/amd/iommu.h | 41 ++++++++--------
xen/drivers/passthrough/amd/iommu_init.c | 22 ++++-----
xen/drivers/passthrough/amd/iommu_intr.c | 18 +++----
xen/drivers/passthrough/amd/iommu_map.c | 22 +++++----
xen/drivers/passthrough/amd/pci_amd_iommu.c | 32 ++++++------
xen/drivers/passthrough/pci.c | 7 +--
xen/drivers/passthrough/vtd/dmar.c | 3 +-
xen/drivers/passthrough/vtd/extern.h | 36 +++++++-------
xen/drivers/passthrough/vtd/intremap.c | 14 +++---
xen/drivers/passthrough/vtd/iommu.c | 75 +++++++++++++++--------------
xen/drivers/passthrough/vtd/qinval.c | 28 +++++------
12 files changed, 153 insertions(+), 149 deletions(-)
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index ec8ba54bb66e..077ded4a754c 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -23,8 +23,8 @@ struct get_reserved_device_memory {
unsigned int used_entries;
};
-static int get_reserved_device_memory(xen_pfn_t start, xen_ulong_t nr,
- u32 id, void *ctxt)
+static int cf_check get_reserved_device_memory(
+ xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
{
struct get_reserved_device_memory *grdm = ctxt;
uint32_t sbdf = PCI_SBDF3(grdm->map.dev.pci.seg, grdm->map.dev.pci.bus,
diff --git a/xen/drivers/passthrough/amd/iommu.h b/xen/drivers/passthrough/amd/iommu.h
index 99be9aafcc53..03811fedea57 100644
--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -236,25 +236,26 @@ int amd_iommu_init_late(void);
int amd_iommu_update_ivrs_mapping_acpi(void);
int cf_check iov_adjust_irq_affinities(void);
-int amd_iommu_quarantine_init(struct domain *d);
+int cf_check amd_iommu_quarantine_init(struct domain *d);
/* mapping functions */
-int __must_check amd_iommu_map_page(struct domain *d, dfn_t dfn,
- mfn_t mfn, unsigned int flags,
- unsigned int *flush_flags);
-int __must_check amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags);
+int __must_check cf_check amd_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags);
+int __must_check cf_check amd_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags);
int __must_check amd_iommu_alloc_root(struct domain *d);
int amd_iommu_reserve_domain_unity_map(struct domain *domain,
const struct ivrs_unity_map *map,
unsigned int flag);
int amd_iommu_reserve_domain_unity_unmap(struct domain *d,
const struct ivrs_unity_map *map);
-int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt);
-int __must_check amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags);
-int __must_check amd_iommu_flush_iotlb_all(struct domain *d);
+int cf_check amd_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt);
+int __must_check cf_check amd_iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags);
+int __must_check cf_check amd_iommu_flush_iotlb_all(struct domain *d);
/* device table functions */
int get_dma_requestor_id(uint16_t seg, uint16_t bdf);
@@ -282,21 +283,21 @@ void amd_iommu_flush_all_caches(struct amd_iommu *iommu);
struct amd_iommu *find_iommu_for_device(int seg, int bdf);
/* interrupt remapping */
-bool iov_supports_xt(void);
+bool cf_check iov_supports_xt(void);
int amd_iommu_setup_ioapic_remapping(void);
void *amd_iommu_alloc_intremap_table(
const struct amd_iommu *, unsigned long **, unsigned int nr);
-int amd_iommu_free_intremap_table(
+int cf_check amd_iommu_free_intremap_table(
const struct amd_iommu *, struct ivrs_mappings *, uint16_t);
unsigned int amd_iommu_intremap_table_order(
const void *irt, const struct amd_iommu *iommu);
-void amd_iommu_ioapic_update_ire(
+void cf_check amd_iommu_ioapic_update_ire(
unsigned int apic, unsigned int reg, unsigned int value);
-unsigned int amd_iommu_read_ioapic_from_ire(
+unsigned int cf_check amd_iommu_read_ioapic_from_ire(
unsigned int apic, unsigned int reg);
-int amd_iommu_msi_msg_update_ire(
+int cf_check amd_iommu_msi_msg_update_ire(
struct msi_desc *msi_desc, struct msi_msg *msg);
-int amd_setup_hpet_msi(struct msi_desc *msi_desc);
+int cf_check amd_setup_hpet_msi(struct msi_desc *msi_desc);
void cf_check amd_iommu_dump_intremap_tables(unsigned char key);
extern struct ioapic_sbdf {
@@ -327,9 +328,9 @@ extern void *shared_intremap_table;
extern unsigned long *shared_intremap_inuse;
/* power management support */
-void amd_iommu_resume(void);
-int __must_check amd_iommu_suspend(void);
-void amd_iommu_crash_shutdown(void);
+void cf_check amd_iommu_resume(void);
+int __must_check cf_check amd_iommu_suspend(void);
+void cf_check amd_iommu_crash_shutdown(void);
/* guest iommu support */
#ifdef CONFIG_HVM
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 34a9e49f1c5a..06b4d2b1fea0 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -258,8 +258,8 @@ static void register_iommu_exclusion_range(struct amd_iommu *iommu)
writel(entry, iommu->mmio_base+IOMMU_EXCLUSION_BASE_LOW_OFFSET);
}
-static void set_iommu_event_log_control(struct amd_iommu *iommu,
- bool enable)
+static void cf_check set_iommu_event_log_control(
+ struct amd_iommu *iommu, bool enable)
{
/* Reset head and tail pointer manually before enablement */
if ( enable )
@@ -275,8 +275,8 @@ static void set_iommu_event_log_control(struct amd_iommu *iommu,
writeq(iommu->ctrl.raw, iommu->mmio_base + IOMMU_CONTROL_MMIO_OFFSET);
}
-static void set_iommu_ppr_log_control(struct amd_iommu *iommu,
- bool enable)
+static void cf_check set_iommu_ppr_log_control(
+ struct amd_iommu *iommu, bool enable)
{
/* Reset head and tail pointer manually before enablement */
if ( enable )
@@ -527,7 +527,7 @@ static hw_irq_controller iommu_x2apic_type = {
.set_affinity = set_x2apic_affinity,
};
-static void parse_event_log_entry(struct amd_iommu *iommu, u32 entry[])
+static void cf_check parse_event_log_entry(struct amd_iommu *iommu, u32 entry[])
{
u32 code;
static const char *const event_str[] = {
@@ -628,7 +628,7 @@ static void iommu_check_event_log(struct amd_iommu *iommu)
spin_unlock_irqrestore(&iommu->lock, flags);
}
-void parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[])
+static void cf_check parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[])
{
u16 device_id;
@@ -1243,7 +1243,7 @@ static int __init alloc_ivrs_mappings(u16 seg)
return 0;
}
-static int __init amd_iommu_setup_device_table(
+static int __init cf_check amd_iommu_setup_device_table(
u16 seg, struct ivrs_mappings *ivrs_mappings)
{
struct amd_iommu_dte *dt = IVRS_MAPPINGS_DEVTAB(ivrs_mappings);
@@ -1543,7 +1543,7 @@ static void invalidate_all_domain_pages(void)
amd_iommu_flush_all_pages(d);
}
-static int _invalidate_all_devices(
+static int cf_check _invalidate_all_devices(
u16 seg, struct ivrs_mappings *ivrs_mappings)
{
unsigned int bdf;
@@ -1569,14 +1569,14 @@ static void invalidate_all_devices(void)
iterate_ivrs_mappings(_invalidate_all_devices);
}
-int amd_iommu_suspend(void)
+int cf_check amd_iommu_suspend(void)
{
amd_iommu_crash_shutdown();
return 0;
}
-void amd_iommu_crash_shutdown(void)
+void cf_check amd_iommu_crash_shutdown(void)
{
struct amd_iommu *iommu;
@@ -1584,7 +1584,7 @@ void amd_iommu_crash_shutdown(void)
disable_iommu(iommu);
}
-void amd_iommu_resume(void)
+void cf_check amd_iommu_resume(void)
{
struct amd_iommu *iommu;
diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthrough/amd/iommu_intr.c
index e7804413c7f4..cebf9ceca74e 100644
--- a/xen/drivers/passthrough/amd/iommu_intr.c
+++ b/xen/drivers/passthrough/amd/iommu_intr.c
@@ -349,7 +349,7 @@ static int update_intremap_entry_from_ioapic(
return 0;
}
-void amd_iommu_ioapic_update_ire(
+void cf_check amd_iommu_ioapic_update_ire(
unsigned int apic, unsigned int reg, unsigned int value)
{
struct IO_APIC_route_entry old_rte = { 0 };
@@ -455,7 +455,7 @@ void amd_iommu_ioapic_update_ire(
}
}
-unsigned int amd_iommu_read_ioapic_from_ire(
+unsigned int cf_check amd_iommu_read_ioapic_from_ire(
unsigned int apic, unsigned int reg)
{
unsigned int idx;
@@ -608,7 +608,7 @@ static struct amd_iommu *_find_iommu_for_device(int seg, int bdf)
return ERR_PTR(-EINVAL);
}
-int amd_iommu_msi_msg_update_ire(
+int cf_check amd_iommu_msi_msg_update_ire(
struct msi_desc *msi_desc, struct msi_msg *msg)
{
struct pci_dev *pdev = msi_desc->dev;
@@ -653,7 +653,7 @@ int amd_iommu_msi_msg_update_ire(
return rc;
}
-int amd_iommu_free_intremap_table(
+int cf_check amd_iommu_free_intremap_table(
const struct amd_iommu *iommu, struct ivrs_mappings *ivrs_mapping,
uint16_t bdf)
{
@@ -727,7 +727,7 @@ void *amd_iommu_alloc_intremap_table(
return tb;
}
-bool __init iov_supports_xt(void)
+bool __init cf_check iov_supports_xt(void)
{
unsigned int apic;
@@ -756,7 +756,7 @@ bool __init iov_supports_xt(void)
return true;
}
-int __init amd_setup_hpet_msi(struct msi_desc *msi_desc)
+int __init cf_check amd_setup_hpet_msi(struct msi_desc *msi_desc)
{
const struct amd_iommu *iommu;
spinlock_t *lock;
@@ -826,9 +826,9 @@ static void dump_intremap_table(const struct amd_iommu *iommu,
}
}
-static int dump_intremap_mapping(const struct amd_iommu *iommu,
- struct ivrs_mappings *ivrs_mapping,
- uint16_t unused)
+static int cf_check dump_intremap_mapping(
+ const struct amd_iommu *iommu, struct ivrs_mappings *ivrs_mapping,
+ uint16_t unused)
{
unsigned long flags;
diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c
index b0330157eab5..bf5df5fe5d9a 100644
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -276,8 +276,9 @@ static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
return 0;
}
-int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
- unsigned int flags, unsigned int *flush_flags)
+int cf_check amd_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags)
{
struct domain_iommu *hd = dom_iommu(d);
int rc;
@@ -326,8 +327,8 @@ int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
return 0;
}
-int amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags)
+int cf_check amd_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags)
{
unsigned long pt_mfn = 0;
struct domain_iommu *hd = dom_iommu(d);
@@ -370,9 +371,9 @@ static unsigned long flush_count(unsigned long dfn, unsigned long page_count,
return end - start;
}
-int amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags)
+int cf_check amd_iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags)
{
unsigned long dfn_l = dfn_x(dfn);
@@ -410,7 +411,7 @@ int amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
return 0;
}
-int amd_iommu_flush_iotlb_all(struct domain *d)
+int cf_check amd_iommu_flush_iotlb_all(struct domain *d)
{
amd_iommu_flush_all_pages(d);
@@ -462,7 +463,8 @@ int amd_iommu_reserve_domain_unity_unmap(struct domain *d,
return rc;
}
-int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
+int cf_check amd_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt)
{
unsigned int seg = 0 /* XXX */, bdf;
const struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(seg);
@@ -537,7 +539,7 @@ int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
return 0;
}
-int __init amd_iommu_quarantine_init(struct domain *d)
+int __init cf_check amd_iommu_quarantine_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
unsigned long end_gfn =
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 9642bba43a26..e57f555d00d1 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -197,7 +197,7 @@ int __init acpi_ivrs_init(void)
return 0;
}
-static int __init iov_detect(void)
+static int __init cf_check iov_detect(void)
{
if ( !iommu_enable && !iommu_intremap )
return 0;
@@ -217,7 +217,7 @@ static int __init iov_detect(void)
return 0;
}
-static int iov_enable_xt(void)
+static int cf_check iov_enable_xt(void)
{
int rc;
@@ -253,7 +253,7 @@ int amd_iommu_alloc_root(struct domain *d)
unsigned int __read_mostly amd_iommu_max_paging_mode = 6;
int __read_mostly amd_iommu_min_paging_mode = 1;
-static int amd_iommu_domain_init(struct domain *d)
+static int cf_check amd_iommu_domain_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -275,9 +275,9 @@ static int amd_iommu_domain_init(struct domain *d)
return 0;
}
-static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
+static int cf_check amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
-static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
+static void __hwdom_init cf_check amd_iommu_hwdom_init(struct domain *d)
{
const struct amd_iommu *iommu;
@@ -350,8 +350,9 @@ static void amd_iommu_disable_domain_device(const struct domain *domain,
spin_unlock_irqrestore(&iommu->lock, flags);
}
-static int reassign_device(struct domain *source, struct domain *target,
- u8 devfn, struct pci_dev *pdev)
+static int cf_check reassign_device(
+ struct domain *source, struct domain *target, u8 devfn,
+ struct pci_dev *pdev)
{
struct amd_iommu *iommu;
int bdf, rc;
@@ -404,9 +405,8 @@ static int reassign_device(struct domain *source, struct domain *target,
return 0;
}
-static int amd_iommu_assign_device(struct domain *d, u8 devfn,
- struct pci_dev *pdev,
- u32 flag)
+static int cf_check amd_iommu_assign_device(
+ struct domain *d, u8 devfn, struct pci_dev *pdev, u32 flag)
{
struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(pdev->seg);
int bdf = PCI_BDF2(pdev->bus, devfn);
@@ -435,7 +435,7 @@ static int amd_iommu_assign_device(struct domain *d, u8 devfn,
return rc;
}
-static void amd_iommu_clear_root_pgtable(struct domain *d)
+static void cf_check amd_iommu_clear_root_pgtable(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -444,13 +444,13 @@ static void amd_iommu_clear_root_pgtable(struct domain *d)
spin_unlock(&hd->arch.mapping_lock);
}
-static void amd_iommu_domain_destroy(struct domain *d)
+static void cf_check amd_iommu_domain_destroy(struct domain *d)
{
iommu_identity_map_teardown(d);
ASSERT(!dom_iommu(d)->arch.amd.root_table);
}
-static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
{
struct amd_iommu *iommu;
u16 bdf;
@@ -525,7 +525,7 @@ static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
return amd_iommu_setup_domain_device(pdev->domain, iommu, devfn, pdev);
}
-static int amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
{
struct amd_iommu *iommu;
u16 bdf;
@@ -562,7 +562,7 @@ static int amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
return 0;
}
-static int amd_iommu_group_id(u16 seg, u8 bus, u8 devfn)
+static int cf_check amd_iommu_group_id(u16 seg, u8 bus, u8 devfn)
{
int bdf = PCI_BDF2(bus, devfn);
@@ -616,7 +616,7 @@ static void amd_dump_page_table_level(struct page_info *pg, int level,
unmap_domain_page(table_vaddr);
}
-static void amd_dump_page_tables(struct domain *d)
+static void cf_check amd_dump_page_tables(struct domain *d)
{
const struct domain_iommu *hd = dom_iommu(d);
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 79f49f0e6007..af1a7f9fab6d 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1098,7 +1098,7 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn)
* scan pci devices to add all existed PCI devices to alldevs_list,
* and setup pci hierarchy in array bus2bridge.
*/
-static int __init _scan_pci_devices(struct pci_seg *pseg, void *arg)
+static int __init cf_check _scan_pci_devices(struct pci_seg *pseg, void *arg)
{
struct pci_dev *pdev;
int bus, dev, func;
@@ -1176,7 +1176,8 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
ctxt->d->domain_id, err);
}
-static int __hwdom_init _setup_hwdom_pci_devices(struct pci_seg *pseg, void *arg)
+static int __hwdom_init cf_check _setup_hwdom_pci_devices(
+ struct pci_seg *pseg, void *arg)
{
struct setup_hwdom *ctxt = arg;
int bus, devfn;
@@ -1333,7 +1334,7 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev)
}
#endif
-static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
+static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg)
{
struct pci_dev *pdev;
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index b8e91f5be1ae..63f8642e126a 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -1046,7 +1046,8 @@ bool_t __init platform_supports_x2apic(void)
return cpu_has_x2apic && ((dmar_flags & mask) == ACPI_DMAR_INTR_REMAP);
}
-int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
+int cf_check intel_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt)
{
struct acpi_rmrr_unit *rmrr, *rmrr_cur = NULL;
unsigned int i;
diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h
index 4595b65e5bce..bf625fd4540e 100644
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -33,9 +33,9 @@ void print_iommu_regs(struct acpi_drhd_unit *drhd);
void print_vtd_entries(struct vtd_iommu *iommu, int bus, int devfn, u64 gmfn);
keyhandler_fn_t cf_check vtd_dump_iommu_info;
-bool intel_iommu_supports_eim(void);
-int intel_iommu_enable_eim(void);
-void intel_iommu_disable_eim(void);
+bool cf_check intel_iommu_supports_eim(void);
+int cf_check intel_iommu_enable_eim(void);
+void cf_check intel_iommu_disable_eim(void);
int enable_qinval(struct vtd_iommu *iommu);
void disable_qinval(struct vtd_iommu *iommu);
@@ -51,15 +51,13 @@ int iommu_flush_iec_global(struct vtd_iommu *iommu);
int iommu_flush_iec_index(struct vtd_iommu *iommu, u8 im, u16 iidx);
void clear_fault_bits(struct vtd_iommu *iommu);
-int __must_check vtd_flush_context_reg(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id,
- uint8_t function_mask, uint64_t type,
- bool flush_non_present_entry);
-int __must_check vtd_flush_iotlb_reg(struct vtd_iommu *iommu, uint16_t did,
- uint64_t addr, unsigned int size_order,
- uint64_t type,
- bool flush_non_present_entry,
- bool flush_dev_iotlb);
+int __must_check cf_check vtd_flush_context_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry);
+int __must_check cf_check vtd_flush_iotlb_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb);
struct vtd_iommu *ioapic_to_iommu(unsigned int apic_id);
struct vtd_iommu *hpet_to_iommu(unsigned int hpet_id);
@@ -89,17 +87,19 @@ int domain_context_mapping_one(struct domain *domain, struct vtd_iommu *iommu,
u8 bus, u8 devfn, const struct pci_dev *);
int domain_context_unmap_one(struct domain *domain, struct vtd_iommu *iommu,
u8 bus, u8 devfn);
-int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt);
+int cf_check intel_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt);
-unsigned int io_apic_read_remap_rte(unsigned int apic, unsigned int reg);
-void io_apic_write_remap_rte(unsigned int apic,
- unsigned int reg, unsigned int value);
+unsigned int cf_check io_apic_read_remap_rte(
+ unsigned int apic, unsigned int reg);
+void cf_check io_apic_write_remap_rte(
+ unsigned int apic, unsigned int reg, unsigned int value);
struct msi_desc;
struct msi_msg;
-int msi_msg_write_remap_rte(struct msi_desc *, struct msi_msg *);
+int cf_check msi_msg_write_remap_rte(struct msi_desc *, struct msi_msg *);
-int intel_setup_hpet_msi(struct msi_desc *);
+int cf_check intel_setup_hpet_msi(struct msi_desc *);
int is_igd_vt_enabled_quirk(void);
bool is_azalia_tlb_enabled(const struct acpi_drhd_unit *);
diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c
index 01152f200664..e6ba89591b6f 100644
--- a/xen/drivers/passthrough/vtd/intremap.c
+++ b/xen/drivers/passthrough/vtd/intremap.c
@@ -142,7 +142,7 @@ static void set_hpet_source_id(unsigned int id, struct iremap_entry *ire)
set_ire_sid(ire, SVT_VERIFY_SID_SQ, SQ_13_IGNORE_3, hpetid_to_bdf(id));
}
-bool __init intel_iommu_supports_eim(void)
+bool __init cf_check intel_iommu_supports_eim(void)
{
struct acpi_drhd_unit *drhd;
unsigned int apic;
@@ -414,7 +414,7 @@ static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu,
return 0;
}
-unsigned int io_apic_read_remap_rte(
+unsigned int cf_check io_apic_read_remap_rte(
unsigned int apic, unsigned int reg)
{
unsigned int ioapic_pin = (reg - 0x10) / 2;
@@ -438,7 +438,7 @@ unsigned int io_apic_read_remap_rte(
return (*(((u32 *)&old_rte) + 0));
}
-void io_apic_write_remap_rte(
+void cf_check io_apic_write_remap_rte(
unsigned int apic, unsigned int reg, unsigned int value)
{
unsigned int ioapic_pin = (reg - 0x10) / 2;
@@ -639,7 +639,7 @@ static int msi_msg_to_remap_entry(
return 0;
}
-int msi_msg_write_remap_rte(
+int cf_check msi_msg_write_remap_rte(
struct msi_desc *msi_desc, struct msi_msg *msg)
{
struct pci_dev *pdev = msi_desc->dev;
@@ -651,7 +651,7 @@ int msi_msg_write_remap_rte(
: -EINVAL;
}
-int __init intel_setup_hpet_msi(struct msi_desc *msi_desc)
+int __init cf_check intel_setup_hpet_msi(struct msi_desc *msi_desc)
{
struct vtd_iommu *iommu = hpet_to_iommu(msi_desc->hpet_id);
unsigned long flags;
@@ -802,7 +802,7 @@ void disable_intremap(struct vtd_iommu *iommu)
* This function is used to enable Interrupt remapping when
* enable x2apic
*/
-int intel_iommu_enable_eim(void)
+int cf_check intel_iommu_enable_eim(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -856,7 +856,7 @@ int intel_iommu_enable_eim(void)
* This function is used to disable Interrupt remapping when
* suspend local apic
*/
-void intel_iommu_disable_eim(void)
+void cf_check intel_iommu_disable_eim(void)
{
struct acpi_drhd_unit *drhd;
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 7a9c391bec70..4d7703dd9a20 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -59,7 +59,7 @@ static unsigned int __read_mostly nr_iommus;
static struct iommu_ops vtd_ops;
static struct tasklet vtd_fault_tasklet;
-static int setup_hwdom_device(u8 devfn, struct pci_dev *);
+static int cf_check setup_hwdom_device(u8 devfn, struct pci_dev *);
static void setup_hwdom_rmrr(struct domain *d);
static bool domid_mapping(const struct vtd_iommu *iommu)
@@ -240,7 +240,7 @@ domid_t did_to_domain_id(const struct vtd_iommu *iommu, unsigned int did)
return iommu->domid_map[did];
}
-static void sync_cache(const void *addr, unsigned int size)
+static void cf_check sync_cache(const void *addr, unsigned int size)
{
static unsigned long clflush_size = 0;
const void *end = addr + size;
@@ -475,9 +475,9 @@ static void iommu_flush_write_buffer(struct vtd_iommu *iommu)
}
/* return value determine if we need a write buffer flush */
-int vtd_flush_context_reg(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id, uint8_t function_mask,
- uint64_t type, bool flush_non_present_entry)
+int cf_check vtd_flush_context_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry)
{
unsigned long flags;
@@ -542,9 +542,10 @@ static int __must_check iommu_flush_context_device(struct vtd_iommu *iommu,
}
/* return value determine if we need a write buffer flush */
-int vtd_flush_iotlb_reg(struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
- unsigned int size_order, uint64_t type,
- bool flush_non_present_entry, bool flush_dev_iotlb)
+int cf_check vtd_flush_iotlb_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb)
{
int tlb_offset = ecap_iotlb_offset(iommu->ecap);
uint64_t val = type | DMA_TLB_IVT;
@@ -752,10 +753,9 @@ static int __must_check iommu_flush_iotlb(struct domain *d, dfn_t dfn,
return ret;
}
-static int __must_check iommu_flush_iotlb_pages(struct domain *d,
- dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags)
+static int __must_check cf_check iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags)
{
ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
ASSERT(flush_flags);
@@ -764,7 +764,7 @@ static int __must_check iommu_flush_iotlb_pages(struct domain *d,
page_count);
}
-static int __must_check iommu_flush_iotlb_all(struct domain *d)
+static int __must_check cf_check iommu_flush_iotlb_all(struct domain *d)
{
return iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
}
@@ -1393,7 +1393,7 @@ void __init iommu_free(struct acpi_drhd_unit *drhd)
agaw = 64; \
agaw; })
-static int intel_iommu_domain_init(struct domain *d)
+static int cf_check intel_iommu_domain_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -1407,7 +1407,7 @@ static int intel_iommu_domain_init(struct domain *d)
return 0;
}
-static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
+static void __hwdom_init cf_check intel_iommu_hwdom_init(struct domain *d)
{
struct acpi_drhd_unit *drhd;
@@ -1856,7 +1856,7 @@ static int domain_context_unmap(struct domain *domain, u8 devfn,
return ret;
}
-static void iommu_clear_root_pgtable(struct domain *d)
+static void cf_check iommu_clear_root_pgtable(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -1865,7 +1865,7 @@ static void iommu_clear_root_pgtable(struct domain *d)
spin_unlock(&hd->arch.mapping_lock);
}
-static void iommu_domain_teardown(struct domain *d)
+static void cf_check iommu_domain_teardown(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
const struct acpi_drhd_unit *drhd;
@@ -1883,9 +1883,9 @@ static void iommu_domain_teardown(struct domain *d)
XFREE(hd->arch.vtd.iommu_bitmap);
}
-static int __must_check intel_iommu_map_page(struct domain *d, dfn_t dfn,
- mfn_t mfn, unsigned int flags,
- unsigned int *flush_flags)
+static int __must_check cf_check intel_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags)
{
struct domain_iommu *hd = dom_iommu(d);
struct dma_pte *page, *pte, old, new = {};
@@ -1954,8 +1954,8 @@ static int __must_check intel_iommu_map_page(struct domain *d, dfn_t dfn,
return rc;
}
-static int __must_check intel_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags)
+static int __must_check cf_check intel_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags)
{
/* Do nothing if VT-d shares EPT page table */
if ( iommu_use_hap_pt(d) )
@@ -1970,8 +1970,8 @@ static int __must_check intel_iommu_unmap_page(struct domain *d, dfn_t dfn,
return 0;
}
-static int intel_iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
- unsigned int *flags)
+static int cf_check intel_iommu_lookup_page(
+ struct domain *d, dfn_t dfn, mfn_t *mfn, unsigned int *flags)
{
struct domain_iommu *hd = dom_iommu(d);
struct dma_pte *page, val;
@@ -2023,7 +2023,7 @@ static int __init vtd_ept_page_compatible(struct vtd_iommu *iommu)
(ept_has_1gb(ept_cap) && opt_hap_1gb) <= cap_sps_1gb(vtd_cap);
}
-static int intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
{
struct acpi_rmrr_unit *rmrr;
u16 bdf;
@@ -2066,7 +2066,7 @@ static int intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
return 0;
}
-static int intel_iommu_enable_device(struct pci_dev *pdev)
+static int cf_check intel_iommu_enable_device(struct pci_dev *pdev)
{
struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
int ret = drhd ? ats_device(pdev, drhd) : -ENODEV;
@@ -2081,7 +2081,7 @@ static int intel_iommu_enable_device(struct pci_dev *pdev)
return ret >= 0 ? 0 : ret;
}
-static int intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
{
struct acpi_rmrr_unit *rmrr;
u16 bdf;
@@ -2108,7 +2108,8 @@ static int intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
return domain_context_unmap(pdev->domain, devfn, pdev);
}
-static int __hwdom_init setup_hwdom_device(u8 devfn, struct pci_dev *pdev)
+static int __hwdom_init cf_check setup_hwdom_device(
+ u8 devfn, struct pci_dev *pdev)
{
return domain_context_mapping(pdev->domain, devfn, pdev);
}
@@ -2314,7 +2315,7 @@ static struct iommu_state {
uint32_t fectl;
} *__read_mostly iommu_state;
-static int __init vtd_setup(void)
+static int __init cf_check vtd_setup(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2449,7 +2450,7 @@ static int __init vtd_setup(void)
return ret;
}
-static int reassign_device_ownership(
+static int cf_check reassign_device_ownership(
struct domain *source,
struct domain *target,
u8 devfn, struct pci_dev *pdev)
@@ -2527,7 +2528,7 @@ static int reassign_device_ownership(
return ret;
}
-static int intel_iommu_assign_device(
+static int cf_check intel_iommu_assign_device(
struct domain *d, u8 devfn, struct pci_dev *pdev, u32 flag)
{
struct domain *s = pdev->domain;
@@ -2609,7 +2610,7 @@ static int intel_iommu_assign_device(
return ret;
}
-static int intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
+static int cf_check intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
{
u8 secbus;
if ( find_upstream_bridge(seg, &bus, &devfn, &secbus) < 0 )
@@ -2618,7 +2619,7 @@ static int intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
return PCI_BDF2(bus, devfn);
}
-static int __must_check vtd_suspend(void)
+static int __must_check cf_check vtd_suspend(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2661,7 +2662,7 @@ static int __must_check vtd_suspend(void)
return 0;
}
-static void vtd_crash_shutdown(void)
+static void cf_check vtd_crash_shutdown(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2682,7 +2683,7 @@ static void vtd_crash_shutdown(void)
}
}
-static void vtd_resume(void)
+static void cf_check vtd_resume(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2760,7 +2761,7 @@ static void vtd_dump_page_table_level(paddr_t pt_maddr, int level, paddr_t gpa,
unmap_vtd_domain_page(pt_vaddr);
}
-static void vtd_dump_page_tables(struct domain *d)
+static void cf_check vtd_dump_page_tables(struct domain *d)
{
const struct domain_iommu *hd = dom_iommu(d);
@@ -2770,7 +2771,7 @@ static void vtd_dump_page_tables(struct domain *d)
agaw_to_level(hd->arch.vtd.agaw), 0, 0);
}
-static int __init intel_iommu_quarantine_init(struct domain *d)
+static int __init cf_check intel_iommu_quarantine_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
struct page_info *pg;
diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c
index 9f291f47e518..beeb65f0deec 100644
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -322,9 +322,9 @@ int iommu_flush_iec_index(struct vtd_iommu *iommu, u8 im, u16 iidx)
return queue_invalidate_iec_sync(iommu, IEC_INDEX_INVL, im, iidx);
}
-static int __must_check flush_context_qi(struct vtd_iommu *iommu, u16 did,
- u16 sid, u8 fm, u64 type,
- bool flush_non_present_entry)
+static int __must_check cf_check flush_context_qi(
+ struct vtd_iommu *iommu, u16 did, u16 sid, u8 fm, u64 type,
+ bool flush_non_present_entry)
{
ASSERT(iommu->qinval_maddr);
@@ -346,11 +346,9 @@ static int __must_check flush_context_qi(struct vtd_iommu *iommu, u16 did,
type >> DMA_CCMD_INVL_GRANU_OFFSET);
}
-static int __must_check flush_iotlb_qi(struct vtd_iommu *iommu, u16 did,
- u64 addr,
- unsigned int size_order, u64 type,
- bool flush_non_present_entry,
- bool flush_dev_iotlb)
+static int __must_check cf_check flush_iotlb_qi(
+ struct vtd_iommu *iommu, u16 did, u64 addr, unsigned int size_order,
+ u64 type, bool flush_non_present_entry, bool flush_dev_iotlb)
{
u8 dr = 0, dw = 0;
int ret = 0, rc;
@@ -461,18 +459,18 @@ int enable_qinval(struct vtd_iommu *iommu)
return 0;
}
-static int vtd_flush_context_noop(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id, uint8_t function_mask,
- uint64_t type, bool flush_non_present_entry)
+static int cf_check vtd_flush_context_noop(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry)
{
WARN();
return -EIO;
}
-static int vtd_flush_iotlb_noop(struct vtd_iommu *iommu, uint16_t did,
- uint64_t addr, unsigned int size_order,
- uint64_t type, bool flush_non_present_entry,
- bool flush_dev_iotlb)
+static int cf_check vtd_flush_iotlb_noop(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb)
{
WARN();
return -EIO;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 28/70] xen/video: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (26 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 27/70] xen/iommu: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 29/70] xen/console: " Andrew Cooper
` (43 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/drivers/video/lfb.c | 4 ++--
xen/drivers/video/lfb.h | 4 ++--
xen/drivers/video/vesa.c | 4 ++--
xen/drivers/video/vga.c | 6 +++---
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/xen/drivers/video/lfb.c b/xen/drivers/video/lfb.c
index 75b749b3303b..48c66f8acf10 100644
--- a/xen/drivers/video/lfb.c
+++ b/xen/drivers/video/lfb.c
@@ -53,7 +53,7 @@ static void lfb_show_line(
}
/* Fast mode which redraws all modified parts of a 2D text buffer. */
-void lfb_redraw_puts(const char *s, size_t nr)
+void cf_check lfb_redraw_puts(const char *s, size_t nr)
{
unsigned int i, min_redraw_y = lfb.ypos;
@@ -98,7 +98,7 @@ void lfb_redraw_puts(const char *s, size_t nr)
}
/* Slower line-based scroll mode which interacts better with dom0. */
-void lfb_scroll_puts(const char *s, size_t nr)
+void cf_check lfb_scroll_puts(const char *s, size_t nr)
{
unsigned int i;
diff --git a/xen/drivers/video/lfb.h b/xen/drivers/video/lfb.h
index e743ccdd6b11..42161402d611 100644
--- a/xen/drivers/video/lfb.h
+++ b/xen/drivers/video/lfb.h
@@ -35,8 +35,8 @@ struct lfb_prop {
unsigned int text_rows;
};
-void lfb_redraw_puts(const char *s, size_t nr);
-void lfb_scroll_puts(const char *s, size_t nr);
+void cf_check lfb_redraw_puts(const char *s, size_t nr);
+void cf_check lfb_scroll_puts(const char *s, size_t nr);
void lfb_carriage_return(void);
void lfb_free(void);
diff --git a/xen/drivers/video/vesa.c b/xen/drivers/video/vesa.c
index cb0e443be4dd..155bc09d3237 100644
--- a/xen/drivers/video/vesa.c
+++ b/xen/drivers/video/vesa.c
@@ -17,7 +17,7 @@
#define vlfb_info vga_console_info.u.vesa_lfb
-static void lfb_flush(void);
+static void cf_check lfb_flush(void);
static unsigned char *lfb;
static const struct font_desc *font;
@@ -177,7 +177,7 @@ void __init vesa_mtrr_init(void)
} while ( (size_total >= PAGE_SIZE) && (rc == -EINVAL) );
}
-static void lfb_flush(void)
+static void cf_check lfb_flush(void)
{
if ( vesa_mtrr == 3 )
__asm__ __volatile__ ("sfence" : : : "memory");
diff --git a/xen/drivers/video/vga.c b/xen/drivers/video/vga.c
index b7f04d0d97f4..abe295e477b1 100644
--- a/xen/drivers/video/vga.c
+++ b/xen/drivers/video/vga.c
@@ -19,8 +19,8 @@ static int vgacon_keep;
static unsigned int xpos, ypos;
static unsigned char *video;
-static void vga_text_puts(const char *s, size_t nr);
-static void vga_noop_puts(const char *s, size_t nr) {}
+static void cf_check vga_text_puts(const char *s, size_t nr);
+static void cf_check vga_noop_puts(const char *s, size_t nr) {}
void (*video_puts)(const char *, size_t nr) = vga_noop_puts;
/*
@@ -175,7 +175,7 @@ void __init video_endboot(void)
}
}
-static void vga_text_puts(const char *s, size_t nr)
+static void cf_check vga_text_puts(const char *s, size_t nr)
{
for ( ; nr > 0; nr--, s++ )
{
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 29/70] xen/console: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (27 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 28/70] xen/video: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 30/70] xen/misc: " Andrew Cooper
` (42 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/drivers/char/console.c | 4 ++--
xen/drivers/char/ehci-dbgp.c | 24 +++++++++++++-----------
xen/drivers/char/ns16550.c | 26 +++++++++++++-------------
3 files changed, 28 insertions(+), 26 deletions(-)
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 380765ab02fd..d9d6556c2293 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -552,7 +552,7 @@ static void __serial_rx(char c, struct cpu_user_regs *regs)
#endif
}
-static void serial_rx(char c, struct cpu_user_regs *regs)
+static void cf_check serial_rx(char c, struct cpu_user_regs *regs)
{
static int switch_code_count = 0;
@@ -1286,7 +1286,7 @@ void panic(const char *fmt, ...)
* **************************************************************
*/
-static void suspend_steal_fn(const char *str, size_t nr) { }
+static void cf_check suspend_steal_fn(const char *str, size_t nr) { }
static int suspend_steal_id;
int console_suspend(void)
diff --git a/xen/drivers/char/ehci-dbgp.c b/xen/drivers/char/ehci-dbgp.c
index a6b57fdf2d19..e205c0da6a61 100644
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1000,13 +1000,15 @@ static int ehci_dbgp_external_startup(struct ehci_dbgp *dbgp)
typedef void (*set_debug_port_t)(struct ehci_dbgp *, unsigned int);
-static void default_set_debug_port(struct ehci_dbgp *dbgp, unsigned int port)
+static void cf_check default_set_debug_port(
+ struct ehci_dbgp *dbgp, unsigned int port)
{
}
static set_debug_port_t __read_mostly set_debug_port = default_set_debug_port;
-static void nvidia_set_debug_port(struct ehci_dbgp *dbgp, unsigned int port)
+static void cf_check nvidia_set_debug_port(
+ struct ehci_dbgp *dbgp, unsigned int port)
{
uint32_t dword = pci_conf_read32(PCI_SBDF(0, dbgp->bus, dbgp->slot,
dbgp->func), 0x74);
@@ -1167,7 +1169,7 @@ static inline void _ehci_dbgp_flush(struct ehci_dbgp *dbgp)
dbgp->out.chunk = 0;
}
-static void ehci_dbgp_flush(struct serial_port *port)
+static void cf_check ehci_dbgp_flush(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
s_time_t goal;
@@ -1196,7 +1198,7 @@ static void ehci_dbgp_flush(struct serial_port *port)
set_timer(&dbgp->timer, goal);
}
-static void ehci_dbgp_putc(struct serial_port *port, char c)
+static void cf_check ehci_dbgp_putc(struct serial_port *port, char c)
{
struct ehci_dbgp *dbgp = port->uart;
@@ -1209,7 +1211,7 @@ static void ehci_dbgp_putc(struct serial_port *port, char c)
ehci_dbgp_flush(port);
}
-static int ehci_dbgp_tx_ready(struct serial_port *port)
+static int cf_check ehci_dbgp_tx_ready(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
@@ -1228,7 +1230,7 @@ static int ehci_dbgp_tx_ready(struct serial_port *port)
(dbgp->state == dbgp_idle) * DBGP_MAX_PACKET;
}
-static int ehci_dbgp_getc(struct serial_port *port, char *pc)
+static int cf_check ehci_dbgp_getc(struct serial_port *port, char *pc)
{
struct ehci_dbgp *dbgp = port->uart;
@@ -1309,7 +1311,7 @@ static bool_t ehci_dbgp_setup_preirq(struct ehci_dbgp *dbgp)
return 0;
}
-static void __init ehci_dbgp_init_preirq(struct serial_port *port)
+static void __init cf_check ehci_dbgp_init_preirq(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
u32 debug_port, offset;
@@ -1358,7 +1360,7 @@ static void ehci_dbgp_setup_postirq(struct ehci_dbgp *dbgp)
set_timer(&dbgp->timer, NOW() + MILLISECS(1));
}
-static void __init ehci_dbgp_init_postirq(struct serial_port *port)
+static void __init cf_check ehci_dbgp_init_postirq(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
@@ -1409,12 +1411,12 @@ static int ehci_dbgp_check_release(struct ehci_dbgp *dbgp)
return 0;
}
-static void __init ehci_dbgp_endboot(struct serial_port *port)
+static void __init cf_check ehci_dbgp_endboot(struct serial_port *port)
{
ehci_dbgp_check_release(port->uart);
}
-static void ehci_dbgp_suspend(struct serial_port *port)
+static void cf_check ehci_dbgp_suspend(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
@@ -1431,7 +1433,7 @@ static void ehci_dbgp_suspend(struct serial_port *port)
dbgp->state = dbgp_unsafe;
}
-static void ehci_dbgp_resume(struct serial_port *port)
+static void cf_check ehci_dbgp_resume(struct serial_port *port)
{
struct ehci_dbgp *dbgp = port->uart;
diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index 990cad39fe85..8df1ee4d5c2c 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -174,7 +174,7 @@ static void handle_dw_usr_busy_quirk(struct ns16550 *uart)
}
}
-static void ns16550_interrupt(
+static void cf_check ns16550_interrupt(
int irq, void *dev_id, struct cpu_user_regs *regs)
{
struct serial_port *port = dev_id;
@@ -239,7 +239,7 @@ static void cf_check ns16550_poll(void *data)
#endif
}
-static int ns16550_tx_ready(struct serial_port *port)
+static int cf_check ns16550_tx_ready(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
@@ -250,13 +250,13 @@ static int ns16550_tx_ready(struct serial_port *port)
uart->lsr_mask ) == uart->lsr_mask ) ? uart->fifo_size : 0;
}
-static void ns16550_putc(struct serial_port *port, char c)
+static void cf_check ns16550_putc(struct serial_port *port, char c)
{
struct ns16550 *uart = port->uart;
ns_write_reg(uart, UART_THR, c);
}
-static int ns16550_getc(struct serial_port *port, char *pc)
+static int cf_check ns16550_getc(struct serial_port *port, char *pc)
{
struct ns16550 *uart = port->uart;
@@ -344,7 +344,7 @@ static void ns16550_setup_preirq(struct ns16550 *uart)
UART_FCR_ENABLE | UART_FCR_CLRX | UART_FCR_CLTX | UART_FCR_TRG14);
}
-static void __init ns16550_init_preirq(struct serial_port *port)
+static void __init cf_check ns16550_init_preirq(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
@@ -373,7 +373,7 @@ static void __init ns16550_init_preirq(struct serial_port *port)
uart->fifo_size = 16;
}
-static void __init ns16550_init_irq(struct serial_port *port)
+static void __init cf_check ns16550_init_irq(struct serial_port *port)
{
#ifdef NS16550_PCI
struct ns16550 *uart = port->uart;
@@ -399,7 +399,7 @@ static void ns16550_setup_postirq(struct ns16550 *uart)
set_timer(&uart->timer, NOW() + MILLISECS(uart->timeout_ms));
}
-static void __init ns16550_init_postirq(struct serial_port *port)
+static void __init cf_check ns16550_init_postirq(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
int rc, bits;
@@ -491,7 +491,7 @@ static void __init ns16550_init_postirq(struct serial_port *port)
ns16550_setup_postirq(uart);
}
-static void ns16550_suspend(struct serial_port *port)
+static void cf_check ns16550_suspend(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
@@ -543,7 +543,7 @@ static void cf_check ns16550_delayed_resume(void *data)
_ns16550_resume(port);
}
-static void ns16550_resume(struct serial_port *port)
+static void cf_check ns16550_resume(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
@@ -569,7 +569,7 @@ static void ns16550_resume(struct serial_port *port)
_ns16550_resume(port);
}
-static void __init ns16550_endboot(struct serial_port *port)
+static void __init cf_check ns16550_endboot(struct serial_port *port)
{
#ifdef CONFIG_HAS_IOPORTS
struct ns16550 *uart = port->uart;
@@ -583,13 +583,13 @@ static void __init ns16550_endboot(struct serial_port *port)
#endif
}
-static int __init ns16550_irq(struct serial_port *port)
+static int __init cf_check ns16550_irq(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
return ((uart->irq > 0) ? uart->irq : -1);
}
-static void ns16550_start_tx(struct serial_port *port)
+static void cf_check ns16550_start_tx(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
u8 ier = ns_read_reg(uart, UART_IER);
@@ -599,7 +599,7 @@ static void ns16550_start_tx(struct serial_port *port)
ns_write_reg(uart, UART_IER, ier | UART_IER_ETHREI);
}
-static void ns16550_stop_tx(struct serial_port *port)
+static void cf_check ns16550_stop_tx(struct serial_port *port)
{
struct ns16550 *uart = port->uart;
u8 ier = ns_read_reg(uart, UART_IER);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 30/70] xen/misc: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (28 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 29/70] xen/console: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 31/70] x86: CFI hardening for request_irq() Andrew Cooper
` (41 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/mm.c | 6 ++++--
xen/arch/x86/setup.c | 4 ++--
xen/common/domain.c | 2 +-
xen/common/gdbstub.c | 5 ++---
xen/common/livepatch.c | 7 +++----
xen/common/memory.c | 4 ++--
xen/common/page_alloc.c | 2 +-
xen/common/radix-tree.c | 4 ++--
xen/common/rangeset.c | 2 +-
xen/common/spinlock.c | 6 +++---
xen/common/vm_event.c | 6 +++---
xen/common/xmalloc_tlsf.c | 4 ++--
xen/drivers/passthrough/amd/iommu_init.c | 2 +-
13 files changed, 27 insertions(+), 27 deletions(-)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 3b8bc3dda977..4b6956c5be78 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -835,7 +835,8 @@ struct mmio_emul_range_ctxt {
unsigned long mfn;
};
-static int print_mmio_emul_range(unsigned long s, unsigned long e, void *arg)
+static int cf_check print_mmio_emul_range(
+ unsigned long s, unsigned long e, void *arg)
{
const struct mmio_emul_range_ctxt *ctxt = arg;
@@ -4606,7 +4607,8 @@ static int _handle_iomem_range(unsigned long s, unsigned long e,
return 0;
}
-static int handle_iomem_range(unsigned long s, unsigned long e, void *p)
+static int cf_check handle_iomem_range(
+ unsigned long s, unsigned long e, void *p)
{
int err = 0;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index eceff0a4e2b4..735f69d2cae8 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -2023,8 +2023,8 @@ int __hwdom_init xen_in_range(unsigned long mfn)
return 0;
}
-static int __hwdom_init io_bitmap_cb(unsigned long s, unsigned long e,
- void *ctx)
+static int __hwdom_init cf_check io_bitmap_cb(
+ unsigned long s, unsigned long e, void *ctx)
{
struct domain *d = ctx;
unsigned int i;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index a49c26064601..a3614539e472 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -370,7 +370,7 @@ static void cf_check _free_pirq_struct(struct rcu_head *head)
xfree(container_of(head, struct pirq, rcu_head));
}
-static void free_pirq_struct(void *ptr)
+static void cf_check free_pirq_struct(void *ptr)
{
struct pirq *pirq = ptr;
diff --git a/xen/common/gdbstub.c b/xen/common/gdbstub.c
index 079c3ca9616a..d6872721dc0d 100644
--- a/xen/common/gdbstub.c
+++ b/xen/common/gdbstub.c
@@ -69,7 +69,7 @@ static void gdb_smp_resume(void);
static char __initdata opt_gdb[30];
string_param("gdb", opt_gdb);
-static void gdbstub_console_puts(const char *str, size_t nr);
+static void cf_check gdbstub_console_puts(const char *str, size_t nr);
/* value <-> char (de)serialzers */
static char
@@ -546,8 +546,7 @@ __gdb_ctx = {
};
static struct gdb_context *gdb_ctx = &__gdb_ctx;
-static void
-gdbstub_console_puts(const char *str, size_t nr)
+static void cf_check gdbstub_console_puts(const char *str, size_t nr)
{
const char *p;
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index e8714920dc8f..ec301a9f120c 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -157,10 +157,9 @@ unsigned long livepatch_symbols_lookup_by_name(const char *symname)
return 0;
}
-static const char *livepatch_symbols_lookup(unsigned long addr,
- unsigned long *symbolsize,
- unsigned long *offset,
- char *namebuf)
+static const char *cf_check livepatch_symbols_lookup(
+ unsigned long addr, unsigned long *symbolsize, unsigned long *offset,
+ char *namebuf)
{
const struct payload *data;
unsigned int i, best;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index ede45c4af9db..69b0cd1e50de 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1051,8 +1051,8 @@ struct get_reserved_device_memory {
unsigned int used_entries;
};
-static int get_reserved_device_memory(xen_pfn_t start, xen_ulong_t nr,
- u32 id, void *ctxt)
+static int cf_check get_reserved_device_memory(
+ xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
{
struct get_reserved_device_memory *grdm = ctxt;
uint32_t sbdf = PCI_SBDF3(grdm->map.dev.pci.seg, grdm->map.dev.pci.bus,
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 3caf5c954b24..46357182375a 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1238,7 +1238,7 @@ struct scrub_wait_state {
bool drop;
};
-static void scrub_continue(void *data)
+static void cf_check scrub_continue(void *data)
{
struct scrub_wait_state *st = data;
diff --git a/xen/common/radix-tree.c b/xen/common/radix-tree.c
index 33b47748ae49..adc3034222dc 100644
--- a/xen/common/radix-tree.c
+++ b/xen/common/radix-tree.c
@@ -52,7 +52,7 @@ struct rcu_node {
struct rcu_head rcu_head;
};
-static struct radix_tree_node *rcu_node_alloc(void *arg)
+static struct radix_tree_node *cf_check rcu_node_alloc(void *arg)
{
struct rcu_node *rcu_node = xmalloc(struct rcu_node);
return rcu_node ? &rcu_node->node : NULL;
@@ -65,7 +65,7 @@ static void cf_check _rcu_node_free(struct rcu_head *head)
xfree(rcu_node);
}
-static void rcu_node_free(struct radix_tree_node *node, void *arg)
+static void cf_check rcu_node_free(struct radix_tree_node *node, void *arg)
{
struct rcu_node *rcu_node = container_of(node, struct rcu_node, node);
call_rcu(&rcu_node->rcu_head, _rcu_node_free);
diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c
index 885b6b15c229..a6ef2640462a 100644
--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -384,7 +384,7 @@ int rangeset_consume_ranges(struct rangeset *r,
return rc;
}
-static int merge(unsigned long s, unsigned long e, void *data)
+static int cf_check merge(unsigned long s, unsigned long e, void *data)
{
struct rangeset *r = data;
diff --git a/xen/common/spinlock.c b/xen/common/spinlock.c
index 25bfbf3c47f7..62c83aaa6a73 100644
--- a/xen/common/spinlock.c
+++ b/xen/common/spinlock.c
@@ -375,7 +375,7 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
spin_unlock(&lock_profile_lock);
}
-static void spinlock_profile_print_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
struct spinlock *lock = data->lock;
@@ -404,7 +404,7 @@ void cf_check spinlock_profile_printall(unsigned char key)
spinlock_profile_iterate(spinlock_profile_print_elem, NULL);
}
-static void spinlock_profile_reset_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_reset_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
data->lock_cnt = 0;
@@ -428,7 +428,7 @@ typedef struct {
int rc;
} spinlock_profile_ucopy_t;
-static void spinlock_profile_ucopy_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_ucopy_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
spinlock_profile_ucopy_t *p = par;
diff --git a/xen/common/vm_event.c b/xen/common/vm_event.c
index 70ab3ba406ff..84cf52636bc4 100644
--- a/xen/common/vm_event.c
+++ b/xen/common/vm_event.c
@@ -523,21 +523,21 @@ int __vm_event_claim_slot(struct domain *d, struct vm_event_domain *ved,
#ifdef CONFIG_MEM_PAGING
/* Registered with Xen-bound event channel for incoming notifications. */
-static void mem_paging_notification(struct vcpu *v, unsigned int port)
+static void cf_check mem_paging_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_paging);
}
#endif
/* Registered with Xen-bound event channel for incoming notifications. */
-static void monitor_notification(struct vcpu *v, unsigned int port)
+static void cf_check monitor_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_monitor);
}
#ifdef CONFIG_MEM_SHARING
/* Registered with Xen-bound event channel for incoming notifications. */
-static void mem_sharing_notification(struct vcpu *v, unsigned int port)
+static void cf_check mem_sharing_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_share);
}
diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c
index e3f6886e6b62..d2ad909502d0 100644
--- a/xen/common/xmalloc_tlsf.c
+++ b/xen/common/xmalloc_tlsf.c
@@ -512,13 +512,13 @@ int xmem_pool_maxalloc(struct xmem_pool *pool)
static struct xmem_pool *xenpool;
-static void *xmalloc_pool_get(unsigned long size)
+static void *cf_check xmalloc_pool_get(unsigned long size)
{
ASSERT(size == PAGE_SIZE);
return alloc_xenheap_page();
}
-static void xmalloc_pool_put(void *p)
+static void cf_check xmalloc_pool_put(void *p)
{
free_xenheap_page(p);
}
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 06b4d2b1fea0..cebcd68a6c04 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1073,7 +1073,7 @@ static void * __init allocate_ppr_log(struct amd_iommu *iommu)
#define IVRS_MAPPINGS_DEVTAB(m) (m)[ivrs_bdf_entries].intremap_table
/* Gets passed to radix_tree_destroy(), so its param needs to be void *. */
-static void __init free_ivrs_mapping_callback(void *ptr)
+static void __init cf_check free_ivrs_mapping_callback(void *ptr)
{
const struct ivrs_mappings *ivrs_mappings = ptr;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 31/70] x86: CFI hardening for request_irq()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (29 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 30/70] xen/misc: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 32/70] x86/hvm: CFI hardening for hvm_funcs Andrew Cooper
` (40 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
... and friends; alloc_direct_apic_vector() and set_direct_apic_vector().
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/apic.c | 8 ++++----
xen/arch/x86/cpu/mcheck/mce_intel.c | 4 ++--
xen/arch/x86/guest/xen/xen.c | 2 +-
xen/arch/x86/hpet.c | 4 ++--
xen/arch/x86/hvm/vmx/vmx.c | 4 ++--
xen/arch/x86/include/asm/irq.h | 16 ++++++++--------
xen/arch/x86/irq.c | 2 +-
xen/arch/x86/smp.c | 6 +++---
xen/arch/x86/time.c | 3 ++-
xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
xen/drivers/passthrough/vtd/iommu.c | 4 ++--
11 files changed, 29 insertions(+), 28 deletions(-)
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 68e4d870c749..5a7a58dc9830 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1361,7 +1361,7 @@ int reprogram_timer(s_time_t timeout)
return apic_tmict || !timeout;
}
-void apic_timer_interrupt(struct cpu_user_regs * regs)
+void cf_check apic_timer_interrupt(struct cpu_user_regs *regs)
{
ack_APIC_irq();
perfc_incr(apic_timer);
@@ -1380,7 +1380,7 @@ void smp_send_state_dump(unsigned int cpu)
/*
* Spurious interrupts should _never_ happen with our APIC/SMP architecture.
*/
-void spurious_interrupt(struct cpu_user_regs *regs)
+void cf_check spurious_interrupt(struct cpu_user_regs *regs)
{
/*
* Check if this is a vectored interrupt (most likely, as this is probably
@@ -1411,7 +1411,7 @@ void spurious_interrupt(struct cpu_user_regs *regs)
* This interrupt should never happen with our APIC/SMP architecture
*/
-void error_interrupt(struct cpu_user_regs *regs)
+void cf_check error_interrupt(struct cpu_user_regs *regs)
{
static const char *const esr_fields[] = {
"Send CS error",
@@ -1444,7 +1444,7 @@ void error_interrupt(struct cpu_user_regs *regs)
* This interrupt handles performance counters interrupt
*/
-void pmu_apic_interrupt(struct cpu_user_regs *regs)
+void cf_check pmu_apic_interrupt(struct cpu_user_regs *regs)
{
ack_APIC_irq();
vpmu_do_interrupt(regs);
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c
index a691e10bdcd6..7aaa56fd02eb 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -55,7 +55,7 @@ bool __read_mostly lmce_support;
#define MCE_RING 0x1
static DEFINE_PER_CPU(int, last_state);
-static void intel_thermal_interrupt(struct cpu_user_regs *regs)
+static void cf_check intel_thermal_interrupt(struct cpu_user_regs *regs)
{
uint64_t msr_content;
unsigned int cpu = smp_processor_id();
@@ -639,7 +639,7 @@ static void cpu_mcheck_disable(void)
clear_cmci();
}
-static void cmci_interrupt(struct cpu_user_regs *regs)
+static void cf_check cmci_interrupt(struct cpu_user_regs *regs)
{
mctelem_cookie_t mctc;
struct mca_summary bs;
diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c
index b2aa3a009b4a..17807cdea688 100644
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -170,7 +170,7 @@ static void __init init_memmap(void)
}
}
-static void xen_evtchn_upcall(struct cpu_user_regs *regs)
+static void cf_check xen_evtchn_upcall(struct cpu_user_regs *regs)
{
struct vcpu_info *vcpu_info = this_cpu(vcpu_info);
unsigned long pending;
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 7b009a930498..c31fd97579dc 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -240,8 +240,8 @@ static void handle_hpet_broadcast(struct hpet_event_channel *ch)
}
}
-static void hpet_interrupt_handler(int irq, void *data,
- struct cpu_user_regs *regs)
+static void cf_check hpet_interrupt_handler(
+ int irq, void *data, struct cpu_user_regs *regs)
{
struct hpet_event_channel *ch = data;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 36c8a12cfe7d..dade08f60279 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2533,7 +2533,7 @@ static struct hvm_function_table __initdata vmx_function_table = {
};
/* Handle VT-d posted-interrupt when VCPU is blocked. */
-static void pi_wakeup_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_wakeup_interrupt(struct cpu_user_regs *regs)
{
struct vmx_vcpu *vmx, *tmp;
spinlock_t *lock = &per_cpu(vmx_pi_blocking, smp_processor_id()).lock;
@@ -2565,7 +2565,7 @@ static void pi_wakeup_interrupt(struct cpu_user_regs *regs)
}
/* Handle VT-d posted-interrupt when VCPU is running. */
-static void pi_notification_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_notification_interrupt(struct cpu_user_regs *regs)
{
ack_APIC_irq();
this_cpu(irq_count)++;
diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index 7c825e9d9c0a..b3f49abc5556 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -93,14 +93,14 @@ static inline struct cpu_user_regs *set_irq_regs(struct cpu_user_regs *new_regs)
#define platform_legacy_irq(irq) ((irq) < 16)
-void event_check_interrupt(struct cpu_user_regs *regs);
-void invalidate_interrupt(struct cpu_user_regs *regs);
-void call_function_interrupt(struct cpu_user_regs *regs);
-void apic_timer_interrupt(struct cpu_user_regs *regs);
-void error_interrupt(struct cpu_user_regs *regs);
-void pmu_apic_interrupt(struct cpu_user_regs *regs);
-void spurious_interrupt(struct cpu_user_regs *regs);
-void irq_move_cleanup_interrupt(struct cpu_user_regs *regs);
+void cf_check event_check_interrupt(struct cpu_user_regs *regs);
+void cf_check invalidate_interrupt(struct cpu_user_regs *regs);
+void cf_check call_function_interrupt(struct cpu_user_regs *regs);
+void cf_check apic_timer_interrupt(struct cpu_user_regs *regs);
+void cf_check error_interrupt(struct cpu_user_regs *regs);
+void cf_check pmu_apic_interrupt(struct cpu_user_regs *regs);
+void cf_check spurious_interrupt(struct cpu_user_regs *regs);
+void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs);
uint8_t alloc_hipriority_vector(void);
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index f43b926ed26b..61e09a356f97 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -744,7 +744,7 @@ void move_native_irq(struct irq_desc *desc)
desc->handler->enable(desc);
}
-void irq_move_cleanup_interrupt(struct cpu_user_regs *regs)
+void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs)
{
unsigned vector, me;
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index b9a696f61963..33748e629a21 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -246,7 +246,7 @@ static cpumask_t flush_cpumask;
static const void *flush_va;
static unsigned int flush_flags;
-void invalidate_interrupt(struct cpu_user_regs *regs)
+void cf_check invalidate_interrupt(struct cpu_user_regs *regs)
{
unsigned int flags = flush_flags;
ack_APIC_irq();
@@ -385,14 +385,14 @@ void smp_send_nmi_allbutself(void)
send_IPI_mask(&cpu_online_map, APIC_DM_NMI);
}
-void event_check_interrupt(struct cpu_user_regs *regs)
+void cf_check event_check_interrupt(struct cpu_user_regs *regs)
{
ack_APIC_irq();
perfc_incr(ipis);
this_cpu(irq_count)++;
}
-void call_function_interrupt(struct cpu_user_regs *regs)
+void cf_check call_function_interrupt(struct cpu_user_regs *regs)
{
ack_APIC_irq();
perfc_incr(ipis);
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 3d767d70f61f..b6e690b76fab 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -196,7 +196,8 @@ static void smp_send_timer_broadcast_ipi(void)
}
}
-static void timer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check timer_interrupt(
+ int irq, void *dev_id, struct cpu_user_regs *regs)
{
ASSERT(local_irq_is_enabled());
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index cebcd68a6c04..d2ad282e93d3 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -715,8 +715,8 @@ static void cf_check do_amd_iommu_irq(void *unused)
}
}
-static void iommu_interrupt_handler(int irq, void *dev_id,
- struct cpu_user_regs *regs)
+static void cf_check iommu_interrupt_handler(
+ int irq, void *dev_id, struct cpu_user_regs *regs)
{
unsigned long flags;
struct amd_iommu *iommu = dev_id;
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 4d7703dd9a20..fc3ff064b692 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1117,8 +1117,8 @@ static void cf_check do_iommu_page_fault(void *unused)
__do_iommu_page_fault(drhd->iommu);
}
-static void iommu_page_fault(int irq, void *dev_id,
- struct cpu_user_regs *regs)
+static void cf_check iommu_page_fault(
+ int irq, void *dev_id, struct cpu_user_regs *regs)
{
/*
* Just flag the tasklet as runnable. This is fine, according to VT-d
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 32/70] x86/hvm: CFI hardening for hvm_funcs
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (30 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 31/70] x86: CFI hardening for request_irq() Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 33/70] x86/hvm: CFI hardening for device emulation Andrew Cooper
` (39 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
In svm.c, make a few rearrangements. svm_update_guest_cr() has no external
callers so can become static, but needs moving along with svm_fpu_enter() to
avoid a forward declaration. Move svm_fpu_leave() too, to match. Also move
svm_update_guest_efer() to drop its forward declaration.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v2:
* Move svm_fpu_leave() too.
---
xen/arch/x86/hvm/svm/nestedsvm.c | 22 +-
xen/arch/x86/hvm/svm/svm.c | 390 ++++++++++++++-------------
xen/arch/x86/hvm/vmx/intr.c | 2 +-
xen/arch/x86/hvm/vmx/vmcs.c | 8 +-
xen/arch/x86/hvm/vmx/vmx.c | 143 +++++-----
xen/arch/x86/hvm/vmx/vvmx.c | 16 +-
xen/arch/x86/include/asm/hvm/svm/nestedsvm.h | 18 +-
xen/arch/x86/include/asm/hvm/svm/svm.h | 1 -
xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 8 +-
xen/arch/x86/include/asm/hvm/vmx/vmx.h | 2 +-
xen/arch/x86/include/asm/hvm/vmx/vvmx.h | 18 +-
11 files changed, 320 insertions(+), 308 deletions(-)
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index abc178d8d482..9f5f35f16aff 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -80,7 +80,7 @@ int nestedsvm_vmcb_map(struct vcpu *v, uint64_t vmcbaddr)
}
/* Interface methods */
-int nsvm_vcpu_initialise(struct vcpu *v)
+int cf_check nsvm_vcpu_initialise(struct vcpu *v)
{
void *msrpm;
struct nestedvcpu *nv = &vcpu_nestedhvm(v);
@@ -110,7 +110,7 @@ int nsvm_vcpu_initialise(struct vcpu *v)
return -ENOMEM;
}
-void nsvm_vcpu_destroy(struct vcpu *v)
+void cf_check nsvm_vcpu_destroy(struct vcpu *v)
{
struct nestedvcpu *nv = &vcpu_nestedhvm(v);
struct nestedsvm *svm = &vcpu_nestedsvm(v);
@@ -150,7 +150,7 @@ void nsvm_vcpu_destroy(struct vcpu *v)
svm->ns_iomap = NULL;
}
-int nsvm_vcpu_reset(struct vcpu *v)
+int cf_check nsvm_vcpu_reset(struct vcpu *v)
{
struct nestedsvm *svm = &vcpu_nestedsvm(v);
@@ -855,8 +855,8 @@ nsvm_vcpu_vmexit_inject(struct vcpu *v, struct cpu_user_regs *regs,
return 0;
}
-int
-nsvm_vcpu_vmexit_event(struct vcpu *v, const struct x86_event *trap)
+int cf_check nsvm_vcpu_vmexit_event(
+ struct vcpu *v, const struct x86_event *trap)
{
ASSERT(vcpu_nestedhvm(v).nv_vvmcx != NULL);
@@ -865,7 +865,7 @@ nsvm_vcpu_vmexit_event(struct vcpu *v, const struct x86_event *trap)
return NESTEDHVM_VMEXIT_DONE;
}
-uint64_t nsvm_vcpu_hostcr3(struct vcpu *v)
+uint64_t cf_check nsvm_vcpu_hostcr3(struct vcpu *v)
{
return vcpu_nestedsvm(v).ns_vmcb_hostcr3;
}
@@ -1030,8 +1030,7 @@ nsvm_vmcb_guest_intercepts_exitcode(struct vcpu *v,
return 1;
}
-bool_t
-nsvm_vmcb_guest_intercepts_event(
+bool cf_check nsvm_vmcb_guest_intercepts_event(
struct vcpu *v, unsigned int vector, int errcode)
{
return nsvm_vmcb_guest_intercepts_exitcode(v,
@@ -1206,8 +1205,7 @@ nsvm_vmcb_prepare4vmexit(struct vcpu *v, struct cpu_user_regs *regs)
return 0;
}
-bool_t
-nsvm_vmcb_hap_enabled(struct vcpu *v)
+bool cf_check nsvm_vmcb_hap_enabled(struct vcpu *v)
{
return vcpu_nestedsvm(v).ns_hap_enabled;
}
@@ -1216,7 +1214,7 @@ nsvm_vmcb_hap_enabled(struct vcpu *v)
* walk is successful, the translated value is returned in
* L1_gpa. The result value tells what to do next.
*/
-int nsvm_hap_walk_L1_p2m(
+int cf_check nsvm_hap_walk_L1_p2m(
struct vcpu *v, paddr_t L2_gpa, paddr_t *L1_gpa, unsigned int *page_order,
uint8_t *p2m_acc, struct npfec npfec)
{
@@ -1241,7 +1239,7 @@ int nsvm_hap_walk_L1_p2m(
return NESTEDHVM_PAGEFAULT_DONE;
}
-enum hvm_intblk nsvm_intr_blocked(struct vcpu *v)
+enum hvm_intblk cf_check nsvm_intr_blocked(struct vcpu *v)
{
struct nestedsvm *svm = &vcpu_nestedsvm(v);
struct nestedvcpu *nv = &vcpu_nestedhvm(v);
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index c4ce3f75ab74..de6166241bf1 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -67,8 +67,6 @@ void noreturn svm_asm_do_resume(void);
u32 svm_feature_flags;
-static void svm_update_guest_efer(struct vcpu *);
-
static struct hvm_function_table svm_function_table;
/*
@@ -122,11 +120,166 @@ void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len)
hvm_inject_hw_exception(TRAP_debug, X86_EVENT_NO_EC);
}
-static void svm_cpu_down(void)
+static void cf_check svm_cpu_down(void)
{
write_efer(read_efer() & ~EFER_SVME);
}
+static void svm_fpu_enter(struct vcpu *v)
+{
+ struct vmcb_struct *n1vmcb = vcpu_nestedhvm(v).nv_n1vmcx;
+
+ vcpu_restore_fpu_lazy(v);
+ vmcb_set_exception_intercepts(
+ n1vmcb,
+ vmcb_get_exception_intercepts(n1vmcb) & ~(1U << TRAP_no_device));
+}
+
+static void cf_check svm_fpu_leave(struct vcpu *v)
+{
+ struct vmcb_struct *n1vmcb = vcpu_nestedhvm(v).nv_n1vmcx;
+
+ ASSERT(!v->fpu_dirtied);
+ ASSERT(read_cr0() & X86_CR0_TS);
+
+ /*
+ * If the guest does not have TS enabled then we must cause and handle an
+ * exception on first use of the FPU. If the guest *does* have TS enabled
+ * then this is not necessary: no FPU activity can occur until the guest
+ * clears CR0.TS, and we will initialise the FPU when that happens.
+ */
+ if ( !(v->arch.hvm.guest_cr[0] & X86_CR0_TS) )
+ {
+ vmcb_set_exception_intercepts(
+ n1vmcb,
+ vmcb_get_exception_intercepts(n1vmcb) | (1U << TRAP_no_device));
+ vmcb_set_cr0(n1vmcb, vmcb_get_cr0(n1vmcb) | X86_CR0_TS);
+ }
+}
+
+static void cf_check svm_update_guest_cr(
+ struct vcpu *v, unsigned int cr, unsigned int flags)
+{
+ struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
+ uint64_t value;
+
+ switch ( cr )
+ {
+ case 0:
+ {
+ unsigned long hw_cr0_mask = 0;
+
+ if ( !(v->arch.hvm.guest_cr[0] & X86_CR0_TS) )
+ {
+ if ( v != current )
+ {
+ if ( !v->arch.fully_eager_fpu )
+ hw_cr0_mask |= X86_CR0_TS;
+ }
+ else if ( vmcb_get_cr0(vmcb) & X86_CR0_TS )
+ svm_fpu_enter(v);
+ }
+
+ if ( paging_mode_hap(v->domain) )
+ {
+ uint32_t intercepts = vmcb_get_cr_intercepts(vmcb);
+
+ /* Trap CR3 updates if CR3 memory events are enabled. */
+ if ( v->domain->arch.monitor.write_ctrlreg_enabled &
+ monitor_ctrlreg_bitmask(VM_EVENT_X86_CR3) )
+ vmcb_set_cr_intercepts(vmcb, intercepts | CR_INTERCEPT_CR3_WRITE);
+ }
+
+ value = v->arch.hvm.guest_cr[0] | hw_cr0_mask;
+ if ( !paging_mode_hap(v->domain) )
+ value |= X86_CR0_PG | X86_CR0_WP;
+ vmcb_set_cr0(vmcb, value);
+ break;
+ }
+ case 2:
+ vmcb_set_cr2(vmcb, v->arch.hvm.guest_cr[2]);
+ break;
+ case 3:
+ vmcb_set_cr3(vmcb, v->arch.hvm.hw_cr[3]);
+ if ( !nestedhvm_enabled(v->domain) )
+ {
+ if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
+ hvm_asid_flush_vcpu(v);
+ }
+ else if ( nestedhvm_vmswitch_in_progress(v) )
+ ; /* CR3 switches during VMRUN/VMEXIT do not flush the TLB. */
+ else if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
+ hvm_asid_flush_vcpu_asid(
+ nestedhvm_vcpu_in_guestmode(v)
+ ? &vcpu_nestedhvm(v).nv_n2asid : &v->arch.hvm.n1asid);
+ break;
+ case 4:
+ value = HVM_CR4_HOST_MASK;
+ if ( paging_mode_hap(v->domain) )
+ value &= ~X86_CR4_PAE;
+ value |= v->arch.hvm.guest_cr[4];
+
+ if ( !hvm_paging_enabled(v) )
+ {
+ /*
+ * When the guest thinks paging is disabled, Xen may need to hide
+ * the effects of shadow paging, as hardware runs with the host
+ * paging settings, rather than the guests settings.
+ *
+ * Without CR0.PG, all memory accesses are user mode, so
+ * _PAGE_USER must be set in the shadow pagetables for guest
+ * userspace to function. This in turn trips up guest supervisor
+ * mode if SMEP/SMAP are left active in context. They wouldn't
+ * have any effect if paging was actually disabled, so hide them
+ * behind the back of the guest.
+ */
+ value &= ~(X86_CR4_SMEP | X86_CR4_SMAP);
+ }
+
+ vmcb_set_cr4(vmcb, value);
+ break;
+ default:
+ BUG();
+ }
+}
+
+static void cf_check svm_update_guest_efer(struct vcpu *v)
+{
+ struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
+ unsigned long guest_efer = v->arch.hvm.guest_efer,
+ xen_efer = read_efer();
+
+ if ( paging_mode_shadow(v->domain) )
+ {
+ /* EFER.NX is a Xen-owned bit and is not under guest control. */
+ guest_efer &= ~EFER_NXE;
+ guest_efer |= xen_efer & EFER_NXE;
+
+ /*
+ * CR0.PG is a Xen-owned bit, and remains set even when the guest has
+ * logically disabled paging.
+ *
+ * LMA was calculated using the guest CR0.PG setting, but LME needs
+ * clearing to avoid interacting with Xen's CR0.PG setting. As writes
+ * to CR0 are intercepted, it is safe to leave LME clear at this
+ * point, and fix up both LME and LMA when CR0.PG is set.
+ */
+ if ( !(guest_efer & EFER_LMA) )
+ guest_efer &= ~EFER_LME;
+ }
+
+ /* SVME must remain set in non-root mode. */
+ guest_efer |= EFER_SVME;
+
+ vmcb_set_efer(vmcb, guest_efer);
+
+ ASSERT(nestedhvm_enabled(v->domain) ||
+ !(v->arch.hvm.guest_efer & EFER_SVME));
+
+ if ( nestedhvm_enabled(v->domain) )
+ svm_nested_features_on_efer_update(v);
+}
+
unsigned long *
svm_msrbit(unsigned long *msr_bitmap, uint32_t msr)
{
@@ -165,7 +318,7 @@ void svm_intercept_msr(struct vcpu *v, uint32_t msr, int flags)
__clear_bit(msr * 2 + 1, msr_bit);
}
-static void svm_enable_msr_interception(struct domain *d, uint32_t msr)
+static void cf_check svm_enable_msr_interception(struct domain *d, uint32_t msr)
{
struct vcpu *v;
@@ -377,13 +530,13 @@ static void svm_load_cpu_state(struct vcpu *v, struct hvm_hw_cpu *data)
svm_update_guest_efer(v);
}
-static void svm_save_vmcb_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
+static void cf_check svm_save_vmcb_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
{
svm_save_cpu_state(v, ctxt);
svm_vmcb_save(v, ctxt);
}
-static int svm_load_vmcb_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
+static int cf_check svm_load_vmcb_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
{
svm_load_cpu_state(v, ctxt);
if ( svm_vmcb_restore(v, ctxt) )
@@ -396,39 +549,7 @@ static int svm_load_vmcb_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
return 0;
}
-static void svm_fpu_enter(struct vcpu *v)
-{
- struct vmcb_struct *n1vmcb = vcpu_nestedhvm(v).nv_n1vmcx;
-
- vcpu_restore_fpu_lazy(v);
- vmcb_set_exception_intercepts(
- n1vmcb,
- vmcb_get_exception_intercepts(n1vmcb) & ~(1U << TRAP_no_device));
-}
-
-static void svm_fpu_leave(struct vcpu *v)
-{
- struct vmcb_struct *n1vmcb = vcpu_nestedhvm(v).nv_n1vmcx;
-
- ASSERT(!v->fpu_dirtied);
- ASSERT(read_cr0() & X86_CR0_TS);
-
- /*
- * If the guest does not have TS enabled then we must cause and handle an
- * exception on first use of the FPU. If the guest *does* have TS enabled
- * then this is not necessary: no FPU activity can occur until the guest
- * clears CR0.TS, and we will initialise the FPU when that happens.
- */
- if ( !(v->arch.hvm.guest_cr[0] & X86_CR0_TS) )
- {
- vmcb_set_exception_intercepts(
- n1vmcb,
- vmcb_get_exception_intercepts(n1vmcb) | (1U << TRAP_no_device));
- vmcb_set_cr0(n1vmcb, vmcb_get_cr0(n1vmcb) | X86_CR0_TS);
- }
-}
-
-static unsigned int svm_get_interrupt_shadow(struct vcpu *v)
+static unsigned cf_check int svm_get_interrupt_shadow(struct vcpu *v)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
unsigned int intr_shadow = 0;
@@ -442,7 +563,8 @@ static unsigned int svm_get_interrupt_shadow(struct vcpu *v)
return intr_shadow;
}
-static void svm_set_interrupt_shadow(struct vcpu *v, unsigned int intr_shadow)
+static void cf_check svm_set_interrupt_shadow(
+ struct vcpu *v, unsigned int intr_shadow)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
u32 general1_intercepts = vmcb_get_general1_intercepts(vmcb);
@@ -456,7 +578,7 @@ static void svm_set_interrupt_shadow(struct vcpu *v, unsigned int intr_shadow)
vmcb_set_general1_intercepts(vmcb, general1_intercepts);
}
-static int svm_guest_x86_mode(struct vcpu *v)
+static int cf_check svm_guest_x86_mode(struct vcpu *v)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -469,129 +591,7 @@ static int svm_guest_x86_mode(struct vcpu *v)
return likely(vmcb->cs.db) ? 4 : 2;
}
-void svm_update_guest_cr(struct vcpu *v, unsigned int cr, unsigned int flags)
-{
- struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
- uint64_t value;
-
- switch ( cr )
- {
- case 0:
- {
- unsigned long hw_cr0_mask = 0;
-
- if ( !(v->arch.hvm.guest_cr[0] & X86_CR0_TS) )
- {
- if ( v != current )
- {
- if ( !v->arch.fully_eager_fpu )
- hw_cr0_mask |= X86_CR0_TS;
- }
- else if ( vmcb_get_cr0(vmcb) & X86_CR0_TS )
- svm_fpu_enter(v);
- }
-
- if ( paging_mode_hap(v->domain) )
- {
- uint32_t intercepts = vmcb_get_cr_intercepts(vmcb);
-
- /* Trap CR3 updates if CR3 memory events are enabled. */
- if ( v->domain->arch.monitor.write_ctrlreg_enabled &
- monitor_ctrlreg_bitmask(VM_EVENT_X86_CR3) )
- vmcb_set_cr_intercepts(vmcb, intercepts | CR_INTERCEPT_CR3_WRITE);
- }
-
- value = v->arch.hvm.guest_cr[0] | hw_cr0_mask;
- if ( !paging_mode_hap(v->domain) )
- value |= X86_CR0_PG | X86_CR0_WP;
- vmcb_set_cr0(vmcb, value);
- break;
- }
- case 2:
- vmcb_set_cr2(vmcb, v->arch.hvm.guest_cr[2]);
- break;
- case 3:
- vmcb_set_cr3(vmcb, v->arch.hvm.hw_cr[3]);
- if ( !nestedhvm_enabled(v->domain) )
- {
- if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
- hvm_asid_flush_vcpu(v);
- }
- else if ( nestedhvm_vmswitch_in_progress(v) )
- ; /* CR3 switches during VMRUN/VMEXIT do not flush the TLB. */
- else if ( !(flags & HVM_UPDATE_GUEST_CR3_NOFLUSH) )
- hvm_asid_flush_vcpu_asid(
- nestedhvm_vcpu_in_guestmode(v)
- ? &vcpu_nestedhvm(v).nv_n2asid : &v->arch.hvm.n1asid);
- break;
- case 4:
- value = HVM_CR4_HOST_MASK;
- if ( paging_mode_hap(v->domain) )
- value &= ~X86_CR4_PAE;
- value |= v->arch.hvm.guest_cr[4];
-
- if ( !hvm_paging_enabled(v) )
- {
- /*
- * When the guest thinks paging is disabled, Xen may need to hide
- * the effects of shadow paging, as hardware runs with the host
- * paging settings, rather than the guests settings.
- *
- * Without CR0.PG, all memory accesses are user mode, so
- * _PAGE_USER must be set in the shadow pagetables for guest
- * userspace to function. This in turn trips up guest supervisor
- * mode if SMEP/SMAP are left active in context. They wouldn't
- * have any effect if paging was actually disabled, so hide them
- * behind the back of the guest.
- */
- value &= ~(X86_CR4_SMEP | X86_CR4_SMAP);
- }
-
- vmcb_set_cr4(vmcb, value);
- break;
- default:
- BUG();
- }
-}
-
-static void svm_update_guest_efer(struct vcpu *v)
-{
- struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
- unsigned long guest_efer = v->arch.hvm.guest_efer,
- xen_efer = read_efer();
-
- if ( paging_mode_shadow(v->domain) )
- {
- /* EFER.NX is a Xen-owned bit and is not under guest control. */
- guest_efer &= ~EFER_NXE;
- guest_efer |= xen_efer & EFER_NXE;
-
- /*
- * CR0.PG is a Xen-owned bit, and remains set even when the guest has
- * logically disabled paging.
- *
- * LMA was calculated using the guest CR0.PG setting, but LME needs
- * clearing to avoid interacting with Xen's CR0.PG setting. As writes
- * to CR0 are intercepted, it is safe to leave LME clear at this
- * point, and fix up both LME and LMA when CR0.PG is set.
- */
- if ( !(guest_efer & EFER_LMA) )
- guest_efer &= ~EFER_LME;
- }
-
- /* SVME must remain set in non-root mode. */
- guest_efer |= EFER_SVME;
-
- vmcb_set_efer(vmcb, guest_efer);
-
- ASSERT(nestedhvm_enabled(v->domain) ||
- !(v->arch.hvm.guest_efer & EFER_SVME));
-
- if ( nestedhvm_enabled(v->domain) )
- svm_nested_features_on_efer_update(v);
-}
-
-static void svm_cpuid_policy_changed(struct vcpu *v)
+static void cf_check svm_cpuid_policy_changed(struct vcpu *v)
{
struct svm_vcpu *svm = &v->arch.hvm.svm;
struct vmcb_struct *vmcb = svm->vmcb;
@@ -636,13 +636,13 @@ void svm_sync_vmcb(struct vcpu *v, enum vmcb_sync_state new_state)
}
}
-static unsigned int svm_get_cpl(struct vcpu *v)
+static unsigned int cf_check svm_get_cpl(struct vcpu *v)
{
return vmcb_get_cpl(v->arch.hvm.svm.vmcb);
}
-static void svm_get_segment_register(struct vcpu *v, enum x86_segment seg,
- struct segment_register *reg)
+static void cf_check svm_get_segment_register(
+ struct vcpu *v, enum x86_segment seg, struct segment_register *reg)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -686,8 +686,8 @@ static void svm_get_segment_register(struct vcpu *v, enum x86_segment seg,
}
}
-static void svm_set_segment_register(struct vcpu *v, enum x86_segment seg,
- struct segment_register *reg)
+static void cf_check svm_set_segment_register(
+ struct vcpu *v, enum x86_segment seg, struct segment_register *reg)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -756,12 +756,12 @@ static void svm_set_segment_register(struct vcpu *v, enum x86_segment seg,
}
}
-static unsigned long svm_get_shadow_gs_base(struct vcpu *v)
+static unsigned long cf_check svm_get_shadow_gs_base(struct vcpu *v)
{
return v->arch.hvm.svm.vmcb->kerngsbase;
}
-static int svm_set_guest_pat(struct vcpu *v, u64 gpat)
+static int cf_check svm_set_guest_pat(struct vcpu *v, u64 gpat)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -772,7 +772,7 @@ static int svm_set_guest_pat(struct vcpu *v, u64 gpat)
return 1;
}
-static int svm_get_guest_pat(struct vcpu *v, u64 *gpat)
+static int cf_check svm_get_guest_pat(struct vcpu *v, u64 *gpat)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -820,7 +820,7 @@ static uint64_t svm_get_tsc_offset(uint64_t host_tsc, uint64_t guest_tsc,
return guest_tsc - scale_tsc(host_tsc, ratio);
}
-static void svm_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
+static void cf_check svm_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
struct vmcb_struct *n1vmcb, *n2vmcb;
@@ -856,7 +856,7 @@ static void svm_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
vmcb_set_tsc_offset(vmcb, offset + n2_tsc_offset);
}
-static void svm_set_rdtsc_exiting(struct vcpu *v, bool_t enable)
+static void cf_check svm_set_rdtsc_exiting(struct vcpu *v, bool enable)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
u32 general1_intercepts = vmcb_get_general1_intercepts(vmcb);
@@ -875,7 +875,8 @@ static void svm_set_rdtsc_exiting(struct vcpu *v, bool_t enable)
vmcb_set_general2_intercepts(vmcb, general2_intercepts);
}
-static void svm_set_descriptor_access_exiting(struct vcpu *v, bool enable)
+static void cf_check svm_set_descriptor_access_exiting(
+ struct vcpu *v, bool enable)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
u32 general1_intercepts = vmcb_get_general1_intercepts(vmcb);
@@ -892,7 +893,7 @@ static void svm_set_descriptor_access_exiting(struct vcpu *v, bool enable)
vmcb_set_general1_intercepts(vmcb, general1_intercepts);
}
-static unsigned int svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
+static unsigned int cf_check svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
unsigned int len = v->arch.hvm.svm.cached_insn_len;
@@ -907,7 +908,7 @@ static unsigned int svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
return len;
}
-static void svm_init_hypercall_page(void *p)
+static void cf_check svm_init_hypercall_page(void *p)
{
unsigned int i;
@@ -1148,7 +1149,7 @@ static int acpi_c1e_quirk(int dir, unsigned int port, unsigned int bytes,
return X86EMUL_OKAY;
}
-static int svm_domain_initialise(struct domain *d)
+static int cf_check svm_domain_initialise(struct domain *d)
{
static const struct arch_csw csw = {
.from = svm_ctxt_switch_from,
@@ -1166,7 +1167,7 @@ static int svm_domain_initialise(struct domain *d)
return 0;
}
-static int svm_vcpu_initialise(struct vcpu *v)
+static int cf_check svm_vcpu_initialise(struct vcpu *v)
{
int rc;
@@ -1183,7 +1184,7 @@ static int svm_vcpu_initialise(struct vcpu *v)
return 0;
}
-static void svm_vcpu_destroy(struct vcpu *v)
+static void cf_check svm_vcpu_destroy(struct vcpu *v)
{
svm_destroy_vmcb(v);
passive_domain_destroy(v);
@@ -1304,7 +1305,7 @@ static void svm_emul_swint_injection(struct x86_event *event)
event->error_code = ec;
}
-static void svm_inject_event(const struct x86_event *event)
+static void cf_check svm_inject_event(const struct x86_event *event)
{
struct vcpu *curr = current;
struct vmcb_struct *vmcb = curr->arch.hvm.svm.vmcb;
@@ -1434,12 +1435,12 @@ static void svm_inject_event(const struct x86_event *event)
HVMTRACE_2D(INJ_EXC, _event.vector, _event.error_code);
}
-static bool svm_event_pending(const struct vcpu *v)
+static bool cf_check svm_event_pending(const struct vcpu *v)
{
return v->arch.hvm.svm.vmcb->event_inj.v;
}
-static void svm_cpu_dead(unsigned int cpu)
+static void cf_check svm_cpu_dead(unsigned int cpu)
{
paddr_t *this_hsa = &per_cpu(hsa, cpu);
paddr_t *this_vmcb = &per_cpu(host_vmcb, cpu);
@@ -1465,7 +1466,7 @@ static void svm_cpu_dead(unsigned int cpu)
}
}
-static int svm_cpu_up_prepare(unsigned int cpu)
+static int cf_check svm_cpu_up_prepare(unsigned int cpu)
{
paddr_t *this_hsa = &per_cpu(hsa, cpu);
paddr_t *this_vmcb = &per_cpu(host_vmcb, cpu);
@@ -1620,7 +1621,7 @@ static int _svm_cpu_up(bool bsp)
return 0;
}
-static int svm_cpu_up(void)
+static int cf_check svm_cpu_up(void)
{
return _svm_cpu_up(false);
}
@@ -1749,7 +1750,7 @@ static void svm_do_nested_pgfault(struct vcpu *v,
domain_crash(v->domain);
}
-static void svm_fpu_dirty_intercept(void)
+static void cf_check svm_fpu_dirty_intercept(void)
{
struct vcpu *v = current;
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -1795,7 +1796,8 @@ static void svm_dr_access(struct vcpu *v, struct cpu_user_regs *regs)
__restore_debug_registers(vmcb, v);
}
-static int svm_msr_read_intercept(unsigned int msr, uint64_t *msr_content)
+static int cf_check svm_msr_read_intercept(
+ unsigned int msr, uint64_t *msr_content)
{
struct vcpu *v = current;
const struct domain *d = v->domain;
@@ -1990,7 +1992,8 @@ static int svm_msr_read_intercept(unsigned int msr, uint64_t *msr_content)
return X86EMUL_EXCEPTION;
}
-static int svm_msr_write_intercept(unsigned int msr, uint64_t msr_content)
+static int cf_check svm_msr_write_intercept(
+ unsigned int msr, uint64_t msr_content)
{
struct vcpu *v = current;
struct domain *d = v->domain;
@@ -2409,7 +2412,7 @@ static void svm_vmexit_mce_intercept(
}
}
-static void svm_wbinvd_intercept(void)
+static void cf_check svm_wbinvd_intercept(void)
{
if ( cache_flush_permitted(current->domain) )
flush_all(FLUSH_CACHE);
@@ -2454,12 +2457,13 @@ static bool is_invlpg(const struct x86_emulate_state *state,
(ext & 7) == 7;
}
-static void svm_invlpg(struct vcpu *v, unsigned long linear)
+static void cf_check svm_invlpg(struct vcpu *v, unsigned long linear)
{
svm_asid_g_invlpg(v, linear);
}
-static bool svm_get_pending_event(struct vcpu *v, struct x86_event *info)
+static bool cf_check svm_get_pending_event(
+ struct vcpu *v, struct x86_event *info)
{
const struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -2473,7 +2477,7 @@ static bool svm_get_pending_event(struct vcpu *v, struct x86_event *info)
return true;
}
-static uint64_t svm_get_reg(struct vcpu *v, unsigned int reg)
+static uint64_t cf_check svm_get_reg(struct vcpu *v, unsigned int reg)
{
const struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
struct domain *d = v->domain;
@@ -2491,7 +2495,7 @@ static uint64_t svm_get_reg(struct vcpu *v, unsigned int reg)
}
}
-static void svm_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
+static void cf_check svm_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
struct domain *d = v->domain;
diff --git a/xen/arch/x86/hvm/vmx/intr.c b/xen/arch/x86/hvm/vmx/intr.c
index 80bfbb478782..13bbe8430df5 100644
--- a/xen/arch/x86/hvm/vmx/intr.c
+++ b/xen/arch/x86/hvm/vmx/intr.c
@@ -147,7 +147,7 @@ static void vmx_enable_intr_window(struct vcpu *v, struct hvm_intack intack)
* used but may have negative impact on interrupt performance.
*/
-enum hvm_intblk nvmx_intr_blocked(struct vcpu *v)
+enum hvm_intblk cf_check nvmx_intr_blocked(struct vcpu *v)
{
int r = hvm_intblk_none;
struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v);
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index d2cafd8ca1c5..60b506ac3f40 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -655,7 +655,7 @@ void vmx_vmcs_reload(struct vcpu *v)
vmx_load_vmcs(v);
}
-int vmx_cpu_up_prepare(unsigned int cpu)
+int cf_check vmx_cpu_up_prepare(unsigned int cpu)
{
/*
* If nvmx_cpu_up_prepare() failed, do not return failure and just fallback
@@ -676,7 +676,7 @@ int vmx_cpu_up_prepare(unsigned int cpu)
return -ENOMEM;
}
-void vmx_cpu_dead(unsigned int cpu)
+void cf_check vmx_cpu_dead(unsigned int cpu)
{
vmx_free_vmcs(per_cpu(vmxon_region, cpu));
per_cpu(vmxon_region, cpu) = 0;
@@ -774,12 +774,12 @@ static int _vmx_cpu_up(bool bsp)
return 0;
}
-int vmx_cpu_up()
+int cf_check vmx_cpu_up()
{
return _vmx_cpu_up(false);
}
-void vmx_cpu_down(void)
+void cf_check vmx_cpu_down(void)
{
struct list_head *active_vmcs_list = &this_cpu(active_vmcs_list);
unsigned long flags;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index dade08f60279..2c4804f9b884 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -68,14 +68,16 @@ static void vmx_ctxt_switch_to(struct vcpu *v);
static int alloc_vlapic_mapping(void);
static void vmx_install_vlapic_mapping(struct vcpu *v);
-static void vmx_update_guest_cr(struct vcpu *v, unsigned int cr,
- unsigned int flags);
-static void vmx_update_guest_efer(struct vcpu *v);
-static void vmx_wbinvd_intercept(void);
-static void vmx_fpu_dirty_intercept(void);
-static int vmx_msr_read_intercept(unsigned int msr, uint64_t *msr_content);
-static int vmx_msr_write_intercept(unsigned int msr, uint64_t msr_content);
-static void vmx_invlpg(struct vcpu *v, unsigned long linear);
+static void cf_check vmx_update_guest_cr(
+ struct vcpu *v, unsigned int cr, unsigned int flags);
+static void cf_check vmx_update_guest_efer(struct vcpu *v);
+static void cf_check vmx_wbinvd_intercept(void);
+static void cf_check vmx_fpu_dirty_intercept(void);
+static int cf_check vmx_msr_read_intercept(
+ unsigned int msr, uint64_t *msr_content);
+static int cf_check vmx_msr_write_intercept(
+ unsigned int msr, uint64_t msr_content);
+static void cf_check vmx_invlpg(struct vcpu *v, unsigned long linear);
static mfn_t __read_mostly apic_access_mfn = INVALID_MFN_INITIALIZER;
@@ -103,7 +105,7 @@ void vmx_pi_per_cpu_init(unsigned int cpu)
spin_lock_init(&per_cpu(vmx_pi_blocking, cpu).lock);
}
-static void vmx_vcpu_block(struct vcpu *v)
+static void cf_check vmx_vcpu_block(struct vcpu *v)
{
unsigned long flags;
unsigned int dest;
@@ -395,7 +397,7 @@ void vmx_pi_hooks_deassign(struct domain *d)
domain_unpause(d);
}
-static int vmx_domain_initialise(struct domain *d)
+static int cf_check vmx_domain_initialise(struct domain *d)
{
static const struct arch_csw csw = {
.from = vmx_ctxt_switch_from,
@@ -414,7 +416,7 @@ static int vmx_domain_initialise(struct domain *d)
return 0;
}
-static void domain_creation_finished(struct domain *d)
+static void cf_check domain_creation_finished(struct domain *d)
{
gfn_t gfn = gaddr_to_gfn(APIC_DEFAULT_PHYS_BASE);
bool ipat;
@@ -444,7 +446,7 @@ static void vmx_init_ipt(struct vcpu *v)
v->arch.msrs->rtit.output_limit = size - 1;
}
-static int vmx_vcpu_initialise(struct vcpu *v)
+static int cf_check vmx_vcpu_initialise(struct vcpu *v)
{
int rc;
@@ -491,7 +493,7 @@ static int vmx_vcpu_initialise(struct vcpu *v)
return 0;
}
-static void vmx_vcpu_destroy(struct vcpu *v)
+static void cf_check vmx_vcpu_destroy(struct vcpu *v)
{
/*
* There are cases that domain still remains in log-dirty mode when it is
@@ -589,7 +591,7 @@ void vmx_update_exception_bitmap(struct vcpu *v)
__vmwrite(EXCEPTION_BITMAP, bitmap);
}
-static void vmx_cpuid_policy_changed(struct vcpu *v)
+static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
{
const struct cpuid_policy *cp = v->domain->arch.cpuid;
int rc = 0;
@@ -647,7 +649,7 @@ static void vmx_cpuid_policy_changed(struct vcpu *v)
}
}
-int vmx_guest_x86_mode(struct vcpu *v)
+int cf_check vmx_guest_x86_mode(struct vcpu *v)
{
unsigned long cs_ar_bytes;
@@ -844,7 +846,7 @@ static void vmx_load_cpu_state(struct vcpu *v, struct hvm_hw_cpu *data)
}
-static void vmx_save_vmcs_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
+static void cf_check vmx_save_vmcs_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
{
if ( v == current )
vmx_save_guest_msrs(v);
@@ -853,7 +855,7 @@ static void vmx_save_vmcs_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
vmx_vmcs_save(v, ctxt);
}
-static int vmx_load_vmcs_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
+static int cf_check vmx_load_vmcs_ctxt(struct vcpu *v, struct hvm_hw_cpu *ctxt)
{
/* Not currently safe to use in current context. */
ASSERT(v != current);
@@ -879,7 +881,7 @@ static void vmx_fpu_enter(struct vcpu *v)
__vmwrite(HOST_CR0, v->arch.hvm.vmx.host_cr0);
}
-static void vmx_fpu_leave(struct vcpu *v)
+static void cf_check vmx_fpu_leave(struct vcpu *v)
{
ASSERT(!v->fpu_dirtied);
ASSERT(read_cr0() & X86_CR0_TS);
@@ -956,7 +958,7 @@ unsigned int vmx_get_cpl(void)
return (attr >> 5) & 3;
}
-static unsigned int _vmx_get_cpl(struct vcpu *v)
+static unsigned int cf_check _vmx_get_cpl(struct vcpu *v)
{
unsigned int cpl;
@@ -982,8 +984,8 @@ static unsigned int _vmx_get_cpl(struct vcpu *v)
#define vm86_ds_attr 0xf3
#define vm86_tr_attr 0x8b
-static void vmx_get_segment_register(struct vcpu *v, enum x86_segment seg,
- struct segment_register *reg)
+static void cf_check vmx_get_segment_register(
+ struct vcpu *v, enum x86_segment seg, struct segment_register *reg)
{
unsigned long attr = 0, sel = 0, limit;
unsigned int tmp_seg;
@@ -1084,8 +1086,8 @@ static void vmx_get_segment_register(struct vcpu *v, enum x86_segment seg,
}
}
-static void vmx_set_segment_register(struct vcpu *v, enum x86_segment seg,
- struct segment_register *reg)
+static void cf_check vmx_set_segment_register(
+ struct vcpu *v, enum x86_segment seg, struct segment_register *reg)
{
uint32_t attr, sel, limit;
uint64_t base;
@@ -1174,12 +1176,12 @@ static void vmx_set_segment_register(struct vcpu *v, enum x86_segment seg,
vmx_vmcs_exit(v);
}
-static unsigned long vmx_get_shadow_gs_base(struct vcpu *v)
+static unsigned long cf_check vmx_get_shadow_gs_base(struct vcpu *v)
{
return v->arch.hvm.vmx.shadow_gs;
}
-static int vmx_set_guest_pat(struct vcpu *v, u64 gpat)
+static int cf_check vmx_set_guest_pat(struct vcpu *v, u64 gpat)
{
if ( !paging_mode_hap(v->domain) ||
unlikely(v->arch.hvm.cache_mode == NO_FILL_CACHE_MODE) )
@@ -1191,7 +1193,7 @@ static int vmx_set_guest_pat(struct vcpu *v, u64 gpat)
return 1;
}
-static int vmx_get_guest_pat(struct vcpu *v, u64 *gpat)
+static int cf_check vmx_get_guest_pat(struct vcpu *v, u64 *gpat)
{
if ( !paging_mode_hap(v->domain) ||
unlikely(v->arch.hvm.cache_mode == NO_FILL_CACHE_MODE) )
@@ -1203,7 +1205,7 @@ static int vmx_get_guest_pat(struct vcpu *v, u64 *gpat)
return 1;
}
-static void vmx_handle_cd(struct vcpu *v, unsigned long value)
+static void cf_check vmx_handle_cd(struct vcpu *v, unsigned long value)
{
if ( !paging_mode_hap(v->domain) )
{
@@ -1253,7 +1255,7 @@ static void vmx_handle_cd(struct vcpu *v, unsigned long value)
}
}
-static void vmx_setup_tsc_scaling(struct vcpu *v)
+static void cf_check vmx_setup_tsc_scaling(struct vcpu *v)
{
if ( v->domain->arch.vtsc )
return;
@@ -1263,7 +1265,7 @@ static void vmx_setup_tsc_scaling(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static void vmx_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
+static void cf_check vmx_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
{
vmx_vmcs_enter(v);
@@ -1274,7 +1276,7 @@ static void vmx_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
vmx_vmcs_exit(v);
}
-static void vmx_set_rdtsc_exiting(struct vcpu *v, bool_t enable)
+static void cf_check vmx_set_rdtsc_exiting(struct vcpu *v, bool enable)
{
vmx_vmcs_enter(v);
v->arch.hvm.vmx.exec_control &= ~CPU_BASED_RDTSC_EXITING;
@@ -1284,7 +1286,8 @@ static void vmx_set_rdtsc_exiting(struct vcpu *v, bool_t enable)
vmx_vmcs_exit(v);
}
-static void vmx_set_descriptor_access_exiting(struct vcpu *v, bool enable)
+static void cf_check vmx_set_descriptor_access_exiting(
+ struct vcpu *v, bool enable)
{
if ( enable )
v->arch.hvm.vmx.secondary_exec_control |=
@@ -1298,7 +1301,7 @@ static void vmx_set_descriptor_access_exiting(struct vcpu *v, bool enable)
vmx_vmcs_exit(v);
}
-static void vmx_init_hypercall_page(void *p)
+static void cf_check vmx_init_hypercall_page(void *p)
{
unsigned int i;
@@ -1321,7 +1324,7 @@ static void vmx_init_hypercall_page(void *p)
}
}
-static unsigned int vmx_get_interrupt_shadow(struct vcpu *v)
+static unsigned int cf_check vmx_get_interrupt_shadow(struct vcpu *v)
{
unsigned long intr_shadow;
@@ -1330,7 +1333,8 @@ static unsigned int vmx_get_interrupt_shadow(struct vcpu *v)
return intr_shadow;
}
-static void vmx_set_interrupt_shadow(struct vcpu *v, unsigned int intr_shadow)
+static void cf_check vmx_set_interrupt_shadow(
+ struct vcpu *v, unsigned int intr_shadow)
{
__vmwrite(GUEST_INTERRUPTIBILITY_INFO, intr_shadow);
}
@@ -1381,7 +1385,7 @@ static void vmx_load_pdptrs(struct vcpu *v)
return;
}
-static void vmx_update_host_cr3(struct vcpu *v)
+static void cf_check vmx_update_host_cr3(struct vcpu *v)
{
vmx_vmcs_enter(v);
__vmwrite(HOST_CR3, v->arch.cr3);
@@ -1400,8 +1404,8 @@ void vmx_update_debug_state(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static void vmx_update_guest_cr(struct vcpu *v, unsigned int cr,
- unsigned int flags)
+static void cf_check vmx_update_guest_cr(
+ struct vcpu *v, unsigned int cr, unsigned int flags)
{
vmx_vmcs_enter(v);
@@ -1603,7 +1607,7 @@ static void vmx_update_guest_cr(struct vcpu *v, unsigned int cr,
vmx_vmcs_exit(v);
}
-static void vmx_update_guest_efer(struct vcpu *v)
+static void cf_check vmx_update_guest_efer(struct vcpu *v)
{
unsigned long entry_ctls, guest_efer = v->arch.hvm.guest_efer,
xen_efer = read_efer();
@@ -1705,7 +1709,8 @@ void nvmx_enqueue_n2_exceptions(struct vcpu *v,
nvmx->intr.intr_info, nvmx->intr.error_code);
}
-static int nvmx_vmexit_event(struct vcpu *v, const struct x86_event *event)
+static int cf_check nvmx_vmexit_event(
+ struct vcpu *v, const struct x86_event *event)
{
nvmx_enqueue_n2_exceptions(v, event->vector, event->error_code,
hvm_intsrc_none);
@@ -1791,7 +1796,7 @@ void vmx_inject_nmi(void)
* - #DB is X86_EVENTTYPE_HW_EXCEPTION, except when generated by
* opcode 0xf1 (which is X86_EVENTTYPE_PRI_SW_EXCEPTION)
*/
-static void vmx_inject_event(const struct x86_event *event)
+static void cf_check vmx_inject_event(const struct x86_event *event)
{
unsigned long intr_info;
struct vcpu *curr = current;
@@ -1872,7 +1877,7 @@ static void vmx_inject_event(const struct x86_event *event)
HVMTRACE_2D(INJ_EXC, _event.vector, _event.error_code);
}
-static bool vmx_event_pending(const struct vcpu *v)
+static bool cf_check vmx_event_pending(const struct vcpu *v)
{
unsigned long intr_info;
@@ -1882,7 +1887,7 @@ static bool vmx_event_pending(const struct vcpu *v)
return intr_info & INTR_INFO_VALID_MASK;
}
-static void vmx_set_info_guest(struct vcpu *v)
+static void cf_check vmx_set_info_guest(struct vcpu *v)
{
unsigned long intr_shadow;
@@ -1910,7 +1915,8 @@ static void vmx_set_info_guest(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static void vmx_update_eoi_exit_bitmap(struct vcpu *v, uint8_t vector, bool set)
+static void cf_check vmx_update_eoi_exit_bitmap(
+ struct vcpu *v, uint8_t vector, bool set)
{
if ( set )
vmx_set_eoi_exit_bitmap(v, vector);
@@ -1938,7 +1944,7 @@ static u8 set_svi(int isr)
return old;
}
-static void vmx_process_isr(int isr, struct vcpu *v)
+static void cf_check vmx_process_isr(int isr, struct vcpu *v)
{
unsigned int i;
const struct vlapic *vlapic = vcpu_vlapic(v);
@@ -2026,7 +2032,7 @@ static void __vmx_deliver_posted_interrupt(struct vcpu *v)
}
}
-static void vmx_deliver_posted_intr(struct vcpu *v, u8 vector)
+static void cf_check vmx_deliver_posted_intr(struct vcpu *v, u8 vector)
{
struct pi_desc old, new, prev;
@@ -2073,7 +2079,7 @@ static void vmx_deliver_posted_intr(struct vcpu *v, u8 vector)
__vmx_deliver_posted_interrupt(v);
}
-static void vmx_sync_pir_to_irr(struct vcpu *v)
+static void cf_check vmx_sync_pir_to_irr(struct vcpu *v)
{
struct vlapic *vlapic = vcpu_vlapic(v);
unsigned int group, i;
@@ -2089,12 +2095,12 @@ static void vmx_sync_pir_to_irr(struct vcpu *v)
vlapic_set_vector(i, &vlapic->regs->data[APIC_IRR]);
}
-static bool vmx_test_pir(const struct vcpu *v, uint8_t vec)
+static bool cf_check vmx_test_pir(const struct vcpu *v, uint8_t vec)
{
return pi_test_pir(vec, &v->arch.hvm.vmx.pi_desc);
}
-static void vmx_handle_eoi(uint8_t vector, int isr)
+static void cf_check vmx_handle_eoi(uint8_t vector, int isr)
{
uint8_t old_svi = set_svi(isr);
static bool warned;
@@ -2103,7 +2109,7 @@ static void vmx_handle_eoi(uint8_t vector, int isr)
printk(XENLOG_WARNING "EOI for %02x but SVI=%02x\n", vector, old_svi);
}
-static void vmx_enable_msr_interception(struct domain *d, uint32_t msr)
+static void cf_check vmx_enable_msr_interception(struct domain *d, uint32_t msr)
{
struct vcpu *v;
@@ -2111,7 +2117,7 @@ static void vmx_enable_msr_interception(struct domain *d, uint32_t msr)
vmx_set_msr_intercept(v, msr, VMX_MSR_W);
}
-static void vmx_vcpu_update_eptp(struct vcpu *v)
+static void cf_check vmx_vcpu_update_eptp(struct vcpu *v)
{
struct domain *d = v->domain;
struct p2m_domain *p2m = NULL;
@@ -2136,7 +2142,7 @@ static void vmx_vcpu_update_eptp(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static void vmx_vcpu_update_vmfunc_ve(struct vcpu *v)
+static void cf_check vmx_vcpu_update_vmfunc_ve(struct vcpu *v)
{
struct domain *d = v->domain;
u32 mask = SECONDARY_EXEC_ENABLE_VM_FUNCTIONS;
@@ -2180,7 +2186,7 @@ static void vmx_vcpu_update_vmfunc_ve(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static int vmx_vcpu_emulate_vmfunc(const struct cpu_user_regs *regs)
+static int cf_check vmx_vcpu_emulate_vmfunc(const struct cpu_user_regs *regs)
{
int rc = X86EMUL_EXCEPTION;
struct vcpu *curr = current;
@@ -2193,7 +2199,7 @@ static int vmx_vcpu_emulate_vmfunc(const struct cpu_user_regs *regs)
return rc;
}
-static bool_t vmx_vcpu_emulate_ve(struct vcpu *v)
+static bool cf_check vmx_vcpu_emulate_ve(struct vcpu *v)
{
const struct page_info *pg = vcpu_altp2m(v).veinfo_pg;
ve_info_t *veinfo;
@@ -2230,7 +2236,8 @@ static bool_t vmx_vcpu_emulate_ve(struct vcpu *v)
return rc;
}
-static bool vmx_get_pending_event(struct vcpu *v, struct x86_event *info)
+static bool cf_check vmx_get_pending_event(
+ struct vcpu *v, struct x86_event *info)
{
unsigned long intr_info, error_code;
@@ -2267,7 +2274,8 @@ static bool vmx_get_pending_event(struct vcpu *v, struct x86_event *info)
(RTIT_STATUS_FILTER_EN | RTIT_STATUS_CONTEXT_EN | RTIT_STATUS_TRIGGER_EN | \
RTIT_STATUS_ERROR | RTIT_STATUS_STOPPED)
-static int vmtrace_get_option(struct vcpu *v, uint64_t key, uint64_t *output)
+static int cf_check vmtrace_get_option(
+ struct vcpu *v, uint64_t key, uint64_t *output)
{
const struct vcpu_msrs *msrs = v->arch.msrs;
@@ -2288,7 +2296,8 @@ static int vmtrace_get_option(struct vcpu *v, uint64_t key, uint64_t *output)
return 0;
}
-static int vmtrace_set_option(struct vcpu *v, uint64_t key, uint64_t value)
+static int cf_check vmtrace_set_option(
+ struct vcpu *v, uint64_t key, uint64_t value)
{
struct vcpu_msrs *msrs = v->arch.msrs;
bool new_en, old_en = msrs->rtit.ctl & RTIT_CTL_TRACE_EN;
@@ -2342,7 +2351,7 @@ static int vmtrace_set_option(struct vcpu *v, uint64_t key, uint64_t value)
return 0;
}
-static int vmtrace_control(struct vcpu *v, bool enable, bool reset)
+static int cf_check vmtrace_control(struct vcpu *v, bool enable, bool reset)
{
struct vcpu_msrs *msrs = v->arch.msrs;
uint64_t new_ctl;
@@ -2374,13 +2383,13 @@ static int vmtrace_control(struct vcpu *v, bool enable, bool reset)
return 0;
}
-static int vmtrace_output_position(struct vcpu *v, uint64_t *pos)
+static int cf_check vmtrace_output_position(struct vcpu *v, uint64_t *pos)
{
*pos = v->arch.msrs->rtit.output_offset;
return v->arch.hvm.vmx.ipt_active;
}
-static int vmtrace_reset(struct vcpu *v)
+static int cf_check vmtrace_reset(struct vcpu *v)
{
if ( !v->arch.hvm.vmx.ipt_active )
return -EINVAL;
@@ -2390,7 +2399,7 @@ static int vmtrace_reset(struct vcpu *v)
return 0;
}
-static uint64_t vmx_get_reg(struct vcpu *v, unsigned int reg)
+static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
{
struct domain *d = v->domain;
uint64_t val = 0;
@@ -2429,7 +2438,7 @@ static uint64_t vmx_get_reg(struct vcpu *v, unsigned int reg)
return val;
}
-static void vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
+static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
{
struct domain *d = v->domain;
int rc;
@@ -2823,7 +2832,7 @@ void update_guest_eip(void)
hvm_inject_hw_exception(TRAP_debug, X86_EVENT_NO_EC);
}
-static void vmx_fpu_dirty_intercept(void)
+static void cf_check vmx_fpu_dirty_intercept(void)
{
struct vcpu *curr = current;
@@ -2858,7 +2867,7 @@ static void vmx_invlpg_intercept(unsigned long linear)
paging_invlpg(current, linear);
}
-static void vmx_invlpg(struct vcpu *v, unsigned long linear)
+static void cf_check vmx_invlpg(struct vcpu *v, unsigned long linear)
{
if ( cpu_has_vmx_vpid )
vpid_sync_vcpu_gva(v, linear);
@@ -3184,7 +3193,8 @@ static int is_last_branch_msr(u32 ecx)
return 0;
}
-static int vmx_msr_read_intercept(unsigned int msr, uint64_t *msr_content)
+static int cf_check vmx_msr_read_intercept(
+ unsigned int msr, uint64_t *msr_content)
{
struct vcpu *curr = current;
uint64_t tmp;
@@ -3387,7 +3397,8 @@ void vmx_vlapic_msr_changed(struct vcpu *v)
vmx_vmcs_exit(v);
}
-static int vmx_msr_write_intercept(unsigned int msr, uint64_t msr_content)
+static int cf_check vmx_msr_write_intercept(
+ unsigned int msr, uint64_t msr_content)
{
struct vcpu *v = current;
const struct cpuid_policy *cp = v->domain->arch.cpuid;
@@ -3587,7 +3598,7 @@ static void vmx_do_extint(struct cpu_user_regs *regs)
do_IRQ(regs);
}
-static void vmx_wbinvd_intercept(void)
+static void cf_check vmx_wbinvd_intercept(void)
{
if ( !cache_flush_permitted(current->domain) || iommu_snoop )
return;
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 7419ee9dd0bc..5f5445147585 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -62,7 +62,7 @@ void nvmx_cpu_dead(unsigned int cpu)
XFREE(per_cpu(vvmcs_buf, cpu));
}
-int nvmx_vcpu_initialise(struct vcpu *v)
+int cf_check nvmx_vcpu_initialise(struct vcpu *v)
{
struct domain *d = v->domain;
struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
@@ -150,7 +150,7 @@ int nvmx_vcpu_initialise(struct vcpu *v)
return 0;
}
-void nvmx_vcpu_destroy(struct vcpu *v)
+void cf_check nvmx_vcpu_destroy(struct vcpu *v)
{
struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v);
@@ -199,7 +199,7 @@ static void vcpu_relinquish_resources(struct vcpu *v)
FREE_XENHEAP_PAGE(nvmx->msr_merged);
}
-void nvmx_domain_relinquish_resources(struct domain *d)
+void cf_check nvmx_domain_relinquish_resources(struct domain *d)
{
struct vcpu *v;
@@ -210,17 +210,17 @@ void nvmx_domain_relinquish_resources(struct domain *d)
}
}
-int nvmx_vcpu_reset(struct vcpu *v)
+int cf_check nvmx_vcpu_reset(struct vcpu *v)
{
return 0;
}
-uint64_t nvmx_vcpu_eptp_base(struct vcpu *v)
+uint64_t cf_check nvmx_vcpu_eptp_base(struct vcpu *v)
{
return get_vvmcs(v, EPT_POINTER) & PAGE_MASK;
}
-bool_t nvmx_ept_enabled(struct vcpu *v)
+bool cf_check nvmx_ept_enabled(struct vcpu *v)
{
struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
@@ -514,7 +514,7 @@ static void vmfail(struct cpu_user_regs *regs, enum vmx_insn_errno errno)
vmfail_invalid(regs);
}
-bool_t nvmx_intercepts_exception(
+bool cf_check nvmx_intercepts_exception(
struct vcpu *v, unsigned int vector, int error_code)
{
u32 exception_bitmap, pfec_match=0, pfec_mask=0;
@@ -2346,7 +2346,7 @@ int nvmx_msr_read_intercept(unsigned int msr, u64 *msr_content)
* walk is successful, the translated value is returned in
* L1_gpa. The result value tells what to do next.
*/
-int nvmx_hap_walk_L1_p2m(
+int cf_check nvmx_hap_walk_L1_p2m(
struct vcpu *v, paddr_t L2_gpa, paddr_t *L1_gpa, unsigned int *page_order,
uint8_t *p2m_acc, struct npfec npfec)
{
diff --git a/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h b/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
index c3ef2354140c..656d7d1a9a71 100644
--- a/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
+++ b/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
@@ -107,22 +107,22 @@ nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
void svm_nested_features_on_efer_update(struct vcpu *v);
/* Interface methods */
-void nsvm_vcpu_destroy(struct vcpu *v);
-int nsvm_vcpu_initialise(struct vcpu *v);
-int nsvm_vcpu_reset(struct vcpu *v);
+void cf_check nsvm_vcpu_destroy(struct vcpu *v);
+int cf_check nsvm_vcpu_initialise(struct vcpu *v);
+int cf_check nsvm_vcpu_reset(struct vcpu *v);
int nsvm_vcpu_vmrun(struct vcpu *v, struct cpu_user_regs *regs);
-int nsvm_vcpu_vmexit_event(struct vcpu *v, const struct x86_event *event);
-uint64_t nsvm_vcpu_hostcr3(struct vcpu *v);
-bool_t nsvm_vmcb_guest_intercepts_event(
+int cf_check nsvm_vcpu_vmexit_event(struct vcpu *v, const struct x86_event *event);
+uint64_t cf_check nsvm_vcpu_hostcr3(struct vcpu *v);
+bool cf_check nsvm_vmcb_guest_intercepts_event(
struct vcpu *v, unsigned int vector, int errcode);
-bool_t nsvm_vmcb_hap_enabled(struct vcpu *v);
-enum hvm_intblk nsvm_intr_blocked(struct vcpu *v);
+bool cf_check nsvm_vmcb_hap_enabled(struct vcpu *v);
+enum hvm_intblk cf_check nsvm_intr_blocked(struct vcpu *v);
/* Interrupts, vGIF */
void svm_vmexit_do_clgi(struct cpu_user_regs *regs, struct vcpu *v);
void svm_vmexit_do_stgi(struct cpu_user_regs *regs, struct vcpu *v);
bool_t nestedsvm_gif_isset(struct vcpu *v);
-int nsvm_hap_walk_L1_p2m(
+int cf_check nsvm_hap_walk_L1_p2m(
struct vcpu *v, paddr_t L2_gpa, paddr_t *L1_gpa, unsigned int *page_order,
uint8_t *p2m_acc, struct npfec npfec);
diff --git a/xen/arch/x86/include/asm/hvm/svm/svm.h b/xen/arch/x86/include/asm/hvm/svm/svm.h
index 09c32044ec8a..65e35a4f59b7 100644
--- a/xen/arch/x86/include/asm/hvm/svm/svm.h
+++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
@@ -50,7 +50,6 @@ struct vcpu;
unsigned long *svm_msrbit(unsigned long *msr_bitmap, uint32_t msr);
void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len);
-void svm_update_guest_cr(struct vcpu *, unsigned int cr, unsigned int flags);
/*
* PV context switch helpers. Prefetching the VMCB area itself has been shown
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
index 03c9ccf627ab..9119aa8536df 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -22,10 +22,10 @@
extern void vmcs_dump_vcpu(struct vcpu *v);
extern int vmx_vmcs_init(void);
-extern int vmx_cpu_up_prepare(unsigned int cpu);
-extern void vmx_cpu_dead(unsigned int cpu);
-extern int vmx_cpu_up(void);
-extern void vmx_cpu_down(void);
+int cf_check vmx_cpu_up_prepare(unsigned int cpu);
+void cf_check vmx_cpu_dead(unsigned int cpu);
+int cf_check vmx_cpu_up(void);
+void cf_check vmx_cpu_down(void);
struct vmcs_struct {
u32 vmcs_revision_id;
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmx.h b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
index 97e7652aa12c..5284fe931f62 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
@@ -588,7 +588,7 @@ static inline int __vmxon(u64 addr)
return rc;
}
-int vmx_guest_x86_mode(struct vcpu *v);
+int cf_check vmx_guest_x86_mode(struct vcpu *v);
unsigned int vmx_get_cpl(void);
void vmx_inject_extint(int trap, uint8_t source);
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vvmx.h b/xen/arch/x86/include/asm/hvm/vmx/vvmx.h
index e4ca3bc6ee2b..2c3adb5dd6b3 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vvmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vvmx.h
@@ -84,23 +84,23 @@ union vmx_inst_info {
u32 word;
};
-int nvmx_vcpu_initialise(struct vcpu *v);
-void nvmx_vcpu_destroy(struct vcpu *v);
-int nvmx_vcpu_reset(struct vcpu *v);
-uint64_t nvmx_vcpu_eptp_base(struct vcpu *v);
-enum hvm_intblk nvmx_intr_blocked(struct vcpu *v);
-bool_t nvmx_intercepts_exception(
+int cf_check nvmx_vcpu_initialise(struct vcpu *v);
+void cf_check nvmx_vcpu_destroy(struct vcpu *v);
+int cf_check nvmx_vcpu_reset(struct vcpu *v);
+uint64_t cf_check nvmx_vcpu_eptp_base(struct vcpu *v);
+enum hvm_intblk cf_check nvmx_intr_blocked(struct vcpu *v);
+bool cf_check nvmx_intercepts_exception(
struct vcpu *v, unsigned int vector, int error_code);
-void nvmx_domain_relinquish_resources(struct domain *d);
+void cf_check nvmx_domain_relinquish_resources(struct domain *d);
-bool_t nvmx_ept_enabled(struct vcpu *v);
+bool cf_check nvmx_ept_enabled(struct vcpu *v);
#define EPT_TRANSLATE_SUCCEED 0
#define EPT_TRANSLATE_VIOLATION 1
#define EPT_TRANSLATE_MISCONFIG 2
#define EPT_TRANSLATE_RETRY 3
-int nvmx_hap_walk_L1_p2m(
+int cf_check nvmx_hap_walk_L1_p2m(
struct vcpu *v, paddr_t L2_gpa, paddr_t *L1_gpa, unsigned int *page_order,
uint8_t *p2m_acc, struct npfec npfec);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 33/70] x86/hvm: CFI hardening for device emulation
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (31 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 32/70] x86/hvm: CFI hardening for hvm_funcs Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 34/70] x86/emul: CFI hardening Andrew Cooper
` (38 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/emul-i8254.c | 8 +++----
xen/arch/x86/hvm/emulate.c | 21 ++++++++---------
xen/arch/x86/hvm/hpet.c | 6 ++---
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/hvm/intercept.c | 28 +++++++++++++----------
xen/arch/x86/hvm/io.c | 38 +++++++++++++++++--------------
xen/arch/x86/hvm/ioreq.c | 2 +-
xen/arch/x86/hvm/pmtimer.c | 4 ++--
xen/arch/x86/hvm/rtc.c | 6 ++---
xen/arch/x86/hvm/stdvga.c | 19 ++++++++--------
xen/arch/x86/hvm/svm/svm.c | 4 ++--
xen/arch/x86/hvm/vioapic.c | 8 +++----
xen/arch/x86/hvm/vlapic.c | 11 +++++----
xen/arch/x86/hvm/vmsi.c | 14 +++++++-----
xen/arch/x86/hvm/vpic.c | 4 ++--
xen/arch/x86/include/asm/hvm/vioapic.h | 2 +-
xen/drivers/passthrough/amd/iommu_guest.c | 10 ++++----
17 files changed, 98 insertions(+), 89 deletions(-)
diff --git a/xen/arch/x86/emul-i8254.c b/xen/arch/x86/emul-i8254.c
index 050c784702af..0e09a173187f 100644
--- a/xen/arch/x86/emul-i8254.c
+++ b/xen/arch/x86/emul-i8254.c
@@ -48,9 +48,9 @@
#define RW_STATE_WORD0 3
#define RW_STATE_WORD1 4
-static int handle_pit_io(
+static int cf_check handle_pit_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val);
-static int handle_speaker_io(
+static int cf_check handle_speaker_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val);
#define get_guest_time(v) \
@@ -505,7 +505,7 @@ void pit_deinit(struct domain *d)
}
/* the intercept action for PIT DM retval:0--not handled; 1--handled */
-static int handle_pit_io(
+static int cf_check handle_pit_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct PITState *vpit = vcpu_vpit(current);
@@ -548,7 +548,7 @@ static uint32_t speaker_ioport_read(
(pit_get_out(pit, 2) << 5) | (refresh_clock << 4));
}
-static int handle_speaker_io(
+static int cf_check handle_speaker_io(
int dir, unsigned int port, uint32_t bytes, uint32_t *val)
{
struct PITState *vpit = vcpu_vpit(current);
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 2b3fb4d6ba05..39dac7fd9d6d 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -71,19 +71,17 @@ static void hvmtrace_io_assist(const ioreq_t *p)
trace_var(event, 0/*!cycles*/, size, buffer);
}
-static int null_read(const struct hvm_io_handler *io_handler,
- uint64_t addr,
- uint32_t size,
- uint64_t *data)
+static int cf_check null_read(
+ const struct hvm_io_handler *io_handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
*data = ~0ul;
return X86EMUL_OKAY;
}
-static int null_write(const struct hvm_io_handler *handler,
- uint64_t addr,
- uint32_t size,
- uint64_t data)
+static int cf_check null_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
return X86EMUL_OKAY;
}
@@ -114,10 +112,9 @@ static const struct hvm_io_handler null_handler = {
.ops = &null_ops
};
-static int ioreq_server_read(const struct hvm_io_handler *io_handler,
- uint64_t addr,
- uint32_t size,
- uint64_t *data)
+static int cf_check ioreq_server_read(
+ const struct hvm_io_handler *io_handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
if ( hvm_copy_from_guest_phys(data, addr, size) != HVMTRANS_okay )
return X86EMUL_UNHANDLEABLE;
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 8267f0b8a278..7bdb51cfa1c4 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -162,7 +162,7 @@ static inline int hpet_check_access_length(
return 0;
}
-static int hpet_read(
+static int cf_check hpet_read(
struct vcpu *v, unsigned long addr, unsigned int length,
unsigned long *pval)
{
@@ -351,7 +351,7 @@ static void timer_sanitize_int_route(HPETState *h, unsigned int tn)
HPET_TN_ROUTE);
}
-static int hpet_write(
+static int cf_check hpet_write(
struct vcpu *v, unsigned long addr,
unsigned int length, unsigned long val)
{
@@ -569,7 +569,7 @@ static int hpet_write(
return X86EMUL_OKAY;
}
-static int hpet_range(struct vcpu *v, unsigned long addr)
+static int cf_check hpet_range(struct vcpu *v, unsigned long addr)
{
return ( (addr >= HPET_BASE_ADDRESS) &&
(addr < (HPET_BASE_ADDRESS + HPET_MMAP_SIZE)) );
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 5ec10f30803e..9e4924649077 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -543,7 +543,7 @@ void hvm_do_resume(struct vcpu *v)
}
}
-static int hvm_print_line(
+static int cf_check hvm_print_line(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct domain *cd = current->domain;
diff --git a/xen/arch/x86/hvm/intercept.c b/xen/arch/x86/hvm/intercept.c
index 02ca3b05b05a..ffa31b746716 100644
--- a/xen/arch/x86/hvm/intercept.c
+++ b/xen/arch/x86/hvm/intercept.c
@@ -32,8 +32,8 @@
#include <xen/event.h>
#include <xen/iommu.h>
-static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
- const ioreq_t *p)
+static bool cf_check hvm_mmio_accept(
+ const struct hvm_io_handler *handler, const ioreq_t *p)
{
paddr_t first = ioreq_mmio_first_byte(p), last;
@@ -51,16 +51,18 @@ static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
return 1;
}
-static int hvm_mmio_read(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t *data)
+static int cf_check hvm_mmio_read(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
BUG_ON(handler->type != IOREQ_TYPE_COPY);
return handler->mmio.ops->read(current, addr, size, data);
}
-static int hvm_mmio_write(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t data)
+static int cf_check hvm_mmio_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
BUG_ON(handler->type != IOREQ_TYPE_COPY);
@@ -73,8 +75,8 @@ static const struct hvm_io_ops mmio_ops = {
.write = hvm_mmio_write
};
-static bool_t hvm_portio_accept(const struct hvm_io_handler *handler,
- const ioreq_t *p)
+static bool cf_check hvm_portio_accept(
+ const struct hvm_io_handler *handler, const ioreq_t *p)
{
unsigned int start = handler->portio.port;
unsigned int end = start + handler->portio.size;
@@ -84,8 +86,9 @@ static bool_t hvm_portio_accept(const struct hvm_io_handler *handler,
return (p->addr >= start) && ((p->addr + p->size) <= end);
}
-static int hvm_portio_read(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t *data)
+static int cf_check hvm_portio_read(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
uint32_t val = ~0u;
int rc;
@@ -98,8 +101,9 @@ static int hvm_portio_read(const struct hvm_io_handler *handler,
return rc;
}
-static int hvm_portio_write(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t data)
+static int cf_check hvm_portio_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
uint32_t val = data;
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 93f1d1503fa6..f70bfde90143 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -156,8 +156,8 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
return true;
}
-static bool_t g2m_portio_accept(const struct hvm_io_handler *handler,
- const ioreq_t *p)
+static bool cf_check g2m_portio_accept(
+ const struct hvm_io_handler *handler, const ioreq_t *p)
{
struct vcpu *curr = current;
const struct hvm_domain *hvm = &curr->domain->arch.hvm;
@@ -179,8 +179,9 @@ static bool_t g2m_portio_accept(const struct hvm_io_handler *handler,
return 0;
}
-static int g2m_portio_read(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t *data)
+static int cf_check g2m_portio_read(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
const struct g2m_ioport *g2m_ioport = hvio->g2m_ioport;
@@ -204,8 +205,9 @@ static int g2m_portio_read(const struct hvm_io_handler *handler,
return X86EMUL_OKAY;
}
-static int g2m_portio_write(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t data)
+static int cf_check g2m_portio_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
const struct g2m_ioport *g2m_ioport = hvio->g2m_ioport;
@@ -261,14 +263,15 @@ unsigned int hvm_pci_decode_addr(unsigned int cf8, unsigned int addr,
}
/* vPCI config space IO ports handlers (0xcf8/0xcfc). */
-static bool vpci_portio_accept(const struct hvm_io_handler *handler,
- const ioreq_t *p)
+static bool cf_check vpci_portio_accept(
+ const struct hvm_io_handler *handler, const ioreq_t *p)
{
return (p->addr == 0xcf8 && p->size == 4) || (p->addr & ~3) == 0xcfc;
}
-static int vpci_portio_read(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t *data)
+static int cf_check vpci_portio_read(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t *data)
{
const struct domain *d = current->domain;
unsigned int reg;
@@ -299,8 +302,9 @@ static int vpci_portio_read(const struct hvm_io_handler *handler,
return X86EMUL_OKAY;
}
-static int vpci_portio_write(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t data)
+static int cf_check vpci_portio_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
struct domain *d = current->domain;
unsigned int reg;
@@ -387,7 +391,7 @@ static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg,
return addr & (PCI_CFG_SPACE_EXP_SIZE - 1);
}
-static int vpci_mmcfg_accept(struct vcpu *v, unsigned long addr)
+static int cf_check vpci_mmcfg_accept(struct vcpu *v, unsigned long addr)
{
struct domain *d = v->domain;
bool found;
@@ -399,8 +403,8 @@ static int vpci_mmcfg_accept(struct vcpu *v, unsigned long addr)
return found;
}
-static int vpci_mmcfg_read(struct vcpu *v, unsigned long addr,
- unsigned int len, unsigned long *data)
+static int cf_check vpci_mmcfg_read(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
{
struct domain *d = v->domain;
const struct hvm_mmcfg *mmcfg;
@@ -426,8 +430,8 @@ static int vpci_mmcfg_read(struct vcpu *v, unsigned long addr,
return X86EMUL_OKAY;
}
-static int vpci_mmcfg_write(struct vcpu *v, unsigned long addr,
- unsigned int len, unsigned long data)
+static int cf_check vpci_mmcfg_write(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
{
struct domain *d = v->domain;
const struct hvm_mmcfg *mmcfg;
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 02ad9db56523..8409d910d689 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -319,7 +319,7 @@ bool arch_ioreq_server_get_type_addr(const struct domain *d,
return true;
}
-static int hvm_access_cf8(
+static int cf_check hvm_access_cf8(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct domain *d = current->domain;
diff --git a/xen/arch/x86/hvm/pmtimer.c b/xen/arch/x86/hvm/pmtimer.c
index 808819d1de91..60e3c8de4c6b 100644
--- a/xen/arch/x86/hvm/pmtimer.c
+++ b/xen/arch/x86/hvm/pmtimer.c
@@ -152,7 +152,7 @@ static void cf_check pmt_timer_callback(void *opaque)
}
/* Handle port I/O to the PM1a_STS and PM1a_EN registers */
-static int handle_evt_io(
+static int cf_check handle_evt_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct vcpu *v = current;
@@ -216,7 +216,7 @@ static int handle_evt_io(
/* Handle port I/O to the TMR_VAL register */
-static int handle_pmt_io(
+static int cf_check handle_pmt_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct vcpu *v = current;
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index 09d3501276bc..bdc647e433e9 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -696,7 +696,7 @@ static uint32_t rtc_ioport_read(RTCState *s, uint32_t addr)
return ret;
}
-static int handle_rtc_io(
+static int cf_check handle_rtc_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct RTCState *vrtc = vcpu_vrtc(current);
@@ -809,8 +809,8 @@ void rtc_reset(struct domain *d)
}
/* RTC mediator for HVM hardware domain. */
-static int hw_rtc_io(int dir, unsigned int port, unsigned int size,
- uint32_t *val)
+static int cf_check hw_rtc_io(
+ int dir, unsigned int port, unsigned int size, uint32_t *val)
{
if ( dir == IOREQ_READ )
*val = ~0;
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index ab9781d82a55..be8200c8d072 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -199,7 +199,7 @@ static void stdvga_out(uint32_t port, uint32_t bytes, uint32_t val)
}
}
-static int stdvga_intercept_pio(
+static int cf_check stdvga_intercept_pio(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct hvm_hw_stdvga *s = ¤t->domain->arch.hvm.stdvga;
@@ -302,8 +302,9 @@ static uint8_t stdvga_mem_readb(uint64_t addr)
return ret;
}
-static int stdvga_mem_read(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size, uint64_t *p_data)
+static int cf_check stdvga_mem_read(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t *p_data)
{
uint64_t data = ~0ul;
@@ -453,9 +454,9 @@ static void stdvga_mem_writeb(uint64_t addr, uint32_t val)
}
}
-static int stdvga_mem_write(const struct hvm_io_handler *handler,
- uint64_t addr, uint32_t size,
- uint64_t data)
+static int cf_check stdvga_mem_write(
+ const struct hvm_io_handler *handler, uint64_t addr, uint32_t size,
+ uint64_t data)
{
struct hvm_hw_stdvga *s = ¤t->domain->arch.hvm.stdvga;
ioreq_t p = {
@@ -514,8 +515,8 @@ static int stdvga_mem_write(const struct hvm_io_handler *handler,
return ioreq_send(srv, &p, 1);
}
-static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
- const ioreq_t *p)
+static bool cf_check stdvga_mem_accept(
+ const struct hvm_io_handler *handler, const ioreq_t *p)
{
struct hvm_hw_stdvga *s = ¤t->domain->arch.hvm.stdvga;
@@ -558,7 +559,7 @@ static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
return 0;
}
-static void stdvga_mem_complete(const struct hvm_io_handler *handler)
+static void cf_check stdvga_mem_complete(const struct hvm_io_handler *handler)
{
struct hvm_hw_stdvga *s = ¤t->domain->arch.hvm.stdvga;
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index de6166241bf1..4c4ebda5e6e4 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1133,8 +1133,8 @@ static void svm_host_osvw_init(void)
spin_unlock(&osvw_lock);
}
-static int acpi_c1e_quirk(int dir, unsigned int port, unsigned int bytes,
- uint32_t *val)
+static int cf_check acpi_c1e_quirk(
+ int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
ASSERT(bytes == 1 && port == acpi_smi_cmd);
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 553c0f76eff8..b56549aa22d1 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -135,7 +135,7 @@ static uint32_t vioapic_read_indirect(const struct hvm_vioapic *vioapic)
return result;
}
-static int vioapic_read(
+static int cf_check vioapic_read(
struct vcpu *v, unsigned long addr,
unsigned int length, unsigned long *pval)
{
@@ -351,7 +351,7 @@ static void vioapic_write_indirect(
}
}
-static int vioapic_write(
+static int cf_check vioapic_write(
struct vcpu *v, unsigned long addr,
unsigned int length, unsigned long val)
{
@@ -383,7 +383,7 @@ static int vioapic_write(
return X86EMUL_OKAY;
}
-static int vioapic_range(struct vcpu *v, unsigned long addr)
+static int cf_check vioapic_range(struct vcpu *v, unsigned long addr)
{
return !!addr_vioapic(v->domain, addr);
}
@@ -568,7 +568,7 @@ int vioapic_get_mask(const struct domain *d, unsigned int gsi)
return vioapic->redirtbl[pin].fields.mask;
}
-int vioapic_get_vector(const struct domain *d, unsigned int gsi)
+int cf_check vioapic_get_vector(const struct domain *d, unsigned int gsi)
{
unsigned int pin = 0; /* See gsi_vioapic */
const struct hvm_vioapic *vioapic = gsi_vioapic(d, gsi, &pin);
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index fe375912bef1..652e3cb87f12 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -615,8 +615,9 @@ static uint32_t vlapic_read_aligned(const struct vlapic *vlapic,
return 0;
}
-static int vlapic_mmio_read(struct vcpu *v, unsigned long address,
- unsigned int len, unsigned long *pval)
+static int cf_check vlapic_mmio_read(
+ struct vcpu *v, unsigned long address, unsigned int len,
+ unsigned long *pval)
{
struct vlapic *vlapic = vcpu_vlapic(v);
unsigned int offset = address - vlapic_base_address(vlapic);
@@ -898,8 +899,8 @@ void vlapic_reg_write(struct vcpu *v, unsigned int reg, uint32_t val)
}
}
-static int vlapic_mmio_write(struct vcpu *v, unsigned long address,
- unsigned int len, unsigned long val)
+static int cf_check vlapic_mmio_write(
+ struct vcpu *v, unsigned long address, unsigned int len, unsigned long val)
{
struct vlapic *vlapic = vcpu_vlapic(v);
unsigned int offset = address - vlapic_base_address(vlapic);
@@ -1052,7 +1053,7 @@ int guest_wrmsr_x2apic(struct vcpu *v, uint32_t msr, uint64_t msr_content)
return X86EMUL_OKAY;
}
-static int vlapic_range(struct vcpu *v, unsigned long addr)
+static int cf_check vlapic_range(struct vcpu *v, unsigned long addr)
{
struct vlapic *vlapic = vcpu_vlapic(v);
unsigned long offset = addr - vlapic_base_address(vlapic);
diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 2889575a2035..d4a8c953e23f 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -211,8 +211,9 @@ static struct msi_desc *msixtbl_addr_to_desc(
return NULL;
}
-static int msixtbl_read(const struct hvm_io_handler *handler,
- uint64_t address, uint32_t len, uint64_t *pval)
+static int cf_check msixtbl_read(
+ const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
+ uint64_t *pval)
{
unsigned long offset;
struct msixtbl_entry *entry;
@@ -350,14 +351,15 @@ static int msixtbl_write(struct vcpu *v, unsigned long address,
return r;
}
-static int _msixtbl_write(const struct hvm_io_handler *handler,
- uint64_t address, uint32_t len, uint64_t val)
+static int cf_check _msixtbl_write(
+ const struct hvm_io_handler *handler, uint64_t address, uint32_t len,
+ uint64_t val)
{
return msixtbl_write(current, address, len, val);
}
-static bool_t msixtbl_range(const struct hvm_io_handler *handler,
- const ioreq_t *r)
+static bool cf_check msixtbl_range(
+ const struct hvm_io_handler *handler, const ioreq_t *r)
{
struct vcpu *curr = current;
unsigned long addr = r->addr;
diff --git a/xen/arch/x86/hvm/vpic.c b/xen/arch/x86/hvm/vpic.c
index 91c2c6983393..5d8ef259b710 100644
--- a/xen/arch/x86/hvm/vpic.c
+++ b/xen/arch/x86/hvm/vpic.c
@@ -351,7 +351,7 @@ static uint32_t vpic_ioport_read(struct hvm_hw_vpic *vpic, uint32_t addr)
return vpic->imr;
}
-static int vpic_intercept_pic_io(
+static int cf_check vpic_intercept_pic_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct hvm_hw_vpic *vpic;
@@ -373,7 +373,7 @@ static int vpic_intercept_pic_io(
return X86EMUL_OKAY;
}
-static int vpic_intercept_elcr_io(
+static int cf_check vpic_intercept_elcr_io(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
struct hvm_hw_vpic *vpic;
diff --git a/xen/arch/x86/include/asm/hvm/vioapic.h b/xen/arch/x86/include/asm/hvm/vioapic.h
index 36b64d20d60c..2944ec20dd53 100644
--- a/xen/arch/x86/include/asm/hvm/vioapic.h
+++ b/xen/arch/x86/include/asm/hvm/vioapic.h
@@ -66,7 +66,7 @@ void vioapic_irq_positive_edge(struct domain *d, unsigned int irq);
void vioapic_update_EOI(struct domain *d, u8 vector);
int vioapic_get_mask(const struct domain *d, unsigned int gsi);
-int vioapic_get_vector(const struct domain *d, unsigned int gsi);
+int cf_check vioapic_get_vector(const struct domain *d, unsigned int gsi);
int vioapic_get_trigger_mode(const struct domain *d, unsigned int gsi);
#endif /* __ASM_X86_HVM_VIOAPIC_H__ */
diff --git a/xen/drivers/passthrough/amd/iommu_guest.c b/xen/drivers/passthrough/amd/iommu_guest.c
index 361ff864d846..80a331f546ed 100644
--- a/xen/drivers/passthrough/amd/iommu_guest.c
+++ b/xen/drivers/passthrough/amd/iommu_guest.c
@@ -645,8 +645,8 @@ static uint64_t iommu_mmio_read64(struct guest_iommu *iommu,
return val;
}
-static int guest_iommu_mmio_read(struct vcpu *v, unsigned long addr,
- unsigned int len, unsigned long *pval)
+static int cf_check guest_iommu_mmio_read(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *pval)
{
struct guest_iommu *iommu = vcpu_iommu(v);
unsigned long offset;
@@ -735,8 +735,8 @@ static void guest_iommu_mmio_write64(struct guest_iommu *iommu,
}
}
-static int guest_iommu_mmio_write(struct vcpu *v, unsigned long addr,
- unsigned int len, unsigned long val)
+static int cf_check guest_iommu_mmio_write(
+ struct vcpu *v, unsigned long addr, unsigned int len, unsigned long val)
{
struct guest_iommu *iommu = vcpu_iommu(v);
unsigned long offset;
@@ -819,7 +819,7 @@ static void guest_iommu_reg_init(struct guest_iommu *iommu)
iommu->reg_ext_feature = ef;
}
-static int guest_iommu_mmio_range(struct vcpu *v, unsigned long addr)
+static int cf_check guest_iommu_mmio_range(struct vcpu *v, unsigned long addr)
{
struct guest_iommu *iommu = vcpu_iommu(v);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (32 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 33/70] x86/hvm: CFI hardening for device emulation Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 13:38 ` Jan Beulich
2022-02-14 12:50 ` [PATCH v2 35/70] x86/ucode: " Andrew Cooper
` (37 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
pv_emul_is_mem_write() is only used in a single file. Having it as a static
inline is pointless because it can't be inlined to begin with.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v2:
* Correct details in commit message.
---
xen/arch/x86/hvm/emulate.c | 72 +++++++++++++++++-----------------
xen/arch/x86/hvm/hvm.c | 8 ++--
xen/arch/x86/hvm/svm/svm.c | 4 +-
xen/arch/x86/include/asm/hvm/emulate.h | 8 ++--
xen/arch/x86/include/asm/mm.h | 16 +++-----
xen/arch/x86/mm.c | 4 +-
xen/arch/x86/mm/shadow/hvm.c | 8 ++--
xen/arch/x86/pv/emul-gate-op.c | 9 +++--
xen/arch/x86/pv/emul-priv-op.c | 64 +++++++++++++++---------------
xen/arch/x86/pv/emulate.h | 7 ----
xen/arch/x86/pv/ro-page-fault.c | 31 +++++++++------
xen/arch/x86/x86_emulate.c | 21 +++++-----
xen/arch/x86/x86_emulate/x86_emulate.c | 10 ++---
xen/arch/x86/x86_emulate/x86_emulate.h | 33 ++++++++--------
14 files changed, 148 insertions(+), 147 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 39dac7fd9d6d..e8d510e0be91 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -1272,7 +1272,7 @@ static int __hvmemul_read(
return linear_read(addr, bytes, p_data, pfec, hvmemul_ctxt);
}
-static int hvmemul_read(
+static int cf_check hvmemul_read(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1290,7 +1290,7 @@ static int hvmemul_read(
container_of(ctxt, struct hvm_emulate_ctxt, ctxt));
}
-int hvmemul_insn_fetch(
+int cf_check hvmemul_insn_fetch(
unsigned long offset,
void *p_data,
unsigned int bytes,
@@ -1336,7 +1336,7 @@ int hvmemul_insn_fetch(
return X86EMUL_OKAY;
}
-static int hvmemul_write(
+static int cf_check hvmemul_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1384,7 +1384,7 @@ static int hvmemul_write(
return X86EMUL_OKAY;
}
-static int hvmemul_rmw(
+static int cf_check hvmemul_rmw(
enum x86_segment seg,
unsigned long offset,
unsigned int bytes,
@@ -1437,7 +1437,7 @@ static int hvmemul_rmw(
return rc;
}
-static int hvmemul_blk(
+static int cf_check hvmemul_blk(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1478,7 +1478,7 @@ static int hvmemul_blk(
return rc;
}
-static int hvmemul_write_discard(
+static int cf_check hvmemul_write_discard(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1489,7 +1489,7 @@ static int hvmemul_write_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_ins_discard(
+static int cf_check hvmemul_rep_ins_discard(
uint16_t src_port,
enum x86_segment dst_seg,
unsigned long dst_offset,
@@ -1500,7 +1500,7 @@ static int hvmemul_rep_ins_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_movs_discard(
+static int cf_check hvmemul_rep_movs_discard(
enum x86_segment src_seg,
unsigned long src_offset,
enum x86_segment dst_seg,
@@ -1512,7 +1512,7 @@ static int hvmemul_rep_movs_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_stos_discard(
+static int cf_check hvmemul_rep_stos_discard(
void *p_data,
enum x86_segment seg,
unsigned long offset,
@@ -1523,7 +1523,7 @@ static int hvmemul_rep_stos_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_outs_discard(
+static int cf_check hvmemul_rep_outs_discard(
enum x86_segment src_seg,
unsigned long src_offset,
uint16_t dst_port,
@@ -1534,7 +1534,7 @@ static int hvmemul_rep_outs_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cmpxchg_discard(
+static int cf_check hvmemul_cmpxchg_discard(
enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -1546,7 +1546,7 @@ static int hvmemul_cmpxchg_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_read_io_discard(
+static int cf_check hvmemul_read_io_discard(
unsigned int port,
unsigned int bytes,
unsigned long *val,
@@ -1555,7 +1555,7 @@ static int hvmemul_read_io_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_write_io_discard(
+static int cf_check hvmemul_write_io_discard(
unsigned int port,
unsigned int bytes,
unsigned long val,
@@ -1564,7 +1564,7 @@ static int hvmemul_write_io_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_write_msr_discard(
+static int cf_check hvmemul_write_msr_discard(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -1572,7 +1572,7 @@ static int hvmemul_write_msr_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cache_op_discard(
+static int cf_check hvmemul_cache_op_discard(
enum x86emul_cache_op op,
enum x86_segment seg,
unsigned long offset,
@@ -1581,7 +1581,7 @@ static int hvmemul_cache_op_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cmpxchg(
+static int cf_check hvmemul_cmpxchg(
enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -1675,7 +1675,7 @@ static int hvmemul_cmpxchg(
return rc;
}
-static int hvmemul_validate(
+static int cf_check hvmemul_validate(
const struct x86_emulate_state *state,
struct x86_emulate_ctxt *ctxt)
{
@@ -1688,7 +1688,7 @@ static int hvmemul_validate(
? X86EMUL_OKAY : X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_rep_ins(
+static int cf_check hvmemul_rep_ins(
uint16_t src_port,
enum x86_segment dst_seg,
unsigned long dst_offset,
@@ -1766,7 +1766,7 @@ static int hvmemul_rep_outs_set_context(
return rc;
}
-static int hvmemul_rep_outs(
+static int cf_check hvmemul_rep_outs(
enum x86_segment src_seg,
unsigned long src_offset,
uint16_t dst_port,
@@ -1807,7 +1807,7 @@ static int hvmemul_rep_outs(
!!(ctxt->regs->eflags & X86_EFLAGS_DF), gpa);
}
-static int hvmemul_rep_movs(
+static int cf_check hvmemul_rep_movs(
enum x86_segment src_seg,
unsigned long src_offset,
enum x86_segment dst_seg,
@@ -1977,7 +1977,7 @@ static int hvmemul_rep_movs(
return X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_rep_stos(
+static int cf_check hvmemul_rep_stos(
void *p_data,
enum x86_segment seg,
unsigned long offset,
@@ -2105,7 +2105,7 @@ static int hvmemul_rep_stos(
}
}
-static int hvmemul_read_segment(
+static int cf_check hvmemul_read_segment(
enum x86_segment seg,
struct segment_register *reg,
struct x86_emulate_ctxt *ctxt)
@@ -2122,7 +2122,7 @@ static int hvmemul_read_segment(
return X86EMUL_OKAY;
}
-static int hvmemul_write_segment(
+static int cf_check hvmemul_write_segment(
enum x86_segment seg,
const struct segment_register *reg,
struct x86_emulate_ctxt *ctxt)
@@ -2141,7 +2141,7 @@ static int hvmemul_write_segment(
return X86EMUL_OKAY;
}
-static int hvmemul_read_io(
+static int cf_check hvmemul_read_io(
unsigned int port,
unsigned int bytes,
unsigned long *val,
@@ -2158,7 +2158,7 @@ static int hvmemul_read_io(
return hvmemul_do_pio_buffer(port, bytes, IOREQ_READ, val);
}
-static int hvmemul_write_io(
+static int cf_check hvmemul_write_io(
unsigned int port,
unsigned int bytes,
unsigned long val,
@@ -2167,7 +2167,7 @@ static int hvmemul_write_io(
return hvmemul_do_pio_buffer(port, bytes, IOREQ_WRITE, &val);
}
-static int hvmemul_read_cr(
+static int cf_check hvmemul_read_cr(
unsigned int reg,
unsigned long *val,
struct x86_emulate_ctxt *ctxt)
@@ -2188,7 +2188,7 @@ static int hvmemul_read_cr(
return X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_write_cr(
+static int cf_check hvmemul_write_cr(
unsigned int reg,
unsigned long val,
struct x86_emulate_ctxt *ctxt)
@@ -2232,7 +2232,7 @@ static int hvmemul_write_cr(
return rc;
}
-static int hvmemul_read_xcr(
+static int cf_check hvmemul_read_xcr(
unsigned int reg,
uint64_t *val,
struct x86_emulate_ctxt *ctxt)
@@ -2245,7 +2245,7 @@ static int hvmemul_read_xcr(
return rc;
}
-static int hvmemul_write_xcr(
+static int cf_check hvmemul_write_xcr(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -2255,7 +2255,7 @@ static int hvmemul_write_xcr(
return x86emul_write_xcr(reg, val, ctxt);
}
-static int hvmemul_read_msr(
+static int cf_check hvmemul_read_msr(
unsigned int reg,
uint64_t *val,
struct x86_emulate_ctxt *ctxt)
@@ -2268,7 +2268,7 @@ static int hvmemul_read_msr(
return rc;
}
-static int hvmemul_write_msr(
+static int cf_check hvmemul_write_msr(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -2281,7 +2281,7 @@ static int hvmemul_write_msr(
return rc;
}
-static int hvmemul_cache_op(
+static int cf_check hvmemul_cache_op(
enum x86emul_cache_op op,
enum x86_segment seg,
unsigned long offset,
@@ -2353,7 +2353,7 @@ static int hvmemul_cache_op(
return X86EMUL_OKAY;
}
-static int hvmemul_get_fpu(
+static int cf_check hvmemul_get_fpu(
enum x86_emulate_fpu_type type,
struct x86_emulate_ctxt *ctxt)
{
@@ -2395,7 +2395,7 @@ static int hvmemul_get_fpu(
return X86EMUL_OKAY;
}
-static void hvmemul_put_fpu(
+static void cf_check hvmemul_put_fpu(
struct x86_emulate_ctxt *ctxt,
enum x86_emulate_fpu_type backout,
const struct x86_emul_fpu_aux *aux)
@@ -2482,7 +2482,7 @@ static void hvmemul_put_fpu(
}
}
-static int hvmemul_tlb_op(
+static int cf_check hvmemul_tlb_op(
enum x86emul_tlb_op op,
unsigned long addr,
unsigned long aux,
@@ -2539,7 +2539,7 @@ static int hvmemul_tlb_op(
return rc;
}
-static int hvmemul_vmfunc(
+static int cf_check hvmemul_vmfunc(
struct x86_emulate_ctxt *ctxt)
{
int rc;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9e4924649077..e87e809a945d 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3755,8 +3755,8 @@ void hvm_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
}
}
-static bool is_sysdesc_access(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_sysdesc_access(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
unsigned int ext;
int mode = x86_insn_modrm(state, NULL, &ext);
@@ -3796,8 +3796,8 @@ int hvm_descriptor_access_intercept(uint64_t exit_info,
return X86EMUL_OKAY;
}
-static bool is_cross_vendor(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_cross_vendor(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
switch ( ctxt->opcode )
{
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 4c4ebda5e6e4..dedb2848e6a1 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2447,8 +2447,8 @@ static void svm_invlpg_intercept(unsigned long linear)
paging_invlpg(current, linear);
}
-static bool is_invlpg(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_invlpg(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
unsigned int ext;
diff --git a/xen/arch/x86/include/asm/hvm/emulate.h b/xen/arch/x86/include/asm/hvm/emulate.h
index e67004060345..d8ba2df4e4a2 100644
--- a/xen/arch/x86/include/asm/hvm/emulate.h
+++ b/xen/arch/x86/include/asm/hvm/emulate.h
@@ -92,10 +92,10 @@ static inline bool handle_mmio(void)
return hvm_emulate_one_insn(x86_insn_is_mem_access, "MMIO");
}
-int hvmemul_insn_fetch(unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
+int cf_check hvmemul_insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt);
+
int hvmemul_do_pio_buffer(uint16_t port,
unsigned int size,
uint8_t dir,
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 9b9de4c6bef7..a684ffabfaae 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -538,16 +538,12 @@ struct mmio_ro_emulate_ctxt {
unsigned int seg, bdf;
};
-extern int mmio_ro_emulated_write(enum x86_segment seg,
- unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
-extern int mmcfg_intercept_write(enum x86_segment seg,
- unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
+int cf_check mmio_ro_emulated_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt);
+int cf_check mmcfg_intercept_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt);
int audit_adjust_pgtables(struct domain *d, int dir, int noisy);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 4b6956c5be78..4d0bebbf8aab 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4852,7 +4852,7 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
return 0;
}
-int mmio_ro_emulated_write(
+int cf_check mmio_ro_emulated_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -4873,7 +4873,7 @@ int mmio_ro_emulated_write(
return X86EMUL_OKAY;
}
-int mmcfg_intercept_write(
+int cf_check mmcfg_intercept_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c
index f2991bc176f0..c90d326becb3 100644
--- a/xen/arch/x86/mm/shadow/hvm.c
+++ b/xen/arch/x86/mm/shadow/hvm.c
@@ -148,7 +148,7 @@ hvm_read(enum x86_segment seg,
return X86EMUL_UNHANDLEABLE;
}
-static int
+static int cf_check
hvm_emulate_read(enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -161,7 +161,7 @@ hvm_emulate_read(enum x86_segment seg,
container_of(ctxt, struct sh_emulate_ctxt, ctxt));
}
-static int
+static int cf_check
hvm_emulate_insn_fetch(unsigned long offset,
void *p_data,
unsigned int bytes,
@@ -181,7 +181,7 @@ hvm_emulate_insn_fetch(unsigned long offset,
return X86EMUL_OKAY;
}
-static int
+static int cf_check
hvm_emulate_write(enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -234,7 +234,7 @@ hvm_emulate_write(enum x86_segment seg,
return X86EMUL_OKAY;
}
-static int
+static int cf_check
hvm_emulate_cmpxchg(enum x86_segment seg,
unsigned long offset,
void *p_old,
diff --git a/xen/arch/x86/pv/emul-gate-op.c b/xen/arch/x86/pv/emul-gate-op.c
index 68ec4d11f6bb..758a20ad9df4 100644
--- a/xen/arch/x86/pv/emul-gate-op.c
+++ b/xen/arch/x86/pv/emul-gate-op.c
@@ -96,8 +96,9 @@ struct gate_op_ctxt {
bool insn_fetch;
};
-static int read_mem(enum x86_segment seg, unsigned long offset, void *p_data,
- unsigned int bytes, struct x86_emulate_ctxt *ctxt)
+static int cf_check read_mem(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
const struct gate_op_ctxt *goc =
container_of(ctxt, struct gate_op_ctxt, ctxt);
@@ -163,8 +164,8 @@ static int read_mem(enum x86_segment seg, unsigned long offset, void *p_data,
return X86EMUL_OKAY;
}
-static int fetch(unsigned long offset, void *p_data,
- unsigned int bytes, struct x86_emulate_ctxt *ctxt)
+static int cf_check fetch(unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
return read_mem(x86_seg_cs, offset, p_data, bytes, ctxt);
}
diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index c78be6d92b21..c46c072f93db 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -358,8 +358,9 @@ static unsigned int check_guest_io_breakpoint(struct vcpu *v,
return match;
}
-static int read_io(unsigned int port, unsigned int bytes,
- unsigned long *val, struct x86_emulate_ctxt *ctxt)
+static int cf_check read_io(
+ unsigned int port, unsigned int bytes, unsigned long *val,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -462,8 +463,9 @@ static void guest_io_write(unsigned int port, unsigned int bytes,
}
}
-static int write_io(unsigned int port, unsigned int bytes,
- unsigned long val, struct x86_emulate_ctxt *ctxt)
+static int cf_check write_io(
+ unsigned int port, unsigned int bytes, unsigned long val,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -493,9 +495,9 @@ static int write_io(unsigned int port, unsigned int bytes,
return X86EMUL_OKAY;
}
-static int read_segment(enum x86_segment seg,
- struct segment_register *reg,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_segment(
+ enum x86_segment seg, struct segment_register *reg,
+ struct x86_emulate_ctxt *ctxt)
{
/* Check if this is an attempt to access the I/O bitmap. */
if ( seg == x86_seg_tr )
@@ -607,10 +609,10 @@ static int pv_emul_virt_to_linear(unsigned long base, unsigned long offset,
return rc;
}
-static int rep_ins(uint16_t port,
- enum x86_segment seg, unsigned long offset,
- unsigned int bytes_per_rep, unsigned long *reps,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check rep_ins(
+ uint16_t port, enum x86_segment seg, unsigned long offset,
+ unsigned int bytes_per_rep, unsigned long *reps,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -675,10 +677,10 @@ static int rep_ins(uint16_t port,
return X86EMUL_OKAY;
}
-static int rep_outs(enum x86_segment seg, unsigned long offset,
- uint16_t port,
- unsigned int bytes_per_rep, unsigned long *reps,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check rep_outs(
+ enum x86_segment seg, unsigned long offset, uint16_t port,
+ unsigned int bytes_per_rep, unsigned long *reps,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -744,8 +746,8 @@ static int rep_outs(enum x86_segment seg, unsigned long offset,
return X86EMUL_OKAY;
}
-static int read_cr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_cr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt)
{
const struct vcpu *curr = current;
@@ -787,8 +789,8 @@ static int read_cr(unsigned int reg, unsigned long *val,
return X86EMUL_UNHANDLEABLE;
}
-static int write_cr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check write_cr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -871,8 +873,8 @@ static uint64_t guest_efer(const struct domain *d)
return val;
}
-static int read_msr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_msr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
const struct domain *currd = curr->domain;
@@ -1020,8 +1022,8 @@ static int read_msr(unsigned int reg, uint64_t *val,
return ret;
}
-static int write_msr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check write_msr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
const struct domain *currd = curr->domain;
@@ -1188,8 +1190,9 @@ static int write_msr(unsigned int reg, uint64_t val,
return X86EMUL_UNHANDLEABLE;
}
-static int cache_op(enum x86emul_cache_op op, enum x86_segment seg,
- unsigned long offset, struct x86_emulate_ctxt *ctxt)
+static int cf_check cache_op(
+ enum x86emul_cache_op op, enum x86_segment seg,
+ unsigned long offset, struct x86_emulate_ctxt *ctxt)
{
ASSERT(op == x86emul_wbinvd || op == x86emul_wbnoinvd);
@@ -1208,8 +1211,8 @@ static int cache_op(enum x86emul_cache_op op, enum x86_segment seg,
return X86EMUL_OKAY;
}
-static int validate(const struct x86_emulate_state *state,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check validate(
+ const struct x86_emulate_state *state, struct x86_emulate_ctxt *ctxt)
{
switch ( ctxt->opcode )
{
@@ -1258,10 +1261,9 @@ static int validate(const struct x86_emulate_state *state,
return X86EMUL_UNHANDLEABLE;
}
-static int insn_fetch(unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt)
{
const struct priv_op_ctxt *poc =
container_of(ctxt, struct priv_op_ctxt, ctxt);
diff --git a/xen/arch/x86/pv/emulate.h b/xen/arch/x86/pv/emulate.h
index 4b845b08e372..49a4d34832df 100644
--- a/xen/arch/x86/pv/emulate.h
+++ b/xen/arch/x86/pv/emulate.h
@@ -12,13 +12,6 @@ int pv_emul_read_descriptor(unsigned int sel, const struct vcpu *v,
void pv_emul_instruction_done(struct cpu_user_regs *regs, unsigned long rip);
-static inline int pv_emul_is_mem_write(const struct x86_emulate_state *state,
- struct x86_emulate_ctxt *ctxt)
-{
- return x86_insn_is_mem_write(state, ctxt) ? X86EMUL_OKAY
- : X86EMUL_UNHANDLEABLE;
-}
-
/* Return a pointer to the GDT/LDT descriptor referenced by sel. */
static inline const seg_desc_t *gdt_ldt_desc_ptr(unsigned int sel)
{
diff --git a/xen/arch/x86/pv/ro-page-fault.c b/xen/arch/x86/pv/ro-page-fault.c
index ef4d146c1d9e..5963f5ee2d51 100644
--- a/xen/arch/x86/pv/ro-page-fault.c
+++ b/xen/arch/x86/pv/ro-page-fault.c
@@ -26,6 +26,13 @@
#include "emulate.h"
#include "mm.h"
+static int cf_check pv_emul_is_mem_write(
+ const struct x86_emulate_state *state, struct x86_emulate_ctxt *ctxt)
+{
+ return x86_insn_is_mem_write(state, ctxt) ? X86EMUL_OKAY
+ : X86EMUL_UNHANDLEABLE;
+}
+
/*********************
* Writable Pagetables
*/
@@ -35,9 +42,9 @@ struct ptwr_emulate_ctxt {
l1_pgentry_t pte;
};
-static int ptwr_emulated_read(enum x86_segment seg, unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_read(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
unsigned int rc = bytes;
unsigned long addr = offset;
@@ -52,9 +59,9 @@ static int ptwr_emulated_read(enum x86_segment seg, unsigned long offset,
return X86EMUL_OKAY;
}
-static int ptwr_emulated_insn_fetch(unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt)
{
unsigned int rc = copy_from_guest_pv(p_data, (void *)offset, bytes);
@@ -218,9 +225,9 @@ static int ptwr_emulated_update(unsigned long addr, intpte_t *p_old,
return X86EMUL_OKAY;
}
-static int ptwr_emulated_write(enum x86_segment seg, unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
intpte_t val = 0;
@@ -236,9 +243,9 @@ static int ptwr_emulated_write(enum x86_segment seg, unsigned long offset,
return ptwr_emulated_update(offset, NULL, val, bytes, ctxt);
}
-static int ptwr_emulated_cmpxchg(enum x86_segment seg, unsigned long offset,
- void *p_old, void *p_new, unsigned int bytes,
- bool lock, struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_cmpxchg(
+ enum x86_segment seg, unsigned long offset, void *p_old, void *p_new,
+ unsigned int bytes, bool lock, struct x86_emulate_ctxt *ctxt)
{
intpte_t old = 0, new = 0;
int rc;
diff --git a/xen/arch/x86/x86_emulate.c b/xen/arch/x86/x86_emulate.c
index 1e082e6f3b2d..60191a94dc18 100644
--- a/xen/arch/x86/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate.c
@@ -53,8 +53,8 @@
#include "x86_emulate/x86_emulate.c"
-int x86emul_read_xcr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_read_xcr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt)
{
switch ( reg )
{
@@ -77,8 +77,8 @@ int x86emul_read_xcr(unsigned int reg, uint64_t *val,
}
/* Note: May be called with ctxt=NULL. */
-int x86emul_write_xcr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_write_xcr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt)
{
switch ( reg )
{
@@ -100,8 +100,8 @@ int x86emul_write_xcr(unsigned int reg, uint64_t val,
#ifdef CONFIG_PV
/* Called with NULL ctxt in hypercall context. */
-int x86emul_read_dr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_read_dr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -143,8 +143,8 @@ int x86emul_read_dr(unsigned int reg, unsigned long *val,
return X86EMUL_OKAY;
}
-int x86emul_write_dr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_write_dr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -167,8 +167,9 @@ int x86emul_write_dr(unsigned int reg, unsigned long val,
}
#endif /* CONFIG_PV */
-int x86emul_cpuid(uint32_t leaf, uint32_t subleaf,
- struct cpuid_leaf *res, struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_cpuid(
+ uint32_t leaf, uint32_t subleaf, struct cpuid_leaf *res,
+ struct x86_emulate_ctxt *ctxt)
{
guest_cpuid(current, leaf, subleaf, res);
diff --git a/xen/arch/x86/x86_emulate/x86_emulate.c b/xen/arch/x86/x86_emulate/x86_emulate.c
index 824af9d899b5..5cdef535affd 100644
--- a/xen/arch/x86/x86_emulate/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
@@ -2524,7 +2524,7 @@ static void adjust_bnd(struct x86_emulate_ctxt *ctxt,
done:;
}
-int x86emul_unhandleable_rw(
+int cf_check x86emul_unhandleable_rw(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -12318,7 +12318,7 @@ x86_insn_operand_ea(const struct x86_emulate_state *state,
* memory operand (like POP), but it does not mean e.g. segment selector
* loads, where the descriptor table access is considered an implicit one.
*/
-bool
+bool cf_check
x86_insn_is_mem_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12410,7 +12410,7 @@ x86_insn_is_mem_access(const struct x86_emulate_state *state,
* loads, where the (possible) descriptor table write is considered an
* implicit access.
*/
-bool
+bool cf_check
x86_insn_is_mem_write(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12582,7 +12582,7 @@ x86_insn_is_mem_write(const struct x86_emulate_state *state,
return false;
}
-bool
+bool cf_check
x86_insn_is_portio(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12597,7 +12597,7 @@ x86_insn_is_portio(const struct x86_emulate_state *state,
return false;
}
-bool
+bool cf_check
x86_insn_is_cr_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
diff --git a/xen/arch/x86/x86_emulate/x86_emulate.h b/xen/arch/x86/x86_emulate/x86_emulate.h
index 419def8790a0..4732855c40ed 100644
--- a/xen/arch/x86/x86_emulate/x86_emulate.h
+++ b/xen/arch/x86/x86_emulate/x86_emulate.h
@@ -737,7 +737,7 @@ static inline unsigned long *decode_gpr(struct cpu_user_regs *regs,
}
/* Unhandleable read, write or instruction fetch */
-int
+int cf_check
x86emul_unhandleable_rw(
enum x86_segment seg,
unsigned long offset,
@@ -766,16 +766,16 @@ x86_insn_immediate(const struct x86_emulate_state *state,
unsigned int
x86_insn_length(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_mem_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_mem_write(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_portio(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_cr_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
@@ -787,17 +787,18 @@ void x86_emulate_free_state(struct x86_emulate_state *state);
#ifdef __XEN__
-int x86emul_read_xcr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_write_xcr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt);
-
-int x86emul_read_dr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_write_dr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_cpuid(uint32_t leaf, uint32_t subleaf,
- struct cpuid_leaf *res, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_read_xcr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_write_xcr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt);
+
+int cf_check x86emul_read_dr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_write_dr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_cpuid(
+ uint32_t leaf, uint32_t subleaf, struct cpuid_leaf *res,
+ struct x86_emulate_ctxt *ctxt);
#endif
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-14 12:50 ` [PATCH v2 34/70] x86/emul: CFI hardening Andrew Cooper
@ 2022-02-14 13:38 ` Jan Beulich
2022-02-15 13:43 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-14 13:38 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Xen-devel
On 14.02.2022 13:50, Andrew Cooper wrote:
> Control Flow Integrity schemes use toolchain and optionally hardware support
> to help protect against call/jump/return oriented programming attacks.
>
> Use cf_check to annotate function pointer targets for the toolchain.
>
> pv_emul_is_mem_write() is only used in a single file. Having it as a static
> inline is pointless because it can't be inlined to begin with.
I'd like you to consider to re-word this: It being static inline was for
the case of there appearing a 2nd user. I don't view such as pointless.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-14 13:38 ` Jan Beulich
@ 2022-02-15 13:43 ` Andrew Cooper
2022-02-15 14:13 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-15 13:43 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper; +Cc: Xen-devel
On 14/02/2022 13:38, Jan Beulich wrote:
> On 14.02.2022 13:50, Andrew Cooper wrote:
>> Control Flow Integrity schemes use toolchain and optionally hardware support
>> to help protect against call/jump/return oriented programming attacks.
>>
>> Use cf_check to annotate function pointer targets for the toolchain.
>>
>> pv_emul_is_mem_write() is only used in a single file. Having it as a static
>> inline is pointless because it can't be inlined to begin with.
> I'd like you to consider to re-word this:
This is the reworded version.
> It being static inline was for
> the case of there appearing a 2nd user. I don't view such as pointless.
I find that impossible to reconcile with your normal review feedback.
It is unconditionally forced out of line because of how it's used,
meaning that if it ever got used in a second translation unit we'd end
up with a duplicate function, at which point it would need to be
non-static and exported to pass review. (And sanity.)
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-15 13:43 ` Andrew Cooper
@ 2022-02-15 14:13 ` Jan Beulich
2022-02-16 21:34 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 14:13 UTC (permalink / raw)
To: Andrew Cooper, Andrew Cooper; +Cc: Xen-devel
On 15.02.2022 14:43, Andrew Cooper wrote:
> On 14/02/2022 13:38, Jan Beulich wrote:
>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>> Control Flow Integrity schemes use toolchain and optionally hardware support
>>> to help protect against call/jump/return oriented programming attacks.
>>>
>>> Use cf_check to annotate function pointer targets for the toolchain.
>>>
>>> pv_emul_is_mem_write() is only used in a single file. Having it as a static
>>> inline is pointless because it can't be inlined to begin with.
>> I'd like you to consider to re-word this:
>
> This is the reworded version.
>
>> It being static inline was for
>> the case of there appearing a 2nd user. I don't view such as pointless.
>
> I find that impossible to reconcile with your normal review feedback.
Interesting. I don't think I would have objected to something like
this, if it was conceivable that a 2nd user may appear. I don't
think this is the only inline function we've got with just a single
user. I also don't think this is the only inline function we've got
with its address taken, and hence having an out-of-line instantiation.
> It is unconditionally forced out of line because of how it's used,
> meaning that if it ever got used in a second translation unit we'd end
> up with a duplicate function, at which point it would need to be
> non-static and exported to pass review. (And sanity.)
I'm afraid you've lost me here. What duplicate function? Before and
after the patch the function is static; what changes is merely the
"inline". Two CUs can have identically named static functions, can't
they? Or if that's not the point you try to make, then I have no idea
what it is that you're trying to tell me.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-15 14:13 ` Jan Beulich
@ 2022-02-16 21:34 ` Andrew Cooper
2022-02-17 11:49 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 21:34 UTC (permalink / raw)
To: Jan Beulich; +Cc: Xen-devel
On 15/02/2022 14:13, Jan Beulich wrote:
> On 15.02.2022 14:43, Andrew Cooper wrote:
>> On 14/02/2022 13:38, Jan Beulich wrote:
>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>> Control Flow Integrity schemes use toolchain and optionally hardware support
>>>> to help protect against call/jump/return oriented programming attacks.
>>>>
>>>> Use cf_check to annotate function pointer targets for the toolchain.
>>>>
>>>> pv_emul_is_mem_write() is only used in a single file. Having it as a static
>>>> inline is pointless because it can't be inlined to begin with.
>>> I'd like you to consider to re-word this:
>> This is the reworded version.
>>
>>> It being static inline was for
>>> the case of there appearing a 2nd user. I don't view such as pointless.
>> I find that impossible to reconcile with your normal review feedback.
> Interesting. I don't think I would have objected to something like
> this, if it was conceivable that a 2nd user may appear. I don't
> think this is the only inline function we've got with just a single
> user. I also don't think this is the only inline function we've got
> with its address taken, and hence having an out-of-line instantiation.
>
>> It is unconditionally forced out of line because of how it's used,
>> meaning that if it ever got used in a second translation unit we'd end
>> up with a duplicate function, at which point it would need to be
>> non-static and exported to pass review. (And sanity.)
> I'm afraid you've lost me here. What duplicate function? Before and
> after the patch the function is static; what changes is merely the
> "inline". Two CUs can have identically named static functions, can't
> they? Or if that's not the point you try to make, then I have no idea
> what it is that you're trying to tell me.
Yes, the same static inline can be out-of-lined in multiple translation
units. This creates two identical copies of the logic, and then falls
foul of our unique symbols constraint.
The absence of complaints in the general case shows that we don't
currently any cases where a static inline is out-of-lined in multiple
translation units.
Under IBT, it means more tagged functions, which I suppose doesn't make
a useful difference from the attackers point of view, but it's still
logic duplication in the final build that we'd prefer to avoid.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 34/70] x86/emul: CFI hardening
2022-02-16 21:34 ` Andrew Cooper
@ 2022-02-17 11:49 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 11:49 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Xen-devel
On 16.02.2022 22:34, Andrew Cooper wrote:
> On 15/02/2022 14:13, Jan Beulich wrote:
>> On 15.02.2022 14:43, Andrew Cooper wrote:
>>> On 14/02/2022 13:38, Jan Beulich wrote:
>>>> On 14.02.2022 13:50, Andrew Cooper wrote:
>>>>> Control Flow Integrity schemes use toolchain and optionally hardware support
>>>>> to help protect against call/jump/return oriented programming attacks.
>>>>>
>>>>> Use cf_check to annotate function pointer targets for the toolchain.
>>>>>
>>>>> pv_emul_is_mem_write() is only used in a single file. Having it as a static
>>>>> inline is pointless because it can't be inlined to begin with.
>>>> I'd like you to consider to re-word this:
>>> This is the reworded version.
>>>
>>>> It being static inline was for
>>>> the case of there appearing a 2nd user. I don't view such as pointless.
>>> I find that impossible to reconcile with your normal review feedback.
>> Interesting. I don't think I would have objected to something like
>> this, if it was conceivable that a 2nd user may appear. I don't
>> think this is the only inline function we've got with just a single
>> user. I also don't think this is the only inline function we've got
>> with its address taken, and hence having an out-of-line instantiation.
>>
>>> It is unconditionally forced out of line because of how it's used,
>>> meaning that if it ever got used in a second translation unit we'd end
>>> up with a duplicate function, at which point it would need to be
>>> non-static and exported to pass review. (And sanity.)
>> I'm afraid you've lost me here. What duplicate function? Before and
>> after the patch the function is static; what changes is merely the
>> "inline". Two CUs can have identically named static functions, can't
>> they? Or if that's not the point you try to make, then I have no idea
>> what it is that you're trying to tell me.
>
> Yes, the same static inline can be out-of-lined in multiple translation
> units. This creates two identical copies of the logic, and then falls
> foul of our unique symbols constraint.
How / where / when? If you care about unique symbol names, you've got
a Kconfig setting to enable.
> The absence of complaints in the general case shows that we don't
> currently any cases where a static inline is out-of-lined in multiple
> translation units.
Nope. I see reports of duplicate symbols (in their warning incarnation)
quite frequently. hvm.c#cpu_callback and hvm.c#cpu_nfb, to just name
two. Those aren't inline functions, but the naming constraints apply
equally.
> Under IBT, it means more tagged functions, which I suppose doesn't make
> a useful difference from the attackers point of view, but it's still
> logic duplication in the final build that we'd prefer to avoid.
Right, which is why I didn't object in any way (and you did have my ack
for the patch already anyway), but merely asked that you soften
"pointless" in the description. I really don't like it if, for reasons
I cannot follow, things are criticized more severely than (imo)
warranted. I'd like to point out that effectively you're reverting
08143c5b6c1f ("x86: move pv_emul_is_mem_write to pv/emulate.h") then
(just that the function moves to a different file now, following the
movement of its users), which you did give your R-b. (And yes, I know
views can change over time.)
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 35/70] x86/ucode: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (33 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 34/70] x86/emul: CFI hardening Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 36/70] x86/power: " Andrew Cooper
` (36 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/microcode/amd.c | 9 +++++----
xen/arch/x86/cpu/microcode/core.c | 4 ++--
xen/arch/x86/cpu/microcode/intel.c | 10 +++++-----
3 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
index fe92e594f129..0afa2192bf1d 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -91,7 +91,7 @@ static struct {
uint16_t id;
} equiv __read_mostly;
-static void collect_cpu_info(void)
+static void cf_check collect_cpu_info(void)
{
struct cpu_signature *csig = &this_cpu(cpu_sig);
@@ -204,7 +204,7 @@ static enum microcode_match_result compare_header(
return compare_revisions(old->patch_id, new->patch_id);
}
-static enum microcode_match_result compare_patch(
+static enum microcode_match_result cf_check compare_patch(
const struct microcode_patch *new, const struct microcode_patch *old)
{
/* Both patches to compare are supposed to be applicable to local CPU. */
@@ -214,7 +214,7 @@ static enum microcode_match_result compare_patch(
return compare_header(new, old);
}
-static int apply_microcode(const struct microcode_patch *patch)
+static int cf_check apply_microcode(const struct microcode_patch *patch)
{
int hw_err;
unsigned int cpu = smp_processor_id();
@@ -299,7 +299,8 @@ static int scan_equiv_cpu_table(const struct container_equiv_table *et)
return -ESRCH;
}
-static struct microcode_patch *cpu_request_microcode(const void *buf, size_t size)
+static struct microcode_patch *cf_check cpu_request_microcode(
+ const void *buf, size_t size)
{
const struct microcode_patch *saved = NULL;
struct microcode_patch *patch = NULL;
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 841364208053..c07f68ba350e 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -291,12 +291,12 @@ static int wait_for_condition(bool (*func)(unsigned int data),
return 0;
}
-static bool wait_cpu_callin(unsigned int nr)
+static bool cf_check wait_cpu_callin(unsigned int nr)
{
return cpumask_weight(&cpu_callin_map) >= nr;
}
-static bool wait_cpu_callout(unsigned int nr)
+static bool cf_check wait_cpu_callout(unsigned int nr)
{
return atomic_read(&cpu_out) >= nr;
}
diff --git a/xen/arch/x86/cpu/microcode/intel.c b/xen/arch/x86/cpu/microcode/intel.c
index f6d01490e0ab..d3864b5ab03e 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -116,7 +116,7 @@ static bool signature_matches(const struct cpu_signature *cpu_sig,
return cpu_sig->pf & ucode_pf;
}
-static void collect_cpu_info(void)
+static void cf_check collect_cpu_info(void)
{
struct cpu_signature *csig = &this_cpu(cpu_sig);
uint64_t msr_content;
@@ -271,7 +271,7 @@ static enum microcode_match_result microcode_update_match(
return compare_revisions(cpu_sig->rev, mc->rev);
}
-static enum microcode_match_result compare_patch(
+static enum microcode_match_result cf_check compare_patch(
const struct microcode_patch *new, const struct microcode_patch *old)
{
/*
@@ -284,7 +284,7 @@ static enum microcode_match_result compare_patch(
return compare_revisions(old->rev, new->rev);
}
-static int apply_microcode(const struct microcode_patch *patch)
+static int cf_check apply_microcode(const struct microcode_patch *patch)
{
uint64_t msr_content;
unsigned int cpu = smp_processor_id();
@@ -323,8 +323,8 @@ static int apply_microcode(const struct microcode_patch *patch)
return 0;
}
-static struct microcode_patch *cpu_request_microcode(const void *buf,
- size_t size)
+static struct microcode_patch *cf_check cpu_request_microcode(
+ const void *buf, size_t size)
{
int error = 0;
const struct microcode_patch *saved = NULL;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 36/70] x86/power: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (34 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 35/70] x86/ucode: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 37/70] x86/apic: " Andrew Cooper
` (35 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
cpufreq_governor_dbs() has no external callers so make it static.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpufreq/cpufreq.c | 14 +++++++-------
xen/arch/x86/acpi/cpufreq/powernow.c | 15 ++++++++-------
xen/common/core_parking.c | 4 ++--
xen/drivers/cpufreq/cpufreq_ondemand.c | 6 ++++--
xen/include/acpi/cpufreq/cpufreq.h | 1 -
5 files changed, 21 insertions(+), 19 deletions(-)
diff --git a/xen/arch/x86/acpi/cpufreq/cpufreq.c b/xen/arch/x86/acpi/cpufreq/cpufreq.c
index 8133c2dd958c..c27cbb2304f2 100644
--- a/xen/arch/x86/acpi/cpufreq/cpufreq.c
+++ b/xen/arch/x86/acpi/cpufreq/cpufreq.c
@@ -320,7 +320,7 @@ unsigned int get_measured_perf(unsigned int cpu, unsigned int flag)
return policy->cpuinfo.max_freq * perf_percent / 100;
}
-static unsigned int get_cur_freq_on_cpu(unsigned int cpu)
+static unsigned int cf_check get_cur_freq_on_cpu(unsigned int cpu)
{
struct cpufreq_policy *policy;
struct acpi_cpufreq_data *data;
@@ -369,8 +369,9 @@ static unsigned int check_freqs(const cpumask_t *mask, unsigned int freq,
return 0;
}
-static int acpi_cpufreq_target(struct cpufreq_policy *policy,
- unsigned int target_freq, unsigned int relation)
+static int cf_check acpi_cpufreq_target(
+ struct cpufreq_policy *policy,
+ unsigned int target_freq, unsigned int relation)
{
struct acpi_cpufreq_data *data = cpufreq_drv_data[policy->cpu];
struct processor_performance *perf;
@@ -449,7 +450,7 @@ static int acpi_cpufreq_target(struct cpufreq_policy *policy,
return result;
}
-static int acpi_cpufreq_verify(struct cpufreq_policy *policy)
+static int cf_check acpi_cpufreq_verify(struct cpufreq_policy *policy)
{
struct acpi_cpufreq_data *data;
struct processor_performance *perf;
@@ -494,8 +495,7 @@ acpi_cpufreq_guess_freq(struct acpi_cpufreq_data *data, unsigned int cpu)
}
}
-static int
-acpi_cpufreq_cpu_init(struct cpufreq_policy *policy)
+static int cf_check acpi_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
unsigned int i;
unsigned int valid_states = 0;
@@ -609,7 +609,7 @@ acpi_cpufreq_cpu_init(struct cpufreq_policy *policy)
return result;
}
-static int acpi_cpufreq_cpu_exit(struct cpufreq_policy *policy)
+static int cf_check acpi_cpufreq_cpu_exit(struct cpufreq_policy *policy)
{
struct acpi_cpufreq_data *data = cpufreq_drv_data[policy->cpu];
diff --git a/xen/arch/x86/acpi/cpufreq/powernow.c b/xen/arch/x86/acpi/cpufreq/powernow.c
index ca71ecf72d67..d4c7dcd5d99d 100644
--- a/xen/arch/x86/acpi/cpufreq/powernow.c
+++ b/xen/arch/x86/acpi/cpufreq/powernow.c
@@ -67,8 +67,8 @@ static void cf_check update_cpb(void *data)
}
}
-static int powernow_cpufreq_update (int cpuid,
- struct cpufreq_policy *policy)
+static int cf_check powernow_cpufreq_update(
+ int cpuid, struct cpufreq_policy *policy)
{
if (!cpumask_test_cpu(cpuid, &cpu_online_map))
return -EINVAL;
@@ -78,8 +78,9 @@ static int powernow_cpufreq_update (int cpuid,
return 0;
}
-static int powernow_cpufreq_target(struct cpufreq_policy *policy,
- unsigned int target_freq, unsigned int relation)
+static int cf_check powernow_cpufreq_target(
+ struct cpufreq_policy *policy,
+ unsigned int target_freq, unsigned int relation)
{
struct acpi_cpufreq_data *data = cpufreq_drv_data[policy->cpu];
struct processor_performance *perf;
@@ -180,7 +181,7 @@ static void cf_check get_cpu_data(void *arg)
amd_fixup_frequency(&perf->states[i]);
}
-static int powernow_cpufreq_verify(struct cpufreq_policy *policy)
+static int cf_check powernow_cpufreq_verify(struct cpufreq_policy *policy)
{
struct acpi_cpufreq_data *data;
struct processor_performance *perf;
@@ -197,7 +198,7 @@ static int powernow_cpufreq_verify(struct cpufreq_policy *policy)
return cpufreq_frequency_table_verify(policy, data->freq_table);
}
-static int powernow_cpufreq_cpu_init(struct cpufreq_policy *policy)
+static int cf_check powernow_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
unsigned int i;
unsigned int valid_states = 0;
@@ -303,7 +304,7 @@ static int powernow_cpufreq_cpu_init(struct cpufreq_policy *policy)
return result;
}
-static int powernow_cpufreq_cpu_exit(struct cpufreq_policy *policy)
+static int cf_check powernow_cpufreq_cpu_exit(struct cpufreq_policy *policy)
{
struct acpi_cpufreq_data *data = cpufreq_drv_data[policy->cpu];
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index 4afad04f2f68..c4f01291c0be 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -53,7 +53,7 @@ static int __init cf_check setup_core_parking_option(const char *str)
}
custom_param("core_parking", setup_core_parking_option);
-static unsigned int core_parking_performance(unsigned int event)
+static unsigned int cf_check core_parking_performance(unsigned int event)
{
unsigned int cpu = -1;
@@ -111,7 +111,7 @@ static unsigned int core_parking_performance(unsigned int event)
return cpu;
}
-static unsigned int core_parking_power(unsigned int event)
+static unsigned int cf_check core_parking_power(unsigned int event)
{
unsigned int cpu = -1;
diff --git a/xen/drivers/cpufreq/cpufreq_ondemand.c b/xen/drivers/cpufreq/cpufreq_ondemand.c
index ba03eaa2336d..fbcd14d6c32f 100644
--- a/xen/drivers/cpufreq/cpufreq_ondemand.c
+++ b/xen/drivers/cpufreq/cpufreq_ondemand.c
@@ -215,7 +215,8 @@ static void dbs_timer_exit(struct cpu_dbs_info_s *dbs_info)
kill_timer(&per_cpu(dbs_timer, dbs_info->cpu));
}
-int cpufreq_governor_dbs(struct cpufreq_policy *policy, unsigned int event)
+static int cf_check cpufreq_governor_dbs(
+ struct cpufreq_policy *policy, unsigned int event)
{
unsigned int cpu = policy->cpu;
struct cpu_dbs_info_s *this_dbs_info;
@@ -307,7 +308,8 @@ int cpufreq_governor_dbs(struct cpufreq_policy *policy, unsigned int event)
return 0;
}
-static bool_t __init cpufreq_dbs_handle_option(const char *name, const char *val)
+static bool __init cf_check cpufreq_dbs_handle_option(
+ const char *name, const char *val)
{
if ( !strcmp(name, "rate") && val )
{
diff --git a/xen/include/acpi/cpufreq/cpufreq.h b/xen/include/acpi/cpufreq/cpufreq.h
index 4958d3f7d315..e5e58c6c30ea 100644
--- a/xen/include/acpi/cpufreq/cpufreq.h
+++ b/xen/include/acpi/cpufreq/cpufreq.h
@@ -227,7 +227,6 @@ struct cpu_dbs_info_s {
int8_t stoppable;
};
-int cpufreq_governor_dbs(struct cpufreq_policy *policy, unsigned int event);
int get_cpufreq_ondemand_para(uint32_t *sampling_rate_max,
uint32_t *sampling_rate_min,
uint32_t *sampling_rate,
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 37/70] x86/apic: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (35 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 36/70] x86/power: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 38/70] x86/nmi: " Andrew Cooper
` (34 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/genapic/bigsmp.c | 4 ++--
xen/arch/x86/genapic/delivery.c | 12 ++++++------
xen/arch/x86/genapic/x2apic.c | 16 ++++++++++------
xen/arch/x86/include/asm/genapic.h | 18 +++++++++---------
xen/arch/x86/smp.c | 6 +++---
5 files changed, 30 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/genapic/bigsmp.c b/xen/arch/x86/genapic/bigsmp.c
index b9d976e8abf2..2000383ab0bf 100644
--- a/xen/arch/x86/genapic/bigsmp.c
+++ b/xen/arch/x86/genapic/bigsmp.c
@@ -10,7 +10,7 @@
#include <xen/dmi.h>
#include <asm/io_apic.h>
-static __init int force_bigsmp(const struct dmi_system_id *d)
+static int __init cf_check force_bigsmp(const struct dmi_system_id *d)
{
printk(KERN_NOTICE "%s detected: force use of apic=bigsmp\n", d->ident);
def_to_bigsmp = true;
@@ -27,7 +27,7 @@ static const struct dmi_system_id __initconstrel bigsmp_dmi_table[] = {
};
-static __init int probe_bigsmp(void)
+static int __init cf_check probe_bigsmp(void)
{
/*
* We don't implement cluster mode, so force use of
diff --git a/xen/arch/x86/genapic/delivery.c b/xen/arch/x86/genapic/delivery.c
index 548c33f282dd..d1f99bf6834a 100644
--- a/xen/arch/x86/genapic/delivery.c
+++ b/xen/arch/x86/genapic/delivery.c
@@ -9,7 +9,7 @@
* LOGICAL FLAT DELIVERY MODE (multicast via bitmask to <= 8 logical APIC IDs).
*/
-void init_apic_ldr_flat(void)
+void cf_check init_apic_ldr_flat(void)
{
unsigned long val;
@@ -19,12 +19,12 @@ void init_apic_ldr_flat(void)
apic_write(APIC_LDR, val);
}
-const cpumask_t *vector_allocation_cpumask_flat(int cpu)
+const cpumask_t *cf_check vector_allocation_cpumask_flat(int cpu)
{
return &cpu_online_map;
}
-unsigned int cpu_mask_to_apicid_flat(const cpumask_t *cpumask)
+unsigned int cf_check cpu_mask_to_apicid_flat(const cpumask_t *cpumask)
{
return cpumask_bits(cpumask)[0]&0xFF;
}
@@ -33,17 +33,17 @@ unsigned int cpu_mask_to_apicid_flat(const cpumask_t *cpumask)
* PHYSICAL DELIVERY MODE (unicast to physical APIC IDs).
*/
-void init_apic_ldr_phys(void)
+void cf_check init_apic_ldr_phys(void)
{
/* We only deliver in phys mode - no setup needed. */
}
-const cpumask_t *vector_allocation_cpumask_phys(int cpu)
+const cpumask_t *cf_check vector_allocation_cpumask_phys(int cpu)
{
return cpumask_of(cpu);
}
-unsigned int cpu_mask_to_apicid_phys(const cpumask_t *cpumask)
+unsigned int cf_check cpu_mask_to_apicid_phys(const cpumask_t *cpumask)
{
/* As we are using single CPU as destination, pick only one CPU here */
return cpu_physical_id(cpumask_any(cpumask));
diff --git a/xen/arch/x86/genapic/x2apic.c b/xen/arch/x86/genapic/x2apic.c
index bd44bb753995..de5032f2020a 100644
--- a/xen/arch/x86/genapic/x2apic.c
+++ b/xen/arch/x86/genapic/x2apic.c
@@ -38,7 +38,7 @@ static inline u32 x2apic_cluster(unsigned int cpu)
return per_cpu(cpu_2_logical_apicid, cpu) >> 16;
}
-static void init_apic_ldr_x2apic_cluster(void)
+static void cf_check init_apic_ldr_x2apic_cluster(void)
{
unsigned int cpu, this_cpu = smp_processor_id();
@@ -74,12 +74,14 @@ static void init_apic_ldr_x2apic_cluster(void)
cpumask_set_cpu(this_cpu, per_cpu(cluster_cpus, this_cpu));
}
-static const cpumask_t *vector_allocation_cpumask_x2apic_cluster(int cpu)
+static const cpumask_t *cf_check vector_allocation_cpumask_x2apic_cluster(
+ int cpu)
{
return per_cpu(cluster_cpus, cpu);
}
-static unsigned int cpu_mask_to_apicid_x2apic_cluster(const cpumask_t *cpumask)
+static unsigned int cf_check cpu_mask_to_apicid_x2apic_cluster(
+ const cpumask_t *cpumask)
{
unsigned int cpu = cpumask_any(cpumask);
unsigned int dest = per_cpu(cpu_2_logical_apicid, cpu);
@@ -92,12 +94,13 @@ static unsigned int cpu_mask_to_apicid_x2apic_cluster(const cpumask_t *cpumask)
return dest;
}
-static void send_IPI_self_x2apic(uint8_t vector)
+static void cf_check send_IPI_self_x2apic(uint8_t vector)
{
apic_wrmsr(APIC_SELF_IPI, vector);
}
-static void send_IPI_mask_x2apic_phys(const cpumask_t *cpumask, int vector)
+static void cf_check send_IPI_mask_x2apic_phys(
+ const cpumask_t *cpumask, int vector)
{
unsigned int cpu;
unsigned long flags;
@@ -130,7 +133,8 @@ static void send_IPI_mask_x2apic_phys(const cpumask_t *cpumask, int vector)
local_irq_restore(flags);
}
-static void send_IPI_mask_x2apic_cluster(const cpumask_t *cpumask, int vector)
+static void cf_check send_IPI_mask_x2apic_cluster(
+ const cpumask_t *cpumask, int vector)
{
unsigned int cpu = smp_processor_id();
cpumask_t *ipimask = per_cpu(scratch_mask, cpu);
diff --git a/xen/arch/x86/include/asm/genapic.h b/xen/arch/x86/include/asm/genapic.h
index 51a65d3e0f0c..beeaddf19daa 100644
--- a/xen/arch/x86/include/asm/genapic.h
+++ b/xen/arch/x86/include/asm/genapic.h
@@ -39,12 +39,12 @@ extern struct genapic genapic;
extern const struct genapic apic_default;
extern const struct genapic apic_bigsmp;
-void send_IPI_self_legacy(uint8_t vector);
+void cf_check send_IPI_self_legacy(uint8_t vector);
-void init_apic_ldr_flat(void);
-unsigned int cpu_mask_to_apicid_flat(const cpumask_t *cpumask);
-void send_IPI_mask_flat(const cpumask_t *mask, int vector);
-const cpumask_t *vector_allocation_cpumask_flat(int cpu);
+void cf_check init_apic_ldr_flat(void);
+unsigned int cf_check cpu_mask_to_apicid_flat(const cpumask_t *cpumask);
+void cf_check send_IPI_mask_flat(const cpumask_t *mask, int vector);
+const cpumask_t *cf_check vector_allocation_cpumask_flat(int cpu);
#define GENAPIC_FLAT \
.int_delivery_mode = dest_LowestPrio, \
.int_dest_mode = 1 /* logical delivery */, \
@@ -54,10 +54,10 @@ const cpumask_t *vector_allocation_cpumask_flat(int cpu);
.send_IPI_mask = send_IPI_mask_flat, \
.send_IPI_self = send_IPI_self_legacy
-void init_apic_ldr_phys(void);
-unsigned int cpu_mask_to_apicid_phys(const cpumask_t *cpumask);
-void send_IPI_mask_phys(const cpumask_t *mask, int vector);
-const cpumask_t *vector_allocation_cpumask_phys(int cpu);
+void cf_check init_apic_ldr_phys(void);
+unsigned int cf_check cpu_mask_to_apicid_phys(const cpumask_t *cpumask);
+void cf_check send_IPI_mask_phys(const cpumask_t *mask, int vector);
+const cpumask_t *cf_check vector_allocation_cpumask_phys(int cpu);
#define GENAPIC_PHYS \
.int_delivery_mode = dest_Fixed, \
.int_dest_mode = 0 /* physical delivery */, \
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index 33748e629a21..0a02086966c0 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -161,13 +161,13 @@ void send_IPI_self(int vector)
* The following functions deal with sending IPIs between CPUs.
*/
-void send_IPI_self_legacy(uint8_t vector)
+void cf_check send_IPI_self_legacy(uint8_t vector)
{
/* NMI continuation handling relies on using a shorthand here. */
send_IPI_shortcut(APIC_DEST_SELF, vector, APIC_DEST_PHYSICAL);
}
-void send_IPI_mask_flat(const cpumask_t *cpumask, int vector)
+void cf_check send_IPI_mask_flat(const cpumask_t *cpumask, int vector)
{
unsigned long mask = cpumask_bits(cpumask)[0];
unsigned long cfg;
@@ -204,7 +204,7 @@ void send_IPI_mask_flat(const cpumask_t *cpumask, int vector)
local_irq_restore(flags);
}
-void send_IPI_mask_phys(const cpumask_t *mask, int vector)
+void cf_check send_IPI_mask_phys(const cpumask_t *mask, int vector)
{
unsigned long cfg, flags;
unsigned int query_cpu;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 38/70] x86/nmi: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (36 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 37/70] x86/apic: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 39/70] x86/mtrr: " Andrew Cooper
` (33 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/alternative.c | 4 ++--
xen/arch/x86/cpu/microcode/core.c | 3 ++-
xen/arch/x86/crash.c | 3 ++-
xen/arch/x86/livepatch.c | 2 +-
xen/arch/x86/oprofile/nmi_int.c | 2 +-
xen/arch/x86/traps.c | 3 ++-
6 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c
index 1cb531c9df83..436047abe021 100644
--- a/xen/arch/x86/alternative.c
+++ b/xen/arch/x86/alternative.c
@@ -324,8 +324,8 @@ static unsigned int __initdata alt_done;
* condition where an NMI hits while we are midway though patching some
* instructions in the NMI path.
*/
-static int __init nmi_apply_alternatives(const struct cpu_user_regs *regs,
- int cpu)
+static int __init cf_check nmi_apply_alternatives(
+ const struct cpu_user_regs *regs, int cpu)
{
/*
* More than one NMI may occur between the two set_nmi_callback() below.
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index c07f68ba350e..f84dafa82693 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -376,7 +376,8 @@ static int primary_thread_work(const struct microcode_patch *patch)
return ret;
}
-static int microcode_nmi_callback(const struct cpu_user_regs *regs, int cpu)
+static int cf_check microcode_nmi_callback(
+ const struct cpu_user_regs *regs, int cpu)
{
unsigned int primary = cpumask_first(this_cpu(cpu_sibling_mask));
int ret;
diff --git a/xen/arch/x86/crash.c b/xen/arch/x86/crash.c
index f6264946a681..c383f718f5bd 100644
--- a/xen/arch/x86/crash.c
+++ b/xen/arch/x86/crash.c
@@ -36,7 +36,8 @@ static unsigned int crashing_cpu;
static DEFINE_PER_CPU_READ_MOSTLY(bool, crash_save_done);
/* This becomes the NMI handler for non-crashing CPUs, when Xen is crashing. */
-static int noreturn do_nmi_crash(const struct cpu_user_regs *regs, int cpu)
+static int noreturn cf_check do_nmi_crash(
+ const struct cpu_user_regs *regs, int cpu)
{
stac();
diff --git a/xen/arch/x86/livepatch.c b/xen/arch/x86/livepatch.c
index d056b1ed8b41..37c9b8435eda 100644
--- a/xen/arch/x86/livepatch.c
+++ b/xen/arch/x86/livepatch.c
@@ -175,7 +175,7 @@ static nmi_callback_t *saved_nmi_callback;
* Note that because of this NOP code the do_nmi is not safely patchable.
* Also if we do receive 'real' NMIs we have lost them.
*/
-static int mask_nmi_callback(const struct cpu_user_regs *regs, int cpu)
+static int cf_check mask_nmi_callback(const struct cpu_user_regs *regs, int cpu)
{
/* TODO: Handle missing NMI/MCE.*/
return 1;
diff --git a/xen/arch/x86/oprofile/nmi_int.c b/xen/arch/x86/oprofile/nmi_int.c
index 6ebe20bd1d3e..a90b72825818 100644
--- a/xen/arch/x86/oprofile/nmi_int.c
+++ b/xen/arch/x86/oprofile/nmi_int.c
@@ -95,7 +95,7 @@ bool nmi_oprofile_send_virq(void)
return v;
}
-static int nmi_callback(const struct cpu_user_regs *regs, int cpu)
+static int cf_check nmi_callback(const struct cpu_user_regs *regs, int cpu)
{
int xen_mode, ovf;
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 485bd66971db..7b957101934e 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -752,7 +752,8 @@ static cpumask_t show_state_mask;
static bool opt_show_all;
boolean_param("async-show-all", opt_show_all);
-static int nmi_show_execution_state(const struct cpu_user_regs *regs, int cpu)
+static int cf_check nmi_show_execution_state(
+ const struct cpu_user_regs *regs, int cpu)
{
if ( !cpumask_test_cpu(cpu, &show_state_mask) )
return 0;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 39/70] x86/mtrr: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (37 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 38/70] x86/nmi: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 40/70] x86/idle: " Andrew Cooper
` (32 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/mtrr/generic.c | 18 ++++++++++--------
xen/arch/x86/cpu/mtrr/mtrr.h | 8 ++++----
2 files changed, 14 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/cpu/mtrr/generic.c b/xen/arch/x86/cpu/mtrr/generic.c
index 7cf4cd01f3f6..47aaf76226e0 100644
--- a/xen/arch/x86/cpu/mtrr/generic.c
+++ b/xen/arch/x86/cpu/mtrr/generic.c
@@ -287,7 +287,8 @@ static void set_fixed_range(int msr, bool *changed, unsigned int *msrwords)
}
}
-int generic_get_free_region(unsigned long base, unsigned long size, int replace_reg)
+int cf_check generic_get_free_region(
+ unsigned long base, unsigned long size, int replace_reg)
/* [SUMMARY] Get a free MTRR.
<base> The starting (base) address of the region.
<size> The size (in bytes) of the region.
@@ -309,8 +310,8 @@ int generic_get_free_region(unsigned long base, unsigned long size, int replace_
return -ENOSPC;
}
-static void generic_get_mtrr(unsigned int reg, unsigned long *base,
- unsigned long *size, mtrr_type *type)
+static void cf_check generic_get_mtrr(
+ unsigned int reg, unsigned long *base, unsigned long *size, mtrr_type *type)
{
uint64_t _mask, _base;
@@ -499,7 +500,7 @@ static void post_set(bool pge)
spin_unlock(&set_atomicity_lock);
}
-static void generic_set_all(void)
+static void cf_check generic_set_all(void)
{
unsigned long mask, count;
unsigned long flags;
@@ -522,8 +523,8 @@ static void generic_set_all(void)
}
}
-static void generic_set_mtrr(unsigned int reg, unsigned long base,
- unsigned long size, mtrr_type type)
+static void cf_check generic_set_mtrr(
+ unsigned int reg, unsigned long base, unsigned long size, mtrr_type type)
/* [SUMMARY] Set variable MTRR register on the local CPU.
<reg> The register to set.
<base> The base address of the region.
@@ -566,7 +567,8 @@ static void generic_set_mtrr(unsigned int reg, unsigned long base,
local_irq_restore(flags);
}
-int generic_validate_add_page(unsigned long base, unsigned long size, unsigned int type)
+int cf_check generic_validate_add_page(
+ unsigned long base, unsigned long size, unsigned int type)
{
unsigned long lbase, last;
@@ -584,7 +586,7 @@ int generic_validate_add_page(unsigned long base, unsigned long size, unsigned i
}
-static int generic_have_wrcomb(void)
+static int cf_check generic_have_wrcomb(void)
{
unsigned long config;
rdmsrl(MSR_MTRRcap, config);
diff --git a/xen/arch/x86/cpu/mtrr/mtrr.h b/xen/arch/x86/cpu/mtrr/mtrr.h
index 9a406e6f6199..c7fd44daab27 100644
--- a/xen/arch/x86/cpu/mtrr/mtrr.h
+++ b/xen/arch/x86/cpu/mtrr/mtrr.h
@@ -24,10 +24,10 @@ struct mtrr_ops {
int (*have_wrcomb)(void);
};
-extern int generic_get_free_region(unsigned long base, unsigned long size,
- int replace_reg);
-extern int generic_validate_add_page(unsigned long base, unsigned long size,
- unsigned int type);
+int cf_check generic_get_free_region(
+ unsigned long base, unsigned long size, int replace_reg);
+int cf_check generic_validate_add_page(
+ unsigned long base, unsigned long size, unsigned int type);
extern const struct mtrr_ops generic_mtrr_ops;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 40/70] x86/idle: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (38 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 39/70] x86/mtrr: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 41/70] x86/quirks: " Andrew Cooper
` (31 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/acpi/cpu_idle.c | 31 +++++++++++++++++++++-------
xen/arch/x86/acpi/cpuidle_menu.c | 6 +++---
xen/arch/x86/cpu/mwait-idle.c | 2 +-
xen/arch/x86/domain.c | 6 +++---
xen/arch/x86/hpet.c | 4 ++--
xen/arch/x86/include/asm/cpuidle.h | 4 ++--
xen/arch/x86/include/asm/hpet.h | 4 ++--
xen/arch/x86/include/asm/time.h | 6 +++---
xen/arch/x86/time.c | 6 +++---
xen/drivers/cpufreq/cpufreq_misc_governors.c | 14 ++++++-------
10 files changed, 49 insertions(+), 34 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index 0142671bb836..557bc6ef8642 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -75,7 +75,7 @@
#define GET_CC7_RES(val) GET_HW_RES_IN_NS(0x3FE, val) /* SNB onwards */
#define PHI_CC6_RES(val) GET_HW_RES_IN_NS(0x3FF, val) /* Xeon Phi only */
-static void lapic_timer_nop(void) { }
+static void cf_check lapic_timer_nop(void) { }
void (*__read_mostly lapic_timer_off)(void);
void (*__read_mostly lapic_timer_on)(void);
@@ -310,12 +310,27 @@ static char* acpi_cstate_method_name[] =
"HALT"
};
-static uint64_t get_stime_tick(void) { return (uint64_t)NOW(); }
-static uint64_t stime_ticks_elapsed(uint64_t t1, uint64_t t2) { return t2 - t1; }
-static uint64_t stime_tick_to_ns(uint64_t ticks) { return ticks; }
+static uint64_t cf_check get_stime_tick(void)
+{
+ return NOW();
+}
+
+static uint64_t cf_check stime_ticks_elapsed(uint64_t t1, uint64_t t2)
+{
+ return t2 - t1;
+}
+
+static uint64_t cf_check stime_tick_to_ns(uint64_t ticks)
+{
+ return ticks;
+}
+
+static uint64_t cf_check get_acpi_pm_tick(void)
+{
+ return inl(pmtmr_ioport);
+}
-static uint64_t get_acpi_pm_tick(void) { return (uint64_t)inl(pmtmr_ioport); }
-static uint64_t acpi_pm_ticks_elapsed(uint64_t t1, uint64_t t2)
+static uint64_t cf_check acpi_pm_ticks_elapsed(uint64_t t1, uint64_t t2)
{
if ( t2 >= t1 )
return (t2 - t1);
@@ -664,7 +679,7 @@ void update_idle_stats(struct acpi_processor_power *power,
spin_unlock(&power->stat_lock);
}
-static void acpi_processor_idle(void)
+static void cf_check acpi_processor_idle(void)
{
unsigned int cpu = smp_processor_id();
struct acpi_processor_power *power = processor_powers[cpu];
@@ -869,7 +884,7 @@ static void acpi_processor_idle(void)
cpuidle_current_governor->reflect(power);
}
-void acpi_dead_idle(void)
+void cf_check acpi_dead_idle(void)
{
struct acpi_processor_power *power;
struct acpi_processor_cx *cx;
diff --git a/xen/arch/x86/acpi/cpuidle_menu.c b/xen/arch/x86/acpi/cpuidle_menu.c
index 6ff5fb8ff215..a275436d799c 100644
--- a/xen/arch/x86/acpi/cpuidle_menu.c
+++ b/xen/arch/x86/acpi/cpuidle_menu.c
@@ -185,7 +185,7 @@ static unsigned int get_sleep_length_us(void)
return (us >> 32) ? (unsigned int)-2000 : (unsigned int)us;
}
-static int menu_select(struct acpi_processor_power *power)
+static int cf_check menu_select(struct acpi_processor_power *power)
{
struct menu_device *data = &this_cpu(menu_devices);
int i;
@@ -237,7 +237,7 @@ static int menu_select(struct acpi_processor_power *power)
return data->last_state_idx;
}
-static void menu_reflect(struct acpi_processor_power *power)
+static void cf_check menu_reflect(struct acpi_processor_power *power)
{
struct menu_device *data = &this_cpu(menu_devices);
u64 new_factor;
@@ -275,7 +275,7 @@ static void menu_reflect(struct acpi_processor_power *power)
data->correction_factor[data->bucket] = new_factor;
}
-static int menu_enable_device(struct acpi_processor_power *power)
+static int cf_check menu_enable_device(struct acpi_processor_power *power)
{
memset(&per_cpu(menu_devices, power->cpu), 0, sizeof(struct menu_device));
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index 927ce1b67aa5..f76c64e04b20 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -773,7 +773,7 @@ static const struct cpuidle_state snr_cstates[] = {
{}
};
-static void mwait_idle(void)
+static void cf_check mwait_idle(void)
{
unsigned int cpu = smp_processor_id();
struct acpi_processor_power *power = processor_powers[cpu];
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 1c3a1ec2a080..ae7c88b51af1 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -74,11 +74,11 @@
DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
-static void default_idle(void);
+static void cf_check default_idle(void);
void (*pm_idle) (void) __read_mostly = default_idle;
void (*dead_idle) (void) __read_mostly = default_dead_idle;
-static void default_idle(void)
+static void cf_check default_idle(void)
{
struct cpu_info *info = get_cpu_info();
@@ -93,7 +93,7 @@ static void default_idle(void)
local_irq_enable();
}
-void default_dead_idle(void)
+void cf_check default_dead_idle(void)
{
/*
* When going into S3, without flushing caches modified data may be
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index c31fd97579dc..20fca839907c 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -709,7 +709,7 @@ void hpet_disable_legacy_broadcast(void)
smp_send_event_check_mask(&cpu_online_map);
}
-void hpet_broadcast_enter(void)
+void cf_check hpet_broadcast_enter(void)
{
unsigned int cpu = smp_processor_id();
struct hpet_event_channel *ch = per_cpu(cpu_bc_channel, cpu);
@@ -740,7 +740,7 @@ void hpet_broadcast_enter(void)
spin_unlock(&ch->lock);
}
-void hpet_broadcast_exit(void)
+void cf_check hpet_broadcast_exit(void)
{
unsigned int cpu = smp_processor_id();
struct hpet_event_channel *ch = per_cpu(cpu_bc_channel, cpu);
diff --git a/xen/arch/x86/include/asm/cpuidle.h b/xen/arch/x86/include/asm/cpuidle.h
index 0981a8fd6417..3edd7a75d2ef 100644
--- a/xen/arch/x86/include/asm/cpuidle.h
+++ b/xen/arch/x86/include/asm/cpuidle.h
@@ -17,8 +17,8 @@ extern uint64_t (*cpuidle_get_tick)(void);
int mwait_idle_init(struct notifier_block *);
int cpuidle_init_cpu(unsigned int cpu);
-void default_dead_idle(void);
-void acpi_dead_idle(void);
+void cf_check default_dead_idle(void);
+void cf_check acpi_dead_idle(void);
void play_dead(void);
void trace_exit_reason(u32 *irq_traced);
void update_idle_stats(struct acpi_processor_power *,
diff --git a/xen/arch/x86/include/asm/hpet.h b/xen/arch/x86/include/asm/hpet.h
index 8f9725a95e21..f343fe4740f1 100644
--- a/xen/arch/x86/include/asm/hpet.h
+++ b/xen/arch/x86/include/asm/hpet.h
@@ -91,8 +91,8 @@ void hpet_disable_legacy_replacement_mode(void);
*/
void hpet_broadcast_init(void);
void hpet_broadcast_resume(void);
-void hpet_broadcast_enter(void);
-void hpet_broadcast_exit(void);
+void cf_check hpet_broadcast_enter(void);
+void cf_check hpet_broadcast_exit(void);
int hpet_broadcast_is_available(void);
void hpet_disable_legacy_broadcast(void);
diff --git a/xen/arch/x86/include/asm/time.h b/xen/arch/x86/include/asm/time.h
index f06f2bfd8b8f..2a57d930ef30 100644
--- a/xen/arch/x86/include/asm/time.h
+++ b/xen/arch/x86/include/asm/time.h
@@ -43,11 +43,11 @@ int hwdom_pit_access(struct ioreq *ioreq);
int cpu_frequency_change(u64 freq);
-void pit_broadcast_enter(void);
-void pit_broadcast_exit(void);
+void cf_check pit_broadcast_enter(void);
+void cf_check pit_broadcast_exit(void);
int pit_broadcast_is_available(void);
-uint64_t acpi_pm_tick_to_ns(uint64_t ticks);
+uint64_t cf_check acpi_pm_tick_to_ns(uint64_t ticks);
uint64_t tsc_ticks2ns(uint64_t ticks);
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index b6e690b76fab..043be2a8ec1a 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -545,7 +545,7 @@ static __init int cf_check init_pmtmr_scale(void)
}
__initcall(init_pmtmr_scale);
-uint64_t acpi_pm_tick_to_ns(uint64_t ticks)
+uint64_t cf_check acpi_pm_tick_to_ns(uint64_t ticks)
{
return scale_delta(ticks, &pmt_scale);
}
@@ -2237,12 +2237,12 @@ static int __init cf_check disable_pit_irq(void)
}
__initcall(disable_pit_irq);
-void pit_broadcast_enter(void)
+void cf_check pit_broadcast_enter(void)
{
cpumask_set_cpu(smp_processor_id(), &pit_broadcast_mask);
}
-void pit_broadcast_exit(void)
+void cf_check pit_broadcast_exit(void)
{
int cpu = smp_processor_id();
diff --git a/xen/drivers/cpufreq/cpufreq_misc_governors.c b/xen/drivers/cpufreq/cpufreq_misc_governors.c
index ad79d0f5d246..f5571f5486ab 100644
--- a/xen/drivers/cpufreq/cpufreq_misc_governors.c
+++ b/xen/drivers/cpufreq/cpufreq_misc_governors.c
@@ -26,8 +26,8 @@
static unsigned int __read_mostly userspace_cmdline_freq;
static DEFINE_PER_CPU(unsigned int, cpu_set_freq);
-static int cpufreq_governor_userspace(struct cpufreq_policy *policy,
- unsigned int event)
+static int cf_check cpufreq_governor_userspace(
+ struct cpufreq_policy *policy, unsigned int event)
{
int ret = 0;
unsigned int cpu;
@@ -81,7 +81,7 @@ int write_userspace_scaling_setspeed(unsigned int cpu, unsigned int freq)
return __cpufreq_driver_target(policy, freq, CPUFREQ_RELATION_L);
}
-static bool_t __init
+static bool __init cf_check
cpufreq_userspace_handle_option(const char *name, const char *val)
{
if (!strcmp(name, "speed") && val) {
@@ -131,8 +131,8 @@ __initcall(cpufreq_gov_userspace_init);
/*
* cpufreq performance governor
*/
-static int cpufreq_governor_performance(struct cpufreq_policy *policy,
- unsigned int event)
+static int cf_check cpufreq_governor_performance(
+ struct cpufreq_policy *policy, unsigned int event)
{
int ret = 0;
@@ -170,8 +170,8 @@ __initcall(cpufreq_gov_performance_init);
/*
* cpufreq powersave governor
*/
-static int cpufreq_governor_powersave(struct cpufreq_policy *policy,
- unsigned int event)
+static int cf_check cpufreq_governor_powersave(
+ struct cpufreq_policy *policy, unsigned int event)
{
int ret = 0;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 41/70] x86/quirks: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (39 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 40/70] x86/idle: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:50 ` [PATCH v2 42/70] x86/hvmsave: " Andrew Cooper
` (30 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/dmi_scan.c | 10 +++++-----
xen/arch/x86/hvm/quirks.c | 2 +-
xen/arch/x86/shutdown.c | 2 +-
xen/arch/x86/x86_64/mmconfig-shared.c | 8 ++++----
4 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/xen/arch/x86/dmi_scan.c b/xen/arch/x86/dmi_scan.c
index d27cd3450a29..81f80c053a7a 100644
--- a/xen/arch/x86/dmi_scan.c
+++ b/xen/arch/x86/dmi_scan.c
@@ -476,7 +476,7 @@ static void __init dmi_save_ident(const struct dmi_header *dm, int slot, int str
#define NO_MATCH { DMI_NONE, NULL}
#define MATCH DMI_MATCH
-static int __init ich10_bios_quirk(const struct dmi_system_id *d)
+static int __init cf_check ich10_bios_quirk(const struct dmi_system_id *d)
{
u32 port, smictl;
@@ -499,14 +499,14 @@ static int __init ich10_bios_quirk(const struct dmi_system_id *d)
return 0;
}
-static __init int reset_videomode_after_s3(const struct dmi_blacklist *d)
+static __init int cf_check reset_videomode_after_s3(const struct dmi_blacklist *d)
{
/* See wakeup.S */
acpi_video_flags |= 2;
return 0;
}
-static __init int dmi_disable_acpi(const struct dmi_blacklist *d)
+static __init int cf_check dmi_disable_acpi(const struct dmi_blacklist *d)
{
if (!acpi_force) {
printk(KERN_NOTICE "%s detected: acpi off\n",d->ident);
@@ -521,7 +521,7 @@ static __init int dmi_disable_acpi(const struct dmi_blacklist *d)
/*
* Limit ACPI to CPU enumeration for HT
*/
-static __init int force_acpi_ht(const struct dmi_blacklist *d)
+static __init int cf_check force_acpi_ht(const struct dmi_blacklist *d)
{
if (!acpi_force) {
printk(KERN_NOTICE "%s detected: force use of acpi=ht\n", d->ident);
@@ -650,7 +650,7 @@ static const struct dmi_blacklist __initconstrel dmi_blacklist[] = {
* out of here.
*/
-static void __init dmi_decode(const struct dmi_header *dm)
+static void __init cf_check dmi_decode(const struct dmi_header *dm)
{
#ifdef DMI_DEBUG
const uint8_t *data = (const void *)dm;
diff --git a/xen/arch/x86/hvm/quirks.c b/xen/arch/x86/hvm/quirks.c
index 917356b1312c..2adab1f4b84b 100644
--- a/xen/arch/x86/hvm/quirks.c
+++ b/xen/arch/x86/hvm/quirks.c
@@ -25,7 +25,7 @@
s8 __read_mostly hvm_port80_allowed = -1;
boolean_param("hvm_port80", hvm_port80_allowed);
-static int __init dmi_hvm_deny_port80(const struct dmi_system_id *id)
+static int __init cf_check dmi_hvm_deny_port80(const struct dmi_system_id *id)
{
printk(XENLOG_WARNING "%s: port 0x80 access %s allowed for HVM guests\n",
id->ident, hvm_port80_allowed > 0 ? "forcibly" : "not");
diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index 30985d36a612..7619544d14da 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -158,7 +158,7 @@ static void default_reboot_type(void)
reboot_type = BOOT_ACPI;
}
-static int __init override_reboot(const struct dmi_system_id *d)
+static int __init cf_check override_reboot(const struct dmi_system_id *d)
{
enum reboot_type type = (long)d->driver_data;
diff --git a/xen/arch/x86/x86_64/mmconfig-shared.c b/xen/arch/x86/x86_64/mmconfig-shared.c
index 2fa7f3f0bc4b..74b22b71a19c 100644
--- a/xen/arch/x86/x86_64/mmconfig-shared.c
+++ b/xen/arch/x86/x86_64/mmconfig-shared.c
@@ -62,7 +62,7 @@ static int __init cf_check parse_mmcfg(const char *s)
}
custom_param("mmcfg", parse_mmcfg);
-static const char __init *pci_mmcfg_e7520(void)
+static const char *__init cf_check pci_mmcfg_e7520(void)
{
u32 win;
win = pci_conf_read16(PCI_SBDF(0, 0, 0, 0), 0xce);
@@ -84,7 +84,7 @@ static const char __init *pci_mmcfg_e7520(void)
return "Intel Corporation E7520 Memory Controller Hub";
}
-static const char __init *pci_mmcfg_intel_945(void)
+static const char *__init cf_check pci_mmcfg_intel_945(void)
{
u32 pciexbar, mask = 0, len = 0;
@@ -137,7 +137,7 @@ static const char __init *pci_mmcfg_intel_945(void)
return "Intel Corporation 945G/GZ/P/PL Express Memory Controller Hub";
}
-static const char __init *pci_mmcfg_amd_fam10h(void)
+static const char *__init cf_check pci_mmcfg_amd_fam10h(void)
{
uint32_t address;
uint64_t base, msr_content;
@@ -190,7 +190,7 @@ static const char __init *pci_mmcfg_amd_fam10h(void)
return "AMD Family 10h NB";
}
-static const char __init *pci_mmcfg_nvidia_mcp55(void)
+static const char *__init cf_check pci_mmcfg_nvidia_mcp55(void)
{
static bool_t __initdata mcp55_checked;
int bus, i;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 42/70] x86/hvmsave: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (40 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 41/70] x86/quirks: " Andrew Cooper
@ 2022-02-14 12:50 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 43/70] x86/mce: " Andrew Cooper
` (29 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/mcheck/vmce.c | 4 ++--
xen/arch/x86/emul-i8254.c | 4 ++--
xen/arch/x86/hvm/hpet.c | 4 ++--
xen/arch/x86/hvm/hvm.c | 18 ++++++++++--------
xen/arch/x86/hvm/irq.c | 12 ++++++------
xen/arch/x86/hvm/mtrr.c | 4 ++--
xen/arch/x86/hvm/pmtimer.c | 4 ++--
xen/arch/x86/hvm/rtc.c | 4 ++--
xen/arch/x86/hvm/vioapic.c | 4 ++--
xen/arch/x86/hvm/viridian/viridian.c | 15 ++++++++-------
xen/arch/x86/hvm/vlapic.c | 8 ++++----
xen/arch/x86/hvm/vpic.c | 4 ++--
12 files changed, 44 insertions(+), 41 deletions(-)
diff --git a/xen/arch/x86/cpu/mcheck/vmce.c b/xen/arch/x86/cpu/mcheck/vmce.c
index eb6434a3ba20..458120f9ad8d 100644
--- a/xen/arch/x86/cpu/mcheck/vmce.c
+++ b/xen/arch/x86/cpu/mcheck/vmce.c
@@ -353,7 +353,7 @@ int vmce_wrmsr(uint32_t msr, uint64_t val)
}
#if CONFIG_HVM
-static int vmce_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check vmce_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
{
struct hvm_vmce_vcpu ctxt = {
.caps = v->arch.vmce.mcg_cap,
@@ -365,7 +365,7 @@ static int vmce_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(VMCE_VCPU, v->vcpu_id, h, &ctxt);
}
-static int vmce_load_vcpu_ctxt(struct domain *d, hvm_domain_context_t *h)
+static int cf_check vmce_load_vcpu_ctxt(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
diff --git a/xen/arch/x86/emul-i8254.c b/xen/arch/x86/emul-i8254.c
index 0e09a173187f..d170f464d966 100644
--- a/xen/arch/x86/emul-i8254.c
+++ b/xen/arch/x86/emul-i8254.c
@@ -391,7 +391,7 @@ void pit_stop_channel0_irq(PITState *pit)
spin_unlock(&pit->lock);
}
-static int pit_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check pit_save(struct vcpu *v, hvm_domain_context_t *h)
{
struct domain *d = v->domain;
PITState *pit = domain_vpit(d);
@@ -409,7 +409,7 @@ static int pit_save(struct vcpu *v, hvm_domain_context_t *h)
return rc;
}
-static int pit_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check pit_load(struct domain *d, hvm_domain_context_t *h)
{
PITState *pit = domain_vpit(d);
int i, rc = 0;
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 7bdb51cfa1c4..ed512fa65b63 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -582,7 +582,7 @@ static const struct hvm_mmio_ops hpet_mmio_ops = {
};
-static int hpet_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hpet_save(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
HPETState *hp = domain_vhpet(d);
@@ -645,7 +645,7 @@ static int hpet_save(struct vcpu *v, hvm_domain_context_t *h)
return rc;
}
-static int hpet_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hpet_load(struct domain *d, hvm_domain_context_t *h)
{
HPETState *hp = domain_vhpet(d);
struct hvm_hw_hpet *rec;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index e87e809a945d..4cf313a0ad0a 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -756,7 +756,7 @@ void hvm_domain_destroy(struct domain *d)
destroy_vpci_mmcfg(d);
}
-static int hvm_save_tsc_adjust(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hvm_save_tsc_adjust(struct vcpu *v, hvm_domain_context_t *h)
{
struct hvm_tsc_adjust ctxt = {
.tsc_adjust = v->arch.hvm.msr_tsc_adjust,
@@ -765,7 +765,7 @@ static int hvm_save_tsc_adjust(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(TSC_ADJUST, v->vcpu_id, h, &ctxt);
}
-static int hvm_load_tsc_adjust(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hvm_load_tsc_adjust(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
@@ -788,7 +788,7 @@ static int hvm_load_tsc_adjust(struct domain *d, hvm_domain_context_t *h)
HVM_REGISTER_SAVE_RESTORE(TSC_ADJUST, hvm_save_tsc_adjust,
hvm_load_tsc_adjust, 1, HVMSR_PER_VCPU);
-static int hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
{
struct segment_register seg;
struct hvm_hw_cpu ctxt = {
@@ -971,7 +971,7 @@ unsigned long hvm_cr4_guest_valid_bits(const struct domain *d)
(cet ? X86_CR4_CET : 0));
}
-static int hvm_load_cpu_ctxt(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hvm_load_cpu_ctxt(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
@@ -1172,7 +1172,8 @@ HVM_REGISTER_SAVE_RESTORE(CPU, hvm_save_cpu_ctxt, hvm_load_cpu_ctxt, 1,
save_area) + \
xstate_ctxt_size(xcr0))
-static int hvm_save_cpu_xsave_states(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hvm_save_cpu_xsave_states(
+ struct vcpu *v, hvm_domain_context_t *h)
{
struct hvm_hw_cpu_xsave *ctxt;
unsigned int size = HVM_CPU_XSAVE_SIZE(v->arch.xcr0_accum);
@@ -1210,7 +1211,8 @@ CHECK_FIELD_(struct, xsave_hdr, reserved);
#undef compat_xsave_hdr
#undef xen_xsave_hdr
-static int hvm_load_cpu_xsave_states(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hvm_load_cpu_xsave_states(
+ struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid, size;
int err;
@@ -1338,7 +1340,7 @@ static const uint32_t msrs_to_send[] = {
MSR_AMD64_DR3_ADDRESS_MASK,
};
-static int hvm_save_cpu_msrs(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hvm_save_cpu_msrs(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
struct hvm_save_descriptor *desc = _p(&h->data[h->cur]);
@@ -1418,7 +1420,7 @@ static int hvm_save_cpu_msrs(struct vcpu *v, hvm_domain_context_t *h)
return 0;
}
-static int hvm_load_cpu_msrs(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hvm_load_cpu_msrs(struct domain *d, hvm_domain_context_t *h)
{
unsigned int i, vcpuid = hvm_load_instance(h);
struct vcpu *v;
diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index a7f8991a7b84..5a7f39b54ff4 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -657,7 +657,7 @@ static int __init cf_check dump_irq_info_key_init(void)
}
__initcall(dump_irq_info_key_init);
-static int irq_save_pci(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check irq_save_pci(struct vcpu *v, hvm_domain_context_t *h)
{
struct domain *d = v->domain;
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
@@ -690,7 +690,7 @@ static int irq_save_pci(struct vcpu *v, hvm_domain_context_t *h)
return rc;
}
-static int irq_save_isa(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check irq_save_isa(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
@@ -699,7 +699,7 @@ static int irq_save_isa(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(ISA_IRQ, 0, h, &hvm_irq->isa_irq);
}
-static int irq_save_link(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check irq_save_link(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
@@ -708,7 +708,7 @@ static int irq_save_link(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(PCI_LINK, 0, h, &hvm_irq->pci_link);
}
-static int irq_load_pci(struct domain *d, hvm_domain_context_t *h)
+static int cf_check irq_load_pci(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
int link, dev, intx, gsi;
@@ -741,7 +741,7 @@ static int irq_load_pci(struct domain *d, hvm_domain_context_t *h)
return 0;
}
-static int irq_load_isa(struct domain *d, hvm_domain_context_t *h)
+static int cf_check irq_load_isa(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
int irq;
@@ -760,7 +760,7 @@ static int irq_load_isa(struct domain *d, hvm_domain_context_t *h)
}
-static int irq_load_link(struct domain *d, hvm_domain_context_t *h)
+static int cf_check irq_load_link(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
int link, gsi;
diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
index 42f3d8319296..4d2aa6def86d 100644
--- a/xen/arch/x86/hvm/mtrr.c
+++ b/xen/arch/x86/hvm/mtrr.c
@@ -687,7 +687,7 @@ int hvm_set_mem_pinned_cacheattr(struct domain *d, uint64_t gfn_start,
return 0;
}
-static int hvm_save_mtrr_msr(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check hvm_save_mtrr_msr(struct vcpu *v, hvm_domain_context_t *h)
{
const struct mtrr_state *mtrr_state = &v->arch.hvm.mtrr;
struct hvm_hw_mtrr hw_mtrr = {
@@ -725,7 +725,7 @@ static int hvm_save_mtrr_msr(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(MTRR, v->vcpu_id, h, &hw_mtrr);
}
-static int hvm_load_mtrr_msr(struct domain *d, hvm_domain_context_t *h)
+static int cf_check hvm_load_mtrr_msr(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid, i;
struct vcpu *v;
diff --git a/xen/arch/x86/hvm/pmtimer.c b/xen/arch/x86/hvm/pmtimer.c
index 60e3c8de4c6b..2a89bbdfa5b8 100644
--- a/xen/arch/x86/hvm/pmtimer.c
+++ b/xen/arch/x86/hvm/pmtimer.c
@@ -249,7 +249,7 @@ static int cf_check handle_pmt_io(
return X86EMUL_OKAY;
}
-static int acpi_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check acpi_save(struct vcpu *v, hvm_domain_context_t *h)
{
struct domain *d = v->domain;
struct hvm_hw_acpi *acpi = &d->arch.hvm.acpi;
@@ -281,7 +281,7 @@ static int acpi_save(struct vcpu *v, hvm_domain_context_t *h)
return rc;
}
-static int acpi_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check acpi_load(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_hw_acpi *acpi = &d->arch.hvm.acpi;
PMTState *s = &d->arch.hvm.pl_time->vpmt;
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index bdc647e433e9..ed397276faa3 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -738,7 +738,7 @@ void rtc_migrate_timers(struct vcpu *v)
}
/* Save RTC hardware state */
-static int rtc_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check rtc_save(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
RTCState *s = domain_vrtc(d);
@@ -756,7 +756,7 @@ static int rtc_save(struct vcpu *v, hvm_domain_context_t *h)
}
/* Reload the hardware state from a saved domain */
-static int rtc_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check rtc_load(struct domain *d, hvm_domain_context_t *h)
{
RTCState *s = domain_vrtc(d);
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index b56549aa22d1..d5d5b02421e9 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -590,7 +590,7 @@ int vioapic_get_trigger_mode(const struct domain *d, unsigned int gsi)
return vioapic->redirtbl[pin].fields.trig_mode;
}
-static int ioapic_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check ioapic_save(struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
struct hvm_vioapic *s;
@@ -607,7 +607,7 @@ static int ioapic_save(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(IOAPIC, 0, h, &s->domU);
}
-static int ioapic_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check ioapic_load(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_vioapic *s;
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
index 7ebcaa1c899f..25dca93e8b71 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -1104,8 +1104,8 @@ void viridian_unmap_guest_page(struct viridian_page *vp)
put_page_and_type(page);
}
-static int viridian_save_domain_ctxt(struct vcpu *v,
- hvm_domain_context_t *h)
+static int cf_check viridian_save_domain_ctxt(
+ struct vcpu *v, hvm_domain_context_t *h)
{
const struct domain *d = v->domain;
const struct viridian_domain *vd = d->arch.hvm.viridian;
@@ -1123,8 +1123,8 @@ static int viridian_save_domain_ctxt(struct vcpu *v,
return (hvm_save_entry(VIRIDIAN_DOMAIN, 0, h, &ctxt) != 0);
}
-static int viridian_load_domain_ctxt(struct domain *d,
- hvm_domain_context_t *h)
+static int cf_check viridian_load_domain_ctxt(
+ struct domain *d, hvm_domain_context_t *h)
{
struct viridian_domain *vd = d->arch.hvm.viridian;
struct hvm_viridian_domain_context ctxt;
@@ -1144,7 +1144,8 @@ static int viridian_load_domain_ctxt(struct domain *d,
HVM_REGISTER_SAVE_RESTORE(VIRIDIAN_DOMAIN, viridian_save_domain_ctxt,
viridian_load_domain_ctxt, 1, HVMSR_PER_DOM);
-static int viridian_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check viridian_save_vcpu_ctxt(
+ struct vcpu *v, hvm_domain_context_t *h)
{
struct hvm_viridian_vcpu_context ctxt = {};
@@ -1157,8 +1158,8 @@ static int viridian_save_vcpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(VIRIDIAN_VCPU, v->vcpu_id, h, &ctxt);
}
-static int viridian_load_vcpu_ctxt(struct domain *d,
- hvm_domain_context_t *h)
+static int cf_check viridian_load_vcpu_ctxt(
+ struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 652e3cb87f12..d4e29ef1ff1d 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1482,7 +1482,7 @@ static void lapic_rearm(struct vlapic *s)
s->timer_last_update = s->pt.last_plt_gtime;
}
-static int lapic_save_hidden(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check lapic_save_hidden(struct vcpu *v, hvm_domain_context_t *h)
{
if ( !has_vlapic(v->domain) )
return 0;
@@ -1490,7 +1490,7 @@ static int lapic_save_hidden(struct vcpu *v, hvm_domain_context_t *h)
return hvm_save_entry(LAPIC, v->vcpu_id, h, &vcpu_vlapic(v)->hw);
}
-static int lapic_save_regs(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check lapic_save_regs(struct vcpu *v, hvm_domain_context_t *h)
{
if ( !has_vlapic(v->domain) )
return 0;
@@ -1529,7 +1529,7 @@ static void lapic_load_fixup(struct vlapic *vlapic)
}
}
-static int lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
+static int cf_check lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
@@ -1563,7 +1563,7 @@ static int lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
return 0;
}
-static int lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
+static int cf_check lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
{
unsigned int vcpuid = hvm_load_instance(h);
struct vcpu *v;
diff --git a/xen/arch/x86/hvm/vpic.c b/xen/arch/x86/hvm/vpic.c
index 5d8ef259b710..b3cafaab8fad 100644
--- a/xen/arch/x86/hvm/vpic.c
+++ b/xen/arch/x86/hvm/vpic.c
@@ -400,7 +400,7 @@ static int cf_check vpic_intercept_elcr_io(
return X86EMUL_OKAY;
}
-static int vpic_save(struct vcpu *v, hvm_domain_context_t *h)
+static int cf_check vpic_save(struct vcpu *v, hvm_domain_context_t *h)
{
struct domain *d = v->domain;
struct hvm_hw_vpic *s;
@@ -420,7 +420,7 @@ static int vpic_save(struct vcpu *v, hvm_domain_context_t *h)
return 0;
}
-static int vpic_load(struct domain *d, hvm_domain_context_t *h)
+static int cf_check vpic_load(struct domain *d, hvm_domain_context_t *h)
{
struct hvm_hw_vpic *s;
unsigned int inst = hvm_load_instance(h);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 43/70] x86/mce: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (41 preceding siblings ...)
2022-02-14 12:50 ` [PATCH v2 42/70] x86/hvmsave: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 44/70] x86/pmu: " Andrew Cooper
` (28 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/mcheck/mce.c | 8 ++++----
xen/arch/x86/cpu/mcheck/mce.h | 2 +-
xen/arch/x86/cpu/mcheck/mce_amd.c | 9 ++++----
xen/arch/x86/cpu/mcheck/mce_amd.h | 4 ++--
xen/arch/x86/cpu/mcheck/mce_intel.c | 41 +++++++++++++++++--------------------
5 files changed, 31 insertions(+), 33 deletions(-)
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index 3467e0f1a315..275c54be7c0f 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -75,7 +75,7 @@ static int __init cf_check mce_set_verbosity(const char *str)
custom_param("mce_verbosity", mce_set_verbosity);
/* Handle unconfigured int18 (should never happen) */
-static void unexpected_machine_check(const struct cpu_user_regs *regs)
+static void cf_check unexpected_machine_check(const struct cpu_user_regs *regs)
{
console_force_unlock();
printk("Unexpected Machine Check Exception\n");
@@ -469,7 +469,7 @@ static int mce_urgent_action(const struct cpu_user_regs *regs,
}
/* Shared #MC handler. */
-void mcheck_cmn_handler(const struct cpu_user_regs *regs)
+void cf_check mcheck_cmn_handler(const struct cpu_user_regs *regs)
{
static DEFINE_MCE_BARRIER(mce_trap_bar);
static atomic_t severity_cpu = ATOMIC_INIT(-1);
@@ -1684,7 +1684,7 @@ long cf_check do_mca(XEN_GUEST_HANDLE_PARAM(xen_mc_t) u_xen_mc)
}
int mcinfo_dumpped;
-static int x86_mcinfo_dump_panic(mctelem_cookie_t mctc)
+static int cf_check x86_mcinfo_dump_panic(mctelem_cookie_t mctc)
{
struct mc_info *mcip = mctelem_dataptr(mctc);
@@ -1801,7 +1801,7 @@ static enum mce_result mce_action(const struct cpu_user_regs *regs,
* should be committed for dom0 consumption, 0 if it should be
* dismissed.
*/
-static int mce_delayed_action(mctelem_cookie_t mctc)
+static int cf_check mce_delayed_action(mctelem_cookie_t mctc)
{
enum mce_result result;
int ret = 0;
diff --git a/xen/arch/x86/cpu/mcheck/mce.h b/xen/arch/x86/cpu/mcheck/mce.h
index 195362691904..535d0abf8f9b 100644
--- a/xen/arch/x86/cpu/mcheck/mce.h
+++ b/xen/arch/x86/cpu/mcheck/mce.h
@@ -70,7 +70,7 @@ extern void x86_mce_vector_register(x86_mce_vector_t);
* Common generic MCE handler that implementations may nominate
* via x86_mce_vector_register.
*/
-extern void mcheck_cmn_handler(const struct cpu_user_regs *regs);
+void cf_check mcheck_cmn_handler(const struct cpu_user_regs *regs);
/* Register a handler for judging whether mce is recoverable. */
typedef bool (*mce_recoverable_t)(uint64_t status);
diff --git a/xen/arch/x86/cpu/mcheck/mce_amd.c b/xen/arch/x86/cpu/mcheck/mce_amd.c
index 279a8e6f122d..d7ae8919df77 100644
--- a/xen/arch/x86/cpu/mcheck/mce_amd.c
+++ b/xen/arch/x86/cpu/mcheck/mce_amd.c
@@ -113,7 +113,7 @@ mc_ec2type(uint16_t errorcode)
return 0;
}
-bool mc_amd_recoverable_scan(uint64_t status)
+bool cf_check mc_amd_recoverable_scan(uint64_t status)
{
bool ret = false;
enum mc_ec_type ectype;
@@ -143,7 +143,7 @@ bool mc_amd_recoverable_scan(uint64_t status)
return ret;
}
-bool mc_amd_addrcheck(uint64_t status, uint64_t misc, int addrtype)
+bool cf_check mc_amd_addrcheck(uint64_t status, uint64_t misc, int addrtype)
{
enum mc_ec_type ectype;
uint16_t errorcode;
@@ -216,7 +216,7 @@ static void mcequirk_amd_apply(enum mcequirk_amd_flags flags)
}
}
-static struct mcinfo_extended *
+static struct mcinfo_extended *cf_check
amd_f10_handler(struct mc_info *mi, uint16_t bank, uint64_t status)
{
struct mcinfo_extended *mc_ext;
@@ -252,7 +252,8 @@ amd_f10_handler(struct mc_info *mi, uint16_t bank, uint64_t status)
return mc_ext;
}
-static bool amd_need_clearbank_scan(enum mca_source who, uint64_t status)
+static bool cf_check amd_need_clearbank_scan(
+ enum mca_source who, uint64_t status)
{
if ( who != MCA_MCE_SCAN )
return true;
diff --git a/xen/arch/x86/cpu/mcheck/mce_amd.h b/xen/arch/x86/cpu/mcheck/mce_amd.h
index 67c45454707d..c12c25d74567 100644
--- a/xen/arch/x86/cpu/mcheck/mce_amd.h
+++ b/xen/arch/x86/cpu/mcheck/mce_amd.h
@@ -1,7 +1,7 @@
#ifndef _MCHECK_AMD_H
#define _MCHECK_AMD_H
-bool mc_amd_recoverable_scan(uint64_t status);
-bool mc_amd_addrcheck(uint64_t status, uint64_t misc, int addrtype);
+bool cf_check mc_amd_recoverable_scan(uint64_t status);
+bool cf_check mc_amd_addrcheck(uint64_t status, uint64_t misc, int addrtype);
#endif
diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c
index 7aaa56fd02eb..50198e0c295c 100644
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -271,12 +271,13 @@ static void intel_memerr_dhandler(
mc_memerr_dhandler(binfo, result, regs);
}
-static bool intel_srar_check(uint64_t status)
+static bool cf_check intel_srar_check(uint64_t status)
{
return (intel_check_mce_type(status) == intel_mce_ucr_srar);
}
-static bool intel_checkaddr(uint64_t status, uint64_t misc, int addrtype)
+static bool cf_check intel_checkaddr(
+ uint64_t status, uint64_t misc, int addrtype)
{
if ( !(status & MCi_STATUS_ADDRV) ||
!(status & MCi_STATUS_MISCV) ||
@@ -287,10 +288,9 @@ static bool intel_checkaddr(uint64_t status, uint64_t misc, int addrtype)
return (addrtype == MC_ADDR_PHYSICAL);
}
-static void intel_srar_dhandler(
- struct mca_binfo *binfo,
- enum mce_result *result,
- const struct cpu_user_regs *regs)
+static void cf_check intel_srar_dhandler(
+ struct mca_binfo *binfo, enum mce_result *result,
+ const struct cpu_user_regs *regs)
{
uint64_t status = binfo->mib->mc_status;
@@ -306,15 +306,14 @@ static void intel_srar_dhandler(
}
}
-static bool intel_srao_check(uint64_t status)
+static bool cf_check intel_srao_check(uint64_t status)
{
return (intel_check_mce_type(status) == intel_mce_ucr_srao);
}
-static void intel_srao_dhandler(
- struct mca_binfo *binfo,
- enum mce_result *result,
- const struct cpu_user_regs *regs)
+static void cf_check intel_srao_dhandler(
+ struct mca_binfo *binfo, enum mce_result *result,
+ const struct cpu_user_regs *regs)
{
uint64_t status = binfo->mib->mc_status;
@@ -333,15 +332,14 @@ static void intel_srao_dhandler(
}
}
-static bool intel_default_check(uint64_t status)
+static bool cf_check intel_default_check(uint64_t status)
{
return true;
}
-static void intel_default_mce_dhandler(
- struct mca_binfo *binfo,
- enum mce_result *result,
- const struct cpu_user_regs * regs)
+static void cf_check intel_default_mce_dhandler(
+ struct mca_binfo *binfo, enum mce_result *result,
+ const struct cpu_user_regs * regs)
{
uint64_t status = binfo->mib->mc_status;
enum intel_mce_type type;
@@ -360,10 +358,9 @@ static const struct mca_error_handler intel_mce_dhandlers[] = {
{intel_default_check, intel_default_mce_dhandler}
};
-static void intel_default_mce_uhandler(
- struct mca_binfo *binfo,
- enum mce_result *result,
- const struct cpu_user_regs *regs)
+static void cf_check intel_default_mce_uhandler(
+ struct mca_binfo *binfo, enum mce_result *result,
+ const struct cpu_user_regs *regs)
{
uint64_t status = binfo->mib->mc_status;
enum intel_mce_type type;
@@ -396,7 +393,7 @@ static const struct mca_error_handler intel_mce_uhandlers[] = {
* 3) ser_support = 1, SRAO, UC = 1, S = 1, AR = 0, [EN = 1]
*/
-static bool intel_need_clearbank_scan(enum mca_source who, u64 status)
+static bool cf_check intel_need_clearbank_scan(enum mca_source who, u64 status)
{
if ( who == MCA_CMCI_HANDLER )
{
@@ -453,7 +450,7 @@ static bool intel_need_clearbank_scan(enum mca_source who, u64 status)
* 4) SRAO ser_support = 1, PCC = 0, S = 1, AR = 0, EN = 1 [UC = 1]
* 5) UCNA ser_support = 1, OVER = 0, EN = 1, PCC = 0, S = 0, AR = 0, [UC = 1]
*/
-static bool intel_recoverable_scan(uint64_t status)
+static bool cf_check intel_recoverable_scan(uint64_t status)
{
if ( !(status & MCi_STATUS_UC ) )
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 44/70] x86/pmu: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (42 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 43/70] x86/mce: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 45/70] x86/cpu: " Andrew Cooper
` (27 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/vpmu_amd.c | 16 ++++++++--------
xen/arch/x86/cpu/vpmu_intel.c | 16 ++++++++--------
xen/arch/x86/oprofile/op_model_athlon.c | 16 ++++++++--------
xen/arch/x86/oprofile/op_model_p4.c | 14 +++++++-------
xen/arch/x86/oprofile/op_model_ppro.c | 26 ++++++++++++++------------
5 files changed, 45 insertions(+), 43 deletions(-)
diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
index 25ad4ecf48a4..5963ce90150a 100644
--- a/xen/arch/x86/cpu/vpmu_amd.c
+++ b/xen/arch/x86/cpu/vpmu_amd.c
@@ -186,7 +186,7 @@ static void amd_vpmu_unset_msr_bitmap(struct vcpu *v)
msr_bitmap_off(vpmu);
}
-static int amd_vpmu_do_interrupt(struct cpu_user_regs *regs)
+static int cf_check amd_vpmu_do_interrupt(struct cpu_user_regs *regs)
{
return 1;
}
@@ -206,7 +206,7 @@ static inline void context_load(struct vcpu *v)
}
}
-static int amd_vpmu_load(struct vcpu *v, bool_t from_guest)
+static int cf_check amd_vpmu_load(struct vcpu *v, bool from_guest)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
struct xen_pmu_amd_ctxt *ctxt;
@@ -280,7 +280,7 @@ static inline void context_save(struct vcpu *v)
rdmsrl(counters[i], counter_regs[i]);
}
-static int amd_vpmu_save(struct vcpu *v, bool_t to_guest)
+static int cf_check amd_vpmu_save(struct vcpu *v, bool to_guest)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
unsigned int i;
@@ -348,7 +348,7 @@ static void context_update(unsigned int msr, u64 msr_content)
}
}
-static int amd_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
+static int cf_check amd_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
{
struct vcpu *v = current;
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -404,7 +404,7 @@ static int amd_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
return 0;
}
-static int amd_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
+static int cf_check amd_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
{
struct vcpu *v = current;
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -422,7 +422,7 @@ static int amd_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
return 0;
}
-static void amd_vpmu_destroy(struct vcpu *v)
+static void cf_check amd_vpmu_destroy(struct vcpu *v)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -440,7 +440,7 @@ static void amd_vpmu_destroy(struct vcpu *v)
}
/* VPMU part of the 'q' keyhandler */
-static void amd_vpmu_dump(const struct vcpu *v)
+static void cf_check amd_vpmu_dump(const struct vcpu *v)
{
const struct vpmu_struct *vpmu = vcpu_vpmu(v);
const struct xen_pmu_amd_ctxt *ctxt = vpmu->context;
@@ -480,7 +480,7 @@ static void amd_vpmu_dump(const struct vcpu *v)
}
}
-static int svm_vpmu_initialise(struct vcpu *v)
+static int cf_check svm_vpmu_initialise(struct vcpu *v)
{
struct xen_pmu_amd_ctxt *ctxt;
struct vpmu_struct *vpmu = vcpu_vpmu(v);
diff --git a/xen/arch/x86/cpu/vpmu_intel.c b/xen/arch/x86/cpu/vpmu_intel.c
index 22dd4469d920..48b81ab6f018 100644
--- a/xen/arch/x86/cpu/vpmu_intel.c
+++ b/xen/arch/x86/cpu/vpmu_intel.c
@@ -288,7 +288,7 @@ static inline void __core2_vpmu_save(struct vcpu *v)
rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, core2_vpmu_cxt->global_status);
}
-static int core2_vpmu_save(struct vcpu *v, bool_t to_guest)
+static int cf_check core2_vpmu_save(struct vcpu *v, bool to_guest)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -407,7 +407,7 @@ static int core2_vpmu_verify(struct vcpu *v)
return 0;
}
-static int core2_vpmu_load(struct vcpu *v, bool_t from_guest)
+static int cf_check core2_vpmu_load(struct vcpu *v, bool from_guest)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -522,7 +522,7 @@ static int core2_vpmu_msr_common_check(u32 msr_index, int *type, int *index)
return 1;
}
-static int core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
+static int cf_check core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
{
int i, tmp;
int type = -1, index = -1;
@@ -690,7 +690,7 @@ static int core2_vpmu_do_wrmsr(unsigned int msr, uint64_t msr_content)
return 0;
}
-static int core2_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
+static int cf_check core2_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
{
int type = -1, index = -1;
struct vcpu *v = current;
@@ -730,7 +730,7 @@ static int core2_vpmu_do_rdmsr(unsigned int msr, uint64_t *msr_content)
}
/* Dump vpmu info on console, called in the context of keyhandler 'q'. */
-static void core2_vpmu_dump(const struct vcpu *v)
+static void cf_check core2_vpmu_dump(const struct vcpu *v)
{
const struct vpmu_struct *vpmu = vcpu_vpmu(v);
unsigned int i;
@@ -775,7 +775,7 @@ static void core2_vpmu_dump(const struct vcpu *v)
}
}
-static int core2_vpmu_do_interrupt(struct cpu_user_regs *regs)
+static int cf_check core2_vpmu_do_interrupt(struct cpu_user_regs *regs)
{
struct vcpu *v = current;
u64 msr_content;
@@ -802,7 +802,7 @@ static int core2_vpmu_do_interrupt(struct cpu_user_regs *regs)
return 1;
}
-static void core2_vpmu_destroy(struct vcpu *v)
+static void cf_check core2_vpmu_destroy(struct vcpu *v)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -816,7 +816,7 @@ static void core2_vpmu_destroy(struct vcpu *v)
vpmu_clear(vpmu);
}
-static int vmx_vpmu_initialise(struct vcpu *v)
+static int cf_check vmx_vpmu_initialise(struct vcpu *v)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
u64 msr_content;
diff --git a/xen/arch/x86/oprofile/op_model_athlon.c b/xen/arch/x86/oprofile/op_model_athlon.c
index 2177f02946e2..7bc5853a6c23 100644
--- a/xen/arch/x86/oprofile/op_model_athlon.c
+++ b/xen/arch/x86/oprofile/op_model_athlon.c
@@ -164,7 +164,7 @@ static inline u64 op_amd_randomize_ibs_op(u64 val)
return val;
}
-static void athlon_fill_in_addresses(struct op_msrs * const msrs)
+static void cf_check athlon_fill_in_addresses(struct op_msrs * const msrs)
{
msrs->counters[0].addr = MSR_K7_PERFCTR0;
msrs->counters[1].addr = MSR_K7_PERFCTR1;
@@ -177,7 +177,7 @@ static void athlon_fill_in_addresses(struct op_msrs * const msrs)
msrs->controls[3].addr = MSR_K7_EVNTSEL3;
}
-static void fam15h_fill_in_addresses(struct op_msrs * const msrs)
+static void cf_check fam15h_fill_in_addresses(struct op_msrs * const msrs)
{
msrs->counters[0].addr = MSR_AMD_FAM15H_PERFCTR0;
msrs->counters[1].addr = MSR_AMD_FAM15H_PERFCTR1;
@@ -194,7 +194,7 @@ static void fam15h_fill_in_addresses(struct op_msrs * const msrs)
msrs->controls[5].addr = MSR_AMD_FAM15H_EVNTSEL5;
}
-static void athlon_setup_ctrs(struct op_msrs const * const msrs)
+static void cf_check athlon_setup_ctrs(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
@@ -308,9 +308,9 @@ static inline int handle_ibs(int mode, struct cpu_user_regs const * const regs)
return 1;
}
-static int athlon_check_ctrs(unsigned int const cpu,
- struct op_msrs const * const msrs,
- struct cpu_user_regs const * const regs)
+static int cf_check athlon_check_ctrs(
+ unsigned int const cpu, struct op_msrs const * const msrs,
+ struct cpu_user_regs const * const regs)
{
uint64_t msr_content;
@@ -386,7 +386,7 @@ static inline void start_ibs(void)
}
}
-static void athlon_start(struct op_msrs const * const msrs)
+static void cf_check athlon_start(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
@@ -415,7 +415,7 @@ static void stop_ibs(void)
wrmsrl(MSR_AMD64_IBSOPCTL, 0);
}
-static void athlon_stop(struct op_msrs const * const msrs)
+static void cf_check athlon_stop(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
diff --git a/xen/arch/x86/oprofile/op_model_p4.c b/xen/arch/x86/oprofile/op_model_p4.c
index b08ba53cbd39..d047258644db 100644
--- a/xen/arch/x86/oprofile/op_model_p4.c
+++ b/xen/arch/x86/oprofile/op_model_p4.c
@@ -390,7 +390,7 @@ static unsigned int get_stagger(void)
static unsigned long reset_value[NUM_COUNTERS_NON_HT];
-static void p4_fill_in_addresses(struct op_msrs * const msrs)
+static void cf_check p4_fill_in_addresses(struct op_msrs * const msrs)
{
unsigned int i;
unsigned int addr, stag;
@@ -530,7 +530,7 @@ static void pmc_setup_one_p4_counter(unsigned int ctr)
}
-static void p4_setup_ctrs(struct op_msrs const * const msrs)
+static void cf_check p4_setup_ctrs(struct op_msrs const * const msrs)
{
unsigned int i;
uint64_t msr_content;
@@ -609,9 +609,9 @@ static void p4_setup_ctrs(struct op_msrs const * const msrs)
}
}
-static int p4_check_ctrs(unsigned int const cpu,
- struct op_msrs const * const msrs,
- struct cpu_user_regs const * const regs)
+static int cf_check p4_check_ctrs(
+ unsigned int const cpu, struct op_msrs const * const msrs,
+ struct cpu_user_regs const * const regs)
{
unsigned long ctr, stag, real;
uint64_t msr_content;
@@ -665,7 +665,7 @@ static int p4_check_ctrs(unsigned int const cpu,
}
-static void p4_start(struct op_msrs const * const msrs)
+static void cf_check p4_start(struct op_msrs const * const msrs)
{
unsigned int stag;
uint64_t msr_content;
@@ -683,7 +683,7 @@ static void p4_start(struct op_msrs const * const msrs)
}
-static void p4_stop(struct op_msrs const * const msrs)
+static void cf_check p4_stop(struct op_msrs const * const msrs)
{
unsigned int stag;
uint64_t msr_content;
diff --git a/xen/arch/x86/oprofile/op_model_ppro.c b/xen/arch/x86/oprofile/op_model_ppro.c
index 72c504a10216..8d7e13ea8777 100644
--- a/xen/arch/x86/oprofile/op_model_ppro.c
+++ b/xen/arch/x86/oprofile/op_model_ppro.c
@@ -63,7 +63,7 @@ static int counter_width = 32;
static unsigned long reset_value[OP_MAX_COUNTER];
int ppro_has_global_ctrl = 0;
-static void ppro_fill_in_addresses(struct op_msrs * const msrs)
+static void cf_check ppro_fill_in_addresses(struct op_msrs * const msrs)
{
int i;
@@ -74,7 +74,7 @@ static void ppro_fill_in_addresses(struct op_msrs * const msrs)
}
-static void ppro_setup_ctrs(struct op_msrs const * const msrs)
+static void cf_check ppro_setup_ctrs(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
@@ -128,9 +128,9 @@ static void ppro_setup_ctrs(struct op_msrs const * const msrs)
}
}
-static int ppro_check_ctrs(unsigned int const cpu,
- struct op_msrs const * const msrs,
- struct cpu_user_regs const * const regs)
+static int cf_check ppro_check_ctrs(
+ unsigned int const cpu, struct op_msrs const * const msrs,
+ struct cpu_user_regs const * const regs)
{
u64 val;
int i;
@@ -170,7 +170,7 @@ static int ppro_check_ctrs(unsigned int const cpu,
}
-static void ppro_start(struct op_msrs const * const msrs)
+static void cf_check ppro_start(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
@@ -190,7 +190,7 @@ static void ppro_start(struct op_msrs const * const msrs)
}
-static void ppro_stop(struct op_msrs const * const msrs)
+static void cf_check ppro_stop(struct op_msrs const * const msrs)
{
uint64_t msr_content;
int i;
@@ -206,7 +206,7 @@ static void ppro_stop(struct op_msrs const * const msrs)
wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, 0x0ULL);
}
-static int ppro_is_arch_pmu_msr(u64 msr_index, int *type, int *index)
+static int cf_check ppro_is_arch_pmu_msr(u64 msr_index, int *type, int *index)
{
if ( (msr_index >= MSR_IA32_PERFCTR0) &&
(msr_index < (MSR_IA32_PERFCTR0 + num_counters)) )
@@ -226,7 +226,7 @@ static int ppro_is_arch_pmu_msr(u64 msr_index, int *type, int *index)
return 0;
}
-static int ppro_allocate_msr(struct vcpu *v)
+static int cf_check ppro_allocate_msr(struct vcpu *v)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
struct arch_msr_pair *msr_content;
@@ -245,7 +245,7 @@ static int ppro_allocate_msr(struct vcpu *v)
return 0;
}
-static void ppro_free_msr(struct vcpu *v)
+static void cf_check ppro_free_msr(struct vcpu *v)
{
struct vpmu_struct *vpmu = vcpu_vpmu(v);
@@ -255,7 +255,8 @@ static void ppro_free_msr(struct vcpu *v)
vpmu_reset(vpmu, VPMU_PASSIVE_DOMAIN_ALLOCATED);
}
-static void ppro_load_msr(struct vcpu *v, int type, int index, u64 *msr_content)
+static void cf_check ppro_load_msr(
+ struct vcpu *v, int type, int index, u64 *msr_content)
{
struct arch_msr_pair *msrs = vcpu_vpmu(v)->context;
switch ( type )
@@ -269,7 +270,8 @@ static void ppro_load_msr(struct vcpu *v, int type, int index, u64 *msr_content)
}
}
-static void ppro_save_msr(struct vcpu *v, int type, int index, u64 msr_content)
+static void cf_check ppro_save_msr(
+ struct vcpu *v, int type, int index, u64 msr_content)
{
struct arch_msr_pair *msrs = vcpu_vpmu(v)->context;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 45/70] x86/cpu: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (43 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 44/70] x86/pmu: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 46/70] x86/guest: " Andrew Cooper
` (26 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/cpu/amd.c | 6 +++---
xen/arch/x86/cpu/centaur.c | 2 +-
xen/arch/x86/cpu/common.c | 2 +-
xen/arch/x86/cpu/cpu.h | 2 +-
xen/arch/x86/cpu/hygon.c | 2 +-
xen/arch/x86/cpu/intel.c | 6 +++---
xen/arch/x86/cpu/shanghai.c | 2 +-
7 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 2d18223f20ef..4999f8be2b11 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -208,7 +208,7 @@ static void __init noinline probe_masking_msrs(void)
* parameter of NULL is used to context switch to the default host state (by
* the cpu bringup-code, crash path, etc).
*/
-static void amd_ctxt_switch_masking(const struct vcpu *next)
+static void cf_check amd_ctxt_switch_masking(const struct vcpu *next)
{
struct cpuidmasks *these_masks = &this_cpu(cpuidmasks);
const struct domain *nextd = next ? next->domain : NULL;
@@ -634,7 +634,7 @@ void amd_log_freq(const struct cpuinfo_x86 *c)
#undef FREQ
}
-void early_init_amd(struct cpuinfo_x86 *c)
+void cf_check early_init_amd(struct cpuinfo_x86 *c)
{
if (c == &boot_cpu_data)
amd_init_levelling();
@@ -744,7 +744,7 @@ void __init detect_zen2_null_seg_behaviour(void)
}
-static void init_amd(struct cpuinfo_x86 *c)
+static void cf_check init_amd(struct cpuinfo_x86 *c)
{
u32 l, h;
diff --git a/xen/arch/x86/cpu/centaur.c b/xen/arch/x86/cpu/centaur.c
index 34a5bfcaeef2..eac49d78db62 100644
--- a/xen/arch/x86/cpu/centaur.c
+++ b/xen/arch/x86/cpu/centaur.c
@@ -48,7 +48,7 @@ static void init_c3(struct cpuinfo_x86 *c)
display_cacheinfo(c);
}
-static void init_centaur(struct cpuinfo_x86 *c)
+static void cf_check init_centaur(struct cpuinfo_x86 *c)
{
if (c->x86 == 6)
init_c3(c);
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index c4f07f2d1da4..6b674bf15e8b 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -104,7 +104,7 @@ bool __init is_forced_cpu_cap(unsigned int cap)
return test_bit(cap, forced_caps);
}
-static void default_init(struct cpuinfo_x86 * c)
+static void cf_check default_init(struct cpuinfo_x86 * c)
{
/* Not much we can do here... */
/* Check if at least it has cpuid */
diff --git a/xen/arch/x86/cpu/cpu.h b/xen/arch/x86/cpu/cpu.h
index b593bd85f04f..a228087f9157 100644
--- a/xen/arch/x86/cpu/cpu.h
+++ b/xen/arch/x86/cpu/cpu.h
@@ -18,7 +18,7 @@ extern void display_cacheinfo(struct cpuinfo_x86 *c);
extern void detect_ht(struct cpuinfo_x86 *c);
extern bool detect_extended_topology(struct cpuinfo_x86 *c);
-void early_init_amd(struct cpuinfo_x86 *c);
+void cf_check early_init_amd(struct cpuinfo_x86 *c);
void amd_log_freq(const struct cpuinfo_x86 *c);
void amd_init_lfence(struct cpuinfo_x86 *c);
void amd_init_ssbd(const struct cpuinfo_x86 *c);
diff --git a/xen/arch/x86/cpu/hygon.c b/xen/arch/x86/cpu/hygon.c
index cdc94130dd2e..3c8516e014c3 100644
--- a/xen/arch/x86/cpu/hygon.c
+++ b/xen/arch/x86/cpu/hygon.c
@@ -28,7 +28,7 @@ static void hygon_get_topology(struct cpuinfo_x86 *c)
c->phys_proc_id, c->cpu_core_id);
}
-static void init_hygon(struct cpuinfo_x86 *c)
+static void cf_check init_hygon(struct cpuinfo_x86 *c)
{
unsigned long long value;
diff --git a/xen/arch/x86/cpu/intel.c b/xen/arch/x86/cpu/intel.c
index 06b0e552cc8f..ff7c02223687 100644
--- a/xen/arch/x86/cpu/intel.c
+++ b/xen/arch/x86/cpu/intel.c
@@ -176,7 +176,7 @@ static void __init probe_masking_msrs(void)
* parameter of NULL is used to context switch to the default host state (by
* the cpu bringup-code, crash path, etc).
*/
-static void intel_ctxt_switch_masking(const struct vcpu *next)
+static void cf_check intel_ctxt_switch_masking(const struct vcpu *next)
{
struct cpuidmasks *these_masks = &this_cpu(cpuidmasks);
const struct domain *nextd = next ? next->domain : NULL;
@@ -286,7 +286,7 @@ static void __init noinline intel_init_levelling(void)
ctxt_switch_masking = intel_ctxt_switch_masking;
}
-static void early_init_intel(struct cpuinfo_x86 *c)
+static void cf_check early_init_intel(struct cpuinfo_x86 *c)
{
u64 misc_enable, disable;
@@ -500,7 +500,7 @@ static void intel_log_freq(const struct cpuinfo_x86 *c)
}
}
-static void init_intel(struct cpuinfo_x86 *c)
+static void cf_check init_intel(struct cpuinfo_x86 *c)
{
/* Detect the extended topology information if available */
detect_extended_topology(c);
diff --git a/xen/arch/x86/cpu/shanghai.c b/xen/arch/x86/cpu/shanghai.c
index 08a81f0f0c8e..95ae544f8c54 100644
--- a/xen/arch/x86/cpu/shanghai.c
+++ b/xen/arch/x86/cpu/shanghai.c
@@ -3,7 +3,7 @@
#include <asm/processor.h>
#include "cpu.h"
-static void init_shanghai(struct cpuinfo_x86 *c)
+static void cf_check init_shanghai(struct cpuinfo_x86 *c)
{
if ( cpu_has(c, X86_FEATURE_ITSC) )
{
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 46/70] x86/guest: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (44 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 45/70] x86/cpu: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 47/70] x86/logdirty: " Andrew Cooper
` (25 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/guest/hyperv/hyperv.c | 10 +++++-----
xen/arch/x86/guest/xen/xen.c | 11 ++++++-----
2 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 84221b751453..b101ba3080b4 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -175,7 +175,7 @@ static int setup_vp_assist(void)
return 0;
}
-static void __init setup(void)
+static void __init cf_check setup(void)
{
ASM_CONSTANT(HV_HCALL_PAGE, __fix_x_to_virt(FIX_X_HYPERV_HCALL));
@@ -188,7 +188,7 @@ static void __init setup(void)
panic("VP assist page setup failed\n");
}
-static int ap_setup(void)
+static int cf_check ap_setup(void)
{
int rc;
@@ -199,7 +199,7 @@ static int ap_setup(void)
return setup_vp_assist();
}
-static void __init e820_fixup(struct e820map *e820)
+static void __init cf_check e820_fixup(struct e820map *e820)
{
uint64_t s = HV_HCALL_MFN << PAGE_SHIFT;
@@ -207,8 +207,8 @@ static void __init e820_fixup(struct e820map *e820)
panic("Unable to reserve Hyper-V hypercall range\n");
}
-static int flush_tlb(const cpumask_t *mask, const void *va,
- unsigned int flags)
+static int cf_check flush_tlb(
+ const cpumask_t *mask, const void *va, unsigned int flags)
{
if ( !(ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) )
return -EOPNOTSUPP;
diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c
index 17807cdea688..9c2defaa6621 100644
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -237,7 +237,7 @@ static int init_evtchn(void)
return rc;
}
-static void __init setup(void)
+static void __init cf_check setup(void)
{
init_memmap();
@@ -265,7 +265,7 @@ static void __init setup(void)
BUG_ON(init_evtchn());
}
-static int ap_setup(void)
+static int cf_check ap_setup(void)
{
set_vcpu_id();
@@ -295,7 +295,7 @@ static void cf_check ap_resume(void *unused)
BUG_ON(init_evtchn());
}
-static void resume(void)
+static void cf_check resume(void)
{
/* Reset shared info page. */
map_shared_info();
@@ -318,13 +318,14 @@ static void resume(void)
pv_console_init();
}
-static void __init e820_fixup(struct e820map *e820)
+static void __init cf_check e820_fixup(struct e820map *e820)
{
if ( pv_shim )
pv_shim_fixup_e820(e820);
}
-static int flush_tlb(const cpumask_t *mask, const void *va, unsigned int flags)
+static int cf_check flush_tlb(
+ const cpumask_t *mask, const void *va, unsigned int flags)
{
return xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL);
}
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 47/70] x86/logdirty: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (45 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 46/70] x86/guest: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 48/70] x86/shadow: " Andrew Cooper
` (24 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/mm/hap/hap.c | 6 +++---
xen/arch/x86/mm/shadow/common.c | 12 ++++++------
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index de4b13565ab4..ed5112b00b63 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -180,7 +180,7 @@ int hap_track_dirty_vram(struct domain *d,
* NB: Domain that having device assigned should not set log_global. Because
* there is no way to track the memory updating from device.
*/
-static int hap_enable_log_dirty(struct domain *d, bool_t log_global)
+static int cf_check hap_enable_log_dirty(struct domain *d, bool log_global)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
@@ -211,7 +211,7 @@ static int hap_enable_log_dirty(struct domain *d, bool_t log_global)
return 0;
}
-static int hap_disable_log_dirty(struct domain *d)
+static int cf_check hap_disable_log_dirty(struct domain *d)
{
paging_lock(d);
d->arch.paging.mode &= ~PG_log_dirty;
@@ -228,7 +228,7 @@ static int hap_disable_log_dirty(struct domain *d)
return 0;
}
-static void hap_clean_dirty_bitmap(struct domain *d)
+static void cf_check hap_clean_dirty_bitmap(struct domain *d)
{
/*
* Switch to log-dirty mode, either by setting l1e entries of P2M table to
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 83dedc8870aa..071a19adce82 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -40,9 +40,9 @@
DEFINE_PER_CPU(uint32_t,trace_shadow_path_flags);
-static int sh_enable_log_dirty(struct domain *, bool log_global);
-static int sh_disable_log_dirty(struct domain *);
-static void sh_clean_dirty_bitmap(struct domain *);
+static int cf_check sh_enable_log_dirty(struct domain *, bool log_global);
+static int cf_check sh_disable_log_dirty(struct domain *);
+static void cf_check sh_clean_dirty_bitmap(struct domain *);
/* Set up the shadow-specific parts of a domain struct at start of day.
* Called for every domain from arch_domain_create() */
@@ -3016,7 +3016,7 @@ static int shadow_test_disable(struct domain *d)
/* Shadow specific code which is called in paging_log_dirty_enable().
* Return 0 if no problem found.
*/
-static int sh_enable_log_dirty(struct domain *d, bool log_global)
+static int cf_check sh_enable_log_dirty(struct domain *d, bool log_global)
{
int ret;
@@ -3044,7 +3044,7 @@ static int sh_enable_log_dirty(struct domain *d, bool log_global)
}
/* shadow specfic code which is called in paging_log_dirty_disable() */
-static int sh_disable_log_dirty(struct domain *d)
+static int cf_check sh_disable_log_dirty(struct domain *d)
{
int ret;
@@ -3058,7 +3058,7 @@ static int sh_disable_log_dirty(struct domain *d)
/* This function is called when we CLEAN log dirty bitmap. See
* paging_log_dirty_op() for details.
*/
-static void sh_clean_dirty_bitmap(struct domain *d)
+static void cf_check sh_clean_dirty_bitmap(struct domain *d)
{
paging_lock(d);
/* Need to revoke write access to the domain's pages again.
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 48/70] x86/shadow: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (46 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 47/70] x86/logdirty: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 49/70] x86/hap: " Andrew Cooper
` (23 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/mm/shadow/common.c | 11 +++---
xen/arch/x86/mm/shadow/hvm.c | 8 ++--
xen/arch/x86/mm/shadow/multi.c | 80 ++++++++++++++++++++--------------------
xen/arch/x86/mm/shadow/multi.h | 20 +++++-----
xen/arch/x86/mm/shadow/private.h | 12 +++---
xen/arch/x86/mm/shadow/pv.c | 4 +-
6 files changed, 67 insertions(+), 68 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 071a19adce82..8f111901730f 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1215,7 +1215,7 @@ void shadow_free(struct domain *d, mfn_t smfn)
* This action is irreversible: the p2m mapping only ever grows.
* That's OK because the p2m table only exists for translated domains,
* and those domains can't ever turn off shadow mode. */
-static struct page_info *
+static struct page_info *cf_check
shadow_alloc_p2m_page(struct domain *d)
{
struct page_info *pg;
@@ -1251,7 +1251,7 @@ shadow_alloc_p2m_page(struct domain *d)
return pg;
}
-static void
+static void cf_check
shadow_free_p2m_page(struct domain *d, struct page_info *pg)
{
struct domain *owner = page_get_owner(pg);
@@ -2290,7 +2290,8 @@ void shadow_prepare_page_type_change(struct domain *d, struct page_info *page,
/* Reset the up-pointers of every L3 shadow to 0.
* This is called when l3 shadows stop being pinnable, to clear out all
* the list-head bits so the up-pointer field is properly inititalised. */
-static int sh_clear_up_pointer(struct vcpu *v, mfn_t smfn, mfn_t unused)
+static int cf_check sh_clear_up_pointer(
+ struct vcpu *v, mfn_t smfn, mfn_t unused)
{
mfn_to_page(smfn)->up = 0;
return 0;
@@ -2490,7 +2491,7 @@ static void sh_update_paging_modes(struct vcpu *v)
v->arch.paging.mode->update_cr3(v, 0, false);
}
-void shadow_update_paging_modes(struct vcpu *v)
+void cf_check shadow_update_paging_modes(struct vcpu *v)
{
paging_lock(v->domain);
sh_update_paging_modes(v);
@@ -3075,7 +3076,7 @@ static bool flush_vcpu(const struct vcpu *v, const unsigned long *vcpu_bitmap)
}
/* Flush TLB of selected vCPUs. NULL for all. */
-bool shadow_flush_tlb(const unsigned long *vcpu_bitmap)
+bool cf_check shadow_flush_tlb(const unsigned long *vcpu_bitmap)
{
static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
cpumask_t *mask = &this_cpu(flush_cpumask);
diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c
index c90d326becb3..27dd99f1a12e 100644
--- a/xen/arch/x86/mm/shadow/hvm.c
+++ b/xen/arch/x86/mm/shadow/hvm.c
@@ -794,9 +794,9 @@ sh_remove_all_shadows_and_parents(struct domain *d, mfn_t gmfn)
* It means extra emulated writes and slows down removal of mappings. */
}
-static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
- l1_pgentry_t old, l1_pgentry_t new,
- unsigned int level)
+static void cf_check sh_unshadow_for_p2m_change(
+ struct domain *d, unsigned long gfn, l1_pgentry_t old, l1_pgentry_t new,
+ unsigned int level)
{
mfn_t omfn = l1e_get_mfn(old);
unsigned int oflags = l1e_get_flags(old);
@@ -879,7 +879,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
}
#if (SHADOW_OPTIMIZATIONS & SHOPT_FAST_FAULT_PATH)
-static void
+static void cf_check
sh_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
{
struct domain *d = p2m->domain;
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index bddef53163f5..b0b1c31ee033 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -369,7 +369,7 @@ static void sh_audit_gw(struct vcpu *v, const walk_t *gw)
#if GUEST_PAGING_LEVELS == 2
/* From one page of a multi-page shadow, find the next one */
-static inline mfn_t sh_next_page(mfn_t smfn)
+static inline mfn_t cf_check sh_next_page(mfn_t smfn)
{
struct page_info *pg = mfn_to_page(smfn), *next;
struct page_list_head h = PAGE_LIST_HEAD_INIT(h);
@@ -399,8 +399,7 @@ guest_index(void *ptr)
return (u32)((unsigned long)ptr & ~PAGE_MASK) / sizeof(guest_l1e_t);
}
-static u32
-shadow_l1_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l1_index(mfn_t *smfn, u32 guest_index)
{
#if (GUEST_PAGING_LEVELS == 2)
ASSERT(mfn_to_page(*smfn)->u.sh.head);
@@ -412,8 +411,7 @@ shadow_l1_index(mfn_t *smfn, u32 guest_index)
#endif
}
-static u32
-shadow_l2_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l2_index(mfn_t *smfn, u32 guest_index)
{
#if (GUEST_PAGING_LEVELS == 2)
int i;
@@ -432,14 +430,12 @@ shadow_l2_index(mfn_t *smfn, u32 guest_index)
#if GUEST_PAGING_LEVELS >= 4
-static u32
-shadow_l3_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l3_index(mfn_t *smfn, u32 guest_index)
{
return guest_index;
}
-static u32
-shadow_l4_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l4_index(mfn_t *smfn, u32 guest_index)
{
return guest_index;
}
@@ -924,7 +920,7 @@ do { \
/**************************************************************************/
/* Create a shadow of a given guest page.
*/
-static mfn_t
+static mfn_t cf_check
sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
{
struct domain *d = v->domain;
@@ -1459,7 +1455,8 @@ void sh_unhook_64b_mappings(struct domain *d, mfn_t sl4mfn, int user_only)
*/
#if GUEST_PAGING_LEVELS >= 4
-static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
+static int cf_check validate_gl4e(
+ struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
{
shadow_l4e_t new_sl4e;
guest_l4e_t new_gl4e = *(guest_l4e_t *)new_ge;
@@ -1518,7 +1515,8 @@ static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
}
-static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
+static int cf_check validate_gl3e(
+ struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
{
struct domain *d = v->domain;
shadow_l3e_t new_sl3e;
@@ -1552,7 +1550,8 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
}
#endif // GUEST_PAGING_LEVELS >= 4
-static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
+static int cf_check validate_gl2e(
+ struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
{
struct domain *d = v->domain;
shadow_l2e_t new_sl2e;
@@ -1599,7 +1598,8 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
return result;
}
-static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
+static int cf_check validate_gl1e(
+ struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
{
struct domain *d = v->domain;
shadow_l1e_t new_sl1e;
@@ -2089,8 +2089,8 @@ static DEFINE_PER_CPU(int,trace_extra_emulation_count);
#endif
static DEFINE_PER_CPU(guest_pa_t,trace_emulate_write_val);
-static void trace_emulate_write_val(const void *ptr, unsigned long vaddr,
- const void *src, unsigned int bytes)
+static void cf_check trace_emulate_write_val(
+ const void *ptr, unsigned long vaddr, const void *src, unsigned int bytes)
{
#if GUEST_PAGING_LEVELS == 3
if ( vaddr == this_cpu(trace_emulate_initial_va) )
@@ -2144,9 +2144,8 @@ static inline void trace_shadow_emulate(guest_l1e_t gl1e, unsigned long va)
* shadow code (and the guest should retry) or 0 if it is not (and the
* fault should be handled elsewhere or passed to the guest). */
-static int sh_page_fault(struct vcpu *v,
- unsigned long va,
- struct cpu_user_regs *regs)
+static int cf_check sh_page_fault(
+ struct vcpu *v, unsigned long va, struct cpu_user_regs *regs)
{
struct domain *d = v->domain;
walk_t gw;
@@ -2898,7 +2897,7 @@ static int sh_page_fault(struct vcpu *v,
* instruction should be issued on the hardware, or false if it's safe not
* to do so.
*/
-static bool sh_invlpg(struct vcpu *v, unsigned long linear)
+static bool cf_check sh_invlpg(struct vcpu *v, unsigned long linear)
{
mfn_t sl1mfn;
shadow_l2e_t sl2e;
@@ -3030,9 +3029,8 @@ static bool sh_invlpg(struct vcpu *v, unsigned long linear)
#ifdef CONFIG_HVM
-static unsigned long
-sh_gva_to_gfn(struct vcpu *v, struct p2m_domain *p2m,
- unsigned long va, uint32_t *pfec)
+static unsigned long cf_check sh_gva_to_gfn(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long va, uint32_t *pfec)
/* Called to translate a guest virtual address to what the *guest*
* pagetables would map it to. */
{
@@ -3196,8 +3194,7 @@ sh_update_linear_entries(struct vcpu *v)
* Removes v->arch.paging.shadow.shadow_table[].
* Does all appropriate management/bookkeeping/refcounting/etc...
*/
-static void
-sh_detach_old_tables(struct vcpu *v)
+static void cf_check sh_detach_old_tables(struct vcpu *v)
{
struct domain *d = v->domain;
mfn_t smfn;
@@ -3216,8 +3213,7 @@ sh_detach_old_tables(struct vcpu *v)
}
}
-static void
-sh_update_cr3(struct vcpu *v, int do_locking, bool noflush)
+static void cf_check sh_update_cr3(struct vcpu *v, int do_locking, bool noflush)
/* Updates vcpu->arch.cr3 after the guest has changed CR3.
* Paravirtual guests should set v->arch.guest_table (and guest_table_user,
* if appropriate).
@@ -3525,7 +3521,8 @@ int sh_rm_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
#endif /* OOS */
#if defined(CONFIG_HVM) && (SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC)
-static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
+static int cf_check sh_guess_wrmap(
+ struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
/* Look up this vaddr in the current shadow and see if it's a writeable
* mapping of this gmfn. If so, remove it. Returns 1 if it worked. */
{
@@ -3589,8 +3586,8 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
}
#endif
-int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
- mfn_t readonly_mfn)
+int cf_check sh_rm_write_access_from_l1(
+ struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn)
/* Excises all writeable mappings to readonly_mfn from this l1 shadow table */
{
shadow_l1e_t *sl1e;
@@ -3626,7 +3623,8 @@ int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
}
-int sh_rm_mappings_from_l1(struct domain *d, mfn_t sl1mfn, mfn_t target_mfn)
+int cf_check sh_rm_mappings_from_l1(
+ struct domain *d, mfn_t sl1mfn, mfn_t target_mfn)
/* Excises all mappings to guest frame from this shadow l1 table */
{
shadow_l1e_t *sl1e;
@@ -3677,7 +3675,7 @@ void sh_clear_shadow_entry(struct domain *d, void *ep, mfn_t smfn)
}
}
-int sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
+int cf_check sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
/* Remove all mappings of this l1 shadow from this l2 shadow */
{
shadow_l2e_t *sl2e;
@@ -3700,7 +3698,7 @@ int sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
}
#if GUEST_PAGING_LEVELS >= 4
-int sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
+int cf_check sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
/* Remove all mappings of this l2 shadow from this l3 shadow */
{
shadow_l3e_t *sl3e;
@@ -3722,7 +3720,7 @@ int sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
return done;
}
-int sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
+int cf_check sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
/* Remove all mappings of this l3 shadow from this l4 shadow */
{
shadow_l4e_t *sl4e;
@@ -3752,7 +3750,7 @@ int sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
* and in the meantime we unhook its top-level user-mode entries. */
#if GUEST_PAGING_LEVELS == 3
-static void sh_pagetable_dying(paddr_t gpa)
+static void cf_check sh_pagetable_dying(paddr_t gpa)
{
struct vcpu *v = current;
struct domain *d = v->domain;
@@ -3833,7 +3831,7 @@ static void sh_pagetable_dying(paddr_t gpa)
put_gfn(d, l3gfn);
}
#else
-static void sh_pagetable_dying(paddr_t gpa)
+static void cf_check sh_pagetable_dying(paddr_t gpa)
{
struct vcpu *v = current;
struct domain *d = v->domain;
@@ -3932,7 +3930,7 @@ static const char *sh_audit_flags(struct vcpu *v, int level,
return NULL;
}
-int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
+int cf_check sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
{
guest_l1e_t *gl1e, *gp;
shadow_l1e_t *sl1e;
@@ -4000,7 +3998,7 @@ int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
return done;
}
-int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
+int cf_check sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
{
guest_l1e_t *gl1e, e;
shadow_l1e_t *sl1e;
@@ -4026,7 +4024,7 @@ int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
return 0;
}
-int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
+int cf_check sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l2e_t *gl2e, *gp;
@@ -4078,7 +4076,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
}
#if GUEST_PAGING_LEVELS >= 4
-int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
+int cf_check sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l3e_t *gl3e, *gp;
@@ -4126,7 +4124,7 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
return 0;
}
-int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
+int cf_check sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l4e_t *gl4e, *gp;
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 0bd6a2d5b787..5bcd6ae1a8da 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -59,10 +59,10 @@ extern void
SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, GUEST_LEVELS)
(struct domain *d, mfn_t sl4mfn, int user_only);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, GUEST_LEVELS)
(struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, GUEST_LEVELS)
(struct domain *d, mfn_t sl1mfn, mfn_t target_mfn);
@@ -70,30 +70,30 @@ extern void
SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, GUEST_LEVELS)
(struct domain *d, void *ep, mfn_t smfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l1_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l2_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l3_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn);
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_fl1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l2_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl2mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l3_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl3mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l4_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl4mfn, mfn_t x);
#endif
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index e4db8d32546a..3dc024e30f20 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -420,15 +420,15 @@ static inline int sh_remove_write_access(struct domain *d, mfn_t readonly_mfn,
#endif
/* Functions that atomically write PV guest PT entries */
-void sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new,
- mfn_t gmfn);
-intpte_t sh_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p, intpte_t old,
- intpte_t new, mfn_t gmfn);
+void cf_check sh_write_guest_entry(
+ struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn);
+intpte_t cf_check sh_cmpxchg_guest_entry(
+ struct vcpu *v, intpte_t *p, intpte_t old, intpte_t new, mfn_t gmfn);
/* Update all the things that are derived from the guest's CR0/CR3/CR4.
* Called to initialize paging structures if the paging mode
* has changed, and when bringing up a VCPU for the first time. */
-void shadow_update_paging_modes(struct vcpu *v);
+void cf_check shadow_update_paging_modes(struct vcpu *v);
/* Unhook the non-Xen mappings in this top-level shadow mfn.
* With user_only == 1, unhooks only the user-mode mappings. */
@@ -922,7 +922,7 @@ static inline int sh_check_page_has_no_refs(struct page_info *page)
}
/* Flush the TLB of the selected vCPUs. */
-bool shadow_flush_tlb(const unsigned long *vcpu_bitmap);
+bool cf_check shadow_flush_tlb(const unsigned long *vcpu_bitmap);
#endif /* _XEN_SHADOW_PRIVATE_H */
diff --git a/xen/arch/x86/mm/shadow/pv.c b/xen/arch/x86/mm/shadow/pv.c
index f51f980f2694..ed10d5479c5e 100644
--- a/xen/arch/x86/mm/shadow/pv.c
+++ b/xen/arch/x86/mm/shadow/pv.c
@@ -28,7 +28,7 @@
* Write a new value into the guest pagetable, and update the shadows
* appropriately.
*/
-void
+void cf_check
sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn)
{
paging_lock(v->domain);
@@ -42,7 +42,7 @@ sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn)
* appropriately. Returns the previous entry found, which the caller is
* expected to check to see if the cmpxchg was successful.
*/
-intpte_t
+intpte_t cf_check
sh_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p, intpte_t old,
intpte_t new, mfn_t gmfn)
{
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 49/70] x86/hap: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (47 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 48/70] x86/shadow: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 50/70] x86/p2m: " Andrew Cooper
` (22 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/mm/hap/guest_walk.c | 4 ++--
xen/arch/x86/mm/hap/hap.c | 21 +++++++++++----------
xen/arch/x86/mm/hap/private.h | 30 ++++++++++++------------------
3 files changed, 25 insertions(+), 30 deletions(-)
diff --git a/xen/arch/x86/mm/hap/guest_walk.c b/xen/arch/x86/mm/hap/guest_walk.c
index 832a8058471e..1da8d3b99edc 100644
--- a/xen/arch/x86/mm/hap/guest_walk.c
+++ b/xen/arch/x86/mm/hap/guest_walk.c
@@ -36,14 +36,14 @@
#include <asm/guest_pt.h>
#include <asm/p2m.h>
-unsigned long hap_gva_to_gfn(GUEST_PAGING_LEVELS)(
+unsigned long cf_check hap_gva_to_gfn(GUEST_PAGING_LEVELS)(
struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t *pfec)
{
unsigned long cr3 = v->arch.hvm.guest_cr[3];
return hap_p2m_ga_to_gfn(GUEST_PAGING_LEVELS)(v, p2m, cr3, gva, pfec, NULL);
}
-unsigned long hap_p2m_ga_to_gfn(GUEST_PAGING_LEVELS)(
+unsigned long cf_check hap_p2m_ga_to_gfn(GUEST_PAGING_LEVELS)(
struct vcpu *v, struct p2m_domain *p2m, unsigned long cr3,
paddr_t ga, uint32_t *pfec, unsigned int *page_order)
{
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index ed5112b00b63..9d67a47f5fe9 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -268,7 +268,7 @@ static void hap_free(struct domain *d, mfn_t mfn)
page_list_add_tail(pg, &d->arch.paging.hap.freelist);
}
-static struct page_info *hap_alloc_p2m_page(struct domain *d)
+static struct page_info *cf_check hap_alloc_p2m_page(struct domain *d)
{
struct page_info *pg;
@@ -294,7 +294,7 @@ static struct page_info *hap_alloc_p2m_page(struct domain *d)
return pg;
}
-static void hap_free_p2m_page(struct domain *d, struct page_info *pg)
+static void cf_check hap_free_p2m_page(struct domain *d, struct page_info *pg)
{
struct domain *owner = page_get_owner(pg);
@@ -662,8 +662,8 @@ void hap_vcpu_init(struct vcpu *v)
* HAP guests can handle page faults (in the guest page tables) without
* needing any action from Xen, so we should not be intercepting them.
*/
-static int hap_page_fault(struct vcpu *v, unsigned long va,
- struct cpu_user_regs *regs)
+static int cf_check hap_page_fault(
+ struct vcpu *v, unsigned long va, struct cpu_user_regs *regs)
{
struct domain *d = v->domain;
@@ -677,7 +677,7 @@ static int hap_page_fault(struct vcpu *v, unsigned long va,
* should not be intercepting it. However, we need to correctly handle
* getting here from instruction emulation.
*/
-static bool_t hap_invlpg(struct vcpu *v, unsigned long linear)
+static bool cf_check hap_invlpg(struct vcpu *v, unsigned long linear)
{
/*
* Emulate INVLPGA:
@@ -690,7 +690,8 @@ static bool_t hap_invlpg(struct vcpu *v, unsigned long linear)
return 1;
}
-static void hap_update_cr3(struct vcpu *v, int do_locking, bool noflush)
+static void cf_check hap_update_cr3(
+ struct vcpu *v, int do_locking, bool noflush)
{
v->arch.hvm.hw_cr[3] = v->arch.hvm.guest_cr[3];
hvm_update_guest_cr3(v, noflush);
@@ -702,7 +703,7 @@ static bool flush_vcpu(const struct vcpu *v, const unsigned long *vcpu_bitmap)
}
/* Flush TLB of selected vCPUs. NULL for all. */
-static bool flush_tlb(const unsigned long *vcpu_bitmap)
+static bool cf_check flush_tlb(const unsigned long *vcpu_bitmap)
{
static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
cpumask_t *mask = &this_cpu(flush_cpumask);
@@ -747,7 +748,7 @@ hap_paging_get_mode(struct vcpu *v)
&hap_paging_protected_mode);
}
-static void hap_update_paging_modes(struct vcpu *v)
+static void cf_check hap_update_paging_modes(struct vcpu *v)
{
struct domain *d = v->domain;
unsigned long cr3_gfn = v->arch.hvm.guest_cr[3] >> PAGE_SHIFT;
@@ -791,13 +792,13 @@ void hap_p2m_init(struct p2m_domain *p2m)
p2m->write_p2m_entry_post = hap_write_p2m_entry_post;
}
-static unsigned long hap_gva_to_gfn_real_mode(
+static unsigned long cf_check hap_gva_to_gfn_real_mode(
struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t *pfec)
{
return ((paddr_t)gva >> PAGE_SHIFT);
}
-static unsigned long hap_p2m_ga_to_gfn_real_mode(
+static unsigned long cf_check hap_p2m_ga_to_gfn_real_mode(
struct vcpu *v, struct p2m_domain *p2m, unsigned long cr3,
paddr_t ga, uint32_t *pfec, unsigned int *page_order)
{
diff --git a/xen/arch/x86/mm/hap/private.h b/xen/arch/x86/mm/hap/private.h
index 973fbe8be59c..1040eaf69f43 100644
--- a/xen/arch/x86/mm/hap/private.h
+++ b/xen/arch/x86/mm/hap/private.h
@@ -24,27 +24,21 @@
/********************************************/
/* GUEST TRANSLATION FUNCS */
/********************************************/
-unsigned long hap_gva_to_gfn_2_levels(struct vcpu *v,
- struct p2m_domain *p2m,
- unsigned long gva,
- uint32_t *pfec);
-unsigned long hap_gva_to_gfn_3_levels(struct vcpu *v,
- struct p2m_domain *p2m,
- unsigned long gva,
- uint32_t *pfec);
-unsigned long hap_gva_to_gfn_4_levels(struct vcpu *v,
- struct p2m_domain *p2m,
- unsigned long gva,
- uint32_t *pfec);
+unsigned long cf_check hap_gva_to_gfn_2_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t *pfec);
+unsigned long cf_check hap_gva_to_gfn_3_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t *pfec);
+unsigned long cf_check hap_gva_to_gfn_4_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t *pfec);
-unsigned long hap_p2m_ga_to_gfn_2_levels(struct vcpu *v,
- struct p2m_domain *p2m, unsigned long cr3,
+unsigned long cf_check hap_p2m_ga_to_gfn_2_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long cr3,
paddr_t ga, uint32_t *pfec, unsigned int *page_order);
-unsigned long hap_p2m_ga_to_gfn_3_levels(struct vcpu *v,
- struct p2m_domain *p2m, unsigned long cr3,
+unsigned long cf_check hap_p2m_ga_to_gfn_3_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long cr3,
paddr_t ga, uint32_t *pfec, unsigned int *page_order);
-unsigned long hap_p2m_ga_to_gfn_4_levels(struct vcpu *v,
- struct p2m_domain *p2m, unsigned long cr3,
+unsigned long cf_check hap_p2m_ga_to_gfn_4_levels(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long cr3,
paddr_t ga, uint32_t *pfec, unsigned int *page_order);
#endif /* __HAP_PRIVATE_H__ */
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 50/70] x86/p2m: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (48 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 49/70] x86/hap: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 51/70] x86/irq: " Andrew Cooper
` (21 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/include/asm/p2m.h | 4 ++--
xen/arch/x86/mm/hap/hap.c | 2 +-
xen/arch/x86/mm/hap/nested_hap.c | 2 +-
xen/arch/x86/mm/p2m-ept.c | 32 +++++++++++++++-----------------
xen/arch/x86/mm/p2m-pt.c | 19 +++++++++----------
5 files changed, 28 insertions(+), 31 deletions(-)
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 6e2206874d62..787374426371 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -820,8 +820,8 @@ void np2m_flush_base(struct vcpu *v, unsigned long np2m_base);
void hap_p2m_init(struct p2m_domain *p2m);
void shadow_p2m_init(struct p2m_domain *p2m);
-void nestedp2m_write_p2m_entry_post(struct p2m_domain *p2m,
- unsigned int oflags);
+void cf_check nestedp2m_write_p2m_entry_post(
+ struct p2m_domain *p2m, unsigned int oflags);
/*
* Alternate p2m: shadow p2m tables used for alternate memory views
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index 9d67a47f5fe9..c19e337d6585 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -778,7 +778,7 @@ static void cf_check hap_update_paging_modes(struct vcpu *v)
put_gfn(d, cr3_gfn);
}
-static void
+static void cf_check
hap_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
{
struct domain *d = p2m->domain;
diff --git a/xen/arch/x86/mm/hap/nested_hap.c b/xen/arch/x86/mm/hap/nested_hap.c
index d8a7b3b40167..dbe5ad23a112 100644
--- a/xen/arch/x86/mm/hap/nested_hap.c
+++ b/xen/arch/x86/mm/hap/nested_hap.c
@@ -71,7 +71,7 @@
/* NESTED VIRT P2M FUNCTIONS */
/********************************************/
-void
+void cf_check
nestedp2m_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
{
if ( oflags & _PAGE_PRESENT )
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index a8a6ad629528..70a401c3a7ea 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -624,7 +624,7 @@ int epte_get_entry_emt(struct domain *d, gfn_t gfn, mfn_t mfn,
* - zero if no adjustment was done,
* - a positive value if at least one adjustment was done.
*/
-static int resolve_misconfig(struct p2m_domain *p2m, unsigned long gfn)
+static int cf_check resolve_misconfig(struct p2m_domain *p2m, unsigned long gfn)
{
struct ept_data *ept = &p2m->ept;
unsigned int level = ept->wl;
@@ -793,7 +793,7 @@ bool_t ept_handle_misconfig(uint64_t gpa)
*
* Returns: 0 for success, -errno for failure
*/
-static int
+static int cf_check
ept_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
unsigned int order, p2m_type_t p2mt, p2m_access_t p2ma,
int sve)
@@ -1002,10 +1002,9 @@ ept_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
}
/* Read ept p2m entries */
-static mfn_t ept_get_entry(struct p2m_domain *p2m,
- gfn_t gfn_, p2m_type_t *t, p2m_access_t* a,
- p2m_query_t q, unsigned int *page_order,
- bool_t *sve)
+static mfn_t cf_check ept_get_entry(
+ struct p2m_domain *p2m, gfn_t gfn_, p2m_type_t *t, p2m_access_t *a,
+ p2m_query_t q, unsigned int *page_order, bool *sve)
{
ept_entry_t *table =
map_domain_page(pagetable_get_mfn(p2m_get_pagetable(p2m)));
@@ -1165,8 +1164,8 @@ void ept_walk_table(struct domain *d, unsigned long gfn)
return;
}
-static void ept_change_entry_type_global(struct p2m_domain *p2m,
- p2m_type_t ot, p2m_type_t nt)
+static void cf_check ept_change_entry_type_global(
+ struct p2m_domain *p2m, p2m_type_t ot, p2m_type_t nt)
{
unsigned long mfn = p2m->ept.mfn;
@@ -1177,10 +1176,9 @@ static void ept_change_entry_type_global(struct p2m_domain *p2m,
ept_sync_domain(p2m);
}
-static int ept_change_entry_type_range(struct p2m_domain *p2m,
- p2m_type_t ot, p2m_type_t nt,
- unsigned long first_gfn,
- unsigned long last_gfn)
+static int cf_check ept_change_entry_type_range(
+ struct p2m_domain *p2m, p2m_type_t ot, p2m_type_t nt,
+ unsigned long first_gfn, unsigned long last_gfn)
{
unsigned int i, wl = p2m->ept.wl;
unsigned long mask = (1 << EPT_TABLE_ORDER) - 1;
@@ -1224,7 +1222,7 @@ static int ept_change_entry_type_range(struct p2m_domain *p2m,
return rc < 0 ? rc : 0;
}
-static void ept_memory_type_changed(struct p2m_domain *p2m)
+static void cf_check ept_memory_type_changed(struct p2m_domain *p2m)
{
unsigned long mfn = p2m->ept.mfn;
@@ -1283,7 +1281,7 @@ void ept_sync_domain(struct p2m_domain *p2m)
ept_sync_domain_mask(p2m, d->dirty_cpumask);
}
-static void ept_tlb_flush(struct p2m_domain *p2m)
+static void cf_check ept_tlb_flush(struct p2m_domain *p2m)
{
ept_sync_domain_mask(p2m, p2m->domain->dirty_cpumask);
}
@@ -1346,7 +1344,7 @@ static void ept_disable_pml(struct p2m_domain *p2m)
vmx_domain_update_eptp(p2m->domain);
}
-static void ept_enable_hardware_log_dirty(struct p2m_domain *p2m)
+static void cf_check ept_enable_hardware_log_dirty(struct p2m_domain *p2m)
{
struct p2m_domain *hostp2m = p2m_get_hostp2m(p2m->domain);
@@ -1355,7 +1353,7 @@ static void ept_enable_hardware_log_dirty(struct p2m_domain *p2m)
p2m_unlock(hostp2m);
}
-static void ept_disable_hardware_log_dirty(struct p2m_domain *p2m)
+static void cf_check ept_disable_hardware_log_dirty(struct p2m_domain *p2m)
{
struct p2m_domain *hostp2m = p2m_get_hostp2m(p2m->domain);
@@ -1364,7 +1362,7 @@ static void ept_disable_hardware_log_dirty(struct p2m_domain *p2m)
p2m_unlock(hostp2m);
}
-static void ept_flush_pml_buffers(struct p2m_domain *p2m)
+static void cf_check ept_flush_pml_buffers(struct p2m_domain *p2m)
{
/* Domain must have been paused */
ASSERT(atomic_read(&p2m->domain->pause_count));
diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index ef3f8e02a415..eaba2b0fb4e6 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -399,7 +399,7 @@ static int p2m_pt_set_recalc_range(struct p2m_domain *p2m,
* GFN. Propagate the re-calculation flag down to the next page table level
* for entries not involved in the translation of the given GFN.
*/
-static int do_recalc(struct p2m_domain *p2m, unsigned long gfn)
+static int cf_check do_recalc(struct p2m_domain *p2m, unsigned long gfn)
{
void *table;
unsigned long gfn_remainder = gfn;
@@ -573,7 +573,7 @@ static void check_entry(mfn_t mfn, p2m_type_t new, p2m_type_t old,
}
/* Returns: 0 for success, -errno for failure */
-static int
+static int cf_check
p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
unsigned int page_order, p2m_type_t p2mt, p2m_access_t p2ma,
int sve)
@@ -774,7 +774,7 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
return rc;
}
-static mfn_t
+static mfn_t cf_check
p2m_pt_get_entry(struct p2m_domain *p2m, gfn_t gfn_,
p2m_type_t *t, p2m_access_t *a, p2m_query_t q,
unsigned int *page_order, bool_t *sve)
@@ -943,8 +943,8 @@ p2m_pt_get_entry(struct p2m_domain *p2m, gfn_t gfn_,
return (p2m_is_valid(*t) || p2m_is_any_ram(*t)) ? mfn : INVALID_MFN;
}
-static void p2m_pt_change_entry_type_global(struct p2m_domain *p2m,
- p2m_type_t ot, p2m_type_t nt)
+static void cf_check p2m_pt_change_entry_type_global(
+ struct p2m_domain *p2m, p2m_type_t ot, p2m_type_t nt)
{
l1_pgentry_t *tab;
unsigned long gfn = 0;
@@ -983,10 +983,9 @@ static void p2m_pt_change_entry_type_global(struct p2m_domain *p2m,
guest_flush_tlb_mask(d, d->dirty_cpumask);
}
-static int p2m_pt_change_entry_type_range(struct p2m_domain *p2m,
- p2m_type_t ot, p2m_type_t nt,
- unsigned long first_gfn,
- unsigned long last_gfn)
+static int cf_check p2m_pt_change_entry_type_range(
+ struct p2m_domain *p2m, p2m_type_t ot, p2m_type_t nt,
+ unsigned long first_gfn, unsigned long last_gfn)
{
unsigned long mask = (1 << PAGETABLE_ORDER) - 1;
unsigned int i;
@@ -1025,7 +1024,7 @@ static int p2m_pt_change_entry_type_range(struct p2m_domain *p2m,
}
#if P2M_AUDIT
-static long p2m_pt_audit_p2m(struct p2m_domain *p2m)
+static long cf_check p2m_pt_audit_p2m(struct p2m_domain *p2m)
{
unsigned long entry_count = 0, pmbad = 0;
unsigned long mfn, gfn, m2pfn;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 51/70] x86/irq: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (49 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 50/70] x86/p2m: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 52/70] x86/apei: " Andrew Cooper
` (20 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/hpet.c | 11 ++++++-----
xen/arch/x86/i8259.c | 10 +++++-----
xen/arch/x86/include/asm/irq.h | 8 ++++----
xen/arch/x86/include/asm/msi.h | 8 ++++----
xen/arch/x86/io_apic.c | 24 ++++++++++++------------
xen/arch/x86/irq.c | 6 +++---
xen/arch/x86/msi.c | 14 +++++++-------
xen/common/irq.c | 6 +++---
xen/drivers/passthrough/amd/iommu_init.c | 15 ++++++++-------
xen/drivers/passthrough/vtd/iommu.c | 13 +++++++------
xen/include/xen/irq.h | 6 +++---
11 files changed, 62 insertions(+), 59 deletions(-)
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 20fca839907c..dcc9e16693e9 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -256,7 +256,7 @@ static void cf_check hpet_interrupt_handler(
ch->event_handler(ch);
}
-static void hpet_msi_unmask(struct irq_desc *desc)
+static void cf_check hpet_msi_unmask(struct irq_desc *desc)
{
u32 cfg;
struct hpet_event_channel *ch = desc->action->dev_id;
@@ -267,7 +267,7 @@ static void hpet_msi_unmask(struct irq_desc *desc)
ch->msi.msi_attrib.host_masked = 0;
}
-static void hpet_msi_mask(struct irq_desc *desc)
+static void cf_check hpet_msi_mask(struct irq_desc *desc)
{
u32 cfg;
struct hpet_event_channel *ch = desc->action->dev_id;
@@ -296,7 +296,7 @@ static int hpet_msi_write(struct hpet_event_channel *ch, struct msi_msg *msg)
return 0;
}
-static unsigned int hpet_msi_startup(struct irq_desc *desc)
+static unsigned int cf_check hpet_msi_startup(struct irq_desc *desc)
{
hpet_msi_unmask(desc);
return 0;
@@ -304,14 +304,15 @@ static unsigned int hpet_msi_startup(struct irq_desc *desc)
#define hpet_msi_shutdown hpet_msi_mask
-static void hpet_msi_ack(struct irq_desc *desc)
+static void cf_check hpet_msi_ack(struct irq_desc *desc)
{
irq_complete_move(desc);
move_native_irq(desc);
ack_APIC_irq();
}
-static void hpet_msi_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+static void cf_check hpet_msi_set_affinity(
+ struct irq_desc *desc, const cpumask_t *mask)
{
struct hpet_event_channel *ch = desc->action->dev_id;
struct msi_msg msg = ch->msi.msg;
diff --git a/xen/arch/x86/i8259.c b/xen/arch/x86/i8259.c
index b389bb217622..6b35be10f09a 100644
--- a/xen/arch/x86/i8259.c
+++ b/xen/arch/x86/i8259.c
@@ -40,18 +40,18 @@ bool bogus_8259A_irq(unsigned int irq)
return _mask_and_ack_8259A_irq(irq);
}
-static void mask_and_ack_8259A_irq(struct irq_desc *desc)
+static void cf_check mask_and_ack_8259A_irq(struct irq_desc *desc)
{
_mask_and_ack_8259A_irq(desc->irq);
}
-static unsigned int startup_8259A_irq(struct irq_desc *desc)
+static unsigned int cf_check startup_8259A_irq(struct irq_desc *desc)
{
enable_8259A_irq(desc);
return 0; /* never anything pending */
}
-static void end_8259A_irq(struct irq_desc *desc, u8 vector)
+static void cf_check end_8259A_irq(struct irq_desc *desc, u8 vector)
{
if (!(desc->status & (IRQ_DISABLED|IRQ_INPROGRESS)))
enable_8259A_irq(desc);
@@ -108,12 +108,12 @@ static void _disable_8259A_irq(unsigned int irq)
spin_unlock_irqrestore(&i8259A_lock, flags);
}
-void disable_8259A_irq(struct irq_desc *desc)
+void cf_check disable_8259A_irq(struct irq_desc *desc)
{
_disable_8259A_irq(desc->irq);
}
-void enable_8259A_irq(struct irq_desc *desc)
+void cf_check enable_8259A_irq(struct irq_desc *desc)
{
unsigned int mask = ~(1 << desc->irq);
unsigned long flags;
diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index b3f49abc5556..76e6ed6d60aa 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -111,8 +111,8 @@ void alloc_direct_apic_vector(
void do_IRQ(struct cpu_user_regs *regs);
-void disable_8259A_irq(struct irq_desc *);
-void enable_8259A_irq(struct irq_desc *);
+void cf_check disable_8259A_irq(struct irq_desc *);
+void cf_check enable_8259A_irq(struct irq_desc *);
int i8259A_irq_pending(unsigned int irq);
void mask_8259A(void);
void unmask_8259A(void);
@@ -173,7 +173,7 @@ int create_irq(nodeid_t node, bool grant_access);
void destroy_irq(unsigned int irq);
int assign_irq_vector(int irq, const cpumask_t *);
-extern void irq_complete_move(struct irq_desc *);
+void cf_check irq_complete_move(struct irq_desc *);
extern struct irq_desc *irq_desc;
@@ -187,7 +187,7 @@ void move_masked_irq(struct irq_desc *);
int bind_irq_vector(int irq, int vector, const cpumask_t *);
-void end_nonmaskable_irq(struct irq_desc *, uint8_t vector);
+void cf_check end_nonmaskable_irq(struct irq_desc *, uint8_t vector);
void irq_set_affinity(struct irq_desc *, const cpumask_t *mask);
int init_domain_irq_mapping(struct domain *);
diff --git a/xen/arch/x86/include/asm/msi.h b/xen/arch/x86/include/asm/msi.h
index e228b0f3f344..117379318f2c 100644
--- a/xen/arch/x86/include/asm/msi.h
+++ b/xen/arch/x86/include/asm/msi.h
@@ -247,10 +247,10 @@ void early_msi_init(void);
void msi_compose_msg(unsigned vector, const cpumask_t *mask,
struct msi_msg *msg);
void __msi_set_enable(u16 seg, u8 bus, u8 slot, u8 func, int pos, int enable);
-void mask_msi_irq(struct irq_desc *);
-void unmask_msi_irq(struct irq_desc *);
+void cf_check mask_msi_irq(struct irq_desc *);
+void cf_check unmask_msi_irq(struct irq_desc *);
void guest_mask_msi_irq(struct irq_desc *, bool mask);
-void ack_nonmaskable_msi_irq(struct irq_desc *);
-void set_msi_affinity(struct irq_desc *, const cpumask_t *);
+void cf_check ack_nonmaskable_msi_irq(struct irq_desc *);
+void cf_check set_msi_affinity(struct irq_desc *, const cpumask_t *);
#endif /* __ASM_MSI_H */
diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index 4c5eaef86273..c086f40f6314 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -473,7 +473,7 @@ static void __level_IO_APIC_irq (unsigned int irq)
modify_IO_APIC_irq(irq, IO_APIC_REDIR_LEVEL_TRIGGER, 0);
}
-static void mask_IO_APIC_irq(struct irq_desc *desc)
+static void cf_check mask_IO_APIC_irq(struct irq_desc *desc)
{
unsigned long flags;
@@ -482,7 +482,7 @@ static void mask_IO_APIC_irq(struct irq_desc *desc)
spin_unlock_irqrestore(&ioapic_lock, flags);
}
-static void unmask_IO_APIC_irq(struct irq_desc *desc)
+static void cf_check unmask_IO_APIC_irq(struct irq_desc *desc)
{
unsigned long flags;
@@ -567,7 +567,7 @@ static void clear_IO_APIC (void)
}
}
-static void
+static void cf_check
set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask)
{
unsigned int dest;
@@ -1547,7 +1547,7 @@ static int __init timer_irq_works(void)
* This is not complete - we should be able to fake
* an edge even if it isn't on the 8259A...
*/
-static unsigned int startup_edge_ioapic_irq(struct irq_desc *desc)
+static unsigned int cf_check startup_edge_ioapic_irq(struct irq_desc *desc)
{
int was_pending = 0;
unsigned long flags;
@@ -1569,7 +1569,7 @@ static unsigned int startup_edge_ioapic_irq(struct irq_desc *desc)
* interrupt for real. This prevents IRQ storms from unhandled
* devices.
*/
-static void ack_edge_ioapic_irq(struct irq_desc *desc)
+static void cf_check ack_edge_ioapic_irq(struct irq_desc *desc)
{
irq_complete_move(desc);
move_native_irq(desc);
@@ -1594,7 +1594,7 @@ static void ack_edge_ioapic_irq(struct irq_desc *desc)
* generic IRQ layer and by the fact that an unacked local
* APIC does not accept IRQs.
*/
-static unsigned int startup_level_ioapic_irq(struct irq_desc *desc)
+static unsigned int cf_check startup_level_ioapic_irq(struct irq_desc *desc)
{
unmask_IO_APIC_irq(desc);
@@ -1652,7 +1652,7 @@ static bool io_apic_level_ack_pending(unsigned int irq)
return 0;
}
-static void mask_and_ack_level_ioapic_irq(struct irq_desc *desc)
+static void cf_check mask_and_ack_level_ioapic_irq(struct irq_desc *desc)
{
unsigned long v;
int i;
@@ -1702,7 +1702,7 @@ static void mask_and_ack_level_ioapic_irq(struct irq_desc *desc)
}
}
-static void end_level_ioapic_irq_old(struct irq_desc *desc, u8 vector)
+static void cf_check end_level_ioapic_irq_old(struct irq_desc *desc, u8 vector)
{
if ( directed_eoi_enabled )
{
@@ -1723,7 +1723,7 @@ static void end_level_ioapic_irq_old(struct irq_desc *desc, u8 vector)
unmask_IO_APIC_irq(desc);
}
-static void end_level_ioapic_irq_new(struct irq_desc *desc, u8 vector)
+static void cf_check end_level_ioapic_irq_new(struct irq_desc *desc, u8 vector)
{
/*
* It appears there is an erratum which affects at least version 0x11
@@ -1807,7 +1807,7 @@ static inline void init_IO_APIC_traps(void)
make_8259A_irq(irq);
}
-static void enable_lapic_irq(struct irq_desc *desc)
+static void cf_check enable_lapic_irq(struct irq_desc *desc)
{
unsigned long v;
@@ -1815,7 +1815,7 @@ static void enable_lapic_irq(struct irq_desc *desc)
apic_write(APIC_LVT0, v & ~APIC_LVT_MASKED);
}
-static void disable_lapic_irq(struct irq_desc *desc)
+static void cf_check disable_lapic_irq(struct irq_desc *desc)
{
unsigned long v;
@@ -1823,7 +1823,7 @@ static void disable_lapic_irq(struct irq_desc *desc)
apic_write(APIC_LVT0, v | APIC_LVT_MASKED);
}
-static void ack_lapic_irq(struct irq_desc *desc)
+static void cf_check ack_lapic_irq(struct irq_desc *desc)
{
ack_APIC_irq();
}
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 61e09a356f97..285ac399fbfd 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -468,7 +468,7 @@ int __init init_irq_data(void)
return 0;
}
-static void ack_none(struct irq_desc *desc)
+static void cf_check ack_none(struct irq_desc *desc)
{
ack_bad_irq(desc->irq);
}
@@ -832,7 +832,7 @@ static void send_cleanup_vector(struct irq_desc *desc)
desc->arch.move_in_progress = 0;
}
-void irq_complete_move(struct irq_desc *desc)
+void cf_check irq_complete_move(struct irq_desc *desc)
{
unsigned vector, me;
@@ -1086,7 +1086,7 @@ bool cpu_has_pending_apic_eoi(void)
return pending_eoi_sp(this_cpu(pending_eoi)) != 0;
}
-void end_nonmaskable_irq(struct irq_desc *desc, uint8_t vector)
+void cf_check end_nonmaskable_irq(struct irq_desc *desc, uint8_t vector)
{
struct pending_eoi *peoi = this_cpu(pending_eoi);
unsigned int sp = pending_eoi_sp(peoi);
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 77a4fbf13f9f..b32b1378f84e 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -241,7 +241,7 @@ static int write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
return 0;
}
-void set_msi_affinity(struct irq_desc *desc, const cpumask_t *mask)
+void cf_check set_msi_affinity(struct irq_desc *desc, const cpumask_t *mask)
{
struct msi_msg msg;
unsigned int dest;
@@ -416,14 +416,14 @@ static int msi_get_mask_bit(const struct msi_desc *entry)
return -1;
}
-void mask_msi_irq(struct irq_desc *desc)
+void cf_check mask_msi_irq(struct irq_desc *desc)
{
if ( unlikely(!msi_set_mask_bit(desc, 1,
desc->msi_desc->msi_attrib.guest_masked)) )
BUG_ON(!(desc->status & IRQ_DISABLED));
}
-void unmask_msi_irq(struct irq_desc *desc)
+void cf_check unmask_msi_irq(struct irq_desc *desc)
{
if ( unlikely(!msi_set_mask_bit(desc, 0,
desc->msi_desc->msi_attrib.guest_masked)) )
@@ -435,26 +435,26 @@ void guest_mask_msi_irq(struct irq_desc *desc, bool mask)
msi_set_mask_bit(desc, desc->msi_desc->msi_attrib.host_masked, mask);
}
-static unsigned int startup_msi_irq(struct irq_desc *desc)
+static unsigned int cf_check startup_msi_irq(struct irq_desc *desc)
{
if ( unlikely(!msi_set_mask_bit(desc, 0, !!(desc->status & IRQ_GUEST))) )
WARN();
return 0;
}
-static void shutdown_msi_irq(struct irq_desc *desc)
+static void cf_check shutdown_msi_irq(struct irq_desc *desc)
{
if ( unlikely(!msi_set_mask_bit(desc, 1, 1)) )
BUG_ON(!(desc->status & IRQ_DISABLED));
}
-void ack_nonmaskable_msi_irq(struct irq_desc *desc)
+void cf_check ack_nonmaskable_msi_irq(struct irq_desc *desc)
{
irq_complete_move(desc);
move_native_irq(desc);
}
-static void ack_maskable_msi_irq(struct irq_desc *desc)
+static void cf_check ack_maskable_msi_irq(struct irq_desc *desc)
{
ack_nonmaskable_msi_irq(desc);
ack_APIC_irq(); /* ACKTYPE_NONE */
diff --git a/xen/common/irq.c b/xen/common/irq.c
index f42512db33b7..727cf8bd22da 100644
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -27,15 +27,15 @@ int init_one_irq_desc(struct irq_desc *desc)
return err;
}
-void no_action(int cpl, void *dev_id, struct cpu_user_regs *regs)
+void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs)
{
}
-void irq_actor_none(struct irq_desc *desc)
+void cf_check irq_actor_none(struct irq_desc *desc)
{
}
-unsigned int irq_startup_none(struct irq_desc *desc)
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
{
return 0;
}
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index d2ad282e93d3..657c7f619a51 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -410,7 +410,7 @@ static void amd_iommu_msi_enable(struct amd_iommu *iommu, int flag)
PCI_FUNC(iommu->bdf), iommu->msi.msi_attrib.pos, flag);
}
-static void iommu_msi_unmask(struct irq_desc *desc)
+static void cf_check iommu_msi_unmask(struct irq_desc *desc)
{
unsigned long flags;
struct amd_iommu *iommu = desc->action->dev_id;
@@ -421,7 +421,7 @@ static void iommu_msi_unmask(struct irq_desc *desc)
iommu->msi.msi_attrib.host_masked = 0;
}
-static void iommu_msi_mask(struct irq_desc *desc)
+static void cf_check iommu_msi_mask(struct irq_desc *desc)
{
unsigned long flags;
struct amd_iommu *iommu = desc->action->dev_id;
@@ -434,13 +434,13 @@ static void iommu_msi_mask(struct irq_desc *desc)
iommu->msi.msi_attrib.host_masked = 1;
}
-static unsigned int iommu_msi_startup(struct irq_desc *desc)
+static unsigned int cf_check iommu_msi_startup(struct irq_desc *desc)
{
iommu_msi_unmask(desc);
return 0;
}
-static void iommu_msi_end(struct irq_desc *desc, u8 vector)
+static void cf_check iommu_msi_end(struct irq_desc *desc, u8 vector)
{
iommu_msi_unmask(desc);
end_nonmaskable_irq(desc, vector);
@@ -458,14 +458,14 @@ static hw_irq_controller iommu_msi_type = {
.set_affinity = set_msi_affinity,
};
-static unsigned int iommu_maskable_msi_startup(struct irq_desc *desc)
+static unsigned int cf_check iommu_maskable_msi_startup(struct irq_desc *desc)
{
iommu_msi_unmask(desc);
unmask_msi_irq(desc);
return 0;
}
-static void iommu_maskable_msi_shutdown(struct irq_desc *desc)
+static void cf_check iommu_maskable_msi_shutdown(struct irq_desc *desc)
{
mask_msi_irq(desc);
iommu_msi_mask(desc);
@@ -489,7 +489,8 @@ static hw_irq_controller iommu_maskable_msi_type = {
.set_affinity = set_msi_affinity,
};
-static void set_x2apic_affinity(struct irq_desc *desc, const cpumask_t *mask)
+static void cf_check set_x2apic_affinity(
+ struct irq_desc *desc, const cpumask_t *mask)
{
struct amd_iommu *iommu = desc->action->dev_id;
unsigned int dest = set_desc_affinity(desc, mask);
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index fc3ff064b692..c05670eb68a8 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1128,7 +1128,7 @@ static void cf_check iommu_page_fault(
tasklet_schedule(&vtd_fault_tasklet);
}
-static void dma_msi_unmask(struct irq_desc *desc)
+static void cf_check dma_msi_unmask(struct irq_desc *desc)
{
struct vtd_iommu *iommu = desc->action->dev_id;
unsigned long flags;
@@ -1143,7 +1143,7 @@ static void dma_msi_unmask(struct irq_desc *desc)
iommu->msi.msi_attrib.host_masked = 0;
}
-static void dma_msi_mask(struct irq_desc *desc)
+static void cf_check dma_msi_mask(struct irq_desc *desc)
{
unsigned long flags;
struct vtd_iommu *iommu = desc->action->dev_id;
@@ -1158,26 +1158,27 @@ static void dma_msi_mask(struct irq_desc *desc)
iommu->msi.msi_attrib.host_masked = 1;
}
-static unsigned int dma_msi_startup(struct irq_desc *desc)
+static unsigned int cf_check dma_msi_startup(struct irq_desc *desc)
{
dma_msi_unmask(desc);
return 0;
}
-static void dma_msi_ack(struct irq_desc *desc)
+static void cf_check dma_msi_ack(struct irq_desc *desc)
{
irq_complete_move(desc);
dma_msi_mask(desc);
move_masked_irq(desc);
}
-static void dma_msi_end(struct irq_desc *desc, u8 vector)
+static void cf_check dma_msi_end(struct irq_desc *desc, u8 vector)
{
dma_msi_unmask(desc);
end_nonmaskable_irq(desc, vector);
}
-static void dma_msi_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+static void cf_check dma_msi_set_affinity(
+ struct irq_desc *desc, const cpumask_t *mask)
{
struct msi_msg msg;
unsigned int dest;
diff --git a/xen/include/xen/irq.h b/xen/include/xen/irq.h
index 43d567fe445c..d8beadd16b9f 100644
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -116,9 +116,9 @@ extern int request_irq(unsigned int irq, unsigned int irqflags,
const char * devname, void *dev_id);
extern hw_irq_controller no_irq_type;
-extern void no_action(int cpl, void *dev_id, struct cpu_user_regs *regs);
-extern unsigned int irq_startup_none(struct irq_desc *);
-extern void irq_actor_none(struct irq_desc *);
+void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs);
+unsigned int cf_check irq_startup_none(struct irq_desc *);
+void cf_check irq_actor_none(struct irq_desc *);
#define irq_shutdown_none irq_actor_none
#define irq_disable_none irq_actor_none
#define irq_enable_none irq_actor_none
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 52/70] x86/apei: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (50 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 51/70] x86/irq: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 53/70] x86/psr: " Andrew Cooper
` (19 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/drivers/acpi/apei/apei-base.c | 32 ++++++++++----------
xen/drivers/acpi/apei/apei-internal.h | 20 ++++++------
xen/drivers/acpi/apei/erst.c | 57 +++++++++++++++++------------------
3 files changed, 54 insertions(+), 55 deletions(-)
diff --git a/xen/drivers/acpi/apei/apei-base.c b/xen/drivers/acpi/apei/apei-base.c
index 6f81e7fa366a..de75c1cef992 100644
--- a/xen/drivers/acpi/apei/apei-base.c
+++ b/xen/drivers/acpi/apei/apei-base.c
@@ -80,8 +80,8 @@ int __apei_exec_read_register(struct acpi_whea_header *entry, u64 *val)
return 0;
}
-int apei_exec_read_register(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+int cf_check apei_exec_read_register(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 val = 0;
@@ -94,8 +94,8 @@ int apei_exec_read_register(struct apei_exec_context *ctx,
return 0;
}
-int apei_exec_read_register_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+int cf_check apei_exec_read_register_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
@@ -126,14 +126,14 @@ int __apei_exec_write_register(struct acpi_whea_header *entry, u64 val)
return rc;
}
-int apei_exec_write_register(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+int cf_check apei_exec_write_register(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_write_register(entry, ctx->value);
}
-int apei_exec_write_register_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+int cf_check apei_exec_write_register_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
@@ -143,8 +143,8 @@ int apei_exec_write_register_value(struct apei_exec_context *ctx,
return rc;
}
-int apei_exec_noop(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+int cf_check apei_exec_noop(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return 0;
}
@@ -230,9 +230,9 @@ static int __init apei_exec_for_each_entry(struct apei_exec_context *ctx,
return 0;
}
-static int __init pre_map_gar_callback(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry,
- void *data)
+static int __init cf_check pre_map_gar_callback(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry,
+ void *data)
{
u8 ins = entry->instruction;
@@ -259,9 +259,9 @@ int __init apei_exec_pre_map_gars(struct apei_exec_context *ctx)
return rc;
}
-static int __init post_unmap_gar_callback(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry,
- void *data)
+static int __init cf_check post_unmap_gar_callback(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry,
+ void *data)
{
u8 ins = entry->instruction;
diff --git a/xen/drivers/acpi/apei/apei-internal.h b/xen/drivers/acpi/apei/apei-internal.h
index b813d55b92a9..360e94b9c877 100644
--- a/xen/drivers/acpi/apei/apei-internal.h
+++ b/xen/drivers/acpi/apei/apei-internal.h
@@ -68,16 +68,16 @@ static inline int apei_exec_run_optional(struct apei_exec_context *ctx, u8 actio
int __apei_exec_read_register(struct acpi_whea_header *entry, u64 *val);
int __apei_exec_write_register(struct acpi_whea_header *entry, u64 val);
-int apei_exec_read_register(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry);
-int apei_exec_read_register_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry);
-int apei_exec_write_register(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry);
-int apei_exec_write_register_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry);
-int apei_exec_noop(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry);
+int cf_check apei_exec_read_register(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry);
+int cf_check apei_exec_read_register_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry);
+int cf_check apei_exec_write_register(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry);
+int cf_check apei_exec_write_register_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry);
+int cf_check apei_exec_noop(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry);
int apei_exec_pre_map_gars(struct apei_exec_context *ctx);
int apei_exec_post_unmap_gars(struct apei_exec_context *ctx);
diff --git a/xen/drivers/acpi/apei/erst.c b/xen/drivers/acpi/apei/erst.c
index c5df512b9838..40d8f00270d0 100644
--- a/xen/drivers/acpi/apei/erst.c
+++ b/xen/drivers/acpi/apei/erst.c
@@ -114,40 +114,40 @@ static int erst_timedout(u64 *t, u64 spin_unit)
return 0;
}
-static int erst_exec_load_var1(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_load_var1(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_read_register(entry, &ctx->var1);
}
-static int erst_exec_load_var2(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_load_var2(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_read_register(entry, &ctx->var2);
}
-static int erst_exec_store_var1(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_store_var1(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_write_register(entry, ctx->var1);
}
-static int erst_exec_add(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_add(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
ctx->var1 += ctx->var2;
return 0;
}
-static int erst_exec_subtract(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_subtract(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
ctx->var1 -= ctx->var2;
return 0;
}
-static int erst_exec_add_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_add_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 val;
@@ -160,8 +160,8 @@ static int erst_exec_add_value(struct apei_exec_context *ctx,
return rc;
}
-static int erst_exec_subtract_value(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_subtract_value(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 val;
@@ -174,8 +174,8 @@ static int erst_exec_subtract_value(struct apei_exec_context *ctx,
return rc;
}
-static int erst_exec_stall(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_stall(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
udelay((ctx->var1 > FIRMWARE_MAX_STALL) ?
FIRMWARE_MAX_STALL :
@@ -183,8 +183,8 @@ static int erst_exec_stall(struct apei_exec_context *ctx,
return 0;
}
-static int erst_exec_stall_while_true(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_stall_while_true(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 val;
@@ -205,9 +205,8 @@ static int erst_exec_stall_while_true(struct apei_exec_context *ctx,
return 0;
}
-static int erst_exec_skip_next_instruction_if_true(
- struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_skip_next_instruction_if_true(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 val;
@@ -223,27 +222,27 @@ static int erst_exec_skip_next_instruction_if_true(
return 0;
}
-static int erst_exec_goto(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_goto(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
ctx->ip = ctx->value;
return APEI_EXEC_SET_IP;
}
-static int erst_exec_set_src_address_base(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_set_src_address_base(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_read_register(entry, &ctx->src_base);
}
-static int erst_exec_set_dst_address_base(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_set_dst_address_base(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
return __apei_exec_read_register(entry, &ctx->dst_base);
}
-static int erst_exec_move_data(struct apei_exec_context *ctx,
- struct acpi_whea_header *entry)
+static int cf_check erst_exec_move_data(
+ struct apei_exec_context *ctx, struct acpi_whea_header *entry)
{
int rc;
u64 offset;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 53/70] x86/psr: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (51 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 52/70] x86/apei: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 54/70] x86/dpci: " Andrew Cooper
` (18 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/psr.c | 33 +++++++++++++++++----------------
1 file changed, 17 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 6c9cabf3843d..ccb761998f9a 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -282,7 +282,7 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
}
/* Implementation of allocation features' functions. */
-static bool cat_check_cbm(const struct feat_node *feat, uint32_t *val)
+static bool cf_check cat_check_cbm(const struct feat_node *feat, uint32_t *val)
{
unsigned int first_bit, zero_bit;
unsigned int cbm_len = feat->cat.cbm_len;
@@ -417,8 +417,8 @@ static bool mba_init_feature(const struct cpuid_leaf *regs,
return true;
}
-static bool cat_get_feat_info(const struct feat_node *feat,
- uint32_t data[], unsigned int array_len)
+static bool cf_check cat_get_feat_info(
+ const struct feat_node *feat, uint32_t data[], unsigned int array_len)
{
if ( array_len != PSR_INFO_ARRAY_SIZE )
return false;
@@ -431,8 +431,8 @@ static bool cat_get_feat_info(const struct feat_node *feat,
}
/* L3 CAT props */
-static void l3_cat_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static void cf_check l3_cat_write_msr(
+ unsigned int cos, uint32_t val, enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val);
}
@@ -447,8 +447,8 @@ static const struct feat_props l3_cat_props = {
};
/* L3 CDP props */
-static bool l3_cdp_get_feat_info(const struct feat_node *feat,
- uint32_t data[], uint32_t array_len)
+static bool cf_check l3_cdp_get_feat_info(
+ const struct feat_node *feat, uint32_t data[], uint32_t array_len)
{
if ( !cat_get_feat_info(feat, data, array_len) )
return false;
@@ -458,8 +458,8 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
-static void l3_cdp_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static void cf_check l3_cdp_write_msr(
+ unsigned int cos, uint32_t val, enum psr_type type)
{
wrmsrl(((type == PSR_TYPE_L3_DATA) ?
MSR_IA32_PSR_L3_MASK_DATA(cos) :
@@ -478,8 +478,8 @@ static const struct feat_props l3_cdp_props = {
};
/* L2 CAT props */
-static void l2_cat_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static void cf_check l2_cat_write_msr(
+ unsigned int cos, uint32_t val, enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val);
}
@@ -494,8 +494,8 @@ static const struct feat_props l2_cat_props = {
};
/* MBA props */
-static bool mba_get_feat_info(const struct feat_node *feat,
- uint32_t data[], unsigned int array_len)
+static bool cf_check mba_get_feat_info(
+ const struct feat_node *feat, uint32_t data[], unsigned int array_len)
{
ASSERT(array_len == PSR_INFO_ARRAY_SIZE);
@@ -508,13 +508,14 @@ static bool mba_get_feat_info(const struct feat_node *feat,
return true;
}
-static void mba_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static void cf_check mba_write_msr(
+ unsigned int cos, uint32_t val, enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
}
-static bool mba_sanitize_thrtl(const struct feat_node *feat, uint32_t *thrtl)
+static bool cf_check mba_sanitize_thrtl(
+ const struct feat_node *feat, uint32_t *thrtl)
{
/*
* Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 54/70] x86/dpci: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (52 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 53/70] x86/psr: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 55/70] x86/pt: " Andrew Cooper
` (17 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/hvm/hvm.c | 4 ++--
xen/drivers/passthrough/vtd/x86/hvm.c | 4 ++--
xen/drivers/passthrough/x86/hvm.c | 8 ++++----
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4cf313a0ad0a..cdd1529014f2 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -474,8 +474,8 @@ void hvm_migrate_pirq(struct hvm_pirq_dpci *pirq_dpci, const struct vcpu *v)
}
}
-static int migrate_pirq(struct domain *d, struct hvm_pirq_dpci *pirq_dpci,
- void *arg)
+static int cf_check migrate_pirq(
+ struct domain *d, struct hvm_pirq_dpci *pirq_dpci, void *arg)
{
hvm_migrate_pirq(pirq_dpci, arg);
diff --git a/xen/drivers/passthrough/vtd/x86/hvm.c b/xen/drivers/passthrough/vtd/x86/hvm.c
index b531fe907a94..132d252d1cca 100644
--- a/xen/drivers/passthrough/vtd/x86/hvm.c
+++ b/xen/drivers/passthrough/vtd/x86/hvm.c
@@ -21,8 +21,8 @@
#include <xen/irq.h>
#include <xen/sched.h>
-static int _hvm_dpci_isairq_eoi(struct domain *d,
- struct hvm_pirq_dpci *pirq_dpci, void *arg)
+static int cf_check _hvm_dpci_isairq_eoi(
+ struct domain *d, struct hvm_pirq_dpci *pirq_dpci, void *arg)
{
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
unsigned int isairq = (long)arg;
diff --git a/xen/drivers/passthrough/x86/hvm.c b/xen/drivers/passthrough/x86/hvm.c
index 0e3c0f6aeed3..0f94203af817 100644
--- a/xen/drivers/passthrough/x86/hvm.c
+++ b/xen/drivers/passthrough/x86/hvm.c
@@ -777,8 +777,8 @@ static void __msi_pirq_eoi(struct hvm_pirq_dpci *pirq_dpci)
}
}
-static int _hvm_dpci_msi_eoi(struct domain *d,
- struct hvm_pirq_dpci *pirq_dpci, void *arg)
+static int cf_check _hvm_dpci_msi_eoi(
+ struct domain *d, struct hvm_pirq_dpci *pirq_dpci, void *arg)
{
int vector = (long)arg;
@@ -947,8 +947,8 @@ void hvm_dpci_eoi(struct domain *d, unsigned int guest_gsi)
spin_unlock(&d->event_lock);
}
-static int pci_clean_dpci_irq(struct domain *d,
- struct hvm_pirq_dpci *pirq_dpci, void *arg)
+static int cf_check pci_clean_dpci_irq(
+ struct domain *d, struct hvm_pirq_dpci *pirq_dpci, void *arg)
{
struct dev_intx_gsi_link *digl, *tmp;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 55/70] x86/pt: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (53 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 54/70] x86/dpci: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 56/70] x86/time: " Andrew Cooper
` (16 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/emul-i8254.c | 2 +-
xen/arch/x86/hvm/hpet.c | 2 +-
xen/arch/x86/hvm/rtc.c | 2 +-
xen/arch/x86/hvm/vlapic.c | 4 ++--
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/xen/arch/x86/emul-i8254.c b/xen/arch/x86/emul-i8254.c
index d170f464d966..18894b63488e 100644
--- a/xen/arch/x86/emul-i8254.c
+++ b/xen/arch/x86/emul-i8254.c
@@ -156,7 +156,7 @@ static int pit_get_gate(PITState *pit, int channel)
return pit->hw.channels[channel].gate;
}
-static void pit_time_fired(struct vcpu *v, void *priv)
+static void cf_check pit_time_fired(struct vcpu *v, void *priv)
{
uint64_t *count_load_time = priv;
TRACE_0D(TRC_HVM_EMUL_PIT_TIMER_CB);
diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index ed512fa65b63..45c7b9b40688 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -219,7 +219,7 @@ static void hpet_stop_timer(HPETState *h, unsigned int tn,
hpet_get_comparator(h, tn, guest_time);
}
-static void hpet_timer_fired(struct vcpu *v, void *data)
+static void cf_check hpet_timer_fired(struct vcpu *v, void *data)
{
unsigned int tn = (unsigned long)data;
HPETState *h = vcpu_vhpet(v);
diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index ed397276faa3..d21925db08bc 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -81,7 +81,7 @@ static void rtc_update_irq(RTCState *s)
/* Called by the VPT code after it's injected a PF interrupt for us.
* Fix up the register state to reflect what happened. */
-static void rtc_pf_callback(struct vcpu *v, void *opaque)
+static void cf_check rtc_pf_callback(struct vcpu *v, void *opaque)
{
RTCState *s = opaque;
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index d4e29ef1ff1d..49be9c8ea4fe 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -691,13 +691,13 @@ int guest_rdmsr_x2apic(const struct vcpu *v, uint32_t msr, uint64_t *val)
return X86EMUL_OKAY;
}
-static void vlapic_pt_cb(struct vcpu *v, void *data)
+static void cf_check vlapic_pt_cb(struct vcpu *v, void *data)
{
TRACE_0D(TRC_HVM_EMUL_LAPIC_TIMER_CB);
*(s_time_t *)data = hvm_get_guest_time(v);
}
-static void vlapic_tdt_pt_cb(struct vcpu *v, void *data)
+static void cf_check vlapic_tdt_pt_cb(struct vcpu *v, void *data)
{
*(s_time_t *)data = hvm_get_guest_time(v);
vcpu_vlapic(v)->hw.tdt_msr = 0;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 56/70] x86/time: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (54 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 55/70] x86/pt: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 57/70] x86/misc: " Andrew Cooper
` (15 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/hpet.c | 8 ++++----
xen/arch/x86/include/asm/hpet.h | 4 ++--
xen/arch/x86/time.c | 33 +++++++++++++++++----------------
3 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index dcc9e16693e9..2b00c30d1153 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -196,7 +196,7 @@ static void evt_do_broadcast(cpumask_t *mask)
cpumask_raise_softirq(mask, TIMER_SOFTIRQ);
}
-static void handle_hpet_broadcast(struct hpet_event_channel *ch)
+static void cf_check handle_hpet_broadcast(struct hpet_event_channel *ch)
{
cpumask_t mask;
s_time_t now, next_event;
@@ -553,7 +553,7 @@ static void hpet_detach_channel(unsigned int cpu,
void (*__read_mostly pv_rtc_handler)(uint8_t index, uint8_t value);
-static void handle_rtc_once(uint8_t index, uint8_t value)
+static void cf_check handle_rtc_once(uint8_t index, uint8_t value)
{
if ( index != RTC_REG_B )
return;
@@ -566,7 +566,7 @@ static void handle_rtc_once(uint8_t index, uint8_t value)
}
}
-void __init hpet_broadcast_init(void)
+void __init cf_check hpet_broadcast_init(void)
{
u64 hpet_rate = hpet_setup();
u32 hpet_id, cfg;
@@ -637,7 +637,7 @@ void __init hpet_broadcast_init(void)
hpet_events->flags = HPET_EVT_LEGACY;
}
-void hpet_broadcast_resume(void)
+void cf_check hpet_broadcast_resume(void)
{
u32 cfg;
unsigned int i, n;
diff --git a/xen/arch/x86/include/asm/hpet.h b/xen/arch/x86/include/asm/hpet.h
index f343fe4740f1..9919f7473071 100644
--- a/xen/arch/x86/include/asm/hpet.h
+++ b/xen/arch/x86/include/asm/hpet.h
@@ -89,8 +89,8 @@ void hpet_disable_legacy_replacement_mode(void);
* Temporarily use an HPET event counter for timer interrupt handling,
* rather than using the LAPIC timer. Used for Cx state entry.
*/
-void hpet_broadcast_init(void);
-void hpet_broadcast_resume(void);
+void cf_check hpet_broadcast_init(void);
+void cf_check hpet_broadcast_resume(void);
void cf_check hpet_broadcast_enter(void);
void cf_check hpet_broadcast_exit(void);
int hpet_broadcast_is_available(void);
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 043be2a8ec1a..2a1758ebb33b 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -309,7 +309,7 @@ static uint64_t adjust_elapsed(uint64_t elapsed, uint32_t actual,
* PLATFORM TIMER 1: PROGRAMMABLE INTERVAL TIMER (LEGACY PIT)
*/
-static u64 read_pit_count(void)
+static u64 cf_check read_pit_count(void)
{
u16 count16;
u32 count32;
@@ -328,7 +328,7 @@ static u64 read_pit_count(void)
return count32;
}
-static s64 __init init_pit(struct platform_timesource *pts)
+static s64 __init cf_check init_pit(struct platform_timesource *pts)
{
u8 portb = inb(0x61);
u64 start, end;
@@ -366,7 +366,7 @@ static s64 __init init_pit(struct platform_timesource *pts)
return (end - start) * CALIBRATE_FRAC;
}
-static void resume_pit(struct platform_timesource *pts)
+static void cf_check resume_pit(struct platform_timesource *pts)
{
/* Set CTC channel 2 to mode 0 again; initial value does not matter. */
outb(0xb0, PIT_MODE); /* binary, mode 0, LSB/MSB, Ch 2 */
@@ -389,12 +389,12 @@ static struct platform_timesource __initdata plt_pit =
* PLATFORM TIMER 2: HIGH PRECISION EVENT TIMER (HPET)
*/
-static u64 read_hpet_count(void)
+static u64 cf_check read_hpet_count(void)
{
return hpet_read32(HPET_COUNTER);
}
-static int64_t __init init_hpet(struct platform_timesource *pts)
+static int64_t __init cf_check init_hpet(struct platform_timesource *pts)
{
uint64_t hpet_rate, start;
uint32_t count, target, elapsed;
@@ -477,7 +477,7 @@ static int64_t __init init_hpet(struct platform_timesource *pts)
return adjust_elapsed(rdtsc_ordered() - start, elapsed, target);
}
-static void resume_hpet(struct platform_timesource *pts)
+static void cf_check resume_hpet(struct platform_timesource *pts)
{
hpet_resume(NULL);
}
@@ -502,12 +502,12 @@ unsigned int __initdata pmtmr_width;
/* ACPI PM timer ticks at 3.579545 MHz. */
#define ACPI_PM_FREQUENCY 3579545
-static u64 read_pmtimer_count(void)
+static u64 cf_check read_pmtimer_count(void)
{
return inl(pmtmr_ioport);
}
-static s64 __init init_pmtimer(struct platform_timesource *pts)
+static s64 __init cf_check init_pmtimer(struct platform_timesource *pts)
{
uint64_t start;
uint32_t count, target, mask, elapsed;
@@ -562,7 +562,7 @@ static unsigned int __initdata tsc_flags;
* Called in verify_tsc_reliability() under reliable TSC conditions
* thus reusing all the checks already performed there.
*/
-static s64 __init init_tsc(struct platform_timesource *pts)
+static s64 __init cf_check init_tsc(struct platform_timesource *pts)
{
u64 ret = pts->frequency;
@@ -584,7 +584,7 @@ static s64 __init init_tsc(struct platform_timesource *pts)
return ret;
}
-static u64 read_tsc(void)
+static u64 cf_check read_tsc(void)
{
return rdtsc_ordered();
}
@@ -626,7 +626,7 @@ static uint64_t xen_timer_cpu_frequency(void)
return freq;
}
-static int64_t __init init_xen_timer(struct platform_timesource *pts)
+static int64_t __init cf_check init_xen_timer(struct platform_timesource *pts)
{
if ( !xen_guest )
return 0;
@@ -647,7 +647,7 @@ static always_inline uint64_t read_cycle(const struct vcpu_time_info *info,
return info->system_time + offset;
}
-static uint64_t read_xen_timer(void)
+static uint64_t cf_check read_xen_timer(void)
{
struct vcpu_time_info *info = &this_cpu(vcpu_info)->time;
uint32_t version;
@@ -676,7 +676,7 @@ static uint64_t read_xen_timer(void)
return ret;
}
-static void resume_xen_timer(struct platform_timesource *pts)
+static void cf_check resume_xen_timer(struct platform_timesource *pts)
{
write_atomic(&xen_timer_last, 0);
}
@@ -702,7 +702,8 @@ static struct platform_timesource __initdata plt_xen_timer =
static struct ms_hyperv_tsc_page *hyperv_tsc;
static struct page_info *hyperv_tsc_page;
-static int64_t __init init_hyperv_timer(struct platform_timesource *pts)
+static int64_t __init cf_check init_hyperv_timer(
+ struct platform_timesource *pts)
{
paddr_t maddr;
uint64_t tsc_msr, freq;
@@ -745,7 +746,7 @@ static int64_t __init init_hyperv_timer(struct platform_timesource *pts)
return freq;
}
-static uint64_t read_hyperv_timer(void)
+static uint64_t cf_check read_hyperv_timer(void)
{
uint64_t scale, ret, tsc;
int64_t offset;
@@ -1721,7 +1722,7 @@ time_calibration_rendezvous_tail(const struct calibration_rendezvous *r,
* Keep TSCs in sync when they run at the same rate, but may stop in
* deep-sleep C states.
*/
-static void time_calibration_tsc_rendezvous(void *_r)
+static void cf_check time_calibration_tsc_rendezvous(void *_r)
{
int i;
struct calibration_rendezvous *r = _r;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 57/70] x86/misc: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (55 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 56/70] x86/time: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 58/70] x86/stack: " Andrew Cooper
` (14 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/extable.c | 4 ++--
xen/common/efi/boot.c | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/extable.c b/xen/arch/x86/extable.c
index 51ef863d786c..4d1875585f9d 100644
--- a/xen/arch/x86/extable.c
+++ b/xen/arch/x86/extable.c
@@ -23,7 +23,7 @@ static inline unsigned long ex_cont(const struct exception_table_entry *x)
return EX_FIELD(x, cont);
}
-static int init_or_livepatch cmp_ex(const void *a, const void *b)
+static int init_or_livepatch cf_check cmp_ex(const void *a, const void *b)
{
const struct exception_table_entry *l = a, *r = b;
unsigned long lip = ex_addr(l);
@@ -37,7 +37,7 @@ static int init_or_livepatch cmp_ex(const void *a, const void *b)
return 0;
}
-static void init_or_livepatch swap_ex(void *a, void *b, size_t size)
+static void init_or_livepatch cf_check swap_ex(void *a, void *b, size_t size)
{
struct exception_table_entry *l = a, *r = b, tmp;
long delta = b - a;
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index f31f68fd4cd1..4dd5ea6a0602 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1497,7 +1497,7 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end,
unmap_domain_page(l3dst);
}
-static bool __init ram_range_valid(unsigned long smfn, unsigned long emfn)
+static bool __init cf_check ram_range_valid(unsigned long smfn, unsigned long emfn)
{
unsigned long sz = pfn_to_pdx(emfn - 1) / PDX_GROUP_COUNT + 1;
@@ -1506,7 +1506,7 @@ static bool __init ram_range_valid(unsigned long smfn, unsigned long emfn)
pfn_to_pdx(smfn) / PDX_GROUP_COUNT) < sz;
}
-static bool __init rt_range_valid(unsigned long smfn, unsigned long emfn)
+static bool __init cf_check rt_range_valid(unsigned long smfn, unsigned long emfn)
{
return true;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 58/70] x86/stack: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (56 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 57/70] x86/misc: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 59/70] x86/bugframe: " Andrew Cooper
` (13 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The function typecheck in switch_stack_and_jump() is incompatible with control
flow typechecking. It's ok for reset_stack_and_jump_ind(), but for
reset_stack_and_jump(), it would force us to ENDBR64 the targets which are
branched to directly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v2:
* Extend reset_stack_and_jump_ind() with ({ })
---
xen/arch/x86/domain.c | 6 +++---
xen/arch/x86/hvm/svm/svm.c | 6 +++---
xen/arch/x86/hvm/vmx/vmcs.c | 2 +-
xen/arch/x86/hvm/vmx/vmx.c | 8 ++++----
xen/arch/x86/include/asm/current.h | 6 ++++--
xen/arch/x86/include/asm/hvm/vmx/vmx.h | 2 +-
xen/arch/x86/include/asm/pv/domain.h | 4 ++--
xen/arch/x86/pv/domain.c | 2 +-
xen/arch/x86/x86_64/entry.S | 1 +
9 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ae7c88b51af1..afccc1525f8b 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -132,7 +132,7 @@ void play_dead(void)
dead_idle();
}
-static void noreturn idle_loop(void)
+static void noreturn cf_check idle_loop(void)
{
unsigned int cpu = smp_processor_id();
/*
@@ -1790,7 +1790,7 @@ static void save_segments(struct vcpu *v)
}
}
-void paravirt_ctxt_switch_from(struct vcpu *v)
+void cf_check paravirt_ctxt_switch_from(struct vcpu *v)
{
save_segments(v);
@@ -1804,7 +1804,7 @@ void paravirt_ctxt_switch_from(struct vcpu *v)
write_debugreg(7, 0);
}
-void paravirt_ctxt_switch_to(struct vcpu *v)
+void cf_check paravirt_ctxt_switch_to(struct vcpu *v)
{
root_pgentry_t *root_pgt = this_cpu(root_pgt);
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index dedb2848e6a1..63535a74b504 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -944,7 +944,7 @@ static inline void svm_tsc_ratio_load(struct vcpu *v)
wrmsrl(MSR_AMD64_TSC_RATIO, hvm_tsc_scaling_ratio(v->domain));
}
-static void svm_ctxt_switch_from(struct vcpu *v)
+static void cf_check svm_ctxt_switch_from(struct vcpu *v)
{
int cpu = smp_processor_id();
@@ -969,7 +969,7 @@ static void svm_ctxt_switch_from(struct vcpu *v)
enable_each_ist(idt_tables[cpu]);
}
-static void svm_ctxt_switch_to(struct vcpu *v)
+static void cf_check svm_ctxt_switch_to(struct vcpu *v)
{
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
int cpu = smp_processor_id();
@@ -996,7 +996,7 @@ static void svm_ctxt_switch_to(struct vcpu *v)
wrmsr_tsc_aux(v->arch.msrs->tsc_aux);
}
-static void noreturn svm_do_resume(void)
+static void noreturn cf_check svm_do_resume(void)
{
struct vcpu *v = current;
struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 60b506ac3f40..e1e1fa14e65e 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -1865,7 +1865,7 @@ void vmx_vmentry_failure(void)
void noreturn vmx_asm_do_vmentry(void);
-void vmx_do_resume(void)
+void cf_check vmx_do_resume(void)
{
struct vcpu *v = current;
bool_t debug_state;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 2c4804f9b884..41db538a9e3d 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -63,8 +63,8 @@
static bool_t __initdata opt_force_ept;
boolean_param("force-ept", opt_force_ept);
-static void vmx_ctxt_switch_from(struct vcpu *v);
-static void vmx_ctxt_switch_to(struct vcpu *v);
+static void cf_check vmx_ctxt_switch_from(struct vcpu *v);
+static void cf_check vmx_ctxt_switch_to(struct vcpu *v);
static int alloc_vlapic_mapping(void);
static void vmx_install_vlapic_mapping(struct vcpu *v);
@@ -907,7 +907,7 @@ static void cf_check vmx_fpu_leave(struct vcpu *v)
}
}
-static void vmx_ctxt_switch_from(struct vcpu *v)
+static void cf_check vmx_ctxt_switch_from(struct vcpu *v)
{
/*
* Return early if trying to do a context switch without VMX enabled,
@@ -939,7 +939,7 @@ static void vmx_ctxt_switch_from(struct vcpu *v)
vmx_pi_switch_from(v);
}
-static void vmx_ctxt_switch_to(struct vcpu *v)
+static void cf_check vmx_ctxt_switch_to(struct vcpu *v)
{
vmx_restore_guest_msrs(v);
vmx_restore_dr(v);
diff --git a/xen/arch/x86/include/asm/current.h b/xen/arch/x86/include/asm/current.h
index dc0edd9ed07d..da5e152a10cc 100644
--- a/xen/arch/x86/include/asm/current.h
+++ b/xen/arch/x86/include/asm/current.h
@@ -173,7 +173,6 @@ unsigned long get_stack_dump_bottom (unsigned long sp);
#define switch_stack_and_jump(fn, instr, constr) \
({ \
unsigned int tmp; \
- (void)((fn) == (void (*)(void))NULL); \
BUILD_BUG_ON(!ssaj_has_attr_noreturn(fn)); \
__asm__ __volatile__ ( \
SHADOW_STACK_WORK \
@@ -198,7 +197,10 @@ unsigned long get_stack_dump_bottom (unsigned long sp);
/* The constraint may only specify non-call-clobbered registers. */
#define reset_stack_and_jump_ind(fn) \
- switch_stack_and_jump(fn, "INDIRECT_JMP %", "b")
+ ({ \
+ (void)((fn) == (void (*)(void))NULL); \
+ switch_stack_and_jump(fn, "INDIRECT_JMP %", "b"); \
+ })
/*
* Which VCPU's state is currently running on each CPU?
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmx.h b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
index 5284fe931f62..c2ebdd6864a5 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
@@ -93,7 +93,7 @@ typedef enum {
void vmx_asm_vmexit_handler(struct cpu_user_regs);
void vmx_intr_assist(void);
-void noreturn vmx_do_resume(void);
+void noreturn cf_check vmx_do_resume(void);
void vmx_vlapic_msr_changed(struct vcpu *v);
struct hvm_emulate_ctxt;
void vmx_realmode_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt);
diff --git a/xen/arch/x86/include/asm/pv/domain.h b/xen/arch/x86/include/asm/pv/domain.h
index 6b16da9d187b..924508bbb4f0 100644
--- a/xen/arch/x86/include/asm/pv/domain.h
+++ b/xen/arch/x86/include/asm/pv/domain.h
@@ -118,8 +118,8 @@ static inline void pv_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
#endif /* CONFIG_PV */
-void paravirt_ctxt_switch_from(struct vcpu *v);
-void paravirt_ctxt_switch_to(struct vcpu *v);
+void cf_check paravirt_ctxt_switch_from(struct vcpu *v);
+void cf_check paravirt_ctxt_switch_to(struct vcpu *v);
#endif /* __X86_PV_DOMAIN_H__ */
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 55146c15c853..f94f28c8e271 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -351,7 +351,7 @@ void pv_domain_destroy(struct domain *d)
FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab);
}
-void noreturn continue_pv_domain(void);
+void noreturn cf_check continue_pv_domain(void);
int pv_domain_initialise(struct domain *d)
{
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 3eaf0e67b2b9..8494b97a54a2 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -625,6 +625,7 @@ ENTRY(dom_crash_sync_extable)
/* No special register assumptions. */
#ifdef CONFIG_PV
ENTRY(continue_pv_domain)
+ ENDBR64
call check_wakeup_from_wait
ret_from_intr:
GET_CURRENT(bx)
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 59/70] x86/bugframe: CFI hardening
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (57 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 58/70] x86/stack: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 60/70] x86: Use control flow typechecking where possible Andrew Cooper
` (12 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
run_in_exception_handler() managed to escape typechecking, as the compiler
can't see where function pointer gets called. After adding some ad-hoc
typechecking, it turns out that dump_execution_state() alone differs in
const-ness from the other users of run_in_exception_handler().
Introduce a new show_execution_state_nonconst() to make the typechecking
happy.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/include/asm/bug.h | 10 +++++++++-
xen/arch/x86/include/asm/processor.h | 4 +++-
xen/arch/x86/traps.c | 5 +++++
xen/common/keyhandler.c | 4 ++--
xen/drivers/char/ehci-dbgp.c | 2 +-
xen/drivers/char/ns16550.c | 2 +-
xen/include/xen/lib.h | 2 +-
7 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/xen/arch/x86/include/asm/bug.h b/xen/arch/x86/include/asm/bug.h
index 9bb4a194202f..b7265bdfbe33 100644
--- a/xen/arch/x86/include/asm/bug.h
+++ b/xen/arch/x86/include/asm/bug.h
@@ -65,7 +65,15 @@ struct bug_frame {
unreachable(); \
} while (0)
-#define run_in_exception_handler(fn) BUG_FRAME(BUGFRAME_run_fn, 0, fn, 0, NULL)
+/*
+ * TODO: untangle header dependences, break BUILD_BUG_ON() out of xen/lib.h,
+ * and use a real static inline here to get proper type checking of fn().
+ */
+#define run_in_exception_handler(fn) \
+ do { \
+ (void)((fn) == (void (*)(struct cpu_user_regs *))NULL); \
+ BUG_FRAME(BUGFRAME_run_fn, 0, fn, 0, NULL); \
+ } while ( 0 )
#define assert_failed(msg) do { \
BUG_FRAME(BUGFRAME_assert, __LINE__, __FILE__, 1, msg); \
diff --git a/xen/arch/x86/include/asm/processor.h b/xen/arch/x86/include/asm/processor.h
index 23639d5479a3..8e2816fae9b9 100644
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -496,7 +496,9 @@ void show_code(const struct cpu_user_regs *regs);
void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs);
void show_registers(const struct cpu_user_regs *regs);
void show_execution_state(const struct cpu_user_regs *regs);
-#define dump_execution_state() run_in_exception_handler(show_execution_state)
+void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs);
+#define dump_execution_state() \
+ run_in_exception_handler(show_execution_state_nonconst)
void show_page_walk(unsigned long addr);
void noreturn fatal_trap(const struct cpu_user_regs *regs, bool_t show_remote);
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 7b957101934e..a2278d9499d0 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -681,6 +681,11 @@ void show_execution_state(const struct cpu_user_regs *regs)
console_unlock_recursive_irqrestore(flags);
}
+void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
+{
+ show_execution_state(regs);
+}
+
void vcpu_show_execution_state(struct vcpu *v)
{
unsigned long flags = 0;
diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 5dc650a37c5c..b6e22d8120b1 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -138,7 +138,7 @@ static void cf_check show_handlers(unsigned char key)
static cpumask_t dump_execstate_mask;
-void dump_execstate(struct cpu_user_regs *regs)
+void cf_check dump_execstate(struct cpu_user_regs *regs)
{
unsigned int cpu = smp_processor_id();
@@ -490,7 +490,7 @@ static void cf_check run_all_keyhandlers(
tasklet_schedule(&run_all_keyhandlers_tasklet);
}
-static void do_debugger_trap_fatal(struct cpu_user_regs *regs)
+static void cf_check do_debugger_trap_fatal(struct cpu_user_regs *regs)
{
(void)debugger_trap_fatal(0xf001, regs);
diff --git a/xen/drivers/char/ehci-dbgp.c b/xen/drivers/char/ehci-dbgp.c
index e205c0da6a61..16c8ff394d5c 100644
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1247,7 +1247,7 @@ static int cf_check ehci_dbgp_getc(struct serial_port *port, char *pc)
/* Safe: ehci_dbgp_poll() runs as timer handler, so not reentrant. */
static struct serial_port *poll_port;
-static void _ehci_dbgp_poll(struct cpu_user_regs *regs)
+static void cf_check _ehci_dbgp_poll(struct cpu_user_regs *regs)
{
struct serial_port *port = poll_port;
struct ehci_dbgp *dbgp = port->uart;
diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index 8df1ee4d5c2c..e5b4a9085516 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -206,7 +206,7 @@ static void cf_check ns16550_interrupt(
/* Safe: ns16550_poll() runs as softirq so not reentrant on a given CPU. */
static DEFINE_PER_CPU(struct serial_port *, poll_port);
-static void __ns16550_poll(struct cpu_user_regs *regs)
+static void cf_check __ns16550_poll(struct cpu_user_regs *regs)
{
struct serial_port *port = this_cpu(poll_port);
struct ns16550 *uart = port->uart;
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index c6987973bf88..3a1fdaf7e35a 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -199,7 +199,7 @@ extern char *print_tainted(char *str);
extern void add_taint(unsigned int taint);
struct cpu_user_regs;
-void dump_execstate(struct cpu_user_regs *);
+void cf_check dump_execstate(struct cpu_user_regs *);
void init_constructors(void);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 60/70] x86: Use control flow typechecking where possible
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (58 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 59/70] x86/bugframe: " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-15 16:26 ` Jan Beulich
2022-02-14 12:51 ` [PATCH v2 61/70] x86/setup: Read CR4 earlier in __start_xen() Andrew Cooper
` (11 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Now all callees have been annotated, turn on typechecking to catch issues in
the future.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
RFC. This is still an experimental compiler extention
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102953
However, it is also the entire basis of being able to sanely use
-mmanual-endbr in the first place, so is very important.
---
xen/arch/x86/arch.mk | 1 +
1 file changed, 1 insertion(+)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index 8b88f0240e85..1710d056af3f 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -49,6 +49,7 @@ CFLAGS-$(CONFIG_INDIRECT_THUNK) += -fno-jump-tables
ifdef CONFIG_XEN_IBT
CFLAGS += -fcf-protection=branch -mmanual-endbr
+$(call cc-option-add,CFLAGS,CC,-fcf-check-attribute=no)
else
$(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
endif
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 61/70] x86/setup: Read CR4 earlier in __start_xen()
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (59 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 60/70] x86: Use control flow typechecking where possible Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 62/70] x86/alternatives: Clear CR4.CET when clearing CR0.WP Andrew Cooper
` (10 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
This is necessary for read_cr4() to function correctly. Move the EFER caching
at the same time.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/setup.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 735f69d2cae8..2b1192d85b77 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -888,6 +888,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
/* Full exception support from here on in. */
+ rdmsrl(MSR_EFER, this_cpu(efer));
+ asm volatile ( "mov %%cr4,%0" : "=r" (get_cpu_info()->cr4) );
+
/* Enable NMIs. Our loader (e.g. Tboot) may have left them disabled. */
enable_nmis();
@@ -934,9 +937,6 @@ void __init noreturn __start_xen(unsigned long mbi_p)
parse_video_info();
- rdmsrl(MSR_EFER, this_cpu(efer));
- asm volatile ( "mov %%cr4,%0" : "=r" (get_cpu_info()->cr4) );
-
/* We initialise the serial devices very early so we can get debugging. */
ns16550.io_base = 0x3f8;
ns16550.irq = 4;
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 62/70] x86/alternatives: Clear CR4.CET when clearing CR0.WP
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (60 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 61/70] x86/setup: Read CR4 earlier in __start_xen() Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 63/70] x86/traps: Rework write_stub_trampoline() to not hardcode the jmp Andrew Cooper
` (9 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
This allows us to have CET active much earlier in boot.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/alternative.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c
index 436047abe021..ec24692e9595 100644
--- a/xen/arch/x86/alternative.c
+++ b/xen/arch/x86/alternative.c
@@ -333,9 +333,13 @@ static int __init cf_check nmi_apply_alternatives(
*/
if ( !(alt_done & alt_todo) )
{
- unsigned long cr0;
+ unsigned long cr0, cr4;
cr0 = read_cr0();
+ cr4 = read_cr4();
+
+ if ( cr4 & X86_CR4_CET )
+ write_cr4(cr4 & ~X86_CR4_CET);
/* Disable WP to allow patching read-only pages. */
write_cr0(cr0 & ~X86_CR0_WP);
@@ -345,6 +349,9 @@ static int __init cf_check nmi_apply_alternatives(
write_cr0(cr0);
+ if ( cr4 & X86_CR4_CET )
+ write_cr4(cr4);
+
alt_done |= alt_todo;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 63/70] x86/traps: Rework write_stub_trampoline() to not hardcode the jmp
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (61 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 62/70] x86/alternatives: Clear CR4.CET when clearing CR0.WP Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions Andrew Cooper
` (8 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
For CET-IBT, we will need to optionally insert an endbr64 instruction at the
start of the stub. Don't hardcode the jmp displacement assuming that it
starts at byte 24 of the stub.
Also add extra comments describing what is going on. The mix of %rax and %rsp
is far from trivial to follow.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Retain the rounding up to 16 bytes.
---
xen/arch/x86/x86_64/traps.c | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
index d661d7ffcaaf..edc6820b85c7 100644
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -293,30 +293,39 @@ static unsigned int write_stub_trampoline(
unsigned char *stub, unsigned long stub_va,
unsigned long stack_bottom, unsigned long target_va)
{
+ unsigned char *p = stub;
+
+ /* Store guest %rax into %ss slot */
/* movabsq %rax, stack_bottom - 8 */
- stub[0] = 0x48;
- stub[1] = 0xa3;
- *(uint64_t *)&stub[2] = stack_bottom - 8;
+ *p++ = 0x48;
+ *p++ = 0xa3;
+ *(uint64_t *)p = stack_bottom - 8;
+ p += 8;
+ /* Store guest %rsp in %rax */
/* movq %rsp, %rax */
- stub[10] = 0x48;
- stub[11] = 0x89;
- stub[12] = 0xe0;
+ *p++ = 0x48;
+ *p++ = 0x89;
+ *p++ = 0xe0;
+ /* Switch to Xen stack */
/* movabsq $stack_bottom - 8, %rsp */
- stub[13] = 0x48;
- stub[14] = 0xbc;
- *(uint64_t *)&stub[15] = stack_bottom - 8;
+ *p++ = 0x48;
+ *p++ = 0xbc;
+ *(uint64_t *)p = stack_bottom - 8;
+ p += 8;
+ /* Store guest %rsp into %rsp slot */
/* pushq %rax */
- stub[23] = 0x50;
+ *p++ = 0x50;
/* jmp target_va */
- stub[24] = 0xe9;
- *(int32_t *)&stub[25] = target_va - (stub_va + 29);
+ *p++ = 0xe9;
+ *(int32_t *)p = target_va - (stub_va + (p - stub) + 4);
+ p += 4;
/* Round up to a multiple of 16 bytes. */
- return 32;
+ return ROUNDUP(p - stub, 16);
}
DEFINE_PER_CPU(struct stubs, stubs);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (62 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 63/70] x86/traps: Rework write_stub_trampoline() to not hardcode the jmp Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 16:14 ` Andrew Cooper
2022-02-15 16:31 ` Jan Beulich
2022-02-14 12:51 ` [PATCH v2 65/70] x86/emul: Update emulation stubs to be CET-IBT compatible Andrew Cooper
` (7 subsequent siblings)
71 siblings, 2 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
... to prevent the optimiser creating unsafe code. See the code comment for
full details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Fix include to let the header be standalone
* Add earlyclobber to asm
v1.1:
* New
---
xen/arch/x86/include/asm/endbr.h | 53 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
create mode 100644 xen/arch/x86/include/asm/endbr.h
diff --git a/xen/arch/x86/include/asm/endbr.h b/xen/arch/x86/include/asm/endbr.h
new file mode 100644
index 000000000000..6b6f46afaf29
--- /dev/null
+++ b/xen/arch/x86/include/asm/endbr.h
@@ -0,0 +1,53 @@
+/******************************************************************************
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Copyright (c) 2021-2022 Citrix Systems Ltd.
+ */
+#ifndef XEN_ASM_ENDBR_H
+#define XEN_ASM_ENDBR_H
+
+#include <xen/types.h>
+
+/*
+ * In some cases we need to inspect/insert endbr64 instructions.
+ *
+ * The naive way, mem{cmp,cpy}(ptr, "\xf3\x0f\x1e\xfa", 4), optimises unsafely
+ * by placing 0xfa1e0ff3 in an imm32 operand, and marks a legal indirect
+ * branch target as far as the CPU is concerned.
+ *
+ * gen_endbr64() is written deliberately to avoid the problematic operand, and
+ * marked __const__ as it is safe for the optimiser to hoist/merge/etc.
+ */
+static inline uint32_t __attribute_const__ gen_endbr64(void)
+{
+ uint32_t res;
+
+ asm ( "mov $~0xfa1e0ff3, %[res]\n\t"
+ "not %[res]\n\t"
+ : [res] "=&r" (res) );
+
+ return res;
+}
+
+static inline bool is_endbr64(const void *ptr)
+{
+ return *(const uint32_t *)ptr == gen_endbr64();
+}
+
+static inline void place_endbr64(void *ptr)
+{
+ *(uint32_t *)ptr = gen_endbr64();
+}
+
+#endif /* XEN_ASM_ENDBR_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions
2022-02-14 12:51 ` [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions Andrew Cooper
@ 2022-02-14 16:14 ` Andrew Cooper
2022-02-15 16:31 ` Jan Beulich
1 sibling, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 16:14 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich, Roger Pau Monne, Wei Liu
On 14/02/2022 12:51, Andrew Cooper wrote:
> ... to prevent the optimiser creating unsafe code. See the code comment for
> full details.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
From review in the follow-up series, I've merged this delta:
diff --git a/xen/arch/x86/include/asm/endbr.h
b/xen/arch/x86/include/asm/endbr.h
index 6b6f46afaf29..6090afeb0bd8 100644
--- a/xen/arch/x86/include/asm/endbr.h
+++ b/xen/arch/x86/include/asm/endbr.h
@@ -19,6 +19,8 @@
#include <xen/types.h>
+#define ENDBR64_LEN 4
+
/*
* In some cases we need to inspect/insert endbr64 instructions.
*
in, to replace some raw 4's.
~Andrew
^ permalink raw reply related [flat|nested] 123+ messages in thread
* Re: [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions
2022-02-14 12:51 ` [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions Andrew Cooper
2022-02-14 16:14 ` Andrew Cooper
@ 2022-02-15 16:31 ` Jan Beulich
1 sibling, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 16:31 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 14.02.2022 13:51, Andrew Cooper wrote:
> ... to prevent the optimiser creating unsafe code. See the code comment for
> full details.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 65/70] x86/emul: Update emulation stubs to be CET-IBT compatible
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (63 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 64/70] x86: Introduce helpers/checks for endbr64 instructions Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 66/70] x86/entry: Make syscall/sysenter entrypoints " Andrew Cooper
` (6 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
All indirect branches need to land on an endbr64 instruction.
For stub_selftests(), use endbr64 unconditionally for simplicity. For ioport
and instruction emulation, add endbr64 conditionally.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Use local endbr64 define rather than raw opcodes in stub_selftest()
v1.1:
* Update to use endbr helpers
---
xen/arch/x86/extable.c | 12 +++++++-----
xen/arch/x86/pv/emul-priv-op.c | 7 +++++++
xen/arch/x86/x86_emulate.c | 13 +++++++++++--
3 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/xen/arch/x86/extable.c b/xen/arch/x86/extable.c
index 4d1875585f9d..4913c4a6dd5d 100644
--- a/xen/arch/x86/extable.c
+++ b/xen/arch/x86/extable.c
@@ -129,20 +129,22 @@ search_exception_table(const struct cpu_user_regs *regs)
static int __init cf_check stub_selftest(void)
{
static const struct {
- uint8_t opc[4];
+ uint8_t opc[8];
uint64_t rax;
union stub_exception_token res;
} tests[] __initconst = {
- { .opc = { 0x0f, 0xb9, 0xc3, 0xc3 }, /* ud1 */
+#define endbr64 0xf3, 0x0f, 0x1e, 0xfa
+ { .opc = { endbr64, 0x0f, 0xb9, 0xc3, 0xc3 }, /* ud1 */
.res.fields.trapnr = TRAP_invalid_op },
- { .opc = { 0x90, 0x02, 0x00, 0xc3 }, /* nop; add (%rax),%al */
+ { .opc = { endbr64, 0x90, 0x02, 0x00, 0xc3 }, /* nop; add (%rax),%al */
.rax = 0x0123456789abcdef,
.res.fields.trapnr = TRAP_gp_fault },
- { .opc = { 0x02, 0x04, 0x04, 0xc3 }, /* add (%rsp,%rax),%al */
+ { .opc = { endbr64, 0x02, 0x04, 0x04, 0xc3 }, /* add (%rsp,%rax),%al */
.rax = 0xfedcba9876543210,
.res.fields.trapnr = TRAP_stack_error },
- { .opc = { 0xcc, 0xc3, 0xc3, 0xc3 }, /* int3 */
+ { .opc = { endbr64, 0xcc, 0xc3, 0xc3, 0xc3 }, /* int3 */
.res.fields.trapnr = TRAP_int3 },
+#undef endbr64
};
unsigned long addr = this_cpu(stubs.addr) + STUB_BUF_SIZE / 2;
unsigned int i;
diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index c46c072f93db..22b10dec2a6e 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -26,6 +26,7 @@
#include <asm/amd.h>
#include <asm/debugreg.h>
+#include <asm/endbr.h>
#include <asm/hpet.h>
#include <asm/hypercall.h>
#include <asm/mc146818rtc.h>
@@ -111,6 +112,12 @@ static io_emul_stub_t *io_emul_stub_setup(struct priv_op_ctxt *ctxt, u8 opcode,
p = ctxt->io_emul_stub;
+ if ( cpu_has_xen_ibt )
+ {
+ place_endbr64(p);
+ p += 4;
+ }
+
APPEND_BUFF(prologue);
APPEND_CALL(load_guest_gprs);
diff --git a/xen/arch/x86/x86_emulate.c b/xen/arch/x86/x86_emulate.c
index 60191a94dc18..720740f29b84 100644
--- a/xen/arch/x86/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate.c
@@ -17,6 +17,7 @@
#include <asm/xstate.h>
#include <asm/amd.h> /* cpu_has_amd_erratum() */
#include <asm/debugreg.h>
+#include <asm/endbr.h>
/* Avoid namespace pollution. */
#undef cmpxchg
@@ -29,11 +30,19 @@
cpu_has_amd_erratum(¤t_cpu_data, AMD_ERRATUM_##nr)
#define get_stub(stb) ({ \
+ void *ptr; \
BUILD_BUG_ON(STUB_BUF_SIZE / 2 < MAX_INST_LEN + 1); \
ASSERT(!(stb).ptr); \
(stb).addr = this_cpu(stubs.addr) + STUB_BUF_SIZE / 2; \
- memset(((stb).ptr = map_domain_page(_mfn(this_cpu(stubs.mfn)))) + \
- ((stb).addr & ~PAGE_MASK), 0xcc, STUB_BUF_SIZE / 2); \
+ (stb).ptr = map_domain_page(_mfn(this_cpu(stubs.mfn))) + \
+ ((stb).addr & ~PAGE_MASK); \
+ ptr = memset((stb).ptr, 0xcc, STUB_BUF_SIZE / 2); \
+ if ( cpu_has_xen_ibt ) \
+ { \
+ place_endbr64(ptr); \
+ ptr += 4; \
+ } \
+ ptr; \
})
#define put_stub(stb) ({ \
if ( (stb).ptr ) \
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 66/70] x86/entry: Make syscall/sysenter entrypoints CET-IBT compatible
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (64 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 65/70] x86/emul: Update emulation stubs to be CET-IBT compatible Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 67/70] x86/entry: Make IDT " Andrew Cooper
` (5 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Each of MSR_{L,C}STAR and MSR_SYSENTER_EIP need to land on an endbr64
instruction. For sysenter, this is easy.
Unfortunately for syscall, the stubs are already 29 byte long with a limit of
32. endbr64 is 4 bytes. Luckily, there is a 1 byte instruction which can
move from the stubs into the main handlers.
Move the push %rax out of the stub and into {l,c}star_entry(), allowing room
for the endbr64 instruction when appropriate. Update the comment describing
the entry state.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v1.1:
* Update to use endbr helpers.
---
xen/arch/x86/x86_64/entry.S | 18 +++++++++---------
xen/arch/x86/x86_64/traps.c | 11 +++++++----
2 files changed, 16 insertions(+), 13 deletions(-)
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 8494b97a54a2..9abcf95bd010 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -241,18 +241,17 @@ iret_exit_to_guest:
* When entering SYSCALL from user mode:
* Vector directly to the registered arch.syscall_addr.
*
- * Initial work is done by per-CPU trampolines. At this point %rsp has been
- * initialised to point at the correct Xen stack, %rsp has been saved, and
- * %rax needs to be restored from the %ss save slot. All other registers are
- * still to be saved onto the stack, starting with RFLAGS, and an appropriate
- * %ss must be saved into the space left by the trampoline.
+ * Initial work is done by per-CPU trampolines.
+ * - Guest %rax stored in the %ss slot
+ * - Guest %rsp stored in %rax
+ * - Xen stack loaded, pointing at the %ss slot
*/
ENTRY(lstar_enter)
#ifdef CONFIG_XEN_SHSTK
ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
#endif
- /* sti could live here when we don't switch page tables below. */
- movq 8(%rsp),%rax /* Restore %rax. */
+ push %rax /* Guest %rsp */
+ movq 8(%rsp), %rax /* Restore guest %rax */
movq $FLAT_KERNEL_SS,8(%rsp)
pushq %r11
pushq $FLAT_KERNEL_CS64
@@ -288,9 +287,9 @@ ENTRY(cstar_enter)
#ifdef CONFIG_XEN_SHSTK
ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
#endif
- /* sti could live here when we don't switch page tables below. */
+ push %rax /* Guest %rsp */
CR4_PV32_RESTORE
- movq 8(%rsp), %rax /* Restore %rax. */
+ movq 8(%rsp), %rax /* Restore guest %rax. */
movq $FLAT_USER_SS32, 8(%rsp) /* Assume a 64bit domain. Compat handled lower. */
pushq %r11
pushq $FLAT_USER_CS32
@@ -323,6 +322,7 @@ ENTRY(cstar_enter)
jmp switch_to_kernel
ENTRY(sysenter_entry)
+ ENDBR64
#ifdef CONFIG_XEN_SHSTK
ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
#endif
diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
index edc6820b85c7..fccfb7c17283 100644
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -16,6 +16,7 @@
#include <asm/current.h>
#include <asm/flushtlb.h>
#include <asm/traps.h>
+#include <asm/endbr.h>
#include <asm/event.h>
#include <asm/nmi.h>
#include <asm/msr.h>
@@ -295,6 +296,12 @@ static unsigned int write_stub_trampoline(
{
unsigned char *p = stub;
+ if ( cpu_has_xen_ibt )
+ {
+ place_endbr64(p);
+ p += 4;
+ }
+
/* Store guest %rax into %ss slot */
/* movabsq %rax, stack_bottom - 8 */
*p++ = 0x48;
@@ -315,10 +322,6 @@ static unsigned int write_stub_trampoline(
*(uint64_t *)p = stack_bottom - 8;
p += 8;
- /* Store guest %rsp into %rsp slot */
- /* pushq %rax */
- *p++ = 0x50;
-
/* jmp target_va */
*p++ = 0xe9;
*(int32_t *)p = target_va - (stub_va + (p - stub) + 4);
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 67/70] x86/entry: Make IDT entrypoints CET-IBT compatible
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (65 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 66/70] x86/entry: Make syscall/sysenter entrypoints " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 12:51 ` [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT Andrew Cooper
` (4 subsequent siblings)
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Each IDT vector needs to land on an endbr64 instruction. This is especially
important for the #CP handler, which will recurse indefinitely if the endbr64
is missing, eventually escalating to #DF if guard pages are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Extra newlines in asm
* Reword commit message
---
xen/arch/x86/x86_64/compat/entry.S | 1 +
xen/arch/x86/x86_64/entry.S | 30 ++++++++++++++++++++++++++++--
2 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index c84ff7ea6476..5fd6dbbd4513 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -12,6 +12,7 @@
#include <irq_vectors.h>
ENTRY(entry_int82)
+ ENDBR64
ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
pushq $0
movl $HYPERCALL_VECTOR, 4(%rsp)
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 9abcf95bd010..ea6f0afbc2b4 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -386,6 +386,7 @@ UNLIKELY_END(sysenter_gpf)
jmp .Lbounce_exception
ENTRY(int80_direct_trap)
+ ENDBR64
ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
pushq $0
movl $0x80, 4(%rsp)
@@ -698,6 +699,7 @@ ENTRY(common_interrupt)
jmp ret_from_intr
ENTRY(page_fault)
+ ENDBR64
movl $TRAP_page_fault,4(%rsp)
/* No special register assumptions. */
GLOBAL(handle_exception)
@@ -872,75 +874,91 @@ FATAL_exception_with_ints_disabled:
BUG /* fatal_trap() shouldn't return. */
ENTRY(divide_error)
+ ENDBR64
pushq $0
movl $TRAP_divide_error,4(%rsp)
jmp handle_exception
ENTRY(coprocessor_error)
+ ENDBR64
pushq $0
movl $TRAP_copro_error,4(%rsp)
jmp handle_exception
ENTRY(simd_coprocessor_error)
+ ENDBR64
pushq $0
movl $TRAP_simd_error,4(%rsp)
jmp handle_exception
ENTRY(device_not_available)
+ ENDBR64
pushq $0
movl $TRAP_no_device,4(%rsp)
jmp handle_exception
ENTRY(debug)
+ ENDBR64
pushq $0
movl $TRAP_debug,4(%rsp)
jmp handle_ist_exception
ENTRY(int3)
+ ENDBR64
pushq $0
movl $TRAP_int3,4(%rsp)
jmp handle_exception
ENTRY(overflow)
+ ENDBR64
pushq $0
movl $TRAP_overflow,4(%rsp)
jmp handle_exception
ENTRY(bounds)
+ ENDBR64
pushq $0
movl $TRAP_bounds,4(%rsp)
jmp handle_exception
ENTRY(invalid_op)
+ ENDBR64
pushq $0
movl $TRAP_invalid_op,4(%rsp)
jmp handle_exception
ENTRY(invalid_TSS)
+ ENDBR64
movl $TRAP_invalid_tss,4(%rsp)
jmp handle_exception
ENTRY(segment_not_present)
+ ENDBR64
movl $TRAP_no_segment,4(%rsp)
jmp handle_exception
ENTRY(stack_segment)
+ ENDBR64
movl $TRAP_stack_error,4(%rsp)
jmp handle_exception
ENTRY(general_protection)
+ ENDBR64
movl $TRAP_gp_fault,4(%rsp)
jmp handle_exception
ENTRY(alignment_check)
+ ENDBR64
movl $TRAP_alignment_check,4(%rsp)
jmp handle_exception
ENTRY(entry_CP)
+ ENDBR64
movl $X86_EXC_CP, 4(%rsp)
jmp handle_exception
ENTRY(double_fault)
+ ENDBR64
movl $TRAP_double_fault,4(%rsp)
/* Set AC to reduce chance of further SMAP faults */
ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
@@ -966,6 +984,7 @@ ENTRY(double_fault)
.pushsection .init.text, "ax", @progbits
ENTRY(early_page_fault)
+ ENDBR64
movl $TRAP_page_fault,4(%rsp)
SAVE_ALL
movq %rsp,%rdi
@@ -974,6 +993,7 @@ ENTRY(early_page_fault)
.popsection
ENTRY(nmi)
+ ENDBR64
pushq $0
movl $TRAP_nmi,4(%rsp)
handle_ist_exception:
@@ -1102,12 +1122,14 @@ handle_ist_exception:
#endif
ENTRY(machine_check)
+ ENDBR64
pushq $0
movl $TRAP_machine_check,4(%rsp)
jmp handle_ist_exception
/* No op trap handler. Required for kexec crash path. */
GLOBAL(trap_nop)
+ ENDBR64
iretq
/* Table of automatically generated entry points. One per vector. */
@@ -1136,7 +1158,9 @@ autogen_stubs: /* Automatically generated stubs. */
#endif
ALIGN
-1: pushq $0
+1:
+ ENDBR64
+ pushq $0
movb $vec,4(%rsp)
jmp common_interrupt
@@ -1146,7 +1170,9 @@ autogen_stubs: /* Automatically generated stubs. */
.elseif vec == X86_EXC_CSO || vec == X86_EXC_SPV || \
vec == X86_EXC_VE || (vec > X86_EXC_CP && vec < TRAP_nr)
-1: test $8,%spl /* 64bit exception frames are 16 byte aligned, but the word */
+1:
+ ENDBR64
+ test $8,%spl /* 64bit exception frames are 16 byte aligned, but the word */
jz 2f /* size is 8 bytes. Check whether the processor gave us an */
pushq $0 /* error code, and insert an empty one if not. */
2: movb $vec,4(%rsp)
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (66 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 67/70] x86/entry: Make IDT " Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-15 16:46 ` Jan Beulich
2022-02-14 12:51 ` [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls Andrew Cooper
` (3 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
CET-SS and CET-IBT can be independently controlled, so the configuration of
MSR_S_CET can't be constant any more.
Introduce xen_msr_s_cet_value(), mostly because I don't fancy
writing/maintaining that logic in assembly. Use this in the 3 paths which
alter MSR_S_CET when both features are potentially active.
To active CET-IBT, we only need CR4.CET and MSR_S_CET.ENDBR_EN. This is
common with the CET-SS setup, so reorder the operations to set up CR4 and
MSR_S_CET for any nonzero result from xen_msr_s_cet_value(), and set up
MSR_PL0_SSP and SSP if SHSTK_EN was also set.
Adjust the crash path to disable CET-IBT too.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Asm adjustments. Add comments regarding safety.
---
xen/arch/x86/acpi/wakeup_prot.S | 38 ++++++++++++++++++++++--------------
xen/arch/x86/boot/x86_64.S | 30 +++++++++++++++++-----------
xen/arch/x86/crash.c | 4 ++--
xen/arch/x86/include/asm/msr-index.h | 1 +
xen/arch/x86/setup.c | 17 +++++++++++++++-
5 files changed, 61 insertions(+), 29 deletions(-)
diff --git a/xen/arch/x86/acpi/wakeup_prot.S b/xen/arch/x86/acpi/wakeup_prot.S
index 15052c300fa1..3855ff1ddb94 100644
--- a/xen/arch/x86/acpi/wakeup_prot.S
+++ b/xen/arch/x86/acpi/wakeup_prot.S
@@ -63,7 +63,26 @@ ENTRY(s3_resume)
pushq %rax
lretq
1:
-#ifdef CONFIG_XEN_SHSTK
+#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
+ call xen_msr_s_cet_value
+ test %eax, %eax
+ jz .L_cet_done
+
+ /* Set up MSR_S_CET. */
+ mov $MSR_S_CET, %ecx
+ xor %edx, %edx
+ wrmsr
+
+ /* Enable CR4.CET. */
+ mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
+ mov %rcx, %cr4
+
+ /* WARNING! call/ret now fatal (iff SHSTK) until SETSSBSY loads SSP */
+
+#if defined(CONFIG_XEN_SHSTK)
+ test $CET_SHSTK_EN, %al
+ jz .L_cet_done
+
/*
* Restoring SSP is a little complicated, because we are intercepting
* an in-use shadow stack. Write a temporary token under the stack,
@@ -71,14 +90,6 @@ ENTRY(s3_resume)
* reset MSR_PL0_SSP to its usual value and pop the temporary token.
*/
mov saved_ssp(%rip), %rdi
- cmpq $1, %rdi
- je .L_shstk_done
-
- /* Set up MSR_S_CET. */
- mov $MSR_S_CET, %ecx
- xor %edx, %edx
- mov $CET_SHSTK_EN | CET_WRSS_EN, %eax
- wrmsr
/* Construct the temporary supervisor token under SSP. */
sub $8, %rdi
@@ -90,10 +101,6 @@ ENTRY(s3_resume)
mov %edi, %eax
wrmsr
- /* Enable CET. MSR_INTERRUPT_SSP_TABLE is set up later in load_system_tables(). */
- mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ebx
- mov %rbx, %cr4
-
/* Write the temporary token onto the shadow stack, and activate it. */
wrssq %rdi, (%rdi)
setssbsy
@@ -106,8 +113,9 @@ ENTRY(s3_resume)
/* Pop the temporary token off the stack. */
mov $2, %eax
incsspd %eax
-.L_shstk_done:
-#endif
+#endif /* CONFIG_XEN_SHSTK */
+.L_cet_done:
+#endif /* CONFIG_XEN_SHSTK || CONFIG_XEN_IBT */
call load_system_tables
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 27f52e7a7708..fa41990dde0f 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -30,18 +30,27 @@ ENTRY(__high_start)
test %ebx,%ebx
jz .L_bsp
- /* APs. Set up shadow stacks before entering C. */
-#ifdef CONFIG_XEN_SHSTK
- testl $cpufeat_mask(X86_FEATURE_XEN_SHSTK), \
- CPUINFO_FEATURE_OFFSET(X86_FEATURE_XEN_SHSTK) + boot_cpu_data(%rip)
- je .L_ap_shstk_done
+ /* APs. Set up CET before entering C properly. */
+#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
+ call xen_msr_s_cet_value
+ test %eax, %eax
+ jz .L_ap_cet_done
/* Set up MSR_S_CET. */
mov $MSR_S_CET, %ecx
xor %edx, %edx
- mov $CET_SHSTK_EN | CET_WRSS_EN, %eax
wrmsr
+ /* Enable CR4.CET. */
+ mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
+ mov %rcx, %cr4
+
+ /* WARNING! call/ret now fatal (iff SHSTK) until SETSSBSY loads SSP */
+
+#if defined(CONFIG_XEN_SHSTK)
+ test $CET_SHSTK_EN, %al
+ jz .L_ap_cet_done
+
/* Derive MSR_PL0_SSP from %rsp (token written when stack is allocated). */
mov $MSR_PL0_SSP, %ecx
mov %rsp, %rdx
@@ -51,13 +60,12 @@ ENTRY(__high_start)
or $(PRIMARY_SHSTK_SLOT + 1) * PAGE_SIZE - 8, %eax
wrmsr
- /* Enable CET. MSR_INTERRUPT_SSP_TABLE is set up later in load_system_tables(). */
- mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
- mov %rcx, %cr4
setssbsy
-#endif
-.L_ap_shstk_done:
+#endif /* CONFIG_XEN_SHSTK */
+.L_ap_cet_done:
+#endif /* CONFIG_XEN_SHSTK || CONFIG_XEN_IBT */
+
call start_secondary
BUG /* start_secondary() shouldn't return. */
diff --git a/xen/arch/x86/crash.c b/xen/arch/x86/crash.c
index c383f718f5bd..003222c0f1ac 100644
--- a/xen/arch/x86/crash.c
+++ b/xen/arch/x86/crash.c
@@ -190,8 +190,8 @@ void machine_crash_shutdown(void)
/* Reset CPUID masking and faulting to the host's default. */
ctxt_switch_levelling(NULL);
- /* Disable shadow stacks. */
- if ( cpu_has_xen_shstk )
+ /* Disable CET. */
+ if ( cpu_has_xen_shstk || cpu_has_xen_ibt )
{
wrmsrl(MSR_S_CET, 0);
write_cr4(read_cr4() & ~X86_CR4_CET);
diff --git a/xen/arch/x86/include/asm/msr-index.h b/xen/arch/x86/include/asm/msr-index.h
index 9df1959fe5a1..3e038db618ff 100644
--- a/xen/arch/x86/include/asm/msr-index.h
+++ b/xen/arch/x86/include/asm/msr-index.h
@@ -117,6 +117,7 @@
#define MSR_S_CET 0x000006a2
#define CET_SHSTK_EN (_AC(1, ULL) << 0)
#define CET_WRSS_EN (_AC(1, ULL) << 1)
+#define CET_ENDBR_EN (_AC(1, ULL) << 2)
#define MSR_PL0_SSP 0x000006a4
#define MSR_PL1_SSP 0x000006a5
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 2b1192d85b77..f6a59d5f0412 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -670,6 +670,21 @@ static void noreturn init_done(void)
startup_cpu_idle_loop();
}
+#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
+/*
+ * Used by AP and S3 asm code to calcualte the appropriate MSR_S_CET setting.
+ * Do not use on the BSP before reinit_bsp_stack(), or it may turn SHSTK on
+ * too early.
+ */
+unsigned int xen_msr_s_cet_value(void)
+{
+ return ((cpu_has_xen_shstk ? CET_SHSTK_EN | CET_WRSS_EN : 0) |
+ (cpu_has_xen_ibt ? CET_ENDBR_EN : 0));
+}
+#else
+unsigned int xen_msr_s_cet_value(void); /* To avoid ifdefary */
+#endif
+
/* Reinitalise all state referring to the old virtual address of the stack. */
static void __init noreturn reinit_bsp_stack(void)
{
@@ -693,7 +708,7 @@ static void __init noreturn reinit_bsp_stack(void)
{
wrmsrl(MSR_PL0_SSP,
(unsigned long)stack + (PRIMARY_SHSTK_SLOT + 1) * PAGE_SIZE - 8);
- wrmsrl(MSR_S_CET, CET_SHSTK_EN | CET_WRSS_EN);
+ wrmsrl(MSR_S_CET, xen_msr_s_cet_value());
asm volatile ("setssbsy" ::: "memory");
}
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT
2022-02-14 12:51 ` [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT Andrew Cooper
@ 2022-02-15 16:46 ` Jan Beulich
2022-02-15 20:58 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 16:46 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 14.02.2022 13:51, Andrew Cooper wrote:
> CET-SS and CET-IBT can be independently controlled, so the configuration of
> MSR_S_CET can't be constant any more.
>
> Introduce xen_msr_s_cet_value(), mostly because I don't fancy
> writing/maintaining that logic in assembly. Use this in the 3 paths which
> alter MSR_S_CET when both features are potentially active.
>
> To active CET-IBT, we only need CR4.CET and MSR_S_CET.ENDBR_EN. This is
> common with the CET-SS setup, so reorder the operations to set up CR4 and
> MSR_S_CET for any nonzero result from xen_msr_s_cet_value(), and set up
> MSR_PL0_SSP and SSP if SHSTK_EN was also set.
>
> Adjust the crash path to disable CET-IBT too.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit with a nit and a remark:
> --- a/xen/arch/x86/acpi/wakeup_prot.S
> +++ b/xen/arch/x86/acpi/wakeup_prot.S
> @@ -63,7 +63,26 @@ ENTRY(s3_resume)
> pushq %rax
> lretq
> 1:
> -#ifdef CONFIG_XEN_SHSTK
> +#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
> + call xen_msr_s_cet_value
> + test %eax, %eax
> + jz .L_cet_done
> +
> + /* Set up MSR_S_CET. */
> + mov $MSR_S_CET, %ecx
> + xor %edx, %edx
> + wrmsr
> +
> + /* Enable CR4.CET. */
> + mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
> + mov %rcx, %cr4
> +
> + /* WARNING! call/ret now fatal (iff SHSTK) until SETSSBSY loads SSP */
> +
> +#if defined(CONFIG_XEN_SHSTK)
Just #ifdef, as it was before?
> @@ -90,10 +101,6 @@ ENTRY(s3_resume)
> mov %edi, %eax
> wrmsr
>
> - /* Enable CET. MSR_INTERRUPT_SSP_TABLE is set up later in load_system_tables(). */
> - mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ebx
> - mov %rbx, %cr4
The latter part of this comment could do with retaining.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT
2022-02-15 16:46 ` Jan Beulich
@ 2022-02-15 20:58 ` Andrew Cooper
2022-02-16 8:49 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-15 20:58 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 15/02/2022 16:46, Jan Beulich wrote:
> On 14.02.2022 13:51, Andrew Cooper wrote:
>> CET-SS and CET-IBT can be independently controlled, so the configuration of
>> MSR_S_CET can't be constant any more.
>>
>> Introduce xen_msr_s_cet_value(), mostly because I don't fancy
>> writing/maintaining that logic in assembly. Use this in the 3 paths which
>> alter MSR_S_CET when both features are potentially active.
>>
>> To active CET-IBT, we only need CR4.CET and MSR_S_CET.ENDBR_EN. This is
>> common with the CET-SS setup, so reorder the operations to set up CR4 and
>> MSR_S_CET for any nonzero result from xen_msr_s_cet_value(), and set up
>> MSR_PL0_SSP and SSP if SHSTK_EN was also set.
>>
>> Adjust the crash path to disable CET-IBT too.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks,
> albeit with a nit and a remark:
>
>> --- a/xen/arch/x86/acpi/wakeup_prot.S
>> +++ b/xen/arch/x86/acpi/wakeup_prot.S
>> @@ -63,7 +63,26 @@ ENTRY(s3_resume)
>> pushq %rax
>> lretq
>> 1:
>> -#ifdef CONFIG_XEN_SHSTK
>> +#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
>> + call xen_msr_s_cet_value
>> + test %eax, %eax
>> + jz .L_cet_done
>> +
>> + /* Set up MSR_S_CET. */
>> + mov $MSR_S_CET, %ecx
>> + xor %edx, %edx
>> + wrmsr
>> +
>> + /* Enable CR4.CET. */
>> + mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
>> + mov %rcx, %cr4
>> +
>> + /* WARNING! call/ret now fatal (iff SHSTK) until SETSSBSY loads SSP */
>> +
>> +#if defined(CONFIG_XEN_SHSTK)
> Just #ifdef, as it was before?
I can if you insist, but that's breaking consistency with the other
ifdefary.
>
>> @@ -90,10 +101,6 @@ ENTRY(s3_resume)
>> mov %edi, %eax
>> wrmsr
>>
>> - /* Enable CET. MSR_INTERRUPT_SSP_TABLE is set up later in load_system_tables(). */
>> - mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ebx
>> - mov %rbx, %cr4
> The latter part of this comment could do with retaining.
So I tried that in v1, and concluded not for v2.
There is nowhere appropriate for it to live, anywhere in this block.
And it is an artefact of me bootstrapping SHSTK to start with.
The truth is that nothing about MSR_ISST_TABLE matters until
load_system_table sets up both this and the TSS IST fields together.
IST exceptions are already fatal at this point for non-SHSTK reasons.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT
2022-02-15 20:58 ` Andrew Cooper
@ 2022-02-16 8:49 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-16 8:49 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 15.02.2022 21:58, Andrew Cooper wrote:
> On 15/02/2022 16:46, Jan Beulich wrote:
>> On 14.02.2022 13:51, Andrew Cooper wrote:
>>> CET-SS and CET-IBT can be independently controlled, so the configuration of
>>> MSR_S_CET can't be constant any more.
>>>
>>> Introduce xen_msr_s_cet_value(), mostly because I don't fancy
>>> writing/maintaining that logic in assembly. Use this in the 3 paths which
>>> alter MSR_S_CET when both features are potentially active.
>>>
>>> To active CET-IBT, we only need CR4.CET and MSR_S_CET.ENDBR_EN. This is
>>> common with the CET-SS setup, so reorder the operations to set up CR4 and
>>> MSR_S_CET for any nonzero result from xen_msr_s_cet_value(), and set up
>>> MSR_PL0_SSP and SSP if SHSTK_EN was also set.
>>>
>>> Adjust the crash path to disable CET-IBT too.
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
> Thanks,
>
>> albeit with a nit and a remark:
>>
>>> --- a/xen/arch/x86/acpi/wakeup_prot.S
>>> +++ b/xen/arch/x86/acpi/wakeup_prot.S
>>> @@ -63,7 +63,26 @@ ENTRY(s3_resume)
>>> pushq %rax
>>> lretq
>>> 1:
>>> -#ifdef CONFIG_XEN_SHSTK
>>> +#if defined(CONFIG_XEN_SHSTK) || defined(CONFIG_XEN_IBT)
>>> + call xen_msr_s_cet_value
>>> + test %eax, %eax
>>> + jz .L_cet_done
>>> +
>>> + /* Set up MSR_S_CET. */
>>> + mov $MSR_S_CET, %ecx
>>> + xor %edx, %edx
>>> + wrmsr
>>> +
>>> + /* Enable CR4.CET. */
>>> + mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx
>>> + mov %rcx, %cr4
>>> +
>>> + /* WARNING! call/ret now fatal (iff SHSTK) until SETSSBSY loads SSP */
>>> +
>>> +#if defined(CONFIG_XEN_SHSTK)
>> Just #ifdef, as it was before?
>
> I can if you insist, but that's breaking consistency with the other
> ifdefary.
I guess consistent of not depends on the way you look at it. I
generally think simple conditionals should just use #ifdef. As
soon as there's an #elif or a more complex condition, #if
defined() is of course more consistent. But one #ifdef nested
inside another #if imo isn't a reason to use #if in both places.
Nevertheless, ftaod - I'm not going to insist, as I can see this
being a matter of personal preference.
>>> @@ -90,10 +101,6 @@ ENTRY(s3_resume)
>>> mov %edi, %eax
>>> wrmsr
>>>
>>> - /* Enable CET. MSR_INTERRUPT_SSP_TABLE is set up later in load_system_tables(). */
>>> - mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ebx
>>> - mov %rbx, %cr4
>> The latter part of this comment could do with retaining.
>
> So I tried that in v1, and concluded not for v2.
>
> There is nowhere appropriate for it to live, anywhere in this block.
> And it is an artefact of me bootstrapping SHSTK to start with.
>
> The truth is that nothing about MSR_ISST_TABLE matters until
> load_system_table sets up both this and the TSS IST fields together.
> IST exceptions are already fatal at this point for non-SHSTK reasons.
Well, okay. To me, not being as familiar with this code as you
are, the comments was quite helpful ...
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (67 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 68/70] x86/setup: Rework MSR_S_CET handling for CET-IBT Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-15 16:53 ` Jan Beulich
2022-02-14 12:51 ` [PATCH v2 70/70] x86: Enable CET Indirect Branch Tracking Andrew Cooper
` (2 subsequent siblings)
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
UEFI Runtime services, at the time of writing, aren't CET-IBT compatible.
Work is ongoing to address this. In the meantime, unconditionally disable IBT.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
https://bugzilla.tianocore.org/show_bug.cgi?id=3726 is the upstream tracking
ticket.
v2:
* Rewrite to be an unconditional disable.
---
xen/common/efi/runtime.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/xen/common/efi/runtime.c b/xen/common/efi/runtime.c
index e3ce85d118e4..13b0975866e3 100644
--- a/xen/common/efi/runtime.c
+++ b/xen/common/efi/runtime.c
@@ -21,6 +21,7 @@ struct efi_rs_state {
* don't strictly need that.
*/
unsigned long __aligned(32) cr3;
+ unsigned long msr_s_cet;
#endif
};
@@ -113,6 +114,19 @@ struct efi_rs_state efi_rs_enter(void)
switch_cr3_cr4(mfn_to_maddr(efi_l4_mfn), read_cr4());
+ /*
+ * At the time of writing (2022), no UEFI firwmare is CET-IBT compatible.
+ * Work is under way to remedy this.
+ *
+ * Stash MSR_S_CET and clobber ENDBR_EN. This is necessary because
+ * SHSTK_EN isn't configured until very late on the BSP.
+ */
+ if ( cpu_has_xen_ibt )
+ {
+ rdmsrl(MSR_S_CET, state.msr_s_cet);
+ wrmsrl(MSR_S_CET, state.msr_s_cet & ~CET_ENDBR_EN);
+ }
+
return state;
}
@@ -122,6 +136,10 @@ void efi_rs_leave(struct efi_rs_state *state)
if ( !state->cr3 )
return;
+
+ if ( state->msr_s_cet )
+ wrmsrl(MSR_S_CET, state->msr_s_cet);
+
switch_cr3_cr4(state->cr3, read_cr4());
if ( is_pv_vcpu(curr) && !is_idle_vcpu(curr) )
{
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls
2022-02-14 12:51 ` [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls Andrew Cooper
@ 2022-02-15 16:53 ` Jan Beulich
2022-02-15 23:00 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-15 16:53 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 14.02.2022 13:51, Andrew Cooper wrote:
> UEFI Runtime services, at the time of writing, aren't CET-IBT compatible.
> Work is ongoing to address this. In the meantime, unconditionally disable IBT.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
> --- a/xen/common/efi/runtime.c
> +++ b/xen/common/efi/runtime.c
> @@ -21,6 +21,7 @@ struct efi_rs_state {
> * don't strictly need that.
> */
> unsigned long __aligned(32) cr3;
> + unsigned long msr_s_cet;
> #endif
> };
The latest with the next addition here we will probably want to ...
> @@ -113,6 +114,19 @@ struct efi_rs_state efi_rs_enter(void)
... no longer have this be the function's return type.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls
2022-02-15 16:53 ` Jan Beulich
@ 2022-02-15 23:00 ` Andrew Cooper
2022-02-16 9:14 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-15 23:00 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 15/02/2022 16:53, Jan Beulich wrote:
> On 14.02.2022 13:51, Andrew Cooper wrote:
>> UEFI Runtime services, at the time of writing, aren't CET-IBT compatible.
>> Work is ongoing to address this. In the meantime, unconditionally disable IBT.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks.
>
>> --- a/xen/common/efi/runtime.c
>> +++ b/xen/common/efi/runtime.c
>> @@ -21,6 +21,7 @@ struct efi_rs_state {
>> * don't strictly need that.
>> */
>> unsigned long __aligned(32) cr3;
>> + unsigned long msr_s_cet;
>> #endif
>> };
> The latest with the next addition here we will probably want to ...
>
>> @@ -113,6 +114,19 @@ struct efi_rs_state efi_rs_enter(void)
> ... no longer have this be the function's return type.
So about this.
why aren't we using __attribute__((force_align_arg_pointer)) ? It
exists in at least GCC 4.1 and Clang 6.
We're way way overdue bumping the minimum toolchain versions, and Clang
3.5=>6 is still very obsolete minimum version. This way, we're not
depending on some very subtle ABI mechanics to try and keep the stack
properly aligned.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls
2022-02-15 23:00 ` Andrew Cooper
@ 2022-02-16 9:14 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-16 9:14 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 16.02.2022 00:00, Andrew Cooper wrote:
> On 15/02/2022 16:53, Jan Beulich wrote:
>> On 14.02.2022 13:51, Andrew Cooper wrote:
>>> --- a/xen/common/efi/runtime.c
>>> +++ b/xen/common/efi/runtime.c
>>> @@ -21,6 +21,7 @@ struct efi_rs_state {
>>> * don't strictly need that.
>>> */
>>> unsigned long __aligned(32) cr3;
>>> + unsigned long msr_s_cet;
>>> #endif
>>> };
>> The latest with the next addition here we will probably want to ...
>>
>>> @@ -113,6 +114,19 @@ struct efi_rs_state efi_rs_enter(void)
>> ... no longer have this be the function's return type.
>
> So about this.
>
> why aren't we using __attribute__((force_align_arg_pointer)) ? It
> exists in at least GCC 4.1 and Clang 6.
Perhaps first and foremost because this is the first time I encounter
this attribute, despite it having been around for so long. However,
Clang 6 would be a little too high for the main box I have a Clang
installed on - that's Clang 5 only (and, afaict, no option to upgrade
without also upgrading the distro, while I'd also like to avoid having
to also build myself Clang binaries; maybe sooner or later that's
going to be unavoidable, though). While from binary searching its
libraries it looks to know of that attribute, it still doesn't accept
its use.
The other issue I see is that using it would be fragile: We cannot
afford to forget putting the attribute on any of the relevant
functions. Whereas the present model makes it impossible to miss
any instance.
Finally the attribute's interaction with -mpreferred-stack-boundary=
isn't spelled out anywhere. It looks to behave sanely on gcc 11, but
who knows whether this has always been the case.
Jan
> We're way way overdue bumping the minimum toolchain versions, and Clang
> 3.5=>6 is still very obsolete minimum version. This way, we're not
> depending on some very subtle ABI mechanics to try and keep the stack
> properly aligned.
>
> ~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2 70/70] x86: Enable CET Indirect Branch Tracking
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (68 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 69/70] x86/efi: Disable CET-IBT around Runtime Services calls Andrew Cooper
@ 2022-02-14 12:51 ` Andrew Cooper
2022-02-14 13:10 ` [PATCH v2 00/70] x86: Support for " Andrew Cooper
2022-02-17 10:01 ` [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
71 siblings, 0 replies; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 12:51 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
With all the pieces now in place, turn CET-IBT on when available.
MSR_S_CET, like SMEP/SMAP, controls Ring1 meaning that ENDBR_EN can't be
enabled for Xen independently of PV32 kernels. As we already disable PV32 for
CET-SS, extend this to all CET, adjusting the documentation/comments as
appropriate.
Introduce a cet=no-ibt command line option to allow the admin to disable IBT
even when everything else is configured correctly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* Rebase over change to UEFI RS handling
---
docs/misc/xen-command-line.pandoc | 16 +++++++++++----
xen/arch/x86/cpu/common.c | 1 +
xen/arch/x86/setup.c | 42 ++++++++++++++++++++++++++++++++++-----
3 files changed, 50 insertions(+), 9 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 1ca817f5e1b9..92891a856971 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -271,7 +271,7 @@ enough. Setting this to a high value may cause boot failure, particularly if
the NMI watchdog is also enabled.
### cet
- = List of [ shstk=<bool> ]
+ = List of [ shstk=<bool>, ibt=<bool> ]
Applicability: x86
@@ -279,6 +279,10 @@ Controls for the use of Control-flow Enforcement Technology. CET is group a
of hardware features designed to combat Return-oriented Programming (ROP, also
call/jmp COP/JOP) attacks.
+CET is incompatible with 32bit PV guests. If any CET sub-options are active,
+they will override the `pv=32` boolean to `false`. Backwards compatibility
+can be maintained with the pv-shim mechanism.
+
* The `shstk=` boolean controls whether Xen uses Shadow Stacks for its own
protection.
@@ -287,9 +291,13 @@ call/jmp COP/JOP) attacks.
`cet=no-shstk` will cause Xen not to use Shadow Stacks even when support
is available in hardware.
- Shadow Stacks are incompatible with 32bit PV guests. This option will
- override the `pv=32` boolean to false. Backwards compatibility can be
- maintained with the `pv-shim` mechanism.
+* The `ibt=` boolean controls whether Xen uses Indirect Branch Tracking for
+ its own protection.
+
+ The option is available when `CONFIG_XEN_IBT` is compiled in, and defaults
+ to `true` on hardware supporting CET-IBT. Specifying `cet=no-ibt` will
+ cause Xen not to use Indirect Branch Tracking even when support is
+ available in hardware.
### clocksource (x86)
> `= pit | hpet | acpi | tsc`
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 6b674bf15e8b..bfb8cf9f100b 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -345,6 +345,7 @@ void __init early_cpu_init(void)
if (c->cpuid_level >= 7) {
cpuid_count(7, 0, &eax, &ebx, &ecx, &edx);
c->x86_capability[cpufeat_word(X86_FEATURE_CET_SS)] = ecx;
+ c->x86_capability[cpufeat_word(X86_FEATURE_CET_IBT)] = edx;
}
eax = cpuid_eax(0x80000000);
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index f6a59d5f0412..f5449c972825 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -102,6 +102,12 @@ static bool __initdata opt_xen_shstk = true;
#define opt_xen_shstk false
#endif
+#ifdef CONFIG_XEN_IBT
+static bool __initdata opt_xen_ibt = true;
+#else
+#define opt_xen_ibt false
+#endif
+
static int __init cf_check parse_cet(const char *s)
{
const char *ss;
@@ -120,6 +126,14 @@ static int __init cf_check parse_cet(const char *s)
no_config_param("XEN_SHSTK", "cet", s, ss);
#endif
}
+ else if ( (val = parse_boolean("ibt", s, ss)) >= 0 )
+ {
+#ifdef CONFIG_XEN_IBT
+ opt_xen_ibt = val;
+#else
+ no_config_param("XEN_IBT", "cet", s, ss);
+#endif
+ }
else
rc = -EINVAL;
@@ -1118,11 +1132,33 @@ void __init noreturn __start_xen(unsigned long mbi_p)
printk("Enabling Supervisor Shadow Stacks\n");
setup_force_cpu_cap(X86_FEATURE_XEN_SHSTK);
+ }
+
+ if ( opt_xen_ibt && boot_cpu_has(X86_FEATURE_CET_IBT) )
+ {
+ printk("Enabling Indirect Branch Tracking\n");
+
+ setup_force_cpu_cap(X86_FEATURE_XEN_IBT);
+
+ if ( efi_enabled(EFI_RS) )
+ printk(" - IBT disabled in UEFI Runtime Services\n");
+
+ /*
+ * Enable IBT now. Only require the endbr64 on callees, which is
+ * entirely build-time arrangements.
+ */
+ wrmsrl(MSR_S_CET, CET_ENDBR_EN);
+ }
+
+ if ( cpu_has_xen_shstk || cpu_has_xen_ibt )
+ {
+ set_in_cr4(X86_CR4_CET);
+
#ifdef CONFIG_PV32
if ( opt_pv32 )
{
opt_pv32 = 0;
- printk(" - Disabling PV32 due to Shadow Stacks\n");
+ printk(" - Disabling PV32 due to CET\n");
}
#endif
}
@@ -1849,10 +1885,6 @@ void __init noreturn __start_xen(unsigned long mbi_p)
alternative_branches();
- /* Defer CR4.CET until alternatives have finished playing with CR0.WP */
- if ( cpu_has_xen_shstk )
- set_in_cr4(X86_CR4_CET);
-
/*
* NB: when running as a PV shim VCPUOP_up/down is wired to the shim
* physical cpu_add/remove functions, so launch the guest with only
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (69 preceding siblings ...)
2022-02-14 12:51 ` [PATCH v2 70/70] x86: Enable CET Indirect Branch Tracking Andrew Cooper
@ 2022-02-14 13:10 ` Andrew Cooper
2022-02-14 13:43 ` Jan Beulich
2022-02-17 10:01 ` [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 13:10 UTC (permalink / raw)
To: Xen-devel
Cc: Jan Beulich, Stefano Stabellini, Wei Liu, Julien Grall,
Roger Pau Monne, Juergen Gross, Daniel Smith,
Marek Marczykowski-Górecki
On 14/02/2022 12:50, Andrew Cooper wrote:
> CET Indirect Branch Tracking is a hardware feature designed to protect against
> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
> companion feature to CET Shadow Stacks added in Xen 4.14.
>
> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
> annotations of function pointer targets. Patches 61 thru 70 are the final
> enablement of CET-IBT.
>
> This series functions correctly with GCC 9 and later, although an experimental
> GCC patch is required to get more helpful typechecking at build time.
>
> Tested on a TigerLake NUC.
>
> CI pipelines:
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
> https://cirrus-ci.com/build/4962308362338304
>
> Major changes from v1:
> * Boilerplate for mechanical commits
> * UEFI runtime services unconditionally disable IBT
> * Comprehensive build time check for embedded endbr's
There's one thing I considered, and wanted to discuss.
I'm tempted to rename cf_check to cfi for the function annotation, as
it's shorter without reducing clarity.
Changing now (i.e. before I commit) is easy. Once committed, changing
is far harder.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-14 13:10 ` [PATCH v2 00/70] x86: Support for " Andrew Cooper
@ 2022-02-14 13:43 ` Jan Beulich
2022-02-14 14:15 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-14 13:43 UTC (permalink / raw)
To: Andrew Cooper
Cc: Stefano Stabellini, Wei Liu, Julien Grall, Roger Pau Monne,
Juergen Gross, Daniel Smith, Marek Marczykowski-Górecki,
Xen-devel
On 14.02.2022 14:10, Andrew Cooper wrote:
> On 14/02/2022 12:50, Andrew Cooper wrote:
>> CET Indirect Branch Tracking is a hardware feature designed to protect against
>> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
>> companion feature to CET Shadow Stacks added in Xen 4.14.
>>
>> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
>> annotations of function pointer targets. Patches 61 thru 70 are the final
>> enablement of CET-IBT.
>>
>> This series functions correctly with GCC 9 and later, although an experimental
>> GCC patch is required to get more helpful typechecking at build time.
>>
>> Tested on a TigerLake NUC.
>>
>> CI pipelines:
>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
>> https://cirrus-ci.com/build/4962308362338304
>>
>> Major changes from v1:
>> * Boilerplate for mechanical commits
>> * UEFI runtime services unconditionally disable IBT
>> * Comprehensive build time check for embedded endbr's
>
> There's one thing I considered, and wanted to discuss.
>
> I'm tempted to rename cf_check to cfi for the function annotation, as
> it's shorter without reducing clarity.
What would the 'i' stand for in this acronym? Irrespective of the answer
I'd like to point out the name collision with the CFI directives at
assembler level. This isn't necessarily an objection (I'm certainly for
shortening), but we want to avoid introducing confusion.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-14 13:43 ` Jan Beulich
@ 2022-02-14 14:15 ` Andrew Cooper
2022-02-14 14:38 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-14 14:15 UTC (permalink / raw)
To: Jan Beulich
Cc: Stefano Stabellini, Wei Liu, Julien Grall, Roger Pau Monne,
Juergen Gross, Daniel Smith, Marek Marczykowski-Górecki,
Xen-devel
On 14/02/2022 13:43, Jan Beulich wrote:
> On 14.02.2022 14:10, Andrew Cooper wrote:
>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>> CET Indirect Branch Tracking is a hardware feature designed to protect against
>>> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
>>> companion feature to CET Shadow Stacks added in Xen 4.14.
>>>
>>> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
>>> annotations of function pointer targets. Patches 61 thru 70 are the final
>>> enablement of CET-IBT.
>>>
>>> This series functions correctly with GCC 9 and later, although an experimental
>>> GCC patch is required to get more helpful typechecking at build time.
>>>
>>> Tested on a TigerLake NUC.
>>>
>>> CI pipelines:
>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
>>> https://cirrus-ci.com/build/4962308362338304
>>>
>>> Major changes from v1:
>>> * Boilerplate for mechanical commits
>>> * UEFI runtime services unconditionally disable IBT
>>> * Comprehensive build time check for embedded endbr's
>> There's one thing I considered, and wanted to discuss.
>>
>> I'm tempted to rename cf_check to cfi for the function annotation, as
>> it's shorter without reducing clarity.
> What would the 'i' stand for in this acronym?
The class of techniques is called Control Flow Integrity.
> Irrespective of the answer
> I'd like to point out the name collision with the CFI directives at
> assembler level. This isn't necessarily an objection (I'm certainly for
> shortening), but we want to avoid introducing confusion.
I doubt there is confusion to be had here. One is entirely a compiler
construct which turns into ENDBR64 instructions in the assembler, and
one is a general toolchain construct we explicitly disable.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-14 14:15 ` Andrew Cooper
@ 2022-02-14 14:38 ` Jan Beulich
2022-02-16 21:59 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-14 14:38 UTC (permalink / raw)
To: Andrew Cooper, Roger Pau Monne, Wei Liu
Cc: Stefano Stabellini, Julien Grall, Juergen Gross, Daniel Smith,
Marek Marczykowski-Górecki, Xen-devel
On 14.02.2022 15:15, Andrew Cooper wrote:
> On 14/02/2022 13:43, Jan Beulich wrote:
>> On 14.02.2022 14:10, Andrew Cooper wrote:
>>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>>> CET Indirect Branch Tracking is a hardware feature designed to protect against
>>>> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
>>>> companion feature to CET Shadow Stacks added in Xen 4.14.
>>>>
>>>> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
>>>> annotations of function pointer targets. Patches 61 thru 70 are the final
>>>> enablement of CET-IBT.
>>>>
>>>> This series functions correctly with GCC 9 and later, although an experimental
>>>> GCC patch is required to get more helpful typechecking at build time.
>>>>
>>>> Tested on a TigerLake NUC.
>>>>
>>>> CI pipelines:
>>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
>>>> https://cirrus-ci.com/build/4962308362338304
>>>>
>>>> Major changes from v1:
>>>> * Boilerplate for mechanical commits
>>>> * UEFI runtime services unconditionally disable IBT
>>>> * Comprehensive build time check for embedded endbr's
>>> There's one thing I considered, and wanted to discuss.
>>>
>>> I'm tempted to rename cf_check to cfi for the function annotation, as
>>> it's shorter without reducing clarity.
>> What would the 'i' stand for in this acronym?
>
> The class of techniques is called Control Flow Integrity.
>
>> Irrespective of the answer
>> I'd like to point out the name collision with the CFI directives at
>> assembler level. This isn't necessarily an objection (I'm certainly for
>> shortening), but we want to avoid introducing confusion.
>
> I doubt there is confusion to be had here. One is entirely a compiler
> construct which turns into ENDBR64 instructions in the assembler, and
> one is a general toolchain construct we explicitly disable.
Hmm. I'm still at best half convinced. Plus we generally have been
naming our shorthands after the actual attribute names. By using
"cfi" such a connection would also be largely lost. Roger, Wei,
others - do you opinions either way?
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-14 14:38 ` Jan Beulich
@ 2022-02-16 21:59 ` Andrew Cooper
2022-02-17 9:56 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-16 21:59 UTC (permalink / raw)
To: Jan Beulich, Roger Pau Monne, Wei Liu
Cc: Stefano Stabellini, Julien Grall, Juergen Gross, Daniel Smith,
Marek Marczykowski-Górecki, Xen-devel
On 14/02/2022 14:38, Jan Beulich wrote:
> On 14.02.2022 15:15, Andrew Cooper wrote:
>> On 14/02/2022 13:43, Jan Beulich wrote:
>>> On 14.02.2022 14:10, Andrew Cooper wrote:
>>>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>>>> CET Indirect Branch Tracking is a hardware feature designed to protect against
>>>>> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
>>>>> companion feature to CET Shadow Stacks added in Xen 4.14.
>>>>>
>>>>> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
>>>>> annotations of function pointer targets. Patches 61 thru 70 are the final
>>>>> enablement of CET-IBT.
>>>>>
>>>>> This series functions correctly with GCC 9 and later, although an experimental
>>>>> GCC patch is required to get more helpful typechecking at build time.
>>>>>
>>>>> Tested on a TigerLake NUC.
>>>>>
>>>>> CI pipelines:
>>>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
>>>>> https://cirrus-ci.com/build/4962308362338304
>>>>>
>>>>> Major changes from v1:
>>>>> * Boilerplate for mechanical commits
>>>>> * UEFI runtime services unconditionally disable IBT
>>>>> * Comprehensive build time check for embedded endbr's
>>>> There's one thing I considered, and wanted to discuss.
>>>>
>>>> I'm tempted to rename cf_check to cfi for the function annotation, as
>>>> it's shorter without reducing clarity.
>>> What would the 'i' stand for in this acronym?
>> The class of techniques is called Control Flow Integrity.
>>
>>> Irrespective of the answer
>>> I'd like to point out the name collision with the CFI directives at
>>> assembler level. This isn't necessarily an objection (I'm certainly for
>>> shortening), but we want to avoid introducing confusion.
>> I doubt there is confusion to be had here. One is entirely a compiler
>> construct which turns into ENDBR64 instructions in the assembler, and
>> one is a general toolchain construct we explicitly disable.
> Hmm. I'm still at best half convinced. Plus we generally have been
> naming our shorthands after the actual attribute names. By using
> "cfi" such a connection would also be largely lost. Roger, Wei,
> others - do you opinions either way?
My point is this. Doing nothing is my easiest option.
But if anyone has length/alternative suggestions, dealing with them now
is going to be infinitely easier than once this series is committed.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking
2022-02-16 21:59 ` Andrew Cooper
@ 2022-02-17 9:56 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 9:56 UTC (permalink / raw)
To: Andrew Cooper
Cc: Stefano Stabellini, Julien Grall, Juergen Gross, Daniel Smith,
Marek Marczykowski-Górecki, Xen-devel, Roger Pau Monne,
Wei Liu
On 16.02.2022 22:59, Andrew Cooper wrote:
> On 14/02/2022 14:38, Jan Beulich wrote:
>> On 14.02.2022 15:15, Andrew Cooper wrote:
>>> On 14/02/2022 13:43, Jan Beulich wrote:
>>>> On 14.02.2022 14:10, Andrew Cooper wrote:
>>>>> On 14/02/2022 12:50, Andrew Cooper wrote:
>>>>>> CET Indirect Branch Tracking is a hardware feature designed to protect against
>>>>>> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
>>>>>> companion feature to CET Shadow Stacks added in Xen 4.14.
>>>>>>
>>>>>> Patches 1 thru 5 are prerequisites. Patches 6 thru 60 are fairly mechanical
>>>>>> annotations of function pointer targets. Patches 61 thru 70 are the final
>>>>>> enablement of CET-IBT.
>>>>>>
>>>>>> This series functions correctly with GCC 9 and later, although an experimental
>>>>>> GCC patch is required to get more helpful typechecking at build time.
>>>>>>
>>>>>> Tested on a TigerLake NUC.
>>>>>>
>>>>>> CI pipelines:
>>>>>> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/470453652
>>>>>> https://cirrus-ci.com/build/4962308362338304
>>>>>>
>>>>>> Major changes from v1:
>>>>>> * Boilerplate for mechanical commits
>>>>>> * UEFI runtime services unconditionally disable IBT
>>>>>> * Comprehensive build time check for embedded endbr's
>>>>> There's one thing I considered, and wanted to discuss.
>>>>>
>>>>> I'm tempted to rename cf_check to cfi for the function annotation, as
>>>>> it's shorter without reducing clarity.
>>>> What would the 'i' stand for in this acronym?
>>> The class of techniques is called Control Flow Integrity.
>>>
>>>> Irrespective of the answer
>>>> I'd like to point out the name collision with the CFI directives at
>>>> assembler level. This isn't necessarily an objection (I'm certainly for
>>>> shortening), but we want to avoid introducing confusion.
>>> I doubt there is confusion to be had here. One is entirely a compiler
>>> construct which turns into ENDBR64 instructions in the assembler, and
>>> one is a general toolchain construct we explicitly disable.
>> Hmm. I'm still at best half convinced. Plus we generally have been
>> naming our shorthands after the actual attribute names. By using
>> "cfi" such a connection would also be largely lost. Roger, Wei,
>> others - do you opinions either way?
>
> My point is this. Doing nothing is my easiest option.
>
> But if anyone has length/alternative suggestions, dealing with them now
> is going to be infinitely easier than once this series is committed.
Understood. My personal preference then is to stick with cf_check.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-14 12:50 [PATCH v2 00/70] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (70 preceding siblings ...)
2022-02-14 13:10 ` [PATCH v2 00/70] x86: Support for " Andrew Cooper
@ 2022-02-17 10:01 ` Andrew Cooper
2022-02-17 10:42 ` Jan Beulich
71 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-17 10:01 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Scanning for embedded endbranch instructions involves parsing the .text
disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
treats it as instructions and tries to disassemble. Convert:
ffff82d040396108 <compatibility_mode_far>:
ffff82d040396108: 00 00 add %al,(%rax)
ffff82d04039610a: 00 00 add %al,(%rax)
ffff82d04039610c: 10 00 adc %al,(%rax)
ffff82d04039610e <compat_mode_gdt_desc>:
ffff82d04039610e: 17 (bad)
...
ffff82d040396118 <compat_mode_gdt>:
...
ffff82d040396120: ff (bad)
ffff82d040396121: ff 00 incl (%rax)
ffff82d040396123: 00 00 add %al,(%rax)
ffff82d040396125: 93 xchg %eax,%ebx
ffff82d040396126: cf iret
ffff82d040396127: 00 ff add %bh,%bh
ffff82d040396129: ff 00 incl (%rax)
ffff82d04039612b: 00 00 add %al,(%rax)
ffff82d04039612d: 9b fwait
ffff82d04039612e: cf iret
...
ffff82d040396130 <compat_mode_idt>:
...
ffff82d0403961b6 <kexec_reloc_size>:
ffff82d0403961b6: b6 01 mov $0x1,%dh
...
to:
ffff82d040396108 <compatibility_mode_far>:
ffff82d040396108: 00 00 00 00 10 00 ......
ffff82d04039610e <compat_mode_gdt_desc>:
ffff82d04039610e: 17 00 00 00 00 00 00 00 00 00 ..........
ffff82d040396118 <compat_mode_gdt>:
...
ffff82d040396120: ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................
ffff82d040396130 <compat_mode_idt>:
ffff82d040396130: 00 00 00 00 00 00 ......
ffff82d040396136 <reloc_stack>:
...
Most data just gains type and size metadata. The reloc_stack label is the
wrong end of the data block to have a size, so move it to the lowest address
and introduce .Lreloc_stack_base as a replacement.
While kexec_reloc_size could gain metadata, it's use in the linker
assertion (while correct) is deeply confusing to follow. Drop it entirely,
using a linker symbol instead to denote the end of the trampoline.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
The remainder of the 32bit code has mode-invariant lengths, so disassembles
safely as 64bit. The only differences come from 32/64bit implicit register
sizes.
v2.1:
* New
---
xen/arch/x86/include/asm/machine_kexec.h | 2 +-
xen/arch/x86/machine_kexec.c | 2 +-
xen/arch/x86/x86_64/kexec_reloc.S | 22 +++++++++++++++++-----
xen/arch/x86/xen.lds.S | 3 ++-
4 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/xen/arch/x86/include/asm/machine_kexec.h b/xen/arch/x86/include/asm/machine_kexec.h
index ba0d469d077b..d4880818c1d9 100644
--- a/xen/arch/x86/include/asm/machine_kexec.h
+++ b/xen/arch/x86/include/asm/machine_kexec.h
@@ -9,7 +9,7 @@ extern void kexec_reloc(unsigned long reloc_code, unsigned long reloc_pt,
unsigned long ind_maddr, unsigned long entry_maddr,
unsigned long flags);
-extern unsigned int kexec_reloc_size;
+extern const char kexec_reloc_end[];
#endif
diff --git a/xen/arch/x86/machine_kexec.c b/xen/arch/x86/machine_kexec.c
index 08ec9fd43b1d..751a9efcaf6a 100644
--- a/xen/arch/x86/machine_kexec.c
+++ b/xen/arch/x86/machine_kexec.c
@@ -117,7 +117,7 @@ int machine_kexec_load(struct kexec_image *image)
}
code_page = __map_domain_page(image->control_code_page);
- memcpy(code_page, kexec_reloc, kexec_reloc_size);
+ memcpy(code_page, kexec_reloc, kexec_reloc_end - (char *)kexec_reloc);
unmap_domain_page(code_page);
/*
diff --git a/xen/arch/x86/x86_64/kexec_reloc.S b/xen/arch/x86/x86_64/kexec_reloc.S
index d488d127cfb9..05bf8810cee6 100644
--- a/xen/arch/x86/x86_64/kexec_reloc.S
+++ b/xen/arch/x86/x86_64/kexec_reloc.S
@@ -34,7 +34,7 @@ ENTRY(kexec_reloc)
movq %rcx, %rbp
/* Setup stack. */
- leaq (reloc_stack - kexec_reloc)(%rdi), %rsp
+ leaq (.Lreloc_stack_base - kexec_reloc)(%rdi), %rsp
/* Load reloc page table. */
movq %rsi, %cr3
@@ -175,10 +175,16 @@ compatibility_mode_far:
.long 0x00000000 /* set in call_32_bit above */
.word 0x0010
+ .type compatibility_mode_far, @object
+ .size compatibility_mode_far, . - compatibility_mode_far
+
compat_mode_gdt_desc:
.word .Lcompat_mode_gdt_end - compat_mode_gdt -1
.quad 0x0000000000000000 /* set in call_32_bit above */
+ .type compat_mode_gdt_desc, @object
+ .size compat_mode_gdt_desc, . - compat_mode_gdt_desc
+
.align 8
compat_mode_gdt:
.quad 0x0000000000000000 /* null */
@@ -186,16 +192,22 @@ compat_mode_gdt:
.quad 0x00cf9b000000ffff /* 0x0010 ring 0 code, compatibility */
.Lcompat_mode_gdt_end:
+ .type compat_mode_gdt, @object
+ .size compat_mode_gdt, . - compat_mode_gdt
+
compat_mode_idt:
.word 0 /* limit */
.long 0 /* base */
+ .type compat_mode_idt, @object
+ .size compat_mode_idt, . - compat_mode_idt
+
/*
* 16 words of stack are more than enough.
*/
- .fill 16,8,0
reloc_stack:
+ .fill 16,8,0
+.Lreloc_stack_base:
- .globl kexec_reloc_size
-kexec_reloc_size:
- .long . - kexec_reloc
+ .type reloc_stack, @object
+ .size reloc_stack, . - reloc_stack
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index c399178ac123..13fc7ee008c1 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -87,6 +87,7 @@ SECTIONS
*(.text.unlikely)
*(.fixup)
*(.text.kexec)
+ kexec_reloc_end = .;
*(.gnu.warning)
_etext = .; /* End of text section */
} PHDR(text) = 0x9090
@@ -433,7 +434,7 @@ ASSERT(__2M_rwdata_end <= XEN_VIRT_END - XEN_VIRT_START + __XEN_VIRT_START -
"Xen image overlaps stubs area")
#ifdef CONFIG_KEXEC
-ASSERT(kexec_reloc_size - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
+ASSERT(kexec_reloc_end - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
#endif
/* The Multiboot setup paths relies on this to simplify superpage PTE creation. */
--
2.11.0
^ permalink raw reply related [flat|nested] 123+ messages in thread* Re: [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-17 10:01 ` [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
@ 2022-02-17 10:42 ` Jan Beulich
2022-02-17 12:06 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 10:42 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 17.02.2022 11:01, Andrew Cooper wrote:
> Scanning for embedded endbranch instructions involves parsing the .text
> disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
> treats it as instructions and tries to disassemble. Convert:
>
> ffff82d040396108 <compatibility_mode_far>:
What about the (possible) padding ahead of this? Should the .align
there perhaps specify a filler character?
> ffff82d040396108: 00 00 add %al,(%rax)
> ffff82d04039610a: 00 00 add %al,(%rax)
> ffff82d04039610c: 10 00 adc %al,(%rax)
>
> ffff82d04039610e <compat_mode_gdt_desc>:
> ffff82d04039610e: 17 (bad)
> ...
>
> ffff82d040396118 <compat_mode_gdt>:
> ...
> ffff82d040396120: ff (bad)
> ffff82d040396121: ff 00 incl (%rax)
> ffff82d040396123: 00 00 add %al,(%rax)
> ffff82d040396125: 93 xchg %eax,%ebx
> ffff82d040396126: cf iret
> ffff82d040396127: 00 ff add %bh,%bh
> ffff82d040396129: ff 00 incl (%rax)
> ffff82d04039612b: 00 00 add %al,(%rax)
> ffff82d04039612d: 9b fwait
> ffff82d04039612e: cf iret
> ...
>
> ffff82d040396130 <compat_mode_idt>:
> ...
>
> ffff82d0403961b6 <kexec_reloc_size>:
> ffff82d0403961b6: b6 01 mov $0x1,%dh
> ...
>
> to:
>
> ffff82d040396108 <compatibility_mode_far>:
> ffff82d040396108: 00 00 00 00 10 00 ......
>
> ffff82d04039610e <compat_mode_gdt_desc>:
> ffff82d04039610e: 17 00 00 00 00 00 00 00 00 00 ..........
>
> ffff82d040396118 <compat_mode_gdt>:
> ...
> ffff82d040396120: ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................
>
> ffff82d040396130 <compat_mode_idt>:
> ffff82d040396130: 00 00 00 00 00 00 ......
With the .size directives added, can we rely on consistent (past,
present, and future) objcopy behavior for padding gaps? It just so
happens that there's no 4-byte gap between compat_mode_gdt_desc and
compat_mode_gdt. Changing the .align ahead of compatibility_mode_far
would eliminate the risk of padding appearing if the code further up
changed.
> ffff82d040396136 <reloc_stack>:
> ...
Now this is particularly puzzling: Us setting %rsp to an unaligned
address is clearly not ABI-conforming. Since you're fiddling with
all of this already anyway, how about fixing this at the same time?
Of course there would then appear padding ahead of the stack, unless
the stack was moved up some.
> @@ -175,10 +175,16 @@ compatibility_mode_far:
> .long 0x00000000 /* set in call_32_bit above */
> .word 0x0010
>
> + .type compatibility_mode_far, @object
> + .size compatibility_mode_far, . - compatibility_mode_far
> +
> compat_mode_gdt_desc:
> .word .Lcompat_mode_gdt_end - compat_mode_gdt -1
> .quad 0x0000000000000000 /* set in call_32_bit above */
>
> + .type compat_mode_gdt_desc, @object
> + .size compat_mode_gdt_desc, . - compat_mode_gdt_desc
Side note: We really ought to gain something like OBJECT(name) to avoid
c'n'p mistakes not updating correctly all three symbol name instances.
> --- a/xen/arch/x86/xen.lds.S
> +++ b/xen/arch/x86/xen.lds.S
> @@ -87,6 +87,7 @@ SECTIONS
> *(.text.unlikely)
> *(.fixup)
> *(.text.kexec)
> + kexec_reloc_end = .;
Does this maybe want aligning on a 4- or even 8-byte boundary? If
so, imo preferably not here, but by adding a trailing .align in the
.S file.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-17 10:42 ` Jan Beulich
@ 2022-02-17 12:06 ` Andrew Cooper
2022-02-17 14:48 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-17 12:06 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 17/02/2022 10:42, Jan Beulich wrote:
> On 17.02.2022 11:01, Andrew Cooper wrote:
>> Scanning for embedded endbranch instructions involves parsing the .text
>> disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
>> treats it as instructions and tries to disassemble. Convert:
>>
>> ffff82d040396108 <compatibility_mode_far>:
> What about the (possible) padding ahead of this? Should the .align
> there perhaps specify a filler character?
What about it? It's just long nops like all other padding in .text
ffff82d040396101: ff d5 call *%ebp
ffff82d040396103: 0f 0b ud2
ffff82d040396105: 0f 1f 00 nopl (%eax)
ffff82d040396108 <compatibility_mode_far>:
ffff82d040396108: 00 00 00 00 10
00 ......
And for this problem, we don't need to care about the behaviour of any
pre-CET version of binutils.
>> ffff82d040396108: 00 00 add %al,(%rax)
>> ffff82d04039610a: 00 00 add %al,(%rax)
>> ffff82d04039610c: 10 00 adc %al,(%rax)
>>
>> ffff82d04039610e <compat_mode_gdt_desc>:
>> ffff82d04039610e: 17 (bad)
>> ...
>>
>> ffff82d040396118 <compat_mode_gdt>:
>> ...
>> ffff82d040396120: ff (bad)
>> ffff82d040396121: ff 00 incl (%rax)
>> ffff82d040396123: 00 00 add %al,(%rax)
>> ffff82d040396125: 93 xchg %eax,%ebx
>> ffff82d040396126: cf iret
>> ffff82d040396127: 00 ff add %bh,%bh
>> ffff82d040396129: ff 00 incl (%rax)
>> ffff82d04039612b: 00 00 add %al,(%rax)
>> ffff82d04039612d: 9b fwait
>> ffff82d04039612e: cf iret
>> ...
>>
>> ffff82d040396130 <compat_mode_idt>:
>> ...
>>
>> ffff82d0403961b6 <kexec_reloc_size>:
>> ffff82d0403961b6: b6 01 mov $0x1,%dh
>> ...
>>
>> to:
>>
>> ffff82d040396108 <compatibility_mode_far>:
>> ffff82d040396108: 00 00 00 00 10 00 ......
>>
>> ffff82d04039610e <compat_mode_gdt_desc>:
>> ffff82d04039610e: 17 00 00 00 00 00 00 00 00 00 ..........
>>
>> ffff82d040396118 <compat_mode_gdt>:
>> ...
>> ffff82d040396120: ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................
>>
>> ffff82d040396130 <compat_mode_idt>:
>> ffff82d040396130: 00 00 00 00 00 00 ......
> With the .size directives added, can we rely on consistent (past,
> present, and future) objcopy behavior for padding gaps?
Of course not. We don't know how things will develop in the future.
The best we can do is hope that it doesn't change too much.
But on that note, the way this would go wrong is the binary scan finding
an endbr that wasn't disassembled properly here, for whatever reason.
> It just so
> happens that there's no 4-byte gap between compat_mode_gdt_desc and
> compat_mode_gdt. Changing the .align ahead of compatibility_mode_far
> would eliminate the risk of padding appearing if the code further up
> changed.
Gaps will be formed of long nops because we're in .text, and they merge
with the previous data blob (see below).
>
>> ffff82d040396136 <reloc_stack>:
>> ...
> Now this is particularly puzzling: Us setting %rsp to an unaligned
> address is clearly not ABI-conforming. Since you're fiddling with
> all of this already anyway, how about fixing this at the same time?
> Of course there would then appear padding ahead of the stack, unless
> the stack was moved up some.
Oh - I'd not even noticed that. Luckily there is no ABI which matters,
because it's the call/push/pop's in this file alone.
With an align 8, we get:
ffff82d0403a7138 <compat_mode_idt>:
ffff82d0403a7138: 00 00 00 00 00 00 66
90 ......f.
ffff82d0403a7140 <reloc_stack>:
...
where the 66 90 in compat_mode_idt is the padding. Recall c/s 9fd181540c7e6
>
>> @@ -175,10 +175,16 @@ compatibility_mode_far:
>> .long 0x00000000 /* set in call_32_bit above */
>> .word 0x0010
>>
>> + .type compatibility_mode_far, @object
>> + .size compatibility_mode_far, . - compatibility_mode_far
>> +
>> compat_mode_gdt_desc:
>> .word .Lcompat_mode_gdt_end - compat_mode_gdt -1
>> .quad 0x0000000000000000 /* set in call_32_bit above */
>>
>> + .type compat_mode_gdt_desc, @object
>> + .size compat_mode_gdt_desc, . - compat_mode_gdt_desc
> Side note: We really ought to gain something like OBJECT(name) to avoid
> c'n'p mistakes not updating correctly all three symbol name instances.
I've got an intern working on it.
>
>> --- a/xen/arch/x86/xen.lds.S
>> +++ b/xen/arch/x86/xen.lds.S
>> @@ -87,6 +87,7 @@ SECTIONS
>> *(.text.unlikely)
>> *(.fixup)
>> *(.text.kexec)
>> + kexec_reloc_end = .;
> Does this maybe want aligning on a 4- or even 8-byte boundary? If
> so, imo preferably not here, but by adding a trailing .align in the
> .S file.
There's no special need for it to be aligned, and it is anyway as the
stack is the last object in it.
The sole user is the memcpy() size calculation moving the trampoline to
its destination page in the kexec reserved area.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-17 12:06 ` Andrew Cooper
@ 2022-02-17 14:48 ` Jan Beulich
2022-02-17 16:06 ` Andrew Cooper
0 siblings, 1 reply; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 14:48 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 17.02.2022 13:06, Andrew Cooper wrote:
> On 17/02/2022 10:42, Jan Beulich wrote:
>> On 17.02.2022 11:01, Andrew Cooper wrote:
>>> Scanning for embedded endbranch instructions involves parsing the .text
>>> disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
>>> treats it as instructions and tries to disassemble. Convert:
>>>
>>> ffff82d040396108 <compatibility_mode_far>:
>> What about the (possible) padding ahead of this? Should the .align
>> there perhaps specify a filler character?
>
> What about it? It's just long nops like all other padding in .text
>
> ffff82d040396101: ff d5 call *%ebp
> ffff82d040396103: 0f 0b ud2
> ffff82d040396105: 0f 1f 00 nopl (%eax)
>
> ffff82d040396108 <compatibility_mode_far>:
> ffff82d040396108: 00 00 00 00 10
> 00 ......
>
> And for this problem, we don't need to care about the behaviour of any
> pre-CET version of binutils.
I was about to ask, but yes - this is a good point.
Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>> ffff82d040396108: 00 00 add %al,(%rax)
>>> ffff82d04039610a: 00 00 add %al,(%rax)
>>> ffff82d04039610c: 10 00 adc %al,(%rax)
>>>
>>> ffff82d04039610e <compat_mode_gdt_desc>:
>>> ffff82d04039610e: 17 (bad)
>>> ...
>>>
>>> ffff82d040396118 <compat_mode_gdt>:
>>> ...
>>> ffff82d040396120: ff (bad)
>>> ffff82d040396121: ff 00 incl (%rax)
>>> ffff82d040396123: 00 00 add %al,(%rax)
>>> ffff82d040396125: 93 xchg %eax,%ebx
>>> ffff82d040396126: cf iret
>>> ffff82d040396127: 00 ff add %bh,%bh
>>> ffff82d040396129: ff 00 incl (%rax)
>>> ffff82d04039612b: 00 00 add %al,(%rax)
>>> ffff82d04039612d: 9b fwait
>>> ffff82d04039612e: cf iret
>>> ...
>>>
>>> ffff82d040396130 <compat_mode_idt>:
>>> ...
>>>
>>> ffff82d0403961b6 <kexec_reloc_size>:
>>> ffff82d0403961b6: b6 01 mov $0x1,%dh
>>> ...
>>>
>>> to:
>>>
>>> ffff82d040396108 <compatibility_mode_far>:
>>> ffff82d040396108: 00 00 00 00 10 00 ......
>>>
>>> ffff82d04039610e <compat_mode_gdt_desc>:
>>> ffff82d04039610e: 17 00 00 00 00 00 00 00 00 00 ..........
>>>
>>> ffff82d040396118 <compat_mode_gdt>:
>>> ...
>>> ffff82d040396120: ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................
>>>
>>> ffff82d040396130 <compat_mode_idt>:
>>> ffff82d040396130: 00 00 00 00 00 00 ......
>> With the .size directives added, can we rely on consistent (past,
>> present, and future) objcopy behavior for padding gaps?
>
> Of course not. We don't know how things will develop in the future.
> The best we can do is hope that it doesn't change too much.
>
> But on that note, the way this would go wrong is the binary scan finding
> an endbr that wasn't disassembled properly here, for whatever reason.
True; it'll "just" be a false positive build failure.
>> It just so
>> happens that there's no 4-byte gap between compat_mode_gdt_desc and
>> compat_mode_gdt. Changing the .align ahead of compatibility_mode_far
>> would eliminate the risk of padding appearing if the code further up
>> changed.
>
> Gaps will be formed of long nops because we're in .text, and they merge
> with the previous data blob (see below).
>
>>
>>> ffff82d040396136 <reloc_stack>:
>>> ...
>> Now this is particularly puzzling: Us setting %rsp to an unaligned
>> address is clearly not ABI-conforming. Since you're fiddling with
>> all of this already anyway, how about fixing this at the same time?
>> Of course there would then appear padding ahead of the stack, unless
>> the stack was moved up some.
>
> Oh - I'd not even noticed that. Luckily there is no ABI which matters,
> because it's the call/push/pop's in this file alone.
And the entity transitioned to is forbidden to make use of our stack?
> With an align 8, we get:
>
> ffff82d0403a7138 <compat_mode_idt>:
> ffff82d0403a7138: 00 00 00 00 00 00 66
> 90 ......f.
>
> ffff82d0403a7140 <reloc_stack>:
> ...
>
> where the 66 90 in compat_mode_idt is the padding. Recall c/s 9fd181540c7e6
>
>>> --- a/xen/arch/x86/xen.lds.S
>>> +++ b/xen/arch/x86/xen.lds.S
>>> @@ -87,6 +87,7 @@ SECTIONS
>>> *(.text.unlikely)
>>> *(.fixup)
>>> *(.text.kexec)
>>> + kexec_reloc_end = .;
>> Does this maybe want aligning on a 4- or even 8-byte boundary? If
>> so, imo preferably not here, but by adding a trailing .align in the
>> .S file.
>
> There's no special need for it to be aligned, and it is anyway as the
> stack is the last object in it.
You mean it anyway would be, if the stack was aligned? Or am I to imply
that you've amended the patch to add alignment there?
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-17 14:48 ` Jan Beulich
@ 2022-02-17 16:06 ` Andrew Cooper
2022-02-17 16:16 ` Jan Beulich
0 siblings, 1 reply; 123+ messages in thread
From: Andrew Cooper @ 2022-02-17 16:06 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 17/02/2022 14:48, Jan Beulich wrote:
> On 17.02.2022 13:06, Andrew Cooper wrote:
>> On 17/02/2022 10:42, Jan Beulich wrote:
>>> On 17.02.2022 11:01, Andrew Cooper wrote:
>>>> Scanning for embedded endbranch instructions involves parsing the .text
>>>> disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
>>>> treats it as instructions and tries to disassemble. Convert:
>>>>
>>>> ffff82d040396108 <compatibility_mode_far>:
>>> What about the (possible) padding ahead of this? Should the .align
>>> there perhaps specify a filler character?
>> What about it? It's just long nops like all other padding in .text
>>
>> ffff82d040396101: ff d5 call *%ebp
>> ffff82d040396103: 0f 0b ud2
>> ffff82d040396105: 0f 1f 00 nopl (%eax)
>>
>> ffff82d040396108 <compatibility_mode_far>:
>> ffff82d040396108: 00 00 00 00 10
>> 00 ......
>>
>> And for this problem, we don't need to care about the behaviour of any
>> pre-CET version of binutils.
> I was about to ask, but yes - this is a good point.
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks.
>>> It just so
>>> happens that there's no 4-byte gap between compat_mode_gdt_desc and
>>> compat_mode_gdt. Changing the .align ahead of compatibility_mode_far
>>> would eliminate the risk of padding appearing if the code further up
>>> changed.
>> Gaps will be formed of long nops because we're in .text, and they merge
>> with the previous data blob (see below).
>>
>>>> ffff82d040396136 <reloc_stack>:
>>>> ...
>>> Now this is particularly puzzling: Us setting %rsp to an unaligned
>>> address is clearly not ABI-conforming. Since you're fiddling with
>>> all of this already anyway, how about fixing this at the same time?
>>> Of course there would then appear padding ahead of the stack, unless
>>> the stack was moved up some.
>> Oh - I'd not even noticed that. Luckily there is no ABI which matters,
>> because it's the call/push/pop's in this file alone.
> And the entity transitioned to is forbidden to make use of our stack?
There's no expectation/guarantee of a good stack, no. Purgatory is a
very minimal environment before it sets something new up.
>> With an align 8, we get:
>>
>> ffff82d0403a7138 <compat_mode_idt>:
>> ffff82d0403a7138: 00 00 00 00 00 00 66
>> 90 ......f.
>>
>> ffff82d0403a7140 <reloc_stack>:
>> ...
>>
>> where the 66 90 in compat_mode_idt is the padding. Recall c/s 9fd181540c7e6
>>
>>>> --- a/xen/arch/x86/xen.lds.S
>>>> +++ b/xen/arch/x86/xen.lds.S
>>>> @@ -87,6 +87,7 @@ SECTIONS
>>>> *(.text.unlikely)
>>>> *(.fixup)
>>>> *(.text.kexec)
>>>> + kexec_reloc_end = .;
>>> Does this maybe want aligning on a 4- or even 8-byte boundary? If
>>> so, imo preferably not here, but by adding a trailing .align in the
>>> .S file.
>> There's no special need for it to be aligned, and it is anyway as the
>> stack is the last object in it.
> You mean it anyway would be, if the stack was aligned? Or am I to imply
> that you've amended the patch to add alignment there?
I have aligned reloc_stack stack because that's a no-brainer.
With that suitably aligned, kexec_reloc_end becomes aligned naturally
(because reloc_stack is the final object), and I don't think there's
much point putting anything explicit in the linker script.
It doesn't matter if subsequent things follow immediately, because this
trampoline is copied into the kexec region before being used. In
practice, the thing immediately following it is .init.text.
~Andrew
^ permalink raw reply [flat|nested] 123+ messages in thread
* Re: [PATCH v2.1 6.5/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-17 16:06 ` Andrew Cooper
@ 2022-02-17 16:16 ` Jan Beulich
0 siblings, 0 replies; 123+ messages in thread
From: Jan Beulich @ 2022-02-17 16:16 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 17.02.2022 17:06, Andrew Cooper wrote:
> On 17/02/2022 14:48, Jan Beulich wrote:
>> On 17.02.2022 13:06, Andrew Cooper wrote:
>>> On 17/02/2022 10:42, Jan Beulich wrote:
>>>> On 17.02.2022 11:01, Andrew Cooper wrote:
>>>>> --- a/xen/arch/x86/xen.lds.S
>>>>> +++ b/xen/arch/x86/xen.lds.S
>>>>> @@ -87,6 +87,7 @@ SECTIONS
>>>>> *(.text.unlikely)
>>>>> *(.fixup)
>>>>> *(.text.kexec)
>>>>> + kexec_reloc_end = .;
>>>> Does this maybe want aligning on a 4- or even 8-byte boundary? If
>>>> so, imo preferably not here, but by adding a trailing .align in the
>>>> .S file.
>>> There's no special need for it to be aligned, and it is anyway as the
>>> stack is the last object in it.
>> You mean it anyway would be, if the stack was aligned? Or am I to imply
>> that you've amended the patch to add alignment there?
>
> I have aligned reloc_stack stack because that's a no-brainer.
With this ...
> With that suitably aligned, kexec_reloc_end becomes aligned naturally
> (because reloc_stack is the final object), and I don't think there's
> much point putting anything explicit in the linker script.
... I certainly agree with this.
Jan
^ permalink raw reply [flat|nested] 123+ messages in thread