* [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 13:13 ` Julien Grall
2015-09-02 15:25 ` Ian Campbell
2015-07-27 11:11 ` [PATCH v5 02/22] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
` (20 subsequent siblings)
21 siblings, 2 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
dt_for_each_irq_map() returns error if no irq mapping is found.
With this patch, Ignore error and return success
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/common/device_tree.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 323c3be..1325ad5 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1085,7 +1085,7 @@ int dt_for_each_irq_map(const struct dt_device_node *dev,
if ( imap == NULL )
{
dt_dprintk(" -> no map, ignoring\n");
- goto fail;
+ return 0;
}
imaplen /= sizeof(u32);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping
2015-07-27 11:11 ` [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping vijay.kilari
@ 2015-07-28 13:13 ` Julien Grall
2015-07-28 13:23 ` Ian Campbell
2015-09-02 15:25 ` Ian Campbell
1 sibling, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-28 13:13 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
I think the title is too wide for the patch. I would rename to
"xen/dt: Handle correctly node with interrupt-map in dt_for_each_irq_map"
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> dt_for_each_irq_map() returns error if no irq mapping is found.
> With this patch, Ignore error and return success
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
I think this would be a bug fix for Xen 4.6. Can you send it separately?
> ---
> xen/common/device_tree.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 323c3be..1325ad5 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -1085,7 +1085,7 @@ int dt_for_each_irq_map(const struct dt_device_node *dev,
> if ( imap == NULL )
> {
> dt_dprintk(" -> no map, ignoring\n");
> - goto fail;
> + return 0;
> }
At the same time, can you also fix the comment above the if (i.e "/* No
interrupt map, check for an interrupt parent */") which is wrong?
> imaplen /= sizeof(u32);
>
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping
2015-07-28 13:13 ` Julien Grall
@ 2015-07-28 13:23 ` Ian Campbell
2015-07-28 13:27 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-07-28 13:23 UTC (permalink / raw)
To: Julien Grall, vijay.kilari, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
On Tue, 2015-07-28 at 14:13 +0100, Julien Grall wrote:
> Hi Vijay,
>
> I think the title is too wide for the patch. I would rename to
> "xen/dt: Handle correctly node with interrupt-map in dt_for_each_irq_map"
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> > From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >
> > dt_for_each_irq_map() returns error if no irq mapping is found.
> > With this patch, Ignore error and return success
> >
> > Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> I think this would be a bug fix for Xen 4.6. Can you send it separately?
I'm happy to be pointed at subsets of a series which are suitable to go in
now, no particular need to resend separately, assuming it can be applied in
isolation.
>
> > ---
> > xen/common/device_tree.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> > index 323c3be..1325ad5 100644
> > --- a/xen/common/device_tree.c
> > +++ b/xen/common/device_tree.c
> > @@ -1085,7 +1085,7 @@ int dt_for_each_irq_map(const struct
> > dt_device_node *dev,
> > if ( imap == NULL )
> > {
> > dt_dprintk(" -> no map, ignoring\n");
> > - goto fail;
> > + return 0;
> > }
>
> At the same time, can you also fix the comment above the if (i.e "/* No
> interrupt map, check for an interrupt parent */") which is wrong?
>
> > imaplen /= sizeof(u32);
> >
> >
>
> Regards,
>
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping
2015-07-28 13:23 ` Ian Campbell
@ 2015-07-28 13:27 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-28 13:27 UTC (permalink / raw)
To: Ian Campbell, vijay.kilari, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
On 28/07/15 14:23, Ian Campbell wrote:
> On Tue, 2015-07-28 at 14:13 +0100, Julien Grall wrote:
>> Hi Vijay,
>>
>> I think the title is too wide for the patch. I would rename to
>> "xen/dt: Handle correctly node with interrupt-map in dt_for_each_irq_map"
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> dt_for_each_irq_map() returns error if no irq mapping is found.
>>> With this patch, Ignore error and return success
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> I think this would be a bug fix for Xen 4.6. Can you send it separately?
>
> I'm happy to be pointed at subsets of a series which are suitable to go in
> now, no particular need to resend separately, assuming it can be applied in
> isolation.
Agree, although there is no need to wait that all the patch series is
resent to get a new version of this patch. So we can get it applied as
soon as possible.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping
2015-07-27 11:11 ` [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping vijay.kilari
2015-07-28 13:13 ` Julien Grall
@ 2015-09-02 15:25 ` Ian Campbell
1 sibling, 0 replies; 81+ messages in thread
From: Ian Campbell @ 2015-09-02 15:25 UTC (permalink / raw)
To: vijay.kilari, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
On Mon, 2015-07-27 at 16:41 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> dt_for_each_irq_map() returns error if no irq mapping is found.
> With this patch, Ignore error and return success
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Wei -- I think this fix would be good to have for 4.6. It is not an error
to process a zero-length (==non-existent) array I think.
Julien proposed retitling to "xen/dt: Handle correctly node with interrupt
-map in dt_for_each_irq_map", I think he meant s/with/without/ in that
though.
> ---
> xen/common/device_tree.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 323c3be..1325ad5 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -1085,7 +1085,7 @@ int dt_for_each_irq_map(const struct dt_device_node
> *dev,
> if ( imap == NULL )
> {
> dt_dprintk(" -> no map, ignoring\n");
> - goto fail;
> + return 0;
> }
> imaplen /= sizeof(u32);
>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 02/22] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
2015-07-27 11:11 ` [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-08-11 13:53 ` Jan Beulich
2015-07-27 11:11 ` [PATCH v5 03/22] xen: Add log2 functionality vijay.kilari
` (19 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
manish.jaggi, Ian Jackson, Jan Beulich, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
bitmap_find_next_zero_area helper function will be used
by physical ITS driver. This is imported from linux 4.2
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v5: Ported from Linux 4.2.
Added bitmap_find_next_zero_area_off().
v4: Removed spaces and added tabs
Moved ALIGN macro to lib.h
v3: Moved changes to xen/common/bitmap.c and
xen/include/xen/bitmap.h
---
xen/common/bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
xen/include/xen/bitmap.h | 16 ++++++++++++++++
xen/include/xen/lib.h | 2 ++
3 files changed, 57 insertions(+)
diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 61d1ea4..ad665d1 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -489,6 +489,45 @@ int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
}
EXPORT_SYMBOL(bitmap_allocate_region);
+/*
+ * bitmap_find_next_zero_area_off - find a contiguous aligned zero area
+ * @map: The address to base the search on
+ * @size: The bitmap size in bits
+ * @start: The bitnumber to start searching at
+ * @nr: The number of zeroed bits we're looking for
+ * @align_mask: Alignment mask for zero area
+ * @align_offset: Alignment offset for zero area.
+ *
+ * The @align_mask should be one less than a power of 2; the effect is that
+ * the bit offset of all zero areas this function finds plus @align_offset
+ * is multiple of that power of 2.
+ */
+unsigned long bitmap_find_next_zero_area_off(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask,
+ unsigned long align_offset)
+{
+ unsigned long index, end, i;
+again:
+ index = find_next_zero_bit(map, size, start);
+
+ /* Align allocation */
+ index = ALIGN_MASK(index + align_offset, align_mask) - align_offset;
+
+ end = index + nr;
+ if (end > size)
+ return end;
+ i = find_next_bit(map, end, index);
+ if (i < end) {
+ start = i + 1;
+ goto again;
+ }
+ return index;
+}
+EXPORT_SYMBOL(bitmap_find_next_zero_area_off)
+
#ifdef __BIG_ENDIAN
void bitmap_long_to_byte(uint8_t *bp, const unsigned long *lp, int nbits)
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index e2a3686..161f990 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -101,6 +101,22 @@ extern int bitmap_scnlistprintf(char *buf, unsigned int len,
extern int bitmap_find_free_region(unsigned long *bitmap, int bits, int order);
extern void bitmap_release_region(unsigned long *bitmap, int pos, int order);
extern int bitmap_allocate_region(unsigned long *bitmap, int pos, int order);
+extern unsigned long bitmap_find_next_zero_area_off(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask,
+ unsigned long align_offset);
+
+static inline unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask)
+{
+ return bitmap_find_next_zero_area_off(map, size, start, nr,
+ align_mask, 0);
+}
#define BITMAP_LAST_WORD_MASK(nbits) \
( \
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 4258912..e7d9d95 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -55,6 +55,8 @@
#define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
+
#define reserve_bootmem(_p,_l) ((void)0)
struct domain;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 02/22] xen/arm: Add bitmap_find_next_zero_area helper function
2015-07-27 11:11 ` [PATCH v5 02/22] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-08-11 13:53 ` Jan Beulich
0 siblings, 0 replies; 81+ messages in thread
From: Jan Beulich @ 2015-08-11 13:53 UTC (permalink / raw)
To: vijay.kilari
Cc: Keir Fraser, Ian Campbell, stefano.stabellini, Prasun.Kapoor, tim,
manish.jaggi, Ian Jackson, xen-devel, julien.grall,
stefano.stabellini, Vijaya Kumar K
>>> On 27.07.15 at 13:11, <vijay.kilari@gmail.com> wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> bitmap_find_next_zero_area helper function will be used
> by physical ITS driver. This is imported from linux 4.2
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 03/22] xen: Add log2 functionality
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
2015-07-27 11:11 ` [PATCH v5 01/22] xen/arm: Return success if dt node does not have irq mapping vijay.kilari
2015-07-27 11:11 ` [PATCH v5 02/22] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-27 11:11 ` [PATCH v5 04/22] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
` (18 subsequent siblings)
21 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Keir Fraser, Ian Campbell, vijay.kilari, Prasun.Kapoor,
manish.jaggi, Ian Jackson, Jan Beulich, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
log2 helper apis are ported from linux from
commit 13c07b0286d340275f2d97adf085cecda37ede37
(linux/log2.h: Fix rounddown_pow_of_two(1))
Changes made for xen are:
- Only required functionality is retained
- Replace fls_long with flsl
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Keir Fraser <keir@xen.org>
CC: Tim Deegan <tim@xen.org>
---
v4: - Only retained required functionality
- Replaced fls_long with flsl
- Removed fls_long implementation in bitops.h in v3 version
---
xen/include/xen/log2.h | 167 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 167 insertions(+)
diff --git a/xen/include/xen/log2.h b/xen/include/xen/log2.h
new file mode 100644
index 0000000..86bd861
--- /dev/null
+++ b/xen/include/xen/log2.h
@@ -0,0 +1,167 @@
+/*
+ * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _XEN_LOG2_H
+#define _XEN_LOG2_H
+
+#include <xen/types.h>
+#include <xen/bitops.h>
+
+/*
+ * deal with unrepresentable constant logarithms
+ */
+extern __attribute__((const))
+int ____ilog2_NaN(void);
+
+/*
+ * non-constant log of base 2 calculators
+ * - the arch may override these in asm/bitops.h if they can be implemented
+ * more efficiently than using fls() and fls64()
+ * - the arch is not required to handle n==0 if implementing the fallback
+ */
+static inline __attribute__((const))
+int __ilog2_u32(u32 n)
+{
+ return fls(n) - 1;
+}
+
+static inline __attribute__((const))
+int __ilog2_u64(u64 n)
+{
+ return flsl(n) - 1;
+}
+
+/*
+ * round up to nearest power of two
+ */
+static inline __attribute__((const))
+unsigned long __roundup_pow_of_two(unsigned long n)
+{
+ return 1UL << flsl(n - 1);
+}
+
+/**
+ * ilog2 - log of base 2 of 32-bit or a 64-bit unsigned value
+ * @n - parameter
+ *
+ * constant-capable log of base 2 calculation
+ * - this can be used to initialise global variables from constant data, hence
+ * the massive ternary operator construction
+ *
+ * selects the appropriately-sized optimised version depending on sizeof(n)
+ */
+#define ilog2(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n) < 1 ? ____ilog2_NaN() : \
+ (n) & (1ULL << 63) ? 63 : \
+ (n) & (1ULL << 62) ? 62 : \
+ (n) & (1ULL << 61) ? 61 : \
+ (n) & (1ULL << 60) ? 60 : \
+ (n) & (1ULL << 59) ? 59 : \
+ (n) & (1ULL << 58) ? 58 : \
+ (n) & (1ULL << 57) ? 57 : \
+ (n) & (1ULL << 56) ? 56 : \
+ (n) & (1ULL << 55) ? 55 : \
+ (n) & (1ULL << 54) ? 54 : \
+ (n) & (1ULL << 53) ? 53 : \
+ (n) & (1ULL << 52) ? 52 : \
+ (n) & (1ULL << 51) ? 51 : \
+ (n) & (1ULL << 50) ? 50 : \
+ (n) & (1ULL << 49) ? 49 : \
+ (n) & (1ULL << 48) ? 48 : \
+ (n) & (1ULL << 47) ? 47 : \
+ (n) & (1ULL << 46) ? 46 : \
+ (n) & (1ULL << 45) ? 45 : \
+ (n) & (1ULL << 44) ? 44 : \
+ (n) & (1ULL << 43) ? 43 : \
+ (n) & (1ULL << 42) ? 42 : \
+ (n) & (1ULL << 41) ? 41 : \
+ (n) & (1ULL << 40) ? 40 : \
+ (n) & (1ULL << 39) ? 39 : \
+ (n) & (1ULL << 38) ? 38 : \
+ (n) & (1ULL << 37) ? 37 : \
+ (n) & (1ULL << 36) ? 36 : \
+ (n) & (1ULL << 35) ? 35 : \
+ (n) & (1ULL << 34) ? 34 : \
+ (n) & (1ULL << 33) ? 33 : \
+ (n) & (1ULL << 32) ? 32 : \
+ (n) & (1ULL << 31) ? 31 : \
+ (n) & (1ULL << 30) ? 30 : \
+ (n) & (1ULL << 29) ? 29 : \
+ (n) & (1ULL << 28) ? 28 : \
+ (n) & (1ULL << 27) ? 27 : \
+ (n) & (1ULL << 26) ? 26 : \
+ (n) & (1ULL << 25) ? 25 : \
+ (n) & (1ULL << 24) ? 24 : \
+ (n) & (1ULL << 23) ? 23 : \
+ (n) & (1ULL << 22) ? 22 : \
+ (n) & (1ULL << 21) ? 21 : \
+ (n) & (1ULL << 20) ? 20 : \
+ (n) & (1ULL << 19) ? 19 : \
+ (n) & (1ULL << 18) ? 18 : \
+ (n) & (1ULL << 17) ? 17 : \
+ (n) & (1ULL << 16) ? 16 : \
+ (n) & (1ULL << 15) ? 15 : \
+ (n) & (1ULL << 14) ? 14 : \
+ (n) & (1ULL << 13) ? 13 : \
+ (n) & (1ULL << 12) ? 12 : \
+ (n) & (1ULL << 11) ? 11 : \
+ (n) & (1ULL << 10) ? 10 : \
+ (n) & (1ULL << 9) ? 9 : \
+ (n) & (1ULL << 8) ? 8 : \
+ (n) & (1ULL << 7) ? 7 : \
+ (n) & (1ULL << 6) ? 6 : \
+ (n) & (1ULL << 5) ? 5 : \
+ (n) & (1ULL << 4) ? 4 : \
+ (n) & (1ULL << 3) ? 3 : \
+ (n) & (1ULL << 2) ? 2 : \
+ (n) & (1ULL << 1) ? 1 : \
+ (n) & (1ULL << 0) ? 0 : \
+ ____ilog2_NaN() \
+ ) : \
+ (sizeof(n) <= 4) ? \
+ __ilog2_u32(n) : \
+ __ilog2_u64(n) \
+ )
+
+/**
+ * roundup_pow_of_two - round the given value up to nearest power of two
+ * @n - parameter
+ *
+ * round the given value up to the nearest power of two
+ * - the result is undefined when n == 0
+ * - this can be used to initialise global variables from constant data
+ */
+#define roundup_pow_of_two(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n == 1) ? 1 : \
+ (1UL << (ilog2((n) - 1) + 1)) \
+ ) : \
+ __roundup_pow_of_two(n) \
+ )
+
+/**
+ * order_base_2 - calculate the (rounded up) base 2 order of the argument
+ * @n: parameter
+ *
+ * The first few values calculated by this routine:
+ * ob2(0) = 0
+ * ob2(1) = 0
+ * ob2(2) = 1
+ * ob2(3) = 2
+ * ob2(4) = 2
+ * ob2(5) = 3
+ * ... and so on.
+ */
+
+#define order_base_2(n) ilog2(roundup_pow_of_two(n))
+#endif /* _XEN_LOG2_H */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v5 04/22] xen/arm: Set nr_cpu_ids to available number of cpus
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (2 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 03/22] xen: Add log2 functionality vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 13:21 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
` (17 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
nr_cpu_ids for arm platforms is set to 128 irrespective of
number of cpus supported by platform.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/setup.c | 1 +
xen/arch/arm/smpboot.c | 11 +++++++++++
xen/include/asm-arm/smp.h | 1 +
3 files changed, 13 insertions(+)
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index a46c583..6ca787b 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -772,6 +772,7 @@ void __init start_xen(unsigned long boot_phys_offset,
smp_init_cpus();
cpus = smp_get_max_cpus();
+ set_nr_cpu_ids(cpus);
init_xen_time();
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a96cda2..233ad95 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -243,6 +243,17 @@ void __init smp_init_cpus(void)
}
}
+void __init set_nr_cpu_ids(unsigned int max_cpus)
+{
+ if ( !max_cpus )
+ max_cpus = 1;
+ else if ( max_cpus > NR_CPUS )
+ max_cpus = NR_CPUS;
+
+ printk(XENLOG_INFO "SMP: Allowing %u CPUs\n", max_cpus);
+ nr_cpu_ids = max_cpus;
+}
+
int __init
smp_get_max_cpus (void)
{
diff --git a/xen/include/asm-arm/smp.h b/xen/include/asm-arm/smp.h
index 91b1e52..c23bd0f 100644
--- a/xen/include/asm-arm/smp.h
+++ b/xen/include/asm-arm/smp.h
@@ -29,6 +29,7 @@ extern void init_secondary(void);
extern void smp_init_cpus(void);
extern void smp_clear_cpu_maps (void);
extern int smp_get_max_cpus (void);
+extern void set_nr_cpu_ids(unsigned int max_cpus);
#endif
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 04/22] xen/arm: Set nr_cpu_ids to available number of cpus
2015-07-27 11:11 ` [PATCH v5 04/22] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
@ 2015-07-28 13:21 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-28 13:21 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> nr_cpu_ids for arm platforms is set to 128 irrespective of
NIT: s/128/NR_CPUS/
> number of cpus supported by platform.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/setup.c | 1 +
> xen/arch/arm/smpboot.c | 11 +++++++++++
> xen/include/asm-arm/smp.h | 1 +
> 3 files changed, 13 insertions(+)
>
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index a46c583..6ca787b 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -772,6 +772,7 @@ void __init start_xen(unsigned long boot_phys_offset,
>
> smp_init_cpus();
> cpus = smp_get_max_cpus();
> + set_nr_cpu_ids(cpus);
>
> init_xen_time();
>
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a96cda2..233ad95 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -243,6 +243,17 @@ void __init smp_init_cpus(void)
> }
> }
>
> +void __init set_nr_cpu_ids(unsigned int max_cpus)
> +{
> + if ( !max_cpus )
> + max_cpus = 1;
> + else if ( max_cpus > NR_CPUS )
> + max_cpus = NR_CPUS;
Those check are not useful. smp_get_max_cpus will always return a number
between 1 and NR_CPUS.
You could even open code the function in setup.c. I.e
cpus = smp_get_max_cpus();
printk(XENLOG_INFO "SMP: Allowing %u CPUs\n", cpus);
nr_cpus_ids = cpus;
> +
> + printk(XENLOG_INFO "SMP: Allowing %u CPUs\n", max_cpus);
> + nr_cpu_ids = max_cpus;
> +}
> +
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (3 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 04/22] xen/arm: Set nr_cpu_ids to available number of cpus vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 16:46 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 06/22] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
` (16 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
The linux driver is based on 4.1 with below commit id
3ad2a5f57656a14d964b673a5a0e4ab0e583c870
Only following code from Linux ITS driver is ported
and compiled
- LPI initialization
- ITS configuration code
- Physical command queue management
- ITS command building
Also redistributor information is split into rdist and
rdist_prop structures.
The rdist_prop struct holds the redistributor common
information for all re-distributor and rdist struct
holds the per-cpu specific information.
This per-cpu rdist is defined as global and shared with
physical ITS driver.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5:
- dump_cmd is called from its_flush_cmd
- Added its_lpi_alloc_chunks, lpi_free, its_send_inv, its_send_mapd,
its_send_mapvi to this patch as these functions are ported from
linux and more logical to be in this patch.
For now these functions are non-static. Will be made static
when used.
- Used this_cpu instead of per_cpu
- Moved GITS_* definitions to git-its.h
v4: Major changes
- Redistributor refactoring patch is merged
- Fixed comments from v3 related to coding style and
removing duplicate code.
- Target address is stored from bits[48:16] to avoid
shifting of target address while building ITS commands
- Removed non-static functions
- Removed usage of command builder functions
- Changed its_cmd_block union to include mix of bit and unsigned
variable types to define ITS command structure
v3:
- Only required changes from Linux ITS driver is ported
- Xen coding style is followed.
---
xen/arch/arm/gic-v3-its.c | 1014 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 15 +-
xen/include/asm-arm/gic-its.h | 275 ++++++++++
xen/include/asm-arm/gic.h | 1 +
xen/include/asm-arm/gic_v3_defs.h | 42 +-
5 files changed, 1340 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
new file mode 100644
index 0000000..ba4110f
--- /dev/null
+++ b/xen/arch/arm/gic-v3-its.c
@@ -0,0 +1,1014 @@
+/*
+ * Copyright (C) 2013, 2014 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Xen changes:
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (C) 2014, 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/config.h>
+#include <xen/bitops.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/list.h>
+#include <xen/sizes.h>
+#include <xen/vmap.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+#define its_print(lvl, fmt, ...) \
+ printk("GIC-ITS:" fmt, ## __VA_ARGS__)
+
+#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
+/* TODO: ratelimit for Xen messages */
+#define its_err_ratelimited(fmt, ...) \
+ its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
+
+#define its_dbg(fmt, ...) \
+ its_print(XENLOG_DEBUG, fmt, ## __VA_ARGS__)
+
+#define its_info(fmt, ...) \
+ its_print(XENLOG_INFO, fmt, ## __VA_ARGS__)
+
+#define its_warn(fmt, ...) \
+ its_print(XENLOG_WARNING, fmt, ## __VA_ARGS__)
+
+//#define DEBUG_GIC_ITS
+
+#ifdef DEBUG_GIC_ITS
+# define DPRINTK(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1 << 0)
+#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
+
+/*
+ * The ITS structure - contains most of the infrastructure, with the
+ * msi_controller, the command queue, the collections, and the list of
+ * devices writing to it.
+ */
+struct its_node {
+ spinlock_t lock;
+ struct list_head entry;
+ void __iomem *base;
+ paddr_t phys_base;
+ paddr_t phys_size;
+ its_cmd_block *cmd_base;
+ its_cmd_block *cmd_write;
+ void *tables[GITS_BASER_NR_REGS];
+ u32 order[GITS_BASER_NR_REGS];
+ struct its_collection *collections;
+ u64 flags;
+ u32 ite_size;
+ struct dt_device_node *dt_node;
+};
+
+#define ITS_ITT_ALIGN SZ_256
+
+static LIST_HEAD(its_nodes);
+static DEFINE_SPINLOCK(its_lock);
+static struct rdist_prop *gic_rdists;
+
+#define gic_data_rdist() (this_cpu(rdist))
+
+#ifdef DEBUG_GIC_ITS
+void dump_cmd(its_cmd_block *cmd)
+{
+ printk(XENLOG_DEBUG,
+ "ITS: CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#else
+void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
+#endif
+
+#define ITS_CMD_QUEUE_SZ SZ_64K
+#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
+
+static u64 its_cmd_ptr_to_offset(struct its_node *its, its_cmd_block *ptr)
+{
+ return (ptr - its->cmd_base) * sizeof(*ptr);
+}
+
+static int its_queue_full(struct its_node *its)
+{
+ int widx;
+ int ridx;
+
+ widx = its->cmd_write - its->cmd_base;
+ ridx = readl_relaxed(its->base + GITS_CREADR) / sizeof(its_cmd_block);
+
+ /* This is incredibly unlikely to happen, unless the ITS locks up. */
+ if ( ((widx + 1) % ITS_CMD_QUEUE_NR_ENTRIES) == ridx )
+ return 1;
+
+ return 0;
+}
+
+static its_cmd_block *its_allocate_entry(struct its_node *its)
+{
+ its_cmd_block *cmd;
+ u32 count = 1000000; /* 1s! */
+
+ while ( its_queue_full(its) )
+ {
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimited("ITS queue not draining\n");
+ return NULL;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+
+ cmd = its->cmd_write++;
+
+ /* Handle queue wrapping */
+ if (its->cmd_write == (its->cmd_base + ITS_CMD_QUEUE_NR_ENTRIES))
+ its->cmd_write = its->cmd_base;
+
+ return cmd;
+}
+
+static its_cmd_block *its_post_commands(struct its_node *its)
+{
+ u64 wr = its_cmd_ptr_to_offset(its, its->cmd_write);
+
+ writel_relaxed(wr, its->base + GITS_CWRITER);
+
+ return its->cmd_write;
+}
+
+static void its_flush_cmd(struct its_node *its, its_cmd_block *cmd)
+{
+ /*
+ * Make sure the commands written to memory are observable by
+ * the ITS.
+ */
+ if ( its->flags & ITS_FLAGS_CMDQ_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cmd, sizeof(*cmd));
+ else
+ dsb(ishst);
+
+ dump_cmd(cmd);
+}
+
+static void its_wait_for_range_completion(struct its_node *its,
+ its_cmd_block *from,
+ its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(its, from);
+ to_idx = its_cmd_ptr_to_offset(its, to);
+
+ while ( 1 )
+ {
+ rd_idx = readl_relaxed(its->base + GITS_CREADR);
+ if ( rd_idx >= to_idx || rd_idx < from_idx )
+ break;
+
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimited("ITS queue timeout\n");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(struct its_node *its,
+ its_cmd_block *src,
+ struct its_collection *sync_col)
+{
+ its_cmd_block *cmd, *sync_cmd, *next_cmd;
+ unsigned long flags;
+
+ BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
+
+ spin_lock_irqsave(&its->lock, flags);
+
+ cmd = its_allocate_entry(its);
+ if ( !cmd )
+ { /* We're soooooo screewed... */
+ its_err_ratelimited("ITS can't allocate, dropping command\n");
+ spin_unlock_irqrestore(&its->lock, flags);
+ return;
+ }
+
+ memcpy(cmd, src, sizeof(its_cmd_block));
+ its_flush_cmd(its, cmd);
+
+ if ( sync_col )
+ {
+ sync_cmd = its_allocate_entry(its);
+ if ( !sync_cmd )
+ {
+ its_err_ratelimited("ITS can't SYNC, skipping\n");
+ goto post;
+ }
+ sync_cmd->sync.cmd = GITS_CMD_SYNC;
+ sync_cmd->sync.ta = sync_col->target_address;
+
+ its_flush_cmd(its, sync_cmd);
+ }
+
+post:
+ next_cmd = its_post_commands(its);
+ spin_unlock_irqrestore(&its->lock, flags);
+
+ its_wait_for_range_completion(its, cmd, next_cmd);
+}
+
+void its_send_inv(struct its_device *dev, struct its_collection *col,
+ u32 event_id)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.inv.cmd = GITS_CMD_INV;
+ cmd.inv.devid = dev->device_id;
+ cmd.inv.event = event_id;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+void its_send_mapd(struct its_device *dev, int valid)
+{
+ its_cmd_block cmd;
+ unsigned long itt_addr;
+ u8 size;
+
+ size = max(ilog2(dev->nr_lpis), 1);
+ itt_addr = __pa(dev->itt_addr);
+ itt_addr = ROUNDUP(itt_addr, ITS_ITT_ALIGN);
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapd.cmd = GITS_CMD_MAPD;
+ cmd.mapd.devid = dev->device_id;
+ cmd.mapd.size = size - 1;
+ /*
+ * ITT address field of MAPD command holds bit[48:8] of
+ * itt address. Hence shift by 8.
+ */
+ cmd.mapd.itt = itt_addr >> 8;
+ cmd.mapd.valid = !!valid;
+
+ its_send_single_command(dev->its, &cmd, &dev->its->collections[0]);
+}
+
+static void its_send_mapc(struct its_node *its, struct its_collection *col,
+ int valid)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapc.cmd = GITS_CMD_MAPC;
+ cmd.mapc.col = col->col_id;
+ cmd.mapc.ta = col->target_address;
+ cmd.mapc.valid = !!valid;
+
+ its_send_single_command(its, &cmd, col);
+}
+
+void its_send_mapvi(struct its_device *dev, struct its_collection *col,
+ u32 phys_id, u32 event)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.mapvi.cmd = GITS_CMD_MAPVI;
+ cmd.mapvi.devid = dev->device_id;
+ cmd.mapvi.event = event;
+ cmd.mapvi.phy_id = phys_id;
+ cmd.mapvi.col = col->col_id;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+static void its_send_invall(struct its_node *its, struct its_collection *col)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.invall.cmd = GITS_CMD_INVALL;
+ cmd.invall.col = col->col_id;
+
+ its_send_single_command(its, &cmd, NULL);
+}
+
+void its_send_discard(struct its_device *dev, struct its_collection *col,
+ u32 event)
+{
+ its_cmd_block cmd;
+
+ memset(&cmd, 0x0, sizeof(its_cmd_block));
+ cmd.discard.devid = dev->device_id;
+ cmd.discard.event = event;
+
+ its_send_single_command(dev->its, &cmd, col);
+}
+
+/*
+ * How we allocate LPIs:
+ *
+ * The GIC has id_bits bits for interrupt identifiers. From there, we
+ * must subtract 8192 which are reserved for SGIs/PPIs/SPIs. Then, as
+ * we allocate LPIs by chunks of 32, we can shift the whole thing by 5
+ * bits to the right.
+ *
+ * This gives us (((1UL << id_bits) - 8192) >> 5) possible allocations.
+ */
+#define IRQS_PER_CHUNK_SHIFT 5
+#define IRQS_PER_CHUNK (1 << IRQS_PER_CHUNK_SHIFT)
+
+static unsigned long *lpi_bitmap;
+static u32 lpi_chunks;
+static DEFINE_SPINLOCK(lpi_lock);
+
+static int its_lpi_to_chunk(int lpi)
+{
+ return (lpi - 8192) >> IRQS_PER_CHUNK_SHIFT;
+}
+
+static int its_chunk_to_lpi(int chunk)
+{
+ return (chunk << IRQS_PER_CHUNK_SHIFT) + 8192;
+}
+
+static int its_lpi_init(u32 id_bits)
+{
+ lpi_chunks = its_lpi_to_chunk(1UL << id_bits);
+
+ lpi_bitmap = xzalloc_bytes(BITS_TO_LONGS(lpi_chunks) * sizeof(long));
+ if ( !lpi_bitmap )
+ {
+ lpi_chunks = 0;
+ return -ENOMEM;
+ }
+
+ its_info("ITS: Allocated %d chunks for LPIs\n", (int)lpi_chunks);
+
+ return 0;
+}
+
+unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
+{
+ unsigned long *bitmap = NULL;
+ int chunk_id, nr_chunks, nr_ids, i;
+
+ nr_chunks = DIV_ROUND_UP(nirqs, IRQS_PER_CHUNK);
+
+ spin_lock(&lpi_lock);
+
+ do {
+ chunk_id = bitmap_find_next_zero_area(lpi_bitmap, lpi_chunks,
+ 0, nr_chunks, 0);
+ if ( chunk_id < lpi_chunks )
+ break;
+
+ nr_chunks--;
+ } while ( nr_chunks > 0 );
+
+ if ( !nr_chunks )
+ goto out;
+
+ bitmap = xzalloc_bytes(BITS_TO_LONGS(nr_chunks * IRQS_PER_CHUNK) *
+ sizeof (long));
+ if ( !bitmap )
+ goto out;
+
+ for ( i = 0; i < nr_chunks; i++ )
+ set_bit(chunk_id + i, lpi_bitmap);
+
+ *base = its_chunk_to_lpi(chunk_id);
+ nr_ids = nr_chunks * IRQS_PER_CHUNK;
+
+ if ( nr_ids < nirqs )
+ {
+ xfree(bitmap);
+ bitmap = NULL;
+ }
+
+out:
+ spin_unlock(&lpi_lock);
+
+ return bitmap;
+}
+
+void its_lpi_free(struct its_device *dev)
+{
+ int lpi;
+
+ spin_lock(&lpi_lock);
+
+ for ( lpi = dev->lpi_base; lpi < (dev->lpi_base + dev->nr_lpis);
+ lpi += IRQS_PER_CHUNK )
+ {
+ int chunk = its_lpi_to_chunk(lpi);
+
+ if (chunk > lpi_chunks)
+ its_err("Bad LPI chunk %d\n", chunk);
+ if ( test_bit(chunk, lpi_bitmap) )
+ clear_bit(chunk, lpi_bitmap);
+ }
+
+ spin_unlock(&lpi_lock);
+
+ xfree(dev->lpi_map);
+}
+
+/*
+ * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
+ * deal with (one configuration byte per interrupt). PENDBASE has to
+ * be 64kB aligned (one bit per LPI, plus 8192 bits for SPI/PPI/SGI).
+ */
+#define LPI_PROPBASE_SZ SZ_64K
+#define LPI_PENDBASE_SZ (LPI_PROPBASE_SZ / 8 + SZ_1K)
+
+/*
+ * This is how many bits of ID we need, including the useless ones.
+ */
+#define LPI_NRBITS ilog2(LPI_PROPBASE_SZ + SZ_8K)
+
+static int __init its_alloc_lpi_tables(void)
+{
+ paddr_t paddr;
+
+ gic_rdists->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(LPI_PROPBASE_SZ), 0);
+
+ if ( !gic_rdists->prop_page )
+ {
+ its_err("Failed to allocate PROPBASE\n");
+ return -ENOMEM;
+ }
+
+ paddr = __pa(gic_rdists->prop_page);
+ its_info("GIC: using LPI property table @%pa\n", &paddr);
+
+ /* Set LPI priority and Group-1, but disabled */
+ memset(gic_rdists->prop_page,
+ GIC_PRI_IRQ | LPI_PROP_GROUP1,
+ LPI_PROPBASE_SZ);
+
+ /* Make sure the GIC will observe the written configuration */
+ clean_and_invalidate_dcache_va_range(gic_rdists->prop_page,
+ LPI_PROPBASE_SZ);
+
+ return 0;
+}
+
+static const char *its_base_type_string[] = {
+ [GITS_BASER_TYPE_DEVICE] = "Devices",
+ [GITS_BASER_TYPE_VCPU] = "Virtual CPUs",
+ [GITS_BASER_TYPE_CPU] = "Physical CPUs",
+ [GITS_BASER_TYPE_COLLECTION] = "Interrupt Collections",
+ [GITS_BASER_TYPE_RESERVED5] = "Reserved (5)",
+ [GITS_BASER_TYPE_RESERVED6] = "Reserved (6)",
+ [GITS_BASER_TYPE_RESERVED7] = "Reserved (7)",
+};
+
+static void its_free_tables(struct its_node *its)
+{
+ int i;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ if ( its->tables[i] )
+ {
+ free_xenheap_pages(its->tables[i], its->order[i]);
+ its->tables[i] = NULL;
+ its->order[i] = 0;
+ }
+ }
+}
+
+static int its_alloc_tables(struct its_node *its)
+{
+ int err;
+ int i;
+ int psz = SZ_64K;
+ u64 shr = GITS_BASER_InnerShareable;
+ u64 cache = GITS_BASER_WaWb;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
+ u64 type = GITS_BASER_TYPE(val);
+ u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
+ unsigned int order = get_order_from_bytes(psz);
+ int alloc_size;
+ u64 tmp;
+ void *base;
+
+ if ( type == GITS_BASER_TYPE_NONE )
+ continue;
+
+ /*
+ * Allocate as many entries as required to fit the
+ * range of device IDs that the ITS can grok... The ID
+ * space being incredibly sparse, this results in a
+ * massive waste of memory.
+ *
+ * For other tables, only allocate a single page.
+ */
+ if ( type == GITS_BASER_TYPE_DEVICE )
+ {
+ u64 typer = readq_relaxed(its->base + GITS_TYPER);
+ u32 ids = GITS_TYPER_DEVBITS(typer);
+
+ order = max(get_order_from_bytes((1UL << ids) * entry_size), order);
+ if (order >= MAX_ORDER)
+ {
+ order = MAX_ORDER - 1;
+ its_warn("Device Table too large,reduce its page order to %u\n",
+ order);
+ }
+ }
+
+ alloc_size = (1 << order) * PAGE_SIZE;
+ base = alloc_xenheap_pages(order, 0);
+ if ( !base )
+ {
+ err = -ENOMEM;
+ goto out_free;
+ }
+ memset(base, 0, alloc_size);
+ its->tables[i] = base;
+ its->order[i] = order;
+
+retry_baser:
+ val = (__pa(base) |
+ (type << GITS_BASER_TYPE_SHIFT) |
+ ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ cache |
+ shr |
+ GITS_BASER_VALID);
+
+ switch (psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ val |= (alloc_size / psz) - 1;
+
+ writeq_relaxed(val, its->base + GITS_BASER + i * 8);
+ tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
+
+ if ( (val ^ tmp) & GITS_BASER_SHAREABILITY_MASK )
+ {
+ /*
+ * Shareability didn't stick. Just use
+ * whatever the read reported, which is likely
+ * to be the only thing this redistributor
+ * supports.
+ */
+ shr = tmp & GITS_BASER_SHAREABILITY_MASK;
+ if ( !shr )
+ cache = GITS_BASER_nC;
+ goto retry_baser;
+ }
+
+ if ( (val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK )
+ {
+ /*
+ * Page size didn't stick. Let's try a smaller
+ * size and retry. If we reach 4K, then
+ * something is horribly wrong...
+ */
+ switch (psz) {
+ case SZ_16K:
+ psz = SZ_4K;
+ goto retry_baser;
+ case SZ_64K:
+ psz = SZ_16K;
+ goto retry_baser;
+ }
+ }
+
+ if ( val != tmp )
+ {
+ its_err("ITS: GITS_BASER%d doesn't stick: %lx %lx\n",
+ i, (unsigned long) val, (unsigned long) tmp);
+ err = -ENXIO;
+ goto out_free;
+ }
+
+ its_info("ITS: allocated %d %s @%lx (psz %dK, shr %d)\n",
+ (int)(alloc_size / entry_size),
+ its_base_type_string[type],
+ (unsigned long)__pa(base),
+ psz / SZ_1K, (int)shr >> GITS_BASER_SHAREABILITY_SHIFT);
+ }
+
+ return 0;
+
+out_free:
+ its_free_tables(its);
+
+ return err;
+}
+
+static int its_alloc_collections(struct its_node *its)
+{
+ its->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
+ if ( !its->collections )
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void its_cpu_init_lpis(void)
+{
+ void __iomem *rbase = gic_data_rdist().rbase;
+ void *pend_page;
+ u64 val, tmp;
+
+ /* If we didn't allocate the pending table yet, do it now */
+ pend_page = gic_data_rdist().pend_page;
+ if ( !pend_page )
+ {
+ paddr_t paddr;
+ u32 order;
+ /*
+ * The pending pages have to be at least 64kB aligned,
+ * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
+ */
+ order = get_order_from_bytes(max(LPI_PENDBASE_SZ, SZ_64K));
+ pend_page = alloc_xenheap_pages(order, 0);
+ if ( !pend_page )
+ {
+ its_err("Failed to allocate PENDBASE for CPU%d with order %d\n",
+ smp_processor_id(), order);
+ return;
+ }
+
+ memset(pend_page, 0, max(LPI_PENDBASE_SZ, SZ_64K));
+ /* Make sure the GIC will observe the zero-ed page */
+ clean_and_invalidate_dcache_va_range(pend_page, LPI_PENDBASE_SZ);
+
+ paddr = __pa(pend_page);
+
+ its_info("CPU%d: using LPI pending table @%pa\n",
+ smp_processor_id(), &paddr);
+
+ gic_data_rdist().pend_page = pend_page;
+ }
+
+ /* Disable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /*
+ * Make sure any change to the table is observable by the GIC.
+ */
+ dsb(sy);
+
+ /* set PROPBASE */
+ val = (__pa(gic_rdists->prop_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb |
+ ((LPI_NRBITS - 1) & GICR_PROPBASER_IDBITS_MASK));
+
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ tmp = readq_relaxed(rbase + GICR_PROPBASER);
+
+ if ( (tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK )
+ {
+ if ( !(tmp & GICR_PROPBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as well.
+ */
+ val &= ~(GICR_PROPBASER_SHAREABILITY_MASK |
+ GICR_PROPBASER_CACHEABILITY_MASK);
+ val |= GICR_PROPBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ }
+
+ its_info("GIC: using cache flushing for LPI property table\n");
+ gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
+ }
+
+ /* set PENDBASE */
+ val = (__pa(pend_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb);
+
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ tmp = readq_relaxed(rbase + GICR_PENDBASER);
+
+ if ( !(tmp & GICR_PENDBASER_SHAREABILITY_MASK) )
+ {
+ /*
+ * The HW reports non-shareable, we must remove the
+ * cacheability attributes as well.
+ */
+ val &= ~(GICR_PENDBASER_SHAREABILITY_MASK |
+ GICR_PENDBASER_CACHEABILITY_MASK);
+ val |= GICR_PENDBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ }
+
+ /* Enable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val |= GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /* Make sure the GIC has seen the above */
+ dsb(sy);
+}
+
+static void its_cpu_init_collection(void)
+{
+ struct its_node *its;
+ int cpu;
+
+ spin_lock(&its_lock);
+ cpu = smp_processor_id();
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ u64 target;
+
+ /*
+ * We now have to bind each collection to its target
+ * redistributor.
+ */
+ if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
+ {
+ target = gic_data_rdist().phys_base;
+ /* ITS command considers only [48:16] of the GICR address */
+ target >>= 16;
+ }
+ else
+ {
+ /*
+ * This ITS wants a linear CPU number.
+ */
+ target = readq_relaxed(gic_data_rdist().rbase + GICR_TYPER);
+ target = GICR_TYPER_CPU_NUMBER(target);
+ }
+
+ /* Perform collection mapping */
+ its->collections[cpu].col_id = cpu;
+ its->collections[cpu].target_address = target;
+
+ its_send_mapc(its, &its->collections[cpu], 1);
+ its_send_invall(its, &its->collections[cpu]);
+ }
+
+ spin_unlock(&its_lock);
+}
+
+static int its_force_quiescent(void __iomem *base)
+{
+ u32 count = 1000000; /* 1s */
+ u32 val;
+
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ /* Disable the generation of all interrupts to this ITS */
+ val &= ~GITS_CTLR_ENABLE;
+ writel_relaxed(val, base + GITS_CTLR);
+
+ /* Poll GITS_CTLR and wait until ITS becomes quiescent */
+ while ( 1 )
+ {
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ count--;
+ if ( !count )
+ return -EBUSY;
+
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static int its_probe(struct dt_device_node *node)
+{
+ paddr_t its_addr, its_size;
+ struct its_node *its;
+ void __iomem *its_base;
+ u32 val, typer;
+ u64 baser, tmp;
+ int err;
+
+ if ( !dt_get_property(node, "msi-controller", NULL) )
+ {
+ its_warn("%s: not a msi-controller\n", node->full_name);
+ return -ENXIO;
+ }
+
+ err = dt_device_get_address(node, 0, &its_addr, &its_size);
+ if ( err )
+ {
+ its_warn("%s: no regs?\n", node->full_name);
+ return -ENXIO;
+ }
+
+ its_base = ioremap_nocache(its_addr, its_size);
+ if ( !its_base )
+ {
+ its_warn("%s: unable to map registers\n", node->full_name);
+ return -ENOMEM;
+ }
+
+ val = readl_relaxed(its_base + GITS_PIDR2) & GIC_PIDR2_ARCH_REV_MASK;
+ if ( val != 0x30 && val != 0x40 )
+ {
+ its_warn("%s: no ITS detected, giving up\n", node->full_name);
+ err = -ENODEV;
+ goto out_unmap;
+ }
+
+ err = its_force_quiescent(its_base);
+ if ( err )
+ {
+ its_warn("%s: failed to quiesce, giving up\n",
+ node->full_name);
+ goto out_unmap;
+ }
+
+ its_info("ITS: %s\n", node->full_name);
+
+ its = xzalloc(struct its_node);
+ if ( !its )
+ {
+ err = -ENOMEM;
+ goto out_unmap;
+ }
+
+ spin_lock_init(&its->lock);
+ INIT_LIST_HEAD(&its->entry);
+ its->dt_node = node;
+ its->base = its_base;
+ its->phys_base = its_addr;
+ its->phys_size = its_size;
+ typer = readl_relaxed(its_base + GITS_TYPER);
+ its->ite_size = ((typer >> 4) & 0xf) + 1;
+
+ its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
+ if ( !its->cmd_base )
+ {
+ err = -ENOMEM;
+ goto out_free_its;
+ }
+ its->cmd_write = its->cmd_base;
+
+ err = its_alloc_tables(its);
+ if ( err )
+ goto out_free_cmd;
+
+ err = its_alloc_collections(its);
+ if ( err )
+ goto out_free_tables;
+
+ baser = (__pa(its->cmd_base) |
+ GITS_CBASER_WaWb |
+ GITS_CBASER_InnerShareable |
+ (ITS_CMD_QUEUE_SZ / SZ_4K - 1) |
+ GITS_CBASER_VALID);
+
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ tmp = readq_relaxed(its->base + GITS_CBASER);
+ if ( (tmp ^ baser) & GITS_CBASER_SHAREABILITY_MASK )
+ {
+ if (!(tmp & GITS_CBASER_SHAREABILITY_MASK))
+ {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as
+ * well.
+ */
+ baser &= ~(GITS_CBASER_SHAREABILITY_MASK |
+ GITS_CBASER_CACHEABILITY_MASK);
+ baser |= GITS_CBASER_nC;
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ }
+
+ its_info("ITS: using cache flushing for cmd queue\n");
+ its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
+ }
+
+ writeq_relaxed(0, its->base + GITS_CWRITER);
+ writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
+
+ spin_lock(&its_lock);
+ list_add(&its->entry, &its_nodes);
+ spin_unlock(&its_lock);
+
+ return 0;
+
+out_free_tables:
+ its_free_tables(its);
+out_free_cmd:
+ xfree(its->cmd_base);
+out_free_its:
+ xfree(its);
+out_unmap:
+ iounmap(its_base);
+ its_err("ITS: failed probing %s (%d)\n", node->full_name, err);
+ return err;
+}
+
+static bool gic_rdists_supports_plpis(void)
+{
+ return !!(readl_relaxed(gic_data_rdist().rbase + GICR_TYPER) & GICR_TYPER_PLPIS);
+}
+
+int its_cpu_init(void)
+{
+ if ( !list_empty(&its_nodes) )
+ {
+ if ( !gic_rdists_supports_plpis() )
+ {
+ its_info("CPU%d: LPIs not supported\n", smp_processor_id());
+ return -ENXIO;
+ }
+ its_cpu_init_lpis();
+ its_cpu_init_collection();
+ }
+
+ return 0;
+}
+
+int __init its_init(struct rdist_prop *rdists)
+{
+ struct dt_device_node *np = NULL;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for (np = dt_find_matching_node(NULL, its_device_ids); np;
+ np = dt_find_matching_node(np, its_device_ids))
+ its_probe(np);
+
+ if ( list_empty(&its_nodes) )
+ {
+ its_warn("ITS: No ITS available, not enabling LPIs\n");
+ return -ENXIO;
+ }
+
+ gic_rdists = rdists;
+ its_lpi_init(rdists->id_bits);
+ its_alloc_lpi_tables();
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 2033951..f2784a2 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -45,6 +45,7 @@
/* Global state */
static struct {
void __iomem *map_dbase; /* Mapped address of distributor registers */
+ struct rdist_prop rdist_data;
struct rdist_region *rdist_regions;
uint32_t rdist_stride;
unsigned int rdist_count; /* Number of rdist regions count */
@@ -55,10 +56,10 @@ static struct {
static struct gic_info gicv3_info;
/* per-cpu re-distributor base */
-static DEFINE_PER_CPU(void __iomem*, rbase);
+DEFINE_PER_CPU(struct rdist, rdist);
#define GICD (gicv3.map_dbase)
-#define GICD_RDIST_BASE (this_cpu(rbase))
+#define GICD_RDIST_BASE (this_cpu(rdist).rbase)
#define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
/*
@@ -618,6 +619,7 @@ static int __init gicv3_populate_rdist(void)
uint32_t aff;
uint32_t reg;
uint64_t typer;
+ uint64_t offset;
uint64_t mpidr = cpu_logical_map(smp_processor_id());
/*
@@ -653,9 +655,12 @@ static int __init gicv3_populate_rdist(void)
if ( (typer >> 32) == aff )
{
- this_cpu(rbase) = ptr;
- printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
- smp_processor_id(), i, ptr);
+ offset = ptr - gicv3.rdist_regions[i].map_base;
+ this_cpu(rdist).rbase = ptr;
+ this_cpu(rdist).phys_base = gicv3.rdist_regions[i].base + offset;
+ printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
+ smp_processor_id(), i,
+ this_cpu(rdist).phys_base);
return 0;
}
if ( gicv3.rdist_stride )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
new file mode 100644
index 0000000..5f44d5f
--- /dev/null
+++ b/xen/include/asm-arm/gic-its.h
@@ -0,0 +1,275 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_GIC_ITS_H__
+#define __ASM_ARM_GIC_ITS_H__
+
+#include <asm/gic_v3_defs.h>
+
+/*
+ * ITS registers, offsets from ITS_base
+ */
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER0 0x0100
+#define GITS_BASER1 0x0108
+#define GITS_BASER 0x0100
+#define GITS_BASERN 0x013c
+#define GITS_PIDR0 GICR_PIDR0
+#define GITS_PIDR1 GICR_PIDR1
+#define GITS_PIDR2 GICR_PIDR2
+#define GITS_PIDR3 GICR_PIDR3
+#define GITS_PIDR4 GICR_PIDR4
+#define GITS_PIDR5 GICR_PIDR5
+#define GITS_PIDR7 GICR_PIDR7
+
+#define GITS_TRANSLATER 0x10040
+#define GITS_CTLR_QUIESCENT (1U << 31)
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_DEVBITS(r) ((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_PTA (1UL << 19)
+#define GITS_TYPER_HCC_SHIFT (24)
+
+#define GITS_CBASER_VALID (1UL << 63)
+#define GITS_CBASER_nC (1UL << 59)
+#define GITS_CBASER_WaWb (5UL << 59)
+#define GITS_CBASER_InnerShareable (1UL << 10)
+#define GITS_CBASER_SHAREABILITY_MASK (3UL << 10)
+#define GITS_CBASER_CACHEABILITY_MASK (7UL << 59)
+
+#define GITS_BASER_NR_REGS 8
+
+#define GITS_BASER_VALID (1UL << 63)
+#define GITS_BASER_nC (1UL << 59)
+#define GITS_BASER_WaWb (5UL << 59)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE_MASK (0x7)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0xff) + 1)
+#define GITS_BASER_InnerShareable (1UL << 10)
+#define GITS_BASER_SHAREABILITY_SHIFT (10)
+#define GITS_BASER_SHAREABILITY_MASK (3UL << GITS_BASER_SHAREABILITY_SHIFT)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_VCPU 2
+#define GITS_BASER_TYPE_CPU 3
+#define GITS_BASER_TYPE_COLLECTION 4
+#define GITS_BASER_TYPE_RESERVED5 5
+#define GITS_BASER_TYPE_RESERVED6 6
+#define GITS_BASER_TYPE_RESERVED7 7
+
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPVI 0x0a
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
+#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PROP_GROUP1 (1 << 1)
+
+/*
+ * Collection structure - just an ID, and a redistributor address to
+ * ping. We use one per CPU as collection of interrupts assigned to this
+ * CPU.
+ */
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
+
+/* ITS command structure */
+typedef union {
+ u64 bits[4];
+ struct __packed {
+ uint8_t cmd;
+ uint8_t pad[7];
+ } hdr;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u64 size:5;
+ u64 res2:59;
+ /* XXX: Though itt is 40 bit. Keep it 48 to avoid shift */
+ u64 res3:8;
+ u64 itt:40;
+ u64 res4:15;
+ u64 valid:1;
+ u64 res5;
+ } mapd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u32 ta;
+ u16 res3:15;
+ u16 valid:1;
+ u64 res4;
+ } mapc;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } mapi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 phy_id;
+ u16 col;
+ u8 res2[6];
+ u64 res3;
+ } mapvi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } movi;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } discard;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } inv;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta1;
+ u16 res4;
+ u16 res5;
+ u32 ta2;
+ u16 res6;
+ } movall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 col;
+ u8 res3[6];
+ u64 res4;
+ } invall;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } int_cmd;
+ struct __packed {
+ u8 cmd;
+ u8 res1[3];
+ u32 devid;
+ u32 event;
+ u32 res2;
+ u64 res3;
+ u64 res4;
+ } clear;
+ struct __packed {
+ u8 cmd;
+ u8 res1[7];
+ u64 res2;
+ u16 res3;
+ u32 ta;
+ u16 res4;
+ u64 res5;
+ } sync;
+} its_cmd_block;
+
+/*
+ * The ITS view of a device.
+ */
+struct its_device {
+ /* Physical ITS */
+ struct its_node *its;
+ /* Device ITT address */
+ paddr_t *itt_addr;
+ /* Device ITT size */
+ unsigned long itt_size;
+ /* Physical LPI map */
+ unsigned long *lpi_map;
+ /* First Physical LPI number assigned */
+ u32 lpi_base;
+ /* Number of ITES/Physical LPIs assigned */
+ u32 nr_lpis;
+ /* Physical Device id */
+ u32 device_id;
+};
+
+int its_init(struct rdist_prop *rdists);
+int its_cpu_init(void);
+
+#endif /* __ASM_ARM_GIC_ITS_H__ */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index d343abf..e330fe3 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -161,6 +161,7 @@
DT_MATCH_COMPATIBLE("arm,gic-400")
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
+#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
#ifdef HAS_GICV3
/*
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index bf7b239..2c322da 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -48,6 +48,7 @@
/* Additional bits in GICD_TYPER defined by GICv3 */
#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -59,11 +60,12 @@
#define GICR_WAKER_ProcessorSleep (1U << 1)
#define GICR_WAKER_ChildrenAsleep (1U << 2)
-#define GICD_PIDR2_ARCH_REV_MASK (0xf0)
+#define GIC_PIDR2_ARCH_REV_MASK (0xf0)
+#define GICD_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICD_PIDR2_ARCH_REV_SHIFT (0x4)
#define GICD_PIDR2_ARCH_GICV3 (0x3)
-#define GICR_PIDR2_ARCH_REV_MASK GICD_PIDR2_ARCH_REV_MASK
+#define GICR_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICR_PIDR2_ARCH_REV_SHIFT GICD_PIDR2_ARCH_REV_SHIFT
#define GICR_PIDR2_ARCH_GICV3 GICD_PIDR2_ARCH_GICV3
@@ -113,10 +115,24 @@
#define GICR_ICFGR1 (0x0C04)
#define GICR_NSACR (0x0E00)
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff)
+
+#define GICR_PROPBASER_InnerShareable (1U << 10)
+#define GICR_PROPBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PROPBASER_nC (1U << 7)
+#define GICR_PROPBASER_WaWb (5U << 7)
+#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_PENDBASER_InnerShareable (1U << 10)
+#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PENDBASER_nC (1U << 7)
+#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
+
#define DEFAULT_PMR_VALUE 0xff
#define GICH_VMCR_EOI (1 << 9)
@@ -161,6 +177,28 @@ struct rdist_region {
void __iomem *map_base;
};
+/* Re-distributor per-cpu information */
+struct rdist {
+ /* CPU Re-distributor address */
+ void __iomem *rbase;
+ /* CPU LPI pending table */
+ void *pend_page;
+ /* CPU Re-distributor physical address */
+ paddr_t phys_base;
+};
+
+/* Common Re-distributor information */
+struct rdist_prop {
+ /* CPUs LPI configuration table */
+ void *prop_page;
+ /* Number of ID bits */
+ int id_bits;
+ /* Flags to store rdist attributes */
+ uint64_t flags;
+};
+
+DECLARE_PER_CPU(struct rdist, rdist);
+
#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-27 11:11 ` [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-07-28 16:46 ` Julien Grall
2015-07-29 15:22 ` Vijay Kilari
2015-07-31 10:28 ` Vijay Kilari
0 siblings, 2 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-28 16:46 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> The linux driver is based on 4.1 with below commit id
>
> 3ad2a5f57656a14d964b673a5a0e4ab0e583c870
This doesn't include commit 591e5bec13f15feb13fc445b6c9c59954711c4ac
"irqchip/gicv3-its: Fix mapping of LPIs to collections".
On the version 4 of this series, Ian [1] said that it would be very nice
to have a similar approach in Xen. I would like to see it too.
[..]
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..ba4110f
> --- /dev/null
> +++ b/xen/arch/arm/gic-v3-its.c
[..]
> +#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
> +/* TODO: ratelimit for Xen messages */
> +#define its_err_ratelimited(fmt, ...) \
> + its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
The macro its_err_ratelimited is mostly used in function that are
accessible by the guest (via enable/disable LPI...). Which means that a
guest could theoretically hit the problem and DOS xen.
I would use XENLOG_G_ERR to have a rate limited until we fix it
correctly for XENLOG_ERR.
[..]
> +#ifdef DEBUG_GIC_ITS
> +void dump_cmd(its_cmd_block *cmd)
static void
and const its_cmd_block *cmd
> +{
> + printk(XENLOG_DEBUG,
> + "ITS: CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
> +}
> +#else
> +void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
ditto for static and const
Also, the do {} while 0 is not necessary given you are using function
and not macro.
> +#endif
> +void its_send_inv(struct its_device *dev, struct its_collection *col,
> + u32 event_id)
On a follow-up patch (see #15) you change the prototype to be static.
The static should be set now and not deferring until you use it just
because of compilation issue. This is a call to reorder the patches or
split them.
Also, if you include the commit I mentioned at the beginning of the
mail, you won't need to pass its_collection *col.
It would be more cleaner for the caller and defer the choice of the
collection within the function as Linux does.
[..]
> +void its_send_mapd(struct its_device *dev, int valid)
static void ...
> +{
> + its_cmd_block cmd;
> + unsigned long itt_addr;
> + u8 size;
> +
> + size = max(ilog2(dev->nr_lpis), 1);
Why do you need the max? dev->nr_lpis should always contains a valid value.
[..]
> +void its_send_mapvi(struct its_device *dev, struct its_collection *col,
> + u32 phys_id, u32 event)
All my remark on its_send_inv applies here too.
[..]
> +void its_send_discard(struct its_device *dev, struct its_collection *col,
> + u32 event)
All my remarks on its_send_inv applies here too.
[..]
> +unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
static unsigned long ...
[..]
> +void its_lpi_free(struct its_device *dev)
static void ...
[..]
> +static void its_cpu_init_lpis(void)
> +{
> + void __iomem *rbase = gic_data_rdist().rbase;
> + void *pend_page;
> + u64 val, tmp;
> +
> + /* If we didn't allocate the pending table yet, do it now */
> + pend_page = gic_data_rdist().pend_page;
> + if ( !pend_page )
> + {
> + paddr_t paddr;
> + u32 order;
NIT: Newline here please.
> + /*
> + * The pending pages have to be at least 64kB aligned,
> + * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
> + */
[..]
> +int __init its_init(struct rdist_prop *rdists)
> +{
> + struct dt_device_node *np = NULL;
> +
> + static const struct dt_device_match its_device_ids[] __initconst =
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> +
> + for (np = dt_find_matching_node(NULL, its_device_ids); np;
> + np = dt_find_matching_node(np, its_device_ids))
The indentation looks wrong here.
> + its_probe(np);
> +
> + if ( list_empty(&its_nodes) )
> + {
> + its_warn("ITS: No ITS available, not enabling LPIs\n");
> + return -ENXIO;
> + }
> +
> + gic_rdists = rdists;
> + its_lpi_init(rdists->id_bits);
> + its_alloc_lpi_tables();
I don't see much reason to change the order compare to Linux. Please
do
its_alloc_lpi_tables();
its_its_lpi_init(rdists->id_bits);
Regards,
[1] http://lists.xen.org/archives/html/xen-devel/2015-07/msg03369.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-28 16:46 ` Julien Grall
@ 2015-07-29 15:22 ` Vijay Kilari
2015-07-29 16:06 ` Ian Campbell
2015-07-31 10:28 ` Vijay Kilari
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-29 15:22 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Julien,
Can you please explain what is the problem with making a function
non-static for compilation purpose and later make it static when used?
In anycase we are going to merge all the patches at once.
Regards
Vijay
On Tue, Jul 28, 2015 at 10:16 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> The linux driver is based on 4.1 with below commit id
>>
>> 3ad2a5f57656a14d964b673a5a0e4ab0e583c870
>
> This doesn't include commit 591e5bec13f15feb13fc445b6c9c59954711c4ac
> "irqchip/gicv3-its: Fix mapping of LPIs to collections".
>
> On the version 4 of this series, Ian [1] said that it would be very nice
> to have a similar approach in Xen. I would like to see it too.
>
> [..]
>
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> new file mode 100644
>> index 0000000..ba4110f
>> --- /dev/null
>> +++ b/xen/arch/arm/gic-v3-its.c
>
> [..]
>
>> +#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
>> +/* TODO: ratelimit for Xen messages */
>> +#define its_err_ratelimited(fmt, ...) \
>> + its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
>
> The macro its_err_ratelimited is mostly used in function that are
> accessible by the guest (via enable/disable LPI...). Which means that a
> guest could theoretically hit the problem and DOS xen.
>
> I would use XENLOG_G_ERR to have a rate limited until we fix it
> correctly for XENLOG_ERR.
>
> [..]
>
>> +#ifdef DEBUG_GIC_ITS
>> +void dump_cmd(its_cmd_block *cmd)
>
> static void
>
> and const its_cmd_block *cmd
>
>> +{
>> + printk(XENLOG_DEBUG,
>> + "ITS: CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
>> + cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
>> +}
>> +#else
>> +void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
>
> ditto for static and const
>
> Also, the do {} while 0 is not necessary given you are using function
> and not macro.
>
>> +#endif
>
>> +void its_send_inv(struct its_device *dev, struct its_collection *col,
>> + u32 event_id)
>
> On a follow-up patch (see #15) you change the prototype to be static.
> The static should be set now and not deferring until you use it just
> because of compilation issue. This is a call to reorder the patches or
> split them.
>
> Also, if you include the commit I mentioned at the beginning of the
> mail, you won't need to pass its_collection *col.
>
> It would be more cleaner for the caller and defer the choice of the
> collection within the function as Linux does.
>
> [..]
>
>> +void its_send_mapd(struct its_device *dev, int valid)
>
> static void ...
>
>> +{
>> + its_cmd_block cmd;
>> + unsigned long itt_addr;
>> + u8 size;
>> +
>> + size = max(ilog2(dev->nr_lpis), 1);
>
> Why do you need the max? dev->nr_lpis should always contains a valid value.
>
> [..]
>
>> +void its_send_mapvi(struct its_device *dev, struct its_collection *col,
>> + u32 phys_id, u32 event)
>
> All my remark on its_send_inv applies here too.
>
> [..]
>
>> +void its_send_discard(struct its_device *dev, struct its_collection *col,
>> + u32 event)
>
> All my remarks on its_send_inv applies here too.
>
> [..]
>
>> +unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
>
> static unsigned long ...
>
> [..]
>
>> +void its_lpi_free(struct its_device *dev)
>
> static void ...
>
> [..]
>
>> +static void its_cpu_init_lpis(void)
>> +{
>> + void __iomem *rbase = gic_data_rdist().rbase;
>> + void *pend_page;
>> + u64 val, tmp;
>> +
>> + /* If we didn't allocate the pending table yet, do it now */
>> + pend_page = gic_data_rdist().pend_page;
>> + if ( !pend_page )
>> + {
>> + paddr_t paddr;
>> + u32 order;
>
> NIT: Newline here please.
>
>> + /*
>> + * The pending pages have to be at least 64kB aligned,
>> + * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
>> + */
>
> [..]
>
>> +int __init its_init(struct rdist_prop *rdists)
>> +{
>> + struct dt_device_node *np = NULL;
>> +
>> + static const struct dt_device_match its_device_ids[] __initconst =
>> + {
>> + DT_MATCH_GIC_ITS,
>> + { /* sentinel */ },
>> + };
>> +
>> + for (np = dt_find_matching_node(NULL, its_device_ids); np;
>> + np = dt_find_matching_node(np, its_device_ids))
>
> The indentation looks wrong here.
>
>> + its_probe(np);
>> +
>> + if ( list_empty(&its_nodes) )
>> + {
>> + its_warn("ITS: No ITS available, not enabling LPIs\n");
>> + return -ENXIO;
>> + }
>> +
>> + gic_rdists = rdists;
>> + its_lpi_init(rdists->id_bits);
>> + its_alloc_lpi_tables();
>
> I don't see much reason to change the order compare to Linux. Please
> do
>
> its_alloc_lpi_tables();
> its_its_lpi_init(rdists->id_bits);
>
> Regards,
>
> [1] http://lists.xen.org/archives/html/xen-devel/2015-07/msg03369.html
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-29 15:22 ` Vijay Kilari
@ 2015-07-29 16:06 ` Ian Campbell
2015-07-29 16:18 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-07-29 16:06 UTC (permalink / raw)
To: Vijay Kilari, Julien Grall
Cc: Stefano Stabellini, Prasun Kapoor, manish.jaggi, Tim Deegan,
xen-devel@lists.xen.org, Stefano Stabellini, Vijaya Kumar K
On Wed, 2015-07-29 at 20:52 +0530, Vijay Kilari wrote:
> Hi Julien,
>
> Can you please explain what is the problem with making a function
> non-static for compilation purpose and later make it static when used?
It's noise in the series, which makes it harder to review and it is noise
in the history which makes it hard to follow what happened.
You should structure the series so that each patch individually makes sense
and builds upon the previous patches. Introducing things only to remove
them later, or making the non-static just to keep the compiler happy until
they are used is a sign that your series is badly organised, which you
should fix.
> In anycase we are going to merge all the patches at once.
I'm afraid that doesn't matter, each point in the series should standalone.
http://wiki.xen.org/wiki/Submitting_Xen_Patches#Making_good_patches covers
some of this.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-29 16:06 ` Ian Campbell
@ 2015-07-29 16:18 ` Vijay Kilari
0 siblings, 0 replies; 81+ messages in thread
From: Vijay Kilari @ 2015-07-29 16:18 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, manish.jaggi, Tim Deegan,
xen-devel@lists.xen.org, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
On Wed, Jul 29, 2015 at 9:36 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Wed, 2015-07-29 at 20:52 +0530, Vijay Kilari wrote:
>> Hi Julien,
>>
>> Can you please explain what is the problem with making a function
>> non-static for compilation purpose and later make it static when used?
>
> It's noise in the series, which makes it harder to review and it is noise
> in the history which makes it hard to follow what happened.
>
> You should structure the series so that each patch individually makes sense
> and builds upon the previous patches. Introducing things only to remove
> them later, or making the non-static just to keep the compiler happy until
> they are used is a sign that your series is badly organised, which you
> should fix.
I understand, it is tough to manage 4000+ lines of code.
I will try my best in my next series. but I _cannot_ assure 100% ;-)
>
>> In anycase we are going to merge all the patches at once.
>
> I'm afraid that doesn't matter, each point in the series should standalone.
>
> http://wiki.xen.org/wiki/Submitting_Xen_Patches#Making_good_patches covers
> some of this.
>
> Ian.
>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-28 16:46 ` Julien Grall
2015-07-29 15:22 ` Vijay Kilari
@ 2015-07-31 10:28 ` Vijay Kilari
2015-07-31 11:10 ` Julien Grall
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 10:28 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Tue, Jul 28, 2015 at 10:16 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> The linux driver is based on 4.1 with below commit id
>>
>> 3ad2a5f57656a14d964b673a5a0e4ab0e583c870
>
> This doesn't include commit 591e5bec13f15feb13fc445b6c9c59954711c4ac
> "irqchip/gicv3-its: Fix mapping of LPIs to collections".
>
> On the version 4 of this series, Ian [1] said that it would be very nice
> to have a similar approach in Xen. I would like to see it too.
By storing col_id in msi_desc as below, I am already taking care of
the above commit
in this series
struct msi_desc {
#ifdef HAS_GICV3
unsigned int eventID;
struct its_device *dev;
u16 col_id;
#endif
};
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen
2015-07-31 10:28 ` Vijay Kilari
@ 2015-07-31 11:10 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-31 11:10 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 31/07/15 11:28, Vijay Kilari wrote:
> On Tue, Jul 28, 2015 at 10:16 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> The linux driver is based on 4.1 with below commit id
>>>
>>> 3ad2a5f57656a14d964b673a5a0e4ab0e583c870
>>
>> This doesn't include commit 591e5bec13f15feb13fc445b6c9c59954711c4ac
>> "irqchip/gicv3-its: Fix mapping of LPIs to collections".
>>
>> On the version 4 of this series, Ian [1] said that it would be very nice
>> to have a similar approach in Xen. I would like to see it too.
>
> By storing col_id in msi_desc as below, I am already taking care of
> the above commit
> in this series
>
> struct msi_desc {
> #ifdef HAS_GICV3
> unsigned int eventID;
> struct its_device *dev;
> u16 col_id;
> #endif
> };
Which requires the caller of the ITS command to explicitly pass the
collection ID...
It complicates the caller for nothing and requires you to have
explicitely irqdesc helpers.
Anyway, I don't see any reason to differ from Linux and AFAICT Ian was
also in favor of using the same approach as Linux (i.e the one in the
commit "irqchip/gicv3-its: Fix mapping of LPIs to collections").
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 06/22] xen/arm: ITS: Add helper functions to manage its_devices
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (4 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 05/22] xen/arm: ITS: Port ITS driver to Xen vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-27 11:11 ` [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver vijay.kilari
` (15 subsequent siblings)
21 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper functions to manage its devices using RB-tree
are introduced in physical ITS driver.
This is global list of all the devices.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
---
v5: - Added assert on spinlock
v4: - Remove passing of root node as parameter
- Declare prototype in header file
- Rename find_its_device to its_find_device
---
xen/arch/arm/gic-v3-its.c | 53 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 3 +++
2 files changed, 56 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index ba4110f..aa4d3c5 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -93,6 +93,8 @@ struct its_node {
static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
+static struct rb_root rb_its_dev;
+static DEFINE_SPINLOCK(rb_its_dev_lock);
#define gic_data_rdist() (this_cpu(rdist))
@@ -107,6 +109,55 @@ void dump_cmd(its_cmd_block *cmd)
void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
#endif
+/* RB-tree helpers for its_device */
+struct its_device *its_find_device(u32 devid)
+{
+ struct rb_node *node = rb_its_dev.rb_node;
+
+ ASSERT(spin_is_locked(&rb_its_dev_lock));
+ while ( node )
+ {
+ struct its_device *dev;
+
+ dev = container_of(node, struct its_device, node);
+ if ( devid < dev->device_id )
+ node = node->rb_left;
+ else if ( devid > dev->device_id )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+int its_insert_device(struct its_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ ASSERT(spin_is_locked(&rb_its_dev_lock));
+ new = &rb_its_dev.rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct its_device *this;
+
+ this = container_of(*new, struct its_device, node);
+ parent = *new;
+ if ( dev->device_id < this->device_id )
+ new = &((*new)->rb_left);
+ else if ( dev->device_id > this->device_id )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, &rb_its_dev);
+
+ return 0;
+}
+
#define ITS_CMD_QUEUE_SZ SZ_64K
#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
@@ -942,6 +993,8 @@ static int its_probe(struct dt_device_node *node)
list_add(&its->entry, &its_nodes);
spin_unlock(&its_lock);
+ rb_its_dev = RB_ROOT;
+
return 0;
out_free_tables:
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 5f44d5f..3daba4b 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -19,6 +19,7 @@
#define __ASM_ARM_GIC_ITS_H__
#include <asm/gic_v3_defs.h>
+#include <xen/rbtree.h>
/*
* ITS registers, offsets from ITS_base
@@ -259,6 +260,8 @@ struct its_device {
u32 nr_lpis;
/* Physical Device id */
u32 device_id;
+ /* RB-tree entry */
+ struct rb_node node;
};
int its_init(struct rdist_prop *rdists);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (5 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 06/22] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 17:13 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
` (14 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This patch introduces virtual ITS driver with following
functionality
- Introduces helper functions to manage device table and
ITT table in guest memory
- Helper function to handle virtual ITS devices assigned
to domain
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Removed RB tree that manages vitual ITS devices
v4: - Rename functions {find,remove,insert}_vits_* to
vits_{find,remove,insert}.
- Add common helper function to map and read/write dt
or vitt table entry.
- Removed unused code
---
xen/arch/arm/vgic-v3-its.c | 162 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/gic-its.h | 36 +++++++++
3 files changed, 200 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
new file mode 100644
index 0000000..60f8332
--- /dev/null
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -0,0 +1,162 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/init.h>
+#include <xen/irq.h>
+#include <xen/list.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <xen/domain_page.h>
+#include <asm/device.h>
+#include <asm/mmio.h>
+#include <asm/io.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
+ uint32_t size, bool_t set)
+{
+ struct page_info *page;
+ uint64_t offset;
+ p2m_type_t p2mt;
+ void *p;
+
+ page = get_page_from_gfn(d, paddr_to_pfn(entry), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ printk(XENLOG_G_ERR "d%"PRId32": vITS: Failed to get table entry\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ printk(XENLOG_G_ERR "d%"PRId32": vITS: with wrong attributes\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = entry & ~PAGE_MASK;
+
+ if ( set )
+ memcpy(p + offset, addr, size);
+ else
+ memcpy(addr, p + offset, size);
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
+/* ITS device table helper functions */
+static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
+ struct vdevice_table *entry, bool_t set)
+{
+ uint64_t offset;
+ paddr_t dt_entry;
+ struct vgic_its *vits = d->arch.vgic.vits;
+
+ BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
+
+ offset = dev_id * sizeof(struct vdevice_table);
+ if ( offset > vits->dt_size )
+ {
+ printk(XENLOG_G_ERR
+ "d%"PRId32":vITS:Out of range off 0x%"PRIx64" id 0x%"PRIx32"\n",
+ d->domain_id, offset, dev_id);
+ return -EINVAL;
+ }
+
+ dt_entry = vits->dt_ipa + offset;
+
+ return vits_access_guest_table(d, dt_entry, entry,
+ sizeof(struct vdevice_table), set);
+}
+
+int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 1);
+}
+
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 0);
+}
+
+static int vits_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry, bool_t set)
+{
+ struct vdevice_table dt_entry;
+ paddr_t vitt_entry;
+ uint64_t offset;
+
+ BUILD_BUG_ON(sizeof(struct vitt) != 8);
+
+ if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
+ {
+ printk(XENLOG_G_ERR
+ "d%"PRId32": vITS: Fail to get vdevice for vdev 0x%"PRIx32"\n",
+ d->domain_id, devid);
+ return -EINVAL;
+ }
+
+ /* dt_entry is validated in vits_get_vdevice_entry */
+ offset = event * sizeof(struct vitt);
+ if ( offset > dt_entry.vitt_size )
+ {
+ printk(XENLOG_G_ERR "d%"PRId32": vITS: ITT out of range\n",
+ d->domain_id);
+ return -EINVAL;
+ }
+
+ vitt_entry = dt_entry.vitt_ipa + offset;
+
+ return vits_access_guest_table(d, vitt_entry, entry,
+ sizeof(struct vitt), set);
+}
+
+int vits_set_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 1);
+}
+
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 0);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 56aa208..986a4d6 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -104,6 +104,8 @@ struct arch_domain
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
#ifdef HAS_GICV3
+ /* Virtual ITS */
+ struct vgic_its *vits;
/* GIC V3 addressing */
/* List of contiguous occupied by the redistributors */
struct vgic_rdist_region {
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 3daba4b..66be53a 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -116,6 +116,17 @@ struct its_collection {
u16 col_id;
};
+/*
+ * Per domain virtual ITS structure.
+ */
+struct vgic_its
+{
+ /* vITT device table ipa */
+ paddr_t dt_ipa;
+ /* vITT device table size */
+ uint64_t dt_size;
+};
+
/* ITS command structure */
typedef union {
u64 bits[4];
@@ -264,8 +275,33 @@ struct its_device {
struct rb_node node;
};
+/*
+ * struct vdevice_table and struct vitt are typically stored in memory
+ * which has been provided by the guest out of its own address space
+ * and which remains accessible to the guest.
+ *
+ * Therefore great care _must_ be taken when accessing an entry in
+ * either table to validate the sanity of any values which are used.
+ */
+struct vdevice_table {
+ uint64_t vitt_ipa;
+ uint32_t vitt_size;
+ uint32_t padding;
+};
+
+struct vitt {
+ uint16_t valid:1;
+ uint16_t pad:15;
+ uint16_t vcollection;
+ uint32_t vlpi;
+};
+
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry);
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver
2015-07-27 11:11 ` [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-07-28 17:13 ` Julien Grall
2015-07-31 6:49 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-28 17:13 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> new file mode 100644
> index 0000000..60f8332
> --- /dev/null
> +++ b/xen/arch/arm/vgic-v3-its.c
> +static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
> + uint32_t size, bool_t set)
> +{
> + struct page_info *page;
> + uint64_t offset;
> + p2m_type_t p2mt;
> + void *p;
> +
> + page = get_page_from_gfn(d, paddr_to_pfn(entry), &p2mt, P2M_ALLOC);
> + if ( !page )
> + {
> + printk(XENLOG_G_ERR "d%"PRId32": vITS: Failed to get table entry\n",
A domain id is encoded on 16 bits and not 32 bits. Furthermore it's an
unsigned. Here you will print signed.
My remark will be the same for all similar use within this patch.
[..]
> +/* ITS device table helper functions */
> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> + struct vdevice_table *entry, bool_t set)
> +{
> + uint64_t offset;
> + paddr_t dt_entry;
> + struct vgic_its *vits = d->arch.vgic.vits;
const struct
[..]
> +int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry)
> +{
> + return vits_vdevice_entry(d, devid, entry, 1);
> +}
The prototype is not specified in the header. So either you need to add
the prototype, if used outside the file, or set static.
[..]
> +static int vits_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry, bool_t set)
> +{
> + struct vdevice_table dt_entry;
> + paddr_t vitt_entry;
> + uint64_t offset;
> +
> + BUILD_BUG_ON(sizeof(struct vitt) != 8);
> +
> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
> + {
> + printk(XENLOG_G_ERR
> + "d%"PRId32": vITS: Fail to get vdevice for vdev 0x%"PRIx32"\n",
s/vdev/vdevid/
> + d->domain_id, devid);
> + return -EINVAL;
> + }
> +
> + /* dt_entry is validated in vits_get_vdevice_entry */
s/is validated/has been validated/
[..]
> +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry)
Same remark as vits_set_vdevice_entry.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver
2015-07-28 17:13 ` Julien Grall
@ 2015-07-31 6:49 ` Vijay Kilari
2015-07-31 10:14 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 6:49 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Tue, Jul 28, 2015 at 10:43 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> new file mode 100644
>> index 0000000..60f8332
>> --- /dev/null
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> +static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
>> + uint32_t size, bool_t set)
>> +{
>> + struct page_info *page;
>> + uint64_t offset;
>> + p2m_type_t p2mt;
>> + void *p;
>> +
>> + page = get_page_from_gfn(d, paddr_to_pfn(entry), &p2mt, P2M_ALLOC);
>> + if ( !page )
>> + {
>> + printk(XENLOG_G_ERR "d%"PRId32": vITS: Failed to get table entry\n",
>
> A domain id is encoded on 16 bits and not 32 bits. Furthermore it's an
> unsigned. Here you will print signed.
>
> My remark will be the same for all similar use within this patch.
>
> [..]
>
>> +/* ITS device table helper functions */
>> +static int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
>> + struct vdevice_table *entry, bool_t set)
>> +{
>> + uint64_t offset;
>> + paddr_t dt_entry;
>> + struct vgic_its *vits = d->arch.vgic.vits;
>
> const struct
>
> [..]
>
>> +int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
>> + struct vdevice_table *entry)
>> +{
>> + return vits_vdevice_entry(d, devid, entry, 1);
>> +}
>
> The prototype is not specified in the header. So either you need to add
> the prototype, if used outside the file, or set static.
>
> [..]
>
>> +static int vits_vitt_entry(struct domain *d, uint32_t devid,
>> + uint32_t event, struct vitt *entry, bool_t set)
>> +{
>> + struct vdevice_table dt_entry;
>> + paddr_t vitt_entry;
>> + uint64_t offset;
>> +
>> + BUILD_BUG_ON(sizeof(struct vitt) != 8);
>> +
>> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
>> + {
>> + printk(XENLOG_G_ERR
>> + "d%"PRId32": vITS: Fail to get vdevice for vdev 0x%"PRIx32"\n",
>
> s/vdev/vdevid/
I think, to manage within 80 char, I used just "vdev"
>
>> + d->domain_id, devid);
>> + return -EINVAL;
>> + }
>> +
>> + /* dt_entry is validated in vits_get_vdevice_entry */
>
> s/is validated/has been validated/
>
> [..]
>
>> +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
>> + uint32_t event, struct vitt *entry)
>
> Same remark as vits_set_vdevice_entry.
I have made non-static for compilation purpose. I will try to introduce
this in the patch where it is used. But it is more logical to have this
in this patch. Anyway I forget to make it static in later patches
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver
2015-07-31 6:49 ` Vijay Kilari
@ 2015-07-31 10:14 ` Julien Grall
2015-07-31 10:32 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-31 10:14 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 31/07/15 07:49, Vijay Kilari wrote:
>>> +static int vits_vitt_entry(struct domain *d, uint32_t devid,
>>> + uint32_t event, struct vitt *entry, bool_t set)
>>> +{
>>> + struct vdevice_table dt_entry;
>>> + paddr_t vitt_entry;
>>> + uint64_t offset;
>>> +
>>> + BUILD_BUG_ON(sizeof(struct vitt) != 8);
>>> +
>>> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
>>> + {
>>> + printk(XENLOG_G_ERR
>>> + "d%"PRId32": vITS: Fail to get vdevice for vdev 0x%"PRIx32"\n",
>>
>> s/vdev/vdevid/
>
> I think, to manage within 80 char, I used just "vdev"
80 character in the source file I guess? If so, you should avoid this
kind of cutting just for coding style benefits. We are not so strict on it.
>
>>
>>> + d->domain_id, devid);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + /* dt_entry is validated in vits_get_vdevice_entry */
>>
>> s/is validated/has been validated/
>>
>> [..]
>>
>>> +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
>>> + uint32_t event, struct vitt *entry)
>>
>> Same remark as vits_set_vdevice_entry.
>
> I have made non-static for compilation purpose. I will try to introduce
> this in the patch where it is used. But it is more logical to have this
> in this patch. Anyway I forget to make it static in later patches
Having introduce static here would have avoid forgetting the static
later... It's just a matter of how you split your series.
For instance, if you would have merge this patch with #8, making this
function non-static wouldn't have been necessary.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver
2015-07-31 10:14 ` Julien Grall
@ 2015-07-31 10:32 ` Ian Campbell
0 siblings, 0 replies; 81+ messages in thread
From: Ian Campbell @ 2015-07-31 10:32 UTC (permalink / raw)
To: Julien Grall, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, manish.jaggi, Tim Deegan,
xen-devel@lists.xen.org, Stefano Stabellini, Vijaya Kumar K
On Fri, 2015-07-31 at 11:14 +0100, Julien Grall wrote:
> Hi Vijay,
>
> On 31/07/15 07:49, Vijay Kilari wrote:
> > > > +static int vits_vitt_entry(struct domain *d, uint32_t devid,
> > > > + uint32_t event, struct vitt *entry,
> > > > bool_t set)
> > > > +{
> > > > + struct vdevice_table dt_entry;
> > > > + paddr_t vitt_entry;
> > > > + uint64_t offset;
> > > > +
> > > > + BUILD_BUG_ON(sizeof(struct vitt) != 8);
> > > > +
> > > > + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
> > > > + {
> > > > + printk(XENLOG_G_ERR
> > > > + "d%"PRId32": vITS: Fail to get vdevice for vdev
> > > > 0x%"PRIx32"\n",
> > >
> > > s/vdev/vdevid/
> >
> > I think, to manage within 80 char, I used just "vdev"
>
> 80 character in the source file I guess? If so, you should avoid this
> kind of cutting just for coding style benefits. We are not so strict on
> it.
It's also acceptable to indent string constants by less than what would be
expected to avoid this, e.g.
printk(XENLOG_G_ERR
d%"PRId32": vITS: Fail to get vdevice for vdevid 0x%"PRIx32"\n",
(also, s/Fail/Failed/ or perhaps "Unable").
Also, I don't think PRId32 is what we should use for d->domain_id. The
actual type is uint16_t, which would suggest PRIu16 (or maybe PRId16) is
formally correct but in actuality we normally use %u or %d in this case (%u
is probably more correct, although it appears to be in the minority).
> > > > + d->domain_id, devid);
> > > > + return -EINVAL;
> > > > + }
> > > > +
> > > > + /* dt_entry is validated in vits_get_vdevice_entry */
> > >
> > > s/is validated/has been validated/
> > >
> > > [..]
> > >
> > > > +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
> > > > + uint32_t event, struct vitt *entry)
> > >
> > > Same remark as vits_set_vdevice_entry.
> >
> > I have made non-static for compilation purpose. I will try to introduce
> > this in the patch where it is used. But it is more logical to have this
> > in this patch. Anyway I forget to make it static in later patches
>
> Having introduce static here would have avoid forgetting the static
> later... It's just a matter of how you split your series.
>
> For instance, if you would have merge this patch with #8, making this
> function non-static wouldn't have been necessary.
I've also suggest elsewhere not adding these new files to the build until
later in the series, such that the user of this function is already added
by the time the file starts to get compiled.
That approach is usually acceptable when introducing a large new file which
isn't used until towards the end of the series.
Of course you need to avoid calling any functions in that file until after
you add it to the build, which introduces a different constraint on the
series ordering, so it's not always possible to make use of this approach.
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (6 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 07/22] xen/arm: ITS: Add virtual ITS driver vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 18:04 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
` (13 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Virtual ITS command processing support to Virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Rename vgic_its_*() to vits_*()
v4: - Use helper function to read from command queue
- Add MOVALL
- Removed check for entry in device in domain RB-tree
---
xen/arch/arm/vgic-v3-its.c | 392 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 13 ++
2 files changed, 405 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 60f8332..dfa3435 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -30,8 +30,27 @@
#include <asm/gic.h>
#include <asm/vgic.h>
#include <asm/gic-its.h>
+#include <asm/atomic.h>
#include <xen/log2.h>
+#define DEBUG_ITS
+
+#ifdef DEBUG_ITS
+# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#ifdef DEBUG_ITS
+static void dump_cmd(its_cmd_block *cmd)
+{
+ printk("VITS:CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
+}
+#else
+static void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
+#endif
+
static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
@@ -152,6 +171,379 @@ int vits_get_vitt_entry(struct domain *d, uint32_t devid,
return vits_vitt_entry(d, devid, event, entry, 0);
}
+static int vits_process_sync(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: SYNC: ta 0x%"PRIx32" \n", v, virt_cmd->sync.ta);
+
+ return 0;
+}
+
+static int vits_process_mapvi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint8_t vcol_id, cmd;
+ uint32_t vid, dev_id, event;
+
+ vcol_id = virt_cmd->mapvi.col;
+ vid = virt_cmd->mapvi.phy_id;
+ cmd = virt_cmd->mapvi.cmd;
+ dev_id = virt_cmd->mapvi.devid;
+
+ DPRINTK("%pv: vITS: MAPVI: dev 0x%"PRIx32" vcol %"PRId32" vid %"PRId32"\n",
+ v, dev_id, vcol_id, vid);
+
+ entry.valid = true;
+ entry.vcollection = vcol_id;
+ entry.vlpi = vid;
+
+ if ( cmd == GITS_CMD_MAPI )
+ vits_set_vitt_entry(d, dev_id, vid, &entry);
+ else
+ {
+ event = virt_cmd->mapvi.event;
+ vits_set_vitt_entry(d, dev_id, event, &entry);
+ }
+
+ return 0;
+}
+
+static int vits_process_movi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t dev_id, event;
+ uint8_t vcol_id;
+
+ vcol_id = virt_cmd->movi.col;
+ event = virt_cmd->movi.event;
+ dev_id = virt_cmd->movi.devid;
+
+ DPRINTK("%pv vITS: MOVI: dev_id 0x%"PRIx32" vcol %"PRId32" event %"PRId32"\n",
+ v, dev_id, vcol_id, event);
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.vcollection = vcol_id;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_movall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: MOVALL: ta1 0x%"PRIx32" ta2 0x%"PRIx32" \n",
+ v, virt_cmd->movall.ta1, virt_cmd->movall.ta2);
+
+ return 0;
+}
+
+static int vits_process_discard(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct domain *d = v->domain;
+ uint32_t event, dev_id;
+
+ event = virt_cmd->discard.event;
+ dev_id = virt_cmd->discard.devid;
+
+ DPRINTK("%pv vITS: DISCARD: dev_id 0x%"PRIx32" id %"PRId32"\n",
+ v, virt_cmd->discard.devid, event);
+
+ if ( vits_get_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ entry.valid = false;
+
+ if ( vits_set_vitt_entry(d, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_inv(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv vITS: INV: dev_id 0x%"PRIx32" id %"PRId32"\n",
+ v, virt_cmd->inv.devid, virt_cmd->inv.event);
+
+ return 0;
+}
+
+static int vits_process_clear(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: CLEAR: dev_id 0x%"PRIx32" id %"PRId32"\n",
+ v, virt_cmd->clear.devid, virt_cmd->clear.event);
+
+ return 0;
+}
+
+static int vits_process_invall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* Ignored */
+ DPRINTK("%pv: vITS: INVALL: vCID %"PRId32"\n", v, virt_cmd->invall.col);
+
+ return 0;
+}
+
+static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint32_t event, dev_id ;
+
+ event = virt_cmd->int_cmd.cmd;
+ dev_id = virt_cmd->int_cmd.devid;
+
+ DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRId32"\n",
+ v, dev_id, event);
+
+ /* TODO: Inject LPI */
+
+ return 0;
+}
+
+static int vits_add_device(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct domain *d = v->domain;
+ struct vdevice_table dt_entry;
+ uint32_t dev_id = virt_cmd->mapd.devid;
+
+ DPRINTK("%pv: vITS:Add dev 0x%"PRIx32" ipa = 0x%"PRIx64" size %"PRId32"\n",
+ v, dev_id, (u64)virt_cmd->mapd.itt << MAPC_ITT_IPA_SHIFT,
+ virt_cmd->mapd.size);
+
+ if ( virt_cmd->mapd.valid )
+ {
+ /* itt field is 40 bit. extract 48 bit address by shifting */
+ dt_entry.vitt_ipa = virt_cmd->mapd.itt << MAPC_ITT_IPA_SHIFT;
+ dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) *
+ sizeof(struct vitt);
+ }
+ else
+ {
+ dt_entry.vitt_ipa = INVALID_PADDR;
+ dt_entry.vitt_size = 0;
+ }
+
+ if ( vits_set_vdevice_entry(d, dev_id, &dt_entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vits_process_mapc(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t vcol_id;
+ uint64_t vta = 0;
+
+ vcol_id = virt_cmd->mapc.col;
+ vta = virt_cmd->mapc.ta;
+
+ DPRINTK("%pv: vITS: MAPC: vCID %"PRId32" vTA 0x%"PRIx64" valid %"PRId32"\n",
+ v, vcol_id, vta, virt_cmd->mapc.valid);
+
+ if ( virt_cmd->mapc.valid )
+ {
+ if ( vta > v->domain->max_vcpus )
+ return -EINVAL;
+ vits->collections[vcol_id].target_address = vta;
+ }
+ else
+ vits->collections[vcol_id].target_address = INVALID_PADDR;
+
+ return 0;
+}
+
+#ifdef DEBUG_ITS
+char *cmd_str[] = {
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPVI] = "MAPVI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ };
+#endif
+
+static int vits_parse_its_command(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t cmd = virt_cmd->hdr.cmd;
+ int ret;
+
+ DPRINTK("%pv: vITS: Received cmd %s (0x%"PRIx32")\n", v, cmd_str[cmd], cmd);
+ dump_cmd(virt_cmd);
+
+ switch ( cmd )
+ {
+ case GITS_CMD_MAPD:
+ ret = vits_add_device(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPC:
+ ret = vits_process_mapc(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPI:
+ /* MAPI is same as MAPVI */
+ case GITS_CMD_MAPVI:
+ ret = vits_process_mapvi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVI:
+ ret = vits_process_movi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVALL:
+ ret = vits_process_movall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_DISCARD:
+ ret = vits_process_discard(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INV:
+ ret = vits_process_inv(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INVALL:
+ ret = vits_process_invall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INT:
+ ret = vits_process_int(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_CLEAR:
+ ret = vits_process_clear(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_SYNC:
+ ret = vits_process_sync(v, vits, virt_cmd);
+ break;
+ default:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Unhandled command cmd %"PRIx32"\n",
+ v, cmd);
+ return 1;
+ }
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to handle cmd %"PRIx32"\n",
+ v, cmd);
+ return 1;
+ }
+
+ return 0;
+}
+
+static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ paddr_t maddr;
+ struct domain *d = v->domain;
+ int ret;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ if ( !(vits->cmd_base & GITS_CBASER_VALID) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid CBASER\n", v);
+ return 0;
+ }
+
+ /* CMD Q can be more than 1 page. Map only page that is required */
+ maddr = (vits->cmd_base & MASK_4K) + atomic_read(&vits->cmd_read);
+
+ DPRINTK("%pv: vITS: Mapping CMD Q maddr 0x%"PRIx64" read 0x%"PRIx32"\n",
+ v, maddr, atomic_read(&vits->cmd_read));
+
+ ret = vits_access_guest_table(d, maddr, (void *)virt_cmd,
+ sizeof(its_cmd_block), 0);
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: Failed to get command page @page 0x%"PRIx32"\n",
+ v, atomic_read(&vits->cmd_read));
+ return -EINVAL;
+ }
+
+ /* No command queue is created by vits to check on Q full */
+ atomic_add(sizeof(its_cmd_block), &vits->cmd_read);
+ if ( atomic_read(&vits->cmd_read) == vits->cmd_qsize )
+ {
+ DPRINTK("%pv: vITS: Reset read @ 0x%"PRIx32" qsize 0x%"PRIx64"\n",
+ v, atomic_read(&vits->cmd_read), vits->cmd_qsize);
+
+ atomic_set(&vits->cmd_read, 0);
+ }
+
+ return 0;
+}
+
+int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
+{
+ its_cmd_block virt_cmd;
+
+ ASSERT(spin_is_locked(&vits->lock));
+
+ do {
+ if ( vits_read_virt_cmd(v, vits, &virt_cmd) )
+ goto err;
+ if ( vits_parse_its_command(v, vits, &virt_cmd) )
+ goto err;
+ } while ( vits->cmd_write != atomic_read(&vits->cmd_read) );
+
+ DPRINTK("%pv: vITS: read @ 0x%"PRIx32" write @ 0x%"PRIx64"\n",
+ v, atomic_read(&vits->cmd_read),
+ vits->cmd_write);
+
+ return 1;
+err:
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
+ domain_crash_synchronous();
+
+ return 0;
+}
+
+int vits_domain_init(struct domain *d)
+{
+ struct vgic_its *vits;
+ int i;
+
+ d->arch.vgic.vits = xzalloc(struct vgic_its);
+ if ( !d->arch.vgic.vits )
+ return -ENOMEM;
+
+ vits = d->arch.vgic.vits;
+
+ spin_lock_init(&vits->lock);
+
+ vits->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
+ if ( !vits->collections )
+ {
+ xfree(d->arch.vgic.vits);
+ return -ENOMEM;
+ }
+
+ for ( i = 0; i < nr_cpu_ids; i++ )
+ vits->collections[i].target_address = ~0UL;
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 66be53a..cdb786c 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -21,6 +21,8 @@
#include <asm/gic_v3_defs.h>
#include <xen/rbtree.h>
+#define MASK_4K 0xfffffffff000UL
+#define MAPC_ITT_IPA_SHIFT 8
/*
* ITS registers, offsets from ITS_base
*/
@@ -121,10 +123,21 @@ struct its_collection {
*/
struct vgic_its
{
+ spinlock_t lock;
+ /* Command queue base */
+ paddr_t cmd_base;
+ /* Command queue write pointer */
+ paddr_t cmd_write;
+ /* Command queue read pointer */
+ atomic_t cmd_read;
+ /* Command queue size */
+ unsigned long cmd_qsize;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
uint64_t dt_size;
+ /* collections mapped */
+ struct its_collection *collections;
};
/* ITS command structure */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support
2015-07-27 11:11 ` [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-07-28 18:04 ` Julien Grall
2015-07-31 6:57 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-28 18:04 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Virtual ITS command processing support to Virtual ITS driver
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v5: - Rename vgic_its_*() to vits_*()
The changelog seems very small compared to the amount of discussion we
had on v4.
> v4: - Use helper function to read from command queue
> - Add MOVALL
> - Removed check for entry in device in domain RB-tree
> ---
> xen/arch/arm/vgic-v3-its.c | 392 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 13 ++
> 2 files changed, 405 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 60f8332..dfa3435 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -30,8 +30,27 @@
> #include <asm/gic.h>
> #include <asm/vgic.h>
> #include <asm/gic-its.h>
> +#include <asm/atomic.h>
> #include <xen/log2.h>
>
> +#define DEBUG_ITS
As said on v4, you should directly do "// #define DEBUG_ITS" rather than
changing this line again in patch #10.
> +
> +#ifdef DEBUG_ITS
> +# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> +#else
> +# define DPRINTK(fmt, args...) do {} while ( 0 )
> +#endif
> +
> +#ifdef DEBUG_ITS
> +static void dump_cmd(its_cmd_block *cmd)
> +{
> + printk("VITS:CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->bits[0], cmd->bits[1], cmd->bits[2], cmd->bits[3]);
> +}
> +#else
> +static void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
The do {} while (0) is not necessary in
> +#endif
[..]
> +static int vits_process_mapvi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct domain *d = v->domain;
> + uint8_t vcol_id, cmd;
> + uint32_t vid, dev_id, event;
> +
> + vcol_id = virt_cmd->mapvi.col;
> + vid = virt_cmd->mapvi.phy_id;
> + cmd = virt_cmd->mapvi.cmd;
> + dev_id = virt_cmd->mapvi.devid;
> +
> + DPRINTK("%pv: vITS: MAPVI: dev 0x%"PRIx32" vcol %"PRId32" vid %"PRId32"\n",
You are using the wrong PRI* for vcol. It's an uint8_t not int32_t. If
PRIu8 doesn't exist please introduce it.
I'm sure I will miss some in all the patches. Please review all the
PRId32 you added and use the correct one.
[..]
> +static int vits_process_movi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct domain *d = v->domain;
> + uint32_t dev_id, event;
> + uint8_t vcol_id;
> +
> + vcol_id = virt_cmd->movi.col;
> + event = virt_cmd->movi.event;
> + dev_id = virt_cmd->movi.devid;
> +
> + DPRINTK("%pv vITS: MOVI: dev_id 0x%"PRIx32" vcol %"PRId32" event %"PRId32"\n",
vcol PRIu8
event PRIu32
[..]
> +static int vits_process_clear(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + /* Ignored */
> + DPRINTK("%pv: vITS: CLEAR: dev_id 0x%"PRIx32" id %"PRId32"\n",
id PRIu32
> + v, virt_cmd->clear.devid, virt_cmd->clear.event);
> +
> + return 0;
> +}
> +
> +static int vits_process_invall(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + /* Ignored */
> + DPRINTK("%pv: vITS: INVALL: vCID %"PRId32"\n", v, virt_cmd->invall.col);
vCID PRIu8
> +
> + return 0;
> +}
> +
> +static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + uint32_t event, dev_id ;
> +
> + event = virt_cmd->int_cmd.cmd;
> + dev_id = virt_cmd->int_cmd.devid;
> +
> + DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRId32"\n",
id PRIu32
> + v, dev_id, event);
> +
> + /* TODO: Inject LPI */
Done in a follow-up patch I guess?
> +
> + return 0;
> +}
> +
> +static int vits_add_device(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct domain *d = v->domain;
> + struct vdevice_table dt_entry;
> + uint32_t dev_id = virt_cmd->mapd.devid;
> +
> + DPRINTK("%pv: vITS:Add dev 0x%"PRIx32" ipa = 0x%"PRIx64" size %"PRId32"\n",
size PRIu32
[..]
> +static int vits_process_mapc(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + uint8_t vcol_id;
> + uint64_t vta = 0;
> +
> + vcol_id = virt_cmd->mapc.col;
> + vta = virt_cmd->mapc.ta;
> +
> + DPRINTK("%pv: vITS: MAPC: vCID %"PRId32" vTA 0x%"PRIx64" valid %"PRId32"\n",
> + v, vcol_id, vta, virt_cmd->mapc.valid);
> +
On v4, I only asked to do the check on vta only when the mapc.valid = 1.
The one the collection ID should not have been dropped. Without it a
malicious guest can provide an invalid collection ID which will result
to access outside the array and may crash Xen.
So please re-add this check.
[..]
> +static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + paddr_t maddr;
> + struct domain *d = v->domain;
> + int ret;
> +
> + ASSERT(spin_is_locked(&vits->lock));
> +
> + if ( !(vits->cmd_base & GITS_CBASER_VALID) )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid CBASER\n", v);
> + return 0;
> + }
> +
> + /* CMD Q can be more than 1 page. Map only page that is required */
"Map only the page..."
> + maddr = (vits->cmd_base & MASK_4K) + atomic_read(&vits->cmd_read);
> +
> + DPRINTK("%pv: vITS: Mapping CMD Q maddr 0x%"PRIx64" read 0x%"PRIx32"\n",
> + v, maddr, atomic_read(&vits->cmd_read));
> +
> + ret = vits_access_guest_table(d, maddr, (void *)virt_cmd,
> + sizeof(its_cmd_block), 0);
> + if ( ret )
> + {
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: Failed to get command page @page 0x%"PRIx32"\n",
> + v, atomic_read(&vits->cmd_read));
> + return -EINVAL;
> + }
> +
> + /* No command queue is created by vits to check on Q full */
> + atomic_add(sizeof(its_cmd_block), &vits->cmd_read);
> + if ( atomic_read(&vits->cmd_read) == vits->cmd_qsize )
> + {
> + DPRINTK("%pv: vITS: Reset read @ 0x%"PRIx32" qsize 0x%"PRIx64"\n",
> + v, atomic_read(&vits->cmd_read), vits->cmd_qsize);
> +
> + atomic_set(&vits->cmd_read, 0);
> + }
> +
> + return 0;
> +}
> +
> +int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
Either put a static, if not use outside this file, or add the
declaration in the header.
> +{
> + its_cmd_block virt_cmd;
> +
> + ASSERT(spin_is_locked(&vits->lock));
> +
> + do {
> + if ( vits_read_virt_cmd(v, vits, &virt_cmd) )
> + goto err;
> + if ( vits_parse_its_command(v, vits, &virt_cmd) )
> + goto err;
> + } while ( vits->cmd_write != atomic_read(&vits->cmd_read) );
> +
> + DPRINTK("%pv: vITS: read @ 0x%"PRIx32" write @ 0x%"PRIx64"\n",
> + v, atomic_read(&vits->cmd_read),
> + vits->cmd_write);
> +
> + return 1;
> +err:
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Failed to process guest cmd\n", v);
> + domain_crash_synchronous();
> +
> + return 0;
> +}
> +
> +int vits_domain_init(struct domain *d)
Ditto.
> +{
> + struct vgic_its *vits;
> + int i;
> +
> + d->arch.vgic.vits = xzalloc(struct vgic_its);
> + if ( !d->arch.vgic.vits )
> + return -ENOMEM;
> +
> + vits = d->arch.vgic.vits;
> +
> + spin_lock_init(&vits->lock);
> +
> + vits->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
The number of collection for a domain is based on the number of VCPUs
owned by him (see d->max_vcpus).
Furthermore, you are allocating to few collection, the number of
collection should at least be max_vcpus + 1.
You've introduced vits_get_max_collections in a latter patch (see #10).
Please use it here.
> + if ( !vits->collections )
> + {
> + xfree(d->arch.vgic.vits);
> + return -ENOMEM;
It's not neccesary to take care of free what you allocated here. When a
domain is destroyed the domain_vgic_free will be called to free
everything correctly.
Although that means that you need to introduce a vits_domain_free, which
is in anycase mandatory. I'd like to see it within this patch.
> + }
> +
> + for ( i = 0; i < nr_cpu_ids; i++ )
> + vits->collections[i].target_address = ~0UL;
> +
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 66be53a..cdb786c 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -21,6 +21,8 @@
> #include <asm/gic_v3_defs.h>
> #include <xen/rbtree.h>
>
> +#define MASK_4K 0xfffffffff000UL
If you name the macro MASK_4K this should go in xen/sizes.h and not in
the gic-its.h. Although on v4 [1], Ian suggested to rename into
GITS_CBASER_PA_MASK which IHMO would be better.
Regards,
Regards,
[1] http://lists.xen.org/archives/html/xen-devel/2015-07/msg03032.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support
2015-07-28 18:04 ` Julien Grall
@ 2015-07-31 6:57 ` Vijay Kilari
2015-07-31 10:16 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 6:57 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
>> /*
>> * Local variables:
>> * mode: C
>> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
>> index 66be53a..cdb786c 100644
>> --- a/xen/include/asm-arm/gic-its.h
>> +++ b/xen/include/asm-arm/gic-its.h
>> @@ -21,6 +21,8 @@
>> #include <asm/gic_v3_defs.h>
>> #include <xen/rbtree.h>
>>
>> +#define MASK_4K 0xfffffffff000UL
>
> If you name the macro MASK_4K this should go in xen/sizes.h and not in
> the gic-its.h. Although on v4 [1], Ian suggested to rename into
> GITS_CBASER_PA_MASK which IHMO would be better.
This is used by other generic code as well in later patches.
One option is to introduce separate macro for each usage or generic name like
GITS_PA_MASK
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support
2015-07-31 6:57 ` Vijay Kilari
@ 2015-07-31 10:16 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-31 10:16 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 31/07/15 07:57, Vijay Kilari wrote:
>>> /*
>>> * Local variables:
>>> * mode: C
>>> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
>>> index 66be53a..cdb786c 100644
>>> --- a/xen/include/asm-arm/gic-its.h
>>> +++ b/xen/include/asm-arm/gic-its.h
>>> @@ -21,6 +21,8 @@
>>> #include <asm/gic_v3_defs.h>
>>> #include <xen/rbtree.h>
>>>
>>> +#define MASK_4K 0xfffffffff000UL
>>
>> If you name the macro MASK_4K this should go in xen/sizes.h and not in
>> the gic-its.h. Although on v4 [1], Ian suggested to rename into
>> GITS_CBASER_PA_MASK which IHMO would be better.
>
>
> This is used by other generic code as well in later patches.
> One option is to introduce separate macro for each usage or generic name like
> GITS_PA_MASK
IHMO a mask per register would be better.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (7 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 08/22] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 18:14 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation vijay.kilari
` (12 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Export physical ITS information to virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 27 ++++++++++++++++++++++++++-
xen/arch/arm/vgic-v3-its.c | 9 +++++++++
xen/include/asm-arm/gic-its.h | 14 ++++++++++++++
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index aa4d3c5..e16fa03 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -94,6 +94,7 @@ static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
static struct rb_root rb_its_dev;
+static struct gic_its_info its_data;
static DEFINE_SPINLOCK(rb_its_dev_lock);
#define gic_data_rdist() (this_cpu(rdist))
@@ -942,6 +943,8 @@ static int its_probe(struct dt_device_node *node)
its->phys_size = its_size;
typer = readl_relaxed(its_base + GITS_TYPER);
its->ite_size = ((typer >> 4) & 0xf) + 1;
+ its_data.eventid_bits = GITS_TYPER_IDBITS(typer);
+ its_data.dev_bits = GITS_TYPER_DEVBITS(typer);
its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
if ( !its->cmd_base )
@@ -1032,7 +1035,10 @@ int its_cpu_init(void)
int __init its_init(struct rdist_prop *rdists)
{
+ struct its_node *its;
+ struct its_node_info *info;
struct dt_device_node *np = NULL;
+ uint32_t i, nr_its = 0;
static const struct dt_device_match its_device_ids[] __initconst =
{
@@ -1042,7 +1048,10 @@ int __init its_init(struct rdist_prop *rdists)
for (np = dt_find_matching_node(NULL, its_device_ids); np;
np = dt_find_matching_node(np, its_device_ids))
- its_probe(np);
+ {
+ if ( !its_probe(np) )
+ nr_its++;
+ }
if ( list_empty(&its_nodes) )
{
@@ -1050,6 +1059,22 @@ int __init its_init(struct rdist_prop *rdists)
return -ENXIO;
}
+ info = xzalloc_array(struct its_node_info, nr_its);
+ if ( !info )
+ return -ENOMEM;
+
+ i = 0;
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ info[i].phys_base = its->phys_base;
+ info[i].phys_size = its->phys_size;
+ i++;
+ }
+
+ its_data.nr_its = nr_its;
+ its_data.its_hw = info;
+ vits_setup_hw(&its_data);
+
gic_rdists = rdists;
its_lpi_init(rdists->id_bits);
its_alloc_lpi_tables();
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index dfa3435..3a003d4 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -51,6 +51,15 @@ static void dump_cmd(its_cmd_block *cmd)
static void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
#endif
+static struct {
+ struct gic_its_info *info;
+} vits_hw;
+
+void vits_setup_hw(struct gic_its_info *its_info)
+{
+ vits_hw.info = its_info;
+}
+
static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index cdb786c..23ff66c 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -309,12 +309,26 @@ struct vitt {
uint32_t vlpi;
};
+struct its_node_info
+{
+ paddr_t phys_base;
+ unsigned long phys_size;
+};
+
+struct gic_its_info {
+ uint32_t eventid_bits;
+ uint32_t dev_bits;
+ uint32_t nr_its;
+ struct its_node_info *its_hw;
+};
+
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
struct vdevice_table *entry);
+void vits_setup_hw(struct gic_its_info *info);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS
2015-07-27 11:11 ` [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
@ 2015-07-28 18:14 ` Julien Grall
2015-07-31 7:01 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-28 18:14 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index aa4d3c5..e16fa03 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -94,6 +94,7 @@ static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> static struct rb_root rb_its_dev;
> +static struct gic_its_info its_data;
> static DEFINE_SPINLOCK(rb_its_dev_lock);
>
> #define gic_data_rdist() (this_cpu(rdist))
> @@ -942,6 +943,8 @@ static int its_probe(struct dt_device_node *node)
> its->phys_size = its_size;
> typer = readl_relaxed(its_base + GITS_TYPER);
> its->ite_size = ((typer >> 4) & 0xf) + 1;
> + its_data.eventid_bits = GITS_TYPER_IDBITS(typer);
> + its_data.dev_bits = GITS_TYPER_DEVBITS(typer);
>
> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
> if ( !its->cmd_base )
> @@ -1032,7 +1035,10 @@ int its_cpu_init(void)
>
> int __init its_init(struct rdist_prop *rdists)
> {
> + struct its_node *its;
> + struct its_node_info *info;
> struct dt_device_node *np = NULL;
> + uint32_t i, nr_its = 0;
>
> static const struct dt_device_match its_device_ids[] __initconst =
> {
> @@ -1042,7 +1048,10 @@ int __init its_init(struct rdist_prop *rdists)
>
> for (np = dt_find_matching_node(NULL, its_device_ids); np;
> np = dt_find_matching_node(np, its_device_ids))
> - its_probe(np);
> + {
> + if ( !its_probe(np) )
> + nr_its++;
> + }
> if ( list_empty(&its_nodes) )
> {
> @@ -1050,6 +1059,22 @@ int __init its_init(struct rdist_prop *rdists)
> return -ENXIO;
> }
>
> + info = xzalloc_array(struct its_node_info, nr_its);
> + if ( !info )
> + return -ENOMEM;
> +
> + i = 0;
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + info[i].phys_base = its->phys_base;
> + info[i].phys_size = its->phys_size;
> + i++;
> + }
> +
> + its_data.nr_its = nr_its;
> + its_data.its_hw = info;
> + vits_setup_hw(&its_data);
> +
The vITS will only expose 1 ITS to dom0. So there is no need to try to
allocate memory in order to expose all the ITS.
We can extend vits_setup_hw if we ever want to support more ITS in the
future.
Furthermore I'd like to see all the ITS data pass one by one rather than
in a structure. This would help to catch error if we ever extend the
structure.
Something like:
void vits_hw_setup(paddr_t its0_base, unsigned long its0_size,
uint32_t eventID_bits, uint32_t devID_bits);
> gic_rdists = rdists;
> its_lpi_init(rdists->id_bits);
> its_alloc_lpi_tables();
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index dfa3435..3a003d4 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -51,6 +51,15 @@ static void dump_cmd(its_cmd_block *cmd)
> static void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
> #endif
>
> +static struct {
> + struct gic_its_info *info;
> +} vits_hw;
Please add a boolean valid in the structure. It would help to know if
the vITS can be used or not later.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS
2015-07-28 18:14 ` Julien Grall
@ 2015-07-31 7:01 ` Vijay Kilari
2015-08-03 15:58 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 7:01 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Tue, Jul 28, 2015 at 11:44 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
>> index aa4d3c5..e16fa03 100644
>> --- a/xen/arch/arm/gic-v3-its.c
>> +++ b/xen/arch/arm/gic-v3-its.c
>> @@ -94,6 +94,7 @@ static LIST_HEAD(its_nodes);
>> static DEFINE_SPINLOCK(its_lock);
>> static struct rdist_prop *gic_rdists;
>> static struct rb_root rb_its_dev;
>> +static struct gic_its_info its_data;
>> static DEFINE_SPINLOCK(rb_its_dev_lock);
>>
>> #define gic_data_rdist() (this_cpu(rdist))
>> @@ -942,6 +943,8 @@ static int its_probe(struct dt_device_node *node)
>> its->phys_size = its_size;
>> typer = readl_relaxed(its_base + GITS_TYPER);
>> its->ite_size = ((typer >> 4) & 0xf) + 1;
>> + its_data.eventid_bits = GITS_TYPER_IDBITS(typer);
>> + its_data.dev_bits = GITS_TYPER_DEVBITS(typer);
>>
>> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
>> if ( !its->cmd_base )
>> @@ -1032,7 +1035,10 @@ int its_cpu_init(void)
>>
>> int __init its_init(struct rdist_prop *rdists)
>> {
>> + struct its_node *its;
>> + struct its_node_info *info;
>> struct dt_device_node *np = NULL;
>> + uint32_t i, nr_its = 0;
>>
>> static const struct dt_device_match its_device_ids[] __initconst =
>> {
>> @@ -1042,7 +1048,10 @@ int __init its_init(struct rdist_prop *rdists)
>>
>> for (np = dt_find_matching_node(NULL, its_device_ids); np;
>> np = dt_find_matching_node(np, its_device_ids))
>> - its_probe(np);
>> + {
>> + if ( !its_probe(np) )
>> + nr_its++;
>> + }
>> if ( list_empty(&its_nodes) )
>> {
>> @@ -1050,6 +1059,22 @@ int __init its_init(struct rdist_prop *rdists)
>> return -ENXIO;
>> }
>>
>> + info = xzalloc_array(struct its_node_info, nr_its);
>> + if ( !info )
>> + return -ENOMEM;
>> +
>> + i = 0;
>> + list_for_each_entry(its, &its_nodes, entry)
>> + {
>> + info[i].phys_base = its->phys_base;
>> + info[i].phys_size = its->phys_size;
>> + i++;
>> + }
>> +
>> + its_data.nr_its = nr_its;
>> + its_data.its_hw = info;
>> + vits_setup_hw(&its_data);
>> +
>
> The vITS will only expose 1 ITS to dom0. So there is no need to try to
> allocate memory in order to expose all the ITS.
>
> We can extend vits_setup_hw if we ever want to support more ITS in the
> future.
>
> Furthermore I'd like to see all the ITS data pass one by one rather than
> in a structure. This would help to catch error if we ever extend the
> structure.
>
> Something like:
>
> void vits_hw_setup(paddr_t its0_base, unsigned long its0_size,
> uint32_t eventID_bits, uint32_t devID_bits);
>
Passing every value as argument makes this function take too many arguments.
Validation can be done member variable to catch errors.
>> gic_rdists = rdists;
>> its_lpi_init(rdists->id_bits);
>> its_alloc_lpi_tables();
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index dfa3435..3a003d4 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -51,6 +51,15 @@ static void dump_cmd(its_cmd_block *cmd)
>> static void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
>> #endif
>>
>> +static struct {
>> + struct gic_its_info *info;
>> +} vits_hw;
>
> Please add a boolean valid in the structure. It would help to know if
> the vITS can be used or not later.
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS
2015-07-31 7:01 ` Vijay Kilari
@ 2015-08-03 15:58 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-03 15:58 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 31/07/15 08:01, Vijay Kilari wrote:
>> The vITS will only expose 1 ITS to dom0. So there is no need to try to
>> allocate memory in order to expose all the ITS.
>>
>> We can extend vits_setup_hw if we ever want to support more ITS in the
>> future.
>>
>> Furthermore I'd like to see all the ITS data pass one by one rather than
>> in a structure. This would help to catch error if we ever extend the
>> structure.
>>
>> Something like:
>>
>> void vits_hw_setup(paddr_t its0_base, unsigned long its0_size,
>> uint32_t eventID_bits, uint32_t devID_bits);
>>
>
> Passing every value as argument makes this function take too many arguments.
> Validation can be done member variable to catch errors.
Can you explain why you think that 4 arguments (based on my suggestion)
is too many? IHMO it's more readable to have the argument passed
explicitly rather than pack in a structure that would make the extension
more error-prone. FYI, we use 4 arguments for vgic_v3_hw_setup too.
Furthermore, you can't validate properly the field unless even if you
assume that the guest will zeroed the structure before passing it.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (8 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 09/22] xen/arm: ITS: Export ITS info to Virtual ITS vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-28 19:01 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 11/22] xen/arm: ITS: Enable physical and virtual ITS driver compilation vijay.kilari
` (11 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate GITS* registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v4: - Removed GICR register emulation
---
xen/arch/arm/irq.c | 3 +
xen/arch/arm/vgic-v3-its.c | 365 ++++++++++++++++++++++++++++++++++++++++-
xen/include/asm-arm/gic-its.h | 15 ++
xen/include/asm-arm/gic.h | 1 +
4 files changed, 381 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 1f38605..85cacb0 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,6 +31,9 @@
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
+/* Number of LPI supported in XEN */
+unsigned int num_of_lpis = 8192;
+
/* Describe an IRQ assigned to a guest */
struct irq_guest
{
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 3a003d4..1c7d9b6 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -33,8 +33,16 @@
#include <asm/atomic.h>
#include <xen/log2.h>
-#define DEBUG_ITS
-
+//#define DEBUG_ITS
+
+/* GITS_PIDRn register values for ARM implementations */
+#define GITS_PIDR0_VAL (0x94)
+#define GITS_PIDR1_VAL (0xb4)
+#define GITS_PIDR2_VAL (0x3b)
+#define GITS_PIDR3_VAL (0x00)
+#define GITS_PIDR4_VAL (0x04)
+#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
+ (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
#ifdef DEBUG_ITS
# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
#else
@@ -60,6 +68,14 @@ void vits_setup_hw(struct gic_its_info *its_info)
vits_hw.info = its_info;
}
+static inline uint32_t vits_get_max_collections(struct domain *d)
+{
+ /* Collection ID is only 16 bit */
+ ASSERT(d->max_vcpus < 256);
+
+ return (d->max_vcpus + 1);
+}
+
static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
@@ -502,7 +518,7 @@ static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
return 0;
}
-int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
+static int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
{
its_cmd_block virt_cmd;
@@ -527,11 +543,338 @@ err:
return 0;
}
+static inline uint32_t vits_get_word(uint32_t reg_offset, uint64_t val)
+{
+ if ( (reg_offset % 8) == 0 )
+ return (u32)val;
+ else
+ return (u32)(val >> 32);
+}
+
+static inline void vits_spin_lock(struct vgic_its *vits)
+{
+ spin_lock(&vits->lock);
+}
+
+static inline void vits_spin_unlock(struct vgic_its *vits)
+{
+ spin_unlock(&vits->lock);
+}
+
+static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ uint64_t val = 0;
+ uint32_t gits_reg;
+
+ gits_reg = info->gpa - vits->gits_base;
+ DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
+
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ *r = vits->ctrl | GITS_CTLR_QUIESCENT;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = GICV3_GICD_IIDR_VAL;
+ return 1;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /*
+ * GITS_TYPER.HCC = max_vcpus + 1 (max collection supported )
+ * GITS_TYPER.Devbits = HW supported Devbits size
+ * GITS_TYPER.IDbits = HW supported IDbits size
+ * GITS_TYPER.PTA = 0 ( Target addresses are linear processor numbers
+ * GITS_TYPER.ITTSize = Size of struct vitt
+ * GITS_TYPER.Physical = 1
+ */
+ if ( dabt.size != DABT_DOUBLE_WORD &&
+ dabt.size != DABT_WORD ) goto bad_width;
+ val = ((vits_get_max_collections(v->domain) << GITS_TYPER_HCC_SHIFT ) |
+ ((vits_hw.info->dev_bits - 1) << GITS_TYPER_DEVBITS_SHIFT) |
+ ((vits_hw.info->eventid_bits - 1) << GITS_TYPER_IDBITS_SHIFT) |
+ ((sizeof(struct vitt) - 1) << GITS_TYPER_ITT_SIZE_SHIFT) |
+ GITS_TYPER_PHYSICAL_LPIS);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else
+ *r = vits_get_word(gits_reg, val);
+ return 1;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- read ignored */
+ goto read_as_zero;
+ case GITS_CBASER:
+ case GITS_CBASER + 4:
+ /* Only read support 32/64-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD &&
+ dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_base;
+ else
+ *r = vits_get_word(gits_reg, vits->cmd_base);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ /* Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ val = vits->cmd_write;
+ vits_spin_unlock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else if (dabt.size == DABT_WORD )
+ *r = (u32)val;
+ else
+ goto bad_width;
+ return 1;
+ case GITS_CWRITER + 4:
+ /* BITS[63:20] are RES0 */
+ goto read_as_zero_32;
+ case GITS_CREADR:
+ /* Only read support 32/64-bit access */
+ val = atomic_read(&vits->cmd_read);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else if (dabt.size == DABT_WORD )
+ *r = (u32)val;
+ else
+ goto bad_width;
+ return 1;
+ case GITS_CREADR + 4:
+ /* BITS[63:20] are RES0 */
+ goto read_as_zero_32;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- read ignored */
+ goto read_as_zero;
+ case GITS_BASER0:
+ /* XXX: Support only 32-bit access */
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ {
+ vits_spin_lock(vits);
+ *r = vits->baser0;
+ vits_spin_unlock(vits);
+ }
+ else
+ goto bad_width;
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ goto read_as_zero;
+ case GITS_PIDR0:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR0_VAL;
+ return 1;
+ case GITS_PIDR1:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR1_VAL;
+ return 1;
+ case GITS_PIDR2:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR2_VAL;
+ return 1;
+ case GITS_PIDR3:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR3_VAL;
+ return 1;
+ case GITS_PIDR4:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR4_VAL;
+ return 1;
+ case GITS_PIDR5 ... GITS_PIDR7:
+ goto read_as_zero_32;
+ default:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: unhandled read r%"PRId32" offset 0x%#08"PRIx32"\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: bad read width %d r%"PRId32" offset 0x%#08"PRIx32"\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+read_as_zero_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+read_as_zero:
+ *r = 0;
+ return 1;
+}
+
+/*
+ * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
+ * and GITS_BASER.Shareability[11:10] are read-only.
+ * Mask those fields while emulating GITS_BASER reg.
+ */
+#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
+ (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
+ (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
+
+static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ int ret;
+ uint32_t gits_reg, sz, psz;
+
+ gits_reg = info->gpa - vits->gits_base;
+
+ DPRINTK("%pv: vITS: GITS_MMIO_WRITE offset 0x%"PRIx32"\n", v, gits_reg);
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->ctrl = *r & GITS_CTLR_ENABLE;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ /* RO -- write ignored */
+ goto write_ignore;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /* RO -- write ignored */
+ goto write_ignore;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- write ignored */
+ goto write_ignore;
+ case GITS_CBASER:
+ /* XXX: support 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ if ( vits->ctrl & GITS_CTLR_ENABLE )
+ {
+ /* RO -- write ignored */
+ vits_spin_unlock(vits);
+ goto write_ignore;
+ }
+ vits->cmd_base = *r;
+ vits->cmd_qsize = SZ_4K * ((*r & GITS_BASER_PAGES_MASK_VAL) + 1);
+ if ( vits->cmd_base & GITS_BASER_VALID )
+ atomic_set(&vits->cmd_read, 0);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ /* Only BITS[19:0] are writable */
+ vits->cmd_write = *r & 0xfffe0;
+ ret = 1;
+ if ( vits->ctrl & GITS_CTLR_ENABLE )
+ {
+ /* CWRITER should be within the range */
+ if ( vits->cmd_write < (vits->cmd_qsize & 0xfffe0) )
+ ret = vits_process_cmd(v, vits);
+ }
+ vits_spin_unlock(vits);
+ return ret;
+ case GITS_CWRITER + 4:
+ /* BITS[63:20] are RES0 */
+ goto write_ignore_32;
+ case GITS_CREADR:
+ /* RO -- write ignored */
+ goto write_ignore;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- write ignored */
+ goto write_ignore;
+ case GITS_BASER0:
+ /* XXX: Support 32-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ /* RO -- write ignored if GITS_CTLR.Enable = 1 */
+ if ( vits->ctrl & GITS_CTLR_ENABLE )
+ {
+ vits_spin_unlock(vits);
+ goto write_ignore_64;
+ }
+ vits->baser0 |= (*r & GITS_BASER_MASK);
+ vits->dt_ipa = vits->baser0 & MASK_4K;
+ psz = (vits->baser0 >> GITS_BASER_PAGE_SIZE_SHIFT) &
+ GITS_BASER_PAGE_SIZE_MASK_VAL;
+ if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
+ sz = 4;
+ else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
+ sz = 16;
+ else
+ /* 0x11 and 0x10 are treated as 64K size */
+ sz = 64;
+
+ vits->dt_size = (vits->baser0 & GITS_BASER_PAGES_MASK_VAL)
+ * sz * SZ_1K;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD )
+ goto bad_width;
+ goto write_ignore;
+ case GITS_PIDR7 ... GITS_PIDR0:
+ /* R0 -- write ignored */
+ goto write_ignore_32;
+ default:
+ dprintk(XENLOG_G_ERR,
+ "%pv vITS: unhandled write r%"PRId32" offset 0x%#08"PRIx32"\n",
+ v, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR,
+ "%pv: vITS: bad write width %d r%"PRId32" offset 0x%#08"PRIx32"\n",
+ v, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+write_ignore_64:
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ return 1;
+write_ignore_32:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ return 1;
+write_ignore:
+ return 1;
+}
+
+
+static const struct mmio_handler_ops vgic_gits_mmio_handler = {
+ .read_handler = vgic_v3_gits_mmio_read,
+ .write_handler = vgic_v3_gits_mmio_write,
+};
+
int vits_domain_init(struct domain *d)
{
struct vgic_its *vits;
int i;
+ if ( is_hardware_domain(d) )
+ d->arch.vgic.nr_lpis = num_of_lpis;
+ else
+ d->arch.vgic.nr_lpis = NR_LPIS;
+
d->arch.vgic.vits = xzalloc(struct vgic_its);
if ( !d->arch.vgic.vits )
return -ENOMEM;
@@ -539,6 +882,7 @@ int vits_domain_init(struct domain *d)
vits = d->arch.vgic.vits;
spin_lock_init(&vits->lock);
+ spin_lock_init(&vits->prop_lock);
vits->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
if ( !vits->collections )
@@ -550,6 +894,21 @@ int vits_domain_init(struct domain *d)
for ( i = 0; i < nr_cpu_ids; i++ )
vits->collections[i].target_address = ~0UL;
+ d->arch.vgic.id_bits = get_count_order(d->arch.vgic.nr_lpis + FIRST_GIC_LPI);
+ vits->baser0 = GITS_BASER_INIT_VAL;
+
+ if ( is_hardware_domain(d) )
+ {
+ /*
+ * Only one virtual ITS is provided to domain.
+ * Assign first physical ITS address to Dom0 virtual ITS.
+ */
+ vits->gits_base = vits_hw.info->its_hw[0].phys_base;
+ vits->gits_size = vits_hw.info->its_hw[0].phys_size;
+ }
+
+ register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
+
return 0;
}
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 23ff66c..0854cde 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -21,6 +21,9 @@
#include <asm/gic_v3_defs.h>
#include <xen/rbtree.h>
+/* Number of LPI supported */
+extern unsigned int num_of_lpis;
+
#define MASK_4K 0xfffffffff000UL
#define MAPC_ITT_IPA_SHIFT 8
/*
@@ -54,6 +57,8 @@
#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
#define GITS_TYPER_PTA (1UL << 19)
#define GITS_TYPER_HCC_SHIFT (24)
+#define GITS_TYPER_PHYSICAL_LPIS (1UL)
+#define GITS_TYPER_ITT_SIZE_SHIFT (4)
#define GITS_CBASER_VALID (1UL << 63)
#define GITS_CBASER_nC (1UL << 59)
@@ -80,6 +85,10 @@
#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_4K_VAL (0)
+#define GITS_BASER_PAGE_SIZE_16K_VAL (1)
+#define GITS_BASER_PAGE_SIZE_MASK_VAL (0x3)
+#define GITS_BASER_PAGES_MASK_VAL (0xff)
#define GITS_BASER_TYPE_NONE 0
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_VCPU 2
@@ -124,6 +133,8 @@ struct its_collection {
struct vgic_its
{
spinlock_t lock;
+ /* Emulation of BASER0 */
+ paddr_t baser0;
/* Command queue base */
paddr_t cmd_base;
/* Command queue write pointer */
@@ -132,6 +143,10 @@ struct vgic_its
atomic_t cmd_read;
/* Command queue size */
unsigned long cmd_qsize;
+ /* ITS mmio physical base */
+ paddr_t gits_base;
+ /* GICR ctrl register */
+ uint32_t ctrl;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index e330fe3..5f37f56 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -20,6 +20,7 @@
#define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
#define NR_GIC_SGI 16
+#define FIRST_GIC_LPI 8192
#define MAX_RDIST_COUNT 4
#define GICD_CTLR (0x000)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-07-27 11:11 ` [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-07-28 19:01 ` Julien Grall
2015-07-31 7:25 ` Vijay Kilari
2015-08-01 8:50 ` Vijay Kilari
0 siblings, 2 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-28 19:01 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate GITS* registers
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v4: - Removed GICR register emulation
> ---
> xen/arch/arm/irq.c | 3 +
> xen/arch/arm/vgic-v3-its.c | 365 ++++++++++++++++++++++++++++++++++++++++-
> xen/include/asm-arm/gic-its.h | 15 ++
> xen/include/asm-arm/gic.h | 1 +
> 4 files changed, 381 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 1f38605..85cacb0 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,6 +31,9 @@
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
>
> +/* Number of LPI supported in XEN */
> +unsigned int num_of_lpis = 8192;
> +
It makes little sense to introduce the support of LPIs in Xen in a patch
called "Add GITS registers emulation".
This should go in a specific ITS (not vITS) patch.
Furthermore, you need to explain where to the 8192 comes from...
Lastly I would rename num_of_lpis into nr_lpis.
> /* Describe an IRQ assigned to a guest */
> struct irq_guest
> {
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 3a003d4..1c7d9b6 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -33,8 +33,16 @@
> #include <asm/atomic.h>
> #include <xen/log2.h>
>
> -#define DEBUG_ITS
> -
> +//#define DEBUG_ITS
> +
This change should go in patch #8.
> +/* GITS_PIDRn register values for ARM implementations */
> +#define GITS_PIDR0_VAL (0x94)
> +#define GITS_PIDR1_VAL (0xb4)
> +#define GITS_PIDR2_VAL (0x3b)
> +#define GITS_PIDR3_VAL (0x00)
> +#define GITS_PIDR4_VAL (0x04)
> +#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
> + (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
> #ifdef DEBUG_ITS
> # define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> #else
> @@ -60,6 +68,14 @@ void vits_setup_hw(struct gic_its_info *its_info)
> vits_hw.info = its_info;
> }
>
> +static inline uint32_t vits_get_max_collections(struct domain *d)
> +{
> + /* Collection ID is only 16 bit */
16 bit = 65536 not 256.
You need to explain that the ITS is only supporting 256 collections in
hardware and that our implementation doesn't support memory provisioning
for collection.
Furthermore if the number of collection is based on 16 bits, the
function should return uint16_t not uint32_t.
> + ASSERT(d->max_vcpus < 256);
> +
Please add a comment to explain why d->max_vcpus + 1 with may a
reference to the public spec.
> + return (d->max_vcpus + 1);
> +}
> +
> static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
> uint32_t size, bool_t set)
> {
> @@ -502,7 +518,7 @@ static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
> return 0;
> }
>
> -int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
> +static int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
Please, Move the static where the function has been defined.
> {
> its_cmd_block virt_cmd;
>
> @@ -527,11 +543,338 @@ err:
> return 0;
> }
[..]
> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint64_t val = 0;
> + uint32_t gits_reg;
> +
> + gits_reg = info->gpa - vits->gits_base;
> + DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + *r = vits->ctrl | GITS_CTLR_QUIESCENT;
Why did you put GITS_CTLR_QUIESCENT?
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_IIDR:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + *r = GICV3_GICD_IIDR_VAL;
> + return 1;
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + /*
> + * GITS_TYPER.HCC = max_vcpus + 1 (max collection supported )
> + * GITS_TYPER.Devbits = HW supported Devbits size
> + * GITS_TYPER.IDbits = HW supported IDbits size
> + * GITS_TYPER.PTA = 0 ( Target addresses are linear processor numbers
Missing )
> + * GITS_TYPER.ITTSize = Size of struct vitt
> + * GITS_TYPER.Physical = 1
> + */
> + if ( dabt.size != DABT_DOUBLE_WORD &&
> + dabt.size != DABT_WORD ) goto bad_width;
> + val = ((vits_get_max_collections(v->domain) << GITS_TYPER_HCC_SHIFT ) |
> + ((vits_hw.info->dev_bits - 1) << GITS_TYPER_DEVBITS_SHIFT) |
> + ((vits_hw.info->eventid_bits - 1) << GITS_TYPER_IDBITS_SHIFT) |
> + ((sizeof(struct vitt) - 1) << GITS_TYPER_ITT_SIZE_SHIFT) |
> + GITS_TYPER_PHYSICAL_LPIS);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = val;
> + else
> + *r = vits_get_word(gits_reg, val);
> + return 1;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- read ignored */
> + goto read_as_zero;
> + case GITS_CBASER:
> + case GITS_CBASER + 4:
> + /* Only read support 32/64-bit access */
"Read supports only 32/64-bit access"
> + if ( dabt.size != DABT_DOUBLE_WORD &&
> + dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_base;
> + else
> + *r = vits_get_word(gits_reg, vits->cmd_base);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER:
> + /* Only read support 32/64-bit access */
Ditto
> + vits_spin_lock(vits);
> + val = vits->cmd_write;
> + vits_spin_unlock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = val;
> + else if (dabt.size == DABT_WORD )
> + *r = (u32)val;
> + else
> + goto bad_width;
> + return 1;
> + case GITS_CWRITER + 4:
> + /* BITS[63:20] are RES0 */
> + goto read_as_zero_32;
> + case GITS_CREADR:
> + /* Only read support 32/64-bit access */
Ditto
> + val = atomic_read(&vits->cmd_read);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = val;
> + else if (dabt.size == DABT_WORD )
> + *r = (u32)val;
> + else
> + goto bad_width;
> + return 1;
> + case GITS_CREADR + 4:
> + /* BITS[63:20] are RES0 */
> + goto read_as_zero_32;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- read ignored */
> + goto read_as_zero;
> + case GITS_BASER0:
> + /* XXX: Support only 32-bit access */
As said on v4, this comment is wrong. You only support 64-bit.
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + {
> + vits_spin_lock(vits);
> + *r = vits->baser0;
> + vits_spin_unlock(vits);
> + }
> + else
> + goto bad_width;
I would prefer to see
if ( dabt.size != DATB_DOUBLE_WORD )
goto bad_width;
vits_spin_lock(...)
To avoid one indentation layer more for most of the lines and keep the
error before.
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
> + goto read_as_zero;
> + case GITS_PIDR0:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR0_VAL;
> + return 1;
> + case GITS_PIDR1:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR1_VAL;
> + return 1;
> + case GITS_PIDR2:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR2_VAL;
> + return 1;
> + case GITS_PIDR3:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR3_VAL;
> + return 1;
> + case GITS_PIDR4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR4_VAL;
> + return 1;
> + case GITS_PIDR5 ... GITS_PIDR7:
> + goto read_as_zero_32;
> + default:
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: unhandled read r%"PRId32" offset 0x%#08"PRIx32"\n",
Reg is definitely not a PRId32.
> + v, dabt.reg, gits_reg);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR,
> + "%pv: vITS: bad read width %d r%"PRId32" offset 0x%#08"PRIx32"\n",
Ditto.
> + v, dabt.size, dabt.reg, gits_reg);
> + domain_crash_synchronous();
> + return 0;
> +
> +read_as_zero_32:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> +read_as_zero:
> + *r = 0;
> + return 1;
> +}
> +
> +/*
> + * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
> + * and GITS_BASER.Shareability[11:10] are read-only.
As said on v4, implemented Shareability as fixed (i.e read-only) is
deprecated. I'd like to see a TODO here.
> + * Mask those fields while emulating GITS_BASER reg.
> + */
As said on v4,
Other fields are (or could be RO) in GITS_BASER:
- Indirect: we only support flat table
- Page_Size: it's fine to only support 4KB granularity. It also
means less code.
I don't mind if you don't do the latter. The former is a mandatory.
> +#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
> + (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
> + (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
[..]
> + case GITS_CWRITER:
> + if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + /* Only BITS[19:0] are writable */
> + vits->cmd_write = *r & 0xfffe0;
> + ret = 1;
> + if ( vits->ctrl & GITS_CTLR_ENABLE )
> + {
> + /* CWRITER should be within the range */
> + if ( vits->cmd_write < (vits->cmd_qsize & 0xfffe0) )
> + ret = vits_process_cmd(v, vits);
> + }
Please do only 1 if rather than 2 nested if for only one line.
> + vits_spin_unlock(vits);
> + return ret;
[..]
> int vits_domain_init(struct domain *d)
> {
> struct vgic_its *vits;
> int i;
>
> + if ( is_hardware_domain(d) )
> + d->arch.vgic.nr_lpis = num_of_lpis;
> + else
> + d->arch.vgic.nr_lpis = NR_LPIS;
NR_LPIS is defined in patch #14. And the name seems to be wrong.
Anyway, I don't understand why you are trying to initialize vITS on
guest. We agree that it should only be used on DOM0 for now until we
effectively need it for the guest.
Furthermore, it miss at least the toolstack in order to get the part
guest ready.
So please ensure that the vITS is not initialized for the guest.
> +
> d->arch.vgic.vits = xzalloc(struct vgic_its);
> if ( !d->arch.vgic.vits )
> return -ENOMEM;
> @@ -539,6 +882,7 @@ int vits_domain_init(struct domain *d)
> vits = d->arch.vgic.vits;
>
> spin_lock_init(&vits->lock);
> + spin_lock_init(&vits->prop_lock);
The field prop_lock is added in patch #12. Please move the
spin_lock_init there.
To remind you, *every* patch should be able to compile one by one on
both aarch64, and arm32. I suspect this is not currently the case.
>
> vits->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
> if ( !vits->collections )
> @@ -550,6 +894,21 @@ int vits_domain_init(struct domain *d)
> for ( i = 0; i < nr_cpu_ids; i++ )
> vits->collections[i].target_address = ~0UL;
>
> + d->arch.vgic.id_bits = get_count_order(d->arch.vgic.nr_lpis + FIRST_GIC_LPI);
ID bits is defined in patch #12. And you don't even use it here...
> + vits->baser0 = GITS_BASER_INIT_VAL;
> +
> + if ( is_hardware_domain(d) )
> + {
> + /*
> + * Only one virtual ITS is provided to domain.
> + * Assign first physical ITS address to Dom0 virtual ITS.
> + */
> + vits->gits_base = vits_hw.info->its_hw[0].phys_base;
> + vits->gits_size = vits_hw.info->its_hw[0].phys_size;
> + }
Here another example where the code for the guest is missing. Please be
consistent i.e either try to support it at the whole or not at all.
The latter seems the more plausible for now.
> +
> + register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
> +
> return 0;
> }
>
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 23ff66c..0854cde 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -21,6 +21,9 @@
> #include <asm/gic_v3_defs.h>
> #include <xen/rbtree.h>
>
> +/* Number of LPI supported */
> +extern unsigned int num_of_lpis;
> +
The prototype of anything declared in irq.c should go in irq.h and not
in the GIC ITS header.
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index e330fe3..5f37f56 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -20,6 +20,7 @@
>
> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
> #define NR_GIC_SGI 16
> +#define FIRST_GIC_LPI 8192
It make little sense to define it in a patch called "Add GITS registers
emulation".
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-07-28 19:01 ` Julien Grall
@ 2015-07-31 7:25 ` Vijay Kilari
2015-07-31 10:28 ` Julien Grall
2015-08-01 8:50 ` Vijay Kilari
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 7:25 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Wed, Jul 29, 2015 at 12:31 AM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Emulate GITS* registers
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v4: - Removed GICR register emulation
>> ---
>> xen/arch/arm/irq.c | 3 +
>> xen/arch/arm/vgic-v3-its.c | 365 ++++++++++++++++++++++++++++++++++++++++-
>> xen/include/asm-arm/gic-its.h | 15 ++
>> xen/include/asm-arm/gic.h | 1 +
>> 4 files changed, 381 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 1f38605..85cacb0 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -31,6 +31,9 @@
>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>>
>> +/* Number of LPI supported in XEN */
>> +unsigned int num_of_lpis = 8192;
>> +
>
> It makes little sense to introduce the support of LPIs in Xen in a patch
> called "Add GITS registers emulation".
>
> This should go in a specific ITS (not vITS) patch.
>
> Furthermore, you need to explain where to the 8192 comes from...
Two reasons for setting to 8192
1) Being LPI starts from 8192 the 8192 is 13 and next if id_bits is 14
then it can hold
upto 16K. So 16K-8K = 8K
2) ThunderX requires more than 4K due to large number of PCIe devices
>
> Lastly I would rename num_of_lpis into nr_lpis.
>
>> /* Describe an IRQ assigned to a guest */
>> struct irq_guest
>> {
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 3a003d4..1c7d9b6 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>> @@ -33,8 +33,16 @@
>> #include <asm/atomic.h>
>> #include <xen/log2.h>
>>
>> -#define DEBUG_ITS
>> -
>> +//#define DEBUG_ITS
>> +
>
> This change should go in patch #8.
>
>> +/* GITS_PIDRn register values for ARM implementations */
>> +#define GITS_PIDR0_VAL (0x94)
>> +#define GITS_PIDR1_VAL (0xb4)
>> +#define GITS_PIDR2_VAL (0x3b)
>> +#define GITS_PIDR3_VAL (0x00)
>> +#define GITS_PIDR4_VAL (0x04)
>> +#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
>> + (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
>> #ifdef DEBUG_ITS
>> # define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
>> #else
>> @@ -60,6 +68,14 @@ void vits_setup_hw(struct gic_its_info *its_info)
>> vits_hw.info = its_info;
>> }
>>
>> +static inline uint32_t vits_get_max_collections(struct domain *d)
>> +{
>> + /* Collection ID is only 16 bit */
>
> 16 bit = 65536 not 256.
I will correct the comment
>
> You need to explain that the ITS is only supporting 256 collections in
> hardware and that our implementation doesn't support memory provisioning
> for collection.
>
> Furthermore if the number of collection is based on 16 bits, the
> function should return uint16_t not uint32_t.
>
>
>> + ASSERT(d->max_vcpus < 256);
>> +
>
> Please add a comment to explain why d->max_vcpus + 1 with may a
> reference to the public spec.
>
>> + return (d->max_vcpus + 1);
>> +}
>> +
>> static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
>> uint32_t size, bool_t set)
>> {
>> @@ -502,7 +518,7 @@ static int vits_read_virt_cmd(struct vcpu *v, struct vgic_its *vits,
>> return 0;
>> }
>>
>> -int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
>> +static int vits_process_cmd(struct vcpu *v, struct vgic_its *vits)
>
> Please, Move the static where the function has been defined.
>
>> {
>> its_cmd_block virt_cmd;
>>
>> @@ -527,11 +543,338 @@ err:
>> return 0;
>> }
>
> [..]
>
>> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
>> +{
>> + struct vgic_its *vits = v->domain->arch.vgic.vits;
>> + struct hsr_dabt dabt = info->dabt;
>> + struct cpu_user_regs *regs = guest_cpu_user_regs();
>> + register_t *r = select_user_reg(regs, dabt.reg);
>> + uint64_t val = 0;
>> + uint32_t gits_reg;
>> +
>> + gits_reg = info->gpa - vits->gits_base;
>> + DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
>> +
>> + switch ( gits_reg )
>> + {
>> + case GITS_CTLR:
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + vits_spin_lock(vits);
>> + *r = vits->ctrl | GITS_CTLR_QUIESCENT;
>
> Why did you put GITS_CTLR_QUIESCENT?
The ITS is quiescent, has no translations in progress and has
completed all operations.
So I have set quescent by default.
[...]
>
>> + return 1;
>> + case GITS_BASER1 ... GITS_BASERN:
>> + goto read_as_zero;
>> + case GITS_PIDR0:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR0_VAL;
>> + return 1;
>> + case GITS_PIDR1:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR1_VAL;
>> + return 1;
>> + case GITS_PIDR2:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR2_VAL;
>> + return 1;
>> + case GITS_PIDR3:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR3_VAL;
>> + return 1;
>> + case GITS_PIDR4:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR4_VAL;
>> + return 1;
>> + case GITS_PIDR5 ... GITS_PIDR7:
>> + goto read_as_zero_32;
>> + default:
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: unhandled read r%"PRId32" offset 0x%#08"PRIx32"\n",
>
> Reg is definitely not a PRId32.
>
>> + v, dabt.reg, gits_reg);
>> + return 0;
>> + }
>> +
>> +bad_width:
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: bad read width %d r%"PRId32" offset 0x%#08"PRIx32"\n",
>
> Ditto.
>
>> + v, dabt.size, dabt.reg, gits_reg);
>> + domain_crash_synchronous();
>> + return 0;
>> +
>> +read_as_zero_32:
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> +read_as_zero:
>> + *r = 0;
>> + return 1;
>> +}
>> +
>> +/*
>> + * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
>> + * and GITS_BASER.Shareability[11:10] are read-only.
>
> As said on v4, implemented Shareability as fixed (i.e read-only) is
> deprecated. I'd like to see a TODO here.
>
>> + * Mask those fields while emulating GITS_BASER reg.
>> + */
>
> As said on v4,
>
> Other fields are (or could be RO) in GITS_BASER:
> - Indirect: we only support flat table
By default it is 0. So support flat table. Do you want it explicitly
specify Indirect?
> - Page_Size: it's fine to only support 4KB granularity. It also
> means less code.
Page_size is set by guest. this is not RO
>
> I don't mind if you don't do the latter. The former is a mandatory.
>
>> +#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
>> + (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
>> + (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
>
> [..]
>
>> + case GITS_CWRITER:
>> + if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD )
>> + goto bad_width;
>> + vits_spin_lock(vits);
>> + /* Only BITS[19:0] are writable */
>> + vits->cmd_write = *r & 0xfffe0;
>> + ret = 1;
>> + if ( vits->ctrl & GITS_CTLR_ENABLE )
>> + {
>> + /* CWRITER should be within the range */
>> + if ( vits->cmd_write < (vits->cmd_qsize & 0xfffe0) )
>> + ret = vits_process_cmd(v, vits);
>> + }
>
> Please do only 1 if rather than 2 nested if for only one line.
>
>> + vits_spin_unlock(vits);
>> + return ret;
>
> [..]
>
>> int vits_domain_init(struct domain *d)
>> {
>> struct vgic_its *vits;
>> int i;
>>
>> + if ( is_hardware_domain(d) )
>> + d->arch.vgic.nr_lpis = num_of_lpis;
>> + else
>> + d->arch.vgic.nr_lpis = NR_LPIS;
>
> NR_LPIS is defined in patch #14. And the name seems to be wrong.
>
> Anyway, I don't understand why you are trying to initialize vITS on
> guest. We agree that it should only be used on DOM0 for now until we
> effectively need it for the guest.
>
> Furthermore, it miss at least the toolstack in order to get the part
> guest ready.
>
> So please ensure that the vITS is not initialized for the guest.
In patch#18, this function is called only for DOM0
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-07-31 7:25 ` Vijay Kilari
@ 2015-07-31 10:28 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-31 10:28 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 31/07/15 08:25, Vijay Kilari wrote:
> On Wed, Jul 29, 2015 at 12:31 AM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Emulate GITS* registers
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>> ---
>>> v4: - Removed GICR register emulation
>>> ---
>>> xen/arch/arm/irq.c | 3 +
>>> xen/arch/arm/vgic-v3-its.c | 365 ++++++++++++++++++++++++++++++++++++++++-
>>> xen/include/asm-arm/gic-its.h | 15 ++
>>> xen/include/asm-arm/gic.h | 1 +
>>> 4 files changed, 381 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>> index 1f38605..85cacb0 100644
>>> --- a/xen/arch/arm/irq.c
>>> +++ b/xen/arch/arm/irq.c
>>> @@ -31,6 +31,9 @@
>>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>>>
>>> +/* Number of LPI supported in XEN */
>>> +unsigned int num_of_lpis = 8192;
>>> +
>>
>> It makes little sense to introduce the support of LPIs in Xen in a patch
>> called "Add GITS registers emulation".
>>
>> This should go in a specific ITS (not vITS) patch.
>>
>> Furthermore, you need to explain where to the 8192 comes from...
>
> Two reasons for setting to 8192
>
> 1) Being LPI starts from 8192 the 8192 is 13 and next if id_bits is 14
> then it can hold
> upto 16K. So 16K-8K = 8K
This is a valid reason. Please add it in the code.
[..]
>>> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
>>> +{
>>> + struct vgic_its *vits = v->domain->arch.vgic.vits;
>>> + struct hsr_dabt dabt = info->dabt;
>>> + struct cpu_user_regs *regs = guest_cpu_user_regs();
>>> + register_t *r = select_user_reg(regs, dabt.reg);
>>> + uint64_t val = 0;
>>> + uint32_t gits_reg;
>>> +
>>> + gits_reg = info->gpa - vits->gits_base;
>>> + DPRINTK("%pv: vITS: GITS_MMIO_READ offset 0x%"PRIx32"\n", v, gits_reg);
>>> +
>>> + switch ( gits_reg )
>>> + {
>>> + case GITS_CTLR:
>>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>>> + vits_spin_lock(vits);
>>> + *r = vits->ctrl | GITS_CTLR_QUIESCENT;
>>
>> Why did you put GITS_CTLR_QUIESCENT?
>
> The ITS is quiescent, has no translations in progress and has
> completed all operations.
> So I have set quescent by default.
This needs to be explained in the code.
[..]
>>> + * Mask those fields while emulating GITS_BASER reg.
>>> + */
>>
>> As said on v4,
>>
>> Other fields are (or could be RO) in GITS_BASER:
>> - Indirect: we only support flat table
> By default it is 0. So support flat table. Do you want it explicitly
> specify Indirect?
It's not what I said ... Your implementation in the VITS *only* supports
flat-table (i.e Single Level). You are currently allowing the guest to
set this bit to 1 which means that it may use Two Level.
This won't work with your implementation. So this field *should* be RAZ/WI.
>
>> - Page_Size: it's fine to only support 4KB granularity. It also
>> means less code.
> Page_size is set by guest. this is not RO
Please read the spec (8.19.1 in ARM IHI 0069A): "If the GIC
implementation supports only a single, fixed page size, this field might
be RO."
If you look closely to the Linux code, if it can't set the Page size it
will retry with a small granularity.
Implementing it as RO would have been fine and save a bit of checking.
Anyway, as said this was only a suggestion.
[..]
>>> int vits_domain_init(struct domain *d)
>>> {
>>> struct vgic_its *vits;
>>> int i;
>>>
>>> + if ( is_hardware_domain(d) )
>>> + d->arch.vgic.nr_lpis = num_of_lpis;
>>> + else
>>> + d->arch.vgic.nr_lpis = NR_LPIS;
>>
>> NR_LPIS is defined in patch #14. And the name seems to be wrong.
>>
>> Anyway, I don't understand why you are trying to initialize vITS on
>> guest. We agree that it should only be used on DOM0 for now until we
>> effectively need it for the guest.
>>
>> Furthermore, it miss at least the toolstack in order to get the part
>> guest ready.
>>
>> So please ensure that the vITS is not initialized for the guest.
>
> In patch#18, this function is called only for DOM0
So why do try to half support vITS for guest? As said, there is lots of
things missing to get the support done for guest (toolstack, ITS
mapping,...).
Please drop all the references to the guest in vits_domain_init and add
an ASSERT to check that vits_init_domain is not called for guest.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-07-28 19:01 ` Julien Grall
2015-07-31 7:25 ` Vijay Kilari
@ 2015-08-01 8:50 ` Vijay Kilari
2015-08-03 11:19 ` Julien Grall
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-01 8:50 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Wed, Jul 29, 2015 at 12:31 AM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Emulate GITS* registers
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v4: - Removed GICR register emulation
>> ---
[...]
>> + case GITS_PIDR4:
>> + if ( dabt.size != DABT_WORD )
>> + goto bad_width;
>> + *r = GITS_PIDR4_VAL;
>> + return 1;
>> + case GITS_PIDR5 ... GITS_PIDR7:
>> + goto read_as_zero_32;
>> + default:
>> + dprintk(XENLOG_G_ERR,
>> + "%pv: vITS: unhandled read r%"PRId32" offset 0x%#08"PRIx32"\n",
>
> Reg is definitely not a PRId32.
dabt.reg is unsigned long : 5. In vgic-v3.c it is printed using %d.
[...]
>> +/*
>> + * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
>> + * and GITS_BASER.Shareability[11:10] are read-only.
>
> As said on v4, implemented Shareability as fixed (i.e read-only) is
> deprecated. I'd like to see a TODO here.
What are you expecting to get fixed for shareability with TODO?
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation
2015-08-01 8:50 ` Vijay Kilari
@ 2015-08-03 11:19 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-03 11:19 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 01/08/15 09:50, Vijay Kilari wrote:
> On Wed, Jul 29, 2015 at 12:31 AM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Emulate GITS* registers
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>> ---
>>> v4: - Removed GICR register emulation
>>> ---
> [...]
>>> + case GITS_PIDR4:
>>> + if ( dabt.size != DABT_WORD )
>>> + goto bad_width;
>>> + *r = GITS_PIDR4_VAL;
>>> + return 1;
>>> + case GITS_PIDR5 ... GITS_PIDR7:
>>> + goto read_as_zero_32;
>>> + default:
>>> + dprintk(XENLOG_G_ERR,
>>> + "%pv: vITS: unhandled read r%"PRId32" offset 0x%#08"PRIx32"\n",
>>
>> Reg is definitely not a PRId32.
>
> dabt.reg is unsigned long : 5. In vgic-v3.c it is printed using %d.
> [...]
>>> +/*
>>> + * GITS_BASER.Type[58:56], GITS_BASER.Entry_size[55:48]
>>> + * and GITS_BASER.Shareability[11:10] are read-only.
>>
>> As said on v4, implemented Shareability as fixed (i.e read-only) is
>> deprecated. I'd like to see a TODO here.
>
> What are you expecting to get fixed for shareability with TODO?
To be more explicit, you are implementing a behavior which is
deprecated. It means that we should avoid to use it knowingly in the
virtual ITS.
In this specific case, I would be ok with a TODO explaining that we need
to fix it or why we need to do read-only.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 11/22] xen/arm: ITS: Enable physical and virtual ITS driver compilation
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (9 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 10/22] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-27 11:11 ` [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation vijay.kilari
` (10 subsequent siblings)
21 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Compilation is delayed till this patch.
>From now on functions in physical ITS and virtual ITS
driver are required. So enable compilation
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1ef39f7..14cbf12 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -14,6 +14,7 @@ obj-y += domain_build.o
obj-y += gic.o gic-v2.o
obj-$(CONFIG_ARM_32) += gic-hip04.o
obj-$(HAS_GICV3) += gic-v3.o
+obj-$(HAS_GICV3) += gic-v3-its.o
obj-y += io.o
obj-y += irq.o
obj-y += kernel.o
@@ -32,6 +33,7 @@ obj-y += shutdown.o
obj-y += traps.o
obj-y += vgic.o vgic-v2.o
obj-$(CONFIG_ARM_64) += vgic-v3.o
+obj-$(HAS_GICV3) += vgic-v3-its.o
obj-y += vtimer.o
obj-y += vuart.o
obj-y += hvm.o
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (10 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 11/22] xen/arm: ITS: Enable physical and virtual ITS driver compilation vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-30 17:04 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 13/22] xen/arm: ITS: Implement gic_is_lpi helper function vijay.kilari
` (9 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate LPI related changes to GICR registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Handled all sizes access to LPI configuration table
- Rename vits_unmap_lpi_prop as vits_map_lpi_prop
v4: - Added LPI configuration table emulation
- Rename function inline with vits
- Copied guest lpi configuration table to xen
---
xen/arch/arm/gic-v3.c | 10 +++
xen/arch/arm/gic.c | 5 ++
xen/arch/arm/vgic-v3-its.c | 168 +++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 104 +++++++++++++++++++----
xen/include/asm-arm/domain.h | 3 +
xen/include/asm-arm/gic-its.h | 12 +++
xen/include/asm-arm/gic.h | 5 ++
xen/include/asm-arm/gic_v3_defs.h | 1 +
8 files changed, 294 insertions(+), 14 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index f2784a2..91c1b74 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -681,6 +681,11 @@ static int __init gicv3_populate_rdist(void)
return -ENODEV;
}
+static int gicv3_dist_supports_lpis(void)
+{
+ return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
+}
+
static int __cpuinit gicv3_cpu_init(void)
{
int i;
@@ -1274,6 +1279,11 @@ static int __init gicv3_init(void)
spin_lock(&gicv3.lock);
+ if ( gicv3_dist_supports_lpis() )
+ gicv3_info.lpi_supported = 1;
+ else
+ gicv3_info.lpi_supported = 0;
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 1757193..af8a34b 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -67,6 +67,11 @@ unsigned int gic_number_lines(void)
return gic_hw_ops->info->nr_lines;
}
+bool_t gic_lpi_supported(void)
+{
+ return gic_hw_ops->info->lpi_supported;
+}
+
void gic_save_state(struct vcpu *v)
{
ASSERT(!local_irq_is_enabled());
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 1c7d9b6..4afb62b 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -28,6 +28,7 @@
#include <asm/io.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
#include <asm/gic-its.h>
#include <asm/atomic.h>
@@ -76,6 +77,34 @@ static inline uint32_t vits_get_max_collections(struct domain *d)
return (d->max_vcpus + 1);
}
+static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
+{
+ struct pending_irq *p;
+
+ p = irq_to_pending(v, vlpi);
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ gic_remove_from_queues(v, vlpi);
+}
+
+static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+
+ p = irq_to_pending(v, vlpi);
+
+ set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ /*XXX: raise on right vcpu */
+ if ( !list_empty(&p->inflight) &&
+ !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ gic_raise_guest_irq(v, vlpi, p->priority);
+
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+}
+
static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
uint32_t size, bool_t set)
{
@@ -551,6 +580,145 @@ static inline uint32_t vits_get_word(uint32_t reg_offset, uint64_t val)
return (u32)(val >> 32);
}
+static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa - (vits->propbase & MASK_4K);
+
+ DPRINTK("%pv: vITS: LPI Table read offset 0x%"PRIx32"\n", v, offset);
+ spin_lock(&vits->prop_lock);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = *((u64*)vits->prop_page + offset);
+ else if (dabt.size == DABT_WORD )
+ *r = *((u32*)vits->prop_page + offset);
+ else if (dabt.size == DABT_HALF_WORD )
+ *r = *((u16*)vits->prop_page + offset);
+ else
+ *r = *((u8*)vits->prop_page + offset);
+ spin_unlock(&vits->prop_lock);
+
+ return 1;
+}
+
+static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset, vid;
+ uint8_t cfg, *p, i, iter;
+ bool_t enable;
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa - (vits->propbase & MASK_4K);
+
+ DPRINTK("%pv: vITS: LPI Table write offset 0x%"PRIx32"\n", v, offset);
+
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ iter = 8;
+ else if ( dabt.size == DABT_WORD )
+ iter = 4;
+ else if ( dabt.size == DABT_HALF_WORD )
+ iter = 2;
+ else
+ iter = 1;
+
+ spin_lock(&vits->prop_lock);
+ p = ((u8*)vits->prop_page + offset);
+
+ vid = offset + FIRST_GIC_LPI;
+ for ( i = 0 ; i < iter; i++ )
+ {
+ cfg = *p;
+ enable = (cfg & *r) & 0x1;
+
+ if ( !enable )
+ vits_enable_lpi(v, vid, (*r & LPI_PRIORITY_MASK));
+ else
+ vits_disable_lpi(v, vid);
+
+ /* Update virtual prop page */
+ *p = (*r & 0xff);
+ p++;
+ vid++;
+ }
+ spin_unlock(&vits->prop_lock);
+
+ return 1;
+}
+
+static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
+ .read_handler = vgic_v3_gits_lpi_mmio_read,
+ .write_handler = vgic_v3_gits_lpi_mmio_write,
+};
+
+int vits_map_lpi_prop(struct domain *d)
+{
+ struct vgic_its *vits = d->arch.vgic.vits;
+ paddr_t gaddr, addr;
+ unsigned long mfn;
+ uint32_t lpi_size, id_bits;
+ int i;
+
+ gaddr = vits->propbase & MASK_4K;
+ id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
+
+ if ( id_bits > d->arch.vgic.id_bits )
+ id_bits = d->arch.vgic.id_bits;
+
+ lpi_size = 1UL << id_bits;
+
+ DPRINTK("vITS:Map LPI conf table @ 0x%"PRIx64" size 0x%"PRIx32"\n",
+ gaddr, lpi_size);
+
+ vits->prop_size = lpi_size;
+ /* Allocate Virtual LPI Property table */
+ /* TODO: To re-use guest property table? */
+ vits->prop_page = alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
+ if ( !vits->prop_page )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to allocate LPI Prop page\n", current);
+ return 0;
+ }
+
+ addr = gaddr;
+ for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
+ {
+ vits_access_guest_table(d, addr,
+ (void *)(vits->prop_page + i * PAGE_SIZE),
+ PAGE_SIZE, 0);
+ addr += PAGE_SIZE;
+ }
+
+ /*
+ * Each re-distributor shares a common LPI configuration table
+ * So one set of mmio handlers to manage configuration table is enough
+ */
+ addr = gaddr;
+ for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
+ {
+ mfn = gmfn_to_mfn(d, paddr_to_pfn(addr));
+ if ( unlikely(!mfn_valid(mfn)) )
+ {
+ dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid propbaser address\n", current);
+ return 0;
+ }
+ guest_physmap_remove_page(d, paddr_to_pfn(addr), mfn, 0);
+ addr += PAGE_SIZE;
+ }
+
+ /* Register mmio handlers for this region */
+ register_mmio_handler(d, &vgic_gits_lpi_mmio_handler,
+ gaddr, lpi_size);
+
+ return 1;
+}
+
static inline void vits_spin_lock(struct vgic_its *vits)
{
spin_lock(&vits->lock);
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 683e3cc..a466a8f 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -29,6 +29,8 @@
#include <asm/current.h>
#include <asm/mmio.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
/* GICD_PIDRn register values for ARM implementations */
@@ -109,29 +111,47 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
struct hsr_dabt dabt = info->dabt;
struct cpu_user_regs *regs = guest_cpu_user_regs();
register_t *r = select_user_reg(regs, dabt.reg);
- uint64_t aff;
+ uint64_t aff, val;
switch ( gicr_reg )
{
case GICR_CTLR:
- /* We have not implemented LPI's, read zero */
- goto read_as_zero_32;
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ *r = v->domain->arch.vgic.gicr_ctlr;
+ vgic_unlock(v);
+ return 1;
case GICR_IIDR:
if ( dabt.size != DABT_WORD ) goto bad_width;
*r = GICV3_GICR_IIDR_VAL;
return 1;
case GICR_TYPER:
- if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
- /* TBD: Update processor id in [23:8] when ITS support is added */
+ case GICR_TYPER + 4:
+ if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD)
+ goto bad_width;
aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
- *r = aff;
-
+ val = aff;
+ if ( gic_lpi_supported() )
+ {
+ /* Set Physical LPIs support */
+ val |= GICR_TYPER_PLPIS;
+ /* GITS_TYPER.PTA is 0. Provide vcpu number as target address */
+ val |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
+ }
if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
- *r |= GICR_TYPER_LAST;
-
+ val |= GICR_TYPER_LAST;
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else
+ {
+ if ( gicr_reg == GICR_TYPER)
+ *r = (u32)val;
+ else
+ *r = (u32)(val >> 32);
+ }
return 1;
case GICR_STATUSR:
/* Not implemented */
@@ -146,10 +166,20 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
/* WO. Read as zero */
goto read_as_zero_64;
case GICR_PROPBASER:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ *r = v->domain->arch.vgic.vits->propbase;
+ return 1;
+ }
goto read_as_zero_64;
case GICR_PENDBASER:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ *r = v->domain->arch.vgic.vits->pendbase[v->vcpu_id];
+ return 1;
+ }
goto read_as_zero_64;
case GICR_INVLPIR:
/* WO. Read as zero */
@@ -224,7 +254,18 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* LPI's not implemented */
+ if ( gic_lpi_supported() )
+ {
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR is not supported.
+ */
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vgic.gicr_ctlr = (*r) & GICR_CTLR_ENABLE_LPIS;
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_32;
case GICR_IIDR:
/* RO */
@@ -245,10 +286,37 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
/* LPI is not implemented */
goto write_ignore_64;
case GICR_PROPBASER:
- /* LPI is not implemented */
+ if ( gic_lpi_supported() )
+ {
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ /* LPI configuration tables are shared across cpus. Should be same */
+ /*
+ * Allow updating on propbase only once with below check.
+ * TODO: Manage change in property table.
+ */
+ if ( v->domain->arch.vgic.vits->propbase != 0 )
+ {
+ printk(XENLOG_G_WARNING
+ "%pv: vGICR: Updating LPI propbase is not allowed\n", v);
+ vgic_unlock(v);
+ return 1;
+ }
+ v->domain->arch.vgic.vits->propbase = *r;
+ vgic_unlock(v);
+ return vits_map_lpi_prop(v->domain);
+ }
goto write_ignore_64;
case GICR_PENDBASER:
- /* LPI is not implemented */
+ if ( gic_lpi_supported() )
+ {
+ /* Just hold pendbaser value for guest read */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vgic.vits->pendbase[v->vcpu_id] = *r;
+ vgic_unlock(v);
+ return 1;
+ }
goto write_ignore_64;
case GICR_INVLPIR:
/* LPI is not implemented */
@@ -715,6 +783,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
*r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
+ if ( gic_lpi_supported() )
+ {
+ irq_bits = v->domain->arch.vgic.id_bits;
+ *r |= GICD_TYPE_LPIS;
+ }
+ else
+ irq_bits = get_count_order(vgic_num_irqs(v->domain));
+
*r |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
return 1;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 986a4d6..c4cf65e 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -93,6 +93,8 @@ struct arch_domain
spinlock_t lock;
int ctlr;
int nr_spis; /* Number of SPIs */
+ int nr_lpis; /* Number of LPIs */
+ int id_bits;
unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
struct vgic_irq_rank *shared_irqs;
/*
@@ -106,6 +108,7 @@ struct arch_domain
#ifdef HAS_GICV3
/* Virtual ITS */
struct vgic_its *vits;
+ int gicr_ctlr;
/* GIC V3 addressing */
/* List of contiguous occupied by the redistributors */
struct vgic_rdist_region {
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 0854cde..30316fd 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -116,6 +116,7 @@ extern unsigned int num_of_lpis;
#define LPI_PROP_ENABLED (1 << 0)
#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PRIORITY_MASK (0xfc)
/*
* Collection structure - just an ID, and a redistributor address to
@@ -147,6 +148,16 @@ struct vgic_its
paddr_t gits_base;
/* GICR ctrl register */
uint32_t ctrl;
+ /* LPI propbase */
+ paddr_t propbase;
+ /* percpu pendbase */
+ paddr_t pendbase[MAX_VIRT_CPUS];
+ /* Virtual LPI property table */
+ void *prop_page;
+ /* Virtual LPI property size */
+ uint64_t prop_size;
+ /* spinlock to protect lpi property table */
+ spinlock_t prop_lock;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
@@ -344,6 +355,7 @@ int vits_get_vitt_entry(struct domain *d, uint32_t devid,
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
struct vdevice_table *entry);
void vits_setup_hw(struct gic_its_info *info);
+int vits_map_lpi_prop(struct domain *d);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 5f37f56..a9a5874 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -97,6 +97,7 @@
#define GICD_TYPE_CPUS_SHIFT 5
#define GICD_TYPE_CPUS 0x0e0
#define GICD_TYPE_SEC 0x400
+#define GICD_TYPE_LPIS (0x1UL << 17)
#define GICC_CTL_ENABLE 0x1
#define GICC_CTL_EOI (0x1 << 9)
@@ -283,6 +284,8 @@ extern void gic_dump_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+/* LPI support info */
+bool_t gic_lpi_supported(void);
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
@@ -300,6 +303,8 @@ struct gic_info {
unsigned int maintenance_irq;
/* Pointer to the device tree node representing the interrupt controller */
const struct dt_device_node *node;
+ /* LPIs are support information */
+ bool_t lpi_supported;
};
struct gic_hw_operations {
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 2c322da..4233bd5 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -127,6 +127,7 @@
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_TYPER_PROCESSOR_SHIFT (8)
#define GICR_PENDBASER_InnerShareable (1U << 10)
#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-07-27 11:11 ` [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-07-30 17:04 ` Julien Grall
2015-07-31 9:08 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-30 17:04 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
[..]
> +static int gicv3_dist_supports_lpis(void)
> +{
> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
> +}
> +
> static int __cpuinit gicv3_cpu_init(void)
> {
> int i;
> @@ -1274,6 +1279,11 @@ static int __init gicv3_init(void)
>
> spin_lock(&gicv3.lock);
>
> + if ( gicv3_dist_supports_lpis() )
> + gicv3_info.lpi_supported = 1;
> + else
> + gicv3_info.lpi_supported = 0;
> +
You will avoid 3 lines if you do:
gicv3_info.lpi_supported = !!gicv3_dist_supports_lpis();
> gicv3_dist_init();
> res = gicv3_cpu_init();
> gicv3_hyp_init();
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 1757193..af8a34b 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -67,6 +67,11 @@ unsigned int gic_number_lines(void)
> return gic_hw_ops->info->nr_lines;
> }
>
> +bool_t gic_lpi_supported(void)
> +{
> + return gic_hw_ops->info->lpi_supported;
> +}
> +
> void gic_save_state(struct vcpu *v)
> {
> ASSERT(!local_irq_is_enabled());
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 1c7d9b6..4afb62b 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
Can you explain why the emulation of the LPI property table has to be
done in the vITS code?
Overall, there is nothing vITS specific in this code and all the
functions you've introduced within this file are called only by the
vgic-v3 code.
> @@ -28,6 +28,7 @@
> #include <asm/io.h>
> #include <asm/gic_v3_defs.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
> #include <asm/gic-its.h>
> #include <asm/atomic.h>
> @@ -76,6 +77,34 @@ static inline uint32_t vits_get_max_collections(struct domain *d)
> return (d->max_vcpus + 1);
> }
>
> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
> +{
> + struct pending_irq *p;
> +
> + p = irq_to_pending(v, vlpi);
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + gic_remove_from_queues(v, vlpi);
> +}
> +
> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + p = irq_to_pending(v, vlpi);
> +
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
> +
> + /*XXX: raise on right vcpu */
As said on the previous versions, I think there will be locking issue
given that pending_irq structure is protected by the vCPU where the IRQ
is locked.
If you think it's not the case please explain why but don't leave the
question unanswered.
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v, vlpi, p->priority);
> +
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> +}
> +
> static int vits_access_guest_table(struct domain *d, paddr_t entry, void *addr,
> uint32_t size, bool_t set)
> {
> @@ -551,6 +580,145 @@ static inline uint32_t vits_get_word(uint32_t reg_offset, uint64_t val)
> return (u32)(val >> 32);
> }
>
> +static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa - (vits->propbase & MASK_4K);
> +
> + DPRINTK("%pv: vITS: LPI Table read offset 0x%"PRIx32"\n", v, offset);
> + spin_lock(&vits->prop_lock);
There is a potential deadlock with this lock here and the one in
vits_get_priority (see patch #16):
- The spin lock is taken in vgic_v3_gits_lpi_mmio_read
<==== LPI received on the same CPU
- vits_get_priority try to get the lock
=> Not possible because already locked
This could be avoid by disabling the IRQ when taking the spin lock and
reenable it when you release it.
> + if ( dabt.size == DABT_DOUBLE_WORD )
Please use a switch rather than if/else if/else if... and assuming the
last else the 8bit reading.
> + *r = *((u64*)vits->prop_page + offset);
> + else if (dabt.size == DABT_WORD )
> + *r = *((u32*)vits->prop_page + offset);
> + else if (dabt.size == DABT_HALF_WORD )
> + *r = *((u16*)vits->prop_page + offset);
The returned value won't be correct for 64, 32 and 16 bit read.
The cast is only applied to vits->prop_page, therefore the pointer will
be increment by offset * sizeof(uN).
You need to do the arithmetic on the pointer and then doing the cast.
> + else
> + *r = *((u8*)vits->prop_page + offset);
> + spin_unlock(&vits->prop_lock);
> +
> + return 1;
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset, vid;
> + uint8_t cfg, *p, i, iter;
> + bool_t enable;
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa - (vits->propbase & MASK_4K);
> +
> + DPRINTK("%pv: vITS: LPI Table write offset 0x%"PRIx32"\n", v, offset);
> +
> + if ( dabt.size == DABT_DOUBLE_WORD )
Please use a switch rather than if/else if/else if...
> + iter = 8;
> + else if ( dabt.size == DABT_WORD )
> + iter = 4;
> + else if ( dabt.size == DABT_HALF_WORD )
> + iter = 2;
> + else
> + iter = 1;
> +
> + spin_lock(&vits->prop_lock);
Same remark as the previous spin_lock.
> + p = ((u8*)vits->prop_page + offset);
> +
> + vid = offset + FIRST_GIC_LPI;
> + for ( i = 0 ; i < iter; i++ )
> + {
> + cfg = *p;
> + enable = (cfg & *r) & 0x1;
As said on v4, please use a define rather than 0x1. It would have been
more clear that we are checking the enable bit.
> +
> + if ( !enable )
> + vits_enable_lpi(v, vid, (*r & LPI_PRIORITY_MASK));
> + else
> + vits_disable_lpi(v, vid);
> +
> + /* Update virtual prop page */
> + *p = (*r & 0xff);
> + p++;
> + vid++;
> + }
> + spin_unlock(&vits->prop_lock);
> +
> + return 1;
> +}
> +
> +static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
> + .read_handler = vgic_v3_gits_lpi_mmio_read,
> + .write_handler = vgic_v3_gits_lpi_mmio_write,
> +};
> +
> +int vits_map_lpi_prop(struct domain *d)
> +{
> + struct vgic_its *vits = d->arch.vgic.vits;
> + paddr_t gaddr, addr;
> + unsigned long mfn;
> + uint32_t lpi_size, id_bits;
> + int i;
> +
> + gaddr = vits->propbase & MASK_4K;
The physical address is only bits [47:12]. The uppers are either RES0 or
used for the OuterCache.
> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
> +
> + if ( id_bits > d->arch.vgic.id_bits )
> + id_bits = d->arch.vgic.id_bits;
As said on v4, you are allowing the possibility to have a smaller
property table than the effective number of LPIs.
An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
of the size of the property table provided by the guest. This will
surely crash Xen in debug mode, and who knows what will happen in
production mode.
> +
> + lpi_size = 1UL << id_bits;
> +
> + DPRINTK("vITS:Map LPI conf table @ 0x%"PRIx64" size 0x%"PRIx32"\n",
Missing space after ":"
s/PRIx64/PRIpaddr/
> + gaddr, lpi_size);
> +
> + vits->prop_size = lpi_size;
> + /* Allocate Virtual LPI Property table */
> + /* TODO: To re-use guest property table? */
> + vits->prop_page = alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
> + if ( !vits->prop_page )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Fail to allocate LPI Prop page\n", current);
I think you should use d->domain_id and only print the domain id for
more consistency with the function rather than current.
> + return 0;
> + }
> +
> + addr = gaddr;
> + for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
> + {
> + vits_access_guest_table(d, addr,
> + (void *)(vits->prop_page + i * PAGE_SIZE),
> + PAGE_SIZE, 0);
> + addr += PAGE_SIZE;
As said on v4, it's good to copy the value from the guest memory to our
internal table. But there is no point to do it if you don't replicate
the value in the internal LPIs. I.e enable/disable the LPIs when
necessary....
Note that it may be a long process. So either you correctly handle the
copy or you don't do it at all. But avoid to do something in the middle,
because it will break the state in Xen.
> + }
> +
> + /*
> + * Each re-distributor shares a common LPI configuration table
> + * So one set of mmio handlers to manage configuration table is enough
> + */
> + addr = gaddr;
> + for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
> + {
> + mfn = gmfn_to_mfn(d, paddr_to_pfn(addr));
> + if ( unlikely(!mfn_valid(mfn)) )
> + {
> + dprintk(XENLOG_G_ERR, "%pv: vITS: Invalid propbaser address\n", current);
> + return 0;
> + }
> + guest_physmap_remove_page(d, paddr_to_pfn(addr), mfn, 0);
> + addr += PAGE_SIZE;
> + }
> +
> + /* Register mmio handlers for this region */
> + register_mmio_handler(d, &vgic_gits_lpi_mmio_handler,
> + gaddr, lpi_size);
> +
> + return 1;
> +}
> +
> static inline void vits_spin_lock(struct vgic_its *vits)
> {
> spin_lock(&vits->lock);
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 683e3cc..a466a8f 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -29,6 +29,8 @@
> #include <asm/current.h>
> #include <asm/mmio.h>
> #include <asm/gic_v3_defs.h>
> +#include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <asm/vgic.h>
>
> /* GICD_PIDRn register values for ARM implementations */
> @@ -109,29 +111,47 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
> struct hsr_dabt dabt = info->dabt;
> struct cpu_user_regs *regs = guest_cpu_user_regs();
> register_t *r = select_user_reg(regs, dabt.reg);
> - uint64_t aff;
> + uint64_t aff, val;
>
> switch ( gicr_reg )
> {
> case GICR_CTLR:
> - /* We have not implemented LPI's, read zero */
> - goto read_as_zero_32;
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + vgic_lock(v);
> + *r = v->domain->arch.vgic.gicr_ctlr;
> + vgic_unlock(v);
> + return 1;
> case GICR_IIDR:
> if ( dabt.size != DABT_WORD ) goto bad_width;
> *r = GICV3_GICR_IIDR_VAL;
> return 1;
> case GICR_TYPER:
> - if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> - /* TBD: Update processor id in [23:8] when ITS support is added */
> + case GICR_TYPER + 4:
Why did you introduce it only in v5? This change is not related to the
LPI but a correct implementation of access size on GICv3 register.
Overall, I think this should be done in a separate patch for all the
registers and not only this one. It would make the code change less
complicate to read.
Please fix the write version of TYPER which happen to be 64 bits only too.
> + if ( dabt.size != DABT_DOUBLE_WORD && dabt.size != DABT_WORD)
> + goto bad_width;
> aff = (MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 3) << 56 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
> MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
> - *r = aff;
> -
> + val = aff;
> + if ( gic_lpi_supported() )
The registers/values used only for LPIs should only be exposed for guest
using ITS. I.e only for DOM0.
Please introduce an helpers vgic_is_lpi_supported(d) which return true
only when the ITS is in-use.
In general seen direct call to the hardware in the vgic is a wrong
things. The vGIC could should be pretty much hardware agnostic.
> + {
> + /* Set Physical LPIs support */
> + val |= GICR_TYPER_PLPIS;
> + /* GITS_TYPER.PTA is 0. Provide vcpu number as target address */
Well this is not specific to the ITS neither LPIs. This could be used by
the guest is we ever happen to support GICv2 on GICv3. Therefore this
should be outside the if.
[..]
> @@ -245,10 +286,37 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
> /* LPI is not implemented */
> goto write_ignore_64;
> case GICR_PROPBASER:
> - /* LPI is not implemented */
> + if ( gic_lpi_supported() )
> + {
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + vgic_lock(v);
> + /* LPI configuration tables are shared across cpus. Should be same */
> + /*
> + * Allow updating on propbase only once with below check.
> + * TODO: Manage change in property table.
> + */
> + if ( v->domain->arch.vgic.vits->propbase != 0 )
> + {
> + printk(XENLOG_G_WARNING
> + "%pv: vGICR: Updating LPI propbase is not allowed\n", v);
Hmmmm, AFAICT this warning will be printed even in valid case. i.e for
every VCPUs but VCPU0 setting the propbaser.
This warning should only be printed when it's necessary to avoid
question from user about false positive.
> + vgic_unlock(v);
> + return 1;
> + }
> + v->domain->arch.vgic.vits->propbase = *r;
> + vgic_unlock(v);
> + return vits_map_lpi_prop(v->domain);
> + }
> goto write_ignore_64;
> case GICR_PENDBASER:
> - /* LPI is not implemented */
> + if ( gic_lpi_supported() )
> + {
> + /* Just hold pendbaser value for guest read */
> + if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
> + vgic_lock(v);
> + v->domain->arch.vgic.vits->pendbase[v->vcpu_id] = *r;
If you store the value, you should at least implementing correctly for
the return (see 8.11.18 ARM IHI 0069A).
> + vgic_unlock(v);
> + return 1;
> + }
> goto write_ignore_64;
> case GICR_INVLPIR:
> /* LPI is not implemented */
> @@ -715,6 +783,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> *r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
> DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
>
> + if ( gic_lpi_supported() )
> + {
> + irq_bits = v->domain->arch.vgic.id_bits;
> + *r |= GICD_TYPE_LPIS;
> + }
> + else
> + irq_bits = get_count_order(vgic_num_irqs(v->domain));
Why do you do a specific case when the lpi is not supported? The id_bits
ought to be valid for both case.
> *r |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
>
> return 1;
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 986a4d6..c4cf65e 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -93,6 +93,8 @@ struct arch_domain
> spinlock_t lock;
> int ctlr;
> int nr_spis; /* Number of SPIs */
> + int nr_lpis; /* Number of LPIs */
> + int id_bits;
Please add a comment explaing id_bits.
> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
> struct vgic_irq_rank *shared_irqs;
> /*
> @@ -106,6 +108,7 @@ struct arch_domain
> #ifdef HAS_GICV3
> /* Virtual ITS */
> struct vgic_its *vits;
> + int gicr_ctlr;
> /* GIC V3 addressing */
> /* List of contiguous occupied by the redistributors */
> struct vgic_rdist_region {
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 0854cde..30316fd 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -116,6 +116,7 @@ extern unsigned int num_of_lpis;
>
> #define LPI_PROP_ENABLED (1 << 0)
> #define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PRIORITY_MASK (0xfc)
>
> /*
> * Collection structure - just an ID, and a redistributor address to
> @@ -147,6 +148,16 @@ struct vgic_its
> paddr_t gits_base;
> /* GICR ctrl register */
> uint32_t ctrl;
> + /* LPI propbase */
> + paddr_t propbase;
> + /* percpu pendbase */
> + paddr_t pendbase[MAX_VIRT_CPUS];
Please create a field in arch_vcpu.vgic structure rather than having a
static array in arch_domain.
> + /* Virtual LPI property table */
> + void *prop_page;
> + /* Virtual LPI property size */
> + uint64_t prop_size;
Please use uint32_t here, the size is only encoding on 32 bits.
> + /* spinlock to protect lpi property table */
> + spinlock_t prop_lock;
> /* vITT device table ipa */
> paddr_t dt_ipa;
> /* vITT device table size */
> @@ -344,6 +355,7 @@ int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
> struct vdevice_table *entry);
> void vits_setup_hw(struct gic_its_info *info);
> +int vits_map_lpi_prop(struct domain *d);
>
> #endif /* __ASM_ARM_GIC_ITS_H__ */
> /*
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 5f37f56..a9a5874 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -97,6 +97,7 @@
> #define GICD_TYPE_CPUS_SHIFT 5
> #define GICD_TYPE_CPUS 0x0e0
> #define GICD_TYPE_SEC 0x400
> +#define GICD_TYPE_LPIS (0x1UL << 17)
This field is GICv3 specific. Therefore this should go in gic_v3_defs.h.
>
> #define GICC_CTL_ENABLE 0x1
> #define GICC_CTL_EOI (0x1 << 9)
> @@ -283,6 +284,8 @@ extern void gic_dump_info(struct vcpu *v);
>
> /* Number of interrupt lines */
> extern unsigned int gic_number_lines(void);
> +/* LPI support info */
> +bool_t gic_lpi_supported(void);
I'm not convinced that this new function in the common GIC is not
necesarry. At least not within this patch. If you appear to need it in
another patch please introduce it there.
> /* IRQ translation function for the device tree */
> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> @@ -300,6 +303,8 @@ struct gic_info {
> unsigned int maintenance_irq;
> /* Pointer to the device tree node representing the interrupt controller */
> const struct dt_device_node *node;
> + /* LPIs are support information */
What do you mean?
> + bool_t lpi_supported;
> };
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-07-30 17:04 ` Julien Grall
@ 2015-07-31 9:08 ` Vijay Kilari
2015-07-31 11:05 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-07-31 9:08 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Thu, Jul 30, 2015 at 10:34 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>
> [..]
>
>> +static int gicv3_dist_supports_lpis(void)
>> +{
>> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
>> +}
>> +
>> static int __cpuinit gicv3_cpu_init(void)
>> {
>> int i;
>> @@ -1274,6 +1279,11 @@ static int __init gicv3_init(void)
>>
>> spin_lock(&gicv3.lock);
>>
>> + if ( gicv3_dist_supports_lpis() )
>> + gicv3_info.lpi_supported = 1;
>> + else
>> + gicv3_info.lpi_supported = 0;
>> +
>
> You will avoid 3 lines if you do:
>
> gicv3_info.lpi_supported = !!gicv3_dist_supports_lpis();
>
This will change in patch #17 where we do check for its_probe() to
be succesful
to set gicv3_info.lpi_supported
>> gicv3_dist_init();
>> res = gicv3_cpu_init();
>> gicv3_hyp_init();
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 1757193..af8a34b 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -67,6 +67,11 @@ unsigned int gic_number_lines(void)
>> return gic_hw_ops->info->nr_lines;
>> }
>>
>> +bool_t gic_lpi_supported(void)
>> +{
>> + return gic_hw_ops->info->lpi_supported;
>> +}
>> +
>> void gic_save_state(struct vcpu *v)
>> {
>> ASSERT(!local_irq_is_enabled());
>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>> index 1c7d9b6..4afb62b 100644
>> --- a/xen/arch/arm/vgic-v3-its.c
>> +++ b/xen/arch/arm/vgic-v3-its.c
>
> Can you explain why the emulation of the LPI property table has to be
> done in the vITS code?
>
> Overall, there is nothing vITS specific in this code and all the
> functions you've introduced within this file are called only by the
> vgic-v3 code.
yes, it is called from vgic-v3 code because it is emulating GICR_PROP/PEND
registers.
This is emulating LPI property table. Hence all the LPI handling code is kept
in vits file.
GICR_PROP/PEND is defined only for LPI case.
>
>> @@ -28,6 +28,7 @@
>> #include <asm/io.h>
>> #include <asm/gic_v3_defs.h>
>> #include <asm/gic.h>
>> +#include <asm/gic-its.h>
>> #include <asm/vgic.h>
>> #include <asm/gic-its.h>
>> #include <asm/atomic.h>
>> @@ -76,6 +77,34 @@ static inline uint32_t vits_get_max_collections(struct domain *d)
>> return (d->max_vcpus + 1);
>> }
>>
>> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
>> +{
>> + struct pending_irq *p;
>> +
>> + p = irq_to_pending(v, vlpi);
>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> + gic_remove_from_queues(v, vlpi);
>> +}
>> +
>> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
>> +{
>> + struct pending_irq *p;
>> + unsigned long flags;
>> +
>> + p = irq_to_pending(v, vlpi);
>> +
>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>> +
>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>> +
>> + /*XXX: raise on right vcpu */
>
> As said on the previous versions, I think there will be locking issue
> given that pending_irq structure is protected by the vCPU where the IRQ
> is locked.
Can you please explain in detail why there is a locking issue.
I remember this locking mechanism is coming from enable_irqs()
as we follow same infrastructure to inject/process LPIs
>
> If you think it's not the case please explain why but don't leave the
> question unanswered.
>
[...]
>>
>> +
>> +int vits_map_lpi_prop(struct domain *d)
>> +{
>> + struct vgic_its *vits = d->arch.vgic.vits;
>> + paddr_t gaddr, addr;
>> + unsigned long mfn;
>> + uint32_t lpi_size, id_bits;
>> + int i;
>> +
>> + gaddr = vits->propbase & MASK_4K;
>
> The physical address is only bits [47:12]. The uppers are either RES0 or
> used for the OuterCache.
Can you please be more explicit. What is the issue here?
>
>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>> +
>> + if ( id_bits > d->arch.vgic.id_bits )
>> + id_bits = d->arch.vgic.id_bits;
>
> As said on v4, you are allowing the possibility to have a smaller
> property table than the effective number of LPIs.
>
> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
> of the size of the property table provided by the guest. This will
> surely crash Xen in debug mode, and who knows what will happen in
> production mode.
lpi_size is calculated based on id_bits. If it is smaller, the lpi_size will be
smaller where only size of lpi_size is considered.
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 683e3cc..a466a8f 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -29,6 +29,8 @@
>> #include <asm/current.h>
>> #include <asm/mmio.h>
>> #include <asm/gic_v3_defs.h>
>> +#include <asm/gic.h>
>> +#include <asm/gic-its.h>
>> #include <asm/vgic.h>
>>
>> /* GICD_PIDRn register values for ARM implementations */
>> @@ -109,29 +111,47 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
>> struct hsr_dabt dabt = info->dabt;
>> struct cpu_user_regs *regs = guest_cpu_user_regs();
>> register_t *r = select_user_reg(regs, dabt.reg);
>> - uint64_t aff;
>> + uint64_t aff, val;
>>
>> switch ( gicr_reg )
>> {
>> case GICR_CTLR:
>> - /* We have not implemented LPI's, read zero */
>> - goto read_as_zero_32;
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + vgic_lock(v);
>> + *r = v->domain->arch.vgic.gicr_ctlr;
>> + vgic_unlock(v);
>> + return 1;
>> case GICR_IIDR:
>> if ( dabt.size != DABT_WORD ) goto bad_width;
>> *r = GICV3_GICR_IIDR_VAL;
>> return 1;
>> case GICR_TYPER:
>> - if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
>> - /* TBD: Update processor id in [23:8] when ITS support is added */
>> + case GICR_TYPER + 4:
>
> Why did you introduce it only in v5? This change is not related to the
> LPI but a correct implementation of access size on GICv3 register.
>
> Overall, I think this should be done in a separate patch for all the
> registers and not only this one. It would make the code change less
> complicate to read.
>
> Please fix the write version of TYPER which happen to be 64 bits only too.
I added it because, after updating my Linux kernel driver, I see that it
is making 32-bit access to TYPER register
static bool gic_rdists_supports_plpis(void)
{
return !!(readl_relaxed(gic_data_rdist_rd_base() + GICR_TYPER) &
GICR_TYPER_PLPIS);
}
But spec doesn't say it supports 32-bit access.
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-07-31 9:08 ` Vijay Kilari
@ 2015-07-31 11:05 ` Julien Grall
2015-08-01 10:25 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-07-31 11:05 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 31/07/15 10:08, Vijay Kilari wrote:
> On Thu, Jul 30, 2015 at 10:34 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>
>> [..]
>>
>>> +static int gicv3_dist_supports_lpis(void)
>>> +{
>>> + return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
>>> +}
>>> +
>>> static int __cpuinit gicv3_cpu_init(void)
>>> {
>>> int i;
>>> @@ -1274,6 +1279,11 @@ static int __init gicv3_init(void)
>>>
>>> spin_lock(&gicv3.lock);
>>>
>>> + if ( gicv3_dist_supports_lpis() )
>>> + gicv3_info.lpi_supported = 1;
>>> + else
>>> + gicv3_info.lpi_supported = 0;
>>> +
>>
>> You will avoid 3 lines if you do:
>>
>> gicv3_info.lpi_supported = !!gicv3_dist_supports_lpis();
>>
>
> This will change in patch #17 where we do check for its_probe() to
> be succesful
> to set gicv3_info.lpi_supported
Then move this addition of lpi_supported in patch #17. This is not
necessary at all here. And this is quite confusing to see GIC specific
changes in vGIC code.
Overall, I think your series would benefits to be split in two parts.
First part would contains all the code for the hardware GIC/ITS driver.
The second part would be for virtual GIC/ITS changes.
>>> gicv3_dist_init();
>>> res = gicv3_cpu_init();
>>> gicv3_hyp_init();
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index 1757193..af8a34b 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -67,6 +67,11 @@ unsigned int gic_number_lines(void)
>>> return gic_hw_ops->info->nr_lines;
>>> }
>>>
>>> +bool_t gic_lpi_supported(void)
>>> +{
>>> + return gic_hw_ops->info->lpi_supported;
>>> +}
>>> +
>>> void gic_save_state(struct vcpu *v)
>>> {
>>> ASSERT(!local_irq_is_enabled());
>>> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
>>> index 1c7d9b6..4afb62b 100644
>>> --- a/xen/arch/arm/vgic-v3-its.c
>>> +++ b/xen/arch/arm/vgic-v3-its.c
>>
>> Can you explain why the emulation of the LPI property table has to be
>> done in the vITS code?
>>
>> Overall, there is nothing vITS specific in this code and all the
>> functions you've introduced within this file are called only by the
>> vgic-v3 code.
>
> yes, it is called from vgic-v3 code because it is emulating GICR_PROP/PEND
> registers.
>
> This is emulating LPI property table. Hence all the LPI handling code is kept
> in vits file.
> GICR_PROP/PEND is defined only for LPI case.
You don't explain why the LPI handling code is kept in the vits file....
The LPI property table can exist even without the presence of
the ITS.
There is nothing ITS specific in the emulation, and your design prove it
(you are calling all the emulation code from the vGICv3).
>>
>>> @@ -28,6 +28,7 @@
>>> #include <asm/io.h>
>>> #include <asm/gic_v3_defs.h>
>>> #include <asm/gic.h>
>>> +#include <asm/gic-its.h>
>>> #include <asm/vgic.h>
>>> #include <asm/gic-its.h>
>>> #include <asm/atomic.h>
>>> @@ -76,6 +77,34 @@ static inline uint32_t vits_get_max_collections(struct domain *d)
>>> return (d->max_vcpus + 1);
>>> }
>>>
>>> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
>>> +{
>>> + struct pending_irq *p;
>>> +
>>> + p = irq_to_pending(v, vlpi);
>>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>> + gic_remove_from_queues(v, vlpi);
>>> +}
>>> +
>>> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
>>> +{
>>> + struct pending_irq *p;
>>> + unsigned long flags;
>>> +
>>> + p = irq_to_pending(v, vlpi);
>>> +
>>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>> +
>>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>> +
>>> + /*XXX: raise on right vcpu */
>>
>> As said on the previous versions, I think there will be locking issue
>> given that pending_irq structure is protected by the vCPU where the IRQ
>> is locked.
>
> Can you please explain in detail why there is a locking issue.
> I remember this locking mechanism is coming from enable_irqs()
> as we follow same infrastructure to inject/process LPIs
I guess you mean vgic_enable_irqs? And no what you've implemented is
definitely not the same as vgic_enable_irqs.
vgic_enable_irqs is locking the pending_irq structure using the vgic
lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
Here, you are locking with the current vCPU, i.e the vCPU which wrote
into the LPI property table.
All the vGIC code is doing the same, so using the wrong locking won't
protect this structure.
>
>>
>> If you think it's not the case please explain why but don't leave the
>> question unanswered.
>>
> [...]
>>>
>>> +
>>> +int vits_map_lpi_prop(struct domain *d)
>>> +{
>>> + struct vgic_its *vits = d->arch.vgic.vits;
>>> + paddr_t gaddr, addr;
>>> + unsigned long mfn;
>>> + uint32_t lpi_size, id_bits;
>>> + int i;
>>> +
>>> + gaddr = vits->propbase & MASK_4K;
>>
>> The physical address is only bits [47:12]. The uppers are either RES0 or
>> used for the OuterCache.
>
> Can you please be more explicit. What is the issue here?
Sorry, I was confused by the MASK_4K which actually is not implemented
as the name suggested but which clearing the bits [63:48].
A correct MASK_4K would haven't clear the top bits. So the name of this
define is not right.
>
>>
>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>> +
>>> + if ( id_bits > d->arch.vgic.id_bits )
>>> + id_bits = d->arch.vgic.id_bits;
>>
>> As said on v4, you are allowing the possibility to have a smaller
>> property table than the effective number of LPIs.
>>
>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>> of the size of the property table provided by the guest. This will
>> surely crash Xen in debug mode, and who knows what will happen in
>> production mode.
>
> lpi_size is calculated based on id_bits. If it is smaller, the lpi_size will be
> smaller where only size of lpi_size is considered.
Where id_bits is based on what the guest provides in GICR_PROPBASER.
Although the guest, malicious or not, can decide to setup this id_bits
smaller than the number of vLPIs effectively supported by the gic-v3.
In this case, if a vLPI higher than this number is injected you will hit
the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
*should* not be able to crash Xen because it decides to send valid input.
>
>>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>>> index 683e3cc..a466a8f 100644
>>> --- a/xen/arch/arm/vgic-v3.c
>>> +++ b/xen/arch/arm/vgic-v3.c
>>> @@ -29,6 +29,8 @@
>>> #include <asm/current.h>
>>> #include <asm/mmio.h>
>>> #include <asm/gic_v3_defs.h>
>>> +#include <asm/gic.h>
>>> +#include <asm/gic-its.h>
>>> #include <asm/vgic.h>
>>>
>>> /* GICD_PIDRn register values for ARM implementations */
>>> @@ -109,29 +111,47 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
>>> struct hsr_dabt dabt = info->dabt;
>>> struct cpu_user_regs *regs = guest_cpu_user_regs();
>>> register_t *r = select_user_reg(regs, dabt.reg);
>>> - uint64_t aff;
>>> + uint64_t aff, val;
>>>
>>> switch ( gicr_reg )
>>> {
>>> case GICR_CTLR:
>>> - /* We have not implemented LPI's, read zero */
>>> - goto read_as_zero_32;
>>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>>> + vgic_lock(v);
>>> + *r = v->domain->arch.vgic.gicr_ctlr;
>>> + vgic_unlock(v);
>>> + return 1;
>>> case GICR_IIDR:
>>> if ( dabt.size != DABT_WORD ) goto bad_width;
>>> *r = GICV3_GICR_IIDR_VAL;
>>> return 1;
>>> case GICR_TYPER:
>>> - if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
>>> - /* TBD: Update processor id in [23:8] when ITS support is added */
>>> + case GICR_TYPER + 4:
>>
>> Why did you introduce it only in v5? This change is not related to the
>> LPI but a correct implementation of access size on GICv3 register.
>>
>> Overall, I think this should be done in a separate patch for all the
>> registers and not only this one. It would make the code change less
>> complicate to read.
>>
>> Please fix the write version of TYPER which happen to be 64 bits only too.
>
> I added it because, after updating my Linux kernel driver, I see that it
> is making 32-bit access to TYPER register
This change should go in a separate patch then. It's not related to this
patch.
>
> static bool gic_rdists_supports_plpis(void)
> {
> return !!(readl_relaxed(gic_data_rdist_rd_base() + GICR_TYPER) &
> GICR_TYPER_PLPIS);
> }
>
> But spec doesn't say it supports 32-bit access.
8.1.3 ARM IHI 0069A
"For the GITS_*, GICD_* and GICR_* registers, the upper 32 bits and the
lower 32 bits can be accessed
independently, unless the register requires a 64 bit access."
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-07-31 11:05 ` Julien Grall
@ 2015-08-01 10:25 ` Vijay Kilari
2015-08-01 15:51 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-01 10:25 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Fri, Jul 31, 2015 at 4:35 PM, Julien Grall <julien.grall@citrix.com> wrote:
> On 31/07/15 10:08, Vijay Kilari wrote:
>> On Thu, Jul 30, 2015 at 10:34 PM, Julien Grall <julien.grall@citrix.com> wrote:
>>> Hi Vijay,
[...]
>>>> +static void vits_disable_lpi(struct vcpu *v, uint32_t vlpi)
>>>> +{
>>>> + struct pending_irq *p;
>>>> +
>>>> + p = irq_to_pending(v, vlpi);
>>>> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>>> + gic_remove_from_queues(v, vlpi);
>>>> +}
>>>> +
>>>> +static void vits_enable_lpi(struct vcpu *v, uint32_t vlpi, uint8_t priority)
>>>> +{
>>>> + struct pending_irq *p;
>>>> + unsigned long flags;
>>>> +
>>>> + p = irq_to_pending(v, vlpi);
>>>> +
>>>> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
>>>> +
>>>> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
>>>> +
>>>> + /*XXX: raise on right vcpu */
>>>
>>> As said on the previous versions, I think there will be locking issue
>>> given that pending_irq structure is protected by the vCPU where the IRQ
>>> is locked.
>>
>> Can you please explain in detail why there is a locking issue.
>> I remember this locking mechanism is coming from enable_irqs()
>> as we follow same infrastructure to inject/process LPIs
>
> I guess you mean vgic_enable_irqs? And no what you've implemented is
> definitely not the same as vgic_enable_irqs.
>
> vgic_enable_irqs is locking the pending_irq structure using the vgic
> lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
>
> Here, you are locking with the current vCPU, i.e the vCPU which wrote
> into the LPI property table.
>
> All the vGIC code is doing the same, so using the wrong locking won't
> protect this structure.
With just vlpi, we cannot get target vcpu without devid. Now
question is there a
need to call gic_raise_guest_irq() for inflight LPIs?
>>>
>>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>>> +
>>>> + if ( id_bits > d->arch.vgic.id_bits )
>>>> + id_bits = d->arch.vgic.id_bits;
>>>
>>> As said on v4, you are allowing the possibility to have a smaller
>>> property table than the effective number of LPIs.
>>>
>>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>>> of the size of the property table provided by the guest. This will
>>> surely crash Xen in debug mode, and who knows what will happen in
>>> production mode.
>>
>> lpi_size is calculated based on id_bits. If it is smaller, the lpi_size will be
>> smaller where only size of lpi_size is considered.
>
> Where id_bits is based on what the guest provides in GICR_PROPBASER.
> Although the guest, malicious or not, can decide to setup this id_bits
> smaller than the number of vLPIs effectively supported by the gic-v3.
>
> In this case, if a vLPI higher than this number is injected you will hit
> the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
> *should* not be able to crash Xen because it decides to send valid input.
>
>From 8.11.19, if id_bits is < 8192 (as below statement), GIC treats
LPIs as out of range.
"If the value of this field is less than 0b1101, indicating that the
largest interrupt ID is less than 8192
(the smallest LPI interrupt ID), the GIC will behave as if all
physical LPIs are out of range."
Based on this, we should make a check on this GICR_PROPBASER.id_bits
before injecting LPI to domain when LPI is received.
>>
>>>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>>>> index 683e3cc..a466a8f 100644
>>>> --- a/xen/arch/arm/vgic-v3.c
>>>> +++ b/xen/arch/arm/vgic-v3.c
>>>> @@ -29,6 +29,8 @@
>>>> #include <asm/current.h>
>>>> #include <asm/mmio.h>
>>>> #include <asm/gic_v3_defs.h>
>>>> +#include <asm/gic.h>
>>>> +#include <asm/gic-its.h>
>>>> #include <asm/vgic.h>
>>>>
>>>> /* GICD_PIDRn register values for ARM implementations */
>>>> @@ -109,29 +111,47 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
>>>> struct hsr_dabt dabt = info->dabt;
>>>> struct cpu_user_regs *regs = guest_cpu_user_regs();
>>>> register_t *r = select_user_reg(regs, dabt.reg);
>>>> - uint64_t aff;
>>>> + uint64_t aff, val;
>>>>
>>>> switch ( gicr_reg )
>>>> {
>>>> case GICR_CTLR:
>>>> - /* We have not implemented LPI's, read zero */
>>>> - goto read_as_zero_32;
>>>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>>>> + vgic_lock(v);
>>>> + *r = v->domain->arch.vgic.gicr_ctlr;
>>>> + vgic_unlock(v);
>>>> + return 1;
>>>> case GICR_IIDR:
>>>> if ( dabt.size != DABT_WORD ) goto bad_width;
>>>> *r = GICV3_GICR_IIDR_VAL;
>>>> return 1;
>>>> case GICR_TYPER:
>>>> - if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
>>>> - /* TBD: Update processor id in [23:8] when ITS support is added */
>>>> + case GICR_TYPER + 4:
>>>
>>> Why did you introduce it only in v5? This change is not related to the
>>> LPI but a correct implementation of access size on GICv3 register.
>>>
>>> Overall, I think this should be done in a separate patch for all the
>>> registers and not only this one. It would make the code change less
>>> complicate to read.
>>>
>>> Please fix the write version of TYPER which happen to be 64 bits only too.
>>
>> I added it because, after updating my Linux kernel driver, I see that it
>> is making 32-bit access to TYPER register
>
> This change should go in a separate patch then. It's not related to this
> patch.
Why separate patch?. This change could be part of GICR* reg emulation
done for LPI
Regards
Vija
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-01 10:25 ` Vijay Kilari
@ 2015-08-01 15:51 ` Julien Grall
2015-08-03 9:36 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-01 15:51 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 01/08/2015 11:25, Vijay Kilari wrote:
>> I guess you mean vgic_enable_irqs? And no what you've implemented is
>> definitely not the same as vgic_enable_irqs.
>>
>> vgic_enable_irqs is locking the pending_irq structure using the vgic
>> lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
>>
>> Here, you are locking with the current vCPU, i.e the vCPU which wrote
>> into the LPI property table.
>>
>> All the vGIC code is doing the same, so using the wrong locking won't
>> protect this structure.
>
> With just vlpi, we cannot get target vcpu without devid. Now
> question is there a
> need to call gic_raise_guest_irq() for inflight LPIs?
Yes it's necessary. Physical LPIs can come up at any time before the
guest enables the virtual LPI. This is because we enable the physical
LPIs and route to the guest as soon as the device is assigned to it. You
may be interesting by the reading of [1].
You will have to find a way to get the correct vCPU because this may
occur more often than you think.
>>>>
>>>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>>>> +
>>>>> + if ( id_bits > d->arch.vgic.id_bits )
>>>>> + id_bits = d->arch.vgic.id_bits;
>>>>
>>>> As said on v4, you are allowing the possibility to have a smaller
>>>> property table than the effective number of LPIs.
>>>>
>>>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>>>> of the size of the property table provided by the guest. This will
>>>> surely crash Xen in debug mode, and who knows what will happen in
>>>> production mode.
>>>
>>> lpi_size is calculated based on id_bits. If it is smaller, the lpi_size will be
>>> smaller where only size of lpi_size is considered.
>>
>> Where id_bits is based on what the guest provides in GICR_PROPBASER.
>> Although the guest, malicious or not, can decide to setup this id_bits
>> smaller than the number of vLPIs effectively supported by the gic-v3.
>>
>> In this case, if a vLPI higher than this number is injected you will hit
>> the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
>> *should* not be able to crash Xen because it decides to send valid input.
>>
>
> From 8.11.19, if id_bits is < 8192 (as below statement), GIC treats
> LPIs as out of range.
When you quote the spec, please give both the section and which spec. I
have about 5 different docs for the GICv3, and I had to guess which one
you were using.
>
> "If the value of this field is less than 0b1101, indicating that the
> largest interrupt ID is less than 8192
> (the smallest LPI interrupt ID), the GIC will behave as if all
> physical LPIs are out of range."
Thank you for the quoting. I helps me to not a latent bug in your LPI
property table emulation. I should have read more carefully the spec.
The field IDBits gives you the number of interrupt ID bits supported.
The LPI property table size is only describing the LPI, i.e the offset 0
of the table is the IntID 8192.
So when you compute the size of the table, you have to substract 8192.
Otherwise you will remove 2 4KB pages from the guest which could be used
by it. You will also, possible Xen unsafe as we may read out of the
pending IRQ array (we are trusting the handler to be registered on valid
input).
> Based on this, we should make a check on this GICR_PROPBASER.id_bits
> before injecting LPI to domain when LPI is received.
And doing what? The paragraph you quote doesn't say anything on what
happen when the LPI interrupt ID is higher than the number of bits. It
only explain what happen the this number if higher and smaller than a bound.
What would happen if the LPI is injected before the GICR_PROPBASER is
enabled? See for more details on the problem here [1]
I think, you just have to make sure that the function reading the
priority (i.e vits_get_priority) is not trying to read out of the array
and return a dummy value (??).
>>> I added it because, after updating my Linux kernel driver, I see that it
>>> is making 32-bit access to TYPER register
>>
>> This change should go in a separate patch then. It's not related to this
>> patch.
>
> Why separate patch?. This change could be part of GICR* reg emulation
> done for LPI
Because this change is not part of the re-distributor emulation done for
LPI. You don't even mention it in your commit message. I discovered it
by comparing on what you did with the previous version.
Furthermore, as said earlier, if you handle 32-bit access for GICR_TYPER
you have to do it for every others registers.
Anyway, I will send a patch myself to handle 32-bit access on 64-bit
registers.
Regards,
[1]
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg02591.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-01 15:51 ` Julien Grall
@ 2015-08-03 9:36 ` Vijay Kilari
2015-08-03 13:01 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-03 9:36 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Sat, Aug 1, 2015 at 9:21 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 01/08/2015 11:25, Vijay Kilari wrote:
>>>
>>> I guess you mean vgic_enable_irqs? And no what you've implemented is
>>> definitely not the same as vgic_enable_irqs.
>>>
>>> vgic_enable_irqs is locking the pending_irq structure using the vgic
>>> lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
>>>
>>> Here, you are locking with the current vCPU, i.e the vCPU which wrote
>>> into the LPI property table.
>>>
>>> All the vGIC code is doing the same, so using the wrong locking won't
>>> protect this structure.
>>
>>
>> With just vlpi, we cannot get target vcpu without devid. Now
>> question is there a
>> need to call gic_raise_guest_irq() for inflight LPIs?
>
>
> Yes it's necessary. Physical LPIs can come up at any time before the guest
> enables the virtual LPI. This is because we enable the physical LPIs and
> route to the guest as soon as the device is assigned to it. You may be
> interesting by the reading of [1].
>
> You will have to find a way to get the correct vCPU because this may occur
> more often than you think.
>
One possible way that I can think of is
In MAPVI command is sent by guest, we know vlpi. Store col_id in the
pending_irq structure
for the vLPI and use this collection id to retrieve correct vCPU on
which we can take vgic lock.
>>>>>
>>>>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>>>>> +
>>>>>> + if ( id_bits > d->arch.vgic.id_bits )
>>>>>> + id_bits = d->arch.vgic.id_bits;
>>>>>
>>>>>
>>>>> As said on v4, you are allowing the possibility to have a smaller
>>>>> property table than the effective number of LPIs.
>>>>>
>>>>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>>>>> of the size of the property table provided by the guest. This will
>>>>> surely crash Xen in debug mode, and who knows what will happen in
>>>>> production mode.
>>>>
>>>>
>>>> lpi_size is calculated based on id_bits. If it is smaller, the
>>>> lpi_size will be
>>>> smaller where only size of lpi_size is considered.
>>>
>>>
>>> Where id_bits is based on what the guest provides in GICR_PROPBASER.
>>> Although the guest, malicious or not, can decide to setup this id_bits
>>> smaller than the number of vLPIs effectively supported by the gic-v3.
>>>
>>> In this case, if a vLPI higher than this number is injected you will hit
>>> the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
>>> *should* not be able to crash Xen because it decides to send valid input.
>>>
>>
>> From 8.11.19, if id_bits is < 8192 (as below statement), GIC treats
>> LPIs as out of range.
>
>
> When you quote the spec, please give both the section and which spec. I have
> about 5 different docs for the GICv3, and I had to guess which one you were
> using.
>
>>
>> "If the value of this field is less than 0b1101, indicating that the
>> largest interrupt ID is less than 8192
>> (the smallest LPI interrupt ID), the GIC will behave as if all
>> physical LPIs are out of range."
>
>
> Thank you for the quoting. I helps me to not a latent bug in your LPI
> property table emulation. I should have read more carefully the spec.
>
> The field IDBits gives you the number of interrupt ID bits supported. The
> LPI property table size is only describing the LPI, i.e the offset 0 of the
> table is the IntID 8192.
>
> So when you compute the size of the table, you have to substract 8192.
> Otherwise you will remove 2 4KB pages from the guest which could be used by
> it. You will also, possible Xen unsafe as we may read out of the pending IRQ
> array (we are trusting the handler to be registered on valid input).
>
>> Based on this, we should make a check on this GICR_PROPBASER.id_bits
>> before injecting LPI to domain when LPI is received.
>
>
> And doing what? The paragraph you quote doesn't say anything on what happen
> when the LPI interrupt ID is higher than the number of bits. It only explain
> what happen the this number if higher and smaller than a bound.
My intention was to convey that any LPI interrupt ID outside of number
of ID bits
set in GICR_PROPBASER.id_bits is treated by GIC as out of range.
Also the GICR_PROPBASER.id_bits should be more than 13.
So three cases are
1) If GICR_PROPBASER.id_bits is greater vgic.id_bits
then we can limit LPI property table size to vgic.id_bits
2) If GICR_PROPBASER.id_bits is greater than 13 and less than vgic.id_bits
then we can limit vgic.id_bits to GICR_PROPBASER.id_bits and LPI property table
size to GICR_PROPBASER.id_bits
3) If GIC_PROPBASER.id_bits is less than 13
then we can limit vgic.id_bits to GIC_PROPBASER.id_bits and disable LPI support
for the guest by setting.
OR do not allow any guest that is setting GICR_PROPBASER.id_bits less than
vgic.id_bits
>
> What would happen if the LPI is injected before the GICR_PROPBASER is
> enabled? See for more details on the problem here [1]
Check is required before accessing LPI property table if property
table is available
or not?.
>
> I think, you just have to make sure that the function reading the priority
> (i.e vits_get_priority) is not trying to read out of the array and return a
> dummy value (??).
>
>>>> I added it because, after updating my Linux kernel driver, I see that it
>>>> is making 32-bit access to TYPER register
>>>
>>>
>>> This change should go in a separate patch then. It's not related to this
>>> patch.
>>
>>
>> Why separate patch?. This change could be part of GICR* reg emulation
>> done for LPI
>
>
> Because this change is not part of the re-distributor emulation done for
> LPI. You don't even mention it in your commit message. I discovered it by
> comparing on what you did with the previous version.
>
> Furthermore, as said earlier, if you handle 32-bit access for GICR_TYPER you
> have to do it for every others registers.
>
> Anyway, I will send a patch myself to handle 32-bit access on 64-bit
> registers.
>
> Regards,
>
> [1]
> http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg02591.html
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-03 9:36 ` Vijay Kilari
@ 2015-08-03 13:01 ` Julien Grall
2015-08-03 13:51 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-03 13:01 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 03/08/15 10:36, Vijay Kilari wrote:
> On Sat, Aug 1, 2015 at 9:21 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 01/08/2015 11:25, Vijay Kilari wrote:
>>>>
>>>> I guess you mean vgic_enable_irqs? And no what you've implemented is
>>>> definitely not the same as vgic_enable_irqs.
>>>>
>>>> vgic_enable_irqs is locking the pending_irq structure using the vgic
>>>> lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
>>>>
>>>> Here, you are locking with the current vCPU, i.e the vCPU which wrote
>>>> into the LPI property table.
>>>>
>>>> All the vGIC code is doing the same, so using the wrong locking won't
>>>> protect this structure.
>>>
>>>
>>> With just vlpi, we cannot get target vcpu without devid. Now
>>> question is there a
>>> need to call gic_raise_guest_irq() for inflight LPIs?
>>
>>
>> Yes it's necessary. Physical LPIs can come up at any time before the guest
>> enables the virtual LPI. This is because we enable the physical LPIs and
>> route to the guest as soon as the device is assigned to it. You may be
>> interesting by the reading of [1].
>>
>> You will have to find a way to get the correct vCPU because this may occur
>> more often than you think.
>>
>
> One possible way that I can think of is
>
> In MAPVI command is sent by guest, we know vlpi. Store col_id in the
> pending_irq structure
> for the vLPI and use this collection id to retrieve correct vCPU on
> which we can take vgic lock.
MAPVI wouldn't be the only command where would have to hook up for
storing the col_id. MOVI is another candidate.
Anyway, unfortunately your solution won't work in complicate case.
The vLPI could be associated to multiple (DeviceID, EventID), this means
that different call to MAPVI may target to different collections (i.e
different VCPU).
So we would end up to protect the pending_irq structure with different
lock which could result to corrupt the Xen state.
There is a similar problem coming if the collection is moving from one
vCPU to another while an interrupt is inflight.
The only way I can see to get accurately is to store it in the
vgic_vcpu_inject_lpi. Of course this would be done only when the vLPI is
not inflight/pending.
Stefano, can you confirm this would work?
>>>>>>
>>>>>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>>>>>> +
>>>>>>> + if ( id_bits > d->arch.vgic.id_bits )
>>>>>>> + id_bits = d->arch.vgic.id_bits;
>>>>>>
>>>>>>
>>>>>> As said on v4, you are allowing the possibility to have a smaller
>>>>>> property table than the effective number of LPIs.
>>>>>>
>>>>>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>>>>>> of the size of the property table provided by the guest. This will
>>>>>> surely crash Xen in debug mode, and who knows what will happen in
>>>>>> production mode.
>>>>>
>>>>>
>>>>> lpi_size is calculated based on id_bits. If it is smaller, the
>>>>> lpi_size will be
>>>>> smaller where only size of lpi_size is considered.
>>>>
>>>>
>>>> Where id_bits is based on what the guest provides in GICR_PROPBASER.
>>>> Although the guest, malicious or not, can decide to setup this id_bits
>>>> smaller than the number of vLPIs effectively supported by the gic-v3.
>>>>
>>>> In this case, if a vLPI higher than this number is injected you will hit
>>>> the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
>>>> *should* not be able to crash Xen because it decides to send valid input.
>>>>
>>>
>>> From 8.11.19, if id_bits is < 8192 (as below statement), GIC treats
>>> LPIs as out of range.
>>
>>
>> When you quote the spec, please give both the section and which spec. I have
>> about 5 different docs for the GICv3, and I had to guess which one you were
>> using.
>>
>>>
>>> "If the value of this field is less than 0b1101, indicating that the
>>> largest interrupt ID is less than 8192
>>> (the smallest LPI interrupt ID), the GIC will behave as if all
>>> physical LPIs are out of range."
>>
>>
>> Thank you for the quoting. I helps me to not a latent bug in your LPI
>> property table emulation. I should have read more carefully the spec.
>>
>> The field IDBits gives you the number of interrupt ID bits supported. The
>> LPI property table size is only describing the LPI, i.e the offset 0 of the
>> table is the IntID 8192.
>>
>> So when you compute the size of the table, you have to substract 8192.
>> Otherwise you will remove 2 4KB pages from the guest which could be used by
>> it. You will also, possible Xen unsafe as we may read out of the pending IRQ
>> array (we are trusting the handler to be registered on valid input).
>>
>>> Based on this, we should make a check on this GICR_PROPBASER.id_bits
>>> before injecting LPI to domain when LPI is received.
>>
>>
>> And doing what? The paragraph you quote doesn't say anything on what happen
>> when the LPI interrupt ID is higher than the number of bits. It only explain
>> what happen the this number if higher and smaller than a bound.
>
> My intention was to convey that any LPI interrupt ID outside of number
> of ID bits
> set in GICR_PROPBASER.id_bits is treated by GIC as out of range.
>
> Also the GICR_PROPBASER.id_bits should be more than 13.
>
> So three cases are
>
> 1) If GICR_PROPBASER.id_bits is greater vgic.id_bits
> then we can limit LPI property table size to vgic.id_bits
Ack. This is what you already do.
> 2) If GICR_PROPBASER.id_bits is greater than 13 and less than vgic.id_bits
> then we can limit vgic.id_bits to GICR_PROPBASER.id_bits and LPI property table
> size to GICR_PROPBASER.id_bits
What do you mean by limit vgic.id_bits? You mean changing value of it?
If so, this is wrong for 2 reasons:
1) The guest may decide to setup a bigger property table later.
2) vgic.id_bits should never be touched. The GITS_TYPER is static and
defined before the guest is booted.
>
> 3) If GIC_PROPBASER.id_bits is less than 13
> then we can limit vgic.id_bits to GIC_PROPBASER.id_bits and disable LPI support
> for the guest by setting.
Same remark as 2).
> OR do not allow any guest that is setting GICR_PROPBASER.id_bits less than
> vgic.id_bits
Nack. It's valid to have GICR_PROPBASER.id_bits smaller than the number
of vgic.id_bits. For instance the guest may decide to handle less LPIs
than the number supported by the vGIC.
I may have a solution, but it would require some rework in
vgic_vcpu_inject_irq. Currently, the priority is retrieved in order to
add the IRQ in the inflight list by priority order.
The inflight list is used in various place in order to know whether we
need to inject an IRQ to the guest and/or evict an IRQ in favor of an
higher one.
Although, having non-enabled IRQ in here is IHMO pointless and is less
efficient. While this is ok currently, SPIs are disabled when the guest
is not using them, with your series LPIs may come at any point even
though the guest didn't enable it (this is because we don't replicate
the virtual enable bit in the hardware).
Furthermore, an LPI can't be enabled if there is a non-corresponding
byte in the LPI property table. This means that the priority won't be
there too. Therefore we can only safely retrieve the priority when the
IRQ is enabled.
I think we could solve the 2 problems I mentioned by a single solution.
Rather having one list for all the inflights IRQ, we could introduce a
new one to link only inflight enabled IRQ. The IRQ would be added in
this list in:
- vgic_vcpu_inject_irq if the IRQ is already enabled
- when the guest is enabling the IRQ and we need to inject the IRQ.
I'd like Stefano inputs on it here too before doing any modification.
FIY, I would be happy to provide a patch to implement what I'm saying if
you want.
>
>>
>> What would happen if the LPI is injected before the GICR_PROPBASER is
>> enabled? See for more details on the problem here [1]
>
> Check is required before accessing LPI property table if property
> table is available
> or not?.
You didn't answer to my question... What would you do if the property is
not available? Ignoring the LPI? Would the LPI can fire back after the
property table is set and the LPI enabled?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-03 13:01 ` Julien Grall
@ 2015-08-03 13:51 ` Vijay Kilari
2015-08-03 13:58 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-03 13:51 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Julien,
On Mon, Aug 3, 2015 at 6:31 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 03/08/15 10:36, Vijay Kilari wrote:
>> On Sat, Aug 1, 2015 at 9:21 PM, Julien Grall <julien.grall@citrix.com> wrote:
>>> On 01/08/2015 11:25, Vijay Kilari wrote:
>>>>>
>>>>> I guess you mean vgic_enable_irqs? And no what you've implemented is
>>>>> definitely not the same as vgic_enable_irqs.
>>>>>
>>>>> vgic_enable_irqs is locking the pending_irq structure using the vgic
>>>>> lock of the targeting VCPU (see v_target = ... ->get_target_cpu(...)).
>>>>>
>>>>> Here, you are locking with the current vCPU, i.e the vCPU which wrote
>>>>> into the LPI property table.
>>>>>
>>>>> All the vGIC code is doing the same, so using the wrong locking won't
>>>>> protect this structure.
>>>>
>>>>
>>>> With just vlpi, we cannot get target vcpu without devid. Now
>>>> question is there a
>>>> need to call gic_raise_guest_irq() for inflight LPIs?
>>>
>>>
>>> Yes it's necessary. Physical LPIs can come up at any time before the guest
>>> enables the virtual LPI. This is because we enable the physical LPIs and
>>> route to the guest as soon as the device is assigned to it. You may be
>>> interesting by the reading of [1].
>>>
>>> You will have to find a way to get the correct vCPU because this may occur
>>> more often than you think.
>>>
>>
>> One possible way that I can think of is
>>
>> In MAPVI command is sent by guest, we know vlpi. Store col_id in the
>> pending_irq structure
>> for the vLPI and use this collection id to retrieve correct vCPU on
>> which we can take vgic lock.
>
> MAPVI wouldn't be the only command where would have to hook up for
> storing the col_id. MOVI is another candidate.
>
> Anyway, unfortunately your solution won't work in complicate case.
>
> The vLPI could be associated to multiple (DeviceID, EventID), this means
> that different call to MAPVI may target to different collections (i.e
> different VCPU).
>
> So we would end up to protect the pending_irq structure with different
> lock which could result to corrupt the Xen state.
>
> There is a similar problem coming if the collection is moving from one
> vCPU to another while an interrupt is inflight.
>
> The only way I can see to get accurately is to store it in the
> vgic_vcpu_inject_lpi. Of course this would be done only when the vLPI is
> not inflight/pending.
>
> Stefano, can you confirm this would work?
>
>>>>>>>
>>>>>>>> + id_bits = ((vits->propbase & GICR_PROPBASER_IDBITS_MASK) + 1);
>>>>>>>> +
>>>>>>>> + if ( id_bits > d->arch.vgic.id_bits )
>>>>>>>> + id_bits = d->arch.vgic.id_bits;
>>>>>>>
>>>>>>>
>>>>>>> As said on v4, you are allowing the possibility to have a smaller
>>>>>>> property table than the effective number of LPIs.
>>>>>>>
>>>>>>> An ASSERT in vits_get_priority (patch #16) doesn't ensure the validity
>>>>>>> of the size of the property table provided by the guest. This will
>>>>>>> surely crash Xen in debug mode, and who knows what will happen in
>>>>>>> production mode.
>>>>>>
>>>>>>
>>>>>> lpi_size is calculated based on id_bits. If it is smaller, the
>>>>>> lpi_size will be
>>>>>> smaller where only size of lpi_size is considered.
>>>>>
>>>>>
>>>>> Where id_bits is based on what the guest provides in GICR_PROPBASER.
>>>>> Although the guest, malicious or not, can decide to setup this id_bits
>>>>> smaller than the number of vLPIs effectively supported by the gic-v3.
>>>>>
>>>>> In this case, if a vLPI higher than this number is injected you will hit
>>>>> the ASSERT(vlpi < vits->prop_size) in vits_get_priority. A guest
>>>>> *should* not be able to crash Xen because it decides to send valid input.
>>>>>
>>>>
>>>> From 8.11.19, if id_bits is < 8192 (as below statement), GIC treats
>>>> LPIs as out of range.
>>>
>>>
>>> When you quote the spec, please give both the section and which spec. I have
>>> about 5 different docs for the GICv3, and I had to guess which one you were
>>> using.
>>>
>>>>
>>>> "If the value of this field is less than 0b1101, indicating that the
>>>> largest interrupt ID is less than 8192
>>>> (the smallest LPI interrupt ID), the GIC will behave as if all
>>>> physical LPIs are out of range."
>>>
>>>
>>> Thank you for the quoting. I helps me to not a latent bug in your LPI
>>> property table emulation. I should have read more carefully the spec.
>>>
>>> The field IDBits gives you the number of interrupt ID bits supported. The
>>> LPI property table size is only describing the LPI, i.e the offset 0 of the
>>> table is the IntID 8192.
>>>
>>> So when you compute the size of the table, you have to substract 8192.
>>> Otherwise you will remove 2 4KB pages from the guest which could be used by
>>> it. You will also, possible Xen unsafe as we may read out of the pending IRQ
>>> array (we are trusting the handler to be registered on valid input).
>>>
>>>> Based on this, we should make a check on this GICR_PROPBASER.id_bits
>>>> before injecting LPI to domain when LPI is received.
>>>
>>>
>>> And doing what? The paragraph you quote doesn't say anything on what happen
>>> when the LPI interrupt ID is higher than the number of bits. It only explain
>>> what happen the this number if higher and smaller than a bound.
>>
>> My intention was to convey that any LPI interrupt ID outside of number
>> of ID bits
>> set in GICR_PROPBASER.id_bits is treated by GIC as out of range.
>>
>> Also the GICR_PROPBASER.id_bits should be more than 13.
>>
>> So three cases are
>>
>> 1) If GICR_PROPBASER.id_bits is greater vgic.id_bits
>> then we can limit LPI property table size to vgic.id_bits
>
> Ack. This is what you already do.
>
>> 2) If GICR_PROPBASER.id_bits is greater than 13 and less than vgic.id_bits
>> then we can limit vgic.id_bits to GICR_PROPBASER.id_bits and LPI property table
>> size to GICR_PROPBASER.id_bits
>
> What do you mean by limit vgic.id_bits? You mean changing value of it?
>
> If so, this is wrong for 2 reasons:
> 1) The guest may decide to setup a bigger property table later.
> 2) vgic.id_bits should never be touched. The GITS_TYPER is static and
> defined before the guest is booted.
If GICR_PROPBASER.id_bits is always subset of GITS_TYPER then
we should always limit size of LPI property table to id_bits or
vgic.id_bits whichever
is smaller.
While injecting LPI drop which is beyond LPI property table or LPI
property table is not
set.
>
>>
>> 3) If GIC_PROPBASER.id_bits is less than 13
>> then we can limit vgic.id_bits to GIC_PROPBASER.id_bits and disable LPI support
>> for the guest by setting.
>
> Same remark as 2).
>
>> OR do not allow any guest that is setting GICR_PROPBASER.id_bits less than
>> vgic.id_bits
>
> Nack. It's valid to have GICR_PROPBASER.id_bits smaller than the number
> of vgic.id_bits. For instance the guest may decide to handle less LPIs
> than the number supported by the vGIC.
>
> I may have a solution, but it would require some rework in
> vgic_vcpu_inject_irq. Currently, the priority is retrieved in order to
> add the IRQ in the inflight list by priority order.
>
> The inflight list is used in various place in order to know whether we
> need to inject an IRQ to the guest and/or evict an IRQ in favor of an
> higher one.
>
> Although, having non-enabled IRQ in here is IHMO pointless and is less
> efficient. While this is ok currently, SPIs are disabled when the guest
> is not using them, with your series LPIs may come at any point even
> though the guest didn't enable it (this is because we don't replicate
> the virtual enable bit in the hardware).
>
> Furthermore, an LPI can't be enabled if there is a non-corresponding
> byte in the LPI property table. This means that the priority won't be
> there too. Therefore we can only safely retrieve the priority when the
> IRQ is enabled.
>
> I think we could solve the 2 problems I mentioned by a single solution.
> Rather having one list for all the inflights IRQ, we could introduce a
> new one to link only inflight enabled IRQ. The IRQ would be added in
> this list in:
> - vgic_vcpu_inject_irq if the IRQ is already enabled
> - when the guest is enabling the IRQ and we need to inject the IRQ.
>
> I'd like Stefano inputs on it here too before doing any modification.
>
> FIY, I would be happy to provide a patch to implement what I'm saying if
> you want.
Yes, you can provide the patch as you have better understanding of this code.
>
>>
>>>
>>> What would happen if the LPI is injected before the GICR_PROPBASER is
>>> enabled? See for more details on the problem here [1]
>>
>> Check is required before accessing LPI property table if property
>> table is available
>> or not?.
>
> You didn't answer to my question... What would you do if the property is
> not available? Ignoring the LPI? Would the LPI can fire back after the
> property table is set and the LPI enabled?
Same as above solution. Ignore LPI as we don't have LPI property
information to inject.
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-03 13:51 ` Vijay Kilari
@ 2015-08-03 13:58 ` Julien Grall
2015-08-04 6:55 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-03 13:58 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 03/08/15 14:51, Vijay Kilari wrote:
>> If so, this is wrong for 2 reasons:
>> 1) The guest may decide to setup a bigger property table later.
>> 2) vgic.id_bits should never be touched. The GITS_TYPER is static and
>> defined before the guest is booted.
>
> If GICR_PROPBASER.id_bits is always subset of GITS_TYPER then
> we should always limit size of LPI property table to id_bits or
> vgic.id_bits whichever
> is smaller.
>
> While injecting LPI drop which is beyond LPI property table or LPI
> property table is not
> set.
I don't understand what you mean here.
>>>>
>>>> What would happen if the LPI is injected before the GICR_PROPBASER is
>>>> enabled? See for more details on the problem here [1]
>>>
>>> Check is required before accessing LPI property table if property
>>> table is available
>>> or not?.
>>
>> You didn't answer to my question... What would you do if the property is
>> not available? Ignoring the LPI? Would the LPI can fire back after the
>> property table is set and the LPI enabled?
>
> Same as above solution. Ignore LPI as we don't have LPI property
> information to inject.
That still not answering to my question. What happen if we ignore the
LPI? Would the device resend the MSI and therefore the LPI later? If
not, how do you plan to handle this?
With the LPI implementation, it's valid to receive an LPI before the
guest effectively setup the ITS (i.e property table and the mapping to a
vLPI). This is because we enable to LPI as soon as we route it to the guest.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-03 13:58 ` Julien Grall
@ 2015-08-04 6:55 ` Vijay Kilari
2015-08-04 8:44 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-04 6:55 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Mon, Aug 3, 2015 at 7:28 PM, Julien Grall <julien.grall@citrix.com> wrote:
> On 03/08/15 14:51, Vijay Kilari wrote:
>>> If so, this is wrong for 2 reasons:
>>> 1) The guest may decide to setup a bigger property table later.
>>> 2) vgic.id_bits should never be touched. The GITS_TYPER is static and
>>> defined before the guest is booted.
>>
>> If GICR_PROPBASER.id_bits is always subset of GITS_TYPER then
>> we should always limit size of LPI property table to id_bits or
>> vgic.id_bits whichever
>> is smaller.
>>
>> While injecting LPI drop which is beyond LPI property table or LPI
>> property table is not
>> set.
>
> I don't understand what you mean here.
>
>>>>>
>>>>> What would happen if the LPI is injected before the GICR_PROPBASER is
>>>>> enabled? See for more details on the problem here [1]
>>>>
>>>> Check is required before accessing LPI property table if property
>>>> table is available
>>>> or not?.
>>>
>>> You didn't answer to my question... What would you do if the property is
>>> not available? Ignoring the LPI? Would the LPI can fire back after the
>>> property table is set and the LPI enabled?
>>
>> Same as above solution. Ignore LPI as we don't have LPI property
>> information to inject.
>
> That still not answering to my question. What happen if we ignore the
> LPI? Would the device resend the MSI and therefore the LPI later? If
> not, how do you plan to handle this?
>
LPIs are edge triggered, so device won't raise again.
We can put in inflight and wait for guest to enabled it, But we are not sure
when LPI property table will be set for this vLPI. On GICR_PROPBASER update
with new property table, we parse the LPI property table and call
vits_{enable/disable}_lpi
> With the LPI implementation, it's valid to receive an LPI before the
> guest effectively setup the ITS (i.e property table and the mapping to a
> vLPI). This is because we enable to LPI as soon as we route it to the guest.
>
> Regards,
>
> --
> Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation
2015-08-04 6:55 ` Vijay Kilari
@ 2015-08-04 8:44 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-04 8:44 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 04/08/2015 07:55, Vijay Kilari wrote:
>>>>>>
>>>>>> What would happen if the LPI is injected before the GICR_PROPBASER is
>>>>>> enabled? See for more details on the problem here [1]
>>>>>
>>>>> Check is required before accessing LPI property table if property
>>>>> table is available
>>>>> or not?.
>>>>
>>>> You didn't answer to my question... What would you do if the property is
>>>> not available? Ignoring the LPI? Would the LPI can fire back after the
>>>> property table is set and the LPI enabled?
>>>
>>> Same as above solution. Ignore LPI as we don't have LPI property
>>> information to inject.
>>
>> That still not answering to my question. What happen if we ignore the
>> LPI? Would the device resend the MSI and therefore the LPI later? If
>> not, how do you plan to handle this?
>>
> LPIs are edge triggered, so device won't raise again.
>
> We can put in inflight and wait for guest to enabled it, But we are not sure
> when LPI property table will be set for this vLPI. On GICR_PROPBASER update
> with new property table, we parse the LPI property table and call
> vits_{enable/disable}_lpi
If we don't put it inflight, we will surely loose some interrupt at some
point if the device is not quiet before the guest has booted and/or
setup the device before enabling the LPI.
>> With the LPI implementation, it's valid to receive an LPI before the
>> guest effectively setup the ITS (i.e property table and the mapping to a
>> vLPI). This is because we enable to LPI as soon as we route it to the guest.
>>
>> Regards,
>>
>> --
>> Julien Grall
[1]
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg02591.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 13/22] xen/arm: ITS: Implement gic_is_lpi helper function
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (11 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 12/22] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-07-30 17:14 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 14/22] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
` (8 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper function gic_is_lpi() is used to find
if irq is lpi or not. For GICv2 platforms this function
returns 0.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-hip04.c | 6 ++++++
xen/arch/arm/gic-v2.c | 6 ++++++
xen/arch/arm/gic-v3.c | 6 ++++++
xen/arch/arm/gic.c | 5 +++++
xen/include/asm-arm/gic.h | 2 ++
5 files changed, 25 insertions(+)
diff --git a/xen/arch/arm/gic-hip04.c b/xen/arch/arm/gic-hip04.c
index c5ed545..fdd428a 100644
--- a/xen/arch/arm/gic-hip04.c
+++ b/xen/arch/arm/gic-hip04.c
@@ -87,6 +87,11 @@ static DEFINE_PER_CPU(u16, gic_cpu_id);
#define HIP04_GICH_APR 0x70
#define HIP04_GICH_LR 0x80
+static bool_t hip04gic_is_lpi(unsigned int irq)
+{
+ return 0;
+}
+
static inline void writeb_gicd(uint8_t val, unsigned int offset)
{
writeb_relaxed(val, gicv2.map_dbase + offset);
@@ -727,6 +732,7 @@ const static struct gic_hw_operations hip04gic_ops = {
.read_vmcr_priority = hip04gic_read_vmcr_priority,
.read_apr = hip04gic_read_apr,
.make_hwdom_dt_node = hip04gic_make_hwdom_dt_node,
+ .is_lpi = hip04gic_is_lpi,
};
/* Set up the GIC */
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 596126d..5cc2bca 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -81,6 +81,11 @@ static DEFINE_PER_CPU(u8, gic_cpu_id);
/* Maximum cpu interface per GIC */
#define NR_GIC_CPU_IF 8
+static bool_t gicv2_is_lpi(unsigned int irq)
+{
+ return 0;
+}
+
static inline void writeb_gicd(uint8_t val, unsigned int offset)
{
writeb_relaxed(val, gicv2.map_dbase + offset);
@@ -713,6 +718,7 @@ const static struct gic_hw_operations gicv2_ops = {
.read_vmcr_priority = gicv2_read_vmcr_priority,
.read_apr = gicv2_read_apr,
.make_hwdom_dt_node = gicv2_make_hwdom_dt_node,
+ .is_lpi = gicv2_is_lpi,
};
/* Set up the GIC */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 91c1b74..3b4dea3 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -62,6 +62,11 @@ DEFINE_PER_CPU(struct rdist, rdist);
#define GICD_RDIST_BASE (this_cpu(rdist).rbase)
#define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
+static bool_t gicv3_is_lpi(u32 irq)
+{
+ return (irq >= FIRST_GIC_LPI && irq < (1 << gicv3_info.nr_id_bits));
+}
+
/*
* Saves all 16(Max) LR registers. Though number of LRs implemented
* is implementation specific.
@@ -1316,6 +1321,7 @@ static const struct gic_hw_operations gicv3_ops = {
.read_apr = gicv3_read_apr,
.secondary_init = gicv3_secondary_cpu_init,
.make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
+ .is_lpi = gicv3_is_lpi,
};
static int __init gicv3_preinit(struct dt_device_node *node, const void *data)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index af8a34b..cb4cdc8 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -62,6 +62,11 @@ enum gic_version gic_hw_version(void)
return gic_hw_ops->info->hw_version;
}
+bool_t gic_is_lpi(unsigned int irq)
+{
+ return gic_hw_ops->is_lpi(irq);
+}
+
unsigned int gic_number_lines(void)
{
return gic_hw_ops->info->nr_lines;
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index a9a5874..f80f291 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -359,12 +359,14 @@ struct gic_hw_operations {
int (*secondary_init)(void);
int (*make_hwdom_dt_node)(const struct domain *d,
const struct dt_device_node *node, void *fdt);
+ bool_t (*is_lpi)(unsigned int irq);
};
void register_gic_ops(const struct gic_hw_operations *ops);
int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt);
+bool_t gic_is_lpi(unsigned int irq);
#endif /* __ASSEMBLY__ */
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 13/22] xen/arm: ITS: Implement gic_is_lpi helper function
2015-07-27 11:11 ` [PATCH v5 13/22] xen/arm: ITS: Implement gic_is_lpi helper function vijay.kilari
@ 2015-07-30 17:14 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-07-30 17:14 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index a9a5874..f80f291 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -359,12 +359,14 @@ struct gic_hw_operations {
> int (*secondary_init)(void);
> int (*make_hwdom_dt_node)(const struct domain *d,
> const struct dt_device_node *node, void *fdt);
> + bool_t (*is_lpi)(unsigned int irq);
I would much prefer to see a new field nr_lpis in the gic_info structure
and get the check directly in gic_is_lpi.
> };
>
> void register_gic_ops(const struct gic_hw_operations *ops);
> int gic_make_hwdom_dt_node(const struct domain *d,
> const struct dt_device_node *node,
> void *fdt);
> +bool_t gic_is_lpi(unsigned int irq);
>
> #endif /* __ASSEMBLY__ */
> #endif
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 14/22] xen/arm: ITS: Allocate irq descriptors for LPIs
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (12 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 13/22] xen/arm: ITS: Implement gic_is_lpi helper function vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-08-04 13:21 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller " vijay.kilari
` (7 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate irq descriptors for LPIs dynamically and
also update irq_to_pending helper for LPIs
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 15 +++++++++++++++
xen/arch/arm/gic-v3.c | 6 ++++++
xen/arch/arm/irq.c | 12 +++++++++++-
xen/arch/arm/vgic-v3-its.c | 22 ++++++++++++++++++++++
xen/arch/arm/vgic.c | 13 ++++++++++---
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/gic-its.h | 6 ++++++
xen/include/asm-arm/gic.h | 2 ++
xen/include/asm-arm/gic_v3_defs.h | 3 ++-
9 files changed, 75 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index e16fa03..0d17885 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -1038,6 +1038,7 @@ int __init its_init(struct rdist_prop *rdists)
struct its_node *its;
struct its_node_info *info;
struct dt_device_node *np = NULL;
+ struct irq_desc *desc;
uint32_t i, nr_its = 0;
static const struct dt_device_match its_device_ids[] __initconst =
@@ -1079,6 +1080,20 @@ int __init its_init(struct rdist_prop *rdists)
its_lpi_init(rdists->id_bits);
its_alloc_lpi_tables();
+ /* Allocate LPI irq descriptors */
+ irq_desc_lpi = xzalloc_array(struct irq_desc, num_of_lpis);
+ if ( !irq_desc_lpi )
+ return -ENOSPC;
+
+ for ( i = 0; i < num_of_lpis; i++ )
+ {
+ desc = &irq_desc_lpi[i];
+ init_one_irq_desc(desc);
+ desc->irq = FIRST_GIC_LPI + i;
+ desc->arch.type = DT_IRQ_TYPE_EDGE_BOTH;
+ desc->action = NULL;
+ }
+
return 0;
}
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 3b4dea3..98d45bc 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1280,6 +1280,12 @@ static int __init gicv3_init(void)
gicv3.rdist_stride);
gicv3_init_v2(node, dbase);
+ reg = readl_relaxed(GICD + GICD_TYPER);
+
+ gicv3.rdist_data.id_bits = ((reg >> GICD_TYPE_ID_BITS_SHIFT) &
+ GICD_TYPE_ID_BITS_MASK) + 1;
+ gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
+
spin_lock_init(&gicv3.lock);
spin_lock(&gicv3.lock);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 85cacb0..63feb43 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,6 +31,7 @@
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
+irq_desc_t *irq_desc_lpi;
/* Number of LPI supported in XEN */
unsigned int num_of_lpis = 8192;
@@ -64,7 +65,16 @@ static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
irq_desc_t *__irq_to_desc(int irq)
{
if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
- return &irq_desc[irq-NR_LOCAL_IRQS];
+ else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
+ return &irq_desc[irq-NR_LOCAL_IRQS];
+#ifdef HAS_GICV3
+ else if ( gic_is_lpi(irq) )
+ {
+ ASSERT(irq_desc_lpi != NULL);
+ return &irq_desc_lpi[irq - FIRST_GIC_LPI];
+ }
+#endif
+ return NULL;
}
int __init arch_init_one_irq_desc(struct irq_desc *desc)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 4afb62b..b8f32ed 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -69,6 +69,12 @@ void vits_setup_hw(struct gic_its_info *its_info)
vits_hw.info = its_info;
}
+bool_t is_domain_lpi(struct domain *d, unsigned int lpi)
+{
+ return ((lpi >= FIRST_GIC_LPI) &&
+ (lpi < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
+}
+
static inline uint32_t vits_get_max_collections(struct domain *d)
{
/* Collection ID is only 16 bit */
@@ -1049,6 +1055,21 @@ int vits_domain_init(struct domain *d)
vits = d->arch.vgic.vits;
+ d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq,
+ d->arch.vgic.nr_lpis);
+ if ( d->arch.vgic.pending_lpis == NULL )
+ {
+ xfree(d->arch.vgic.vits);
+ return -ENOMEM;
+ }
+
+ for ( i = 0; i < d->arch.vgic.nr_lpis; i++ )
+ {
+ INIT_LIST_HEAD(&d->arch.vgic.pending_lpis[i].inflight);
+ INIT_LIST_HEAD(&d->arch.vgic.pending_lpis[i].lr_queue);
+ d->arch.vgic.pending_lpis[i].irq = FIRST_GIC_LPI + i;
+ }
+
spin_lock_init(&vits->lock);
spin_lock_init(&vits->prop_lock);
@@ -1056,6 +1077,7 @@ int vits_domain_init(struct domain *d)
if ( !vits->collections )
{
xfree(d->arch.vgic.vits);
+ xfree(d->arch.vgic.pending_lpis);
return -ENOMEM;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index a6835a8..ab5e81b 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -30,6 +30,7 @@
#include <asm/mmio.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
@@ -375,13 +376,19 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
{
- struct pending_irq *n;
+ struct pending_irq *n = NULL;
/* Pending irqs allocation strategy: the first vgic.nr_spis irqs
- * are used for SPIs; the rests are used for per cpu irqs */
+ * are used for SPIs; the rests are used for per cpu irqs.
+ * For LPIs pending_irq structures are allocated separately */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
- else
+ else if ( irq < vgic_num_irqs(v->domain) )
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
+#ifdef HAS_GICV3
+ else if ( is_domain_lpi(v->domain, irq) )
+ n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
+#endif
+ ASSERT(n != NULL);
return n;
}
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index c4cf65e..3f26d17 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -108,6 +108,7 @@ struct arch_domain
#ifdef HAS_GICV3
/* Virtual ITS */
struct vgic_its *vits;
+ struct pending_irq *pending_lpis;
int gicr_ctlr;
/* GIC V3 addressing */
/* List of contiguous occupied by the redistributors */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 30316fd..7bb645e 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -21,11 +21,14 @@
#include <asm/gic_v3_defs.h>
#include <xen/rbtree.h>
+extern irq_desc_t *irq_desc_lpi;
/* Number of LPI supported */
extern unsigned int num_of_lpis;
#define MASK_4K 0xfffffffff000UL
#define MAPC_ITT_IPA_SHIFT 8
+/* Number of LPIs allocated per domain */
+#define NR_LPIS 8192
/*
* ITS registers, offsets from ITS_base
*/
@@ -146,6 +149,8 @@ struct vgic_its
unsigned long cmd_qsize;
/* ITS mmio physical base */
paddr_t gits_base;
+ /* ITS mmio physical size */
+ unsigned long gits_size;
/* GICR ctrl register */
uint32_t ctrl;
/* LPI propbase */
@@ -348,6 +353,7 @@ struct gic_its_info {
struct its_node_info *its_hw;
};
+bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index f80f291..2f5d5e3 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -303,6 +303,8 @@ struct gic_info {
unsigned int maintenance_irq;
/* Pointer to the device tree node representing the interrupt controller */
const struct dt_device_node *node;
+ /* Number of IRQ ID bits supported */
+ uint32_t nr_id_bits;
/* LPIs are support information */
bool_t lpi_supported;
};
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 4233bd5..68978e8 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -46,7 +46,8 @@
#define GICC_SRE_EL2_ENEL1 (1UL << 3)
/* Additional bits in GICD_TYPER defined by GICv3 */
-#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPE_ID_BITS_MASK 0x1f
#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 14/22] xen/arm: ITS: Allocate irq descriptors for LPIs
2015-07-27 11:11 ` [PATCH v5 14/22] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
@ 2015-08-04 13:21 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-04 13:21 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Allocate irq descriptors for LPIs dynamically and
> also update irq_to_pending helper for LPIs
You are mixing irq_desc and pending_irq aspect in the code which made
quite difficult to review.
It would have been helpful to have a patch for allocate irq_desc and
another for pending_irq.
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index e16fa03..0d17885 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -1038,6 +1038,7 @@ int __init its_init(struct rdist_prop *rdists)
> struct its_node *its;
> struct its_node_info *info;
> struct dt_device_node *np = NULL;
> + struct irq_desc *desc;
> uint32_t i, nr_its = 0;
>
> static const struct dt_device_match its_device_ids[] __initconst =
> @@ -1079,6 +1080,20 @@ int __init its_init(struct rdist_prop *rdists)
> its_lpi_init(rdists->id_bits);
> its_alloc_lpi_tables();
>
> + /* Allocate LPI irq descriptors */
> + irq_desc_lpi = xzalloc_array(struct irq_desc, num_of_lpis);
> + if ( !irq_desc_lpi )
> + return -ENOSPC;
> +
> + for ( i = 0; i < num_of_lpis; i++ )
> + {
> + desc = &irq_desc_lpi[i];
> + init_one_irq_desc(desc);
> + desc->irq = FIRST_GIC_LPI + i;
> + desc->arch.type = DT_IRQ_TYPE_EDGE_BOTH;
> + desc->action = NULL;
> + }
> +
This code doesn't below to the gic-v3 ITS. This should go either to
irq.c or a new file irq-lpi.c.
But IHMO, there is no harm to build it on ARM, so irq.c would be the
best place. It would avoid a lot of #ifdef HAS_GICV3 in the code and
help when we will support new version of the GIC hardware.
> return 0;
> }
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 3b4dea3..98d45bc 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1280,6 +1280,12 @@ static int __init gicv3_init(void)
> gicv3.rdist_stride);
> gicv3_init_v2(node, dbase);
>
> + reg = readl_relaxed(GICD + GICD_TYPER);
> +
> + gicv3.rdist_data.id_bits = ((reg >> GICD_TYPE_ID_BITS_SHIFT) &
> + GICD_TYPE_ID_BITS_MASK) + 1;
> + gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
> +
This is not related to IRQ allocation and you make usage of nr_id_bits
in the previous patch.
> spin_lock_init(&gicv3.lock);
>
> spin_lock(&gicv3.lock);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 85cacb0..63feb43 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,6 +31,7 @@
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
>
> +irq_desc_t *irq_desc_lpi;
If you would have handle the irq_desc_lpi allocation within this file
you could have remove the static. Which make the exposed "API" more generic.
> /* Number of LPI supported in XEN */
> unsigned int num_of_lpis = 8192;
>
> @@ -64,7 +65,16 @@ static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
> irq_desc_t *__irq_to_desc(int irq)
> {
> if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
> - return &irq_desc[irq-NR_LOCAL_IRQS];
> + else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
First, the irq >= NR_LOCAL_IRQS is not necessary at all your are in the
else of (irq < NR_LOCAL_IRQS).
Secondly, it's weird to use NR_IRQS here. After all, an LPIs is an
IRQ... That's a call for renaming the define.
> + return &irq_desc[irq-NR_LOCAL_IRQS];
> +#ifdef HAS_GICV3
> + else if ( gic_is_lpi(irq) )
> + {
> + ASSERT(irq_desc_lpi != NULL);
> + return &irq_desc_lpi[irq - FIRST_GIC_LPI];
> + }
> +#endif
I don't think the HAS_GICV3 is necessary (see my point above). If you
really want to keep it, please ensure that you don't have extra cost on
ARM32 by moving the SPI case at the end. I.e
if (irq < NR_LOCAL_IRQS)
SGIs/PPIs
#ifdef HAS_GICV3
else if ( gic_is_lpi(irq) )
LPIs
#endif
else
/* SPIs */
We expect the IRQ to be valid, so it's perfectly fine to optimize given
that the code will heavily be used.
> + return NULL;
> }
>
> int __init arch_init_one_irq_desc(struct irq_desc *desc)
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 4afb62b..b8f32ed 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -69,6 +69,12 @@ void vits_setup_hw(struct gic_its_info *its_info)
> vits_hw.info = its_info;
> }
>
> +bool_t is_domain_lpi(struct domain *d, unsigned int lpi)
> +{
> + return ((lpi >= FIRST_GIC_LPI) &&
> + (lpi < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
> +}
> +
This should go in vgic.c and not vgic-v3-its.c.
> static inline uint32_t vits_get_max_collections(struct domain *d)
> {
> /* Collection ID is only 16 bit */
> @@ -1049,6 +1055,21 @@ int vits_domain_init(struct domain *d)
>
> vits = d->arch.vgic.vits;
>
> + d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq,
> + d->arch.vgic.nr_lpis);
> + if ( d->arch.vgic.pending_lpis == NULL )
> + {
> + xfree(d->arch.vgic.vits);
> + return -ENOMEM;
> + }
> +
> + for ( i = 0; i < d->arch.vgic.nr_lpis; i++ )
> + {
> + INIT_LIST_HEAD(&d->arch.vgic.pending_lpis[i].inflight);
> + INIT_LIST_HEAD(&d->arch.vgic.pending_lpis[i].lr_queue);
> + d->arch.vgic.pending_lpis[i].irq = FIRST_GIC_LPI + i;
Please use vgic_init_pending_irq rather than hardcoding the initialization.
> + }
> +
> spin_lock_init(&vits->lock);
> spin_lock_init(&vits->prop_lock);
>
> @@ -1056,6 +1077,7 @@ int vits_domain_init(struct domain *d)
> if ( !vits->collections )
> {
> xfree(d->arch.vgic.vits);
> + xfree(d->arch.vgic.pending_lpis);
> return -ENOMEM;
> }
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a6835a8..ab5e81b 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -30,6 +30,7 @@
>
> #include <asm/mmio.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
I though I said it on the previous version... I'm against of any
inclusion of GIC ITS headers or any ITS call in the vgic.c.
This should be abstract and we should never see #ifdef HAS_GICV3 in the
common vgic code.
> #include <asm/vgic.h>
>
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
> @@ -375,13 +376,19 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
>
> struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> {
> - struct pending_irq *n;
> + struct pending_irq *n = NULL;
> /* Pending irqs allocation strategy: the first vgic.nr_spis irqs
> - * are used for SPIs; the rests are used for per cpu irqs */
> + * are used for SPIs; the rests are used for per cpu irqs.
> + * For LPIs pending_irq structures are allocated separately */
> if ( irq < 32 )
> n = &v->arch.vgic.pending_irqs[irq];
> - else
> + else if ( irq < vgic_num_irqs(v->domain) )
Another call for renaming, the LPI is also an IRQ... Please change the
name when it's necessary rather than keeping the current one. They may
not be valid after with your LPIs series.
> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
> +#ifdef HAS_GICV3
> + else if ( is_domain_lpi(v->domain, irq) )
> + n = &v->domain->arch.vgic.pending_lpis[irq - FIRST_GIC_LPI];
> +#endif
My remarks in __irq_to_desc are also valid here.
> + ASSERT(n != NULL);
> return n;
> }
>
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index c4cf65e..3f26d17 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -108,6 +108,7 @@ struct arch_domain
> #ifdef HAS_GICV3
> /* Virtual ITS */
> struct vgic_its *vits;
> + struct pending_irq *pending_lpis;
I would prefer to see it with pending_irqs above.
> int gicr_ctlr;
> /* GIC V3 addressing */
> /* List of contiguous occupied by the redistributors */
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 30316fd..7bb645e 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -21,11 +21,14 @@
> #include <asm/gic_v3_defs.h>
> #include <xen/rbtree.h>
>
> +extern irq_desc_t *irq_desc_lpi;
This is not gic-its specific.
> /* Number of LPI supported */
> extern unsigned int num_of_lpis;
>
> #define MASK_4K 0xfffffffff000UL
> #define MAPC_ITT_IPA_SHIFT 8
> +/* Number of LPIs allocated per domain */
> +#define NR_LPIS 8192
Ditto.
> /*
> * ITS registers, offsets from ITS_base
> */
> @@ -146,6 +149,8 @@ struct vgic_its
> unsigned long cmd_qsize;
> /* ITS mmio physical base */
> paddr_t gits_base;
> + /* ITS mmio physical size */
> + unsigned long gits_size;
> /* GICR ctrl register */
> uint32_t ctrl;
> /* LPI propbase */
> @@ -348,6 +353,7 @@ struct gic_its_info {
> struct its_node_info *its_hw;
> };
>
> +bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
Ditto.
> int its_init(struct rdist_prop *rdists);
> int its_cpu_init(void);
> int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index f80f291..2f5d5e3 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -303,6 +303,8 @@ struct gic_info {
> unsigned int maintenance_irq;
> /* Pointer to the device tree node representing the interrupt controller */
> const struct dt_device_node *node;
> + /* Number of IRQ ID bits supported */
> + uint32_t nr_id_bits;
Care to explain why you need to export it? IHMO, this is not related to
your patch.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (13 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 14/22] xen/arm: ITS: Allocate irq descriptors for LPIs vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-08-04 13:45 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 16/22] xen/arm: ITS: Route LPIs vijay.kilari
` (6 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Implements hw_irq_controller api's required
to handle LPI's
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Fixed review comments
- Exposed gicv3_[host|guest]_irq_end and hook to its
v4: - Implement separate hw_irq_controller for LPIs
- Drop setting LPI affinity
- virq and vid are moved under union
- Introduced inv command handling
- its_device is stored in irq_desc
---
xen/arch/arm/gic-hip04.c | 14 ++++-
xen/arch/arm/gic-v2.c | 14 ++++-
xen/arch/arm/gic-v3-its.c | 124 ++++++++++++++++++++++++++++++++++++++++-
xen/arch/arm/gic-v3.c | 33 +++++++++--
xen/arch/arm/gic.c | 14 ++++-
xen/arch/arm/irq.c | 46 +++++++++++++++
xen/include/asm-arm/gic-its.h | 2 +
xen/include/asm-arm/gic.h | 13 ++++-
xen/include/asm-arm/irq.h | 15 ++++-
9 files changed, 258 insertions(+), 17 deletions(-)
diff --git a/xen/arch/arm/gic-hip04.c b/xen/arch/arm/gic-hip04.c
index fdd428a..9ad8b6a 100644
--- a/xen/arch/arm/gic-hip04.c
+++ b/xen/arch/arm/gic-hip04.c
@@ -634,6 +634,16 @@ static hw_irq_controller hip04gic_guest_irq_type = {
.set_affinity = hip04gic_irq_set_affinity,
};
+static hw_irq_controller *hip04gic_get_host_irq_type(unsigned int irq)
+{
+ return &hip04gic_host_irq_type;
+}
+
+static hw_irq_controller *hip04gic_get_guest_irq_type(unsigned int irq)
+{
+ return &hip04gic_guest_irq_type;
+}
+
static int __init hip04gic_init(void)
{
int res;
@@ -716,8 +726,8 @@ const static struct gic_hw_operations hip04gic_ops = {
.save_state = hip04gic_save_state,
.restore_state = hip04gic_restore_state,
.dump_state = hip04gic_dump_state,
- .gic_host_irq_type = &hip04gic_host_irq_type,
- .gic_guest_irq_type = &hip04gic_guest_irq_type,
+ .gic_get_host_irq_type = hip04gic_get_host_irq_type,
+ .gic_get_guest_irq_type = hip04gic_get_guest_irq_type,
.eoi_irq = hip04gic_eoi_irq,
.deactivate_irq = hip04gic_dir_irq,
.read_irq = hip04gic_read_irq,
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 5cc2bca..5c580bc 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -620,6 +620,16 @@ static hw_irq_controller gicv2_guest_irq_type = {
.set_affinity = gicv2_irq_set_affinity,
};
+static hw_irq_controller *gicv2_get_host_irq_type(unsigned int irq)
+{
+ return &gicv2_host_irq_type;
+}
+
+static hw_irq_controller *gicv2_get_guest_irq_type(unsigned int irq)
+{
+ return &gicv2_guest_irq_type;
+}
+
static int __init gicv2_init(void)
{
int res;
@@ -702,8 +712,8 @@ const static struct gic_hw_operations gicv2_ops = {
.save_state = gicv2_save_state,
.restore_state = gicv2_restore_state,
.dump_state = gicv2_dump_state,
- .gic_host_irq_type = &gicv2_host_irq_type,
- .gic_guest_irq_type = &gicv2_guest_irq_type,
+ .gic_get_host_irq_type = gicv2_get_host_irq_type,
+ .gic_get_guest_irq_type = gicv2_get_guest_irq_type,
.eoi_irq = gicv2_eoi_irq,
.deactivate_irq = gicv2_dir_irq,
.read_irq = gicv2_read_irq,
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 0d17885..5ffd52f 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -301,8 +301,8 @@ post:
its_wait_for_range_completion(its, cmd, next_cmd);
}
-void its_send_inv(struct its_device *dev, struct its_collection *col,
- u32 event_id)
+static void its_send_inv(struct its_device *dev, struct its_collection *col,
+ u32 event_id)
{
its_cmd_block cmd;
@@ -390,6 +390,126 @@ void its_send_discard(struct its_device *dev, struct its_collection *col,
its_send_single_command(dev->its, &cmd, col);
}
+static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
+{
+ struct its_collection *col;
+ struct its_device *its_dev = get_irq_its_device(desc);
+ u32 vid = irq_to_vid(desc);
+ u16 col_id;
+
+ ASSERT(vid < its_dev->nr_lpis);
+
+ /*
+ * Make the above write visible to the redistributors.
+ * And yes, we're flushing exactly: One. Single. Byte.
+ * Humpf...
+ */
+ if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
+ else
+ dsb(ishst);
+
+ /* Get collection id for this event id */
+ col_id = irqdesc_get_collection(desc);
+ col = &its_dev->its->collections[col_id];
+ its_send_inv(its_dev, col, vid);
+}
+
+static void its_set_lpi_state(struct irq_desc *desc, int enable)
+{
+ u8 *cfg;
+
+ ASSERT(spin_is_locked(&its_lock));
+
+ cfg = gic_rdists->prop_page + desc->irq - FIRST_GIC_LPI;
+ if ( enable )
+ *cfg |= LPI_PROP_ENABLED;
+ else
+ *cfg &= ~LPI_PROP_ENABLED;
+
+ its_flush_and_invalidate_prop(desc, cfg);
+}
+
+static void its_irq_enable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ clear_bit(_IRQ_DISABLED, &desc->status);
+ dsb(sy);
+ its_set_lpi_state(desc, 1);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static void its_irq_disable(struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ ASSERT(spin_is_locked(&desc->lock));
+
+ spin_lock_irqsave(&its_lock, flags);
+ its_set_lpi_state(desc, 0);
+ set_bit(_IRQ_DISABLED, &desc->status);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
+static unsigned int its_irq_startup(struct irq_desc *desc)
+{
+ its_irq_enable(desc);
+
+ return 0;
+}
+
+static void its_irq_shutdown(struct irq_desc *desc)
+{
+ its_irq_disable(desc);
+}
+
+static void its_irq_ack(struct irq_desc *desc)
+{
+ /* No ACK -- reading IAR has done this for us */
+}
+
+static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
+{
+ /*TODO: Yet to support */
+ return;
+}
+
+static const hw_irq_controller its_host_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = gicv3_host_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
+static const hw_irq_controller its_guest_lpi_type = {
+ .typename = "gic-its",
+ .startup = its_irq_startup,
+ .shutdown = its_irq_shutdown,
+ .enable = its_irq_enable,
+ .disable = its_irq_disable,
+ .ack = its_irq_ack,
+ .end = gicv3_guest_irq_end,
+ .set_affinity = its_irq_set_affinity,
+};
+
+hw_irq_controller *its_get_host_lpi_type(void)
+{
+ return &its_host_lpi_type;
+}
+
+hw_irq_controller *its_get_guest_lpi_type(void)
+{
+ return &its_guest_lpi_type;
+}
+
/*
* How we allocate LPIs:
*
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 98d45bc..58e878e 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -40,6 +40,7 @@
#include <asm/device.h>
#include <asm/gic.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
#include <asm/cpufeature.h>
/* Global state */
@@ -1033,15 +1034,19 @@ static void gicv3_irq_ack(struct irq_desc *desc)
/* No ACK -- reading IAR has done this for us */
}
-static void gicv3_host_irq_end(struct irq_desc *desc)
+void gicv3_host_irq_end(struct irq_desc *desc)
{
/* Lower the priority */
gicv3_eoi_irq(desc);
- /* Deactivate */
- gicv3_dir_irq(desc);
+ /*
+ * LPIs does not have active state. Do do not deactivate,
+ * when EOI mode is set to 1.
+ */
+ if ( !gic_is_lpi(desc->irq) )
+ gicv3_dir_irq(desc);
}
-static void gicv3_guest_irq_end(struct irq_desc *desc)
+void gicv3_guest_irq_end(struct irq_desc *desc)
{
/* Lower the priority of the IRQ */
gicv3_eoi_irq(desc);
@@ -1146,6 +1151,22 @@ static const hw_irq_controller gicv3_guest_irq_type = {
.set_affinity = gicv3_irq_set_affinity,
};
+static hw_irq_controller *gicv3_get_host_irq_type(unsigned int irq)
+{
+ if ( gic_is_lpi(irq) )
+ return its_get_host_lpi_type();
+
+ return &gicv3_host_irq_type;
+}
+
+static hw_irq_controller *gicv3_get_guest_irq_type(unsigned int irq)
+{
+ if ( gic_is_lpi(irq) )
+ return its_get_guest_lpi_type();
+
+ return &gicv3_guest_irq_type;
+}
+
static int __init cmp_rdist(const void *a, const void *b)
{
const struct rdist_region *l = a, *r = a;
@@ -1310,8 +1331,8 @@ static const struct gic_hw_operations gicv3_ops = {
.save_state = gicv3_save_state,
.restore_state = gicv3_restore_state,
.dump_state = gicv3_dump_state,
- .gic_host_irq_type = &gicv3_host_irq_type,
- .gic_guest_irq_type = &gicv3_guest_irq_type,
+ .gic_get_host_irq_type = gicv3_get_host_irq_type,
+ .gic_get_guest_irq_type = gicv3_get_guest_irq_type,
.eoi_irq = gicv3_eoi_irq,
.deactivate_irq = gicv3_dir_irq,
.read_irq = gicv3_read_irq,
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index cb4cdc8..092087d 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -104,6 +104,16 @@ void gic_restore_state(struct vcpu *v)
gic_restore_pending_irqs(v);
}
+static inline hw_irq_controller *get_host_hw_irq_controller(unsigned int irq)
+{
+ return gic_hw_ops->gic_get_host_irq_type(irq);
+}
+
+static inline hw_irq_controller *get_guest_hw_irq_controller(unsigned int irq)
+{
+ return gic_hw_ops->gic_get_guest_irq_type(irq);
+}
+
/*
* needs to be called with a valid cpu_mask, ie each cpu in the mask has
* already called gic_cpu_init
@@ -128,7 +138,7 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
- desc->handler = gic_hw_ops->gic_host_irq_type;
+ desc->handler = get_host_hw_irq_controller(desc->irq);
gic_set_irq_properties(desc, cpu_mask, priority);
}
@@ -159,7 +169,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) )
goto out;
- desc->handler = gic_hw_ops->gic_guest_irq_type;
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
set_bit(_IRQ_GUEST, &desc->status);
gic_set_irq_properties(desc, cpumask_of(v_target->processor), priority);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 63feb43..ccbe088 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -156,6 +156,52 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
return irq_get_guest_info(desc)->d;
}
+unsigned int irq_to_virq(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->virq;
+}
+
+#ifdef HAS_GICV3
+unsigned int irq_to_vid(struct irq_desc *desc)
+{
+ ASSERT(desc->msi_desc != NULL);
+
+ return desc->msi_desc->eventID;
+}
+
+struct its_device *get_irq_its_device(struct irq_desc *desc)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(desc->msi_desc != NULL);
+
+ return desc->msi_desc->dev;
+}
+
+void set_irq_its_device(struct irq_desc *desc, struct its_device *dev)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(desc->msi_desc != NULL);
+
+ desc->msi_desc->dev = dev;
+}
+
+u16 irqdesc_get_collection(struct irq_desc *desc)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(desc->msi_desc != NULL);
+
+ return desc->msi_desc->col_id;
+}
+
+void irqdesc_set_collection(struct irq_desc *desc, uint16_t col_id)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(desc->msi_desc != NULL);
+
+ desc->msi_desc->col_id = col_id;
+}
+#endif
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 7bb645e..110516a 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -354,6 +354,8 @@ struct gic_its_info {
};
bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
+hw_irq_controller *its_get_host_lpi_type(void);
+hw_irq_controller *its_get_guest_lpi_type(void);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 2f5d5e3..2edf9de 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -287,6 +287,8 @@ extern unsigned int gic_number_lines(void);
/* LPI support info */
bool_t gic_lpi_supported(void);
+void gicv3_host_irq_end(struct irq_desc *desc);
+void gicv3_guest_irq_end(struct irq_desc *desc);
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
unsigned int *out_hwirq, unsigned int *out_type);
@@ -322,10 +324,10 @@ struct gic_hw_operations {
void (*dump_state)(const struct vcpu *);
/* hw_irq_controller to enable/disable/eoi host irq */
- hw_irq_controller *gic_host_irq_type;
+ hw_irq_controller *(*gic_get_host_irq_type)(unsigned int irq);
/* hw_irq_controller to enable/disable/eoi guest irq */
- hw_irq_controller *gic_guest_irq_type;
+ hw_irq_controller *(*gic_get_guest_irq_type)(unsigned int irq);
/* End of Interrupt */
void (*eoi_irq)(struct irq_desc *irqd);
@@ -364,6 +366,13 @@ struct gic_hw_operations {
bool_t (*is_lpi)(unsigned int irq);
};
+struct its_hw_operations {
+ /* hw_irq_controller to enable/disable/eoi host lpi */
+ hw_irq_controller *lpi_host_irq_type;
+ /* hw_irq_controller to enable/disable/eoi guest lpi */
+ hw_irq_controller *lpi_guest_irq_type;
+};
+
void register_gic_ops(const struct gic_hw_operations *ops);
int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index f33c331..5923127 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -18,6 +18,14 @@ struct arch_irq_desc {
unsigned int type;
};
+struct msi_desc {
+#ifdef HAS_GICV3
+ unsigned int eventID;
+ struct its_device *dev;
+ u16 col_id;
+#endif
+};
+
#define NR_LOCAL_IRQS 32
#define NR_IRQS 1024
@@ -51,7 +59,12 @@ void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
int irq_set_spi_type(unsigned int spi, unsigned int type);
-
+unsigned int irq_to_virq(struct irq_desc *desc);
+unsigned int irq_to_vid(struct irq_desc *desc);
+struct its_device *get_irq_its_device(struct irq_desc *desc);
+void set_irq_its_device(struct irq_desc *desc, struct its_device *dev);
+u16 irqdesc_get_collection(struct irq_desc *desc);
+void irqdesc_set_collection(struct irq_desc *desc, u16 col_id);
int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-27 11:11 ` [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller " vijay.kilari
@ 2015-08-04 13:45 ` Julien Grall
2015-08-06 8:15 ` Vijay Kilari
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-04 13:45 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Implements hw_irq_controller api's required
> to handle LPI's
You need to explain in your commit message why you switched to callback.
Relevant maintainers should be CCed too (see scripts/get_maintainers.pl).
[..]
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 5cc2bca..5c580bc 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -620,6 +620,16 @@ static hw_irq_controller gicv2_guest_irq_type = {
> .set_affinity = gicv2_irq_set_affinity,
> };
>
> +static hw_irq_controller *gicv2_get_host_irq_type(unsigned int irq)
> +{
> + return &gicv2_host_irq_type;
> +}
> +
> +static hw_irq_controller *gicv2_get_guest_irq_type(unsigned int irq)
> +{
> + return &gicv2_guest_irq_type;
> +}
> +
> static int __init gicv2_init(void)
> {
> int res;
> @@ -702,8 +712,8 @@ const static struct gic_hw_operations gicv2_ops = {
> .save_state = gicv2_save_state,
> .restore_state = gicv2_restore_state,
> .dump_state = gicv2_dump_state,
> - .gic_host_irq_type = &gicv2_host_irq_type,
> - .gic_guest_irq_type = &gicv2_guest_irq_type,
> + .gic_get_host_irq_type = gicv2_get_host_irq_type,
> + .gic_get_guest_irq_type = gicv2_get_guest_irq_type,
> .eoi_irq = gicv2_eoi_irq,
> .deactivate_irq = gicv2_dir_irq,
> .read_irq = gicv2_read_irq,
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 0d17885..5ffd52f 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -301,8 +301,8 @@ post:
> its_wait_for_range_completion(its, cmd, next_cmd);
> }
>
> -void its_send_inv(struct its_device *dev, struct its_collection *col,
> - u32 event_id)
> +static void its_send_inv(struct its_device *dev, struct its_collection *col,
> + u32 event_id)
The static belongs to the patch where you introduced its_send_inv.
> {
> its_cmd_block cmd;
>
> @@ -390,6 +390,126 @@ void its_send_discard(struct its_device *dev, struct its_collection *col,
> its_send_single_command(dev->its, &cmd, col);
> }
>
> +static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
> +{
> + struct its_collection *col;
> + struct its_device *its_dev = get_irq_its_device(desc);
> + u32 vid = irq_to_vid(desc);
> + u16 col_id;
> +
> + ASSERT(vid < its_dev->nr_lpis);
> +
> + /*
> + * Make the above write visible to the redistributors.
> + * And yes, we're flushing exactly: One. Single. Byte.
> + * Humpf...
> + */
> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> + else
> + dsb(ishst);
> +
> + /* Get collection id for this event id */
> + col_id = irqdesc_get_collection(desc);
> + col = &its_dev->its->collections[col_id];
> + its_send_inv(its_dev, col, vid);
If you would have implemented the same solution as Linux for the
collection, you could have avoid those 2 lines to get the collection.
> +}
> +static void its_irq_shutdown(struct irq_desc *desc)
> +{
> + its_irq_disable(desc);
> +}
> +
> +static void its_irq_ack(struct irq_desc *desc)
> +{
> + /* No ACK -- reading IAR has done this for us */
> +}
> +
> +static void its_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> +{
> + /*TODO: Yet to support */
/* TODO: ... */
A print/BUG_ON would be helpful to detect if we ever go here.
> + return;
> +}
> +
> +static const hw_irq_controller its_host_lpi_type = {
> + .typename = "gic-its",
> + .startup = its_irq_startup,
> + .shutdown = its_irq_shutdown,
> + .enable = its_irq_enable,
> + .disable = its_irq_disable,
> + .ack = its_irq_ack,
> + .end = gicv3_host_irq_end,
> + .set_affinity = its_irq_set_affinity,
> +};
> +
> +static const hw_irq_controller its_guest_lpi_type = {
> + .typename = "gic-its",
> + .startup = its_irq_startup,
> + .shutdown = its_irq_shutdown,
> + .enable = its_irq_enable,
> + .disable = its_irq_disable,
> + .ack = its_irq_ack,
> + .end = gicv3_guest_irq_end,
> + .set_affinity = its_irq_set_affinity,
> +};
> +
> +hw_irq_controller *its_get_host_lpi_type(void)
> +{
> + return &its_host_lpi_type;
> +}
> +
> +hw_irq_controller *its_get_guest_lpi_type(void)
> +{
> + return &its_guest_lpi_type;
> +}
> +
Please directly export the variables.
> /*
> * How we allocate LPIs:
> *
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 98d45bc..58e878e 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -40,6 +40,7 @@
> #include <asm/device.h>
> #include <asm/gic.h>
> #include <asm/gic_v3_defs.h>
> +#include <asm/gic-its.h>
> #include <asm/cpufeature.h>
>
> /* Global state */
> @@ -1033,15 +1034,19 @@ static void gicv3_irq_ack(struct irq_desc *desc)
> /* No ACK -- reading IAR has done this for us */
> }
>
> -static void gicv3_host_irq_end(struct irq_desc *desc)
> +void gicv3_host_irq_end(struct irq_desc *desc)
> {
> /* Lower the priority */
> gicv3_eoi_irq(desc);
> - /* Deactivate */
> - gicv3_dir_irq(desc);
> + /*
> + * LPIs does not have active state. Do do not deactivate,
> + * when EOI mode is set to 1.
> + */
> + if ( !gic_is_lpi(desc->irq) )
> + gicv3_dir_irq(desc);
I already told you that the goal of the hw_irq_controller is to avoid
checking the interrupt for every callback.
If the current function doesn't work for you, introduce a new one. But
don't put an if inside.
> }
>
[..]
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 63feb43..ccbe088 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -156,6 +156,52 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
> return irq_get_guest_info(desc)->d;
> }
>
> +unsigned int irq_to_virq(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->virq;
> +}
This helper is never used.
> +
> +#ifdef HAS_GICV3
> +unsigned int irq_to_vid(struct irq_desc *desc)
> +{
> + ASSERT(desc->msi_desc != NULL);
> +
> + return desc->msi_desc->eventID;
> +}
> +
> +struct its_device *get_irq_its_device(struct irq_desc *desc)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(desc->msi_desc != NULL);
> +
> + return desc->msi_desc->dev;
> +}
> +
> +void set_irq_its_device(struct irq_desc *desc, struct its_device *dev)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(desc->msi_desc != NULL);
> +
> + desc->msi_desc->dev = dev;
> +}
> +
> +u16 irqdesc_get_collection(struct irq_desc *desc)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(desc->msi_desc != NULL);
> +
> + return desc->msi_desc->col_id;
> +}
> +
> +void irqdesc_set_collection(struct irq_desc *desc, uint16_t col_id)
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(desc->msi_desc != NULL);
> +
> + desc->msi_desc->col_id = col_id;
> +}
> +#endif
Your naming is not consistent at all. Please use irqdesc_{set/get}_
everywhere rather than trying to reinvent a new help for each pair.
But IHMO, those helpers are not necessary given that they should only be
used internally within the GICv3 ITS and in a couple if not only one
places. I would prefer to see helpers to get the msi_desc and the rest
done within the GICv3 ITS code.
> +
> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
> {
> if ( desc != NULL )
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 7bb645e..110516a 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -354,6 +354,8 @@ struct gic_its_info {
> };
>
> bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
> +hw_irq_controller *its_get_host_lpi_type(void);
> +hw_irq_controller *its_get_guest_lpi_type(void);
> int its_init(struct rdist_prop *rdists);
> int its_cpu_init(void);
> int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 2f5d5e3..2edf9de 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -287,6 +287,8 @@ extern unsigned int gic_number_lines(void);
> /* LPI support info */
> bool_t gic_lpi_supported(void);
>
> +void gicv3_host_irq_end(struct irq_desc *desc);
> +void gicv3_guest_irq_end(struct irq_desc *desc);
Nack. No GICv3 specific function in the common gic header.
> /* IRQ translation function for the device tree */
> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> unsigned int *out_hwirq, unsigned int *out_type);
> @@ -322,10 +324,10 @@ struct gic_hw_operations {
> void (*dump_state)(const struct vcpu *);
>
> /* hw_irq_controller to enable/disable/eoi host irq */
> - hw_irq_controller *gic_host_irq_type;
> + hw_irq_controller *(*gic_get_host_irq_type)(unsigned int irq);
const hw_irq_controller ...
>
> /* hw_irq_controller to enable/disable/eoi guest irq */
> - hw_irq_controller *gic_guest_irq_type;
> + hw_irq_controller *(*gic_get_guest_irq_type)(unsigned int irq);
Same here.
>
> /* End of Interrupt */
> void (*eoi_irq)(struct irq_desc *irqd);
> @@ -364,6 +366,13 @@ struct gic_hw_operations {
> bool_t (*is_lpi)(unsigned int irq);
> };
>
> +struct its_hw_operations {
> + /* hw_irq_controller to enable/disable/eoi host lpi */
> + hw_irq_controller *lpi_host_irq_type;
> + /* hw_irq_controller to enable/disable/eoi guest lpi */
> + hw_irq_controller *lpi_guest_irq_type;
> +};
> +
What's for? You never use it.
> void register_gic_ops(const struct gic_hw_operations *ops);
> int gic_make_hwdom_dt_node(const struct domain *d,
> const struct dt_device_node *node,
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index f33c331..5923127 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -18,6 +18,14 @@ struct arch_irq_desc {
> unsigned int type;
> };
>
> +struct msi_desc {
> +#ifdef HAS_GICV3
> + unsigned int eventID;
> + struct its_device *dev;
> + u16 col_id;
> +#endif
> +};
> +
> #define NR_LOCAL_IRQS 32
> #define NR_IRQS 1024
>
> @@ -51,7 +59,12 @@ void arch_move_irqs(struct vcpu *v);
>
> /* Set IRQ type for an SPI */
> int irq_set_spi_type(unsigned int spi, unsigned int type);
> -
> +unsigned int irq_to_virq(struct irq_desc *desc);
> +unsigned int irq_to_vid(struct irq_desc *desc);
> +struct its_device *get_irq_its_device(struct irq_desc *desc);
> +void set_irq_its_device(struct irq_desc *desc, struct its_device *dev);
> +u16 irqdesc_get_collection(struct irq_desc *desc);
> +void irqdesc_set_collection(struct irq_desc *desc, u16 col_id);
This will likely fail to build on ARM 32 bits. That another reason I'd
like to see only a msi_desc helpers.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-08-04 13:45 ` Julien Grall
@ 2015-08-06 8:15 ` Vijay Kilari
2015-08-06 10:05 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-06 8:15 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Tue, Aug 4, 2015 at 7:15 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>> +
>> +static const hw_irq_controller its_host_lpi_type = {
>> + .typename = "gic-its",
>> + .startup = its_irq_startup,
>> + .shutdown = its_irq_shutdown,
>> + .enable = its_irq_enable,
>> + .disable = its_irq_disable,
>> + .ack = its_irq_ack,
>> + .end = gicv3_host_irq_end,
>> + .set_affinity = its_irq_set_affinity,
>> +};
>> +
>> +static const hw_irq_controller its_guest_lpi_type = {
>> + .typename = "gic-its",
>> + .startup = its_irq_startup,
>> + .shutdown = its_irq_shutdown,
>> + .enable = its_irq_enable,
>> + .disable = its_irq_disable,
>> + .ack = its_irq_ack,
>> + .end = gicv3_guest_irq_end,
>> + .set_affinity = its_irq_set_affinity,
>> +};
>> +
>> +hw_irq_controller *its_get_host_lpi_type(void)
>> +{
>> + return &its_host_lpi_type;
>> +}
>> +
>> +hw_irq_controller *its_get_guest_lpi_type(void)
>> +{
>> + return &its_guest_lpi_type;
>> +}
>> +
>
> Please directly export the variables.
These API's are called to assign hw_irq_handler to LPI
by generic code
>
>> /*
>> * How we allocate LPIs:
>> *
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 98d45bc..58e878e 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -40,6 +40,7 @@
>> #include <asm/device.h>
>> #include <asm/gic.h>
>> #include <asm/gic_v3_defs.h>
>> +#include <asm/gic-its.h>
>> #include <asm/cpufeature.h>
>>
>> /* Global state */
>> @@ -1033,15 +1034,19 @@ static void gicv3_irq_ack(struct irq_desc *desc)
>> /* No ACK -- reading IAR has done this for us */
>> }
>>
>> -static void gicv3_host_irq_end(struct irq_desc *desc)
>> +void gicv3_host_irq_end(struct irq_desc *desc)
>> {
>> /* Lower the priority */
>> gicv3_eoi_irq(desc);
>> - /* Deactivate */
>> - gicv3_dir_irq(desc);
>> + /*
>> + * LPIs does not have active state. Do do not deactivate,
>> + * when EOI mode is set to 1.
>> + */
>> + if ( !gic_is_lpi(desc->irq) )
>> + gicv3_dir_irq(desc);
>
> I already told you that the goal of the hw_irq_controller is to avoid
> checking the interrupt for every callback.
>
> If the current function doesn't work for you, introduce a new one. But
> don't put an if inside.
This is what I have done in patch v4
http://lists.xen.org/archives/html/xen-devel/2015-07/msg02128.html
But Ian suggested to export gicv3_host_irq_end instead of calling
gicv3_eoi_irq()
Regards
Vijay
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-08-06 8:15 ` Vijay Kilari
@ 2015-08-06 10:05 ` Julien Grall
2015-08-06 10:11 ` Julien Grall
0 siblings, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-06 10:05 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 06/08/15 09:15, Vijay Kilari wrote:
> On Tue, Aug 4, 2015 at 7:15 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
>>> +
>>> +static const hw_irq_controller its_host_lpi_type = {
>>> + .typename = "gic-its",
>>> + .startup = its_irq_startup,
>>> + .shutdown = its_irq_shutdown,
>>> + .enable = its_irq_enable,
>>> + .disable = its_irq_disable,
>>> + .ack = its_irq_ack,
>>> + .end = gicv3_host_irq_end,
>>> + .set_affinity = its_irq_set_affinity,
>>> +};
>>> +
>>> +static const hw_irq_controller its_guest_lpi_type = {
>>> + .typename = "gic-its",
>>> + .startup = its_irq_startup,
>>> + .shutdown = its_irq_shutdown,
>>> + .enable = its_irq_enable,
>>> + .disable = its_irq_disable,
>>> + .ack = its_irq_ack,
>>> + .end = gicv3_guest_irq_end,
>>> + .set_affinity = its_irq_set_affinity,
>>> +};
>>> +
>>> +hw_irq_controller *its_get_host_lpi_type(void)
>>> +{
>>> + return &its_host_lpi_type;
>>> +}
>>> +
>>> +hw_irq_controller *its_get_guest_lpi_type(void)
>>> +{
>>> + return &its_guest_lpi_type;
>>> +}
>>> +
>>
>> Please directly export the variables.
>
> These API's are called to assign hw_irq_handler to LPI
> by generic code
You didn't understand what I meant.
I meant dropping static from the 2 variables and directly use them in
the GICv3 code. The functions are pointless here and add another
indirection for nothing.
>
>>
>>> /*
>>> * How we allocate LPIs:
>>> *
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>> index 98d45bc..58e878e 100644
>>> --- a/xen/arch/arm/gic-v3.c
>>> +++ b/xen/arch/arm/gic-v3.c
>>> @@ -40,6 +40,7 @@
>>> #include <asm/device.h>
>>> #include <asm/gic.h>
>>> #include <asm/gic_v3_defs.h>
>>> +#include <asm/gic-its.h>
>>> #include <asm/cpufeature.h>
>>>
>>> /* Global state */
>>> @@ -1033,15 +1034,19 @@ static void gicv3_irq_ack(struct irq_desc *desc)
>>> /* No ACK -- reading IAR has done this for us */
>>> }
>>>
>>> -static void gicv3_host_irq_end(struct irq_desc *desc)
>>> +void gicv3_host_irq_end(struct irq_desc *desc)
>>> {
>>> /* Lower the priority */
>>> gicv3_eoi_irq(desc);
>>> - /* Deactivate */
>>> - gicv3_dir_irq(desc);
>>> + /*
>>> + * LPIs does not have active state. Do do not deactivate,
>>> + * when EOI mode is set to 1.
>>> + */
>>> + if ( !gic_is_lpi(desc->irq) )
>>> + gicv3_dir_irq(desc);
>>
>> I already told you that the goal of the hw_irq_controller is to avoid
>> checking the interrupt for every callback.
>>
>> If the current function doesn't work for you, introduce a new one. But
>> don't put an if inside.
>
> This is what I have done in patch v4
>
> http://lists.xen.org/archives/html/xen-devel/2015-07/msg02128.html
>
> But Ian suggested to export gicv3_host_irq_end instead of calling
> gicv3_eoi_irq()
And Ian said "Exposing those two gicv3 functions is a bit unfortunate,
but I think it will do for now.".
Which means he was opposed to the previous solution.
Anyway, I think it's a fair trade compare to the overhead you had for
everytime you received an IRQ used by Xen. This is more true given that
we don't even support LPI for Xen...
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-08-06 10:05 ` Julien Grall
@ 2015-08-06 10:11 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-06 10:11 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 06/08/15 11:05, Julien Grall wrote:
>> But Ian suggested to export gicv3_host_irq_end instead of calling
>> gicv3_eoi_irq()
>
> And Ian said "Exposing those two gicv3 functions is a bit unfortunate,
> but I think it will do for now.".
>
> Which means he was opposed to the previous solution.
*he wasn't opposed.
>
> Anyway, I think it's a fair trade compare to the overhead you had for
> everytime you received an IRQ used by Xen. This is more true given that
> we don't even support LPI for Xen...
>
> Regards,
>
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 16/22] xen/arm: ITS: Route LPIs
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (14 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 15/22] xen/arm: ITS: implement hw_irq_controller " vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-08-04 14:54 ` Julien Grall
2015-07-27 11:11 ` [PATCH v5 17/22] xen/arm: ITS: Initialize physical ITS vijay.kilari
` (5 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Allocate and initialize irq descriptor for LPIs and
route LPIs to guest
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 20 +++++++
xen/arch/arm/gic-v3.c | 17 +++++-
xen/arch/arm/gic.c | 38 ++++++++++++-
xen/arch/arm/irq.c | 126 ++++++++++++++++++++++++++++++++++++++---
xen/arch/arm/vgic-v2.c | 5 +-
xen/arch/arm/vgic-v3-its.c | 22 ++++++-
xen/arch/arm/vgic-v3.c | 17 ++++--
xen/arch/arm/vgic.c | 72 ++++++++++++++++++++++-
xen/include/asm-arm/gic-its.h | 10 ++++
xen/include/asm-arm/gic.h | 11 +++-
xen/include/asm-arm/irq.h | 2 +
xen/include/asm-arm/vgic.h | 2 +
12 files changed, 319 insertions(+), 23 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 5ffd52f..0d8582a 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -110,6 +110,11 @@ void dump_cmd(its_cmd_block *cmd)
void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
#endif
+u32 its_get_nr_event_ids(void)
+{
+ return (1 << its_data.eventid_bits);
+}
+
/* RB-tree helpers for its_device */
struct its_device *its_find_device(u32 devid)
{
@@ -415,6 +420,21 @@ static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
its_send_inv(its_dev, col, vid);
}
+void its_set_lpi_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority)
+{
+ unsigned long flags;
+ u8 *cfg;
+
+ spin_lock_irqsave(&its_lock, flags);
+ cfg = gic_rdists->prop_page + desc->irq - FIRST_GIC_LPI;
+ *cfg = (*cfg & 3) | (priority & LPI_PRIORITY_MASK) ;
+
+ its_flush_and_invalidate_prop(desc, cfg);
+ spin_unlock_irqrestore(&its_lock, flags);
+}
+
static void its_set_lpi_state(struct irq_desc *desc, int enable)
{
u8 *cfg;
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 58e878e..8c7c5cf 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -535,6 +535,16 @@ static void gicv3_set_irq_properties(struct irq_desc *desc,
spin_unlock(&gicv3.lock);
}
+static void gicv3_set_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority)
+{
+ if ( gic_is_lpi(desc->irq) )
+ its_set_lpi_properties(desc, cpu_mask, priority);
+ else
+ gicv3_set_irq_properties(desc, cpu_mask, priority);
+}
+
static void __init gicv3_dist_init(void)
{
uint32_t type;
@@ -912,7 +922,7 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
- if ( p->desc != NULL )
+ if ( p->desc != NULL && !(gic_is_lpi(p->irq)) )
val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
<< GICH_LR_PHYSICAL_SHIFT);
@@ -1312,7 +1322,10 @@ static int __init gicv3_init(void)
spin_lock(&gicv3.lock);
if ( gicv3_dist_supports_lpis() )
+ {
gicv3_info.lpi_supported = 1;
+ gicv3_info.nr_event_ids = its_get_nr_event_ids();
+ }
else
gicv3_info.lpi_supported = 0;
@@ -1336,7 +1349,7 @@ static const struct gic_hw_operations gicv3_ops = {
.eoi_irq = gicv3_eoi_irq,
.deactivate_irq = gicv3_dir_irq,
.read_irq = gicv3_read_irq,
- .set_irq_properties = gicv3_set_irq_properties,
+ .set_irq_properties = gicv3_set_properties,
.send_SGI = gicv3_send_sgi,
.disable_interface = gicv3_disable_interface,
.update_lr = gicv3_update_lr,
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 092087d..f6be0e9 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -33,6 +33,7 @@
#include <asm/device.h>
#include <asm/io.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
static void gic_restore_pending_irqs(struct vcpu *v);
@@ -67,11 +68,23 @@ bool_t gic_is_lpi(unsigned int irq)
return gic_hw_ops->is_lpi(irq);
}
+/* Returns number of PPIs/SGIs/SPIs supported */
unsigned int gic_number_lines(void)
{
return gic_hw_ops->info->nr_lines;
}
+/* Validates PPIs/SGIs/SPIs/LPIs supported */
+bool_t gic_is_valid_irq(unsigned int irq)
+{
+ return ((irq < gic_hw_ops->info->nr_lines) || gic_is_lpi(irq));
+}
+
+unsigned int gic_nr_event_ids(void)
+{
+ return gic_hw_ops->info->nr_event_ids;
+}
+
bool_t gic_lpi_supported(void)
{
return gic_hw_ops->info->lpi_supported;
@@ -134,7 +147,8 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
unsigned int priority)
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
- ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+ /* Can't route interrupts that don't exist */
+ ASSERT(gic_is_valid_irq(desc->irq));
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
@@ -183,6 +197,24 @@ out:
return res;
}
+int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc, unsigned int priority)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(virq < gic_nr_event_ids());
+
+ desc->handler = get_guest_hw_irq_controller(desc->irq);
+ set_bit(_IRQ_GUEST, &desc->status);
+
+ /* Set cpumask to current processor */
+ gic_set_irq_properties(desc, cpumask_of(smp_processor_id()), priority);
+
+ /* Enable LPI by default? */
+ desc->handler->enable(desc);
+
+ return 0;
+}
+
/* This function only works with SPIs for now */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
struct irq_desc *desc)
@@ -440,7 +472,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
{
- if ( p->desc == NULL )
+ if ( p->desc == NULL || gic_is_lpi(irq) )
{
lr_val.state |= GICH_LR_PENDING;
gic_hw_ops->write_lr(i, &lr_val);
@@ -663,7 +695,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
/* Reading IRQ will ACK it */
irq = gic_hw_ops->read_irq();
- if ( likely(irq >= 16 && irq < 1020) )
+ if ( (likely(irq >= 16 && irq < 1020)) || likely(gic_is_lpi(irq)) )
{
local_irq_enable();
do_IRQ(regs, irq, is_fiq);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index ccbe088..ae301a4 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -26,6 +26,7 @@
#include <xen/sched.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
@@ -221,7 +222,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
* which interrupt is which (messes up the interrupt freeing
* logic etc).
*/
- if ( irq >= nr_irqs )
+ if ( !gic_is_valid_irq(irq) )
return -EINVAL;
if ( !handler )
return -EINVAL;
@@ -279,11 +280,28 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
set_bit(_IRQ_INPROGRESS, &desc->status);
- /*
- * The irq cannot be a PPI, we only support delivery of SPIs to
- * guests.
- */
- vgic_vcpu_inject_spi(info->d, info->virq);
+#ifdef HAS_GICV3
+ if ( gic_is_lpi(irq) )
+ {
+ struct its_device *dev = get_irq_its_device(desc);
+ unsigned int vdevid = dev->virt_device_id;
+ unsigned int eventID = irq_to_vid(desc);
+
+ if ( info->d->domain_id != dev->domain_id)
+ {
+ printk("LPI %"PRId32" not assigned to domain.. dropping \n",
+ irq);
+ goto out_no_end;
+ }
+ vgic_vcpu_inject_lpi(info->d, vdevid, eventID);
+ }
+ else
+#endif
+ /*
+ * The irq cannot be a PPI, we only support delivery of SPIs to
+ * guests
+ */
+ vgic_vcpu_inject_spi(info->d, info->virq);
goto out_no_end;
}
@@ -449,10 +467,102 @@ err:
bool_t is_assignable_irq(unsigned int irq)
{
- /* For now, we can only route SPIs to the guest */
- return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
+ /* For now, we can only route SPI/LPIs to the guest */
+ return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
+ gic_is_lpi(irq));
}
+#ifdef HAS_GICV3
+/*
+ * Route an LPI to a specific guest.
+ */
+int route_lpi_to_guest(struct domain *d, unsigned int vid,
+ unsigned int irq, const char *devname)
+{
+ struct irqaction *action;
+ struct irq_guest *info;
+ struct irq_desc *desc;
+ unsigned long flags;
+ int retval = 0;
+
+ if ( !gic_is_lpi(irq) )
+ {
+ printk(XENLOG_G_ERR "Only LPI can be routed \n");
+ return -EINVAL;
+ }
+
+ action = xmalloc(struct irqaction);
+ if ( !action )
+ return -ENOMEM;
+
+ info = xmalloc(struct irq_guest);
+ if ( !info )
+ {
+ xfree(action);
+ return -ENOMEM;
+ }
+ info->d = d;
+
+ action->dev_id = info;
+ action->name = devname;
+ action->free_on_release = 1;
+
+ desc = irq_to_desc(irq);
+ spin_lock_irqsave(&desc->lock, flags);
+
+ ASSERT(desc->msi_desc != NULL);
+ desc->msi_desc->eventID = vid;
+ if ( desc->arch.type == DT_IRQ_TYPE_INVALID )
+ {
+ printk(XENLOG_G_ERR "LPI %u has not been configured\n", irq);
+ retval = -EIO;
+ goto out;
+ }
+
+ /*
+ * If the IRQ is already used by by same domain
+ */
+ if ( desc->action != NULL )
+ {
+ struct domain *ad = irq_get_domain(desc);
+
+ if ( test_bit(_IRQ_GUEST, &desc->status) && d == ad )
+ {
+ printk(XENLOG_G_ERR
+ "d%u: vID %u is already assigned to domain %u\n",
+ d->domain_id, irq, d->domain_id);
+ retval = -EBUSY;
+ goto out;
+ }
+ }
+
+ retval = __setup_irq(desc, 0, action);
+ if ( retval )
+ goto out;
+
+ retval = gic_route_lpi_to_guest(d, vid, desc, GIC_PRI_IRQ);
+
+ spin_unlock_irqrestore(&desc->lock, flags);
+
+ if ( retval )
+ {
+ /* TODO: for LPI */
+ release_irq(desc->irq, info);
+ goto free_info;
+ }
+
+ return 0;
+
+out:
+ spin_unlock_irqrestore(&desc->lock, flags);
+ xfree(action);
+free_info:
+ xfree(info);
+
+ return retval;
+}
+#endif
+
/*
* Route an IRQ to a specific guest.
* For now only SPIs are assignable to the guest.
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 524787b..4517a62 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -520,11 +520,14 @@ static int vgic_v2_get_irq_priority(struct vcpu *v, unsigned int irq)
{
int priority;
struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
+ unsigned long flags;
- ASSERT(spin_is_locked(&rank->lock));
+ vgic_lock_rank(v, rank, flags);
priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
+ vgic_lock_rank(v, rank, flags);
+
return priority;
}
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index b8f32ed..5323192 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -75,6 +75,11 @@ bool_t is_domain_lpi(struct domain *d, unsigned int lpi)
(lpi < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
}
+bool_t is_valid_collection(struct domain *d, uint32_t col)
+{
+ return (col <= (d->max_vcpus + 1));
+}
+
static inline uint32_t vits_get_max_collections(struct domain *d)
{
/* Collection ID is only 16 bit */
@@ -371,7 +376,7 @@ static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRId32"\n",
v, dev_id, event);
- /* TODO: Inject LPI */
+ vgic_vcpu_inject_lpi(v->domain, dev_id, event);
return 0;
}
@@ -586,6 +591,21 @@ static inline uint32_t vits_get_word(uint32_t reg_offset, uint64_t val)
return (u32)(val >> 32);
}
+uint8_t vits_get_priority(struct vcpu *v, uint32_t vlpi)
+{
+ struct vgic_its *vits = v->domain->arch.vgic.vits;
+ uint8_t priority;
+
+ ASSERT(vlpi < vits->prop_size);
+
+ spin_lock(&vits->prop_lock);
+ priority = *((u8*)vits->prop_page + vlpi);
+ priority &= LPI_PRIORITY_MASK;
+ spin_unlock(&vits->prop_lock);
+
+ return priority;
+}
+
static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
{
uint32_t offset;
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index a466a8f..9e6e3ff 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1140,12 +1140,21 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
{
- int priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
+ int priority = 0;
+ struct vgic_irq_rank *rank;
+ unsigned long flags;
- ASSERT(spin_is_locked(&rank->lock));
- priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
+ if ( !gic_is_lpi(irq) )
+ {
+ rank = vgic_rank_irq(v, irq);
+
+ vgic_lock_rank(v, rank, flags);
+ priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
irq, DABT_WORD)], 0, irq & 0x3);
+ vgic_unlock_rank(v, rank, flags);
+ }
+ else if ( gic_lpi_supported() && is_domain_lpi(v->domain, irq) )
+ priority = vits_get_priority(v, irq);
return priority;
}
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index ab5e81b..57c0f52 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -33,6 +33,12 @@
#include <asm/gic-its.h>
#include <asm/vgic.h>
+static inline bool_t is_domains_lpi(struct domain *d, unsigned int irq)
+{
+ return ((irq > FIRST_GIC_LPI) &&
+ (irq < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
+}
+
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
{
if ( rank == 0 )
@@ -414,14 +420,11 @@ void vgic_clear_pending_irqs(struct vcpu *v)
void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
{
uint8_t priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
struct pending_irq *iter, *n = irq_to_pending(v, virq);
unsigned long flags;
bool_t running;
- vgic_lock_rank(v, rank, flags);
priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
- vgic_unlock_rank(v, rank, flags);
spin_lock_irqsave(&v->arch.vgic.lock, flags);
@@ -478,6 +481,69 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
vgic_vcpu_inject_irq(v, virq);
}
+#ifdef HAS_GICV3
+void vgic_vcpu_inject_lpi(struct domain *d, unsigned int vdevid,
+ unsigned int eventID)
+{
+ struct vdevice_table dt_entry;
+ struct vitt vitt_entry;
+ uint32_t col_id, target;
+
+ if ( eventID > gic_nr_event_ids() )
+ {
+ dprintk(XENLOG_WARNING,
+ "Invalid eventID %"PRId32" ..dropping\n", vdevid);
+ return;
+ }
+
+ if ( vits_get_vdevice_entry(d, vdevid, &dt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "Failed to read dt entry for dev 0x%"PRIx32" ..dropping\n",
+ vdevid);
+ return;
+ }
+
+ if ( dt_entry.vitt_ipa == INVALID_PADDR )
+ {
+ dprintk(XENLOG_WARNING,
+ "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
+ eventID, vdevid);
+ return;
+ }
+
+ if ( vits_get_vitt_entry(d, vdevid, eventID, &vitt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
+ eventID, vdevid);
+ return;
+ }
+
+ col_id = vitt_entry.vcollection;
+
+ if ( !vitt_entry.valid || !is_valid_collection(d, col_id) ||
+ !is_domain_lpi(d, vitt_entry.vlpi) )
+ {
+ dprintk(XENLOG_WARNING,
+ "vlpi %"PRId32" for dev 0x%"PRIx32" is not valid..dropping\n",
+ vitt_entry.vlpi, vdevid);
+ return;
+ }
+
+ target = d->arch.vgic.vits->collections[col_id].target_address;
+ if ( target > d->max_vcpus )
+ {
+ dprintk(XENLOG_WARNING,
+ "vlpi %"PRId32" got invalid target %"PRId32" dev 0x%"PRIx32"\n",
+ vitt_entry.vlpi, target, vdevid);
+ return;
+ }
+
+ vgic_vcpu_inject_irq(d->vcpu[target], vitt_entry.vlpi);
+}
+#endif
+
void arch_evtchn_inject(struct vcpu *v)
{
vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 110516a..870c9a8 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -315,6 +315,10 @@ struct its_device {
u32 nr_lpis;
/* Physical Device id */
u32 device_id;
+ /* Virtual Device id */
+ u32 virt_device_id;
+ /* Domain id */
+ int domain_id;
/* RB-tree entry */
struct rb_node node;
};
@@ -353,17 +357,23 @@ struct gic_its_info {
struct its_node_info *its_hw;
};
+bool_t is_valid_collection(struct domain *d, uint32_t col);
bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
hw_irq_controller *its_get_host_lpi_type(void);
hw_irq_controller *its_get_guest_lpi_type(void);
+u32 its_get_nr_event_ids(void);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
+void its_set_lpi_properties(struct irq_desc *desc,
+ const cpumask_t *cpu_mask,
+ unsigned int priority);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
struct vdevice_table *entry);
void vits_setup_hw(struct gic_its_info *info);
int vits_map_lpi_prop(struct domain *d);
+uint8_t vits_get_priority(struct vcpu *v, uint32_t pid);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 2edf9de..15bd6eb 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -226,6 +226,9 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mas
extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
struct irq_desc *desc,
unsigned int priority);
+extern int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc,
+ unsigned int priority);
/* Remove an IRQ passthrough to a guest */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
@@ -282,8 +285,12 @@ extern void send_SGI_allbutself(enum gic_sgi sgi);
/* print useful debug info */
extern void gic_dump_info(struct vcpu *v);
-/* Number of interrupt lines */
+/* Number of interrupt lines (PPIs + SGIs + SPIs)*/
extern unsigned int gic_number_lines(void);
+/* Check if irq is valid */
+bool_t gic_is_valid_irq(unsigned int irq);
+/* Number of event ids supported */
+extern unsigned int gic_nr_event_ids(void);
/* LPI support info */
bool_t gic_lpi_supported(void);
@@ -307,6 +314,8 @@ struct gic_info {
const struct dt_device_node *node;
/* Number of IRQ ID bits supported */
uint32_t nr_id_bits;
+ /* Number of Event IDs supported */
+ uint32_t nr_event_ids;
/* LPIs are support information */
bool_t lpi_supported;
};
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 5923127..00d5345 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -49,6 +49,8 @@ bool_t is_assignable_irq(unsigned int irq);
void init_IRQ(void);
void init_secondary_IRQ(void);
+int route_lpi_to_guest(struct domain *d, unsigned int vid,
+ unsigned int irq, const char *devname);
int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned int irq, const char *devname);
int release_guest_irq(struct domain *d, unsigned int irq);
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 41cadb1..b11faa0 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -196,6 +196,8 @@ extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
+extern void vgic_vcpu_inject_lpi(struct domain *d, unsigned int devid,
+ unsigned int eventID);
extern void vgic_clear_pending_irqs(struct vcpu *v);
extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 16/22] xen/arm: ITS: Route LPIs
2015-07-27 11:11 ` [PATCH v5 16/22] xen/arm: ITS: Route LPIs vijay.kilari
@ 2015-08-04 14:54 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-04 14:54 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/15 12:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Allocate and initialize irq descriptor for LPIs and
> route LPIs to guest
On v4, I've asked to see a commit message explaining how you route the
LPI and why you don't set GICH_LR.HW.
I don't see it know, so I expect to see it in v6...
In general your commit message should explain what you are doing in the
patch. It's helpful when you go back in the log to understand what was
done and for reviewers.
[..]
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 5ffd52f..0d8582a 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -110,6 +110,11 @@ void dump_cmd(its_cmd_block *cmd)
> void dump_cmd(its_cmd_block *cmd) { do {} while ( 0 ); }
> #endif
>
> +u32 its_get_nr_event_ids(void)
> +{
> + return (1 << its_data.eventid_bits);
> +}
> +
> /* RB-tree helpers for its_device */
> struct its_device *its_find_device(u32 devid)
> {
> @@ -415,6 +420,21 @@ static void its_flush_and_invalidate_prop(struct irq_desc *desc, u8 *cfg)
> its_send_inv(its_dev, col, vid);
> }
>
> +void its_set_lpi_properties(struct irq_desc *desc,
> + const cpumask_t *cpu_mask,
> + unsigned int priority)
> +{
> + unsigned long flags;
> + u8 *cfg;
> +
> + spin_lock_irqsave(&its_lock, flags);
> + cfg = gic_rdists->prop_page + desc->irq - FIRST_GIC_LPI;
> + *cfg = (*cfg & 3) | (priority & LPI_PRIORITY_MASK) ;
> +
> + its_flush_and_invalidate_prop(desc, cfg);
> + spin_unlock_irqrestore(&its_lock, flags);
> +}
> +
> static void its_set_lpi_state(struct irq_desc *desc, int enable)
> {
> u8 *cfg;
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 58e878e..8c7c5cf 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -535,6 +535,16 @@ static void gicv3_set_irq_properties(struct irq_desc *desc,
> spin_unlock(&gicv3.lock);
> }
>
> +static void gicv3_set_properties(struct irq_desc *desc,
> + const cpumask_t *cpu_mask,
> + unsigned int priority)
> +{
> + if ( gic_is_lpi(desc->irq) )
> + its_set_lpi_properties(desc, cpu_mask, priority);
> + else
> + gicv3_set_irq_properties(desc, cpu_mask, priority);
I'm sure I've said that on a previous version. The naming of the
function/variable/macro/define is not set. It was fitting our need and
may not after the LPIs series. The name should be updated rather kept
and making the code less readable to read.
In this case, your rename the callback to gicv3_set_properties. But
properties of what? You are setting the property of an IRQ (LPIs, SPIs,
PPIs,... are all IRQs).
IHMO, the old gicv3_set_irq_properties should have been renamed in
gicv3_set_line_properties.
> +}
> +
> static void __init gicv3_dist_init(void)
> {
> uint32_t type;
> @@ -912,7 +922,7 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>
> - if ( p->desc != NULL )
> + if ( p->desc != NULL && !(gic_is_lpi(p->irq)) )
As said on v4, can you please explain why you need this? You never set
p->desc for LPI so it's not necessary.
> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> << GICH_LR_PHYSICAL_SHIFT);
>
> @@ -1312,7 +1322,10 @@ static int __init gicv3_init(void)
> spin_lock(&gicv3.lock);
>
> if ( gicv3_dist_supports_lpis() )
> + {
> gicv3_info.lpi_supported = 1;
> + gicv3_info.nr_event_ids = its_get_nr_event_ids();
> + }
> else
> gicv3_info.lpi_supported = 0;
>
> @@ -1336,7 +1349,7 @@ static const struct gic_hw_operations gicv3_ops = {
> .eoi_irq = gicv3_eoi_irq,
> .deactivate_irq = gicv3_dir_irq,
> .read_irq = gicv3_read_irq,
> - .set_irq_properties = gicv3_set_irq_properties,
> + .set_irq_properties = gicv3_set_properties,
> .send_SGI = gicv3_send_sgi,
> .disable_interface = gicv3_disable_interface,
> .update_lr = gicv3_update_lr,
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 092087d..f6be0e9 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -33,6 +33,7 @@
> #include <asm/device.h>
> #include <asm/io.h>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
I don't avoid any include of gic-its.h in the common code. The common
code shouldn't contain any GIC hardware specific.
> #include <asm/vgic.h>
>
> static void gic_restore_pending_irqs(struct vcpu *v);
> @@ -67,11 +68,23 @@ bool_t gic_is_lpi(unsigned int irq)
> return gic_hw_ops->is_lpi(irq);
> }
>
> +/* Returns number of PPIs/SGIs/SPIs supported */
> unsigned int gic_number_lines(void)
> {
> return gic_hw_ops->info->nr_lines;
> }
>
> +/* Validates PPIs/SGIs/SPIs/LPIs supported */
> +bool_t gic_is_valid_irq(unsigned int irq)
> +{
> + return ((irq < gic_hw_ops->info->nr_lines) || gic_is_lpi(irq));
> +}
> +
> +unsigned int gic_nr_event_ids(void)
> +{
> + return gic_hw_ops->info->nr_event_ids;
> +}
> +
> bool_t gic_lpi_supported(void)
> {
> return gic_hw_ops->info->lpi_supported;
> @@ -134,7 +147,8 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> unsigned int priority)
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> + /* Can't route interrupts that don't exist */
> + ASSERT(gic_is_valid_irq(desc->irq));
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> @@ -183,6 +197,24 @@ out:
> return res;
> }
>
> +int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc, unsigned int priority)
I don't understand why you have a virq parameter. You don't have a vLPI
when you route the IRQ.
> +{
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(virq < gic_nr_event_ids());
> +
> + desc->handler = get_guest_hw_irq_controller(desc->irq);
> + set_bit(_IRQ_GUEST, &desc->status);
> +
> + /* Set cpumask to current processor */
> + gic_set_irq_properties(desc, cpumask_of(smp_processor_id()), priority);
> +
> + /* Enable LPI by default? */
Please see the draft [1] and update the comment accordingly.
> + desc->handler->enable(desc);
> +
> + return 0;
> +}
> +
> /* This function only works with SPIs for now */
> int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> struct irq_desc *desc)
> @@ -440,7 +472,7 @@ static void gic_update_one_lr(struct vcpu *v, int i)
> if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) &&
> test_and_clear_bit(GIC_IRQ_GUEST_QUEUED, &p->status) )
> {
> - if ( p->desc == NULL )
> + if ( p->desc == NULL || gic_is_lpi(irq) )
Same remark as gicv3_update_lr. I would add that the LPI as no pending
state so this change is technically invalid too. Am I wrong?
> {
> lr_val.state |= GICH_LR_PENDING;
> gic_hw_ops->write_lr(i, &lr_val);
> @@ -663,7 +695,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
> /* Reading IRQ will ACK it */
> irq = gic_hw_ops->read_irq();
>
> - if ( likely(irq >= 16 && irq < 1020) )
> + if ( (likely(irq >= 16 && irq < 1020)) || likely(gic_is_lpi(irq)) )
When I asked to put in likely, I meant
likely((irq >= 16 && irq < 1020) || gic_is_lpi(irq))
> {
> local_irq_enable();
> do_IRQ(regs, irq, is_fiq);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index ccbe088..ae301a4 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -26,6 +26,7 @@
> #include <xen/sched.h>
>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
Same remark as the include in gic.c
> #include <asm/vgic.h>
>
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> @@ -221,7 +222,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
> * which interrupt is which (messes up the interrupt freeing
> * logic etc).
> */
> - if ( irq >= nr_irqs )
> + if ( !gic_is_valid_irq(irq) )
> return -EINVAL;
> if ( !handler )
> return -EINVAL;
> @@ -279,11 +280,28 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>
> set_bit(_IRQ_INPROGRESS, &desc->status);
>
> - /*
> - * The irq cannot be a PPI, we only support delivery of SPIs to
> - * guests.
> - */
> - vgic_vcpu_inject_spi(info->d, info->virq);
> +#ifdef HAS_GICV3
> + if ( gic_is_lpi(irq) )
> + {
> + struct its_device *dev = get_irq_its_device(desc);
> + unsigned int vdevid = dev->virt_device_id;
> + unsigned int eventID = irq_to_vid(desc);
> +
> + if ( info->d->domain_id != dev->domain_id)
Why this check? IHMO this could never happen if you implement correctly
the routing.
> + {
> + printk("LPI %"PRId32" not assigned to domain.. dropping \n",
> + irq);
> + goto out_no_end;
> + }
> + vgic_vcpu_inject_lpi(info->d, vdevid, eventID);
> + }
> + else
> +#endif
> + /*
> + * The irq cannot be a PPI, we only support delivery of SPIs to
> + * guests
> + */
> + vgic_vcpu_inject_spi(info->d, info->virq);
> goto out_no_end;
> }
>
> @@ -449,10 +467,102 @@ err:
>
> bool_t is_assignable_irq(unsigned int irq)
> {
> - /* For now, we can only route SPIs to the guest */
> - return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
> + /* For now, we can only route SPI/LPIs to the guest */
> + return (((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines())) ||
> + gic_is_lpi(irq));
> }
>
> +#ifdef HAS_GICV3
> +/*
> + * Route an LPI to a specific guest.
> + */
> +int route_lpi_to_guest(struct domain *d, unsigned int vid,
What does vid stand for? virtual eventID? If so, shouldn't it the same
as the physical eventID?
Overall, route_lpi_to_guest should only take an LPI in parameter.
> + unsigned int irq, const char *devname)
> +{
> + struct irqaction *action;
> + struct irq_guest *info;
> + struct irq_desc *desc;
> + unsigned long flags;
> + int retval = 0;
> +
> + if ( !gic_is_lpi(irq) )
> + {
> + printk(XENLOG_G_ERR "Only LPI can be routed \n");
> + return -EINVAL;
> + }
> +
> + action = xmalloc(struct irqaction);
> + if ( !action )
> + return -ENOMEM;
> +
> + info = xmalloc(struct irq_guest);
> + if ( !info )
> + {
> + xfree(action);
> + return -ENOMEM;
> + }
> + info->d = d;
> +
> + action->dev_id = info;
> + action->name = devname;
> + action->free_on_release = 1;
> +
> + desc = irq_to_desc(irq);
> + spin_lock_irqsave(&desc->lock, flags);
> +
> + ASSERT(desc->msi_desc != NULL);
> + desc->msi_desc->eventID = vid;
Why?
> + if ( desc->arch.type == DT_IRQ_TYPE_INVALID )
> + {
> + printk(XENLOG_G_ERR "LPI %u has not been configured\n", irq);
> + retval = -EIO;
> + goto out;
> + }
> +
> + /*
> + * If the IRQ is already used by by same domain
I suspect that you forgot to finish this comment...
> + */
> + if ( desc->action != NULL )
> + {
> + struct domain *ad = irq_get_domain(desc);
> +
> + if ( test_bit(_IRQ_GUEST, &desc->status) && d == ad )
> + {
> + printk(XENLOG_G_ERR
> + "d%u: vID %u is already assigned to domain %u\n",
> + d->domain_id, irq, d->domain_id);
> + retval = -EBUSY;
> + goto out;
> + }
> + }
> +
> + retval = __setup_irq(desc, 0, action);
> + if ( retval )
> + goto out;
> +
> + retval = gic_route_lpi_to_guest(d, vid, desc, GIC_PRI_IRQ);
> +
> + spin_unlock_irqrestore(&desc->lock, flags);
> +
> + if ( retval )
> + {
> + /* TODO: for LPI */
So if we merge this series and it's failing, you will just screw
knowingly Xen... You should just make sure that gic_route_lpi_to_guest
never return an error or support release_irq.
> /*
> * Route an IRQ to a specific guest.
> * For now only SPIs are assignable to the guest.
[..]
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index b8f32ed..5323192 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -75,6 +75,11 @@ bool_t is_domain_lpi(struct domain *d, unsigned int lpi)
> (lpi < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
> }
>
> +bool_t is_valid_collection(struct domain *d, uint32_t col)
> +{
> + return (col <= (d->max_vcpus + 1));
> +}
> +
> static inline uint32_t vits_get_max_collections(struct domain *d)
> {
> /* Collection ID is only 16 bit */
> @@ -371,7 +376,7 @@ static int vits_process_int(struct vcpu *v, struct vgic_its *vits,
> DPRINTK("%pv: vITS: INT: Device 0x%"PRIx32" id %"PRId32"\n",
> v, dev_id, event);
>
> - /* TODO: Inject LPI */
> + vgic_vcpu_inject_lpi(v->domain, dev_id, event);
>
> return 0;
> }
> @@ -586,6 +591,21 @@ static inline uint32_t vits_get_word(uint32_t reg_offset, uint64_t val)
> return (u32)(val >> 32);
> }
>
> +uint8_t vits_get_priority(struct vcpu *v, uint32_t vlpi)
> +{
> + struct vgic_its *vits = v->domain->arch.vgic.vits;
> + uint8_t priority;
> +
> + ASSERT(vlpi < vits->prop_size);
> +
> + spin_lock(&vits->prop_lock);
Please see my comment on #12 about the lock. I won't repeat here.
> + priority = *((u8*)vits->prop_page + vlpi);
Looks like I forgot to mention it on v4. You should substract 8192 to
the vLPI in order to the offset in the property table.
> + priority &= LPI_PRIORITY_MASK;
You need to explain why you just only mask and not shift.
> + spin_unlock(&vits->prop_lock);
> +
> + return priority;
> +}
> +
> static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> {
> uint32_t offset;
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index a466a8f..9e6e3ff 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1140,12 +1140,21 @@ static const struct mmio_handler_ops vgic_distr_mmio_handler = {
>
> static int vgic_v3_get_irq_priority(struct vcpu *v, unsigned int irq)
> {
> - int priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, irq);
> + int priority = 0;
> + struct vgic_irq_rank *rank;
> + unsigned long flags;
>
> - ASSERT(spin_is_locked(&rank->lock));
> - priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> + if ( !gic_is_lpi(irq) )
> + {
> + rank = vgic_rank_irq(v, irq);
> +
> + vgic_lock_rank(v, rank, flags);
> + priority = vgic_byte_read(rank->ipriority[REG_RANK_INDEX(8,
> irq, DABT_WORD)], 0, irq & 0x3);
> + vgic_unlock_rank(v, rank, flags);
> + }
> + else if ( gic_lpi_supported() && is_domain_lpi(v->domain, irq) )
The gic_lpi_supported is pointless here. If the LPI is not supported,
the guest would have the nr_lpis fields equal to 0.
> + priority = vits_get_priority(v, irq);
>
> return priority;
> }
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index ab5e81b..57c0f52 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -33,6 +33,12 @@
> #include <asm/gic-its.h>
> #include <asm/vgic.h>
>
> +static inline bool_t is_domains_lpi(struct domain *d, unsigned int irq)
> +{
> + return ((irq > FIRST_GIC_LPI) &&
> + (irq < (d->arch.vgic.nr_lpis + FIRST_GIC_LPI)));
> +}
> +
This not used.
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
> {
> if ( rank == 0 )
> @@ -414,14 +420,11 @@ void vgic_clear_pending_irqs(struct vcpu *v)
> void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
> {
> uint8_t priority;
> - struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
> struct pending_irq *iter, *n = irq_to_pending(v, virq);
> unsigned long flags;
> bool_t running;
>
> - vgic_lock_rank(v, rank, flags);
> priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
> - vgic_unlock_rank(v, rank, flags);
>
> spin_lock_irqsave(&v->arch.vgic.lock, flags);
>
> @@ -478,6 +481,69 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
> vgic_vcpu_inject_irq(v, virq);
> }
>
> +#ifdef HAS_GICV3
> +void vgic_vcpu_inject_lpi(struct domain *d, unsigned int vdevid,
> + unsigned int eventID)
You will have to address the problem mentioned in [2] at some point
before it's merged. This is a real one, and as you said in a previous
mail the interrupt will never came back.
> +{
> + struct vdevice_table dt_entry;
> + struct vitt vitt_entry;
> + uint32_t col_id, target;
> +
> + if ( eventID > gic_nr_event_ids() )
> + {
> + dprintk(XENLOG_WARNING,
> + "Invalid eventID %"PRId32" ..dropping\n", vdevid);
> + return;
> + }
This is not possible. Please drop this check.
> +
> + if ( vits_get_vdevice_entry(d, vdevid, &dt_entry) )
> + {
> + dprintk(XENLOG_WARNING,
> + "Failed to read dt entry for dev 0x%"PRIx32" ..dropping\n",
> + vdevid);
> + return;
> + }
> +
> + if ( dt_entry.vitt_ipa == INVALID_PADDR )
> + {
> + dprintk(XENLOG_WARNING,
> + "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
> + eventID, vdevid);
> + return;
> + }
> +
> + if ( vits_get_vitt_entry(d, vdevid, eventID, &vitt_entry) )
> + {
> + dprintk(XENLOG_WARNING,
> + "Event %"PRId32" of dev 0x%"PRIx32" is invalid..dropping\n",
> + eventID, vdevid);
> + return;
> + }
> +
> + col_id = vitt_entry.vcollection;
> +
> + if ( !vitt_entry.valid || !is_valid_collection(d, col_id) ||
> + !is_domain_lpi(d, vitt_entry.vlpi) )
> + {
> + dprintk(XENLOG_WARNING,
> + "vlpi %"PRId32" for dev 0x%"PRIx32" is not valid..dropping\n",
> + vitt_entry.vlpi, vdevid);
> + return;
> + }
> +
> + target = d->arch.vgic.vits->collections[col_id].target_address;
> + if ( target > d->max_vcpus )
> + {
> + dprintk(XENLOG_WARNING,
> + "vlpi %"PRId32" got invalid target %"PRId32" dev 0x%"PRIx32"\n",
> + vitt_entry.vlpi, target, vdevid);
> + return;
> + }
> +
> + vgic_vcpu_inject_irq(d->vcpu[target], vitt_entry.vlpi);
> +}
> +#endif
> +
> void arch_evtchn_inject(struct vcpu *v)
> {
> vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
Regards,
[1]
http://xenbits.xen.org/people/ianc/vits/draftF.html#handling-of-unroutedspurious-lpis
[2]
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg02591.html
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 17/22] xen/arm: ITS: Initialize physical ITS
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (15 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 16/22] xen/arm: ITS: Route LPIs vijay.kilari
@ 2015-07-27 11:11 ` vijay.kilari
2015-08-17 19:00 ` Julien Grall
2015-07-27 11:12 ` [PATCH v5 18/22] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
` (4 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:11 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Initialize physical ITS driver from GIC v3 driver
if LPIs are supported by hardware
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: Made check of its dt node availability before
setting lpi_supported flag
---
xen/arch/arm/gic-v3.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 8c7c5cf..23eb47c 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -714,6 +714,10 @@ static int __cpuinit gicv3_cpu_init(void)
if ( gicv3_enable_redist() )
return -ENODEV;
+ /* Give LPIs a spin */
+ if ( gicv3_info.lpi_supported )
+ its_cpu_init();
+
/* Set priority on PPI and SGI interrupts */
priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
GIC_PRI_IPI);
@@ -1323,11 +1327,18 @@ static int __init gicv3_init(void)
if ( gicv3_dist_supports_lpis() )
{
- gicv3_info.lpi_supported = 1;
- gicv3_info.nr_event_ids = its_get_nr_event_ids();
+ /*
+ * LPI support is enabled only if HW supports it and
+ * ITS dt node is available
+ */
+ if ( !its_init(&gicv3.rdist_data) )
+ {
+ gicv3_info.lpi_supported = 1;
+ gicv3_info.nr_event_ids = its_get_nr_event_ids();
+ }
+ else
+ gicv3_info.lpi_supported = 0;
}
- else
- gicv3_info.lpi_supported = 0;
gicv3_dist_init();
res = gicv3_cpu_init();
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 17/22] xen/arm: ITS: Initialize physical ITS
2015-07-27 11:11 ` [PATCH v5 17/22] xen/arm: ITS: Initialize physical ITS vijay.kilari
@ 2015-08-17 19:00 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-17 19:00 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/2015 04:11, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Initialize physical ITS driver from GIC v3 driver
> if LPIs are supported by hardware
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v5: Made check of its dt node availability before
> setting lpi_supported flag
> ---
> xen/arch/arm/gic-v3.c | 19 +++++++++++++++----
> 1 file changed, 15 insertions(+), 4 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 8c7c5cf..23eb47c 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -714,6 +714,10 @@ static int __cpuinit gicv3_cpu_init(void)
> if ( gicv3_enable_redist() )
> return -ENODEV;
>
> + /* Give LPIs a spin */
> + if ( gicv3_info.lpi_supported )
> + its_cpu_init();
> +
> /* Set priority on PPI and SGI interrupts */
> priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
> GIC_PRI_IPI);
> @@ -1323,11 +1327,18 @@ static int __init gicv3_init(void)
>
> if ( gicv3_dist_supports_lpis() )
> {
> - gicv3_info.lpi_supported = 1;
> - gicv3_info.nr_event_ids = its_get_nr_event_ids();
> + /*
> + * LPI support is enabled only if HW supports it and
> + * ITS dt node is available
> + */
> + if ( !its_init(&gicv3.rdist_data) )
> + {
> + gicv3_info.lpi_supported = 1;
> + gicv3_info.nr_event_ids = its_get_nr_event_ids();
> + }
> + else
> + gicv3_info.lpi_supported = 0;
I think this is wrong to use LPI supported to know whether ITS is
present or not.
AFAICT, most the usage of lpi_supported is only for vgic where you
should have a proper field in the vgic structure rather than using this
field.
For the rest, we may want to support LPIs without ITS at some point. So
we should keep 2 distincts field for this purpose. Although, I believe
that we can drop most of them.
> }
> - else
> - gicv3_info.lpi_supported = 0;
Why did you drop the else? It was valid...
>
> gicv3_dist_init();
> res = gicv3_cpu_init();
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 18/22] xen/arm: ITS: Add domain specific ITS initialization
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (16 preceding siblings ...)
2015-07-27 11:11 ` [PATCH v5 17/22] xen/arm: ITS: Initialize physical ITS vijay.kilari
@ 2015-07-27 11:12 ` vijay.kilari
2015-08-17 18:57 ` Julien Grall
2015-07-27 11:12 ` [PATCH v5 19/22] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
` (3 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:12 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Domain and vcpu specific ITS initialization
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/vgic-v3-its.c | 10 ++++++++++
xen/arch/arm/vgic-v3.c | 10 ++++++++++
xen/arch/arm/vgic.c | 3 +++
xen/include/asm-arm/gic-its.h | 2 ++
xen/include/asm-arm/vgic.h | 3 +++
5 files changed, 28 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 5323192..e182cee 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -44,6 +44,7 @@
#define GITS_PIDR4_VAL (0x04)
#define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
(0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
+//#define DEBUG_ITS
#ifdef DEBUG_ITS
# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
#else
@@ -1122,6 +1123,15 @@ int vits_domain_init(struct domain *d)
return 0;
}
+void vits_domain_free(struct domain *d)
+{
+ free_xenheap_pages(d->arch.vgic.vits->prop_page,
+ get_order_from_bytes(d->arch.vgic.vits->prop_size));
+ xfree(d->arch.vgic.pending_lpis);
+ xfree(d->arch.vgic.vits->collections);
+ xfree(d->arch.vgic.vits);
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 9e6e3ff..a09ba36 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1299,12 +1299,22 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
+ if ( is_hardware_domain(d) && gic_lpi_supported() )
+ vits_domain_init(d);
+
return 0;
}
+void vgic_v3_domain_free(struct domain *d)
+{
+ if ( is_hardware_domain(d) && gic_lpi_supported() )
+ vits_domain_free(d);
+}
+
static const struct vgic_ops v3_ops = {
.vcpu_init = vgic_v3_vcpu_init,
.domain_init = vgic_v3_domain_init,
+ .domain_free = vgic_v3_domain_free,
.get_irq_priority = vgic_v3_get_irq_priority,
.get_target_vcpu = vgic_v3_get_target_vcpu,
.emulate_sysreg = vgic_v3_emulate_sysreg,
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 57c0f52..e2bfdb6 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -166,6 +166,9 @@ void domain_vgic_free(struct domain *d)
xfree(d->arch.vgic.shared_irqs);
xfree(d->arch.vgic.pending_irqs);
xfree(d->arch.vgic.allocated_irqs);
+
+ if ( !d->arch.vgic.handler->domain_free )
+ d->arch.vgic.handler->domain_free(d);
}
int vcpu_vgic_init(struct vcpu *v)
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 870c9a8..da689a4 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -367,6 +367,8 @@ int its_cpu_init(void);
void its_set_lpi_properties(struct irq_desc *desc,
const cpumask_t *cpu_mask,
unsigned int priority);
+int vits_domain_init(struct domain *d);
+void vits_domain_free(struct domain *d);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index b11faa0..853df04 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -114,6 +114,8 @@ struct vgic_ops {
int (*vcpu_init)(struct vcpu *v);
/* Domain specific initialization of vGIC */
int (*domain_init)(struct domain *d);
+ /* Free domain specific resources */
+ void (*domain_free)(struct domain *d);
/* Get priority for a given irq stored in vgic structure */
int (*get_irq_priority)(struct vcpu *v, unsigned int irq);
/* Get the target vcpu for a given virq. The rank lock is already taken
@@ -191,6 +193,7 @@ enum gic_sgi_mode;
#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
+extern void vgic_its_init(void);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 18/22] xen/arm: ITS: Add domain specific ITS initialization
2015-07-27 11:12 ` [PATCH v5 18/22] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-08-17 18:57 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-17 18:57 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
This patch doesn't only do initialization but also destruction of vGIC info.
Although, a part of the domain initialization is done in part in #8 and
#8. It's very confusing to see the initialization split in 3 different
patch.
I would prefer to see the initialization/destruction in a single patch.
It would help to catch whether you forgot to see you forgot some bits.
On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Domain and vcpu specific ITS initialization
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 10 ++++++++++
> xen/arch/arm/vgic-v3.c | 10 ++++++++++
> xen/arch/arm/vgic.c | 3 +++
> xen/include/asm-arm/gic-its.h | 2 ++
> xen/include/asm-arm/vgic.h | 3 +++
> 5 files changed, 28 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 5323192..e182cee 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -44,6 +44,7 @@
> #define GITS_PIDR4_VAL (0x04)
> #define GITS_BASER_INIT_VAL ((1UL << GITS_BASER_TYPE_SHIFT) | \
> (0x7UL << GITS_BASER_ENTRY_SIZE_SHIFT))
> +//#define DEBUG_ITS
Why here?
> #ifdef DEBUG_ITS
> # define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> #else
> @@ -1122,6 +1123,15 @@ int vits_domain_init(struct domain *d)
> return 0;
> }
>
> +void vits_domain_free(struct domain *d)
> +{
> + free_xenheap_pages(d->arch.vgic.vits->prop_page,
> + get_order_from_bytes(d->arch.vgic.vits->prop_size));
> + xfree(d->arch.vgic.pending_lpis);
> + xfree(d->arch.vgic.vits->collections);
> + xfree(d->arch.vgic.vits);
For instance, you add the initialization of these fields in patch #8 and
#10 but free everything here. I have to go in the others patch to get if
you forgot something or not.
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 9e6e3ff..a09ba36 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -1299,12 +1299,22 @@ static int vgic_v3_domain_init(struct domain *d)
>
> d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
>
> + if ( is_hardware_domain(d) && gic_lpi_supported() )
I think gic_lpi_supported is badly name. vITS should only be supported
when ITS is present, not when LPIs is present.
And I know that you unset LPIs when ITS is not present. But this confuse
the reader to do this.
> + vits_domain_init(d);
> +
> return 0;
> }
>
> +void vgic_v3_domain_free(struct domain *d)
> +{
> + if ( is_hardware_domain(d) && gic_lpi_supported() )
> + vits_domain_free(d);
> +}
> +
> static const struct vgic_ops v3_ops = {
> .vcpu_init = vgic_v3_vcpu_init,
> .domain_init = vgic_v3_domain_init,
> + .domain_free = vgic_v3_domain_free,
> .get_irq_priority = vgic_v3_get_irq_priority,
> .get_target_vcpu = vgic_v3_get_target_vcpu,
> .emulate_sysreg = vgic_v3_emulate_sysreg,
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 57c0f52..e2bfdb6 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -166,6 +166,9 @@ void domain_vgic_free(struct domain *d)
> xfree(d->arch.vgic.shared_irqs);
> xfree(d->arch.vgic.pending_irqs);
> xfree(d->arch.vgic.allocated_irqs);
> +
> + if ( !d->arch.vgic.handler->domain_free )
The test is wrong. If domain_free is NULL you will end up to dereference
the pointer and crash.
> + d->arch.vgic.handler->domain_free(d);
> }
>
> int vcpu_vgic_init(struct vcpu *v)
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 870c9a8..da689a4 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -367,6 +367,8 @@ int its_cpu_init(void);
> void its_set_lpi_properties(struct irq_desc *desc,
> const cpumask_t *cpu_mask,
> unsigned int priority);
> +int vits_domain_init(struct domain *d);
> +void vits_domain_free(struct domain *d);
> int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> uint32_t event, struct vitt *entry);
> int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index b11faa0..853df04 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -114,6 +114,8 @@ struct vgic_ops {
> int (*vcpu_init)(struct vcpu *v);
> /* Domain specific initialization of vGIC */
> int (*domain_init)(struct domain *d);
> + /* Free domain specific resources */
> + void (*domain_free)(struct domain *d);
> /* Get priority for a given irq stored in vgic structure */
> int (*get_irq_priority)(struct vcpu *v, unsigned int irq);
> /* Get the target vcpu for a given virq. The rank lock is already taken
> @@ -191,6 +193,7 @@ enum gic_sgi_mode;
> #define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
>
> extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
> +extern void vgic_its_init(void);
Where does it come from?
> extern void domain_vgic_free(struct domain *d);
> extern int vcpu_vgic_init(struct vcpu *v);
> extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 19/22] xen/arm: ITS: Add APIs to add and assign device
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (17 preceding siblings ...)
2015-07-27 11:12 ` [PATCH v5 18/22] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-07-27 11:12 ` vijay.kilari
2015-08-17 19:17 ` Julien Grall
2015-07-27 11:12 ` [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space vijay.kilari
` (2 subsequent siblings)
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:12 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add APIs to add devices to RB-tree, assign and remove
devices to domain.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Removed its_detach_device API
- Pass nr_ites as parameter to its_add_device
v4: - Introduced helper to populate its_device struct
- Fixed freeing of its_device memory
- its_device struct holds domain id
---
xen/arch/arm/gic-v3-its.c | 224 +++++++++++++++++++++++++++++++++++++++--
xen/arch/arm/irq.c | 8 ++
xen/include/asm-arm/gic-its.h | 2 +
xen/include/asm-arm/irq.h | 1 +
4 files changed, 226 insertions(+), 9 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 0d8582a..99f6edc 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -115,8 +115,21 @@ u32 its_get_nr_event_ids(void)
return (1 << its_data.eventid_bits);
}
+static struct its_node *its_get_phys_node(struct dt_device_node *dt)
+{
+ struct its_node *its;
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ if ( its->dt_node == dt )
+ return its;
+ }
+
+ return NULL;
+}
+
/* RB-tree helpers for its_device */
-struct its_device *its_find_device(u32 devid)
+static struct its_device *its_find_device(u32 devid)
{
struct rb_node *node = rb_its_dev.rb_node;
@@ -137,7 +150,7 @@ struct its_device *its_find_device(u32 devid)
return NULL;
}
-int its_insert_device(struct its_device *dev)
+static int its_insert_device(struct its_device *dev)
{
struct rb_node **new, *parent;
@@ -319,7 +332,7 @@ static void its_send_inv(struct its_device *dev, struct its_collection *col,
its_send_single_command(dev->its, &cmd, col);
}
-void its_send_mapd(struct its_device *dev, int valid)
+static void its_send_mapd(struct its_device *dev, int valid)
{
its_cmd_block cmd;
unsigned long itt_addr;
@@ -357,8 +370,8 @@ static void its_send_mapc(struct its_node *its, struct its_collection *col,
its_send_single_command(its, &cmd, col);
}
-void its_send_mapvi(struct its_device *dev, struct its_collection *col,
- u32 phys_id, u32 event)
+static void its_send_mapvi(struct its_device *dev, struct its_collection *col,
+ u32 phys_id, u32 event)
{
its_cmd_block cmd;
@@ -383,8 +396,8 @@ static void its_send_invall(struct its_node *its, struct its_collection *col)
its_send_single_command(its, &cmd, NULL);
}
-void its_send_discard(struct its_device *dev, struct its_collection *col,
- u32 event)
+static void its_send_discard(struct its_device *dev, struct its_collection *col,
+ u32 event)
{
its_cmd_block cmd;
@@ -573,7 +586,7 @@ static int its_lpi_init(u32 id_bits)
return 0;
}
-unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
+static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base)
{
unsigned long *bitmap = NULL;
int chunk_id, nr_chunks, nr_ids, i;
@@ -617,7 +630,7 @@ out:
return bitmap;
}
-void its_lpi_free(struct its_device *dev)
+static void its_lpi_free(struct its_device *dev)
{
int lpi;
@@ -639,6 +652,199 @@ void its_lpi_free(struct its_device *dev)
xfree(dev->lpi_map);
}
+static void its_discard_lpis(struct its_device *dev, u32 ids)
+{
+ struct its_collection *col;
+ int i;
+
+ for ( i = 0; i < ids; i++)
+ {
+ col = &dev->its->collections[(i % nr_cpu_ids)];
+ its_send_discard(dev, col, i);
+ }
+}
+
+static inline u32 its_get_plpi(struct its_device *dev, u32 event)
+{
+ return dev->lpi_base + event;
+}
+
+static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
+{
+ int idx;
+
+ idx = find_first_zero_bit(dev->lpi_map, dev->nr_lpis);
+ if ( idx == dev->nr_lpis )
+ return -ENOSPC;
+
+ *hwirq = its_get_plpi(dev, idx);
+ set_bit(idx, dev->lpi_map);
+
+ return 0;
+}
+
+static void its_free_device(struct its_device *dev)
+{
+ xfree(dev->itt_addr);
+ xfree(dev->lpi_map);
+ xfree(dev);
+}
+
+static struct its_device *its_alloc_device(u32 devid, u32 nr_ites,
+ struct dt_device_node *dt_its)
+{
+ struct its_device *dev;
+ paddr_t *itt;
+ unsigned long *lpi_map;
+ int lpi_base, sz;
+
+ dev = xzalloc(struct its_device);
+ if ( dev == NULL )
+ return NULL;
+
+ dev->its = its_get_phys_node(dt_its);
+ if (dev->its == NULL)
+ {
+ dprintk(XENLOG_G_ERR, "ITS: Failed to find ITS node 0x%"PRIx32"\n",
+ devid);
+ goto err;
+ }
+
+ sz = nr_ites * dev->its->ite_size;
+ sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
+ itt = xzalloc_bytes(sz);
+ if ( !itt )
+ goto err;
+
+ lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base);
+ if ( !lpi_map )
+ goto lpi_err;
+
+ dev->itt_addr = itt;
+ dev->lpi_map = lpi_map;
+ dev->lpi_base = lpi_base;
+ dev->nr_lpis = nr_ites;
+ dev->device_id = devid;
+
+ return dev;
+
+lpi_err:
+ xfree(itt);
+ xfree(lpi_map);
+err:
+ xfree(dev);
+
+ return NULL;
+}
+
+/* Device assignment */
+int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its)
+{
+ struct its_device *dev;
+ u32 i, plpi = 0;
+ struct its_collection *col;
+ struct irq_desc *desc;
+ struct msi_desc *msi = NULL;
+ int res = 0;
+
+ spin_lock(&rb_its_dev_lock);
+ dev = its_find_device(devid);
+ if ( dev )
+ {
+ dprintk(XENLOG_G_ERR, "ITS: Device already exists 0x%"PRIx32"\n",
+ dev->device_id);
+ res = -EEXIST;
+ goto err;
+ }
+
+ dev = its_alloc_device(devid, nr_ites, dt_its);
+ if ( !dev )
+ {
+ res = -ENOMEM;
+ goto err;
+ }
+
+ if ( its_insert_device(dev) )
+ {
+ its_free_device(dev);
+ dprintk(XENLOG_G_ERR, "ITS: failed to insert device 0x%"PRIx32"\n",
+ devid);
+ res = -EINVAL;
+ goto err;
+ }
+
+ DPRINTK("ITS:Add Device 0x%"PRIx32" lpis %"PRId32" base 0x%"PRIx32"\n",
+ dev->device_id, dev->nr_lpis, dev->lpi_base);
+
+ /* Map device to its ITT */
+ its_send_mapd(dev, 1);
+
+ for ( i = 0; i < dev->nr_lpis; i++ )
+ {
+ msi = xzalloc(struct msi_desc);
+ /* Reserve pLPI */
+ if ( its_alloc_device_irq(dev, &plpi) || !msi )
+ {
+ its_discard_lpis(dev, i);
+ its_lpi_free(dev);
+ its_send_mapd(dev, 0);
+ its_free_device(dev);
+ dprintk(XENLOG_G_ERR,"ITS: Cannot add device 0x%"PRIx32"\n", devid);
+ res = -ENOSPC;
+ goto err;
+ }
+
+ /* For each pLPI send MAPVI command */
+ col = &dev->its->collections[(i % nr_cpu_ids)];
+ /* Store collection for this plpi in irq_desc */
+ desc = irq_to_desc(plpi);
+ spin_lock(&desc->lock);
+ desc->msi_desc = msi;
+ set_lpi_event(desc, i);
+ set_irq_its_device(desc, dev);
+ irqdesc_set_collection(desc, col->col_id);
+ spin_unlock(&desc->lock);
+ its_send_mapvi(dev, col, plpi, i);
+ }
+
+err:
+ spin_unlock(&rb_its_dev_lock);
+
+ return res;
+}
+
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
+{
+ struct its_device *pdev;
+ u32 plpi, i;
+
+ DPRINTK("ITS: Assign request for dev 0x%"PRIx32" to domain %"PRId32"\n",
+ vdevid, d->domain_id);
+
+ spin_lock(&rb_its_dev_lock);
+ pdev = its_find_device(pdevid);
+ if ( !pdev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -ENODEV;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ pdev->domain_id = d->domain_id;
+ pdev->virt_device_id = vdevid;
+
+ DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRId32" for dom %"PRId32"\n",
+ pdevid, pdev->nr_lpis, d->domain_id);
+
+ for ( i = 0; i < pdev->nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ /* TODO: Route lpi */
+ }
+
+ return 0;
+}
+
/*
* We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
* deal with (one configuration byte per interrupt). PENDBASE has to
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index ae301a4..f217c35 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -163,6 +163,14 @@ unsigned int irq_to_virq(struct irq_desc *desc)
}
#ifdef HAS_GICV3
+void set_lpi_event(struct irq_desc *desc, unsigned id)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(desc->msi_desc != NULL);
+
+ desc->msi_desc->eventID = id;
+}
+
unsigned int irq_to_vid(struct irq_desc *desc)
{
ASSERT(desc->msi_desc != NULL);
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index da689a4..77f694f 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -369,6 +369,8 @@ void its_set_lpi_properties(struct irq_desc *desc,
unsigned int priority);
int vits_domain_init(struct domain *d);
void vits_domain_free(struct domain *d);
+int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its);
+int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid);
int vits_get_vitt_entry(struct domain *d, uint32_t devid,
uint32_t event, struct vitt *entry);
int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 00d5345..b4ac8bb 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -61,6 +61,7 @@ void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
int irq_set_spi_type(unsigned int spi, unsigned int type);
+void set_lpi_event(struct irq_desc *desc, unsigned id);
unsigned int irq_to_virq(struct irq_desc *desc);
unsigned int irq_to_vid(struct irq_desc *desc);
struct its_device *get_irq_its_device(struct irq_desc *desc);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 19/22] xen/arm: ITS: Add APIs to add and assign device
2015-07-27 11:12 ` [PATCH v5 19/22] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-08-17 19:17 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-17 19:17 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> + for ( i = 0; i < dev->nr_lpis; i++ )
> + {
> + msi = xzalloc(struct msi_desc);
> + /* Reserve pLPI */
> + if ( its_alloc_device_irq(dev, &plpi) || !msi )
> + {
> + its_discard_lpis(dev, i);
> + its_lpi_free(dev);
> + its_send_mapd(dev, 0);
> + its_free_device(dev);
> + dprintk(XENLOG_G_ERR,"ITS: Cannot add device 0x%"PRIx32"\n", devid);
> + res = -ENOSPC;
> + goto err;
> + }
> +
> + /* For each pLPI send MAPVI command */
> + col = &dev->its->collections[(i % nr_cpu_ids)];
> + /* Store collection for this plpi in irq_desc */
> + desc = irq_to_desc(plpi);
> + spin_lock(&desc->lock);
> + desc->msi_desc = msi;
It's rather strange to directly use msi_desc field here when you
introduced helper for all the other fields.
As I said on a previous mail, I would prefer to see an helper to set the
msi_desc and avoid introducing ITS specific helpers for all the other
fields in irq.c.
> + set_lpi_event(desc, i);
> + set_irq_its_device(desc, dev);
> + irqdesc_set_collection(desc, col->col_id);
Please be consistent with the name. 3 helpers doing the same (setting
fields in MSI), 3 completely different naming...
> + spin_unlock(&desc->lock);
> + its_send_mapvi(dev, col, plpi, i);
> + }
> +
> +err:
> + spin_unlock(&rb_its_dev_lock);
> +
> + return res;
> +}
> +
> +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid)
> +{
> + struct its_device *pdev;
> + u32 plpi, i;
> +
> + DPRINTK("ITS: Assign request for dev 0x%"PRIx32" to domain %"PRId32"\n",
> + vdevid, d->domain_id);
> +
> + spin_lock(&rb_its_dev_lock);
> + pdev = its_find_device(pdevid);
> + if ( !pdev )
> + {
> + spin_unlock(&rb_its_dev_lock);
> + return -ENODEV;
> + }
> + spin_unlock(&rb_its_dev_lock);
> +
> + pdev->domain_id = d->domain_id;
> + pdev->virt_device_id = vdevid;
> +
> + DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRId32" for dom %"PRId32"\n",
> + pdevid, pdev->nr_lpis, d->domain_id);
> +
> + for ( i = 0; i < pdev->nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + /* TODO: Route lpi */
Seriously? Why did you drop the code since v4 here?
We are at v5, I'm expecting to see this series working if I'm applying
to my tree. Without the routing, I don't see how PCI can work with ITS
on DOM0... You didn't even mention it in the cover letter!
You should test your series without any additional patch on upstream Xen
before sending it.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (18 preceding siblings ...)
2015-07-27 11:12 ` [PATCH v5 19/22] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-07-27 11:12 ` vijay.kilari
2015-08-17 19:20 ` Julien Grall
2015-08-18 19:14 ` Julien Grall
2015-07-27 11:12 ` [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
2015-07-27 11:12 ` [PATCH v5 22/22] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
21 siblings, 2 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:12 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS translation space contains GITS_TRANSLATOR register
which is written by device to raise LPI. This space needs
to mapped to every domain address space for all physical
ITS available,so that device can access GITS_TRANSLATOR
register using SMMU.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
xen/arch/arm/vgic-v3-its.c | 38 +++++++++++++++++++++++++++++++++++++-
1 file changed, 37 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index e182cee..27523f4 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -1060,6 +1060,42 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+/*
+ * Map the 64K ITS translation space in guest.
+ * This is required purely for device smmu writes.
+*/
+
+static int vits_map_translation_space(struct domain *d)
+{
+ uint64_t addr, size;
+ int ret;
+
+ if ( !is_hardware_domain(d) )
+ return 0;
+
+ ASSERT(is_domain_direct_mapped(d));
+
+ addr = d->arch.vgic.vits->gits_base + SZ_64K;
+ size = SZ_64K;
+
+ /* Using 1:1 mapping to map translation space */
+ /* TODO: Handle DomU mapping */
+ ret = map_mmio_regions(d,
+ paddr_to_pfn(addr & PAGE_MASK),
+ DIV_ROUND_UP(size, PAGE_SIZE),
+ paddr_to_pfn(addr & PAGE_MASK));
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "vITS: Unable to map to dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+ }
+
+ return ret;
+}
+
int vits_domain_init(struct domain *d)
{
struct vgic_its *vits;
@@ -1120,7 +1156,7 @@ int vits_domain_init(struct domain *d)
register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
- return 0;
+ return vits_map_translation_space(d);
}
void vits_domain_free(struct domain *d)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-07-27 11:12 ` [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-08-17 19:20 ` Julien Grall
2015-08-18 19:14 ` Julien Grall
1 sibling, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-17 19:20 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
Hi Vijay,
On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS translation space contains GITS_TRANSLATOR register
> which is written by device to raise LPI. This space needs
> to mapped to every domain address space for all physical
> ITS available,so that device can access GITS_TRANSLATOR
> register using SMMU.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 38 +++++++++++++++++++++++++++++++++++++-
> 1 file changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index e182cee..27523f4 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -1060,6 +1060,42 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
> .write_handler = vgic_v3_gits_mmio_write,
> };
>
> +/*
> + * Map the 64K ITS translation space in guest.
> + * This is required purely for device smmu writes.
> +*/
> +
> +static int vits_map_translation_space(struct domain *d)
> +{
> + uint64_t addr, size;
> + int ret;
> +
> + if ( !is_hardware_domain(d) )
Well this code is surely wrong. If you happen to have a guest domain,
you won't map anything and say you've done it.
Given that vits_domain_init is only called for DOM0, you should drop
this check.
> + return 0;
> +
> + ASSERT(is_domain_direct_mapped(d));
> +
> + addr = d->arch.vgic.vits->gits_base + SZ_64K;
> + size = SZ_64K;
> +
> + /* Using 1:1 mapping to map translation space */
> + /* TODO: Handle DomU mapping */
> + ret = map_mmio_regions(d,
> + paddr_to_pfn(addr & PAGE_MASK),
> + DIV_ROUND_UP(size, PAGE_SIZE),
> + paddr_to_pfn(addr & PAGE_MASK));
> +
> + if ( ret )
> + {
> + dprintk(XENLOG_G_ERR, "vITS: Unable to map to dom%d access to"
> + " 0x%"PRIx64" - 0x%"PRIx64"\n",
"Unable to map 0x...-0x... to dom%d"
> + d->domain_id,
> + addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
> + }
> +
> + return ret;
> +}
> +
> int vits_domain_init(struct domain *d)
> {
> struct vgic_its *vits;
> @@ -1120,7 +1156,7 @@ int vits_domain_init(struct domain *d)
>
> register_mmio_handler(d, &vgic_gits_mmio_handler, vits->gits_base, SZ_64K);
>
> - return 0;
> + return vits_map_translation_space(d);
> }
>
> void vits_domain_free(struct domain *d)
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-07-27 11:12 ` [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space vijay.kilari
2015-08-17 19:20 ` Julien Grall
@ 2015-08-18 19:14 ` Julien Grall
2015-08-18 22:37 ` Marc Zyngier
1 sibling, 1 reply; 81+ messages in thread
From: Julien Grall @ 2015-08-18 19:14 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, Stefano Stabellini,
stefano.stabellini, tim, xen-devel
Cc: Michal Marek, Marc Zyngier, manish.jaggi, Vijaya Kumar K
Hi,
On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> ITS translation space contains GITS_TRANSLATOR register
s/GITS_TRANSLATOR/GITS_TRANSLATOR/
> which is written by device to raise LPI. This space needs
> to mapped to every domain address space for all physical
> ITS available,so that device can access GITS_TRANSLATOR
Ditto
> register using SMMU.
Marc pointed me today that if the processor is writing into
GITS_TRANSLATER it may be able to deadlock the system.
Reading more closely the spec (8.1.3 IHI0069A), there is undefined
behavior when writing to this register with wrong access size.
Currently the page table are shared between the processor and the SMMU,
so that means that a domain will be able to deadlock the processor and
therefore the whole platform.
So we should never expose GITS_TRANSLATER into the processor page table.
Which means unsharing some parts if not all of the page tables between
the processor and the SMMU.
While it's not required for this series, you don't support guest, this
would be mandatory to fix it before any usage of the vITS by a guest.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-08-18 19:14 ` Julien Grall
@ 2015-08-18 22:37 ` Marc Zyngier
2015-09-02 15:45 ` Ian Campbell
0 siblings, 1 reply; 81+ messages in thread
From: Marc Zyngier @ 2015-08-18 22:37 UTC (permalink / raw)
To: Julien Grall
Cc: Michal Marek, Ian.Campbell@citrix.com, vijay.kilari@gmail.com,
Stefano Stabellini, manish.jaggi@caviumnetworks.com, tim@xen.org,
xen-devel@lists.xen.org, stefano.stabellini@citrix.com,
Vijaya Kumar K
On Tue, 18 Aug 2015 20:14:43 +0100
Julien Grall <julien.grall@citrix.com> wrote:
> Hi,
>
> On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> > From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >
> > ITS translation space contains GITS_TRANSLATOR register
>
> s/GITS_TRANSLATOR/GITS_TRANSLATOR/
I assume you mean GITS_TRANSLATER? ;-)
>
> > which is written by device to raise LPI. This space needs
> > to mapped to every domain address space for all physical
> > ITS available,so that device can access GITS_TRANSLATOR
>
> Ditto
>
> > register using SMMU.
>
> Marc pointed me today that if the processor is writing into
> GITS_TRANSLATER it may be able to deadlock the system.
>
> Reading more closely the spec (8.1.3 IHI0069A), there is undefined
> behavior when writing to this register with wrong access size.
>
> Currently the page table are shared between the processor and the SMMU,
> so that means that a domain will be able to deadlock the processor and
> therefore the whole platform.
Indeed. A CPU should *never* be able to write to the GITS_TRANSLATER
register. What would be the meaning anyway? How would a DeviceID be
sampled? This is definitely UNPREDICTIBLE territory, and you want to
make sure a guest cannot directly write to the HW.
> So we should never expose GITS_TRANSLATER into the processor page table.
> Which means unsharing some parts if not all of the page tables between
> the processor and the SMMU.
Agreed. It looks to me like the CPU should only see the the virtual
ITS, and nothing else.
Thanks,
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-08-18 22:37 ` Marc Zyngier
@ 2015-09-02 15:45 ` Ian Campbell
2015-09-02 15:59 ` Marc Zyngier
0 siblings, 1 reply; 81+ messages in thread
From: Ian Campbell @ 2015-09-02 15:45 UTC (permalink / raw)
To: Marc Zyngier, Julien Grall
Cc: Michal Marek, vijay.kilari@gmail.com, Stefano Stabellini,
manish.jaggi@caviumnetworks.com, tim@xen.org,
xen-devel@lists.xen.org, stefano.stabellini@citrix.com,
Vijaya Kumar K
On Tue, 2015-08-18 at 23:37 +0100, Marc Zyngier wrote:
> On Tue, 18 Aug 2015 20:14:43 +0100 Julien Grall <julien.grall@citrix.com> wrote:
>
> > Marc pointed me today that if the processor is writing into
> > GITS_TRANSLATER it may be able to deadlock the system.
> >
> > Reading more closely the spec (8.1.3 IHI0069A), there is undefined
> > behavior when writing to this register with wrong access size.
> >
> > Currently the page table are shared between the processor and the SMMU,
> >
> > so that means that a domain will be able to deadlock the processor and
> > therefore the whole platform.
>
> Indeed. A CPU should *never* be able to write to the GITS_TRANSLATER
> register. What would be the meaning anyway? How would a DeviceID be
> sampled? This is definitely UNPREDICTIBLE territory, and you want to
> make sure a guest cannot directly write to the HW.
>
> > So we should never expose GITS_TRANSLATER into the processor page
> > table.
> > Which means unsharing some parts if not all of the page tables between
> > the processor and the SMMU.
>
> Agreed. It looks to me like the CPU should only see the the virtual
> ITS, and nothing else.
It's rather unfortunate that using an ITS therefore precludes sharing stage
-2 page tables between MMU and SMMU, which it seems otherwise the
architecture designers have tried hard to allow.
Do you know if this will be fixed in some future revision (although given
we now need to have the functionality anyway I'm not sure it help more than
saving a few pages of memory :-()
Ian.
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space
2015-09-02 15:45 ` Ian Campbell
@ 2015-09-02 15:59 ` Marc Zyngier
0 siblings, 0 replies; 81+ messages in thread
From: Marc Zyngier @ 2015-09-02 15:59 UTC (permalink / raw)
To: Ian Campbell, Julien Grall
Cc: Michal Marek, vijay.kilari@gmail.com, Stefano Stabellini,
manish.jaggi@caviumnetworks.com, tim@xen.org,
xen-devel@lists.xen.org, stefano.stabellini@citrix.com,
Vijaya Kumar K
On 02/09/15 16:45, Ian Campbell wrote:
> On Tue, 2015-08-18 at 23:37 +0100, Marc Zyngier wrote:
>> On Tue, 18 Aug 2015 20:14:43 +0100 Julien Grall <julien.grall@citrix.com> wrote:
>>
>>> Marc pointed me today that if the processor is writing into
>>> GITS_TRANSLATER it may be able to deadlock the system.
>>>
>>> Reading more closely the spec (8.1.3 IHI0069A), there is undefined
>>> behavior when writing to this register with wrong access size.
>>>
>>> Currently the page table are shared between the processor and the SMMU,
>>>
>>> so that means that a domain will be able to deadlock the processor and
>>> therefore the whole platform.
>>
>> Indeed. A CPU should *never* be able to write to the GITS_TRANSLATER
>> register. What would be the meaning anyway? How would a DeviceID be
>> sampled? This is definitely UNPREDICTIBLE territory, and you want to
>> make sure a guest cannot directly write to the HW.
>>
>>> So we should never expose GITS_TRANSLATER into the processor page
>>> table.
>>> Which means unsharing some parts if not all of the page tables between
>>> the processor and the SMMU.
>>
>> Agreed. It looks to me like the CPU should only see the the virtual
>> ITS, and nothing else.
>
> It's rather unfortunate that using an ITS therefore precludes sharing stage
> -2 page tables between MMU and SMMU, which it seems otherwise the
> architecture designers have tried hard to allow.
>
> Do you know if this will be fixed in some future revision (although given
> we now need to have the functionality anyway I'm not sure it help more than
> saving a few pages of memory :-()
I don't have any idea if something is being worked on to address this,
but I think you may be able to share at least the page tables describing
the memory, which should really be the bulk of the page tables.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (19 preceding siblings ...)
2015-07-27 11:12 ` [PATCH v5 20/22] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-07-27 11:12 ` vijay.kilari
2015-08-17 19:41 ` Julien Grall
2015-07-27 11:12 ` [PATCH v5 22/22] xen/arm: ITS: Add pci devices in ThunderX vijay.kilari
21 siblings, 1 reply; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:12 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Parse host dt and generate ITS node for Dom0.
ITS node resides inside GIC node so when GIC node
is encountered look for ITS node.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v5: - Moved ITS dt node generation to ITS driver
v4: - Generate only one ITS node for Dom0
- Replace msi-parent references to single its phandle
---
xen/arch/arm/domain_build.c | 17 ++++++++++
xen/arch/arm/gic-v3-its.c | 74 +++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 29 ++++++++++++++++
xen/arch/arm/gic.c | 18 ++++++++++
xen/include/asm-arm/gic-its.h | 3 ++
xen/include/asm-arm/gic.h | 7 ++++
6 files changed, 148 insertions(+)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 8556afd..6b6f013 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -469,6 +469,19 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
continue;
}
+ /*
+ * Replace all msi-parent phandle references to single ITS node
+ * generated for Dom0
+ */
+ if ( dt_property_name_is_equal(prop, "msi-parent") )
+ {
+ fdt32_t phandle = gic_get_msi_handle();
+ DPRINT(" Set msi-parent(ITS) phandle 0x%x\n",phandle);
+ fdt_property(kinfo->fdt, prop->name, (void *)&phandle,
+ sizeof(phandle));
+ continue;
+ }
+
res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
xfree(new_data);
@@ -875,6 +888,10 @@ static int make_gic_node(const struct domain *d, void *fdt,
return res;
res = fdt_end_node(fdt);
+ if ( res )
+ return res;
+
+ res = gic_its_hwdom_dt_node(d, node, fdt);
return res;
}
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 99f6edc..042c70d 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -27,6 +27,8 @@
#include <xen/sched.h>
#include <xen/errno.h>
#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/list.h>
#include <xen/sizes.h>
#include <xen/vmap.h>
@@ -96,6 +98,7 @@ static struct rdist_prop *gic_rdists;
static struct rb_root rb_its_dev;
static struct gic_its_info its_data;
static DEFINE_SPINLOCK(rb_its_dev_lock);
+static fdt32_t its_phandle;
#define gic_data_rdist() (this_cpu(rdist))
@@ -1198,6 +1201,77 @@ static void its_cpu_init_collection(void)
spin_unlock(&its_lock);
}
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt)
+{
+ struct its_node *its;
+ const struct dt_device_node *gic;
+ const void *compatible = NULL;
+ u32 len;
+ __be32 *new_cells, *tmp;
+ int res = 0;
+
+ /* Will pass only first ITS node info */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ if ( !its )
+ {
+ dprintk(XENLOG_ERR, "ITS node not found\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ gic = its->dt_node;
+
+ compatible = dt_get_property(gic, "compatible", &len);
+ if ( !compatible )
+ {
+ dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ res = fdt_begin_node(fdt, "gic-its");
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "compatible", compatible, len);
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "msi-controller", NULL, 0);
+ if ( res )
+ return res;
+
+ len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+
+ new_cells = xzalloc_bytes(len);
+ if ( new_cells == NULL )
+ return -FDT_ERR_XEN(ENOMEM);
+ tmp = new_cells;
+
+ dt_set_range(&tmp, node, its->phys_base, its->phys_size);
+
+ res = fdt_property(fdt, "reg", new_cells, len);
+ xfree(new_cells);
+
+ if ( node->phandle )
+ {
+ res = fdt_property_cell(fdt, "phandle", node->phandle);
+ if ( res )
+ return res;
+
+ its_phandle = cpu_to_fdt32(node->phandle);
+ }
+
+ res = fdt_end_node(fdt);
+
+ return res;
+}
+
+
+fdt32_t its_get_lpi_handle(void)
+{
+ return its_phandle;
+}
+
static int its_force_quiescent(void __iomem *base)
{
u32 count = 1000000; /* 1s */
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 23eb47c..828bf27 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1143,6 +1143,34 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
return res;
}
+static int gicv3_make_hwdom_its_dt_node(const struct domain *d,
+ const struct dt_device_node *node,
+ void *fdt)
+{
+ struct dt_device_node *gic_child;
+ int res = 0;
+
+ static const struct dt_device_match its_matches[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ dt_for_each_child_node(node, gic_child)
+ {
+ if ( gic_child != NULL )
+ {
+ if ( dt_match_node(its_matches, gic_child) )
+ {
+ res = its_make_dt_node(d, gic_child, fdt);
+ break;
+ }
+ }
+ }
+
+ return res;
+}
+
static const hw_irq_controller gicv3_host_irq_type = {
.typename = "gic-v3",
.startup = gicv3_irq_startup,
@@ -1372,6 +1400,7 @@ static const struct gic_hw_operations gicv3_ops = {
.read_apr = gicv3_read_apr,
.secondary_init = gicv3_secondary_cpu_init,
.make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
+ .make_hwdom_its_dt_node = gicv3_make_hwdom_its_dt_node,
.is_lpi = gicv3_is_lpi,
};
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index f6be0e9..88c1427 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -753,6 +753,15 @@ void __cpuinit init_maintenance_interrupt(void)
"irq-maintenance", NULL);
}
+fdt32_t gic_get_msi_handle(void)
+{
+#ifdef HAS_GICV3
+ if ( gic_lpi_supported() )
+ return its_get_lpi_handle();
+#endif
+ return 0;
+}
+
int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt)
@@ -760,6 +769,15 @@ int gic_make_hwdom_dt_node(const struct domain *d,
return gic_hw_ops->make_hwdom_dt_node(d, node, fdt);
}
+int gic_its_hwdom_dt_node(const struct domain *d,
+ const struct dt_device_node *node,
+ void *fdt)
+{
+ if ( gic_lpi_supported() )
+ return gic_hw_ops->make_hwdom_its_dt_node(d, node, fdt);
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 77f694f..f312a88 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -362,11 +362,14 @@ bool_t is_domain_lpi(struct domain *d, unsigned int lpi);
hw_irq_controller *its_get_host_lpi_type(void);
hw_irq_controller *its_get_guest_lpi_type(void);
u32 its_get_nr_event_ids(void);
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt);
int its_init(struct rdist_prop *rdists);
int its_cpu_init(void);
void its_set_lpi_properties(struct irq_desc *desc,
const cpumask_t *cpu_mask,
unsigned int priority);
+fdt32_t its_get_lpi_handle(void);
int vits_domain_init(struct domain *d);
void vits_domain_free(struct domain *d);
int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 15bd6eb..d7050a9 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -152,6 +152,7 @@
#ifndef __ASSEMBLY__
#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/irq.h>
#include <asm-arm/vgic.h>
@@ -372,6 +373,8 @@ struct gic_hw_operations {
int (*secondary_init)(void);
int (*make_hwdom_dt_node)(const struct domain *d,
const struct dt_device_node *node, void *fdt);
+ int (*make_hwdom_its_dt_node)(const struct domain *d,
+ const struct dt_device_node *node, void *fdt);
bool_t (*is_lpi)(unsigned int irq);
};
@@ -387,6 +390,10 @@ int gic_make_hwdom_dt_node(const struct domain *d,
const struct dt_device_node *node,
void *fdt);
bool_t gic_is_lpi(unsigned int irq);
+int gic_its_hwdom_dt_node(const struct domain *d,
+ const struct dt_device_node *node,
+ void *fdt);
+fdt32_t gic_get_msi_handle(void);
#endif /* __ASSEMBLY__ */
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-07-27 11:12 ` [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-08-17 19:41 ` Julien Grall
2015-08-21 23:02 ` Vijay Kilari
2015-08-26 12:40 ` Vijay Kilari
0 siblings, 2 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-17 19:41 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K
On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Parse host dt and generate ITS node for Dom0.
> ITS node resides inside GIC node so when GIC node
> is encountered look for ITS node.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v5: - Moved ITS dt node generation to ITS driver
> v4: - Generate only one ITS node for Dom0
> - Replace msi-parent references to single its phandle
> ---
> xen/arch/arm/domain_build.c | 17 ++++++++++
> xen/arch/arm/gic-v3-its.c | 74 +++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3.c | 29 ++++++++++++++++
> xen/arch/arm/gic.c | 18 ++++++++++
> xen/include/asm-arm/gic-its.h | 3 ++
> xen/include/asm-arm/gic.h | 7 ++++
> 6 files changed, 148 insertions(+)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 8556afd..6b6f013 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -469,6 +469,19 @@ static int write_properties(struct domain *d, struct kernel_info *kinfo,
> continue;
> }
>
> + /*
> + * Replace all msi-parent phandle references to single ITS node
> + * generated for Dom0
> + */
> + if ( dt_property_name_is_equal(prop, "msi-parent") )
I think this need more care than replacing every msi-parent without any
checking.
You need to make sure that the msi-parent points to an ITS MSI just in
case there is other possibility of MSI.
Furthermore, I would do this a ITS specific callback (gic_rewrite_node
or smth similar) to avoid replacing msi-parent when it's not necessary.
I have in mind GICv2M.
> + {
> + fdt32_t phandle = gic_get_msi_handle();
> + DPRINT(" Set msi-parent(ITS) phandle 0x%x\n",phandle);
> + fdt_property(kinfo->fdt, prop->name, (void *)&phandle,
> + sizeof(phandle));
> + continue;
> + }
> +
> res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>
> xfree(new_data);
> @@ -875,6 +888,10 @@ static int make_gic_node(const struct domain *d, void *fdt,
> return res;
>
> res = fdt_end_node(fdt);
> + if ( res )
> + return res;
> +
> + res = gic_its_hwdom_dt_node(d, node, fdt);
Can you explain why you didn't follow my suggestion to plumb the ITS
node creation in gic_hwdow_dt_node? IHMO there is no need of new callback.
Furthermore the call is misplaced. You will end up to have a DT looking like
gic {
}
gic-its {
}
rather than
gic {
gic-its {
}
}
> return res;
> }
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 99f6edc..042c70d 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> +int its_make_dt_node(const struct domain *d,
> + const struct dt_device_node *node, void *fdt)
> +{
> + struct its_node *its;
> + const struct dt_device_node *gic;
> + const void *compatible = NULL;
> + u32 len;
> + __be32 *new_cells, *tmp;
> + int res = 0;
> +
> + /* Will pass only first ITS node info */
> + its = list_first_entry(&its_nodes, struct its_node, entry);
> + if ( !its )
> + {
> + dprintk(XENLOG_ERR, "ITS node not found\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + gic = its->dt_node;
> +
> + compatible = dt_get_property(gic, "compatible", &len);
> + if ( !compatible )
> + {
> + dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + res = fdt_begin_node(fdt, "gic-its");
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "compatible", compatible, len);
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "msi-controller", NULL, 0);
> + if ( res )
> + return res;
> +
> + len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
> +
> + new_cells = xzalloc_bytes(len);
> + if ( new_cells == NULL )
> + return -FDT_ERR_XEN(ENOMEM);
> + tmp = new_cells;
> +
> + dt_set_range(&tmp, node, its->phys_base, its->phys_size);
> +
> + res = fdt_property(fdt, "reg", new_cells, len);
> + xfree(new_cells);
> +
> + if ( node->phandle )
> + {
> + res = fdt_property_cell(fdt, "phandle", node->phandle);
> + if ( res )
> + return res;
> +
> + its_phandle = cpu_to_fdt32(node->phandle);
Why this is set in make_hwdom_dt_node? The ITS phandle may be used
before we effectively parse the GIC node.
> + }
> +
> + res = fdt_end_node(fdt);
> +
> + return res;
> +}
> +
> +
> +fdt32_t its_get_lpi_handle(void)
> +{
> + return its_phandle;
> +}
> +
> static int its_force_quiescent(void __iomem *base)
> {
> u32 count = 1000000; /* 1s */
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 23eb47c..828bf27 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -1143,6 +1143,34 @@ static int gicv3_make_hwdom_dt_node(const struct domain *d,
> return res;
> }
>
> +static int gicv3_make_hwdom_its_dt_node(const struct domain *d,
> + const struct dt_device_node *node,
> + void *fdt)
> +{
> + struct dt_device_node *gic_child;
> + int res = 0;
> +
> + static const struct dt_device_match its_matches[] __initconst =
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> +
> + dt_for_each_child_node(node, gic_child)
> + {
> + if ( gic_child != NULL )
> + {
> + if ( dt_match_node(its_matches, gic_child) )
> + {
> + res = its_make_dt_node(d, gic_child, fdt);
> + break;
> + }
> + }
> + }
I already asked on v4, why do you need this loop? The GIC node for the
DOM0 is recreating from scratch, there is no need of looping on the
current GIC node...
> +
> + return res;
> +}
> +
> static const hw_irq_controller gicv3_host_irq_type = {
> .typename = "gic-v3",
> .startup = gicv3_irq_startup,
> @@ -1372,6 +1400,7 @@ static const struct gic_hw_operations gicv3_ops = {
> .read_apr = gicv3_read_apr,
> .secondary_init = gicv3_secondary_cpu_init,
> .make_hwdom_dt_node = gicv3_make_hwdom_dt_node,
> + .make_hwdom_its_dt_node = gicv3_make_hwdom_its_dt_node,
> .is_lpi = gicv3_is_lpi,
> };
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index f6be0e9..88c1427 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -753,6 +753,15 @@ void __cpuinit init_maintenance_interrupt(void)
> "irq-maintenance", NULL);
> }
>
> +fdt32_t gic_get_msi_handle(void)
> +{
> +#ifdef HAS_GICV3
> + if ( gic_lpi_supported() )
> + return its_get_lpi_handle();
> +#endif
If you need this introduce a new callback but don't add #ifdef in the
common code unless you have a strong argument.
> + return 0;
> +}
> +
[..]
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-08-17 19:41 ` Julien Grall
@ 2015-08-21 23:02 ` Vijay Kilari
2015-08-21 23:48 ` Julien Grall
2015-08-26 12:40 ` Vijay Kilari
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-21 23:02 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Mon, Aug 17, 2015 at 12:41 PM, Julien Grall <julien.grall@citrix.com> wrote:
>
>
> On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
>>
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Parse host dt and generate ITS node for Dom0.
>> ITS node resides inside GIC node so when GIC node
>> is encountered look for ITS node.
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v5: - Moved ITS dt node generation to ITS driver
>> v4: - Generate only one ITS node for Dom0
>> - Replace msi-parent references to single its phandle
>> ---
>> xen/arch/arm/domain_build.c | 17 ++++++++++
>> xen/arch/arm/gic-v3-its.c | 74
>> +++++++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/gic-v3.c | 29 ++++++++++++++++
>> xen/arch/arm/gic.c | 18 ++++++++++
>> xen/include/asm-arm/gic-its.h | 3 ++
>> xen/include/asm-arm/gic.h | 7 ++++
>> 6 files changed, 148 insertions(+)
>>
[...]
>> +static int gicv3_make_hwdom_its_dt_node(const struct domain *d,
>> + const struct dt_device_node
>> *node,
>> + void *fdt)
>> +{
>> + struct dt_device_node *gic_child;
>> + int res = 0;
>> +
>> + static const struct dt_device_match its_matches[] __initconst =
>> + {
>> + DT_MATCH_GIC_ITS,
>> + { /* sentinel */ },
>> + };
>> +
>> + dt_for_each_child_node(node, gic_child)
>> + {
>> + if ( gic_child != NULL )
>> + {
>> + if ( dt_match_node(its_matches, gic_child) )
>> + {
>> + res = its_make_dt_node(d, gic_child, fdt);
>> + break;
>> + }
>> + }
>> + }
>
>
> I already asked on v4, why do you need this loop? The GIC node for the DOM0
> is recreating from scratch, there is no need of looping on the current GIC
> node...
It ensures that ITS node for Dom0 is generated only if dt_host has it.
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-08-21 23:02 ` Vijay Kilari
@ 2015-08-21 23:48 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-21 23:48 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On 21/08/2015 16:02, Vijay Kilari wrote:
> On Mon, Aug 17, 2015 at 12:41 PM, Julien Grall <julien.grall@citrix.com> wrote:
>>
>>
>> On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
>>>
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Parse host dt and generate ITS node for Dom0.
>>> ITS node resides inside GIC node so when GIC node
>>> is encountered look for ITS node.
>>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>> ---
>>> v5: - Moved ITS dt node generation to ITS driver
>>> v4: - Generate only one ITS node for Dom0
>>> - Replace msi-parent references to single its phandle
>>> ---
>>> xen/arch/arm/domain_build.c | 17 ++++++++++
>>> xen/arch/arm/gic-v3-its.c | 74
>>> +++++++++++++++++++++++++++++++++++++++++
>>> xen/arch/arm/gic-v3.c | 29 ++++++++++++++++
>>> xen/arch/arm/gic.c | 18 ++++++++++
>>> xen/include/asm-arm/gic-its.h | 3 ++
>>> xen/include/asm-arm/gic.h | 7 ++++
>>> 6 files changed, 148 insertions(+)
>>>
> [...]
>>> +static int gicv3_make_hwdom_its_dt_node(const struct domain *d,
>>> + const struct dt_device_node
>>> *node,
>>> + void *fdt)
>>> +{
>>> + struct dt_device_node *gic_child;
>>> + int res = 0;
>>> +
>>> + static const struct dt_device_match its_matches[] __initconst =
>>> + {
>>> + DT_MATCH_GIC_ITS,
>>> + { /* sentinel */ },
>>> + };
>>> +
>>> + dt_for_each_child_node(node, gic_child)
>>> + {
>>> + if ( gic_child != NULL )
>>> + {
>>> + if ( dt_match_node(its_matches, gic_child) )
>>> + {
>>> + res = its_make_dt_node(d, gic_child, fdt);
>>> + break;
>>> + }
>>> + }
>>> + }
>>
>>
>> I already asked on v4, why do you need this loop? The GIC node for the DOM0
>> is recreating from scratch, there is no need of looping on the current GIC
>> node...
>
> It ensures that ITS node for Dom0 is generated only if dt_host has it.
That's wrong. The ITS node maybe be present in the DT but the ITS driver
never initialized. Furthermore, we may decide at some point to not
exposed the ITS to DOM0.
You should introduced a variable to know whether the ITS is used for
DOM0 or not and not relying on the hw DT. This is pointless and make the
code harder to read.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-08-17 19:41 ` Julien Grall
2015-08-21 23:02 ` Vijay Kilari
@ 2015-08-26 12:40 ` Vijay Kilari
2015-08-27 0:02 ` Julien Grall
1 sibling, 1 reply; 81+ messages in thread
From: Vijay Kilari @ 2015-08-26 12:40 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
On Tue, Aug 18, 2015 at 1:11 AM, Julien Grall <julien.grall@citrix.com> wrote:
>
>
> On 27/07/2015 04:12, vijay.kilari@gmail.com wrote:
>>
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Parse host dt and generate ITS node for Dom0.
>> ITS node resides inside GIC node so when GIC node
>> is encountered look for ITS node.
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> v5: - Moved ITS dt node generation to ITS driver
>> v4: - Generate only one ITS node for Dom0
>> - Replace msi-parent references to single its phandle
>> ---
>> xen/arch/arm/domain_build.c | 17 ++++++++++
>> xen/arch/arm/gic-v3-its.c | 74
>> +++++++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/gic-v3.c | 29 ++++++++++++++++
>> xen/arch/arm/gic.c | 18 ++++++++++
>> xen/include/asm-arm/gic-its.h | 3 ++
>> xen/include/asm-arm/gic.h | 7 ++++
>> 6 files changed, 148 insertions(+)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 8556afd..6b6f013 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -469,6 +469,19 @@ static int write_properties(struct domain *d, struct
>> kernel_info *kinfo,
>> continue;
>> }
>>
>> + /*
>> + * Replace all msi-parent phandle references to single ITS node
>> + * generated for Dom0
>> + */
>> + if ( dt_property_name_is_equal(prop, "msi-parent") )
>
>
> I think this need more care than replacing every msi-parent without any
> checking.
>
> You need to make sure that the msi-parent points to an ITS MSI just in case
> there is other possibility of MSI.
>
> Furthermore, I would do this a ITS specific callback (gic_rewrite_node or
> smth similar) to avoid replacing msi-parent when it's not necessary. I have
> in mind GICv2M.
>
>> + {
>> + fdt32_t phandle = gic_get_msi_handle();
>> + DPRINT(" Set msi-parent(ITS) phandle 0x%x\n",phandle);
>> + fdt_property(kinfo->fdt, prop->name, (void *)&phandle,
>> + sizeof(phandle));
>> + continue;
>> + }
>> +
>> res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>>
>> xfree(new_data);
>> @@ -875,6 +888,10 @@ static int make_gic_node(const struct domain *d, void
>> *fdt,
>> return res;
>>
>> res = fdt_end_node(fdt);
>> + if ( res )
>> + return res;
>> +
>> + res = gic_its_hwdom_dt_node(d, node, fdt);
>
>
> Can you explain why you didn't follow my suggestion to plumb the ITS node
> creation in gic_hwdow_dt_node? IHMO there is no need of new callback.
>
> Furthermore the call is misplaced. You will end up to have a DT looking like
>
> gic {
> }
>
> gic-its {
> }
>
> rather than
>
> gic {
> gic-its {
> }
> }
As discussed, I have tried to generate ITS node inside gic node and call
fdt_end_node() for gic after generating ITS node. But dom0 fails to
find ITS node in the generated device tree. Any clue?
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0
2015-08-26 12:40 ` Vijay Kilari
@ 2015-08-27 0:02 ` Julien Grall
0 siblings, 0 replies; 81+ messages in thread
From: Julien Grall @ 2015-08-27 0:02 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel@lists.xen.org, Stefano Stabellini,
Vijaya Kumar K
Hi Vijay,
On 26/08/2015 05:40, Vijay Kilari wrote:
>>> + {
>>> + fdt32_t phandle = gic_get_msi_handle();
>>> + DPRINT(" Set msi-parent(ITS) phandle 0x%x\n",phandle);
>>> + fdt_property(kinfo->fdt, prop->name, (void *)&phandle,
>>> + sizeof(phandle));
>>> + continue;
>>> + }
>>> +
>>> res = fdt_property(kinfo->fdt, prop->name, prop_data, prop_len);
>>>
>>> xfree(new_data);
>>> @@ -875,6 +888,10 @@ static int make_gic_node(const struct domain *d, void
>>> *fdt,
>>> return res;
>>>
>>> res = fdt_end_node(fdt);
>>> + if ( res )
>>> + return res;
>>> +
>>> + res = gic_its_hwdom_dt_node(d, node, fdt);
>>
>>
>> Can you explain why you didn't follow my suggestion to plumb the ITS node
>> creation in gic_hwdow_dt_node? IHMO there is no need of new callback.
>>
>> Furthermore the call is misplaced. You will end up to have a DT looking like
>>
>> gic {
>> }
>>
>> gic-its {
>> }
>>
>> rather than
>>
>> gic {
>> gic-its {
>> }
>> }
>
> As discussed, I have tried to generate ITS node inside gic node and call
> fdt_end_node() for gic after generating ITS node. But dom0 fails to
> find ITS node in the generated device tree. Any clue?
I can't give any clue without any log from Linux or code you've written...
I still think that your previous code is wrong (see
Documentation/devicetree/bindings/arm/gic-v3.txt in Linux tree), because
fdt_end_node of the parent should be called after all the children has
been created i.e
fdt_begin_node(fdt, "gic");
/* properties of the gic node */
fdt_begin_node(fdt, "ITS");
/* properties of the ITS node */
fdt_end_node(fdt); // End of ITS
fdt_end_node(fdt); // End of GIC
Note that *all* the properties of the GIC node should have been written
before creating the children nodes. Otherwise the device tree is
considered as invalid.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v5 22/22] xen/arm: ITS: Add pci devices in ThunderX
2015-07-27 11:11 [PATCH v5 00/22] Add ITS support vijay.kilari
` (20 preceding siblings ...)
2015-07-27 11:12 ` [PATCH v5 21/22] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-07-27 11:12 ` vijay.kilari
21 siblings, 0 replies; 81+ messages in thread
From: vijay.kilari @ 2015-07-27 11:12 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, manish.jaggi, Vijaya Kumar K, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS initialization required for all PCI devices in
ThunderX platform are done by calling from specific
mapping function.
This patch can be reverted once XEN PCI passthrough
framework for arm64 is in available.
For now all the PCI devices are assigned to Dom0
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
xen/arch/arm/platforms/Makefile | 1 +
xen/arch/arm/platforms/thunderx.c | 151 +++++++++++++++++++++++++++++++++++++
2 files changed, 152 insertions(+)
diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
index e173fec..d9f98f9 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_ARM_32) += sunxi.o
obj-$(CONFIG_ARM_32) += rcar2.o
obj-$(CONFIG_ARM_64) += seattle.o
obj-$(CONFIG_ARM_64) += xgene-storm.o
+obj-$(CONFIG_ARM_64) += thunderx.o
diff --git a/xen/arch/arm/platforms/thunderx.c b/xen/arch/arm/platforms/thunderx.c
new file mode 100644
index 0000000..7c335ba
--- /dev/null
+++ b/xen/arch/arm/platforms/thunderx.c
@@ -0,0 +1,151 @@
+/*
+ * xen/arch/arm/platforms/thunderx.c
+ *
+ * Cavium Thunder specific settings
+ *
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (c) 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/config.h>
+#include <asm/platform.h>
+#include <asm/gic-its.h>
+
+struct pci_dev_list
+{
+ uint32_t seg;
+ uint32_t bus;
+ uint32_t dev;
+ uint32_t func;
+};
+
+#define NUM_DEVIDS 54
+
+static struct pci_dev_list bdf[NUM_DEVIDS] =
+{
+ {0, 0, 2, 0}, /* 1 */
+ {0, 0, 6, 0},
+ {0, 0, 7, 0},
+ {0, 0, 10, 0},
+ {0, 0, 11, 0},
+ {0, 1, 0, 0},
+ {0, 1, 0, 1},
+ {0, 1, 0, 5},
+ {0, 1, 1, 4},
+ {0, 1, 9, 0}, /* 10 */
+ {0, 1, 9, 1},
+ {0, 1, 9, 2},
+ {0, 1, 9, 3},
+ {0, 1, 9, 4},
+ {0, 1, 9, 5},
+ {0, 1, 10, 0},
+ {0, 1, 10, 1},
+ {0, 1, 10, 2},
+ {0, 1, 10, 3},
+ {0, 1, 14, 0}, /* 20 */
+ {0, 1, 14, 2},
+ {0, 1, 14, 4},
+ {0, 1, 16, 0},
+ {0, 1, 16, 1},
+ {0, 2, 0, 0},
+ {0, 3, 0, 0},
+ {0, 4, 0, 0},
+ {1, 0, 8, 0},
+ {1, 0, 9, 0},
+ {1, 0, 10, 0}, /* 30 */
+ {1, 0, 11, 0},
+ {2, 0, 1, 0},
+ {2, 0, 3, 0},
+ {2, 1, 0, 0},
+ {2, 1, 0, 1},
+ {2, 1, 0, 2},
+ {2, 1, 0, 3},
+ {2, 1, 0, 4},
+ {2, 1, 0, 5},
+ {2, 1, 0, 6}, /* 40 */
+ {2, 1, 0, 7},
+ {2, 1, 1, 0},
+ {2, 1, 1, 1},
+ {2, 1, 1, 2},
+ {2, 1, 1, 3},
+ {2, 1, 1, 4},
+ {2, 1, 1, 5},
+ {2, 1, 1, 6},
+ {2, 1, 1, 7},
+ {2, 1, 2, 0}, /* 50 */
+ {2, 1, 2, 1},
+ {2, 1, 2, 2},
+ {2, 1, 1, 7},
+ {3, 0, 1, 0}, /* 54 */
+};
+
+#define BDF_TO_DEVID(seg, bus, dev, func) (seg << 16 | bus << 8 | dev << 3| func)
+
+/* TODO: add and assign devices using PCI framework */
+static int thunderx_specific_mapping(struct domain *d)
+{
+ struct dt_device_node *dt_its;
+ uint32_t devid, i;
+ int res;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for (dt_its = dt_find_matching_node(NULL, its_device_ids); dt_its;
+ dt_its = dt_find_matching_node(dt_its, its_device_ids))
+ {
+ break;
+ }
+
+ if ( dt_its == NULL )
+ {
+ dprintk(XENLOG_ERR, "ThunderX: ITS node not found to add device\n");
+ return 0;
+ }
+
+ for ( i = 0; i < NUM_DEVIDS; i++ )
+ {
+ devid = BDF_TO_DEVID(bdf[i].seg, bdf[i].bus,bdf[i].dev, bdf[i].func);
+ res = its_add_device(devid, 32, dt_its);
+ if ( res )
+ return res;
+ res = its_assign_device(d, devid, devid);
+ if ( res )
+ return res;
+ }
+
+ return 0;
+}
+
+static const char * const thunderx_dt_compat[] __initconst =
+{
+ "cavium,thunder-88xx",
+ NULL
+};
+
+PLATFORM_START(thunderx, "THUNDERX")
+ .compatible = thunderx_dt_compat,
+ .specific_mapping = thunderx_specific_mapping,
+PLATFORM_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 81+ messages in thread