Re: Xen on ARM vITS Handling Draft B (Was Re: Xen/arm: Virtual ITS command queue handling)

[Ian Campbell <ian.campbell@citrix.com>]

On Sun, 2015-05-24 at 11:35 +0100, Julien Grall wrote:
[...]

> I wrote my though for the validation bits (see below) and add some 
> definitions useful for people which don't have the spec.

Thanks for this.

> 
> Emulation of ITS commands
> =========================
> 
> # Introduction
> 
> This document is based on the section 5.13 of GICv3 specification
> (PRD03-GENC-010745 24.0). The goal is to provide insight of the cost
> to emulate ITS commands in Xen.
> 
> The ITS provides 12 commands in order to manage interrupt collection,
> device and interrupts.
> 
> # Definitions

I tried to integrate your text here into the introduction section of the
VITs doc.

[...]
> # Validation of the parameters

And this bit I used as the basis for a new "ITS Command Translation"
chapter.

I have some questions which I have inserted into the next draft of the
doc with XXX markers. I intend to post a new draft very soon, rather
than wait for any discussion, so you might prefer to wait and answer
them in that thread.

> Each command contains parameters that needs to be validated before any 
> usage in Xen or passing to the hardware.
> 
> This section will describe the validation of the main parameters.
> 
> ## Device ID
> 
> This parameter is used in commands which manage the device and the 
> interrupts associated to this device. Checking if a device is present 
> and retrieving the data structure must be fast.
> 
> The device identifiers may not be assigned contiguously and the maximum 
> number is very high (2^32).

That's true for the host, but I think the lookup here needs to be based
on the virtual device id, not necessarily the physical one, so we have
the opportunity to arrange things for our convenience.

In particular we could arrange for device ids to be contiguous (or in a
small number of ranges) and we know that N is going to be much lower
that 2^32 in practice.

So I think we could almost get away with either a simple array or a much
simpler M-level look up (for small M, say 2).

Or is there some constraint which means we cannot virtualise the device
id?

>  The possible efficient data structure would be:
>      1) List: The lookup/deletion is in O(n) and the insertion will 
> depend if the device should be sort following their identifier. The 
> memory overhead is 18 bytes per element.
>      2) Red-black tree: All the operations are O(log(n)). The memory 
> overhead is 24 bytes per element.
> 
> The solution 2) seems the more suitable for having fast deviceID 
> validation even though the memory overhead is a bit higher compare to 
> the list.

Vijay's text discussed Event ID too. I've added some words about that,
they may be rubbish, please check the next draft.

> ## Collection
> 
> This parameter is used in commands which manage collections and 
> interrupt in order to move them for one CPU to another. The ITS is only 
> mandatory to implement N + 1 collections where N is the number of 
> processor on the platform. Furthermore, the identifier are always 
> contiguous.
> 
> If we decide to implement the strict minimum (i.e N + 1), an array is
> enough and will allow operations in O(1).

May not even need that since [0..NR_CPUS+1] would allow us to go
straight to either vcpu->collection_id or domain->collection_id (the
latter being the +1).

> 
> ## Target Address
> 
>        This parameter is used in commands to manage collection.

It's also, I think, the output of the ITS Translation table?

>                              It's a unique
> identifier per processor. The format is different following the value
> of the bit GITS_TYPER.PTA (see definition). The value of the field 
> pre-defined by the ITS and the software has to handle the 2 cases.

IOW the bit is r/o and fixed by the ITS implementor?

> The solution with GITS_TYPER.PTA set to one will require some computation
> in order to find the VCPU associated with the redistributor address. It 
> will be similar to get_vcpu_from_rdist in the vGICv3 emulation 
> (xen/arch/arm/vgic-v3.c).
> 
> On another hand, setting GITS_TYPER.PTA to zero will give us control to
> decide the linear process number  which could simply be the vcpu_id (always
> linear).

Does this get more complicated with large numbers of vcpus on gic v3
(i.e. once AFFR>0 gets involved)?

>