Re: Xen/arm: Virtual ITS command queue handling

[Julien Grall <julien.grall@citrix.com>]

On 05/05/15 17:09, Vijay Kilari wrote:
> On Tue, May 5, 2015 at 7:39 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 05/05/15 13:14, Vijay Kilari wrote:
>>> Proposal 2:
>>> ----------------
>>> Here when guest writes command to vITS queue and updates CWRITER registers,
>>> it is trapped in XEN and below steps are followed to process ITS command
>>>
>>> - Dom0 creates a ITS completion device with device id (00:00.1) and reserves
>>>   n number (256 or so) irqs (LPIs) for this device.
>>> - One irq/LPI (called completion_irq) of this completion device is
>>> allocated per domain
>>> - With this irq/LPI descriptor we can identify the domain/vITS.
>>> - Info of all the ongoing ITS requests(put in pITS Queue) of this domain is
>>>   stored in ITS command status array (called its_requests). This is
>>> managed per vITS.
>>>
>>> 1) Trap of CWRITER write by guest
>>> 2) Take vITS lock
>>> 3) Read all the commands written by guest, translate it
>>>     - If one of the guest command is INT command
>>
>> Why do you need a specific handling for the guest INT command?
> 
>   If guest driver is using interrupt mechanism instead of polling
> then INT command is passed by guest. To make sure that CREADER is updated
> before INT command raises interrupt to guest, Xen has to insert completion
> interrupt and update CREADER

Hmmm I see what you mean now. Although, if I understand correctly, Xen
would receive two interrupts: one for the completion, and the other for
the guest.

It would be better if we avoid the first by re-using the INT command
from the guest. If it's not to difficult of course.

>>>        a) Append INT command with completion_irq and write this batch as
>>>           seperate request and goto (3) to process next commands
>>>     - If more than 'n' commands are sent by guest, start a timer to process
>>>       remaining commands
>>
>> Hmmm... How are you sure the time for the timer would be enough?
>>
>    Not thought of how much time. May be the number of pending
>    commands in physical queue might give some hueristic on timer value.

I'm wondering if a tasklet would be better here.

>>> 4) Append INT command with completion_irq of current domain
>>> 5) Release vITS lock
>>> 6) Take physical ITS (pITS) lock
>>> 7) Write translated cmds to physical ITS
>>> 8) Add entry in its_requests
>>
>> You don't explain what is its_requests.
>>
>>> 9) Release pITS lock
>>> 10) return from trap
>>>
>>> One receiving completion interrupt:
>>>
>>> 1) Take the first pending request from its_requests.
>>
>> I'm assuming that you have some kind of array/list to store the pending
>> request? I think this would be more difficult to manage than only
>> supporting one batch per domain at any time.
> 
>   Yes, If only one batch per domain is processed at a time,
> then the array could store only one entry. I will tune it when I implement

You won't need an array in this case...

>>> 2) Update vITS CREADER of the guest indicating completion of command to guest
>>>
>>> Cons:
>>>    - Has overhead of processing completion interrupt.
>>>    - Need to reserve a fake device to generate completion interrupt and
>>>      reserve one LPI per-domain
>>>
>>> Pros:
>>>    - VCPU does not poll in Xen for completion of commands.
>>>    - Handles guest flooding command queue with commands. But needs timer
>>>
>>> Handling Command queue state:
>>>  - Physical Queue cannot be full as it 64KB there by it can accomodate
>>> 1K ITS commands.
>>
>> I don't understand this sentence. Why do you think the physical queue
>> cannot be full?
> 
>   I mean that it is unlikely that physical ITS command Q would be full
> because of 64KB size. If at all if it full then below action is taken

Oh ok. I though you were saying it's not possible :).

> 
>>
>>>    In case it is full, VCPU has to poll with timeout till physical
>>> Queue is empty before it post
>>>    next command
>>>  - If vITS Queue condition should be managed by guest ITS driver.
>>
>> Same here.
> 
>     vITS Queue is under guest control. If Xen is processing commands slowly
> and if guest sees its queue is full then guest driver will handle it.

This paragraph is easier to understand thanks.

Regards,

-- 
Julien Grall