From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: Re: Xen/arm: Virtual ITS command queue handling Date: Tue, 5 May 2015 15:09:37 +0100 Message-ID: <5548CF21.2090108@citrix.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Vijay Kilari , Stefano Stabellini , Ian Campbell , Stefano Stabellini , Julien Grall Cc: Prasun Kapoor , manish.jaggi@caviumnetworks.com, "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 05/05/15 13:14, Vijay Kilari wrote: > Hi, > Hi Vijay, > As discussed, here is the design doc/txt. I will comment on the proposal 2 as it seems to be the preferred one assuming you are able to find why it's slow. > Proposal 2: > ---------------- > Here when guest writes command to vITS queue and updates CWRITER registers, > it is trapped in XEN and below steps are followed to process ITS command > > - Dom0 creates a ITS completion device with device id (00:00.1) and reserves > n number (256 or so) irqs (LPIs) for this device. > - One irq/LPI (called completion_irq) of this completion device is > allocated per domain > - With this irq/LPI descriptor we can identify the domain/vITS. > - Info of all the ongoing ITS requests(put in pITS Queue) of this domain is > stored in ITS command status array (called its_requests). This is > managed per vITS. > > 1) Trap of CWRITER write by guest > 2) Take vITS lock > 3) Read all the commands written by guest, translate it > - If one of the guest command is INT command Why do you need a specific handling for the guest INT command? > a) Append INT command with completion_irq and write this batch as > seperate request and goto (3) to process next commands > - If more than 'n' commands are sent by guest, start a timer to process > remaining commands Hmmm... How are you sure the time for the timer would be enough? > 4) Append INT command with completion_irq of current domain > 5) Release vITS lock > 6) Take physical ITS (pITS) lock > 7) Write translated cmds to physical ITS > 8) Add entry in its_requests You don't explain what is its_requests. > 9) Release pITS lock > 10) return from trap > > One receiving completion interrupt: > > 1) Take the first pending request from its_requests. I'm assuming that you have some kind of array/list to store the pending request? I think this would be more difficult to manage than only supporting one batch per domain at any time. > 2) Update vITS CREADER of the guest indicating completion of command to guest > > Cons: > - Has overhead of processing completion interrupt. > - Need to reserve a fake device to generate completion interrupt and > reserve one LPI per-domain > > Pros: > - VCPU does not poll in Xen for completion of commands. > - Handles guest flooding command queue with commands. But needs timer > > Handling Command queue state: > - Physical Queue cannot be full as it 64KB there by it can accomodate > 1K ITS commands. I don't understand this sentence. Why do you think the physical queue cannot be full? > In case it is full, VCPU has to poll with timeout till physical > Queue is empty before it post > next command > - If vITS Queue condition should be managed by guest ITS driver. Same here. > Behaviour of Polling and completion interrupt based guest driver: > - If completion interrupt (INT) is used by guest driver, then insert > Xen completion > INT command so that CREADER is updated before guest's INT command is injected > - If polling mode is used, trap on CREADER checks for completion of command > Regards, -- Julien Grall