Re: [PATCH v3 0/2] Inter-VM shared memory PCI device

[Avi Kivity <avi@redhat.com>]

On 03/25/2010 08:17 PM, Cam Macdonell wrote:
>
>>> I had a hunch it was probably considered.  That explains why irqfd
>>> doesn't have a datamatch field.  I guess supporting multiple MSI
>>> vectors with one doorbell per guest isn't possible if one 1 bit of
>>> information can be communicated.
>>>
>>>        
>> Actually you can have one doorbell supporting multiple vectors and guests,
>> simply divide the data value into two bit fields, one for the vector and one
>> for the guest.  A single write gets both values into the host, which can
>> then use datamatch to trigger the correct eventfd (which is wired to an
>> irqfd in another guest).
>>      
> At 4-bits per guest, a single write is then limited to 8 guests (with
> 32-bit registers), we could got to 64-bit.
>    

I meant a unicast doorbell: 16 bits for guest ID, 16 bits for vector number.

>>      
>>> So, ioeventfd/irqfd restricts MSI to 1 vector between guests.  Should
>>> multi-MSI even be supported then in the non-ioeventfd/irq case?
>>> Otherwise ioeventfd/irqfd become more than an implementation detail.
>>>
>>>        
>> I lost you.  Please re-explain.
>>      
> An irqfd can only trigger a single vector in a guest.  Right now I
> only have one eventfd per guest.    So ioeventfd/irqfd restricts the
> current implementation to a single vector that a guest can trigger.
> Without irqfd, eventfds can be used like registers a write the number
> of the vector they want to trigger, but as you point out it is racy.
>    

You can't use eventfds as registers.  The next write will add to the 
current value.

> So, supporting multiple vectors via irqfd requires multiple eventfds
> for each guest (one per vector).   a total of (# of guests) X (# of
> vectors) are required.  If we're limited to 8 or 16 guests that's not
> too bad, but since the server opens them all we're restricted to 1024,
> but that's a pretty high ceiling for this purpose.
>    

I'm sure we can raise the fd ulimit for this.  Note, I think qemus need 
the ulimit raised as well, since an fd passed via SCM_RIGHTS probably 
counts as an open file.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.