Re: [RFC] Proposed XenStore Interactions for Multi-Queue VIFs

["Egger, Christoph" <chegger@amazon.de>]

On 27.06.13 12:11, Andrew Bennieston wrote:
> On 27/06/13 10:07, Jan Beulich wrote:
>>>>> On 26.06.13 at 18:59, Andrew Bennieston
>>>>> <andrew.bennieston@citrix.com> wrote:
>>> 2. Backend feature advertising
>>> ------------------------------
>>> The backend advertises the features it supports via keys of the form
>>>
>>>       /local/domain/0/backend/vif/X/Y/feature-NNN = "1"
>>>
>>> where X is the domain ID and Y is the virtual network device number.
>>>
>>> In this proposal, a backend that wishes to support multi-queue VIFs
>>> would add the key
>>>
>>>       /local/domain/0/backend/vif/X/Y/feature-multi-queue = "1"
>>>
>>> If this key exists and is set to "1", the frontend may request a
>>> multi-queue configuration. If the key is set to "0", or does not exist,
>>> the backend either does not support this feature, or it has been
>>> disabled.
>>>
>>> In addition to the feature flag, a backend which supports
>>> feature-multi-queue would advertise a maximum number of queues, via the
>>> key:
>>>
>>>       /local/domain/0/backend/vif/X/Y/multi-queue-max-queues
>>>
>>> This value is the maximum number of supported ring pairs; each queue
>>> consists of a pair of rings supporting Tx (from guest) and Rx (to
>>> guest). The number of rings in total is twice the value of
>>> multi-queue-max-queues.
>>
>> I pretty much dislike this redundant advertisement - a single
>> key absolutely suffices here - absence of the key or a value
>> <= 1 are a sufficient indication of the feature not being
>> supported.
>>
> 
> You're right; I explicitly did not put a 'request-feature-multi-queue'
> in the frontend keys for this reason, but this one slipped past! The
> presence of multi-queue-max-queues > 1 is certainly sufficient to
> advertise multi-queue support.

I advertise a scheme that enlists all features in the Dom0 with

  xenstore-ls | grep "feature-"

That makes it easier to get a clue what is put in place and what
is used.

Christoph

> 
>>> 3.2 Shared ring grant references and event channels
>>> ---------------------------------------------------
>>> 3.2.1 Ring pages
>>> ----------------
>>> It is the responsibility of the frontend to allocate one page for each
>>> ring (i.e. two pages for each queue) and provide a grant reference to
>>> each page, so that the backend may map them. In the single-queue case,
>>> this is done as usual with the tx-ring-ref and rx-ring-ref keys.
>>>
>>> For multi-queue, a hierarchical structure is proposed. This serves the
>>> dual purpose of clean separation of grant references between queues and
>>> allows additional mechanisms (e.g. split event channels, multi-page
>>> rings) to replicate their XenStore keys for each queue without name
>>> collisions. For each queue, the frontend should set up the following
>>> keys:
>>>
>>>       /local/domain/X/device/vif/Y/queue-N/tx-ring-ref
>>>       /local/domain/X/device/vif/Y/queue-N/rx-ring-ref
>>>
>>> where X is the domain ID, Y is the device ID and N is the queue number
>>> (beginning at zero).
>>>
>>> 3.2.2 Event channels
>>> --------------------
>>> The upstream netback and netfront code supports
>>> feature-split-event-channels, allowing one channel per ring (instead of
>>> one channel per VIF). When multiple queues are used, the frontend must
>>> write either:
>>>
>>>       /local/domain/X/device/vif/Y/queue-N/event-channel = "M"
>>>
>>> to use a single event channel (number M) for that queue, or
>>>
>>>       /local/domain/X/device/vif/Y/queue-N/tx-event-channel = "M"
>>>       /local/domain/X/device/vif/Y/queue-N/rx-event-channel = "L"
>>>
>>> to use split event channels (numbers L, M) for that queue.
>>
>> Other than Wei, I'm actually in favor of this model. I don't see this
>> adding much complexity to the parsing logic in the backend: It's a
>> simply loop over the requested queue count, otherwise doing the
>> same operations as it does currently.
> 
> Indeed. I think Wei was referring to the hash-specific parameters, which
> will be "grouped" (but have one key per parameter) according to, e.g.
> /local/domain/X/device/vif/Y/multi-queue-hash-params-alg1/key = "..."
> /local/domain/X/device/vif/Y/multi-queue-hash-params-alg1/map = "..."
> 
> Andrew