[virtio-dev] Packed ring only devices clarification

[virtio-dev] Packed ring only devices clarification

[Maxime Coquelin]

Hello,

In the Virtqueues section of the Virtio specification [0], it is
mentioned that:

"
Every driver and device supports either the Packed or the Split
Virtqueue format, or both.
"

However, it seems there is no way to differentiate whether the device
supports both split and packed ring, or only packed ring by reading
its features bits.

In the case of live-migration, it means we would need to trigger a
features negotiation before the migration is initiated, and see if
FEATURES_OK device status bit is set. Still, we could not be sure a
failure would be due to the device supporting only packed ring.

Is my understanding correct?

If so, maybe we would need to introduce a new feature bit for split
ring, and also a feature bit indicating compliance with the new spec
version for backward compatibility:

VIRTIO_F_VERSION_1_2(39):
    This indicates compliance with Virtio version 1.2 specification.

VIRTIO_F_RING_SPLIT(40):
    This feature indicates support for the split virtqueue layout as
    described in 2.6 Split Virtqueues.


Best regards,
Maxime

[0]:
https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Packed ring only devices clarification

[Jason Wang]

On 2021/1/29 下午4:38, Maxime Coquelin wrote:
> Hello,
>
> In the Virtqueues section of the Virtio specification [0], it is
> mentioned that:
>
> "
> Every driver and device supports either the Packed or the Split
> Virtqueue format, or both.
> "
>
> However, it seems there is no way to differentiate whether the device
> supports both split and packed ring, or only packed ring by reading
> its features bits.
>
> In the case of live-migration, it means we would need to trigger a
> features negotiation before the migration is initiated, and see if
> FEATURES_OK device status bit is set. Still, we could not be sure a
> failure would be due to the device supporting only packed ring.
>
> Is my understanding correct?


Yes, I think so.


>
> If so, maybe we would need to introduce a new feature bit for split
> ring, and also a feature bit indicating compliance with the new spec
> version for backward compatibility:
>
> VIRTIO_F_VERSION_1_2(39):
>      This indicates compliance with Virtio version 1.2 specification.
>
> VIRTIO_F_RING_SPLIT(40):
>      This feature indicates support for the split virtqueue layout as
>      described in 2.6 Split Virtqueues.


A question here, do we want to fix this issues with old driver (when 
both of the above are not negotiated)?

Thanks


>
>
> Best regards,
> Maxime
>
> [0]:
> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Packed ring only devices clarification

[Maxime Coquelin]

On 1/29/21 10:04 AM, Jason Wang wrote:
> 
> On 2021/1/29 下午4:38, Maxime Coquelin wrote:
>> Hello,
>>
>> In the Virtqueues section of the Virtio specification [0], it is
>> mentioned that:
>>
>> "
>> Every driver and device supports either the Packed or the Split
>> Virtqueue format, or both.
>> "
>>
>> However, it seems there is no way to differentiate whether the device
>> supports both split and packed ring, or only packed ring by reading
>> its features bits.
>>
>> In the case of live-migration, it means we would need to trigger a
>> features negotiation before the migration is initiated, and see if
>> FEATURES_OK device status bit is set. Still, we could not be sure a
>> failure would be due to the device supporting only packed ring.
>>
>> Is my understanding correct?
> 
> 
> Yes, I think so.
> 
> 
>>
>> If so, maybe we would need to introduce a new feature bit for split
>> ring, and also a feature bit indicating compliance with the new spec
>> version for backward compatibility:
>>
>> VIRTIO_F_VERSION_1_2(39):
>>      This indicates compliance with Virtio version 1.2 specification.
>>
>> VIRTIO_F_RING_SPLIT(40):
>>      This feature indicates support for the split virtqueue layout as
>>      described in 2.6 Split Virtqueues.
> 
> 
> A question here, do we want to fix this issues with old driver (when
> both of the above are not negotiated)?

Could you elaborate please? I am not sure to understand your question.

For existing devices only supporting packed ring (if it exists, I don't
know any for now), maybe their drivers could have a quirk adding above
features. If we do that, we may want to rename VIRTIO_F_VERSION_1_2 with
something else to restrict it to rings layout support.

Thanks,
Maxime

> Thanks
> 
> 
>>
>>
>> Best regards,
>> Maxime
>>
>> [0]:
>> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Packed ring only devices clarification

[Jason Wang]

On 2021/1/29 下午5:14, Maxime Coquelin wrote:
>
> On 1/29/21 10:04 AM, Jason Wang wrote:
>> On 2021/1/29 下午4:38, Maxime Coquelin wrote:
>>> Hello,
>>>
>>> In the Virtqueues section of the Virtio specification [0], it is
>>> mentioned that:
>>>
>>> "
>>> Every driver and device supports either the Packed or the Split
>>> Virtqueue format, or both.
>>> "
>>>
>>> However, it seems there is no way to differentiate whether the device
>>> supports both split and packed ring, or only packed ring by reading
>>> its features bits.
>>>
>>> In the case of live-migration, it means we would need to trigger a
>>> features negotiation before the migration is initiated, and see if
>>> FEATURES_OK device status bit is set. Still, we could not be sure a
>>> failure would be due to the device supporting only packed ring.
>>>
>>> Is my understanding correct?
>>
>> Yes, I think so.
>>
>>
>>> If so, maybe we would need to introduce a new feature bit for split
>>> ring, and also a feature bit indicating compliance with the new spec
>>> version for backward compatibility:
>>>
>>> VIRTIO_F_VERSION_1_2(39):
>>>       This indicates compliance with Virtio version 1.2 specification.
>>>
>>> VIRTIO_F_RING_SPLIT(40):
>>>       This feature indicates support for the split virtqueue layout as
>>>       described in 2.6 Split Virtqueues.
>>
>> A question here, do we want to fix this issues with old driver (when
>> both of the above are not negotiated)?
> Could you elaborate please? I am not sure to understand your question.


I meant it looks to me we can't avoid the feature pre-negotiation, if we 
migrate to an elder qemu?


>
> For existing devices only supporting packed ring (if it exists, I don't
> know any for now),


Spec doesn't forbid that so I guess there would be some hardware 
implementation will do packed only.


>   maybe their drivers could have a quirk adding above
> features.


It won't be easy consider there will be a software updating in the guest.


>   If we do that, we may want to rename VIRTIO_F_VERSION_1_2 with
> something else to restrict it to rings layout support.


This looks like a common issue which is not packed specific, it might 
happen when destination mandate some features but src doesn't (e.g 
VIRTIO_F_ACCESS_PLATFORM).

Maybe it's the time to introduce an API in the spec to know the features 
that is forced by the device.

Thanks


>
> Thanks,
> Maxime
>
>> Thanks
>>
>>
>>>
>>> Best regards,
>>> Maxime
>>>
>>> [0]:
>>> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>>> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: Packed ring only devices clarification

[Michael S. Tsirkin]

On Fri, Jan 29, 2021 at 09:38:25AM +0100, Maxime Coquelin wrote:
> Hello,
> 
> In the Virtqueues section of the Virtio specification [0], it is
> mentioned that:
> 
> "
> Every driver and device supports either the Packed or the Split
> Virtqueue format, or both.
> "
> 
> However, it seems there is no way to differentiate whether the device
> supports both split and packed ring, or only packed ring by reading
> its features bits.
> 
> In the case of live-migration, it means we would need to trigger a
> features negotiation before the migration is initiated, and see if
> FEATURES_OK device status bit is set. Still, we could not be sure a
> failure would be due to the device supporting only packed ring.
> 
> Is my understanding correct?


The simple answer is that live migration is transparent to
guests and so isn't ATM described by the virtio spec.

devices implementing live migration need not constrain
themselves to using the feature bits described in the spec,
e.g. vhost-user has its own "protocol features".

> If so, maybe we would need to introduce a new feature bit for split
> ring, and also a feature bit indicating compliance with the new spec
> version for backward compatibility:
> 
> VIRTIO_F_VERSION_1_2(39):
>     This indicates compliance with Virtio version 1.2 specification.
> 
> VIRTIO_F_RING_SPLIT(40):
>     This feature indicates support for the split virtqueue layout as
>     described in 2.6 Split Virtqueues.
> 
> 
> Best regards,
> Maxime
> 
> [0]:
> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: Packed ring only devices clarification

[Jason Wang]

On 2021/1/29 下午11:58, Michael S. Tsirkin wrote:
> On Fri, Jan 29, 2021 at 09:38:25AM +0100, Maxime Coquelin wrote:
>> Hello,
>>
>> In the Virtqueues section of the Virtio specification [0], it is
>> mentioned that:
>>
>> "
>> Every driver and device supports either the Packed or the Split
>> Virtqueue format, or both.
>> "
>>
>> However, it seems there is no way to differentiate whether the device
>> supports both split and packed ring, or only packed ring by reading
>> its features bits.
>>
>> In the case of live-migration, it means we would need to trigger a
>> features negotiation before the migration is initiated, and see if
>> FEATURES_OK device status bit is set. Still, we could not be sure a
>> failure would be due to the device supporting only packed ring.
>>
>> Is my understanding correct?
>
> The simple answer is that live migration is transparent to
> guests and so isn't ATM described by the virtio spec.


This might require more thoughts, consider in a cloud with multiple 
virtio implementations. How do management know it's safe to migrate 
between two nodes? We probably need something like the concept of 
cluster but there should be way for management to know which parts of 
features is supported and mandated by a device.

Thanks


>
> devices implementing live migration need not constrain
> themselves to using the feature bits described in the spec,
> e.g. vhost-user has its own "protocol features".
>
>> If so, maybe we would need to introduce a new feature bit for split
>> ring, and also a feature bit indicating compliance with the new spec
>> version for backward compatibility:
>>
>> VIRTIO_F_VERSION_1_2(39):
>>      This indicates compliance with Virtio version 1.2 specification.
>>
>> VIRTIO_F_RING_SPLIT(40):
>>      This feature indicates support for the split virtqueue layout as
>>      described in 2.6 Split Virtqueues.
>>
>>
>> Best regards,
>> Maxime
>>
>> [0]:
>> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-230005


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org