Re: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

["Zhu, Lingshan" <lingshan.zhu@intel.com>]

On 10/24/2023 6:37 PM, Parav Pandit wrote:
>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
>> Sent: Tuesday, October 24, 2023 4:00 PM
>>
>> On 10/23/2023 6:14 PM, Parav Pandit wrote:
>>>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
>>>> Sent: Monday, October 23, 2023 3:39 PM
>>>>
>>>> On 10/20/2023 8:54 PM, Parav Pandit wrote:
>>>>>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
>>>>>> Sent: Friday, October 20, 2023 3:01 PM
>>>>>>
>>>>>> On 10/19/2023 6:33 PM, Parav Pandit wrote:
>>>>>>>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
>>>>>>>> Sent: Thursday, October 19, 2023 2:48 PM
>>>>>>>>
>>>>>>>> On 10/19/2023 5:14 PM, Michael S. Tsirkin wrote:
>>>>>>>>> On Thu, Oct 19, 2023 at 09:13:16AM +0000, Parav Pandit wrote:
>>>>>>>>>>> Oh, really? Quite interesting, do you want to move all config
>>>>>>>>>>> space fields in VF to admin vq? Have a plan?
>>>>>>>>>> Not in my plan for spec 1.4 time frame.
>>>>>>>>>> I do not want to divert the discussion, would like to focus on
>>>>>>>>>> device
>>>>>>>> migration phases.
>>>>>>>>>> Lets please discuss in some other dedicated thread.
>>>>>>>>> Possibly, if there's a way to send admin commands to vf itself
>>>>>>>>> then Lingshan will be happy?
>>>>>>>> still need to prove why admin commands are better than registers.
>>>>>>> Virtio spec development is not proof based approach. Please stop
>>>>>>> asking for
>>>> it.
>>>>>>> I tried my best to have technical answer in [1].
>>>>>>> I explained that registers simply do not work for passthrough mode
>>>>>>> (if this is what you are asking when you are asking prove its better).
>>>>>>> They can work for non_passthrough mediated mode.
>>>>>>>
>>>>>>> A member device may do admin commands using registers. Michael and
>>>>>>> I are
>>>>>> discussing presently in the same thread.
>>>>>>> Since there are multiple things to be done for device migration,
>>>>>>> dedicated
>>>>>> register set for each functionality do not scale well, hard to
>>>>>> maintain and extend.
>>>>>>> A register holding a command content make sense.
>>>>>>>
>>>>>>> Now, with that, if this can be useful only for non_passthrough, I
>>>>>>> made humble
>>>>>> request to transport them using AQ, this way, you get all benefits of AQ.
>>>>>>> And trying to understand, why AQ cannot possible or inferior?
>>>>>>>
>>>>>>> If you have commands like suspend/resume device, register or queue
>>>>>> transport simply don’t work, because it's wrong to bifurcate the
>>>>>> device with such weird API.
>>>>>>> If you want to biferacate for mediation software, it probably
>>>>>>> makes sense to
>>>>>> operate at each VQ level, config space level. Such are very
>>>>>> different commands than passthrough.
>>>>>>> I think vdpa has demonstrated that very well on how to do specific
>>>>>>> work for
>>>>>> specific device type. So some of those work can be done using AQ.
>>>>>>> [1]
>>>>>>> https://lore.kernel.org/virtio-comment/870ace02-f99c-4582-932f-bd1
>>>>>>> 03
>>>>>>> 36
>>>>>>> 2dae9@intel.com/T/#m37743aa924536d0256d6b3b8e83a11c750f28794
>>>>>> We have been through your statement for many times.
>>>>>> This is not about how many times you repeated, if you think this is
>>>>>> true, you need to prove that with solid evidence.
>>>>>>
>>>>> I will not respond to this comment anymore.
>>>> Ok if you choose not to respond.
>>>>>> For pass-through, I still recommend you to take a reference of
>>>>>> current virito-pci implementation, it works for pass-through, right?
>>>>> What do you mean by current virtio-pci implementation?
>>>> current virito-pci works for pass-through
>>> I still don’t understand what is "current virtio-pci".
>>> Do you mean qemu implementation of emulated virtio-pci or you mean
>> virtio-pci specification for passthrough?
>>> What do you want me to refer to for passthrough? Please clarify.
>> you know guest vcpu and its vRC can not access host side devices, and there
>> must be a driver helping the pass-through use cases, like vDPA and vfio
> I am not sure how to corelate this answer to the question of "virtio-pci for passthrough".
> :(
>
> Today when a virtio-pci member device is passthrough to the guest VM, hypervisor is not involved in virtio interface such as config space, cvq, data vq etc.
> Do you agree?
Can vCPU access host side device config space? It needs a pass-through 
helper driver like vfio, right?
>
>>>>>> For scale, I already told you for many times that they are
>>>>>> per-device facilities. How can a per-device facility not scale?
>>>>> Each VF device must implement new set of on-chip memory-based
>>>>> registers
>>>> which demands more power, die area which does not scale efficiently
>>>> to thousands of VFs.
>>>> that can be fpga gates or SOC implementing new features, you think
>>>> that is a waste?
>>> It is waste in hw, if there is a better approach possible to not burn them as
>> gates and save on resources for rarely used items.
>> Is a new entry in MSIX table a waste of HW?
> Not as must as existing MSI-X table entries which requires linear amount of on-chip memory.
anyway, even only one MSIX entry cost my HW resource than the amount of 
new registers in my proposal.
>
>> Can I say implementing admin vq in SOC is a waste of cores?
> Which cores in the SoC?
> If it is on the PF, there is only handful of AQs for scale of N VFs.
I see you got the point anyway, new features cost extra resource
>
>>>
>>>>>> vDPA works fine on config space.
>>>>>>
>>>>>> So, if you still insist admin vq is better than config space like
>>>>>> in other thread you have concluded, you may imply that config space
>>>>>> interfaces should be re-factored to admin vq.
>>>>> Whatever is done in past is done, there is no way to change history.
>>>>> An new non init time registers should not be placed in device
>>>>> specific config
>>>> space as virtio spec has clear guideline on it for good.
>>>>> Device context reading, dirty page address reading, changing vf
>>>>> device modes,
>>>> all of these are clearly not a init time settings.
>>>>> Hence, they do not belong to the registers.
>>>> reset vq? and you get it from Appendix B. Creating New Device Types,
>>>> are we implementing a new type of device???
>>> I don’t understand your question.
>>> I replied the history of reset_vq.
>>> Take good examples to follow, reset_vq clearly is not the one.
>> so again, we are not implementing new device type, so your citation doesn't
>> apply.
> I disagree.
> I am engineer to build practical systems considering limitations and also advancements of the transport; while listening to other industry efforts,
> I am no from legal department.
> Hence, Appendix B makes a sense to me to apply to the existing device which also has the section for "device improvements".
it titled as "new device", and I think this discussion is non-sense. So 
if you want to fix this statement, works for me.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/