* Re: memory locking in vhost-vdpa [not found] <780fba4c-458f-0abc-ae48-4cf806401228@virtuozzo.com> @ 2022-08-04 1:52 ` Jason Wang 2022-08-04 6:30 ` Michael S. Tsirkin 2022-08-04 6:39 ` Stefano Garzarella 0 siblings, 2 replies; 5+ messages in thread From: Jason Wang @ 2022-08-04 1:52 UTC (permalink / raw) To: Andrey Zhadchenko; +Cc: den, virtualization, Michael S. Tsirkin On Thu, Aug 4, 2022 at 1:46 AM Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com> wrote: > > Hi! > > Recently I sent vhost-blk patchset and Stefano suggested to instead join > efforts on developing vdpa-blk. > I played with it a bit and looks like vdpa itself pins the whole guest > memory. Is there a way to control it or reduce pinned amount to the > device pages? > Looks like even vdpa-sim requires all memory to be pinned [1]. Pinning > this much will surely impact guest density. It depends on the parent. When allocating the vDPA device, the parent can clams it supports virtual address then pinning is avoided: /** * __vdpa_alloc_device - allocate and initilaize a vDPA device * This allows driver to some prepartion after device is * initialized but before registered. ... * @use_va: indicate whether virtual address must be used by this device */ The only user so far is VDUSE which is a software parent in the userspace with a customized swiotlb for kernel drivers. Simulator came before this feature so we stick to the pinning method, technically we can switch to use the va mode, but it might have some performance impact (mostly the copy_from|to_user()). This option might be useful for hardware parent if PRI or device page fault is supported in the future. Thanks > > Kind regards, > Andrey > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1868535 > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: memory locking in vhost-vdpa 2022-08-04 1:52 ` memory locking in vhost-vdpa Jason Wang @ 2022-08-04 6:30 ` Michael S. Tsirkin 2022-08-04 6:39 ` Stefano Garzarella 1 sibling, 0 replies; 5+ messages in thread From: Michael S. Tsirkin @ 2022-08-04 6:30 UTC (permalink / raw) To: Jason Wang; +Cc: Andrey Zhadchenko, den, virtualization On Thu, Aug 04, 2022 at 09:52:47AM +0800, Jason Wang wrote: > On Thu, Aug 4, 2022 at 1:46 AM Andrey Zhadchenko > <andrey.zhadchenko@virtuozzo.com> wrote: > > > > Hi! > > > > Recently I sent vhost-blk patchset and Stefano suggested to instead join > > efforts on developing vdpa-blk. > > I played with it a bit and looks like vdpa itself pins the whole guest > > memory. Is there a way to control it or reduce pinned amount to the > > device pages? > > Looks like even vdpa-sim requires all memory to be pinned [1]. Pinning > > this much will surely impact guest density. > > It depends on the parent. > > When allocating the vDPA device, the parent can clams it supports > virtual address then pinning is avoided: > > /** > * __vdpa_alloc_device - allocate and initilaize a vDPA device > * This allows driver to some prepartion after device is > * initialized but before registered. > ... > * @use_va: indicate whether virtual address must be used by this device > */ > > The only user so far is VDUSE which is a software parent in the > userspace with a customized swiotlb for kernel drivers. > > Simulator came before this feature so we stick to the pinning method, > technically we can switch to use the va mode, but it might have some > performance impact (mostly the copy_from|to_user()). > > This option might be useful for hardware parent if PRI or device page > fault is supported in the future. > > Thanks Well VDUSE has this funky bounce buffer design. It works but is costly performance wise. > > > > Kind regards, > > Andrey > > > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1868535 > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: memory locking in vhost-vdpa 2022-08-04 1:52 ` memory locking in vhost-vdpa Jason Wang 2022-08-04 6:30 ` Michael S. Tsirkin @ 2022-08-04 6:39 ` Stefano Garzarella 2022-08-04 6:43 ` Michael S. Tsirkin 2022-08-04 7:27 ` Jason Wang 1 sibling, 2 replies; 5+ messages in thread From: Stefano Garzarella @ 2022-08-04 6:39 UTC (permalink / raw) To: Jason Wang; +Cc: Andrey Zhadchenko, virtualization, den, Michael S. Tsirkin On Thu, Aug 04, 2022 at 09:52:47AM +0800, Jason Wang wrote: >On Thu, Aug 4, 2022 at 1:46 AM Andrey Zhadchenko ><andrey.zhadchenko@virtuozzo.com> wrote: >> >> Hi! >> >> Recently I sent vhost-blk patchset and Stefano suggested to instead join >> efforts on developing vdpa-blk. >> I played with it a bit and looks like vdpa itself pins the whole guest >> memory. Is there a way to control it or reduce pinned amount to the >> device pages? >> Looks like even vdpa-sim requires all memory to be pinned [1]. Pinning >> this much will surely impact guest density. > >It depends on the parent. > >When allocating the vDPA device, the parent can clams it supports >virtual address then pinning is avoided: > >/** > * __vdpa_alloc_device - allocate and initilaize a vDPA device > * This allows driver to some prepartion after device is > * initialized but before registered. >... > * @use_va: indicate whether virtual address must be used by this device > */ > >The only user so far is VDUSE which is a software parent in the >userspace with a customized swiotlb for kernel drivers. > >Simulator came before this feature so we stick to the pinning method, I based vdpa-blk PoC on the simulator and didn't realize this, maybe I should have used this. >technically we can switch to use the va mode, but it might have some >performance impact (mostly the copy_from|to_user()). Would the cost be comparable to implementing a vhost-blk device? (IIRC vq in vhost uses copy_from/to_user, right?) Thanks, Stefano _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: memory locking in vhost-vdpa 2022-08-04 6:39 ` Stefano Garzarella @ 2022-08-04 6:43 ` Michael S. Tsirkin 2022-08-04 7:27 ` Jason Wang 1 sibling, 0 replies; 5+ messages in thread From: Michael S. Tsirkin @ 2022-08-04 6:43 UTC (permalink / raw) To: Stefano Garzarella; +Cc: Andrey Zhadchenko, den, virtualization On Thu, Aug 04, 2022 at 08:39:11AM +0200, Stefano Garzarella wrote: > > technically we can switch to use the va mode, but it might have some > > performance impact (mostly the copy_from|to_user()). > > Would the cost be comparable to implementing a vhost-blk device? (IIRC vq in > vhost uses copy_from/to_user, right?) > > Thanks, > Stefano Well not in exactly the same way. -- MST _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: memory locking in vhost-vdpa 2022-08-04 6:39 ` Stefano Garzarella 2022-08-04 6:43 ` Michael S. Tsirkin @ 2022-08-04 7:27 ` Jason Wang 1 sibling, 0 replies; 5+ messages in thread From: Jason Wang @ 2022-08-04 7:27 UTC (permalink / raw) To: Stefano Garzarella Cc: Andrey Zhadchenko, virtualization, den, Michael S. Tsirkin On Thu, Aug 4, 2022 at 2:39 PM Stefano Garzarella <sgarzare@redhat.com> wrote: > > On Thu, Aug 04, 2022 at 09:52:47AM +0800, Jason Wang wrote: > >On Thu, Aug 4, 2022 at 1:46 AM Andrey Zhadchenko > ><andrey.zhadchenko@virtuozzo.com> wrote: > >> > >> Hi! > >> > >> Recently I sent vhost-blk patchset and Stefano suggested to instead join > >> efforts on developing vdpa-blk. > >> I played with it a bit and looks like vdpa itself pins the whole guest > >> memory. Is there a way to control it or reduce pinned amount to the > >> device pages? > >> Looks like even vdpa-sim requires all memory to be pinned [1]. Pinning > >> this much will surely impact guest density. > > > >It depends on the parent. > > > >When allocating the vDPA device, the parent can clams it supports > >virtual address then pinning is avoided: > > > >/** > > * __vdpa_alloc_device - allocate and initilaize a vDPA device > > * This allows driver to some prepartion after device is > > * initialized but before registered. > >... > > * @use_va: indicate whether virtual address must be used by this device > > */ > > > >The only user so far is VDUSE which is a software parent in the > >userspace with a customized swiotlb for kernel drivers. > > > >Simulator came before this feature so we stick to the pinning method, > > I based vdpa-blk PoC on the simulator and didn't realize this, maybe I > should have used this. > > >technically we can switch to use the va mode, but it might have some > >performance impact (mostly the copy_from|to_user()). > > Would the cost be comparable to implementing a vhost-blk device? (IIRC > vq in vhost uses copy_from/to_user, right?) It would not be that simple. We need to differentiate kernel va from userspace va at least, since the buffer could belong to kernel (via virtio-vdpa). Thanks > > Thanks, > Stefano > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-08-04 7:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <780fba4c-458f-0abc-ae48-4cf806401228@virtuozzo.com>
2022-08-04 1:52 ` memory locking in vhost-vdpa Jason Wang
2022-08-04 6:30 ` Michael S. Tsirkin
2022-08-04 6:39 ` Stefano Garzarella
2022-08-04 6:43 ` Michael S. Tsirkin
2022-08-04 7:27 ` Jason Wang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.