* Re: [PATCH net-next v2 0/5] virtio: support packed ring
From: Jason Wang @ 2018-07-13 0:52 UTC (permalink / raw)
To: David Miller, tiwei.bie
Cc: virtio-dev, mst, netdev, linux-kernel, virtualization, wexu
In-Reply-To: <20180712.144458.2076041018423659380.davem@davemloft.net>
On 2018年07月13日 05:44, David Miller wrote:
> From: Tiwei Bie <tiwei.bie@intel.com>
> Date: Wed, 11 Jul 2018 10:27:06 +0800
>
>> Hello everyone,
>>
>> This patch set implements packed ring support in virtio driver.
>>
>> Some functional tests have been done with Jason's
>> packed ring implementation in vhost:
>>
>> https://lkml.org/lkml/2018/7/3/33
>>
>> Both of ping and netperf worked as expected.
> Michael and Jason, where are we with this series?
For the series:
Acked-by: Jason Wang <jasowang@redhat.com>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wei Wang @ 2018-07-13 0:33 UTC (permalink / raw)
To: Michal Hocko
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal,
Linus Torvalds
In-Reply-To: <20180712114946.GI32648@dhcp22.suse.cz>
On 07/12/2018 07:49 PM, Michal Hocko wrote:
> On Thu 12-07-18 19:34:16, Wei Wang wrote:
>> On 07/12/2018 04:13 PM, Michal Hocko wrote:
>>> On Thu 12-07-18 10:52:08, Wei Wang wrote:
>>>> On 07/12/2018 10:30 AM, Linus Torvalds wrote:
>>>>> On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
>>>>>> Would it be better to remove __GFP_THISNODE? We actually want to get all
>>>>>> the guest free pages (from all the nodes).
>>>>> Maybe. Or maybe it would be better to have the memory balloon logic be
>>>>> per-node? Maybe you don't want to remove too much memory from one
>>>>> node? I think it's one of those "play with it" things.
>>>>>
>>>>> I don't think that's the big issue, actually. I think the real issue
>>>>> is how to react quickly and gracefully to "oops, I'm trying to give
>>>>> memory away, but now the guest wants it back" while you're in the
>>>>> middle of trying to create that 2TB list of pages.
>>>> OK. virtio-balloon has already registered an oom notifier
>>>> (virtballoon_oom_notify). I plan to add some control there. If oom happens,
>>>> - stop the page allocation;
>>>> - immediately give back the allocated pages to mm.
>>> Please don't. Oom notifier is an absolutely hideous interface which
>>> should go away sooner or later (I would much rather like the former) so
>>> do not build a new logic on top of it. I would appreciate if you
>>> actually remove the notifier much more.
>>>
>>> You can give memory back from the standard shrinker interface. If we are
>>> reaching low reclaim priorities then we are struggling to reclaim memory
>>> and then you can start returning pages back.
>> OK. Just curious why oom notifier is thought to be hideous, and has it been
>> a consensus?
> Because it is a completely non-transparent callout from the OOM context
> which is really subtle on its own. It is just too easy to end up in
> weird corner cases. We really have to be careful and be as swift as
> possible. Any potential sleep would make the OOM situation much worse
> because nobody would be able to make a forward progress or (in)direct
> dependency on MM subsystem can easily deadlock. Those are really hard
> to track down and defining the notifier as blockable by design which
> just asks for bad implementations because most people simply do not
> realize how subtle the oom context is.
>
> Another thing is that it happens way too late when we have basically
> reclaimed the world and didn't get out of the memory pressure so you can
> expect any workload is suffering already. Anybody sitting on a large
> amount of reclaimable memory should have released that memory by that
> time. Proportionally to the reclaim pressure ideally.
>
> The notifier API is completely unaware of oom constrains. Just imagine
> you are OOM in a subset of numa nodes. Callback doesn't have any idea
> about that.
>
> Moreover we do have proper reclaim mechanism that has a feedback
> loop and that should be always preferable to an abrupt reclaim.
Sounds very reasonable, thanks for the elaboration. I'll try with shrinker.
Best,
Wei
^ permalink raw reply
* Re: [PATCH net-next v2 0/5] virtio: support packed ring
From: David Miller @ 2018-07-12 21:44 UTC (permalink / raw)
To: tiwei.bie; +Cc: virtio-dev, mst, netdev, linux-kernel, virtualization, wexu
In-Reply-To: <20180711022711.7090-1-tiwei.bie@intel.com>
From: Tiwei Bie <tiwei.bie@intel.com>
Date: Wed, 11 Jul 2018 10:27:06 +0800
> Hello everyone,
>
> This patch set implements packed ring support in virtio driver.
>
> Some functional tests have been done with Jason's
> packed ring implementation in vhost:
>
> https://lkml.org/lkml/2018/7/3/33
>
> Both of ping and netperf worked as expected.
Michael and Jason, where are we with this series?
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-12 13:12 UTC (permalink / raw)
To: Linus Torvalds
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal
In-Reply-To: <CA+55aFwku2tDH4+rfaC67xc4-cEwSrXgnQaci=e2id5ZCRE9JQ@mail.gmail.com>
[Hmm this one somehow got stuck in my outgoing emails]
On Wed 11-07-18 09:23:54, Linus Torvalds wrote:
[...]
> So I'm open to new interfaces. I just want those new interfaces to
> make sense, and be low latency and simple for the VM to do. I'm
> objecting to the incredibly baroque and heavy-weight one that can
> return near-infinite amounts of memory.
Mel was suggesting a bulk page allocator a year ago [1]. I can see only
slab bulk api so I am not sure what happened with that work. Anyway
I think that starting with what we have right now is much more
appropriate than over design this thing from the early beginning.
[1] http://lkml.kernel.org/r/20170109163518.6001-5-mgorman@techsingularity.net
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-12 11:49 UTC (permalink / raw)
To: Wei Wang
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal,
Linus Torvalds
In-Reply-To: <5B473CB8.1050306@intel.com>
On Thu 12-07-18 19:34:16, Wei Wang wrote:
> On 07/12/2018 04:13 PM, Michal Hocko wrote:
> > On Thu 12-07-18 10:52:08, Wei Wang wrote:
> > > On 07/12/2018 10:30 AM, Linus Torvalds wrote:
> > > > On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
> > > > > Would it be better to remove __GFP_THISNODE? We actually want to get all
> > > > > the guest free pages (from all the nodes).
> > > > Maybe. Or maybe it would be better to have the memory balloon logic be
> > > > per-node? Maybe you don't want to remove too much memory from one
> > > > node? I think it's one of those "play with it" things.
> > > >
> > > > I don't think that's the big issue, actually. I think the real issue
> > > > is how to react quickly and gracefully to "oops, I'm trying to give
> > > > memory away, but now the guest wants it back" while you're in the
> > > > middle of trying to create that 2TB list of pages.
> > > OK. virtio-balloon has already registered an oom notifier
> > > (virtballoon_oom_notify). I plan to add some control there. If oom happens,
> > > - stop the page allocation;
> > > - immediately give back the allocated pages to mm.
> > Please don't. Oom notifier is an absolutely hideous interface which
> > should go away sooner or later (I would much rather like the former) so
> > do not build a new logic on top of it. I would appreciate if you
> > actually remove the notifier much more.
> >
> > You can give memory back from the standard shrinker interface. If we are
> > reaching low reclaim priorities then we are struggling to reclaim memory
> > and then you can start returning pages back.
>
> OK. Just curious why oom notifier is thought to be hideous, and has it been
> a consensus?
Because it is a completely non-transparent callout from the OOM context
which is really subtle on its own. It is just too easy to end up in
weird corner cases. We really have to be careful and be as swift as
possible. Any potential sleep would make the OOM situation much worse
because nobody would be able to make a forward progress or (in)direct
dependency on MM subsystem can easily deadlock. Those are really hard
to track down and defining the notifier as blockable by design which
just asks for bad implementations because most people simply do not
realize how subtle the oom context is.
Another thing is that it happens way too late when we have basically
reclaimed the world and didn't get out of the memory pressure so you can
expect any workload is suffering already. Anybody sitting on a large
amount of reclaimable memory should have released that memory by that
time. Proportionally to the reclaim pressure ideally.
The notifier API is completely unaware of oom constrains. Just imagine
you are OOM in a subset of numa nodes. Callback doesn't have any idea
about that.
Moreover we do have proper reclaim mechanism that has a feedback
loop and that should be always preferable to an abrupt reclaim.
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wei Wang @ 2018-07-12 11:34 UTC (permalink / raw)
To: Michal Hocko
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal,
Linus Torvalds
In-Reply-To: <20180712081317.GD32648@dhcp22.suse.cz>
On 07/12/2018 04:13 PM, Michal Hocko wrote:
> On Thu 12-07-18 10:52:08, Wei Wang wrote:
>> On 07/12/2018 10:30 AM, Linus Torvalds wrote:
>>> On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
>>>> Would it be better to remove __GFP_THISNODE? We actually want to get all
>>>> the guest free pages (from all the nodes).
>>> Maybe. Or maybe it would be better to have the memory balloon logic be
>>> per-node? Maybe you don't want to remove too much memory from one
>>> node? I think it's one of those "play with it" things.
>>>
>>> I don't think that's the big issue, actually. I think the real issue
>>> is how to react quickly and gracefully to "oops, I'm trying to give
>>> memory away, but now the guest wants it back" while you're in the
>>> middle of trying to create that 2TB list of pages.
>> OK. virtio-balloon has already registered an oom notifier
>> (virtballoon_oom_notify). I plan to add some control there. If oom happens,
>> - stop the page allocation;
>> - immediately give back the allocated pages to mm.
> Please don't. Oom notifier is an absolutely hideous interface which
> should go away sooner or later (I would much rather like the former) so
> do not build a new logic on top of it. I would appreciate if you
> actually remove the notifier much more.
>
> You can give memory back from the standard shrinker interface. If we are
> reaching low reclaim priorities then we are struggling to reclaim memory
> and then you can start returning pages back.
OK. Just curious why oom notifier is thought to be hideous, and has it
been a consensus?
Best,
Wei
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-12 8:13 UTC (permalink / raw)
To: Wei Wang
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal,
Linus Torvalds
In-Reply-To: <5B46C258.40601@intel.com>
On Thu 12-07-18 10:52:08, Wei Wang wrote:
> On 07/12/2018 10:30 AM, Linus Torvalds wrote:
> > On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
> > > Would it be better to remove __GFP_THISNODE? We actually want to get all
> > > the guest free pages (from all the nodes).
> > Maybe. Or maybe it would be better to have the memory balloon logic be
> > per-node? Maybe you don't want to remove too much memory from one
> > node? I think it's one of those "play with it" things.
> >
> > I don't think that's the big issue, actually. I think the real issue
> > is how to react quickly and gracefully to "oops, I'm trying to give
> > memory away, but now the guest wants it back" while you're in the
> > middle of trying to create that 2TB list of pages.
>
> OK. virtio-balloon has already registered an oom notifier
> (virtballoon_oom_notify). I plan to add some control there. If oom happens,
> - stop the page allocation;
> - immediately give back the allocated pages to mm.
Please don't. Oom notifier is an absolutely hideous interface which
should go away sooner or later (I would much rather like the former) so
do not build a new logic on top of it. I would appreciate if you
actually remove the notifier much more.
You can give memory back from the standard shrinker interface. If we are
reaching low reclaim priorities then we are struggling to reclaim memory
and then you can start returning pages back.
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Jason Wang @ 2018-07-12 5:51 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <20180712082324-mutt-send-email-mst@kernel.org>
On 2018年07月12日 13:24, Michael S. Tsirkin wrote:
> On Thu, Jul 12, 2018 at 01:21:03PM +0800, Jason Wang wrote:
>>
>> On 2018年07月12日 11:34, Michael S. Tsirkin wrote:
>>> On Thu, Jul 12, 2018 at 11:26:12AM +0800, Jason Wang wrote:
>>>> On 2018年07月11日 19:59, Michael S. Tsirkin wrote:
>>>>> On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
>>>>>> On 2018年07月11日 11:49, Tonghao Zhang wrote:
>>>>>>> On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
>>>>>>>> On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
>>>>>>>>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>>>>>>>>
>>>>>>>>> This patches improve the guest receive and transmit performance.
>>>>>>>>> On the handle_tx side, we poll the sock receive queue at the same time.
>>>>>>>>> handle_rx do that in the same way.
>>>>>>>>>
>>>>>>>>> For more performance report, see patch 4.
>>>>>>>>>
>>>>>>>>> v4 -> v5:
>>>>>>>>> fix some issues
>>>>>>>>>
>>>>>>>>> v3 -> v4:
>>>>>>>>> fix some issues
>>>>>>>>>
>>>>>>>>> v2 -> v3:
>>>>>>>>> This patches are splited from previous big patch:
>>>>>>>>> http://patchwork.ozlabs.org/patch/934673/
>>>>>>>>>
>>>>>>>>> Tonghao Zhang (4):
>>>>>>>>> vhost: lock the vqs one by one
>>>>>>>>> net: vhost: replace magic number of lock annotation
>>>>>>>>> net: vhost: factor out busy polling logic to vhost_net_busy_poll()
>>>>>>>>> net: vhost: add rx busy polling in tx path
>>>>>>>>>
>>>>>>>>> drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
>>>>>>>>> drivers/vhost/vhost.c | 24 ++++-------
>>>>>>>>> 2 files changed, 67 insertions(+), 65 deletions(-)
>>>>>>>>>
>>>>>>>> Hi, any progress on the new version?
>>>>>>>>
>>>>>>>> I plan to send a new series of packed virtqueue support of vhost. If you
>>>>>>>> plan to send it soon, I can wait. Otherwise, I will send my series.
>>>>>>> I rebase the codes. and find there is no improvement anymore, the
>>>>>>> patches of makita may solve the problem. jason you may send your
>>>>>>> patches, and I will do some research on busypoll.
>>>>>> I see. Maybe you can try some bi-directional traffic.
>>>>>>
>>>>>> Btw, lots of optimizations could be done for busy polling. E.g integrating
>>>>>> with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
>>>>>> to work or propose new ideas.
>>>>>>
>>>>>> Thanks
>>>>> It seems clear we do need adaptive polling.
>>>> Yes.
>>>>
>>>>> The difficulty with NAPI
>>>>> polling is it can't access guest memory easily. But maybe
>>>>> get_user_pages on the polled memory+NAPI polling can work.
>>>> You mean something like zerocopy? Looks like we can do busy polling without
>>>> it. I mean something like https://patchwork.kernel.org/patch/8707511/.
>>>>
>>>> Thanks
>>> How does this patch work? vhost_vq_avail_empty can sleep,
>>> you are calling it within an rcu read side critical section.
>> Ok, I get your meaning. I have patches to access vring through
>> get_user_pages + vmap() which should help here. (And it increase PPS about
>> 10%-20%).
> Remember you must mark it as dirty on unpin too ...
Ok.
>
>
>>> That's not the only problem btw, another one is that the
>>> CPU time spent polling isn't accounted with the VM.
>>
>> Yes, but it's not the 'issue' of this patch.
> Yes it is. polling within thread context accounts CPU correctly.
>
>> And I believe cgroup can help?
>>
>> Thanks
>
> cgroups are what's broken by polling in irq context.
But I think the NAPI busy polling is still done in process context.
Thanks
>
>>>>>>>> Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Michael S. Tsirkin @ 2018-07-12 5:24 UTC (permalink / raw)
To: Jason Wang; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <2898b8aa-8a7c-06ad-9dc0-0e97e2f8e417@redhat.com>
On Thu, Jul 12, 2018 at 01:21:03PM +0800, Jason Wang wrote:
>
>
> On 2018年07月12日 11:34, Michael S. Tsirkin wrote:
> > On Thu, Jul 12, 2018 at 11:26:12AM +0800, Jason Wang wrote:
> > >
> > > On 2018年07月11日 19:59, Michael S. Tsirkin wrote:
> > > > On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
> > > > > On 2018年07月11日 11:49, Tonghao Zhang wrote:
> > > > > > On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
> > > > > > > On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
> > > > > > > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > > > > > > >
> > > > > > > > This patches improve the guest receive and transmit performance.
> > > > > > > > On the handle_tx side, we poll the sock receive queue at the same time.
> > > > > > > > handle_rx do that in the same way.
> > > > > > > >
> > > > > > > > For more performance report, see patch 4.
> > > > > > > >
> > > > > > > > v4 -> v5:
> > > > > > > > fix some issues
> > > > > > > >
> > > > > > > > v3 -> v4:
> > > > > > > > fix some issues
> > > > > > > >
> > > > > > > > v2 -> v3:
> > > > > > > > This patches are splited from previous big patch:
> > > > > > > > http://patchwork.ozlabs.org/patch/934673/
> > > > > > > >
> > > > > > > > Tonghao Zhang (4):
> > > > > > > > vhost: lock the vqs one by one
> > > > > > > > net: vhost: replace magic number of lock annotation
> > > > > > > > net: vhost: factor out busy polling logic to vhost_net_busy_poll()
> > > > > > > > net: vhost: add rx busy polling in tx path
> > > > > > > >
> > > > > > > > drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
> > > > > > > > drivers/vhost/vhost.c | 24 ++++-------
> > > > > > > > 2 files changed, 67 insertions(+), 65 deletions(-)
> > > > > > > >
> > > > > > > Hi, any progress on the new version?
> > > > > > >
> > > > > > > I plan to send a new series of packed virtqueue support of vhost. If you
> > > > > > > plan to send it soon, I can wait. Otherwise, I will send my series.
> > > > > > I rebase the codes. and find there is no improvement anymore, the
> > > > > > patches of makita may solve the problem. jason you may send your
> > > > > > patches, and I will do some research on busypoll.
> > > > > I see. Maybe you can try some bi-directional traffic.
> > > > >
> > > > > Btw, lots of optimizations could be done for busy polling. E.g integrating
> > > > > with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
> > > > > to work or propose new ideas.
> > > > >
> > > > > Thanks
> > > > It seems clear we do need adaptive polling.
> > > Yes.
> > >
> > > > The difficulty with NAPI
> > > > polling is it can't access guest memory easily. But maybe
> > > > get_user_pages on the polled memory+NAPI polling can work.
> > > You mean something like zerocopy? Looks like we can do busy polling without
> > > it. I mean something like https://patchwork.kernel.org/patch/8707511/.
> > >
> > > Thanks
> > How does this patch work? vhost_vq_avail_empty can sleep,
> > you are calling it within an rcu read side critical section.
>
> Ok, I get your meaning. I have patches to access vring through
> get_user_pages + vmap() which should help here. (And it increase PPS about
> 10%-20%).
Remember you must mark it as dirty on unpin too ...
> >
> > That's not the only problem btw, another one is that the
> > CPU time spent polling isn't accounted with the VM.
>
>
> Yes, but it's not the 'issue' of this patch.
Yes it is. polling within thread context accounts CPU correctly.
> And I believe cgroup can help?
>
> Thanks
cgroups are what's broken by polling in irq context.
> >
> > > > > > > Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Jason Wang @ 2018-07-12 5:21 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <20180712062756-mutt-send-email-mst@kernel.org>
On 2018年07月12日 11:34, Michael S. Tsirkin wrote:
> On Thu, Jul 12, 2018 at 11:26:12AM +0800, Jason Wang wrote:
>>
>> On 2018年07月11日 19:59, Michael S. Tsirkin wrote:
>>> On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
>>>> On 2018年07月11日 11:49, Tonghao Zhang wrote:
>>>>> On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
>>>>>> On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
>>>>>>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>>>>>>
>>>>>>> This patches improve the guest receive and transmit performance.
>>>>>>> On the handle_tx side, we poll the sock receive queue at the same time.
>>>>>>> handle_rx do that in the same way.
>>>>>>>
>>>>>>> For more performance report, see patch 4.
>>>>>>>
>>>>>>> v4 -> v5:
>>>>>>> fix some issues
>>>>>>>
>>>>>>> v3 -> v4:
>>>>>>> fix some issues
>>>>>>>
>>>>>>> v2 -> v3:
>>>>>>> This patches are splited from previous big patch:
>>>>>>> http://patchwork.ozlabs.org/patch/934673/
>>>>>>>
>>>>>>> Tonghao Zhang (4):
>>>>>>> vhost: lock the vqs one by one
>>>>>>> net: vhost: replace magic number of lock annotation
>>>>>>> net: vhost: factor out busy polling logic to vhost_net_busy_poll()
>>>>>>> net: vhost: add rx busy polling in tx path
>>>>>>>
>>>>>>> drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
>>>>>>> drivers/vhost/vhost.c | 24 ++++-------
>>>>>>> 2 files changed, 67 insertions(+), 65 deletions(-)
>>>>>>>
>>>>>> Hi, any progress on the new version?
>>>>>>
>>>>>> I plan to send a new series of packed virtqueue support of vhost. If you
>>>>>> plan to send it soon, I can wait. Otherwise, I will send my series.
>>>>> I rebase the codes. and find there is no improvement anymore, the
>>>>> patches of makita may solve the problem. jason you may send your
>>>>> patches, and I will do some research on busypoll.
>>>> I see. Maybe you can try some bi-directional traffic.
>>>>
>>>> Btw, lots of optimizations could be done for busy polling. E.g integrating
>>>> with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
>>>> to work or propose new ideas.
>>>>
>>>> Thanks
>>> It seems clear we do need adaptive polling.
>> Yes.
>>
>>> The difficulty with NAPI
>>> polling is it can't access guest memory easily. But maybe
>>> get_user_pages on the polled memory+NAPI polling can work.
>> You mean something like zerocopy? Looks like we can do busy polling without
>> it. I mean something like https://patchwork.kernel.org/patch/8707511/.
>>
>> Thanks
> How does this patch work? vhost_vq_avail_empty can sleep,
> you are calling it within an rcu read side critical section.
Ok, I get your meaning. I have patches to access vring through
get_user_pages + vmap() which should help here. (And it increase PPS
about 10%-20%).
>
> That's not the only problem btw, another one is that the
> CPU time spent polling isn't accounted with the VM.
Yes, but it's not the 'issue' of this patch. And I believe cgroup can help?
Thanks
>
>>>>>> Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Michael S. Tsirkin @ 2018-07-12 3:34 UTC (permalink / raw)
To: Jason Wang; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <0dcaf8f1-01b5-417d-420b-d5b716a82a8a@redhat.com>
On Thu, Jul 12, 2018 at 11:26:12AM +0800, Jason Wang wrote:
>
>
> On 2018年07月11日 19:59, Michael S. Tsirkin wrote:
> > On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
> > >
> > > On 2018年07月11日 11:49, Tonghao Zhang wrote:
> > > > On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
> > > > >
> > > > > On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
> > > > > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > > > > >
> > > > > > This patches improve the guest receive and transmit performance.
> > > > > > On the handle_tx side, we poll the sock receive queue at the same time.
> > > > > > handle_rx do that in the same way.
> > > > > >
> > > > > > For more performance report, see patch 4.
> > > > > >
> > > > > > v4 -> v5:
> > > > > > fix some issues
> > > > > >
> > > > > > v3 -> v4:
> > > > > > fix some issues
> > > > > >
> > > > > > v2 -> v3:
> > > > > > This patches are splited from previous big patch:
> > > > > > http://patchwork.ozlabs.org/patch/934673/
> > > > > >
> > > > > > Tonghao Zhang (4):
> > > > > > vhost: lock the vqs one by one
> > > > > > net: vhost: replace magic number of lock annotation
> > > > > > net: vhost: factor out busy polling logic to vhost_net_busy_poll()
> > > > > > net: vhost: add rx busy polling in tx path
> > > > > >
> > > > > > drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
> > > > > > drivers/vhost/vhost.c | 24 ++++-------
> > > > > > 2 files changed, 67 insertions(+), 65 deletions(-)
> > > > > >
> > > > > Hi, any progress on the new version?
> > > > >
> > > > > I plan to send a new series of packed virtqueue support of vhost. If you
> > > > > plan to send it soon, I can wait. Otherwise, I will send my series.
> > > > I rebase the codes. and find there is no improvement anymore, the
> > > > patches of makita may solve the problem. jason you may send your
> > > > patches, and I will do some research on busypoll.
> > > I see. Maybe you can try some bi-directional traffic.
> > >
> > > Btw, lots of optimizations could be done for busy polling. E.g integrating
> > > with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
> > > to work or propose new ideas.
> > >
> > > Thanks
> > It seems clear we do need adaptive polling.
>
> Yes.
>
> > The difficulty with NAPI
> > polling is it can't access guest memory easily. But maybe
> > get_user_pages on the polled memory+NAPI polling can work.
>
> You mean something like zerocopy? Looks like we can do busy polling without
> it. I mean something like https://patchwork.kernel.org/patch/8707511/.
>
> Thanks
How does this patch work? vhost_vq_avail_empty can sleep,
you are calling it within an rcu read side critical section.
That's not the only problem btw, another one is that the
CPU time spent polling isn't accounted with the VM.
> >
> > > > > Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Jason Wang @ 2018-07-12 3:26 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <20180711145421-mutt-send-email-mst@kernel.org>
On 2018年07月11日 19:59, Michael S. Tsirkin wrote:
> On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
>>
>> On 2018年07月11日 11:49, Tonghao Zhang wrote:
>>> On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
>>>>
>>>> On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
>>>>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>>>>
>>>>> This patches improve the guest receive and transmit performance.
>>>>> On the handle_tx side, we poll the sock receive queue at the same time.
>>>>> handle_rx do that in the same way.
>>>>>
>>>>> For more performance report, see patch 4.
>>>>>
>>>>> v4 -> v5:
>>>>> fix some issues
>>>>>
>>>>> v3 -> v4:
>>>>> fix some issues
>>>>>
>>>>> v2 -> v3:
>>>>> This patches are splited from previous big patch:
>>>>> http://patchwork.ozlabs.org/patch/934673/
>>>>>
>>>>> Tonghao Zhang (4):
>>>>> vhost: lock the vqs one by one
>>>>> net: vhost: replace magic number of lock annotation
>>>>> net: vhost: factor out busy polling logic to vhost_net_busy_poll()
>>>>> net: vhost: add rx busy polling in tx path
>>>>>
>>>>> drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
>>>>> drivers/vhost/vhost.c | 24 ++++-------
>>>>> 2 files changed, 67 insertions(+), 65 deletions(-)
>>>>>
>>>> Hi, any progress on the new version?
>>>>
>>>> I plan to send a new series of packed virtqueue support of vhost. If you
>>>> plan to send it soon, I can wait. Otherwise, I will send my series.
>>> I rebase the codes. and find there is no improvement anymore, the
>>> patches of makita may solve the problem. jason you may send your
>>> patches, and I will do some research on busypoll.
>> I see. Maybe you can try some bi-directional traffic.
>>
>> Btw, lots of optimizations could be done for busy polling. E.g integrating
>> with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
>> to work or propose new ideas.
>>
>> Thanks
> It seems clear we do need adaptive polling.
Yes.
> The difficulty with NAPI
> polling is it can't access guest memory easily. But maybe
> get_user_pages on the polled memory+NAPI polling can work.
You mean something like zerocopy? Looks like we can do busy polling
without it. I mean something like
https://patchwork.kernel.org/patch/8707511/.
Thanks
>
>>>> Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wei Wang @ 2018-07-12 2:52 UTC (permalink / raw)
To: Linus Torvalds
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, nilal, liliang.opensource,
Linux Kernel Mailing List, virtualization, linux-mm,
Paolo Bonzini, Andrew Morton, Michal Hocko
In-Reply-To: <CA+55aFxyv=EUAJFUSio=k+pm3ddteojshP7Radjia5ZRgm53zQ@mail.gmail.com>
On 07/12/2018 10:30 AM, Linus Torvalds wrote:
> On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
>> Would it be better to remove __GFP_THISNODE? We actually want to get all
>> the guest free pages (from all the nodes).
> Maybe. Or maybe it would be better to have the memory balloon logic be
> per-node? Maybe you don't want to remove too much memory from one
> node? I think it's one of those "play with it" things.
>
> I don't think that's the big issue, actually. I think the real issue
> is how to react quickly and gracefully to "oops, I'm trying to give
> memory away, but now the guest wants it back" while you're in the
> middle of trying to create that 2TB list of pages.
OK. virtio-balloon has already registered an oom notifier
(virtballoon_oom_notify). I plan to add some control there. If oom happens,
- stop the page allocation;
- immediately give back the allocated pages to mm.
Best,
Wei
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Linus Torvalds @ 2018-07-12 2:30 UTC (permalink / raw)
To: wei.w.wang
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, nilal, liliang.opensource,
Linux Kernel Mailing List, virtualization, linux-mm,
Paolo Bonzini, Andrew Morton, Michal Hocko
In-Reply-To: <5B46BB46.2080802@intel.com>
On Wed, Jul 11, 2018 at 7:17 PM Wei Wang <wei.w.wang@intel.com> wrote:
>
> Would it be better to remove __GFP_THISNODE? We actually want to get all
> the guest free pages (from all the nodes).
Maybe. Or maybe it would be better to have the memory balloon logic be
per-node? Maybe you don't want to remove too much memory from one
node? I think it's one of those "play with it" things.
I don't think that's the big issue, actually. I think the real issue
is how to react quickly and gracefully to "oops, I'm trying to give
memory away, but now the guest wants it back" while you're in the
middle of trying to create that 2TB list of pages.
IOW, I think the real work is in whatever tuning for the righ
tbehavior. But I'm just guessing.
Linus
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wei Wang @ 2018-07-12 2:21 UTC (permalink / raw)
To: Linus Torvalds, Michal Hocko
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal
In-Reply-To: <CA+55aFwku2tDH4+rfaC67xc4-cEwSrXgnQaci=e2id5ZCRE9JQ@mail.gmail.com>
On 07/12/2018 12:23 AM, Linus Torvalds wrote:
> On Wed, Jul 11, 2018 at 2:21 AM Michal Hocko <mhocko@kernel.org> wrote:
>> We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1).
>> So why do we need any array based interface?
> That was actually my original argument in the original thread - that
> the only new interface people might want is one that just tells how
> many of those MAX_ORDER-1 pages there are.
>
> See the thread in v33 with the subject
>
> "[PATCH v33 1/4] mm: add a function to get free page blocks"
>
> and look for me suggesting just using
>
> #define GFP_MINFLAGS (__GFP_NORETRY | __GFP_NOWARN |
> __GFP_THISNODE | __GFP_NOMEMALLOC)
Would it be better to remove __GFP_THISNODE? We actually want to get all
the guest free pages (from all the nodes).
Best,
Wei
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michael S. Tsirkin @ 2018-07-11 19:36 UTC (permalink / raw)
To: Michal Hocko
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
liliang.opensource, Linux Kernel Mailing List, virtualization,
linux-mm, Paolo Bonzini, Andrew Morton, nilal, Linus Torvalds
In-Reply-To: <20180711110949.GJ20050@dhcp22.suse.cz>
On Wed, Jul 11, 2018 at 01:09:49PM +0200, Michal Hocko wrote:
> But let me note that I am not really convinced how this (or previous)
> approach will really work in most workloads. We tend to cache heavily so
> there is rarely any memory free.
It might be that it's worth flushing the cache when VM is
migrating. Or maybe we should implement virtio-tmem or add
transcendent memory support to the balloon.
--
MST
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Linus Torvalds @ 2018-07-11 16:23 UTC (permalink / raw)
To: Michal Hocko
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal
In-Reply-To: <20180711092152.GE20050@dhcp22.suse.cz>
On Wed, Jul 11, 2018 at 2:21 AM Michal Hocko <mhocko@kernel.org> wrote:
>
> We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1).
> So why do we need any array based interface?
That was actually my original argument in the original thread - that
the only new interface people might want is one that just tells how
many of those MAX_ORDER-1 pages there are.
See the thread in v33 with the subject
"[PATCH v33 1/4] mm: add a function to get free page blocks"
and look for me suggesting just using
#define GFP_MINFLAGS (__GFP_NORETRY | __GFP_NOWARN |
__GFP_THISNODE | __GFP_NOMEMALLOC)
struct page *page = alloc_pages(GFP_MINFLAGS, MAX_ORDER-1);
for this all.
But I could also see an argument for "allocate N pages of size
MAX_ORDER-1", with some small N, simply because I can see the
advantage of not taking and releasing the locking and looking up the
zone individually N times.
If you want to get gigabytes of memory (or terabytes), doing it in
bigger chunks than one single maximum-sized page sounds fairly
reasonable.
I just don't think that "thousands of pages" is reasonable. But "tens
of max-sized pages" sounds fair enough to me, and it would certainly
not be a pain for the VM.
So I'm open to new interfaces. I just want those new interfaces to
make sense, and be low latency and simple for the VM to do. I'm
objecting to the incredibly baroque and heavy-weight one that can
return near-infinite amounts of memory.
The real advantage of jjuist the existing "alloc_pages()" model is
that I think the ballooning people can use that to *test* things out.
If it turns out that taking and releasing the VM locks is a big cost,
we can see if a batch interface that allows you to get tens of pages
at the same time is worth it.
So yes, I'd suggest starting with just the existing alloc_pages. Maybe
it's not enough, but it should be good enough for testing.
Linus
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-11 14:38 UTC (permalink / raw)
To: Wang, Wei W
Cc: yang.zhang.wz@gmail.com, virtio-dev@lists.oasis-open.org,
Rik van Riel, quan.xu0@gmail.com, KVM list, Michael S. Tsirkin,
liliang.opensource@gmail.com, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton,
nilal@redhat.com, Linus Torvalds
In-Reply-To: <286AC319A985734F985F78AFA26841F7396EEFD8@SHSMSX101.ccr.corp.intel.com>
On Wed 11-07-18 13:55:15, Wang, Wei W wrote:
> On Wednesday, July 11, 2018 7:10 PM, Michal Hocko wrote:
> > On Wed 11-07-18 18:52:45, Wei Wang wrote:
> > > On 07/11/2018 05:21 PM, Michal Hocko wrote:
> > > > On Tue 10-07-18 18:44:34, Linus Torvalds wrote:
> > > > [...]
> > > > > That was what I tried to encourage with actually removing the
> > > > > pages form the page list. That would be an _incremental_
> > > > > interface. You can remove MAX_ORDER-1 pages one by one (or a
> > > > > hundred at a time), and mark them free for ballooning that way.
> > > > > And if you still feel you have tons of free memory, just continue
> > removing more pages from the free list.
> > > > We already have an interface for that. alloc_pages(GFP_NOWAIT,
> > MAX_ORDER -1).
> > > > So why do we need any array based interface?
> > >
> > > Yes, I'm trying to get free pages directly via alloc_pages, so there
> > > will be no new mm APIs.
> >
> > OK. The above was just a rough example. In fact you would need a more
> > complex gfp mask. I assume you only want to balloon only memory directly
> > usable by the kernel so it will be
> > (GFP_KERNEL | __GFP_NOWARN) & ~__GFP_RECLAIM
>
> Sounds good to me, thanks.
>
> >
> > > I plan to let free page allocation stop when the remaining system free
> > > memory becomes close to min_free_kbytes (prevent swapping).
> >
> > ~__GFP_RECLAIM will make sure you are allocate as long as there is any
> > memory without reclaim. It will not even poke the kswapd to do the
> > background work. So I do not think you would need much more than that.
>
> "close to min_free_kbytes" - I meant when doing the allocations, we
> intentionally reserve some small amount of memory, e.g. 2 free page
> blocks of "MAX_ORDER - 1". So when other applications happen to do
> some allocation, they may easily get some from the reserved memory
> left on the free list. Without that reserved memory, other allocation
> may cause the system free memory below the WMARK[MIN], and kswapd
> would start to do swapping. This is actually just a small optimization
> to reduce the probability of causing swapping (nice to have, but not
> mandatary because we will allocate free page blocks one by one).
I really have hard time to follow you here. Nothing outside of the core
MM proper should play with watermarks.
> > But let me note that I am not really convinced how this (or previous)
> > approach will really work in most workloads. We tend to cache heavily so
> > there is rarely any memory free.
>
> With less free memory, the improvement becomes less, but should be
> nicer than no optimization. For example, the Linux build workload
> would cause 4~5 GB (out of 8GB) memory to be used as page cache at the
> final stage, there is still ~44% live migration time reduction.
But most systems will stay somewhere around the high watermark if there
is any page cache activity. Especially after a longer uptime.
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* RE: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wang, Wei W @ 2018-07-11 13:55 UTC (permalink / raw)
To: Michal Hocko
Cc: yang.zhang.wz@gmail.com, virtio-dev@lists.oasis-open.org,
Rik van Riel, quan.xu0@gmail.com, KVM list, Michael S. Tsirkin,
liliang.opensource@gmail.com, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton,
nilal@redhat.com, Linus Torvalds
In-Reply-To: <20180711110949.GJ20050@dhcp22.suse.cz>
On Wednesday, July 11, 2018 7:10 PM, Michal Hocko wrote:
> On Wed 11-07-18 18:52:45, Wei Wang wrote:
> > On 07/11/2018 05:21 PM, Michal Hocko wrote:
> > > On Tue 10-07-18 18:44:34, Linus Torvalds wrote:
> > > [...]
> > > > That was what I tried to encourage with actually removing the
> > > > pages form the page list. That would be an _incremental_
> > > > interface. You can remove MAX_ORDER-1 pages one by one (or a
> > > > hundred at a time), and mark them free for ballooning that way.
> > > > And if you still feel you have tons of free memory, just continue
> removing more pages from the free list.
> > > We already have an interface for that. alloc_pages(GFP_NOWAIT,
> MAX_ORDER -1).
> > > So why do we need any array based interface?
> >
> > Yes, I'm trying to get free pages directly via alloc_pages, so there
> > will be no new mm APIs.
>
> OK. The above was just a rough example. In fact you would need a more
> complex gfp mask. I assume you only want to balloon only memory directly
> usable by the kernel so it will be
> (GFP_KERNEL | __GFP_NOWARN) & ~__GFP_RECLAIM
Sounds good to me, thanks.
>
> > I plan to let free page allocation stop when the remaining system free
> > memory becomes close to min_free_kbytes (prevent swapping).
>
> ~__GFP_RECLAIM will make sure you are allocate as long as there is any
> memory without reclaim. It will not even poke the kswapd to do the
> background work. So I do not think you would need much more than that.
"close to min_free_kbytes" - I meant when doing the allocations, we intentionally reserve some small amount of memory, e.g. 2 free page blocks of "MAX_ORDER - 1". So when other applications happen to do some allocation, they may easily get some from the reserved memory left on the free list. Without that reserved memory, other allocation may cause the system free memory below the WMARK[MIN], and kswapd would start to do swapping. This is actually just a small optimization to reduce the probability of causing swapping (nice to have, but not mandatary because we will allocate free page blocks one by one).
> But let me note that I am not really convinced how this (or previous)
> approach will really work in most workloads. We tend to cache heavily so
> there is rarely any memory free.
With less free memory, the improvement becomes less, but should be nicer than no optimization. For example, the Linux build workload would cause 4~5 GB (out of 8GB) memory to be used as page cache at the final stage, there is still ~44% live migration time reduction.
Since we have many cloud customers interested in this feature, I think we can let them test the usefulness.
Best,
Wei
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Michael S. Tsirkin @ 2018-07-11 11:59 UTC (permalink / raw)
To: Jason Wang; +Cc: Linux Kernel Network Developers, virtualization
In-Reply-To: <4901593f-1d9e-2c47-b963-5fc1f48a1bfe@redhat.com>
On Wed, Jul 11, 2018 at 01:12:59PM +0800, Jason Wang wrote:
>
>
> On 2018年07月11日 11:49, Tonghao Zhang wrote:
> > On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
> > >
> > >
> > > On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
> > > > From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > > >
> > > > This patches improve the guest receive and transmit performance.
> > > > On the handle_tx side, we poll the sock receive queue at the same time.
> > > > handle_rx do that in the same way.
> > > >
> > > > For more performance report, see patch 4.
> > > >
> > > > v4 -> v5:
> > > > fix some issues
> > > >
> > > > v3 -> v4:
> > > > fix some issues
> > > >
> > > > v2 -> v3:
> > > > This patches are splited from previous big patch:
> > > > http://patchwork.ozlabs.org/patch/934673/
> > > >
> > > > Tonghao Zhang (4):
> > > > vhost: lock the vqs one by one
> > > > net: vhost: replace magic number of lock annotation
> > > > net: vhost: factor out busy polling logic to vhost_net_busy_poll()
> > > > net: vhost: add rx busy polling in tx path
> > > >
> > > > drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
> > > > drivers/vhost/vhost.c | 24 ++++-------
> > > > 2 files changed, 67 insertions(+), 65 deletions(-)
> > > >
> > > Hi, any progress on the new version?
> > >
> > > I plan to send a new series of packed virtqueue support of vhost. If you
> > > plan to send it soon, I can wait. Otherwise, I will send my series.
> > I rebase the codes. and find there is no improvement anymore, the
> > patches of makita may solve the problem. jason you may send your
> > patches, and I will do some research on busypoll.
>
> I see. Maybe you can try some bi-directional traffic.
>
> Btw, lots of optimizations could be done for busy polling. E.g integrating
> with host NAPI busy polling or a 100% busy polling vhost_net. You're welcome
> to work or propose new ideas.
>
> Thanks
It seems clear we do need adaptive polling. The difficulty with NAPI
polling is it can't access guest memory easily. But maybe
get_user_pages on the polled memory+NAPI polling can work.
> >
> > > Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-11 11:09 UTC (permalink / raw)
To: Wei Wang
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal,
Linus Torvalds
In-Reply-To: <5B45E17D.2090205@intel.com>
On Wed 11-07-18 18:52:45, Wei Wang wrote:
> On 07/11/2018 05:21 PM, Michal Hocko wrote:
> > On Tue 10-07-18 18:44:34, Linus Torvalds wrote:
> > [...]
> > > That was what I tried to encourage with actually removing the pages
> > > form the page list. That would be an _incremental_ interface. You can
> > > remove MAX_ORDER-1 pages one by one (or a hundred at a time), and mark
> > > them free for ballooning that way. And if you still feel you have tons
> > > of free memory, just continue removing more pages from the free list.
> > We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1).
> > So why do we need any array based interface?
>
> Yes, I'm trying to get free pages directly via alloc_pages, so there will be
> no new mm APIs.
OK. The above was just a rough example. In fact you would need a more
complex gfp mask. I assume you only want to balloon only memory directly
usable by the kernel so it will be
(GFP_KERNEL | __GFP_NOWARN) & ~__GFP_RECLAIM
> I plan to let free page allocation stop when the remaining system free
> memory becomes close to min_free_kbytes (prevent swapping).
~__GFP_RECLAIM will make sure you are allocate as long as there is any
memory without reclaim. It will not even poke the kswapd to do the
background work. So I do not think you would need much more than that.
But let me note that I am not really convinced how this (or previous)
approach will really work in most workloads. We tend to cache heavily so
there is rarely any memory free.
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Wei Wang @ 2018-07-11 10:52 UTC (permalink / raw)
To: Michal Hocko, Linus Torvalds
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal
In-Reply-To: <20180711092152.GE20050@dhcp22.suse.cz>
On 07/11/2018 05:21 PM, Michal Hocko wrote:
> On Tue 10-07-18 18:44:34, Linus Torvalds wrote:
> [...]
>> That was what I tried to encourage with actually removing the pages
>> form the page list. That would be an _incremental_ interface. You can
>> remove MAX_ORDER-1 pages one by one (or a hundred at a time), and mark
>> them free for ballooning that way. And if you still feel you have tons
>> of free memory, just continue removing more pages from the free list.
> We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1).
> So why do we need any array based interface?
Yes, I'm trying to get free pages directly via alloc_pages, so there
will be no new mm APIs.
I plan to let free page allocation stop when the remaining system free
memory becomes close to min_free_kbytes (prevent swapping).
Best,
Wei
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michal Hocko @ 2018-07-11 9:21 UTC (permalink / raw)
To: Linus Torvalds
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
Michael S. Tsirkin, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton, nilal
In-Reply-To: <CA+55aFzqj8wxXnHAdUTiOomipgFONVbqKMjL_tfk7e5ar1FziQ@mail.gmail.com>
On Tue 10-07-18 18:44:34, Linus Torvalds wrote:
[...]
> That was what I tried to encourage with actually removing the pages
> form the page list. That would be an _incremental_ interface. You can
> remove MAX_ORDER-1 pages one by one (or a hundred at a time), and mark
> them free for ballooning that way. And if you still feel you have tons
> of free memory, just continue removing more pages from the free list.
We already have an interface for that. alloc_pages(GFP_NOWAIT, MAX_ORDER -1).
So why do we need any array based interface?
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH net-next v5 0/4] net: vhost: improve performance when enable busyloop
From: Jason Wang @ 2018-07-11 5:12 UTC (permalink / raw)
To: Tonghao Zhang; +Cc: Linux Kernel Network Developers, virtualization, mst
In-Reply-To: <CAMDZJNWqF95P1YBiu5X+gBbBOp2EVe380D2kzNmdes_4hBH3dw@mail.gmail.com>
On 2018年07月11日 11:49, Tonghao Zhang wrote:
> On Wed, Jul 11, 2018 at 10:56 AM Jason Wang <jasowang@redhat.com> wrote:
>>
>>
>> On 2018年07月04日 12:31, xiangxia.m.yue@gmail.com wrote:
>>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>>>
>>> This patches improve the guest receive and transmit performance.
>>> On the handle_tx side, we poll the sock receive queue at the same time.
>>> handle_rx do that in the same way.
>>>
>>> For more performance report, see patch 4.
>>>
>>> v4 -> v5:
>>> fix some issues
>>>
>>> v3 -> v4:
>>> fix some issues
>>>
>>> v2 -> v3:
>>> This patches are splited from previous big patch:
>>> http://patchwork.ozlabs.org/patch/934673/
>>>
>>> Tonghao Zhang (4):
>>> vhost: lock the vqs one by one
>>> net: vhost: replace magic number of lock annotation
>>> net: vhost: factor out busy polling logic to vhost_net_busy_poll()
>>> net: vhost: add rx busy polling in tx path
>>>
>>> drivers/vhost/net.c | 108 ++++++++++++++++++++++++++++----------------------
>>> drivers/vhost/vhost.c | 24 ++++-------
>>> 2 files changed, 67 insertions(+), 65 deletions(-)
>>>
>> Hi, any progress on the new version?
>>
>> I plan to send a new series of packed virtqueue support of vhost. If you
>> plan to send it soon, I can wait. Otherwise, I will send my series.
> I rebase the codes. and find there is no improvement anymore, the
> patches of makita may solve the problem. jason you may send your
> patches, and I will do some research on busypoll.
I see. Maybe you can try some bi-directional traffic.
Btw, lots of optimizations could be done for busy polling. E.g
integrating with host NAPI busy polling or a 100% busy polling
vhost_net. You're welcome to work or propose new ideas.
Thanks
>
>> Thanks
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH v35 1/5] mm: support to get hints of free page blocks
From: Michael S. Tsirkin @ 2018-07-11 4:04 UTC (permalink / raw)
To: Linus Torvalds
Cc: yang.zhang.wz, virtio-dev, Rik van Riel, quan.xu0, KVM list,
nilal, liliang.opensource, Linux Kernel Mailing List,
virtualization, linux-mm, Paolo Bonzini, Andrew Morton,
Michal Hocko
In-Reply-To: <20180711064709-mutt-send-email-mst@kernel.org>
On Wed, Jul 11, 2018 at 07:00:37AM +0300, Michael S. Tsirkin wrote:
> On Tue, Jul 10, 2018 at 10:33:08AM -0700, Linus Torvalds wrote:
> > NAK.
> >
> > On Tue, Jul 10, 2018 at 2:56 AM Wei Wang <wei.w.wang@intel.com> wrote:
> > >
> > > +
> > > + buf_page = list_first_entry_or_null(pages, struct page, lru);
> > > + if (!buf_page)
> > > + return -EINVAL;
> > > + buf = (__le64 *)page_address(buf_page);
> >
> > Stop this garbage.
> >
> > Why the hell would you pass in some crazy "liost of pages" that uses
> > that lru list?
> >
> > That's just insane shit.
> >
> > Just pass in a an array to fill in.
> > No idiotic games like this with
> > odd list entries (what's the locking?) and crazy casting to
> >
> > So if you want an array of page addresses, pass that in as such. If
> > you want to do it in a page, do it with
> >
> > u64 *array = page_address(page);
> > int nr = PAGE_SIZE / sizeof(u64);
> >
> > and now you pass that array in to the thing. None of this completely
> > insane crazy crap interfaces.
>
> Question was raised what to do if there are so many free
> MAX_ORDER pages that their addresses don't fit in a single MAX_ORDER
> page.
Oh you answered already, I spoke too soon. Nevermind, pls ignore me.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox