* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Michael S. Tsirkin @ 2018-01-26 22:14 UTC (permalink / raw)
To: Siwei Liu
Cc: Alexander Duyck, virtio-dev, Jakub Kicinski, Samudrala, Sridhar,
virtualization, Netdev, David Miller
In-Reply-To: <CADGSJ205yeyGV2nJ9Cr=FwPypxAHb12jx3WitS6UmWSJ9VP42Q@mail.gmail.com>
On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
> > and the VM is not expected to do any tuning/optimizations on the VF driver
> > directly,
> > i think the current patch that follows the netvsc model of 2 netdevs(virtio
> > and vf) should
> > work fine.
>
> OK. For your use case that's fine. But that's too specific scenario
> with lots of restrictions IMHO, perhaps very few users will benefit
> from it, I'm not sure. If you're unwilling to move towards it, we'd
> take this one and come back with a generic solution that is able to
> address general use cases for VF/PT live migration .
I think that's a fine approach. Scratch your own itch! I imagine a very
generic virtio-switchdev providing host routing info to guests could
address lots of usecases. A driver could bind to that one and enslave
arbitrary other devices. Sounds reasonable.
But given the fundamental idea of a failover was floated at least as
early as 2013, and made 0 progress since precisely because it kept
trying to address more and more features, and given netvsc is already
using the basic solution with some success, I'm not inclined to block
this specific effort waiting for the generic one.
--
MST
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Jakub Kicinski @ 2018-01-26 22:47 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Alexander Duyck, virtio-dev, Netdev, virtualization, Siwei Liu,
Samudrala, Sridhar, David Miller
In-Reply-To: <20180127000831-mutt-send-email-mst@kernel.org>
On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
> > > and the VM is not expected to do any tuning/optimizations on the VF driver
> > > directly,
> > > i think the current patch that follows the netvsc model of 2 netdevs(virtio
> > > and vf) should
> > > work fine.
> >
> > OK. For your use case that's fine. But that's too specific scenario
> > with lots of restrictions IMHO, perhaps very few users will benefit
> > from it, I'm not sure. If you're unwilling to move towards it, we'd
> > take this one and come back with a generic solution that is able to
> > address general use cases for VF/PT live migration .
>
> I think that's a fine approach. Scratch your own itch! I imagine a very
> generic virtio-switchdev providing host routing info to guests could
> address lots of usecases. A driver could bind to that one and enslave
> arbitrary other devices. Sounds reasonable.
>
> But given the fundamental idea of a failover was floated at least as
> early as 2013, and made 0 progress since precisely because it kept
> trying to address more and more features, and given netvsc is already
> using the basic solution with some success, I'm not inclined to block
> this specific effort waiting for the generic one.
I think there is an agreement that the extra netdev will be useful for
more advanced use cases, and is generally preferable. What is the
argument for not doing that from the start? If it was made I must have
missed it. Is it just unwillingness to write the extra 300 lines of
code? Sounds like a pretty weak argument when adding kernel ABI is at
stake...
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Samudrala, Sridhar @ 2018-01-26 23:30 UTC (permalink / raw)
To: Jakub Kicinski, Michael S. Tsirkin
Cc: Alexander Duyck, virtio-dev, Netdev, virtualization, Siwei Liu,
David Miller
In-Reply-To: <20180126144704.6e1a9628@cakuba.netronome.com>
On 1/26/2018 2:47 PM, Jakub Kicinski wrote:
> On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
>> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
>>>> and the VM is not expected to do any tuning/optimizations on the VF driver
>>>> directly,
>>>> i think the current patch that follows the netvsc model of 2 netdevs(virtio
>>>> and vf) should
>>>> work fine.
>>> OK. For your use case that's fine. But that's too specific scenario
>>> with lots of restrictions IMHO, perhaps very few users will benefit
>>> from it, I'm not sure. If you're unwilling to move towards it, we'd
>>> take this one and come back with a generic solution that is able to
>>> address general use cases for VF/PT live migration .
>> I think that's a fine approach. Scratch your own itch! I imagine a very
>> generic virtio-switchdev providing host routing info to guests could
>> address lots of usecases. A driver could bind to that one and enslave
>> arbitrary other devices. Sounds reasonable.
>>
>> But given the fundamental idea of a failover was floated at least as
>> early as 2013, and made 0 progress since precisely because it kept
>> trying to address more and more features, and given netvsc is already
>> using the basic solution with some success, I'm not inclined to block
>> this specific effort waiting for the generic one.
> I think there is an agreement that the extra netdev will be useful for
> more advanced use cases, and is generally preferable. What is the
> argument for not doing that from the start? If it was made I must have
> missed it. Is it just unwillingness to write the extra 300 lines of
> code? Sounds like a pretty weak argument when adding kernel ABI is at
> stake...
I am still not clear on the need for the extra netdev created by
virtio_net. The only advantage
i can see is that the stats can be broken between VF and virtio
datapaths compared
to the aggregrated stats on virtio netdev as seen with the 2 netdev
approach.
With 2 netdev model, any VM image that has a working network
configuration will transparently get
VF based acceleration without any changes. 3 netdev model breaks this
configuration starting with the
creation and naming of the 2 devices to udev needing to be aware of
master and slave virtio-net devices.
Also, from a user experience point of view, loading a virtio-net with
BACKUP feature
enabled will now show 2 virtio-net netdevs.
For live migration with advanced usecases that Siwei is suggesting, i
think we need a new driver
with a new device type that can track the VF specific feature settings
even when the VF driver is unloaded.
Thanks
Sridhar
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Jakub Kicinski @ 2018-01-27 2:30 UTC (permalink / raw)
To: Samudrala, Sridhar
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Netdev,
virtualization, Siwei Liu, David Miller
In-Reply-To: <5fa8c6c6-4a94-91fa-fdbb-ee7b624d703f@intel.com>
On Fri, 26 Jan 2018 15:30:35 -0800, Samudrala, Sridhar wrote:
> On 1/26/2018 2:47 PM, Jakub Kicinski wrote:
> > On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
> >> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
> >>>> and the VM is not expected to do any tuning/optimizations on the VF driver
> >>>> directly,
> >>>> i think the current patch that follows the netvsc model of 2 netdevs(virtio
> >>>> and vf) should
> >>>> work fine.
> >>> OK. For your use case that's fine. But that's too specific scenario
> >>> with lots of restrictions IMHO, perhaps very few users will benefit
> >>> from it, I'm not sure. If you're unwilling to move towards it, we'd
> >>> take this one and come back with a generic solution that is able to
> >>> address general use cases for VF/PT live migration .
> >> I think that's a fine approach. Scratch your own itch! I imagine a very
> >> generic virtio-switchdev providing host routing info to guests could
> >> address lots of usecases. A driver could bind to that one and enslave
> >> arbitrary other devices. Sounds reasonable.
> >>
> >> But given the fundamental idea of a failover was floated at least as
> >> early as 2013, and made 0 progress since precisely because it kept
> >> trying to address more and more features, and given netvsc is already
> >> using the basic solution with some success, I'm not inclined to block
> >> this specific effort waiting for the generic one.
> > I think there is an agreement that the extra netdev will be useful for
> > more advanced use cases, and is generally preferable. What is the
> > argument for not doing that from the start? If it was made I must have
> > missed it. Is it just unwillingness to write the extra 300 lines of
> > code? Sounds like a pretty weak argument when adding kernel ABI is at
> > stake...
>
> I am still not clear on the need for the extra netdev created by
> virtio_net. The only advantage i can see is that the stats can be
> broken between VF and virtio datapaths compared to the aggregrated
> stats on virtio netdev as seen with the 2 netdev approach.
Maybe you're not convinced but multiple arguments were made.
> With 2 netdev model, any VM image that has a working network
> configuration will transparently get VF based acceleration without
> any changes.
Nothing happens transparently. Things may happen automatically. The
VF netdev doesn't disappear with netvsc. The PV netdev transforms into
something it did not use to be. And configures and reports some
information from the PV (e.g. speed) but PV doesn't pass traffic any
longer.
> 3 netdev model breaks this configuration starting with the creation
> and naming of the 2 devices to udev needing to be aware of master and
> slave virtio-net devices.
I don't understand this comment. There is one virtio-net device and
one "virtio-bond" netdev. And user space has to be aware of the special
automatic arrangement anyway, because it can't touch the VF. It
doesn't make any difference whether it ignores the VF or PV and VF.
It simply can't touch the slaves, no matter how many there are.
> Also, from a user experience point of view, loading a virtio-net with
> BACKUP feature enabled will now show 2 virtio-net netdevs.
One virtio-net and one virtio-bond, which represents what's happening.
> For live migration with advanced usecases that Siwei is suggesting, i
> think we need a new driver with a new device type that can track the
> VF specific feature settings even when the VF driver is unloaded.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Samudrala, Sridhar @ 2018-01-27 5:33 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Netdev,
virtualization, Siwei Liu, David Miller
In-Reply-To: <20180126183003.591cd5c5@cakuba.netronome.com>
On 1/26/2018 6:30 PM, Jakub Kicinski wrote:
> On Fri, 26 Jan 2018 15:30:35 -0800, Samudrala, Sridhar wrote:
>> On 1/26/2018 2:47 PM, Jakub Kicinski wrote:
>>> On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
>>>> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
>>>>>> and the VM is not expected to do any tuning/optimizations on the VF driver
>>>>>> directly,
>>>>>> i think the current patch that follows the netvsc model of 2 netdevs(virtio
>>>>>> and vf) should
>>>>>> work fine.
>>>>> OK. For your use case that's fine. But that's too specific scenario
>>>>> with lots of restrictions IMHO, perhaps very few users will benefit
>>>>> from it, I'm not sure. If you're unwilling to move towards it, we'd
>>>>> take this one and come back with a generic solution that is able to
>>>>> address general use cases for VF/PT live migration .
>>>> I think that's a fine approach. Scratch your own itch! I imagine a very
>>>> generic virtio-switchdev providing host routing info to guests could
>>>> address lots of usecases. A driver could bind to that one and enslave
>>>> arbitrary other devices. Sounds reasonable.
>>>>
>>>> But given the fundamental idea of a failover was floated at least as
>>>> early as 2013, and made 0 progress since precisely because it kept
>>>> trying to address more and more features, and given netvsc is already
>>>> using the basic solution with some success, I'm not inclined to block
>>>> this specific effort waiting for the generic one.
>>> I think there is an agreement that the extra netdev will be useful for
>>> more advanced use cases, and is generally preferable. What is the
>>> argument for not doing that from the start? If it was made I must have
>>> missed it. Is it just unwillingness to write the extra 300 lines of
>>> code? Sounds like a pretty weak argument when adding kernel ABI is at
>>> stake...
>> I am still not clear on the need for the extra netdev created by
>> virtio_net. The only advantage i can see is that the stats can be
>> broken between VF and virtio datapaths compared to the aggregrated
>> stats on virtio netdev as seen with the 2 netdev approach.
> Maybe you're not convinced but multiple arguments were made.
All the arguments seem to either saying that semantically this doesn't
look like
a bond OR suggesting usecases that this patch is not trying to solve.
This approach should help cloud environments where the guest networking
is fully
controlled from the hypervisor via the PF driver or via port representor
when switchdev
mode is enabled. The guest admin is not expected or allowed to make any
networking
changes from the VM.
>
>> With 2 netdev model, any VM image that has a working network
>> configuration will transparently get VF based acceleration without
>> any changes.
> Nothing happens transparently. Things may happen automatically. The
> VF netdev doesn't disappear with netvsc. The PV netdev transforms into
> something it did not use to be. And configures and reports some
> information from the PV (e.g. speed) but PV doesn't pass traffic any
> longer.
>> 3 netdev model breaks this configuration starting with the creation
>> and naming of the 2 devices to udev needing to be aware of master and
>> slave virtio-net devices.
> I don't understand this comment. There is one virtio-net device and
> one "virtio-bond" netdev. And user space has to be aware of the special
> automatic arrangement anyway, because it can't touch the VF. It
> doesn't make any difference whether it ignores the VF or PV and VF.
> It simply can't touch the slaves, no matter how many there are.
If the userspace is not expected to touch the slaves, then why do we need to
take extra effort to expose a netdev that is just not really useful.
>
>> Also, from a user experience point of view, loading a virtio-net with
>> BACKUP feature enabled will now show 2 virtio-net netdevs.
> One virtio-net and one virtio-bond, which represents what's happening.
This again assumes that we want to represent a bond setup. Can't we
treat this
as virtio-net providing an alternate low-latency datapath by taking over
VF datapath?
>
>> For live migration with advanced usecases that Siwei is suggesting, i
>> think we need a new driver with a new device type that can track the
>> VF specific feature settings even when the VF driver is unloaded.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Jakub Kicinski @ 2018-01-27 5:58 UTC (permalink / raw)
To: Samudrala, Sridhar
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Netdev,
virtualization, Siwei Liu, David Miller
In-Reply-To: <e02e21a1-ab5d-132a-fb33-5e0ba42fc9ac@intel.com>
On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
> >> 3 netdev model breaks this configuration starting with the creation
> >> and naming of the 2 devices to udev needing to be aware of master and
> >> slave virtio-net devices.
> > I don't understand this comment. There is one virtio-net device and
> > one "virtio-bond" netdev. And user space has to be aware of the special
> > automatic arrangement anyway, because it can't touch the VF. It
> > doesn't make any difference whether it ignores the VF or PV and VF.
> > It simply can't touch the slaves, no matter how many there are.
>
> If the userspace is not expected to touch the slaves, then why do we need to
> take extra effort to expose a netdev that is just not really useful.
You said:
"[user space] needs to be aware of master and slave virtio-net devices."
I'm saying:
It has to be aware of the special arrangement whether there is an
explicit bond netdev or not.
> >> Also, from a user experience point of view, loading a virtio-net with
> >> BACKUP feature enabled will now show 2 virtio-net netdevs.
> > One virtio-net and one virtio-bond, which represents what's happening.
> This again assumes that we want to represent a bond setup. Can't we
> treat this
> as virtio-net providing an alternate low-latency datapath by taking over
> VF datapath?
Bond is just a familiar name, we can call it something else if you
prefer. The point is there are two data paths which can have
independent low-level settings and a higher level entity with
global settings which represents any path to the outside world.
Hiding low-level netdevs from a lay user requires a generic solution.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* RE: [PATCH v24 1/2] mm: support reporting free page blocks
From: Wang, Wei W @ 2018-01-27 13:13 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: yang.zhang.wz@gmail.com, virtio-dev@lists.oasis-open.org,
riel@redhat.com, quan.xu0@gmail.com, kvm@vger.kernel.org,
nilal@redhat.com, liliang.opensource@gmail.com,
linux-kernel@vger.kernel.org, mhocko@kernel.org,
linux-mm@kvack.org, pbonzini@redhat.com,
akpm@linux-foundation.org,
virtualization@lists.linux-foundation.org
In-Reply-To: <20180126233950-mutt-send-email-mst@kernel.org>
On Saturday, January 27, 2018 5:44 AM, Michael S. Tsirkin wrote:
> On Fri, Jan 26, 2018 at 05:00:09PM +0200, Michael S. Tsirkin wrote:
> > On Fri, Jan 26, 2018 at 11:29:15AM +0800, Wei Wang wrote:
> > > On 01/25/2018 09:41 PM, Michael S. Tsirkin wrote:
> > > > On Wed, Jan 24, 2018 at 06:42:41PM +0800, Wei Wang wrote:
> > > 2) If worse, all the blocks have been split into smaller blocks and
> > > used after the caller comes back.
> > >
> > > where could we continue?
> >
> > I'm not sure. But an alternative appears to be to hold a lock and just
> > block whoever wanted to use any pages. Yes we are sending hints
> > faster but apparently something wanted these pages, and holding the
> > lock is interfering with this something.
>
> I've been thinking about it. How about the following scheme:
> 1. register balloon to get a (new) callback when free list runs empty 2. take
> pages off the free list, add them to the balloon specific list 3. report to host 4.
> readd to free list at tail 5. if callback triggers, interrupt balloon reporting to
> host,
> and readd to free list at tail
So in step 2, when we walk through the free page list, take each block, and add them to the balloon specific list, is this performed under the mm lock? If it is still under the lock, then what would be the difference compared to walking through the free list, and add each block to virtqueue?
Best,
Wei
^ permalink raw reply
* RE: [PATCH v24 1/2] mm: support reporting free page blocks
From: Wang, Wei W @ 2018-01-27 14:00 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: yang.zhang.wz@gmail.com, virtio-dev@lists.oasis-open.org,
riel@redhat.com, quan.xu0@gmail.com, kvm@vger.kernel.org,
nilal@redhat.com, liliang.opensource@gmail.com,
linux-kernel@vger.kernel.org, mhocko@kernel.org,
linux-mm@kvack.org, pbonzini@redhat.com,
akpm@linux-foundation.org,
virtualization@lists.linux-foundation.org
In-Reply-To: <20180126155224-mutt-send-email-mst@kernel.org>
On Friday, January 26, 2018 11:00 PM, Michael S. Tsirkin wrote:
> On Fri, Jan 26, 2018 at 11:29:15AM +0800, Wei Wang wrote:
> > On 01/25/2018 09:41 PM, Michael S. Tsirkin wrote:
> > > On Wed, Jan 24, 2018 at 06:42:41PM +0800, Wei Wang wrote:
> > > > This patch adds support to walk through the free page blocks in
> > > > the system and report them via a callback function. Some page
> > > > blocks may leave the free list after zone->lock is released, so it
> > > > is the caller's responsibility to either detect or prevent the use of such
> pages.
> > > >
> > > > One use example of this patch is to accelerate live migration by
> > > > skipping the transfer of free pages reported from the guest. A
> > > > popular method used by the hypervisor to track which part of
> > > > memory is written during live migration is to write-protect all
> > > > the guest memory. So, those pages that are reported as free pages
> > > > but are written after the report function returns will be captured
> > > > by the hypervisor, and they will be added to the next round of memory
> transfer.
> > > >
> > > > Signed-off-by: Wei Wang <wei.w.wang@intel.com>
> > > > Signed-off-by: Liang Li <liang.z.li@intel.com>
> > > > Cc: Michal Hocko <mhocko@kernel.org>
> > > > Cc: Michael S. Tsirkin <mst@redhat.com>
> > > > Acked-by: Michal Hocko <mhocko@kernel.org>
> > > > ---
> > > > include/linux/mm.h | 6 ++++
> > > > mm/page_alloc.c | 91
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > > 2 files changed, 97 insertions(+)
> > > >
> > > > diff --git a/include/linux/mm.h b/include/linux/mm.h index
> > > > ea818ff..b3077dd 100644
> > > > --- a/include/linux/mm.h
> > > > +++ b/include/linux/mm.h
> > > > @@ -1938,6 +1938,12 @@ extern void free_area_init_node(int nid,
> unsigned long * zones_size,
> > > > unsigned long zone_start_pfn, unsigned long *zholes_size);
> > > > extern void free_initmem(void);
> > > > +extern void walk_free_mem_block(void *opaque,
> > > > + int min_order,
> > > > + bool (*report_pfn_range)(void *opaque,
> > > > + unsigned long pfn,
> > > > + unsigned long num));
> > > > +
> > > > /*
> > > > * Free reserved pages within range [PAGE_ALIGN(start), end &
> PAGE_MASK)
> > > > * into the buddy system. The freed pages will be poisoned with
> > > > pattern diff --git a/mm/page_alloc.c b/mm/page_alloc.c index
> > > > 76c9688..705de22 100644
> > > > --- a/mm/page_alloc.c
> > > > +++ b/mm/page_alloc.c
> > > > @@ -4899,6 +4899,97 @@ void show_free_areas(unsigned int filter,
> nodemask_t *nodemask)
> > > > show_swap_cache_info();
> > > > }
> > > > +/*
> > > > + * Walk through a free page list and report the found pfn range
> > > > +via the
> > > > + * callback.
> > > > + *
> > > > + * Return false if the callback requests to stop reporting.
> > > > +Otherwise,
> > > > + * return true.
> > > > + */
> > > > +static bool walk_free_page_list(void *opaque,
> > > > + struct zone *zone,
> > > > + int order,
> > > > + enum migratetype mt,
> > > > + bool (*report_pfn_range)(void *,
> > > > + unsigned long,
> > > > + unsigned long))
> > > > +{
> > > > + struct page *page;
> > > > + struct list_head *list;
> > > > + unsigned long pfn, flags;
> > > > + bool ret;
> > > > +
> > > > + spin_lock_irqsave(&zone->lock, flags);
> > > > + list = &zone->free_area[order].free_list[mt];
> > > > + list_for_each_entry(page, list, lru) {
> > > > + pfn = page_to_pfn(page);
> > > > + ret = report_pfn_range(opaque, pfn, 1 << order);
> > > > + if (!ret)
> > > > + break;
> > > > + }
> > > > + spin_unlock_irqrestore(&zone->lock, flags);
> > > > +
> > > > + return ret;
> > > > +}
> > > There are two issues with this API. One is that it is not
> > > restarteable: if you return false, you start from the beginning. So
> > > no way to drop lock, do something slow and then proceed.
> > >
> > > Another is that you are using it to report free page hints.
> > > Presumably the point is to drop these pages - keeping them near head
> > > of the list and reusing the reported ones will just make everything
> > > slower invalidating the hint.
> > >
> > > How about rotating these pages towards the end of the list?
> > > Probably not on each call, callect reported pages and then move them
> > > to tail when we exit.
> >
> >
> > I'm not sure how this would help. For example, we have a list of 2M
> > free page blocks:
> > A-->B-->C-->D-->E-->F-->G--H
> >
> > After reporting A and B, and put them to the end and exit, when the
> > caller comes back,
> > 1) if the list remains unchanged, then it will be
> > C-->D-->E-->F-->G-->H-->A-->B
>
> Right. So here we can just scan until we see A, right? It's a harder question
> what to do if A and only A has been consumed. We don't want B to be sent
> twice ideally. OTOH maybe that isn't a big deal if it's only twice. Host might
> know page is already gone - how about host gives us a hint after using the
> buffer?
>
> > 2) If worse, all the blocks have been split into smaller blocks and
> > used after the caller comes back.
> >
> > where could we continue?
>
> I'm not sure. But an alternative appears to be to hold a lock and just block
> whoever wanted to use any pages. Yes we are sending hints faster but
> apparently something wanted these pages, and holding the lock is interfering
> with this something.
>
> >
> > The reason to think about "restart" is the worry about the virtqueue
> > is full, right? But we've agreed that losing some hints to report
> > isn't important, and in practice, the virtqueue won't be full as the
> > host side is faster.
>
> It would be more convincing if we sent e.g. higher order pages first. As it is - it
> won't take long to stuff ring full of 4K pages and it seems highly unlikely that
> host won't ever be scheduled out.
Yes, actually we've already sent higher order pages first, please check this patch, we have:
for (order = MAX_ORDER - 1; order >= min_order; order--)
--> start from high order blocks.
>
> Can we maybe agree on what kind of benchmark makes sense for this work?
> I'm concerned that we are laser focused on just how long does it take to
> migrate ignoring e.g. slowdowns after migration.
Testing the time of linux compilation during migration? or what benchmark do you have in mind?
We can compare how long it takes during legacy live migration and live migration with this feature?
If you really worry about this, we could also provide an configure option, e.g. under /sys/, for the guest to decide whether to enable or disable reporting free page hints to the host at any time. If disabled, in the balloon driver we skip the calling of walk_free_mem_block().
> > I'm concerned that actions on the free list may cause more controversy
> > though it might be safe to do from some aspect, and would be hard to
> > end debating. If possible, we could go with the most prudent approach
> > for now, and have more discussions in future improvement patches. What
> > would you think?
>
> Well I'm not 100% about restartability. But keeping pages freed by host near
> head of the list looks kind of wrong.
> Try to float a patch on top for the rotation and see what happens?
I didn't get it, "pages freed by host", what does that mean?
Best,
Wei
^ permalink raw reply
* Re: Ping Re: [PATCH] virtio: make VIRTIO a menuconfig to ease disabling it all
From: Michael Ellerman @ 2018-01-28 10:14 UTC (permalink / raw)
To: Michael S. Tsirkin, Jason Wang, Vincent Legoll, sfr
Cc: Randy Dunlap, linux-kernel, virtualization
In-Reply-To: <CAEwRq=oJ0apphUFDheoX5d-BB5yjG3rZD8m4yhK75eZcm6eqVg@mail.gmail.com>
Vincent Legoll <vincent.legoll@gmail.com> writes:
> On 1/23/18, Michael Ellerman <mpe@ellerman.id.au> wrote:
>> This has been broken in linux-next for ~6 weeks now, can we please merge
>> this and get it fixed.
>
> Added Stephen Rothwell to cc
It should be fixed in the virtio tree, which the other Michael and Jason
maintain AFAIK.
cheers
^ permalink raw reply
* Re: [PATCH 4/6] x86: Consolidate PCI_MMCONFIG configs
From: Andy Shevchenko @ 2018-01-28 17:26 UTC (permalink / raw)
To: Jan Kiszka
Cc: jailhouse-dev, linux-pci, x86, Linux Kernel Mailing List,
virtualization, Ingo Molnar, H . Peter Anvin, Bjorn Helgaas,
Thomas Gleixner
In-Reply-To: <5dcaec50bb59140c1cb63c35ce566ccc02188bcb.1516601570.git.jan.kiszka@siemens.com>
On Mon, Jan 22, 2018 at 8:12 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Not sure if those two worked by design or just by chance so far. In any
> case, it's at least cleaner and clearer to express this in a single
> config statement.
Congrats! You found by the way a bug in
commit e279b6c1d329e50b766bce96aacc197eae8a053b
Author: Sam Ravnborg <sam@ravnborg.org>
Date: Tue Nov 6 20:41:05 2007 +0100
x86: start unification of arch/x86/Kconfig.*
...and proper fix seems to split PCI stuff to common + X86_32 only + X86_64 only
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Alexander Duyck @ 2018-01-28 17:35 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin,
Samudrala, Sridhar, virtualization, Siwei Liu, Netdev,
David Miller
In-Reply-To: <20180126215829.7b3c6bac@cakuba.netronome.com>
On Fri, Jan 26, 2018 at 9:58 PM, Jakub Kicinski <kubakici@wp.pl> wrote:
> On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
>> >> 3 netdev model breaks this configuration starting with the creation
>> >> and naming of the 2 devices to udev needing to be aware of master and
>> >> slave virtio-net devices.
>> > I don't understand this comment. There is one virtio-net device and
>> > one "virtio-bond" netdev. And user space has to be aware of the special
>> > automatic arrangement anyway, because it can't touch the VF. It
>> > doesn't make any difference whether it ignores the VF or PV and VF.
>> > It simply can't touch the slaves, no matter how many there are.
>>
>> If the userspace is not expected to touch the slaves, then why do we need to
>> take extra effort to expose a netdev that is just not really useful.
>
> You said:
> "[user space] needs to be aware of master and slave virtio-net devices."
>
> I'm saying:
> It has to be aware of the special arrangement whether there is an
> explicit bond netdev or not.
To clarify here the kernel should be aware that there is a device that
is an aggregate of 2 other devices. It isn't as if we need to insert
the new device "above" the virtio.
I have been doing a bit of messing around with a few ideas and it
seems like it would be better if we could replace the virtio interface
with the virtio-bond, renaming my virt-bond concept to this since it
is now supposedly going to live in the virtio driver, interface, and
then push the original virtio down one layer and call it a
virtio-backup. If I am not mistaken we could assign the same dev
pointer used by the virtio netdev to the virtio-bond, and if we
register it first with the "eth%d" name then udev will assume that the
virtio-bond device is the original virtio and all existing scripts
should just work with that. We then would want to change the name of
the virtio interface with the backup feature bit set, maybe call it
something like bkup-00:00:00 where the 00:00:00 would be the last 3
octets of the MAC address. It should solve the issue of inserting an
interface "above" the virtio by making the virtio-bond become the
virtio. The only limitation is that we will probably need to remove
the back-up if the virtio device is removed, however that was already
a limitation of this solution and others like the netvsc solution
anyway.
>> >> Also, from a user experience point of view, loading a virtio-net with
>> >> BACKUP feature enabled will now show 2 virtio-net netdevs.
>> > One virtio-net and one virtio-bond, which represents what's happening.
>> This again assumes that we want to represent a bond setup. Can't we
>> treat this
>> as virtio-net providing an alternate low-latency datapath by taking over
>> VF datapath?
>
> Bond is just a familiar name, we can call it something else if you
> prefer. The point is there are two data paths which can have
> independent low-level settings and a higher level entity with
> global settings which represents any path to the outside world.
>
> Hiding low-level netdevs from a lay user requires a generic solution.
The last thing I think we should be doing is hiding the low level
netdevs. It doesn't solve anythying. We are already trusting the owner
of the VM enough to let them have root access of the VM. That means
they can load/unload any driver they want. They don't have to use the
kernel provided virtio driver, they could load their own. They could
even do something like run DPDK on top of it, or they could run DPDK
on top of the VF. In either case there is no way the bond would ever
be created and all hiding devices does is make it easier to fix
problems when something gets broken. Unless I am mistaken, and
"security through obscurity" has somehow become a valid security
model.
As I mentioned to Sridhar on an off-list thread I think the goal
should be to make it so that the user wants to use the virtio-bond,
not make it so that they have no choice but to use it. The idea is we
should be making things easier for the administrator of the VM, not
harder.
- Alex
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Samudrala, Sridhar @ 2018-01-28 19:18 UTC (permalink / raw)
To: Alexander Duyck, Jakub Kicinski
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Netdev,
virtualization, Siwei Liu, David Miller
In-Reply-To: <CAKgT0Ud4e9O-FoY242Kn_yBX7JWtL_i=yW+Y-MXQ9BGEVBEkgA@mail.gmail.com>
On 1/28/2018 9:35 AM, Alexander Duyck wrote:
> On Fri, Jan 26, 2018 at 9:58 PM, Jakub Kicinski <kubakici@wp.pl> wrote:
>> On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
>>>>> 3 netdev model breaks this configuration starting with the creation
>>>>> and naming of the 2 devices to udev needing to be aware of master and
>>>>> slave virtio-net devices.
>>>> I don't understand this comment. There is one virtio-net device and
>>>> one "virtio-bond" netdev. And user space has to be aware of the special
>>>> automatic arrangement anyway, because it can't touch the VF. It
>>>> doesn't make any difference whether it ignores the VF or PV and VF.
>>>> It simply can't touch the slaves, no matter how many there are.
>>> If the userspace is not expected to touch the slaves, then why do we need to
>>> take extra effort to expose a netdev that is just not really useful.
>> You said:
>> "[user space] needs to be aware of master and slave virtio-net devices."
>>
>> I'm saying:
>> It has to be aware of the special arrangement whether there is an
>> explicit bond netdev or not.
> To clarify here the kernel should be aware that there is a device that
> is an aggregate of 2 other devices. It isn't as if we need to insert
> the new device "above" the virtio.
>
> I have been doing a bit of messing around with a few ideas and it
> seems like it would be better if we could replace the virtio interface
> with the virtio-bond, renaming my virt-bond concept to this since it
> is now supposedly going to live in the virtio driver, interface, and
> then push the original virtio down one layer and call it a
> virtio-backup. If I am not mistaken we could assign the same dev
> pointer used by the virtio netdev to the virtio-bond, and if we
> register it first with the "eth%d" name then udev will assume that the
> virtio-bond device is the original virtio and all existing scripts
> should just work with that. We then would want to change the name of
> the virtio interface with the backup feature bit set, maybe call it
> something like bkup-00:00:00 where the 00:00:00 would be the last 3
> octets of the MAC address. It should solve the issue of inserting an
> interface "above" the virtio by making the virtio-bond become the
> virtio. The only limitation is that we will probably need to remove
> the back-up if the virtio device is removed, however that was already
> a limitation of this solution and others like the netvsc solution
> anyway.
With 3 netdev model, if we make the the master virtio-net associated
with the
real virtio pci device, i think the userspace scripts would not break.
If we go this route, i am still not clear on the purpose of exposing the
bkup netdev.
Can we start with the 2 netdev model and move to 3 netdev model later if we
find out that there are limitiations with the 2 netdev model? I don't
think this will
break any user API as the userspace is not expected to use the bkup netdev.
>
>>>>> Also, from a user experience point of view, loading a virtio-net with
>>>>> BACKUP feature enabled will now show 2 virtio-net netdevs.
>>>> One virtio-net and one virtio-bond, which represents what's happening.
>>> This again assumes that we want to represent a bond setup. Can't we
>>> treat this
>>> as virtio-net providing an alternate low-latency datapath by taking over
>>> VF datapath?
>> Bond is just a familiar name, we can call it something else if you
>> prefer. The point is there are two data paths which can have
>> independent low-level settings and a higher level entity with
>> global settings which represents any path to the outside world.
>>
>> Hiding low-level netdevs from a lay user requires a generic solution.
> The last thing I think we should be doing is hiding the low level
> netdevs. It doesn't solve anythying. We are already trusting the owner
> of the VM enough to let them have root access of the VM. That means
> they can load/unload any driver they want. They don't have to use the
> kernel provided virtio driver, they could load their own. They could
> even do something like run DPDK on top of it, or they could run DPDK
> on top of the VF. In either case there is no way the bond would ever
> be created and all hiding devices does is make it easier to fix
> problems when something gets broken. Unless I am mistaken, and
> "security through obscurity" has somehow become a valid security
> model.
>
> As I mentioned to Sridhar on an off-list thread I think the goal
> should be to make it so that the user wants to use the virtio-bond,
> not make it so that they have no choice but to use it. The idea is we
> should be making things easier for the administrator of the VM, not
> harder.
>
>
When the hypervisor has enabled BACKUP bit along with a VF with the same
MAC address, the VM can use either VF only OR extended virtio with
attached VF.
Although there are 2 datapaths to the device, the hypervisor will enable
only
one at any time. The virtio via PF datapath is only enabled during live
migration
when the VF is unplugged. If not, VF broadcasts/multicasts will get
looped back
to the VM via the PF datapath. In the RFC path i was assuming that the
VM can
use both datapaths and i had broadcasts/multicasts going over virtio
datapath,
but i don' think it is a good idea. It requires the device to disable
broadcast
replication on the VFs.
Thanks
Sridhar
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* CISTI'2018 - Doctoral Symposium
From: Maria Lemos @ 2018-01-28 19:29 UTC (permalink / raw)
To: virtualization
[-- Attachment #1.1: Type: text/plain, Size: 4313 bytes --]
* Proceedings published in IEEE Xplore and indexed by ISI, Scopus, etc.
---------------------------------------------------------------------------------------------------------------------------
Doctoral Symposium of CISTI'2018 - 13th Iberian Conference on Information Systems and Technologies
Caceres, Spain, 16 - 13 June 2018
http://www.cisti.eu/ <http://www.cisti.eu/>
------------------------------------------------------------------------------------------------------------------------------------
The purpose of CISTI'2018’s Doctoral Symposium is to provide graduate students a setting where they can, informally, expose and discuss their work, collecting valuable expert opinions and sharing new ideas, methods and applications. The Doctoral Symposium is an excellent opportunity for PhD students to present and discuss their work in a Workshop format. Each presentation will be evaluated by a panel composed by at least three Information Systems and Technologies experts.
Contributions Submission
The Doctoral Symposium is opened to PhD students whose research area includes the themes proposed for this Conference. Submissions must include an extended abstract (maximum 4 pages), following the Conference style guide <http://cisti.eu/2017/images/templates.zip>. All selected contributions will be handed out along with the Conference Proceedings, in CD with an ISBN. These contributions will be available in the IEEE Xplore <http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7511893> Digital Library and will be sent for indexing in ISI, Scopus, EI-Compendex, INSPEC and Google Scholar.
Submissions must include the field, the PhD institution and the number of months devoted to the development of the work. Additionally, they should include in a clear and succinct manner:
• The problem approached and its significance or relevance
• The research objectives and related investigation topics
• A brief display of what is already known
• A proposed solution methodology for the problem
• Expected results
Important Dates
Paper submission: February 11, 2018
Notification of acceptance: March 18, 2018
Submission of accepted papers: March 30, 2018
Payment of registration, to ensure the inclusion of an accepted paper in the conference proceedings: April 1, 2018
Organizing Committee
Álvaro Rocha, Universidade de Coimbra
Manuel Pérez Cota, Universidad de Vigo
Scientific Committee
Manuel Pérez Cota, Universidad de Vigo (Chair)
Adolfo Lozano Tello, Universidad de Extremadura
Álvaro Rocha, Universidade de Coimbra
Ana Amélia Carvalho, Universidade de Coimbra
Ana Maria Ramalho Correia, Nova Information Management School
Antonio Garcia Loureiro, Universidad de Santiago de Compostela
António Lucas Soares, Universidade do Porto, FEUP
Arnaldo Martins, Universidade de Aveiro
Bráulio Alturas, Instituto Universitário de Lisboa (ISCTE-IUL)
Carlos Ferrás Sexto, Universidad de Santiago de Compostela
David Fonseca, La Salle, Universitat Ramon Llull
Ernest Redondo, Universidad Politécnica de Catalunya
Francisco Restivo, Universidade Católica Portuguesa
Gonçalo Paiva Dias, Universidade de Aveiro
Gonzalo Cuevas Agustin, Universidad Politécnica de Madrid
João Paulo Costa, Universidade de Coimbra
José Borbinha, INESC-ID, IST, Universidade de Lisboa
José Valença, Universidade do Minho
Jose Antonio Calvo-Manzano Villalón, Universidad Politécnica de Madrid
Juan Hernádez, Universidad de Extremadura
Luis Camarinha-Matos, Universidade Nova de Lisboa
Luís Paulo Reis, Universidade do Minho
Marco Painho, Nova Information Management School
Mário Piattini, Universidad de Castilla-La Mancha
Nelson Pacheco Rocha, Universidade de Aveiro
Ramiro Gonçalves, Universidade de Trás-os-Montes e Alto Douro
Doctoral Symposium webpage: http://cisti.eu/index.php?option=com_content&view=article&id=35&Itemid=119&lang=en
Kind regards,
CISTI'2018 Team
http://www.cisti.eu/
---
This email has been checked for viruses by AVG.
http://www.avg.com
[-- Attachment #1.2: Type: text/html, Size: 7685 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Alexander Duyck @ 2018-01-28 20:18 UTC (permalink / raw)
To: Samudrala, Sridhar
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Jakub Kicinski,
Netdev, virtualization, Siwei Liu, David Miller
In-Reply-To: <f6737111-0015-007c-336b-c8c2793ee590@intel.com>
On Sun, Jan 28, 2018 at 11:18 AM, Samudrala, Sridhar
<sridhar.samudrala@intel.com> wrote:
> On 1/28/2018 9:35 AM, Alexander Duyck wrote:
>>
>> On Fri, Jan 26, 2018 at 9:58 PM, Jakub Kicinski <kubakici@wp.pl> wrote:
>>>
>>> On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
>>>>>>
>>>>>> 3 netdev model breaks this configuration starting with the creation
>>>>>> and naming of the 2 devices to udev needing to be aware of master and
>>>>>> slave virtio-net devices.
>>>>>
>>>>> I don't understand this comment. There is one virtio-net device and
>>>>> one "virtio-bond" netdev. And user space has to be aware of the
>>>>> special
>>>>> automatic arrangement anyway, because it can't touch the VF. It
>>>>> doesn't make any difference whether it ignores the VF or PV and VF.
>>>>> It simply can't touch the slaves, no matter how many there are.
>>>>
>>>> If the userspace is not expected to touch the slaves, then why do we
>>>> need to
>>>> take extra effort to expose a netdev that is just not really useful.
>>>
>>> You said:
>>> "[user space] needs to be aware of master and slave virtio-net devices."
>>>
>>> I'm saying:
>>> It has to be aware of the special arrangement whether there is an
>>> explicit bond netdev or not.
>>
>> To clarify here the kernel should be aware that there is a device that
>> is an aggregate of 2 other devices. It isn't as if we need to insert
>> the new device "above" the virtio.
>>
>> I have been doing a bit of messing around with a few ideas and it
>> seems like it would be better if we could replace the virtio interface
>> with the virtio-bond, renaming my virt-bond concept to this since it
>> is now supposedly going to live in the virtio driver, interface, and
>> then push the original virtio down one layer and call it a
>> virtio-backup. If I am not mistaken we could assign the same dev
>> pointer used by the virtio netdev to the virtio-bond, and if we
>> register it first with the "eth%d" name then udev will assume that the
>> virtio-bond device is the original virtio and all existing scripts
>> should just work with that. We then would want to change the name of
>> the virtio interface with the backup feature bit set, maybe call it
>> something like bkup-00:00:00 where the 00:00:00 would be the last 3
>> octets of the MAC address. It should solve the issue of inserting an
>> interface "above" the virtio by making the virtio-bond become the
>> virtio. The only limitation is that we will probably need to remove
>> the back-up if the virtio device is removed, however that was already
>> a limitation of this solution and others like the netvsc solution
>> anyway.
>
>
> With 3 netdev model, if we make the the master virtio-net associated with
> the
> real virtio pci device, i think the userspace scripts would not break.
> If we go this route, i am still not clear on the purpose of exposing the
> bkup netdev.
> Can we start with the 2 netdev model and move to 3 netdev model later if we
> find out that there are limitiations with the 2 netdev model? I don't think
> this will
> break any user API as the userspace is not expected to use the bkup netdev.
The 2 netdev model breaks a large number of expectations of user
space. Among them is XDP since we cannot guarantee a symmetric setup
between any entity and the virtio. How does it make sense that
enabling XDP on virtio shows zero Rx packets, and in the meantime you
are getting all of the packets coming in off of the VF?
In addition we would need to rewrite the VLAN and MAC address
filtering ndo operations since we likely cannot add any VLANs since in
most cases VFs are VLAN locked due to things like port VLAN and we
cannot change the MAC address since the whole bonding concept is built
around it.
The last bit is the netpoll packet routing which the current code
assumes is using the virtio only, but I don't know if that is a valid
assumption since the VF is expected to be the default route for
everything else when it is available.
The point is by the time you are done you will have rewritten pretty
much all the network device ops. With that being the case why add all
the code to virtio itself instead of just coming up with a brand new
set of ndo_ops that belong to this new device, and you could leave the
existing virtio code in place and save yourself a bunch of time by
just accessing it as an existing call as a separate netdev.
>>>>>> Also, from a user experience point of view, loading a virtio-net with
>>>>>> BACKUP feature enabled will now show 2 virtio-net netdevs.
>>>>>
>>>>> One virtio-net and one virtio-bond, which represents what's happening.
>>>>
>>>> This again assumes that we want to represent a bond setup. Can't we
>>>> treat this
>>>> as virtio-net providing an alternate low-latency datapath by taking over
>>>> VF datapath?
>>>
>>> Bond is just a familiar name, we can call it something else if you
>>> prefer. The point is there are two data paths which can have
>>> independent low-level settings and a higher level entity with
>>> global settings which represents any path to the outside world.
>>>
>>> Hiding low-level netdevs from a lay user requires a generic solution.
>>
>> The last thing I think we should be doing is hiding the low level
>> netdevs. It doesn't solve anythying. We are already trusting the owner
>> of the VM enough to let them have root access of the VM. That means
>> they can load/unload any driver they want. They don't have to use the
>> kernel provided virtio driver, they could load their own. They could
>> even do something like run DPDK on top of it, or they could run DPDK
>> on top of the VF. In either case there is no way the bond would ever
>> be created and all hiding devices does is make it easier to fix
>> problems when something gets broken. Unless I am mistaken, and
>> "security through obscurity" has somehow become a valid security
>> model.
>>
>> As I mentioned to Sridhar on an off-list thread I think the goal
>> should be to make it so that the user wants to use the virtio-bond,
>> not make it so that they have no choice but to use it. The idea is we
>> should be making things easier for the administrator of the VM, not
>> harder.
>>
>>
> When the hypervisor has enabled BACKUP bit along with a VF with the same
> MAC address, the VM can use either VF only OR extended virtio with attached
> VF.
> Although there are 2 datapaths to the device, the hypervisor will enable
> only
> one at any time. The virtio via PF datapath is only enabled during live
> migration
> when the VF is unplugged. If not, VF broadcasts/multicasts will get looped
> back
> to the VM via the PF datapath. In the RFC path i was assuming that the VM
> can
> use both datapaths and i had broadcasts/multicasts going over virtio
> datapath,
> but i don' think it is a good idea. It requires the device to disable
> broadcast
> replication on the VFs.
>
> Thanks
> Sridhar
This would work well for the SwitchDev model, but not so well for the
legacy model supported by the Intel drivers. With SwitchDev the
broadcasts/multicasts are coming in on the upllink port representor
anyway is my understanding of how some of the implementations out
there are configured.
For now though we can't assume the logic in the path. We can do that
later when we start looking at v2 which would likely have us spawn a
new device that acts as some sort of smart PCI bridge/PCIe switch that
can then have that device assigned to the virtio-bond/bridge and can
hopefully start taking care of things like providing topology/switch
information and maybe incorporate some DMA tracking on the VF which
would probably be v3.
In the meantime for now I would say we keep this simple. We create a
virtio-bond device that does a bit of identity theft on the existing
virtio by mapping the same device, pushes the existing virtio to a
different name to hide it from udev to avoid naming confusion. We make
the device as dumb as can be and don't let it do any L2 configuration
such as assigning a MAC address or VLAN filter, we could probably flag
it as VLAN challenged. The XDP problem solves itself by us not
exposing any bpf or XDP operations. Then all that is left is dealing
with netpoll which if needed we can probably borrow the existing
approach from the bonding driver to solve.
Anyway that is the direction I see this going in.
Thanks.
- Alex
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Samudrala, Sridhar @ 2018-01-28 21:01 UTC (permalink / raw)
To: Alexander Duyck
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Jakub Kicinski,
Netdev, virtualization, Siwei Liu, David Miller
In-Reply-To: <CAKgT0UdaKtPe6982TuuGbxhhVgeehwS1aAp=s4sok2vKD6wMJg@mail.gmail.com>
On 1/28/2018 12:18 PM, Alexander Duyck wrote:
> On Sun, Jan 28, 2018 at 11:18 AM, Samudrala, Sridhar
> <sridhar.samudrala@intel.com> wrote:
>> On 1/28/2018 9:35 AM, Alexander Duyck wrote:
>>> On Fri, Jan 26, 2018 at 9:58 PM, Jakub Kicinski <kubakici@wp.pl> wrote:
>>>> On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
>>>>>>> 3 netdev model breaks this configuration starting with the creation
>>>>>>> and naming of the 2 devices to udev needing to be aware of master and
>>>>>>> slave virtio-net devices.
>>>>>> I don't understand this comment. There is one virtio-net device and
>>>>>> one "virtio-bond" netdev. And user space has to be aware of the
>>>>>> special
>>>>>> automatic arrangement anyway, because it can't touch the VF. It
>>>>>> doesn't make any difference whether it ignores the VF or PV and VF.
>>>>>> It simply can't touch the slaves, no matter how many there are.
>>>>> If the userspace is not expected to touch the slaves, then why do we
>>>>> need to
>>>>> take extra effort to expose a netdev that is just not really useful.
>>>> You said:
>>>> "[user space] needs to be aware of master and slave virtio-net devices."
>>>>
>>>> I'm saying:
>>>> It has to be aware of the special arrangement whether there is an
>>>> explicit bond netdev or not.
>>> To clarify here the kernel should be aware that there is a device that
>>> is an aggregate of 2 other devices. It isn't as if we need to insert
>>> the new device "above" the virtio.
>>>
>>> I have been doing a bit of messing around with a few ideas and it
>>> seems like it would be better if we could replace the virtio interface
>>> with the virtio-bond, renaming my virt-bond concept to this since it
>>> is now supposedly going to live in the virtio driver, interface, and
>>> then push the original virtio down one layer and call it a
>>> virtio-backup. If I am not mistaken we could assign the same dev
>>> pointer used by the virtio netdev to the virtio-bond, and if we
>>> register it first with the "eth%d" name then udev will assume that the
>>> virtio-bond device is the original virtio and all existing scripts
>>> should just work with that. We then would want to change the name of
>>> the virtio interface with the backup feature bit set, maybe call it
>>> something like bkup-00:00:00 where the 00:00:00 would be the last 3
>>> octets of the MAC address. It should solve the issue of inserting an
>>> interface "above" the virtio by making the virtio-bond become the
>>> virtio. The only limitation is that we will probably need to remove
>>> the back-up if the virtio device is removed, however that was already
>>> a limitation of this solution and others like the netvsc solution
>>> anyway.
>>
>> With 3 netdev model, if we make the the master virtio-net associated with
>> the
>> real virtio pci device, i think the userspace scripts would not break.
>> If we go this route, i am still not clear on the purpose of exposing the
>> bkup netdev.
>> Can we start with the 2 netdev model and move to 3 netdev model later if we
>> find out that there are limitiations with the 2 netdev model? I don't think
>> this will
>> break any user API as the userspace is not expected to use the bkup netdev.
> The 2 netdev model breaks a large number of expectations of user
> space. Among them is XDP since we cannot guarantee a symmetric setup
> between any entity and the virtio. How does it make sense that
> enabling XDP on virtio shows zero Rx packets, and in the meantime you
> are getting all of the packets coming in off of the VF?
Sure we cannot support XDP in this model and it needs to be disabled.
>
> In addition we would need to rewrite the VLAN and MAC address
> filtering ndo operations since we likely cannot add any VLANs since in
> most cases VFs are VLAN locked due to things like port VLAN and we
> cannot change the MAC address since the whole bonding concept is built
> around it.
>
> The last bit is the netpoll packet routing which the current code
> assumes is using the virtio only, but I don't know if that is a valid
> assumption since the VF is expected to be the default route for
> everything else when it is available.
>
> The point is by the time you are done you will have rewritten pretty
> much all the network device ops. With that being the case why add all
> the code to virtio itself instead of just coming up with a brand new
> set of ndo_ops that belong to this new device, and you could leave the
> existing virtio code in place and save yourself a bunch of time by
> just accessing it as an existing call as a separate netdev.
When the BACKUP feature is enabled, we can simply disable most of these
ndo ops
that cannot be supported. Not sure we need an additional netdev and ndo_ops.
When we can support all these usecases along with live migration we can move
to the 3 netdev model and i think we will need a new feature bit so that the
hypervisor can allow VM to use both datapaths and configure PF accordingly.
Thanks
Sridhar
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Stephen Hemminger @ 2018-01-28 23:02 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin,
Samudrala, Sridhar, virtualization, Siwei Liu, Netdev,
David Miller
In-Reply-To: <20180126183003.591cd5c5@cakuba.netronome.com>
On Fri, 26 Jan 2018 18:30:03 -0800
Jakub Kicinski <kubakici@wp.pl> wrote:
> On Fri, 26 Jan 2018 15:30:35 -0800, Samudrala, Sridhar wrote:
> > On 1/26/2018 2:47 PM, Jakub Kicinski wrote:
> > > On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
> > >> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
> > >>>> and the VM is not expected to do any tuning/optimizations on the VF driver
> > >>>> directly,
> > >>>> i think the current patch that follows the netvsc model of 2 netdevs(virtio
> > >>>> and vf) should
> > >>>> work fine.
> > >>> OK. For your use case that's fine. But that's too specific scenario
> > >>> with lots of restrictions IMHO, perhaps very few users will benefit
> > >>> from it, I'm not sure. If you're unwilling to move towards it, we'd
> > >>> take this one and come back with a generic solution that is able to
> > >>> address general use cases for VF/PT live migration .
> > >> I think that's a fine approach. Scratch your own itch! I imagine a very
> > >> generic virtio-switchdev providing host routing info to guests could
> > >> address lots of usecases. A driver could bind to that one and enslave
> > >> arbitrary other devices. Sounds reasonable.
> > >>
> > >> But given the fundamental idea of a failover was floated at least as
> > >> early as 2013, and made 0 progress since precisely because it kept
> > >> trying to address more and more features, and given netvsc is already
> > >> using the basic solution with some success, I'm not inclined to block
> > >> this specific effort waiting for the generic one.
> > > I think there is an agreement that the extra netdev will be useful for
> > > more advanced use cases, and is generally preferable. What is the
> > > argument for not doing that from the start? If it was made I must have
> > > missed it. Is it just unwillingness to write the extra 300 lines of
> > > code? Sounds like a pretty weak argument when adding kernel ABI is at
> > > stake...
> >
> > I am still not clear on the need for the extra netdev created by
> > virtio_net. The only advantage i can see is that the stats can be
> > broken between VF and virtio datapaths compared to the aggregrated
> > stats on virtio netdev as seen with the 2 netdev approach.
>
> Maybe you're not convinced but multiple arguments were made.
>
> > With 2 netdev model, any VM image that has a working network
> > configuration will transparently get VF based acceleration without
> > any changes.
>
> Nothing happens transparently. Things may happen automatically. The
> VF netdev doesn't disappear with netvsc. The PV netdev transforms into
> something it did not use to be. And configures and reports some
> information from the PV (e.g. speed) but PV doesn't pass traffic any
> longer.
>
> > 3 netdev model breaks this configuration starting with the creation
> > and naming of the 2 devices to udev needing to be aware of master and
> > slave virtio-net devices.
>
> I don't understand this comment. There is one virtio-net device and
> one "virtio-bond" netdev. And user space has to be aware of the special
> automatic arrangement anyway, because it can't touch the VF. It
> doesn't make any difference whether it ignores the VF or PV and VF.
> It simply can't touch the slaves, no matter how many there are.
>
> > Also, from a user experience point of view, loading a virtio-net with
> > BACKUP feature enabled will now show 2 virtio-net netdevs.
>
> One virtio-net and one virtio-bond, which represents what's happening.
>
> > For live migration with advanced usecases that Siwei is suggesting, i
> > think we need a new driver with a new device type that can track the
> > VF specific feature settings even when the VF driver is unloaded.
I see no added value of the 3 netdev model, there is no need for a bond
device.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Alexander Duyck @ 2018-01-29 0:58 UTC (permalink / raw)
To: Samudrala, Sridhar
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Jakub Kicinski,
Netdev, virtualization, Siwei Liu, David Miller
In-Reply-To: <b6bf20f8-8881-4d25-db4c-24f93d5e6cba@intel.com>
On Sun, Jan 28, 2018 at 1:01 PM, Samudrala, Sridhar
<sridhar.samudrala@intel.com> wrote:
>
>
> On 1/28/2018 12:18 PM, Alexander Duyck wrote:
>>
>> On Sun, Jan 28, 2018 at 11:18 AM, Samudrala, Sridhar
>> <sridhar.samudrala@intel.com> wrote:
>>>
>>> On 1/28/2018 9:35 AM, Alexander Duyck wrote:
>>>>
>>>> On Fri, Jan 26, 2018 at 9:58 PM, Jakub Kicinski <kubakici@wp.pl> wrote:
>>>>>
>>>>> On Fri, 26 Jan 2018 21:33:01 -0800, Samudrala, Sridhar wrote:
>>>>>>>>
>>>>>>>> 3 netdev model breaks this configuration starting with the creation
>>>>>>>> and naming of the 2 devices to udev needing to be aware of master
>>>>>>>> and
>>>>>>>> slave virtio-net devices.
>>>>>>>
>>>>>>> I don't understand this comment. There is one virtio-net device and
>>>>>>> one "virtio-bond" netdev. And user space has to be aware of the
>>>>>>> special
>>>>>>> automatic arrangement anyway, because it can't touch the VF. It
>>>>>>> doesn't make any difference whether it ignores the VF or PV and VF.
>>>>>>> It simply can't touch the slaves, no matter how many there are.
>>>>>>
>>>>>> If the userspace is not expected to touch the slaves, then why do we
>>>>>> need to
>>>>>> take extra effort to expose a netdev that is just not really useful.
>>>>>
>>>>> You said:
>>>>> "[user space] needs to be aware of master and slave virtio-net
>>>>> devices."
>>>>>
>>>>> I'm saying:
>>>>> It has to be aware of the special arrangement whether there is an
>>>>> explicit bond netdev or not.
>>>>
>>>> To clarify here the kernel should be aware that there is a device that
>>>> is an aggregate of 2 other devices. It isn't as if we need to insert
>>>> the new device "above" the virtio.
>>>>
>>>> I have been doing a bit of messing around with a few ideas and it
>>>> seems like it would be better if we could replace the virtio interface
>>>> with the virtio-bond, renaming my virt-bond concept to this since it
>>>> is now supposedly going to live in the virtio driver, interface, and
>>>> then push the original virtio down one layer and call it a
>>>> virtio-backup. If I am not mistaken we could assign the same dev
>>>> pointer used by the virtio netdev to the virtio-bond, and if we
>>>> register it first with the "eth%d" name then udev will assume that the
>>>> virtio-bond device is the original virtio and all existing scripts
>>>> should just work with that. We then would want to change the name of
>>>> the virtio interface with the backup feature bit set, maybe call it
>>>> something like bkup-00:00:00 where the 00:00:00 would be the last 3
>>>> octets of the MAC address. It should solve the issue of inserting an
>>>> interface "above" the virtio by making the virtio-bond become the
>>>> virtio. The only limitation is that we will probably need to remove
>>>> the back-up if the virtio device is removed, however that was already
>>>> a limitation of this solution and others like the netvsc solution
>>>> anyway.
>>>
>>>
>>> With 3 netdev model, if we make the the master virtio-net associated with
>>> the
>>> real virtio pci device, i think the userspace scripts would not break.
>>> If we go this route, i am still not clear on the purpose of exposing the
>>> bkup netdev.
>>> Can we start with the 2 netdev model and move to 3 netdev model later if
>>> we
>>> find out that there are limitiations with the 2 netdev model? I don't
>>> think
>>> this will
>>> break any user API as the userspace is not expected to use the bkup
>>> netdev.
>>
>> The 2 netdev model breaks a large number of expectations of user
>> space. Among them is XDP since we cannot guarantee a symmetric setup
>> between any entity and the virtio. How does it make sense that
>> enabling XDP on virtio shows zero Rx packets, and in the meantime you
>> are getting all of the packets coming in off of the VF?
>
> Sure we cannot support XDP in this model and it needs to be disabled.
>>
>>
>> In addition we would need to rewrite the VLAN and MAC address
>> filtering ndo operations since we likely cannot add any VLANs since in
>> most cases VFs are VLAN locked due to things like port VLAN and we
>> cannot change the MAC address since the whole bonding concept is built
>> around it.
>>
>> The last bit is the netpoll packet routing which the current code
>> assumes is using the virtio only, but I don't know if that is a valid
>> assumption since the VF is expected to be the default route for
>> everything else when it is available.
>>
>> The point is by the time you are done you will have rewritten pretty
>> much all the network device ops. With that being the case why add all
>> the code to virtio itself instead of just coming up with a brand new
>> set of ndo_ops that belong to this new device, and you could leave the
>> existing virtio code in place and save yourself a bunch of time by
>> just accessing it as an existing call as a separate netdev.
>
>
> When the BACKUP feature is enabled, we can simply disable most of these ndo
> ops
> that cannot be supported. Not sure we need an additional netdev and ndo_ops.
The point is feature bits have to be changed, and network
device/ethtool ops have to be rewritten. At that point you have
essentially forked the virtio interface. What I don't understand is
what your resistance is to doing this as 3 netdevs? I thought the
original issue was the fact that the bond-like interface would come up
with a different name/ID. I think we have a solution for that at this
point. I think the bigger issue is that this 2 netdev approach is far
to invasive to be done in the virtio driver itself and would require
massive changes to it. It makes more sense to just put together a
small new netdev that basically does the "dumb-bond" solution.
> When we can support all these usecases along with live migration we can move
> to the 3 netdev model and i think we will need a new feature bit so that the
> hypervisor can allow VM to use both datapaths and configure PF accordingly.
Right now you aren't supporting any of the use cases you claim to. The
code as it stands is full of corner cases. I have already pointed out
several holes in the code that were overlooked that are going to sink
the whole concept if we were to try to push something like this into
production.
On Thursday I tried seeing how hard it would be to make bonding fit
into the model we need. The bonding driver is carrying a TON of extra
bloat and technical debt that we don't need at this point since it has
all the MII/ARP code, sysfs, procfs, debugfs, notifiers, and modes
that we don't want. Trying to cut out that code and make something
that would be usable for this is probably going to be too expensive in
the long run. In the meantime though I think I have developed an
appreciation of where both solutions are currently lacking.
I think implementing another netdev at this point is the only way to
go forward, otherwise what is going to happen is that virtio is going
to quickly bloat up with a bunch of code that has to disable almost
everything in it if BACKUP is enabled as a feature. So here are the
list of netdev ops for virtio:
.ndo_open = virtnet_open,
.ndo_stop = virtnet_close,
I assume these first 2 will need some changes. I suspect there would
be threads responsible for doing things like maintaining the link
statistics that need to be collected periodically. Also I don't know
if you want to keep the slaves up if the master is up or not. In
addition I don't see any code in your current solution that calls
dev_open. I am assuming something was probably bringing up the
interface for you and then you enslaved it instead of the virtio
interface bringing up the interface itself.
.ndo_start_xmit = start_xmit,
Your original patch included a small set of changes for this function.
One thing you appear to have overlooked was the fact that
qdisc_skb_cb(skb)->slave_dev_queue_mapping relies on ndo_select_queue.
It means you are missing at least one new ndo_op that is not currently
provided by virtio. In addition we may want to look at providing a
change of the queue mapping for the combined interface since the VF
may support more queues than the virtio currently does. In my mind we
probably don't even need a qdisc for the upper level device anyway. We
could probably improve our performance if we could make the device not
have a qdisc and even more if we could make it run with LLTX since the
lower device has both covered for us already.
.ndo_validate_addr = eth_validate_addr,
.ndo_set_mac_address = virtnet_set_mac_address,
.ndo_set_rx_mode = virtnet_set_rx_mode,
I already mentioned most of the L2 address/filtering code would
probably have to go since the VF drivers aren't usually allowed to do
things like go into full promiscuous or change their MAC address
anyway, and we don't want the address of a MAC address based bond to
be changed.
.ndo_get_stats64 = virtnet_stats,
You already modified this in your first rev of the patch. Your
solution is okay I suppose, though it might be more interesting if you
could just provide a merged version of the stats of the 2 interfaces
instead of just capturing some of the Tx/Rx stats.
.ndo_vlan_rx_add_vid = virtnet_vlan_rx_add_vid,
.ndo_vlan_rx_kill_vid = virtnet_vlan_rx_kill_vid,
These can both go since we could assume the combined interface is more
than likely "VLAN challenged" due to the fact that the primary use
case is VF migration. Otherwise we need to be able to guarantee that
any VLAN added to the interface is present on both the VF and the
virtio. Since that is extra work just calling the interface "VLAN
challenged" for now is easier.
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = virtnet_netpoll,
#endif
The polling logic needs to be updated so that the polling follows the
Tx packet path instead of just assuming everything is running through
virtio since a backup interface may or may not be capable of
transmitting packets when not in use.
.ndo_bpf = virtnet_xdp,
.ndo_xdp_xmit = virtnet_xdp_xmit,
.ndo_xdp_flush = virtnet_xdp_flush,
This whole block would have to be dropped for the combined interface.
That way we will at least get symmetric behavior between a VF and the
virtio data paths.
.ndo_features_check = passthru_features_check,
This part can mostly stay, but as I said the feature bits are going to
have to significantly change for the interface that is sitting on top
of the VF and virtio since the feature functionality likely differs
quite a bit.
I would go through the ethtool ops, but the fact is most of them just
don't apply. All bond implements is the get_drvinfo, get_link, and
get_link_ksettings and that is probably all that would be needed for
the virtio-bond interface.
Hopefully that clarifies things. In my mind the 2 netdev approach
would be the approach to consider for later. For now it is probably
easier to just have the virtio-bond, virtio-backup, and the VF all
present as it makes it easier to get away with just reusing existing
code since all the needed ops are exposed by the netdevs. Hiding them
is going to make things more difficult for debugging so I am strongly
opposed to hiding things.
I realize I told you I was willing to disagree and commit on this, but
after spending a full day reviewing the solution proposed and spending
some time crawling through the bonding code to get a better grasp of
things I am going to have to say the 2 netdev approach just isn't
feasible. We need to have 3 separate netdevs exposing 3 separate sets
of ndo/ethtool ops in order for things to work correctly. Anything
else is going to end up severely impacting features and/or
performance.
- Alex
^ permalink raw reply
* Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Jason Wang @ 2018-01-29 3:32 UTC (permalink / raw)
To: Samudrala, Sridhar, mst, stephen, davem, netdev, virtualization,
virtio-dev, jesse.brandeburg, alexander.h.duyck, kubakici
In-Reply-To: <f6d80eca-8f95-861c-6717-7df53af870fe@intel.com>
On 2018年01月24日 00:03, Samudrala, Sridhar wrote:
>
>
> On 1/23/2018 2:33 AM, Jason Wang wrote:
>>
>>
>> On 2018年01月12日 13:58, Sridhar Samudrala wrote:
>>> static netdev_tx_t start_xmit(struct sk_buff *skb, struct
>>> net_device *dev)
>>> {
>>> struct virtnet_info *vi = netdev_priv(dev);
>>> int qnum = skb_get_queue_mapping(skb);
>>> struct send_queue *sq = &vi->sq[qnum];
>>> + struct net_device *vf_netdev;
>>> int err;
>>> struct netdev_queue *txq = netdev_get_tx_queue(dev, qnum);
>>> bool kick = !skb->xmit_more;
>>> bool use_napi = sq->napi.weight;
>>> + /* If VF is present and up then redirect packets
>>> + * called with rcu_read_lock_bh
>>> + */
>>> + vf_netdev = rcu_dereference_bh(vi->vf_netdev);
>>> + if (vf_netdev && netif_running(vf_netdev) &&
>>> + !netpoll_tx_running(dev) &&
>>> + is_unicast_ether_addr(eth_hdr(skb)->h_dest))
>>> + return virtnet_vf_xmit(dev, vf_netdev, skb);
>>> +
>>
>> A question here.
>>
>> If I read the code correctly, all features were validated against
>> virtio instead VF before transmitting. This assumes VF's feature is a
>> superset of virtio, does this really work? Do we need to sanitize the
>> feature before joining? (e.g at last NETIF_R_GSO_ROBUST needs to be
>> removed).
>
> Actually, virtnet_vf_xmit() calls dev_queue_xmit() after updating
> skb->dev to vf netdev.
> So the features get validated against VF features and the right tx
> queue is selected
> before the real transmit.
I see, the the packet will go through qdiscs twice which is suboptimal.
And the device may lose some ability of VF like tunnel GSO.
Thanks
>
> Thanks
> Sridhar
>
>
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Alexander Duyck @ 2018-01-29 4:26 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Alexander Duyck, virtio-dev, Michael S. Tsirkin, Jakub Kicinski,
Samudrala, Sridhar, virtualization, Siwei Liu, Netdev,
David Miller
In-Reply-To: <20180128150237.35237d84@xeon-e3>
On Sun, Jan 28, 2018 at 3:02 PM, Stephen Hemminger
<stephen@networkplumber.org> wrote:
> On Fri, 26 Jan 2018 18:30:03 -0800
> Jakub Kicinski <kubakici@wp.pl> wrote:
>
>> On Fri, 26 Jan 2018 15:30:35 -0800, Samudrala, Sridhar wrote:
>> > On 1/26/2018 2:47 PM, Jakub Kicinski wrote:
>> > > On Sat, 27 Jan 2018 00:14:20 +0200, Michael S. Tsirkin wrote:
>> > >> On Fri, Jan 26, 2018 at 01:46:42PM -0800, Siwei Liu wrote:
>> > >>>> and the VM is not expected to do any tuning/optimizations on the VF driver
>> > >>>> directly,
>> > >>>> i think the current patch that follows the netvsc model of 2 netdevs(virtio
>> > >>>> and vf) should
>> > >>>> work fine.
>> > >>> OK. For your use case that's fine. But that's too specific scenario
>> > >>> with lots of restrictions IMHO, perhaps very few users will benefit
>> > >>> from it, I'm not sure. If you're unwilling to move towards it, we'd
>> > >>> take this one and come back with a generic solution that is able to
>> > >>> address general use cases for VF/PT live migration .
>> > >> I think that's a fine approach. Scratch your own itch! I imagine a very
>> > >> generic virtio-switchdev providing host routing info to guests could
>> > >> address lots of usecases. A driver could bind to that one and enslave
>> > >> arbitrary other devices. Sounds reasonable.
>> > >>
>> > >> But given the fundamental idea of a failover was floated at least as
>> > >> early as 2013, and made 0 progress since precisely because it kept
>> > >> trying to address more and more features, and given netvsc is already
>> > >> using the basic solution with some success, I'm not inclined to block
>> > >> this specific effort waiting for the generic one.
>> > > I think there is an agreement that the extra netdev will be useful for
>> > > more advanced use cases, and is generally preferable. What is the
>> > > argument for not doing that from the start? If it was made I must have
>> > > missed it. Is it just unwillingness to write the extra 300 lines of
>> > > code? Sounds like a pretty weak argument when adding kernel ABI is at
>> > > stake...
>> >
>> > I am still not clear on the need for the extra netdev created by
>> > virtio_net. The only advantage i can see is that the stats can be
>> > broken between VF and virtio datapaths compared to the aggregrated
>> > stats on virtio netdev as seen with the 2 netdev approach.
>>
>> Maybe you're not convinced but multiple arguments were made.
>>
>> > With 2 netdev model, any VM image that has a working network
>> > configuration will transparently get VF based acceleration without
>> > any changes.
>>
>> Nothing happens transparently. Things may happen automatically. The
>> VF netdev doesn't disappear with netvsc. The PV netdev transforms into
>> something it did not use to be. And configures and reports some
>> information from the PV (e.g. speed) but PV doesn't pass traffic any
>> longer.
>>
>> > 3 netdev model breaks this configuration starting with the creation
>> > and naming of the 2 devices to udev needing to be aware of master and
>> > slave virtio-net devices.
>>
>> I don't understand this comment. There is one virtio-net device and
>> one "virtio-bond" netdev. And user space has to be aware of the special
>> automatic arrangement anyway, because it can't touch the VF. It
>> doesn't make any difference whether it ignores the VF or PV and VF.
>> It simply can't touch the slaves, no matter how many there are.
>>
>> > Also, from a user experience point of view, loading a virtio-net with
>> > BACKUP feature enabled will now show 2 virtio-net netdevs.
>>
>> One virtio-net and one virtio-bond, which represents what's happening.
>>
>> > For live migration with advanced usecases that Siwei is suggesting, i
>> > think we need a new driver with a new device type that can track the
>> > VF specific feature settings even when the VF driver is unloaded.
>
> I see no added value of the 3 netdev model, there is no need for a bond
> device.
I agree a full-blown bond isn't what is needed. However, just forking
traffic out from virtio to a VF doesn't really solve things either.
One of the issues as I see it is the fact that the qdisc model with
the merged device gets to be pretty ugly. There is the fact that every
packet that goes to the VF has to pass through the qdisc code twice.
There is the dual nature of the 2 netdev solution that also introduces
the potential for head-of-line blocking since the virtio could put
back pressure on the upper qdisc layer which could stall the VF
traffic when switching over. I hope we could avoid issues like that by
maintaining qdiscs per device queue instead of on an upper device that
is half software interface and half not. Ideally the virtio-bond
device could operate without a qdisc and without needing any
additional locks so there shouldn't be head of line blocking occurring
between the two interfaces and overhead could be kept minimal.
Also in the case of virtio there is support for in-driver XDP. As
Sridhar stated, when using the 2 netdev model "we cannot support XDP
in this model and it needs to be disabled". That sounds like a step
backwards instead of forwards. I would much rather leave the XDP
enabled at the lower dev level, and then if we want we can use the
generic XDP at the virtio-bond level to capture traffic on both
interfaces at the same time.
In the case of netvsc you have control of both sides of a given link
so you can match up the RSS tables, queue configuration and everything
is somewhat symmetric since you are running the PF and all the HyperV
subchannels. Most of the complexity is pushed down into the host and
your subchannel management is synchronized there if I am not mistaken.
We don't have this in the case of this virtio-bond setup. Instead a
single bit is set indicating "backup" without indicating what that
means to topology other than the fact that this virtio interface is
the backup for some other interface. We are essentially blind other
than having the link status for the VF and virtio and knowing that the
virtio is intended to be the backup.
We might be able to get to a 2 or maybe even a 1 netdev solution at
some point in the future, but I don't think that time is now. For now
a virtio-bond type solution would allow us to address most of the use
cases with minimal modification to the existing virtio and can deal
with feature and/or resource asymmetry.
- Alex
^ permalink raw reply
* Re: [PATCH net] vhost_net: stop device during reset owner
From: David Miller @ 2018-01-29 17:35 UTC (permalink / raw)
To: jasowang; +Cc: netdev, virtualization, linux-kernel, mst
In-Reply-To: <1516889032-23510-1-git-send-email-jasowang@redhat.com>
From: Jason Wang <jasowang@redhat.com>
Date: Thu, 25 Jan 2018 22:03:52 +0800
> We don't stop device before reset owner, this means we could try to
> serve any virtqueue kick before reset dev->worker. This will result a
> warn since the work was pending at llist during owner resetting. Fix
> this by stopping device during owner reset.
>
> Reported-by: syzbot+eb17c6162478cc50632c@syzkaller.appspotmail.com
> Fixes: 3a4d5c94e9593 ("vhost_net: a kernel-level virtio server")
> Signed-off-by: Jason Wang <jasowang@redhat.com>
Applied and queued up for -stable.
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Michael S. Tsirkin @ 2018-01-29 18:24 UTC (permalink / raw)
To: Alexander Duyck
Cc: Alexander Duyck, virtio-dev, Jakub Kicinski, Netdev,
virtualization, Siwei Liu, Samudrala, Sridhar, David Miller
In-Reply-To: <CAKgT0UfEHc6T4MbAwFjx4c2TK1hL-_eGDpSiBEY_ryN_4UaPoQ@mail.gmail.com>
On Sun, Jan 28, 2018 at 08:26:53PM -0800, Alexander Duyck wrote:
> >> > For live migration with advanced usecases that Siwei is suggesting, i
> >> > think we need a new driver with a new device type that can track the
> >> > VF specific feature settings even when the VF driver is unloaded.
> >
> > I see no added value of the 3 netdev model, there is no need for a bond
> > device.
>
> I agree a full-blown bond isn't what is needed. However, just forking
> traffic out from virtio to a VF doesn't really solve things either.
>
> One of the issues as I see it is the fact that the qdisc model with
> the merged device gets to be pretty ugly. There is the fact that every
> packet that goes to the VF has to pass through the qdisc code twice.
> There is the dual nature of the 2 netdev solution that also introduces
> the potential for head-of-line blocking since the virtio could put
> back pressure on the upper qdisc layer which could stall the VF
> traffic when switching over. I hope we could avoid issues like that by
> maintaining qdiscs per device queue instead of on an upper device that
> is half software interface and half not. Ideally the virtio-bond
> device could operate without a qdisc and without needing any
> additional locks so there shouldn't be head of line blocking occurring
> between the two interfaces and overhead could be kept minimal.
>
> Also in the case of virtio there is support for in-driver XDP. As
> Sridhar stated, when using the 2 netdev model "we cannot support XDP
> in this model and it needs to be disabled". That sounds like a step
> backwards instead of forwards. I would much rather leave the XDP
> enabled at the lower dev level, and then if we want we can use the
> generic XDP at the virtio-bond level to capture traffic on both
> interfaces at the same time.
I agree dropping XDP makes everything very iffy.
> In the case of netvsc you have control of both sides of a given link
> so you can match up the RSS tables, queue configuration and everything
> is somewhat symmetric since you are running the PF and all the HyperV
> subchannels. Most of the complexity is pushed down into the host and
> your subchannel management is synchronized there if I am not mistaken.
> We don't have this in the case of this virtio-bond setup. Instead a
> single bit is set indicating "backup" without indicating what that
> means to topology other than the fact that this virtio interface is
> the backup for some other interface. We are essentially blind other
> than having the link status for the VF and virtio and knowing that the
> virtio is intended to be the backup.
Would you be interested in posting at least a proof of concept
patch for this approach? It's OK if there are some TODO stubs.
It would be much easier to compare code to code rather than
a high level description to code.
> We might be able to get to a 2 or maybe even a 1 netdev solution at
> some point in the future, but I don't think that time is now. For now
> a virtio-bond type solution would allow us to address most of the use
> cases with minimal modification to the existing virtio and can deal
> with feature and/or resource asymmetry.
>
> - Alex
^ permalink raw reply
* Re: [virtio-dev] Re: [RFC PATCH net-next v2 2/2] virtio_net: Extend virtio to use VF datapath when available
From: Alexander Duyck @ 2018-01-29 20:09 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Alexander Duyck, virtio-dev, Jakub Kicinski, Netdev,
virtualization, Siwei Liu, Samudrala, Sridhar, David Miller
In-Reply-To: <20180129202113-mutt-send-email-mst@kernel.org>
On Mon, Jan 29, 2018 at 10:24 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Sun, Jan 28, 2018 at 08:26:53PM -0800, Alexander Duyck wrote:
>> >> > For live migration with advanced usecases that Siwei is suggesting, i
>> >> > think we need a new driver with a new device type that can track the
>> >> > VF specific feature settings even when the VF driver is unloaded.
>> >
>> > I see no added value of the 3 netdev model, there is no need for a bond
>> > device.
>>
>> I agree a full-blown bond isn't what is needed. However, just forking
>> traffic out from virtio to a VF doesn't really solve things either.
>>
>> One of the issues as I see it is the fact that the qdisc model with
>> the merged device gets to be pretty ugly. There is the fact that every
>> packet that goes to the VF has to pass through the qdisc code twice.
>> There is the dual nature of the 2 netdev solution that also introduces
>> the potential for head-of-line blocking since the virtio could put
>> back pressure on the upper qdisc layer which could stall the VF
>> traffic when switching over. I hope we could avoid issues like that by
>> maintaining qdiscs per device queue instead of on an upper device that
>> is half software interface and half not. Ideally the virtio-bond
>> device could operate without a qdisc and without needing any
>> additional locks so there shouldn't be head of line blocking occurring
>> between the two interfaces and overhead could be kept minimal.
>>
>> Also in the case of virtio there is support for in-driver XDP. As
>> Sridhar stated, when using the 2 netdev model "we cannot support XDP
>> in this model and it needs to be disabled". That sounds like a step
>> backwards instead of forwards. I would much rather leave the XDP
>> enabled at the lower dev level, and then if we want we can use the
>> generic XDP at the virtio-bond level to capture traffic on both
>> interfaces at the same time.
>
> I agree dropping XDP makes everything very iffy.
>
>> In the case of netvsc you have control of both sides of a given link
>> so you can match up the RSS tables, queue configuration and everything
>> is somewhat symmetric since you are running the PF and all the HyperV
>> subchannels. Most of the complexity is pushed down into the host and
>> your subchannel management is synchronized there if I am not mistaken.
>> We don't have this in the case of this virtio-bond setup. Instead a
>> single bit is set indicating "backup" without indicating what that
>> means to topology other than the fact that this virtio interface is
>> the backup for some other interface. We are essentially blind other
>> than having the link status for the VF and virtio and knowing that the
>> virtio is intended to be the backup.
>
> Would you be interested in posting at least a proof of concept
> patch for this approach? It's OK if there are some TODO stubs.
> It would be much easier to compare code to code rather than
> a high level description to code.
That is the general idea. I was hoping to go the bonding route last
week but I got too snarled up trying to untangle the features we
didn't need. I have some code I am working on but don't have an ETA
for when it will be done.
At this point I am hoping we can build something based on Sridhar's
original patch that can addresses the items I brought up and shifts to
more of a 3 netdev model. If I am not mistaken we might be able to do
it as a separate driver that has a pair of calls that allow for
adding/removing a virt-bond that is provided with a MAC address and a
device pointer so that we can do the bits necessary to get ourselves
swapped with the original virtio device and identify the virtio as the
backup channel.
Thanks.
- Alex
^ permalink raw reply
* Re: [virtio-dev] Re: [PATCH v25 2/2] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
From: Michael S. Tsirkin @ 2018-01-30 23:44 UTC (permalink / raw)
To: Wei Wang
Cc: yang.zhang.wz, virtio-dev, riel, quan.xu0, kvm, nilal,
liliang.opensource, linux-kernel, mhocko, linux-mm, pbonzini,
akpm, virtualization
In-Reply-To: <5A6AA107.3000607@intel.com>
On Fri, Jan 26, 2018 at 11:31:19AM +0800, Wei Wang wrote:
> On 01/26/2018 10:42 AM, Michael S. Tsirkin wrote:
> > On Fri, Jan 26, 2018 at 09:40:44AM +0800, Wei Wang wrote:
> > > On 01/25/2018 09:49 PM, Michael S. Tsirkin wrote:
> > > > On Thu, Jan 25, 2018 at 05:14:06PM +0800, Wei Wang wrote:
> > > >
>
> > > The controversy is that the free list is not static
> > > once the lock is dropped, so everything is dynamically changing, including
> > > the state that was recorded. The method we are using is more prudent, IMHO.
> > > How about taking the fundamental solution, and seek to improve incrementally
> > > in the future?
> > >
> > >
> > > Best,
> > > Wei
> > I'd like to see kicks happen outside the spinlock. kick with a spinlock
> > taken looks like a scalability issue that won't be easy to
> > reproduce but hurt workloads at random unexpected times.
> >
>
> Is that "kick inside the spinlock" the only concern you have? I think we can
> remove the kick actually. If we check how the host side works, it is
> worthwhile to let the host poll the virtqueue after it receives the cmd id
> from the guest (kick for cmd id isn't within the lock).
>
>
> Best,
> Wei
So really there are different ways to put free page hints to use.
The current interface requires host to do dirty tracking
for all memory, and it's more or less useless for
things like freeing host memory.
So while your project's needs seem to be addressed, I'm
still a bit disappointed that so little collaboration
happened with e.g. Nitesh's project, to the point where
you don't even CC him on patches.
So I'm kind of trying to bridge this a bit - I would
like the interfaces that we build to at least superficially
look like they might be reusable for other uses of hinting.
Imagine that you don't have dirty tracking on the host.
What would it take to still use hinting information,
e.g. to call MADV_FREE on the pages guest gives us?
I think you need to kick and you need to wait for
host to consume the hint before page is reused.
And we know madvise takes a lot of time sometimes,
so locking out the free list does not sound like a
good idea.
That's why I was talking about kick out of lock,
so that eventually we can reuse that for hinting
and actually wait for an interrupt.
So how about we take a bunch of pages out of the free list, move them to
the balloon, kick (and optionally wait for host to consume), them move
them back? Preferably to end of the list? This will also make things
like sorting them much easier as you can just put them in a binary tree
or something.
For when we need to be careful to make sure we don't
create an OOM situation with this out of thin air,
and for when you can't give everything to host in one go,
you might want some kind of notifier that tells you
that you need to return pages to the free list ASAP.
How'd this sound?
--
MST
^ permalink raw reply
* [vhost:vhost 11/22] drivers//firmware/qemu_fw_cfg.c:379:23: error: 'VMCOREINFO_NOTE_SIZE' undeclared; did you mean 'MEI_CL_NAME_SIZE'?
From: kbuild test robot @ 2018-02-01 7:55 UTC (permalink / raw)
To: Marc-André Lureau
Cc: kvm, Michael S. Tsirkin, netdev, Gabriel Somlo, virtualization,
kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4177 bytes --]
tree: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git vhost
head: 9991b95cb712cdd9a40240bde6274d5415476fb5
commit: 860c7fa9d6d433011e82a4f7f896893e914ce4a9 [11/22] fw_cfg: write vmcoreinfo details
config: i386-randconfig-c0-02011330 (attached as .config)
compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
reproduce:
git checkout 860c7fa9d6d433011e82a4f7f896893e914ce4a9
# save the attached .config to linux build tree
make ARCH=i386
All error/warnings (new ones prefixed by >>):
In file included from include/linux/byteorder/little_endian.h:5:0,
from arch/x86/include/uapi/asm/byteorder.h:5,
from include/asm-generic/bitops/le.h:6,
from arch/x86/include/asm/bitops.h:518,
from include/linux/bitops.h:38,
from include/linux/kernel.h:11,
from include/linux/list.h:9,
from include/linux/module.h:9,
from drivers//firmware/qemu_fw_cfg.c:30:
drivers//firmware/qemu_fw_cfg.c: In function 'write_vmcoreinfo':
>> drivers//firmware/qemu_fw_cfg.c:379:23: error: 'VMCOREINFO_NOTE_SIZE' undeclared (first use in this function); did you mean 'MEI_CL_NAME_SIZE'?
.size = cpu_to_le32(VMCOREINFO_NOTE_SIZE),
^
include/uapi/linux/byteorder/little_endian.h:33:51: note: in definition of macro '__cpu_to_le32'
#define __cpu_to_le32(x) ((__force __le32)(__u32)(x))
^
>> drivers//firmware/qemu_fw_cfg.c:379:11: note: in expansion of macro 'cpu_to_le32'
.size = cpu_to_le32(VMCOREINFO_NOTE_SIZE),
^~~~~~~~~~~
drivers//firmware/qemu_fw_cfg.c:379:23: note: each undeclared identifier is reported only once for each function it appears in
.size = cpu_to_le32(VMCOREINFO_NOTE_SIZE),
^
include/uapi/linux/byteorder/little_endian.h:33:51: note: in definition of macro '__cpu_to_le32'
#define __cpu_to_le32(x) ((__force __le32)(__u32)(x))
^
>> drivers//firmware/qemu_fw_cfg.c:379:11: note: in expansion of macro 'cpu_to_le32'
.size = cpu_to_le32(VMCOREINFO_NOTE_SIZE),
^~~~~~~~~~~
>> drivers//firmware/qemu_fw_cfg.c:380:24: error: implicit declaration of function 'paddr_vmcoreinfo_note'; did you mean 'write_vmcoreinfo'? [-Werror=implicit-function-declaration]
.paddr = cpu_to_le64(paddr_vmcoreinfo_note())
^
include/uapi/linux/byteorder/little_endian.h:31:51: note: in definition of macro '__cpu_to_le64'
#define __cpu_to_le64(x) ((__force __le64)(__u64)(x))
^
>> drivers//firmware/qemu_fw_cfg.c:380:12: note: in expansion of macro 'cpu_to_le64'
.paddr = cpu_to_le64(paddr_vmcoreinfo_note())
^~~~~~~~~~~
cc1: some warnings being treated as errors
vim +379 drivers//firmware/qemu_fw_cfg.c
360
361 #ifdef CONFIG_CRASH_CORE
362 static ssize_t write_vmcoreinfo(struct device *dev, const struct fw_cfg_file *f)
363 {
364 struct vmci {
365 __le16 host_format;
366 __le16 guest_format;
367 __le32 size;
368 __le64 paddr;
369 } __packed;
370 static struct vmci *data;
371 ssize_t ret;
372
373 data = kmalloc(sizeof(struct vmci), GFP_KERNEL);
374 if (!data)
375 return -ENOMEM;
376
377 *data = (struct vmci) {
378 .guest_format = cpu_to_le16(VMCOREINFO_FORMAT_ELF),
> 379 .size = cpu_to_le32(VMCOREINFO_NOTE_SIZE),
> 380 .paddr = cpu_to_le64(paddr_vmcoreinfo_note())
381 };
382 /* spare ourself reading host format support for now since we
383 * don't know what else to format - host may ignore ours
384 */
385 ret = fw_cfg_write_blob(dev, f->select, data, 0, sizeof(struct vmci));
386
387 kfree(data);
388 return ret;
389 }
390 #endif /* CONFIG_CRASH_CORE */
391
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32076 bytes --]
[-- Attachment #3: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [virtio-dev] Re: [PATCH v25 2/2] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
From: Wei Wang @ 2018-02-01 9:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: yang.zhang.wz@gmail.com, virtio-dev@lists.oasis-open.org,
riel@redhat.com, quan.xu0@gmail.com, kvm@vger.kernel.org,
nilal@redhat.com, liliang.opensource@gmail.com,
linux-kernel@vger.kernel.org, mhocko@kernel.org,
linux-mm@kvack.org, pbonzini@redhat.com,
akpm@linux-foundation.org,
virtualization@lists.linux-foundation.org
In-Reply-To: <20180131011423-mutt-send-email-mst@kernel.org>
On 01/31/2018 07:44 AM, Michael S. Tsirkin wrote:
> On Fri, Jan 26, 2018 at 11:31:19AM +0800, Wei Wang wrote:
>> On 01/26/2018 10:42 AM, Michael S. Tsirkin wrote:
>>> On Fri, Jan 26, 2018 at 09:40:44AM +0800, Wei Wang wrote:
>>>> On 01/25/2018 09:49 PM, Michael S. Tsirkin wrote:
>>>>> On Thu, Jan 25, 2018 at 05:14:06PM +0800, Wei Wang wrote:
>>>>>
>>>> The controversy is that the free list is not static
>>>> once the lock is dropped, so everything is dynamically changing, including
>>>> the state that was recorded. The method we are using is more prudent, IMHO.
>>>> How about taking the fundamental solution, and seek to improve incrementally
>>>> in the future?
>>>>
>>>>
>>>> Best,
>>>> Wei
>>> I'd like to see kicks happen outside the spinlock. kick with a spinlock
>>> taken looks like a scalability issue that won't be easy to
>>> reproduce but hurt workloads at random unexpected times.
>>>
>> Is that "kick inside the spinlock" the only concern you have? I think we can
>> remove the kick actually. If we check how the host side works, it is
>> worthwhile to let the host poll the virtqueue after it receives the cmd id
>> from the guest (kick for cmd id isn't within the lock).
>>
>>
>> Best,
>> Wei
> So really there are different ways to put free page hints to use.
>
> The current interface requires host to do dirty tracking
> for all memory, and it's more or less useless for
> things like freeing host memory.
>
> So while your project's needs seem to be addressed, I'm
> still a bit disappointed that so little collaboration
> happened with e.g. Nitesh's project, to the point where
> you don't even CC him on patches.
Isn't "nilal@redhat.com" Nitesh? Actually it's been cc-ed long time ago.
I think we should at least see the performance numbers and a working
prototype from them (I remember they lack the host side implementation).
Btw, this feature is requested by many customers of Linux (not our own
project's need). They want to use this feature to optimize their *live
migration*. Hope the community could understand our need.
> So I'm kind of trying to bridge this a bit - I would
> like the interfaces that we build to at least superficially
> look like they might be reusable for other uses of hinting.
>
> Imagine that you don't have dirty tracking on the host.
> What would it take to still use hinting information,
> e.g. to call MADV_FREE on the pages guest gives us?
>
> I think you need to kick and you need to wait for
> host to consume the hint before page is reused.
> And we know madvise takes a lot of time sometimes,
> so locking out the free list does not sound like a
> good idea.
>
> That's why I was talking about kick out of lock,
> so that eventually we can reuse that for hinting
> and actually wait for an interrupt.
>
> So how about we take a bunch of pages out of the free list, move them to
> the balloon, kick (and optionally wait for host to consume), them move
> them back? Preferably to end of the list? This will also make things
> like sorting them much easier as you can just put them in a binary tree
> or something.
>
> For when we need to be careful to make sure we don't
> create an OOM situation with this out of thin air,
> and for when you can't give everything to host in one go,
> you might want some kind of notifier that tells you
> that you need to return pages to the free list ASAP.
>
> How'd this sound?
>
I think the above is a duplicate function of ballooning, though there
are some differences. Please see below my concerns and different thoughts:
1) From the previous discussion, the only acceptable method to get pages
from mm is to do alloc() (btw, we are not getting pages in this patch,
we are getting hints). The above sounds like we are going to take pages
from the free list without mm's awareness. I'm not sure if you would be
ready to convince the mm folks that this idea is allowed.
2) If the guest has 8G free memory, how much can virtio-balloon take
with the above method? For example, if virtio-balloon only takes 1G,
with 7G left in mm. The next moment, it is possible that something comes
out and needs to use 7.5GB. I think it is barely possible to ensure that
the amount of memory we take to virtio-balloon won't affect the system.
3) Hints means the pages are quite likely to be free pages (no
guarantee). If the pages given to host are going to be freed, then we
really couldn't call them hints, they are true free pages. Ballooning
needs true free pages, while live migration needs hints, would you agree
with this? From the perspective of features, they are two different
features, and should be gated with two feature bits and separated
implementations. Mixing them would cause many unexpected issues (e.g.
the case when the two features function at the same time)
4) If we want to add another function of ballooning, how is this better
than the existing ballooning? The difference I can see is the current
ballooning takes free pages via alloc(), while the above hacks into the
free page list.
Best,
Wei
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox