Re: Elvis upstreaming plan

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Nov 27, 2013 at 04:00:53PM +0100, Stefan Hajnoczi wrote:
> On Wed, Nov 27, 2013 at 09:43:33AM +0200, Joel Nider wrote:
> > Hi,
> > 
> > Razya is out for a few days, so I will try to answer the questions as well
> > as I can:
> > 
> > "Michael S. Tsirkin" <mst@redhat.com> wrote on 26/11/2013 11:11:57 PM:
> > 
> > > From: "Michael S. Tsirkin" <mst@redhat.com>
> > > To: Abel Gordon/Haifa/IBM@IBMIL,
> > > Cc: Anthony Liguori <anthony@codemonkey.ws>, abel.gordon@gmail.com,
> > > asias@redhat.com, digitaleric@google.com, Eran Raichstein/Haifa/
> > > IBM@IBMIL, gleb@redhat.com, jasowang@redhat.com, Joel Nider/Haifa/
> > > IBM@IBMIL, kvm@vger.kernel.org, pbonzini@redhat.com, Razya Ladelsky/
> > > Haifa/IBM@IBMIL
> > > Date: 27/11/2013 01:08 AM
> > > Subject: Re: Elvis upstreaming plan
> > >
> > > On Tue, Nov 26, 2013 at 08:53:47PM +0200, Abel Gordon wrote:
> > > >
> > > >
> > > > Anthony Liguori <anthony@codemonkey.ws> wrote on 26/11/2013 08:05:00
> > PM:
> > > >
> > > > >
> > > > > Razya Ladelsky <RAZYA@il.ibm.com> writes:
> > > > >
> > <edit>
> > > >
> > > > That's why we are proposing to implement a mechanism that will enable
> > > > the management stack to configure 1 thread per I/O device (as it is
> > today)
> > > > or 1 thread for many I/O devices (belonging to the same VM).
> > > >
> > > > > Once you are scheduling multiple guests in a single vhost device, you
> > > > > now create a whole new class of DoS attacks in the best case
> > scenario.
> > > >
> > > > Again, we are NOT proposing to schedule multiple guests in a single
> > > > vhost thread. We are proposing to schedule multiple devices belonging
> > > > to the same guest in a single (or multiple) vhost thread/s.
> > > >
> > >
> > > I guess a question then becomes why have multiple devices?
> > 
> > If you mean "why serve multiple devices from a single thread" the answer is
> > that we cannot rely on the Linux scheduler which has no knowledge of I/O
> > queues to do a decent job of scheduling I/O.  The idea is to take over the
> > I/O scheduling responsibilities from the kernel's thread scheduler with a
> > more efficient I/O scheduler inside each vhost thread.  So by combining all
> > of the I/O devices from the same guest (disks, network cards, etc) in a
> > single I/O thread, it allows us to provide better scheduling by giving us
> > more knowledge of the nature of the work.  So now instead of relying on the
> > linux scheduler to perform context switches between multiple vhost threads,
> > we have a single thread context in which we can do the I/O scheduling more
> > efficiently.  We can closely monitor the performance needs of each queue of
> > each device inside the vhost thread which gives us much more information
> > than relying on the kernel's thread scheduler.
> 
> And now there are 2 performance-critical pieces that need to be
> optimized/tuned instead of just 1:
> 
> 1. Kernel infrastructure that QEMU and vhost use today but you decided
> to bypass.
> 2. The new ELVIS code which only affects vhost devices in the same VM.
> 
> If you split the code paths it results in more effort in the long run
> and the benefit seems quite limited once you acknowledge that isolation
> is important.
>
> Isn't the sane thing to do taking lessons from ELVIS improving existing
> pieces instead of bypassing them?  That way both the single VM and
> host-wide performance improves.  And as a bonus non-virtualization use
> cases may also benefit.
> 
> Stefan

I'm not sure about that. elvis is all about specific behaviour
patterns that are virtualization specific, and general claims
that we can improve scheduler for all workloads seem somewhat
optimistic.

-- 
MST