[Qemu-devel] dataplane, thread and gpu stuff

[Qemu-devel] dataplane, thread and gpu stuff

[Dave Airlie]

Hi,

So after talking to a few people at kvm forum I think the GPU code
should probably use the dataplane stuff from the outset,

The main advantages I think this gives me is being able to dequeue
objects from the vq from a thread and send irq vectors from there as
well.

Though since it appears the dataplane stuff is kvm specific (at least
the irq handling), I was wondering how I should deal with fallbacks
for non-kvm operation, and quite how much falling back I need to do.

Can I still use the dataplane/vring code from the normal bottom half
handlers or do I have to write separate code for both situations.

Dave.

Re: [Qemu-devel] dataplane, thread and gpu stuff

[Stefan Hajnoczi]

On Mon, Nov 18, 2013 at 02:52:53PM +1000, Dave Airlie wrote:
> So after talking to a few people at kvm forum I think the GPU code
> should probably use the dataplane stuff from the outset,
> 
> The main advantages I think this gives me is being able to dequeue
> objects from the vq from a thread and send irq vectors from there as
> well.
> 
> Though since it appears the dataplane stuff is kvm specific (at least
> the irq handling), I was wondering how I should deal with fallbacks
> for non-kvm operation, and quite how much falling back I need to do.
> 
> Can I still use the dataplane/vring code from the normal bottom half
> handlers or do I have to write separate code for both situations.

As of today, there are still two vring implementations in
hw/virtio/virtio.c and hw/virtio/dataplane/vring.c.  This means it isn't
clean and easy to integrate into a new device yet.  Existing dataplane
devices basically take advantage of the fact that the non-dataplane
version sets up the device before I/O.

Paolo can give you details on the latest thread-safe memory API stuff
and whether it's already usable for virtio.

Regarding irqfd, we could emulate it in TCG using an EventNotifier
(eventfd).  At that point I think it's no longer kvm-specific.

Stefan

Re: [Qemu-devel] dataplane, thread and gpu stuff

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

On Nov 18, 2013 7:05 AM, "Stefan Hajnoczi" <stefanha@gmail.com> wrote:
>
> On Mon, Nov 18, 2013 at 02:52:53PM +1000, Dave Airlie wrote:
> > So after talking to a few people at kvm forum I think the GPU code
> > should probably use the dataplane stuff from the outset,
> >
> > The main advantages I think this gives me is being able to dequeue
> > objects from the vq from a thread and send irq vectors from there as
> > well.
> >
> > Though since it appears the dataplane stuff is kvm specific (at least
> > the irq handling), I was wondering how I should deal with fallbacks
> > for non-kvm operation, and quite how much falling back I need to do.
> >
> > Can I still use the dataplane/vring code from the normal bottom half
> > handlers or do I have to write separate code for both situations.
>
> As of today, there are still two vring implementations in
> hw/virtio/virtio.c and hw/virtio/dataplane/vring.c.  This means it isn't
> clean and easy to integrate into a new device yet.  Existing dataplane
> devices basically take advantage of the fact that the non-dataplane
> version sets up the device before I/O.

I think we also need some form of mdroth's GContext prior to introducing
more dataplane devices.  Sticking every device in a seperate thread with no
way to control who is where can actually hurt performance.  I think we
really need to have a M-N device thread model too.

Regards,

Anthony Liguori

>
> Paolo can give you details on the latest thread-safe memory API stuff
> and whether it's already usable for virtio.
>
> Regarding irqfd, we could emulate it in TCG using an EventNotifier
> (eventfd).  At that point I think it's no longer kvm-specific.
>
> Stefan
>

[-- Attachment #2: Type: text/html, Size: 2132 bytes --]

Re: [Qemu-devel] dataplane, thread and gpu stuff

[Paolo Bonzini]

Il 18/11/2013 16:03, Stefan Hajnoczi ha scritto:
> As of today, there are still two vring implementations in
> hw/virtio/virtio.c and hw/virtio/dataplane/vring.c.  This means it isn't
> clean and easy to integrate into a new device yet.  Existing dataplane
> devices basically take advantage of the fact that the non-dataplane
> version sets up the device before I/O.
> 
> Paolo can give you details on the latest thread-safe memory API stuff
> and whether it's already usable for virtio.

vring and virtio are still separate even in my latest patches.  I have
patches to convert vring to use memory_region_find instead of hostmem.

> Regarding irqfd, we could emulate it in TCG using an EventNotifier
> (eventfd).  At that point I think it's no longer kvm-specific.

I think he's talking about ioeventfd though.

Paolo

Re: [Qemu-devel] dataplane, thread and gpu stuff

[Stefan Hajnoczi]

On Mon, Nov 18, 2013 at 05:37:20PM +0100, Paolo Bonzini wrote:
> Il 18/11/2013 16:03, Stefan Hajnoczi ha scritto:
> > As of today, there are still two vring implementations in
> > hw/virtio/virtio.c and hw/virtio/dataplane/vring.c.  This means it isn't
> > clean and easy to integrate into a new device yet.  Existing dataplane
> > devices basically take advantage of the fact that the non-dataplane
> > version sets up the device before I/O.
> > 
> > Paolo can give you details on the latest thread-safe memory API stuff
> > and whether it's already usable for virtio.
> 
> vring and virtio are still separate even in my latest patches.  I have
> patches to convert vring to use memory_region_find instead of hostmem.
> 
> > Regarding irqfd, we could emulate it in TCG using an EventNotifier
> > (eventfd).  At that point I think it's no longer kvm-specific.
> 
> I think he's talking about ioeventfd though.

ioeventfd can be emulated using EventNotifier too, although it depends
on the handler function whether or not it's worth deferring it to
another thread.

Stefan

Re: [Qemu-devel] dataplane, thread and gpu stuff

[Stefan Hajnoczi]

On Mon, Nov 18, 2013 at 08:18:47AM -0800, Anthony Liguori wrote:
> On Nov 18, 2013 7:05 AM, "Stefan Hajnoczi" <stefanha@gmail.com> wrote:
> >
> > On Mon, Nov 18, 2013 at 02:52:53PM +1000, Dave Airlie wrote:
> > > So after talking to a few people at kvm forum I think the GPU code
> > > should probably use the dataplane stuff from the outset,
> > >
> > > The main advantages I think this gives me is being able to dequeue
> > > objects from the vq from a thread and send irq vectors from there as
> > > well.
> > >
> > > Though since it appears the dataplane stuff is kvm specific (at least
> > > the irq handling), I was wondering how I should deal with fallbacks
> > > for non-kvm operation, and quite how much falling back I need to do.
> > >
> > > Can I still use the dataplane/vring code from the normal bottom half
> > > handlers or do I have to write separate code for both situations.
> >
> > As of today, there are still two vring implementations in
> > hw/virtio/virtio.c and hw/virtio/dataplane/vring.c.  This means it isn't
> > clean and easy to integrate into a new device yet.  Existing dataplane
> > devices basically take advantage of the fact that the non-dataplane
> > version sets up the device before I/O.
> 
> I think we also need some form of mdroth's GContext prior to introducing
> more dataplane devices.  Sticking every device in a seperate thread with no
> way to control who is where can actually hurt performance.  I think we
> really need to have a M-N device thread model too.

Yes, I agree.  We need the concept of multiple event loops (QContext).

Stefan