[PULL] virtio

[PULL] virtio

[Rusty Russell]

The following changes since commit f7da9cdf45cbbad5029d4858dcbc0134e06084ed:

  Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net (2012-07-28 06:00:39 -0700)

are available in the git repository at:


  git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux-2.6-for-linus.git tags/virtio-for-linus

for you to fetch changes up to 6a743897144500fb4c4566ced3a498d5180fbb5b:

  virtio-blk: return VIRTIO_BLK_F_FLUSH to header. (2012-07-30 13:30:52 +0930)

----------------------------------------------------------------
Virtio patches, mainly hotplugging fixes.

----------------------------------------------------------------
Amit Shah (5):
      virtio ids: fix comment for virtio-rng
      virtio: rng: allow tasks to be killed that are waiting for rng input
      virtio: rng: don't wait on host when module is going away
      virtio: rng: split out common code in probe / remove for s3/s4 ops
      virtio: rng: s3/s4 support

Asias He (3):
      virtio-blk: Call del_gendisk() before disable guest kick
      virtio-blk: Reset device after blk_cleanup_queue()
      virtio-blk: Use block layer provided spinlock

Paolo Bonzini (1):
      virtio-blk: allow toggling host cache between writeback and writethrough

Rusty Russell (1):
      virtio-blk: return VIRTIO_BLK_F_FLUSH to header.

 drivers/block/virtio_blk.c          |  115 ++++++++++++++++++++++++++++-------
 drivers/char/hw_random/virtio-rng.c |   37 ++++++++++-
 include/linux/virtio_blk.h          |   10 ++-
 include/linux/virtio_ids.h          |    2 +-
 4 files changed, 137 insertions(+), 27 deletions(-)

[PULL] virtio

[Rusty Russell]

The following changes since commit 925a6f0bf8bd122d5d2429af7f0ca0fecf4ae71f:

  Merge tag 'hwspinlock-3.6-fix' of git://git.kernel.org/pub/scm/linux/kernel/git/ohad/hwspinlock (2012-09-18 11:58:54 -0700)

are available in the git repository at:


  git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux.git virtio-next

for you to fetch changes up to ca16f580a5db7e60bfafe59a50bb133bd3347491:

  lguest: fix occasional crash in example launcher. (2012-10-04 12:12:59 +0930)

----------------------------------------------------------------
New workflow: same git trees pulled by linux-next get sent straight to Linus.
Git is awkward at shuffling patches compared with quilt or mq, but that doesn't
happen often once things get into my -next branch.

----------------------------------------------------------------
Alexey Khoroshilov (1):
      virtio: console: fix error handling in init() function

Asias He (3):
      virtio-blk: Add bio-based IO path for virtio-blk
      virtio-blk: Add REQ_FLUSH and REQ_FUA support to bio path
      virtio-blk: Disable callback in virtblk_done()

Brian Foley (2):
      virtio_mmio: fix off by one error allocating queue
      virtio_mmio: Don't attempt to create empty virtqueues

Dan Carpenter (1):
      virtio-blk: fix NULL checking in virtblk_alloc_req()

Jason Wang (2):
      virtio-ring: move queue_index to vring_virtqueue
      virtio: introduce an API to set affinity for a virtqueue

Masami Hiramatsu (5):
      virtio/console: Add splice_write support
      virtio/console: Add a failback for unstealable pipe buffer
      virtio/console: Wait until the port is ready on splice
      ftrace: Allow stealing pages from pipe buffer
      virtio/console: Allocate scatterlist according to the current pipe size

Michael S. Tsirkin (3):
      virtio-balloon: dependency fix
      virtio: support reserved vqs
      virtio: don't crash when device is buggy

Peter Senna Tschudin (1):
      drivers/virtio/virtio_pci.c: fix error return code

Rusty Russell (4):
      virtio_balloon: not EXPERIMENTAL any more.
      virtio: add help to CONFIG_VIRTIO option.
      virtio: remove CONFIG_VIRTIO_RING
      lguest: fix occasional crash in example launcher.

Yoshihiro YUNOMAE (2):
      tools: Add guest trace agent as a user tool
      tools: Fix pthread flag for Makefile of trace-agent used by virtio-trace

 arch/s390/Kconfig                           |    1 -
 arch/x86/lguest/Kconfig                     |    1 -
 drivers/block/virtio_blk.c                  |  306 +++++++++++++++++++++++----
 drivers/char/virtio_console.c               |  210 ++++++++++++++++--
 drivers/lguest/lguest_device.c              |    5 +-
 drivers/remoteproc/remoteproc_virtio.c      |    5 +-
 drivers/rpmsg/Kconfig                       |    1 -
 drivers/s390/kvm/kvm_virtio.c               |    5 +-
 drivers/virtio/Kconfig                      |   17 +-
 drivers/virtio/Makefile                     |    3 +-
 drivers/virtio/virtio.c                     |    2 +-
 drivers/virtio/virtio_mmio.c                |   29 ++-
 drivers/virtio/virtio_pci.c                 |   68 +++++-
 drivers/virtio/virtio_ring.c                |   14 +-
 include/linux/virtio.h                      |    2 +
 include/linux/virtio_config.h               |   23 ++
 include/linux/virtio_ring.h                 |    3 +-
 kernel/trace/trace.c                        |    8 +-
 tools/lguest/lguest.c                       |    1 +
 tools/virtio/virtio-trace/Makefile          |   13 ++
 tools/virtio/virtio-trace/README            |  118 +++++++++++
 tools/virtio/virtio-trace/trace-agent-ctl.c |  137 ++++++++++++
 tools/virtio/virtio-trace/trace-agent-rw.c  |  192 +++++++++++++++++
 tools/virtio/virtio-trace/trace-agent.c     |  270 +++++++++++++++++++++++
 tools/virtio/virtio-trace/trace-agent.h     |   75 +++++++
 25 files changed, 1402 insertions(+), 107 deletions(-)
 create mode 100644 tools/virtio/virtio-trace/Makefile
 create mode 100644 tools/virtio/virtio-trace/README
 create mode 100644 tools/virtio/virtio-trace/trace-agent-ctl.c
 create mode 100644 tools/virtio/virtio-trace/trace-agent-rw.c
 create mode 100644 tools/virtio/virtio-trace/trace-agent.c
 create mode 100644 tools/virtio/virtio-trace/trace-agent.h

Re: [PULL] virtio

[Michael S. Tsirkin]

On Tue, Nov 01, 2011 at 10:39:08AM +1030, Rusty Russell wrote:
>  * [new tag]         rusty@rustcorp.com.au-v3.1-7196-gac5be1e -> rusty@rustcorp.com.au-v3.1-7196-gac5be1e
> 
> The following changes since commit 839d8810747bbf39e0a5a7f223b67bffa7945f8d:
> 
>   Merge branch 'i2c-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging (2011-10-30 15:54:59 -0700)
> 
> are available in the git repository at:
> 
>   git://github.com/rustyrussell/linux.git master
> 
> Alexey Kardashevskiy (1):
>       virtio-pci: Use PCI MMIO instead of PIO when available

I missed this one - wasn't Cc'd neither me, kvm or virtio mailing lists.

It's well known that mmio is much slower than pio on kvm, since
mmio needs to be emulated to get at the address.
So I'm expecting this will cause a performance regression.
IMO we should keep using PIO for VQ and interrupt status access
if PIO is available.

Another consideration is that in an attempt to pack data
densely in the PIO space the layout became messy.
It would be better to have common config space and
per-device config space in separate pages, possibly
with padding between them.

So I'd like a bit more discussion on this patch,
I'm concerned that if this is released in 3.2 as is we'll
have to support this forever. How about a revert for now?


> Amit Shah (8):
>       virtio: console: Fix indentation
>       virtio: console: Ignore port name update request if name already set
>       virtio: console: Use wait_event_freezable instead of _interruptible
>       virtio: console: Fix return type for get_inbuf()
>       virtio: console: make get_inbuf() return port->inbuf if present
>       virtio: console: rename variable
>       virtio: console: make discard_port_data() use get_inbuf()
>       virtio: console: add port stats for bytes received, sent and discarded
> 
> Christian Borntraeger (1):
>       virtio: console: wait for first console port for early console output
> 
> Krishna Kumar (1):
>       virtio: Dont add "config" to list for !per_vq_vector
> 
> Michael S. Tsirkin (1):
>       virtio-blk: use ida to allocate disk index
> 
> Pawel Moll (1):
>       virtio: Add platform bus driver for memory mapped virtio device
> 
> Sasha Levin (3):
>       virtio-console: Use virtio_config_val() for retrieving config
>       virtio_config: Add virtio_config_val_len()
>       virtio-net: Use virtio_config_val() for retrieving config
> 
> Wang Sheng-Hui (2):
>       virtio.h: correct comment for struct virtio_driver
>       virtio: modify vring_init and vring_size to take account of the layout containing *_event_idx
> 
>  Documentation/devicetree/bindings/virtio/mmio.txt |   17 +
>  drivers/block/virtio_blk.c                        |   30 +-
>  drivers/char/virtio_console.c                     |  120 ++++--
>  drivers/net/virtio_net.c                          |   14 +-
>  drivers/virtio/Kconfig                            |   11 +
>  drivers/virtio/Makefile                           |    1 +
>  drivers/virtio/virtio_mmio.c                      |  479 +++++++++++++++++++++
>  drivers/virtio/virtio_pci.c                       |   20 +-
>  include/linux/virtio.h                            |    4 +-
>  include/linux/virtio_config.h                     |    3 +
>  include/linux/virtio_mmio.h                       |  111 +++++
>  include/linux/virtio_ring.h                       |    6 +-
>  12 files changed, 749 insertions(+), 67 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/virtio/mmio.txt
>  create mode 100644 drivers/virtio/virtio_mmio.c
>  create mode 100644 include/linux/virtio_mmio.h

Re: [PULL] virtio

[Sasha Levin]

On Tue, 2011-11-01 at 13:45 +0200, Michael S. Tsirkin wrote:
> On Tue, Nov 01, 2011 at 10:39:08AM +1030, Rusty Russell wrote:
> >  * [new tag]         rusty@rustcorp.com.au-v3.1-7196-gac5be1e -> rusty@rustcorp.com.au-v3.1-7196-gac5be1e
> > 
> > The following changes since commit 839d8810747bbf39e0a5a7f223b67bffa7945f8d:
> > 
> >   Merge branch 'i2c-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging (2011-10-30 15:54:59 -0700)
> > 
> > are available in the git repository at:
> > 
> >   git://github.com/rustyrussell/linux.git master
> > 
> > Alexey Kardashevskiy (1):
> >       virtio-pci: Use PCI MMIO instead of PIO when available
> 
> I missed this one - wasn't Cc'd neither me, kvm or virtio mailing lists.
> 
> It's well known that mmio is much slower than pio on kvm, since
> mmio needs to be emulated to get at the address.
> So I'm expecting this will cause a performance regression.
> IMO we should keep using PIO for VQ and interrupt status access
> if PIO is available.
> 
> Another consideration is that in an attempt to pack data
> densely in the PIO space the layout became messy.
> It would be better to have common config space and
> per-device config space in separate pages, possibly
> with padding between them.
> 
> So I'd like a bit more discussion on this patch,
> I'm concerned that if this is released in 3.2 as is we'll
> have to support this forever. How about a revert for now?

Another thing, the patch tries to map BAR 2 and use it as the
configuration space.

It's both not documented properly anywhere, and is not fully backwards
compatible - we currently use BAR 2 as part of our MSIX handling in the
kvm tool and I'm sure we're not the only ones to assume virtio-pci only
uses BAR 0.

A proper solution would be for example a configuration in the PIO config
space which points to the MMIO BAR to use instead.

Unless Michael pointed this patch out, it would have broken (at least)
the kvm tool in a non obvious way which would require a rather long
session of 'git bisect' to figure out whats wrong.

-- 

Sasha.

Re: [PULL] virtio

[Michael S. Tsirkin]

On Tue, Nov 01, 2011 at 02:33:33PM +0200, Sasha Levin wrote:
> On Tue, 2011-11-01 at 13:45 +0200, Michael S. Tsirkin wrote:
> > On Tue, Nov 01, 2011 at 10:39:08AM +1030, Rusty Russell wrote:
> > >  * [new tag]         rusty@rustcorp.com.au-v3.1-7196-gac5be1e -> rusty@rustcorp.com.au-v3.1-7196-gac5be1e
> > > 
> > > The following changes since commit 839d8810747bbf39e0a5a7f223b67bffa7945f8d:
> > > 
> > >   Merge branch 'i2c-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging (2011-10-30 15:54:59 -0700)
> > > 
> > > are available in the git repository at:
> > > 
> > >   git://github.com/rustyrussell/linux.git master
> > > 
> > > Alexey Kardashevskiy (1):
> > >       virtio-pci: Use PCI MMIO instead of PIO when available
> > 
> > I missed this one - wasn't Cc'd neither me, kvm or virtio mailing lists.
> > 
> > It's well known that mmio is much slower than pio on kvm, since
> > mmio needs to be emulated to get at the address.
> > So I'm expecting this will cause a performance regression.
> > IMO we should keep using PIO for VQ and interrupt status access
> > if PIO is available.
> > 
> > Another consideration is that in an attempt to pack data
> > densely in the PIO space the layout became messy.
> > It would be better to have common config space and
> > per-device config space in separate pages, possibly
> > with padding between them.
> > 
> > So I'd like a bit more discussion on this patch,
> > I'm concerned that if this is released in 3.2 as is we'll
> > have to support this forever. How about a revert for now?
> 
> Another thing, the patch tries to map BAR 2 and use it as the
> configuration space.
> 
> It's both not documented properly anywhere, and is not fully backwards
> compatible - we currently use BAR 2 as part of our MSIX handling in the
> kvm tool and I'm sure we're not the only ones to assume virtio-pci only
> uses BAR 0.
> 
> A proper solution would be for example a configuration in the PIO config
> space which points to the MMIO BAR to use instead.

I think it makes sense to put the configuration in PCI
configuration space, using vendor-specific capability.
This way we can reuse existing functionality for scanning
capability lists.

> Unless Michael pointed this patch out, it would have broken (at least)
> the kvm tool in a non obvious way which would require a rather long
> session of 'git bisect' to figure out whats wrong.
> 
> -- 
> 
> Sasha.

Re: [PULL] virtio

[Sasha Levin]

On Tue, 2011-11-01 at 14:42 +0200, Michael S. Tsirkin wrote:
> On Tue, Nov 01, 2011 at 02:33:33PM +0200, Sasha Levin wrote:
> > On Tue, 2011-11-01 at 13:45 +0200, Michael S. Tsirkin wrote:
> > > On Tue, Nov 01, 2011 at 10:39:08AM +1030, Rusty Russell wrote:
> > > >  * [new tag]         rusty@rustcorp.com.au-v3.1-7196-gac5be1e -> rusty@rustcorp.com.au-v3.1-7196-gac5be1e
> > > > 
> > > > The following changes since commit 839d8810747bbf39e0a5a7f223b67bffa7945f8d:
> > > > 
> > > >   Merge branch 'i2c-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jdelvare/staging (2011-10-30 15:54:59 -0700)
> > > > 
> > > > are available in the git repository at:
> > > > 
> > > >   git://github.com/rustyrussell/linux.git master
> > > > 
> > > > Alexey Kardashevskiy (1):
> > > >       virtio-pci: Use PCI MMIO instead of PIO when available
> > > 
> > > I missed this one - wasn't Cc'd neither me, kvm or virtio mailing lists.
> > > 
> > > It's well known that mmio is much slower than pio on kvm, since
> > > mmio needs to be emulated to get at the address.
> > > So I'm expecting this will cause a performance regression.
> > > IMO we should keep using PIO for VQ and interrupt status access
> > > if PIO is available.
> > > 
> > > Another consideration is that in an attempt to pack data
> > > densely in the PIO space the layout became messy.
> > > It would be better to have common config space and
> > > per-device config space in separate pages, possibly
> > > with padding between them.
> > > 
> > > So I'd like a bit more discussion on this patch,
> > > I'm concerned that if this is released in 3.2 as is we'll
> > > have to support this forever. How about a revert for now?
> > 
> > Another thing, the patch tries to map BAR 2 and use it as the
> > configuration space.
> > 
> > It's both not documented properly anywhere, and is not fully backwards
> > compatible - we currently use BAR 2 as part of our MSIX handling in the
> > kvm tool and I'm sure we're not the only ones to assume virtio-pci only
> > uses BAR 0.
> > 
> > A proper solution would be for example a configuration in the PIO config
> > space which points to the MMIO BAR to use instead.
> 
> I think it makes sense to put the configuration in PCI
> configuration space, using vendor-specific capability.
> This way we can reuse existing functionality for scanning
> capability lists.

Yup, I agree. It would also allow dropping the PIO config BAR altogether
after some period of backwards compatibility.

It was mostly to point out that the patch isn't really backwards
compatible :)

-- 

Sasha.

Re: [PULL] virtio

[Rusty Russell]

On Tue, 1 Nov 2011 14:42:23 +0200, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Tue, Nov 01, 2011 at 02:33:33PM +0200, Sasha Levin wrote:
> > A proper solution would be for example a configuration in the PIO config
> > space which points to the MMIO BAR to use instead.
> 
> I think it makes sense to put the configuration in PCI
> configuration space, using vendor-specific capability.
> This way we can reuse existing functionality for scanning
> capability lists.

Patch welcome; first one to implement it becomes virtio PCI maintainer!
I'll keep maintaining the spec, since it covers much more than the PCI
mechanics, but everything I know about PCI I learn from these
discussions.

Meanwhile, I've dropped this patch; mea culpa.

> > Unless Michael pointed this patch out, it would have broken (at least)
> > the kvm tool in a non obvious way which would require a rather long
> > session of 'git bisect' to figure out whats wrong.

I assumed not, until QEMU provided the second BAR it would simply fall
back.  Or am I completely misunderstanding?

Thanks,
Rusty.

Re: [PULL] virtio

[Sasha Levin]

On Wed, 2011-11-02 at 11:39 +1030, Rusty Russell wrote:
> On Tue, 1 Nov 2011 14:42:23 +0200, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Tue, Nov 01, 2011 at 02:33:33PM +0200, Sasha Levin wrote:
> > > Unless Michael pointed this patch out, it would have broken (at least)
> > > the kvm tool in a non obvious way which would require a rather long
> > > session of 'git bisect' to figure out whats wrong.
> 
> I assumed not, until QEMU provided the second BAR it would simply fall
> back.  Or am I completely misunderstanding?

QEMU uses two bars within each virtio-pci device. First one (BAR 0) is
the PIO config BAR, and the second one (BAR 1) is the MSI-X BAR used for
the MSI-X table and the PBA.

Since the PCI spec allows you to have the MSI-X table and the PBA in
completely different regions pointed by two different BARs, we did so in
kvm tools and have BAR 1 pointing to the MSI-X table, and BAR 2 pointing
to the PBA ( - we *already* provided the second BAR and used it for
something else).

This means that with this patch, we'd load BAR 2 and use it as the
config space, instead of the PBA table which it really represents.

-- 

Sasha.

Re: [PULL] virtio

[Rusty Russell]

On Wed, 02 Nov 2011 06:52:44 +0200, Sasha Levin <levinsasha928@gmail.com> wrote:
> Since the PCI spec allows you to have the MSI-X table and the PBA in
> completely different regions pointed by two different BARs, we did so in
> kvm tools and have BAR 1 pointing to the MSI-X table, and BAR 2 pointing
> to the PBA ( - we *already* provided the second BAR and used it for
> something else).

Got it!  Thanks for the explanation.

Cheers,
Rusty.