* Re: [v3,11/41] mips: reuse asm-generic/barrier.h
From: Paul E. McKenney @ 2016-01-27 23:32 UTC (permalink / raw)
To: Will Deacon
Cc: linux-mips, linux-ia64, Michael S. Tsirkin, Peter Zijlstra,
virtualization, H. Peter Anvin, sparclinux, Ingo Molnar,
linux-arch, linux-s390, Russell King - ARM Linux,
user-mode-linux-devel, linux-sh, Michael Ellerman, x86, xen-devel,
Ingo Molnar, linux-xtensa, james.hogan, Arnd Bergmann,
Stefano Stabellini, adi-buildroot-devel, Leonid Yegoshin,
ddaney.cavm, Thomas Gleixner, linux-metag
In-Reply-To: <20160127102546.GD2390@arm.com>
On Wed, Jan 27, 2016 at 10:25:46AM +0000, Will Deacon wrote:
> On Tue, Jan 26, 2016 at 11:58:20AM -0800, Paul E. McKenney wrote:
> > On Tue, Jan 26, 2016 at 12:16:09PM +0000, Will Deacon wrote:
> > > On Mon, Jan 25, 2016 at 10:03:22PM -0800, Paul E. McKenney wrote:
> > > > On Mon, Jan 25, 2016 at 04:42:43PM +0000, Will Deacon wrote:
> > > > > On Fri, Jan 15, 2016 at 01:58:53PM -0800, Paul E. McKenney wrote:
> > > > > > PPC Overlapping Group-B sets version 4
> > > > > > ""
> > > > > > (* When the Group-B sets from two different barriers involve instructions in
> > > > > > the same thread, within that thread one set must contain the other.
> > > > > >
> > > > > > P0 P1 P2
> > > > > > Rx=1 Wy=1 Wz=2
> > > > > > dep. lwsync lwsync
> > > > > > Ry=0 Wz=1 Wx=1
> > > > > > Rz=1
> > > > > >
> > > > > > assert(!(z=2))
> > > > > >
> > > > > > Forbidden by ppcmem, allowed by herd.
> > > > > > *)
> > > > > > {
> > > > > > 0:r1=x; 0:r2=y; 0:r3=z;
> > > > > > 1:r1=x; 1:r2=y; 1:r3=z; 1:r4=1;
> > > > > > 2:r1=x; 2:r2=y; 2:r3=z; 2:r4=1; 2:r5=2;
> > > > > > }
> > > > > > P0 | P1 | P2 ;
> > > > > > lwz r6,0(r1) | stw r4,0(r2) | stw r5,0(r3) ;
> > > > > > xor r7,r6,r6 | lwsync | lwsync ;
> > > > > > lwzx r7,r7,r2 | stw r4,0(r3) | stw r4,0(r1) ;
> > > > > > lwz r8,0(r3) | | ;
> > > > > >
> > > > > > exists
> > > > > > (z=2 /\ 0:r6=1 /\ 0:r7=0 /\ 0:r8=1)
> > > > >
> > > > > That really hurts. Assuming that the "assert(!(z=2))" is actually there
> > > > > to constrain the coherence order of z to be {0->1->2}, then I think that
> > > > > this test is forbidden on arm using dmb instead of lwsync. That said, I
> > > > > also don't think the Rz=1 in P0 changes anything.
> > > >
> > > > What about the smp_wmb() variant of dmb that orders only stores?
> > >
> > > Tricky, but I think it still works out if the coherence order of z is as
> > > I described above. The line of reasoning is weird though -- I ended up
> > > considering the two cases where P0 reads z before and after it reads x
> > > and what that means for the read of y.
> >
> > By "works out" you mean that ARM prohibits the outcome?
>
> Yes, that's my understanding.
Very good, we have agreement between the two architectures, then. ;-)
Thanx, Paul
^ permalink raw reply
* Re: [PATCH] documentation: Add disclaimer
From: Paul E. McKenney @ 2016-01-27 23:35 UTC (permalink / raw)
To: David Howells
Cc: linux-mips, linux-ia64, Michael S. Tsirkin, Peter Zijlstra,
Will Deacon, virtualization, H. Peter Anvin, sparclinux,
Ingo Molnar, linux-arch, linux-s390, Russell King - ARM Linux,
user-mode-linux-devel, linux-sh, Michael Ellerman, x86, xen-devel,
Ingo Molnar, linux-xtensa, james.hogan, Arnd Bergmann,
Stefano Stabellini, adi-buildroot-devel, Leonid Yegoshin,
ddaney.cavm, Thomas Gleixner
In-Reply-To: <15882.1453906627@warthog.procyon.org.uk>
On Wed, Jan 27, 2016 at 02:57:07PM +0000, David Howells wrote:
> Peter Zijlstra <peterz@infradead.org> wrote:
>
> > +==========
> > +DISCLAIMER
> > +==========
> > +
> > +This document is not a specification; it is intentionally (for the sake of
> > +brevity) and unintentionally (due to being human) incomplete. This document is
> > +meant as a guide to using the various memory barriers provided by Linux, but
> > +in case of any doubt (and there are many) please ask.
> > +
> > +I repeat, this document is not a specification of what Linux expects from
> > +hardware.
>
> The purpose of this document is twofold:
>
> (1) to specify the minimum functionality that one can rely on for any
> particular barrier, and
>
> (2) to provide a guide as to how to use the barriers that are available.
>
> Note that an architecture can provide more than the minimum requirement for
> any particular barrier, but if the barrier provides less than that, it is
> incorrect.
>
> Note also that it is possible that a barrier may be a no-op for an
> architecture because the way that arch works renders an explicit barrier
> unnecessary in that case.
>
> > +
>
> Can you bung an extra blank line in here if you have to redo this at all?
>
> > +========
> > +CONTENTS
> > +========
> >
> > (*) Abstract memory access model.
Good point! Would you be willing to add a Signed-off-by so I
can take the combined change, assuming Peter and Will are good
with it?
Thanx, Paul
^ permalink raw reply
* Re: [v3,11/41] mips: reuse asm-generic/barrier.h
From: Leonid Yegoshin @ 2016-01-28 0:48 UTC (permalink / raw)
To: Maciej W. Rozycki
Cc: linux-mips, linux-ia64, Michael S. Tsirkin, Peter Zijlstra,
Will Deacon, virtualization, H. Peter Anvin, sparclinux,
Ingo Molnar, linux-arch, linux-s390, Russell King - ARM Linux,
user-mode-linux-devel, linux-sh, Michael Ellerman, x86, xen-devel,
Ingo Molnar, paulmck, linux-xtensa, james.hogan, Arnd Bergmann,
Stefano Stabellini, adi-buildroot-devel, ddaney.cavm,
Thomas Gleixner, linux-me
In-Reply-To: <alpine.DEB.2.00.1601271116520.5958@tp.orcam.me.uk>
[-- Attachment #1.1: Type: text/plain, Size: 1357 bytes --]
On 01/27/2016 03:26 AM, Maciej W. Rozycki wrote:
> On Fri, 15 Jan 2016, Leonid Yegoshin wrote:
>
>>> So you need to build a different kernel for some types of MIPS systems?
>>> Or do you do boot-time rewriting, like a number of other arches do?
>> I don't know. I would like to have responses. Ralf asked Maciej about old
>> systems and that came nowhere. Even rewrite - don't know what to do with that:
>> no lightweight SYNC or no SYNC at all - yes, it is still possible that SYNC on
>> some systems can be too heavy or even harmful, nobody tested that.
> I don't recall being asked;
In http://patchwork.linux-mips.org/patch/10505/ the very last mesg
exchange is:
Maciej,
do you have an R4000 / R4600 / R5000 / R7000 / SiByte system at hand to
test this?
...
Ralf
Maciej W. Rozycki
<http://patchwork.linux-mips.org/project/linux-mips/list/?submitter=79>
- June 5, 2015, 9:18 p.m.
On Fri, 5 Jun 2015, Ralf Baechle wrote:
> do you have an R4000 / R4600 / R5000 / R7000 / SiByte system at hand to
> test this?
I should be able to check R4400 (that is virtually the same as R4000)
next week or so. As to SiByte -- not before next month I'm afraid. I
don't have access to any of the other processors you named. You may
want to find a better person if you want to accept this change soon.
Maciej
... and that stops forever...
- Leonid.
[-- Attachment #1.2: Type: text/html, Size: 2237 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [v3,11/41] mips: reuse asm-generic/barrier.h
From: Leonid Yegoshin @ 2016-01-28 0:58 UTC (permalink / raw)
To: Maciej W. Rozycki
Cc: linux-mips, linux-ia64, Michael S. Tsirkin, Peter Zijlstra,
Will Deacon, virtualization, H. Peter Anvin, sparclinux,
Ingo Molnar, linux-arch, linux-s390, Russell King - ARM Linux,
user-mode-linux-devel, linux-sh, Michael Ellerman, x86, xen-devel,
Ingo Molnar, paulmck, linux-xtensa, james.hogan, Arnd Bergmann,
Stefano Stabellini, adi-buildroot-devel, ddaney.cavm,
Thomas Gleixner, linux-me
In-Reply-To: <alpine.DEB.2.00.1601271116520.5958@tp.orcam.me.uk>
On 01/27/2016 03:26 AM, Maciej W. Rozycki wrote:
> On Fri, 15 Jan 2016, Leonid Yegoshin wrote:
>
>>> So you need to build a different kernel for some types of MIPS systems?
>>> Or do you do boot-time rewriting, like a number of other arches do?
>> I don't know. I would like to have responses. Ralf asked Maciej about old
>> systems and that came nowhere. Even rewrite - don't know what to do with that:
>> no lightweight SYNC or no SYNC at all - yes, it is still possible that SYNC on
>> some systems can be too heavy or even harmful, nobody tested that.
> I don't recall being asked; mind that I might not get to messages I have
> not been cc-ed in a timely manner and I may miss some altogether. With
> the amount of mailing list traffic that passes by me my scanner may fail
> to trigger. Sorry if this causes anybody trouble, but such is life.
>
> Coincidentally, I have just posted some notes on SYNC in a different
> thread, see <http://lkml.iu.edu/hypermail/linux/kernel/1601.3/03080.html>.
> There's a reference to an older message of mine there too. I hope this
> answers your questions.
>
> Maciej
In http://patchwork.linux-mips.org/patch/10505/the very last mesg
exchange is:
Maciej,
do you have an R4000 / R4600 / R5000 / R7000 / SiByte system at hand to
test this?
...
Ralf
Maciej W. Rozycki- June 5, 2015, 9:18 p.m.
On Fri, 5 Jun 2015, Ralf Baechle wrote:
> do you have an R4000 / R4600 / R5000 / R7000 / SiByte system at hand to
> test this?
I should be able to check R4400 (that is virtually the same as R4000)
next week or so. As to SiByte -- not before next month I'm afraid. I
don't have access to any of the other processors you named. You may
want to find a better person if you want to accept this change soon.
Maciej
... and that stops forever...
- Leonid.
^ permalink raw reply
* Re: [PULL] virtio: fixes, tests
From: Junio C Hamano @ 2016-01-28 6:26 UTC (permalink / raw)
To: Linus Torvalds
Cc: jmarchan, KVM list, Michael S. Tsirkin, Network Development,
Kamal Mostafa, Linux Kernel Mailing List, virtualization,
Sasha Levin
In-Reply-To: <CA+55aFzC6mopLkYYvuZNK8W510MpqgSuYwhjAAM-XeuAdEfs3A@mail.gmail.com>
Linus Torvalds <torvalds@linux-foundation.org> writes:
> On Wed, Jan 27, 2016 at 2:15 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> Anyway, pulled. Just curious about how that thing happened.
>>
>> That's because apparently diffstat obeys orderfile rules:
>
> Ugh. I guess that makes sense, but it's still very annoying for
> something like a pull request, where now different people end up
> having different diffstats. And the reason I never noticed it is that
> likely there aren't that many people who use an orderfile.
>
> I guess something like "-O /dev/null" in the pull-request would undo
> it, but it is a bit annoying.
>
> I've never actually met anybody (knowingly) that used that option. I
> thought it was a Junio-only use case (it's been around forever as a
> command line option, but the config file entry seems to be somewhat
> recent and I wasn't even aware of it).
>
> Adding Junio just as background to see what he thinks. Looks like the
> diff.orderfile config option hits not just porcelain, but plumbing
> too.
Note: without enough context, I am guessing that you are annoyed
that diffstat graph in a pull request did not match what you got
locally after pulling.
I've never actually met anybody who uses the orderfile, either. And
I tend to agree that it probably is a mistake if the configuration
is not limited to diff_ui_config().
Wait. It _is_ limited to diff_ui_config(). The thing is, "git
request-pull" does use the Porcelain "git diff" to emit the final
diffstat. So there is nothing to fix in diff.c, at least ;-)
I am of two minds about the configuration affecting the output from
"git request-pull" command as a whole. In a sense, "request-pull"
command itself _is_ a UI level thing, i.e. Porcelain, and if a
project chooses to standardize on a certain patch/diff presentation
order using the diff.orderfile facility, there should be a way to
tell your project participants to use the same order when they send
in their patches and their pull requests, but with the hardcoded
"git diff --stat" invocation at the end of "git request-pull", such
a per-project customization must happen via _some_ configuration
variable. We _could_ make up request-pull.difforderfile that is
read by that script and gets turned into "-O $file" arguments to the
"git diff" invocation, but if projects really wants to standardize
on a single patch/diff presentation order, I do not think such a
split configuration buys us anything. We'd be better off using a
single diff.orderfile and have it consistently honoured by tools
everybody who participates in the project uses.
You obviously can declare "With MY project, the standard order in
which you must present patch/diff is with this empty file", and tell
your project participants to set diff.orderfile to /dev/null in the
repository config when they work on the kernel. That would be what
any such hypothetical project would do, when it has a specific
preference of the patch/diff presentation order. After all, you do
have a specific preference, which is "do not futz with the order in
which Git gives the paths by default", so...
^ permalink raw reply
* Re: [RFC v4 0/5] Add virtio transport for AF_VSOCK
From: Stefan Hajnoczi @ 2016-01-28 15:19 UTC (permalink / raw)
To: kvm
Cc: Michael S. Tsirkin, netdev, virtualization, Matt Benjamin,
Christoffer Dall, matt.ma
In-Reply-To: <1450775258-18287-1-git-send-email-stefanha@redhat.com>
[-- Attachment #1.1: Type: text/plain, Size: 5793 bytes --]
On Tue, Dec 22, 2015 at 05:07:33PM +0800, Stefan Hajnoczi wrote:
> This series is based on v4.4-rc2 and the "virtio: make find_vqs()
> checkpatch.pl-friendly" patch I recently submitted.
>
> v4:
> * Addressed code review comments from Alex Bennee
> * MAINTAINERS file entries for new files
> * Trace events instead of pr_debug()
> * RST packet is sent when there is no listen socket
> * Allow guest->host connections again (began discussing netfilter support with
> Matt Benjamin instead of hard-coding security policy in virtio-vsock code)
> * Many checkpatch.pl cleanups (will be 100% clean in v5)
>
> v3:
> * Remove unnecessary 3-way handshake, just do REQUEST/RESPONSE instead
> of REQUEST/RESPONSE/ACK
> * Remove SOCK_DGRAM support and focus on SOCK_STREAM first
> (also drop v2 Patch 1, it's only needed for SOCK_DGRAM)
> * Only allow host->guest connections (same security model as latest
> VMware)
> * Don't put vhost vsock driver into staging
> * Add missing Kconfig dependencies (Arnd Bergmann <arnd@arndb.de>)
> * Remove unneeded variable used to store return value
> (Fengguang Wu <fengguang.wu@intel.com> and Julia Lawall
> <julia.lawall@lip6.fr>)
>
> v2:
> * Rebased onto Linux v4.4-rc2
> * vhost: Refuse to assign reserved CIDs
> * vhost: Refuse guest CID if already in use
> * vhost: Only accept correctly addressed packets (no spoofing!)
> * vhost: Support flexible rx/tx descriptor layout
> * vhost: Add missing total_tx_buf decrement
> * virtio_transport: Fix total_tx_buf accounting
> * virtio_transport: Add virtio_transport global mutex to prevent races
> * common: Notify other side of SOCK_STREAM disconnect (fixes shutdown
> semantics)
> * common: Avoid recursive mutex_lock(tx_lock) for write_space (fixes deadlock)
> * common: Define VIRTIO_VSOCK_TYPE_STREAM/DGRAM hardware interface constants
> * common: Define VIRTIO_VSOCK_SHUTDOWN_RCV/SEND hardware interface constants
> * common: Fix peer_buf_alloc inheritance on child socket
>
> This patch series adds a virtio transport for AF_VSOCK (net/vmw_vsock/).
> AF_VSOCK is designed for communication between virtual machines and
> hypervisors. It is currently only implemented for VMware's VMCI transport.
>
> This series implements the proposed virtio-vsock device specification from
> here:
> http://permalink.gmane.org/gmane.comp.emulators.virtio.devel/980
>
> Most of the work was done by Asias He and Gerd Hoffmann a while back. I have
> picked up the series again.
>
> The QEMU userspace changes are here:
> https://github.com/stefanha/qemu/commits/vsock
>
> Why virtio-vsock?
> -----------------
> Guest<->host communication is currently done over the virtio-serial device.
> This makes it hard to port sockets API-based applications and is limited to
> static ports.
>
> virtio-vsock uses the sockets API so that applications can rely on familiar
> SOCK_STREAM semantics. Applications on the host can easily connect to guest
> agents because the sockets API allows multiple connections to a listen socket
> (unlike virtio-serial). This simplifies the guest<->host communication and
> eliminates the need for extra processes on the host to arbitrate virtio-serial
> ports.
>
> Overview
> --------
> This series adds 3 pieces:
>
> 1. virtio_transport_common.ko - core virtio vsock code that uses vsock.ko
>
> 2. virtio_transport.ko - guest driver
>
> 3. drivers/vhost/vsock.ko - host driver
>
> Howto
> -----
> The following kernel options are needed:
> CONFIG_VSOCKETS=y
> CONFIG_VIRTIO_VSOCKETS=y
> CONFIG_VIRTIO_VSOCKETS_COMMON=y
> CONFIG_VHOST_VSOCK=m
>
> Launch QEMU as follows:
> # qemu ... -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3
>
> Guest and host can communicate via AF_VSOCK sockets. The host's CID (address)
> is 2 and the guest must be assigned a CID (3 in the example above).
>
> Status
> ------
> This patch series implements the latest draft specification. Please review.
>
> Asias He (4):
> VSOCK: Introduce virtio_vsock_common.ko
> VSOCK: Introduce virtio_transport.ko
> VSOCK: Introduce vhost_vsock.ko
> VSOCK: Add Makefile and Kconfig
>
> Stefan Hajnoczi (1):
> VSOCK: transport-specific vsock_transport functions
>
> MAINTAINERS | 13 +
> drivers/vhost/Kconfig | 15 +
> drivers/vhost/Makefile | 4 +
> drivers/vhost/vsock.c | 607 +++++++++++++++
> drivers/vhost/vsock.h | 4 +
> include/linux/virtio_vsock.h | 167 +++++
> include/net/af_vsock.h | 3 +
> .../trace/events/vsock_virtio_transport_common.h | 144 ++++
> include/uapi/linux/virtio_ids.h | 1 +
> include/uapi/linux/virtio_vsock.h | 87 +++
> net/vmw_vsock/Kconfig | 19 +
> net/vmw_vsock/Makefile | 2 +
> net/vmw_vsock/af_vsock.c | 9 +
> net/vmw_vsock/virtio_transport.c | 481 ++++++++++++
> net/vmw_vsock/virtio_transport_common.c | 834 +++++++++++++++++++++
> 15 files changed, 2390 insertions(+)
> create mode 100644 drivers/vhost/vsock.c
> create mode 100644 drivers/vhost/vsock.h
> create mode 100644 include/linux/virtio_vsock.h
> create mode 100644 include/trace/events/vsock_virtio_transport_common.h
> create mode 100644 include/uapi/linux/virtio_vsock.h
> create mode 100644 net/vmw_vsock/virtio_transport.c
> create mode 100644 net/vmw_vsock/virtio_transport_common.c
Michael and Jason: Ping
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* [PATCH v5 0/5] x86: faster smp_mb()+documentation tweaks
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
the arch/x86 maintainers, virtualization, Borislav Petkov,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Ingo Molnar
mb() typically uses mfence on modern x86, but a micro-benchmark shows that it's
2 to 3 times slower than lock; addl that we use on older CPUs.
So we really should use the locked variant everywhere, except that intel manual
says that clflush is only ordered by mfence, so we can't.
Note: some callers of clflush seems to assume sfence will
order it, so there could be existing bugs around this code.
Fortunately no callers of clflush (except one) order it using smp_mb(), so
after fixing that one caller, it seems safe to override smp_mb straight away.
Down the road, it might make sense to introduce clflush_mb() and switch
to that for clflush callers.
While I was at it, I found some inconsistencies in comments in
arch/x86/include/asm/barrier.h
The documentation fixes are included first - I verified that
they do not change the generated code at all. Borislav Petkov
said they will appear in tip eventually, included here for
completeness.
The last patch changes __smp_mb() to lock addl. I was unable to
measure a speed difference on a macro benchmark,
but I noted that even doing
#define mb() barrier()
seems to make no difference for most benchmarks
(it causes hangs sometimes, of course).
Lightly tested on my laptop.
HPA asked that the last patch is deferred until we hear back from
intel, which makes sense of course. So it needs HPA's ack.
Changes from v4:
Fix up the 64 bit version.
Changes from v3:
Leave mb() alone for now since it's used to order
clflush, which requires mfence. Optimize smp_mb instead.
Changes from v2:
add patch adding cc clobber for addl
tweak commit log for patch 2
use addl at SP-4 (as opposed to SP) to reduce data dependencies
Michael S. Tsirkin (5):
x86: add cc clobber for addl
x86: drop a comment left over from X86_OOSTORE
x86: tweak the comment about use of wmb for IO
x86: use mb() around clflush
x86: drop mfence in favor of lock+addl
arch/x86/include/asm/barrier.h | 21 ++++++++++++---------
arch/x86/kernel/process.c | 4 ++--
2 files changed, 14 insertions(+), 11 deletions(-)
--
MST
^ permalink raw reply
* [PATCH v5 1/5] x86: add cc clobber for addl
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
Andrey Konovalov, the arch/x86 maintainers, virtualization,
Ingo Molnar, Borislav Petkov, Borislav Petkov, Andy Lutomirski,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Ingo Molnar
In-Reply-To: <1453921746-16178-1-git-send-email-mst@redhat.com>
addl clobbers flags (such as CF) but barrier.h didn't tell this to gcc.
Historically, gcc doesn't need one on x86, and always considers flags
clobbered. We are probably missing the cc clobber in a *lot* of places
for this reason.
But even if not necessary, it's probably a good thing to add for
documentation, and in case gcc semantcs ever change.
Reported-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
arch/x86/include/asm/barrier.h | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index a584e1c..a65bdb1 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -15,9 +15,12 @@
* Some non-Intel clones support out of order store. wmb() ceases to be a
* nop for these.
*/
-#define mb() alternative("lock; addl $0,0(%%esp)", "mfence", X86_FEATURE_XMM2)
-#define rmb() alternative("lock; addl $0,0(%%esp)", "lfence", X86_FEATURE_XMM2)
-#define wmb() alternative("lock; addl $0,0(%%esp)", "sfence", X86_FEATURE_XMM)
+#define mb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "mfence", \
+ X86_FEATURE_XMM2) ::: "memory", "cc")
+#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "lfence", \
+ X86_FEATURE_XMM2) ::: "memory", "cc")
+#define wmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "sfence", \
+ X86_FEATURE_XMM2) ::: "memory", "cc")
#else
#define mb() asm volatile("mfence":::"memory")
#define rmb() asm volatile("lfence":::"memory")
--
MST
^ permalink raw reply related
* [PATCH v5 2/5] x86: drop a comment left over from X86_OOSTORE
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
Andrey Konovalov, the arch/x86 maintainers, virtualization,
Ingo Molnar, Borislav Petkov, Borislav Petkov, Andy Lutomirski,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Ingo Molnar
In-Reply-To: <1453921746-16178-1-git-send-email-mst@redhat.com>
The comment about wmb being non-nop to deal with non-intel CPUs is a
left over from before commit 09df7c4c8097 ("x86: Remove
CONFIG_X86_OOSTORE").
It makes no sense now: in particular, wmb is not a nop even for regular
intel CPUs because of weird use-cases e.g. dealing with WC memory.
Drop this comment.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
arch/x86/include/asm/barrier.h | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index a65bdb1..a291745 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -11,10 +11,6 @@
*/
#ifdef CONFIG_X86_32
-/*
- * Some non-Intel clones support out of order store. wmb() ceases to be a
- * nop for these.
- */
#define mb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "mfence", \
X86_FEATURE_XMM2) ::: "memory", "cc")
#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "lfence", \
--
MST
^ permalink raw reply related
* [PATCH v5 3/5] x86: tweak the comment about use of wmb for IO
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
Andrey Konovalov, the arch/x86 maintainers, virtualization,
Ingo Molnar, Borislav Petkov, Borislav Petkov, Andy Lutomirski,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Ingo Molnar
In-Reply-To: <1453921746-16178-1-git-send-email-mst@redhat.com>
On x86, we *do* still use the non-nop rmb/wmb for IO barriers, but even
that is generally questionable.
Leave them around as historial unless somebody can point to a case where
they care about the performance, but tweak the comment so people
don't think they are strictly required in all cases.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
arch/x86/include/asm/barrier.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index a291745..bfb28ca 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -6,7 +6,7 @@
/*
* Force strict CPU ordering.
- * And yes, this is required on UP too when we're talking
+ * And yes, this might be required on UP too when we're talking
* to devices.
*/
--
MST
^ permalink raw reply related
* [PATCH v5 4/5] x86: use mb() around clflush
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Len Brown, Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
the arch/x86 maintainers, Oleg Nesterov, virtualization,
Mike Galbraith, Ingo Molnar, Borislav Petkov, Andy Lutomirski,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Ingo Molnar
In-Reply-To: <1453921746-16178-1-git-send-email-mst@redhat.com>
commit f8e617f4582995f7c25ef25b4167213120ad122b ("sched/idle/x86:
Optimize unnecessary mwait_idle() resched IPIs") adds
memory barriers around clflush, but this seems wrong
for UP since barrier() has no effect on clflush.
We really want mfence so switch to mb() instead.
Cc: Mike Galbraith <bitbucket@online.de>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
arch/x86/kernel/process.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 9f7c21c..9decee2 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -418,9 +418,9 @@ static void mwait_idle(void)
if (!current_set_polling_and_test()) {
trace_cpu_idle_rcuidle(1, smp_processor_id());
if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR)) {
- smp_mb(); /* quirk */
+ mb(); /* quirk */
clflush((void *)¤t_thread_info()->flags);
- smp_mb(); /* quirk */
+ mb(); /* quirk */
}
__monitor((void *)¤t_thread_info()->flags, 0, 0);
--
MST
^ permalink raw reply related
* [PATCH v5 5/5] x86: drop mfence in favor of lock+addl
From: Michael S. Tsirkin @ 2016-01-28 17:02 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds
Cc: Davidlohr Bueso, Davidlohr Bueso, Peter Zijlstra,
Andrey Konovalov, the arch/x86 maintainers, virtualization,
Andy Lutomirski, Borislav Petkov, Borislav Petkov,
Andy Lutomirski, H. Peter Anvin, Thomas Gleixner,
Paul E. McKenney, Ingo Molnar, Ingo Molnar
In-Reply-To: <1453921746-16178-1-git-send-email-mst@redhat.com>
mfence appears to be way slower than a locked instruction - let's use
lock+add unconditionally, as we always did on old 32-bit.
Just poking at SP would be the most natural, but if we
then read the value from SP, we get a false dependency
which will slow us down.
This was noted in this article:
http://shipilev.net/blog/2014/on-the-fence-with-dependencies/
And is easy to reproduce by sticking a barrier in a small non-inline
function.
So let's use a negative offset - which avoids this problem since we
build with the red zone disabled.
Unfortunately there's some code that wants to order clflush instructions
using mb(), so we can't replace that - but smp_mb should be safe
to replace.
Update mb/rmb/wmb on 32 bit to use the negative offset, too, for
consistency.
Suggested-by: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
arch/x86/include/asm/barrier.h | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h
index bfb28ca..3c6ba1e 100644
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -11,11 +11,11 @@
*/
#ifdef CONFIG_X86_32
-#define mb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "mfence", \
+#define mb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "mfence", \
X86_FEATURE_XMM2) ::: "memory", "cc")
-#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "lfence", \
+#define rmb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "lfence", \
X86_FEATURE_XMM2) ::: "memory", "cc")
-#define wmb() asm volatile(ALTERNATIVE("lock; addl $0,0(%%esp)", "sfence", \
+#define wmb() asm volatile(ALTERNATIVE("lock; addl $0,-4(%%esp)", "sfence", \
X86_FEATURE_XMM2) ::: "memory", "cc")
#else
#define mb() asm volatile("mfence":::"memory")
@@ -30,7 +30,11 @@
#endif
#define dma_wmb() barrier()
-#define __smp_mb() mb()
+#ifdef CONFIG_X86_32
+#define __smp_mb() asm volatile("lock; addl $0,-4(%%esp)" ::: "memory", "cc")
+#else
+#define __smp_mb() asm volatile("lock; addl $0,-4(%%rsp)" ::: "memory", "cc")
+#endif
#define __smp_rmb() dma_rmb()
#define __smp_wmb() barrier()
#define __smp_store_mb(var, value) do { (void)xchg(&var, value); } while (0)
--
MST
^ permalink raw reply related
* Re: [PATCH v5 4/5] x86: use mb() around clflush
From: Peter Zijlstra @ 2016-01-28 18:25 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Len Brown, Davidlohr Bueso, Davidlohr Bueso, Mike Galbraith,
the arch/x86 maintainers, Oleg Nesterov, linux-kernel,
virtualization, Ingo Molnar, Borislav Petkov, Andy Lutomirski,
H. Peter Anvin, Thomas Gleixner, Paul E. McKenney, Linus Torvalds,
Ingo Molnar
In-Reply-To: <1453921746-16178-5-git-send-email-mst@redhat.com>
On Thu, Jan 28, 2016 at 07:02:51PM +0200, Michael S. Tsirkin wrote:
> commit f8e617f4582995f7c25ef25b4167213120ad122b ("sched/idle/x86:
> Optimize unnecessary mwait_idle() resched IPIs") adds
> memory barriers around clflush, but this seems wrong
> for UP since barrier() has no effect on clflush.
> We really want mfence so switch to mb() instead.
>
> Cc: Mike Galbraith <bitbucket@online.de>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
^ permalink raw reply
* Re: [PATCH] documentation: Add disclaimer
From: David Howells @ 2016-01-28 20:02 UTC (permalink / raw)
To: paulmck
Cc: linux-mips, linux-ia64, Michael S. Tsirkin, Peter Zijlstra,
Will Deacon, virtualization, dhowells, H. Peter Anvin, sparclinux,
Ingo Molnar, linux-arch, linux-s390, Russell King - ARM Linux,
user-mode-linux-devel, linux-sh, Michael Ellerman, x86, xen-devel,
Ingo Molnar, linux-xtensa, james.hogan, Arnd Bergmann,
Stefano Stabellini, adi-buildroot-devel, Leonid Yegoshin,
ddaney.cavm, Thomas
In-Reply-To: <20160127233504.GP4503@linux.vnet.ibm.com>
Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> Good point! Would you be willing to add a Signed-off-by so I
> can take the combined change, assuming Peter and Will are good
> with it?
Sure!
David
^ permalink raw reply
* [PATCH v5 00/10] virtio DMA API, yet again
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
This switches virtio to use the DMA API on Xen and if requested by
module option.
This fixes virtio on Xen, and it should break anything because it's
off by default on everything except Xen PV on x86.
To the Xen people: is this okay? If it doesn't work on other Xen
variants (PVH? HVM?), can you submit follow-up patches to fix it?
To everyone else: we've waffled on this for way too long. I think
we should to get DMA API implementation in with a conservative
policy like this rather than waiting until we achieve perfection.
I'm tired of carrying these patches around.
Michael, if these survive review, can you stage these in your tree?
Can you also take a look at tools/virtio? I probably broke it, but I
couldn't get it to build without these patches either, so I'm stuck.
Changes from v4:
- Bake vring_use_dma_api in from the beginning.
- Automatically enable only on Xen.
- Add module parameter.
- Add s390 and alpha DMA API implementations.
- Rebase to 4.5-rc1.
Changes from v3:
- More big-endian fixes.
- Added better virtio-ring APIs that handle allocation and use them in
virtio-mmio and virtio-pci.
- Switch to Michael's virtio-net patch.
Changes from v2:
- Fix vring_mapping_error incorrect argument
Changes from v1:
- Fix an endian conversion error causing a BUG to hit.
- Fix a DMA ordering issue (swiotlb=force works now).
- Minor cleanups.
Andy Lutomirski (7):
vring: Introduce vring_use_dma_api()
virtio_ring: Support DMA APIs
virtio: Add improved queue allocation API
virtio_mmio: Use the DMA API if enabled
virtio_pci: Use the DMA API if enabled
vring: Use the DMA API on Xen
vring: Add a module parameter to force-enable the DMA API
Christian Borntraeger (3):
dma: Provide simple noop dma ops
alpha/dma: use common noop dma ops
s390/dma: Allow per device dma ops
arch/alpha/kernel/pci-noop.c | 46 +---
arch/s390/Kconfig | 6 +-
arch/s390/include/asm/device.h | 6 +-
arch/s390/include/asm/dma-mapping.h | 6 +-
arch/s390/pci/pci.c | 1 +
arch/s390/pci/pci_dma.c | 4 +-
drivers/virtio/Kconfig | 2 +-
drivers/virtio/virtio_mmio.c | 67 ++----
drivers/virtio/virtio_pci_common.h | 6 -
drivers/virtio/virtio_pci_legacy.c | 42 ++--
drivers/virtio/virtio_pci_modern.c | 61 ++----
drivers/virtio/virtio_ring.c | 412 ++++++++++++++++++++++++++++++------
include/linux/dma-mapping.h | 2 +
include/linux/virtio.h | 23 +-
include/linux/virtio_ring.h | 35 +++
lib/Makefile | 1 +
lib/dma-noop.c | 75 +++++++
tools/virtio/linux/dma-mapping.h | 17 ++
18 files changed, 568 insertions(+), 244 deletions(-)
create mode 100644 lib/dma-noop.c
create mode 100644 tools/virtio/linux/dma-mapping.h
--
2.5.0
^ permalink raw reply
* [PATCH v5 01/10] dma: Provide simple noop dma ops
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel, xen-devel,
sparclinux, Paolo Bonzini, Linux Virtualization, David Woodhouse,
David S. Miller, Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
From: Christian Borntraeger <borntraeger@de.ibm.com>
We are going to require dma_ops for several common drivers, even for
systems that do have an identity mapping. Lets provide some minimal
no-op dma_ops that can be used for that purpose.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
---
include/linux/dma-mapping.h | 2 ++
lib/Makefile | 1 +
lib/dma-noop.c | 75 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 78 insertions(+)
create mode 100644 lib/dma-noop.c
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 75857cda38e9..c0b27ff2c784 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -70,6 +70,8 @@ struct dma_map_ops {
int is_phys;
};
+extern struct dma_map_ops dma_noop_ops;
+
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
#define DMA_MASK_NONE 0x0ULL
diff --git a/lib/Makefile b/lib/Makefile
index a7c26a41a738..a572b86a1b1d 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -18,6 +18,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
lib-$(CONFIG_MMU) += ioremap.o
lib-$(CONFIG_SMP) += cpumask.o
+lib-$(CONFIG_HAS_DMA) += dma-noop.o
lib-y += kobject.o klist.o
obj-y += lockref.o
diff --git a/lib/dma-noop.c b/lib/dma-noop.c
new file mode 100644
index 000000000000..72145646857e
--- /dev/null
+++ b/lib/dma-noop.c
@@ -0,0 +1,75 @@
+/*
+ * lib/dma-noop.c
+ *
+ * Simple DMA noop-ops that map 1:1 with memory
+ */
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+
+static void *dma_noop_alloc(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp,
+ struct dma_attrs *attrs)
+{
+ void *ret;
+
+ ret = (void *)__get_free_pages(gfp, get_order(size));
+ if (ret)
+ *dma_handle = virt_to_phys(ret);
+ return ret;
+}
+
+static void dma_noop_free(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t dma_addr,
+ struct dma_attrs *attrs)
+{
+ free_pages((unsigned long)cpu_addr, get_order(size));
+}
+
+static dma_addr_t dma_noop_map_page(struct device *dev, struct page *page,
+ unsigned long offset, size_t size,
+ enum dma_data_direction dir,
+ struct dma_attrs *attrs)
+{
+ return page_to_phys(page) + offset;
+}
+
+static int dma_noop_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+ enum dma_data_direction dir, struct dma_attrs *attrs)
+{
+ int i;
+ struct scatterlist *sg;
+
+ for_each_sg(sgl, sg, nents, i) {
+ void *va;
+
+ BUG_ON(!sg_page(sg));
+ va = sg_virt(sg);
+ sg_dma_address(sg) = (dma_addr_t)virt_to_phys(va);
+ sg_dma_len(sg) = sg->length;
+ }
+
+ return nents;
+}
+
+static int dma_noop_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+ return 0;
+}
+
+static int dma_noop_supported(struct device *dev, u64 mask)
+{
+ return 1;
+}
+
+struct dma_map_ops dma_noop_ops = {
+ .alloc = dma_noop_alloc,
+ .free = dma_noop_free,
+ .map_page = dma_noop_map_page,
+ .map_sg = dma_noop_map_sg,
+ .mapping_error = dma_noop_mapping_error,
+ .dma_supported = dma_noop_supported,
+};
+
+EXPORT_SYMBOL(dma_noop_ops);
--
2.5.0
^ permalink raw reply related
* [PATCH v5 02/10] alpha/dma: use common noop dma ops
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel, xen-devel,
sparclinux, Paolo Bonzini, Linux Virtualization, David Woodhouse,
David S. Miller, Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
From: Christian Borntraeger <borntraeger@de.ibm.com>
Some of the alpha pci noop dma ops are identical to the common ones.
Use them.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
---
arch/alpha/kernel/pci-noop.c | 46 ++++----------------------------------------
1 file changed, 4 insertions(+), 42 deletions(-)
diff --git a/arch/alpha/kernel/pci-noop.c b/arch/alpha/kernel/pci-noop.c
index 2b1f4a1e9272..8e735b5e56bd 100644
--- a/arch/alpha/kernel/pci-noop.c
+++ b/arch/alpha/kernel/pci-noop.c
@@ -123,44 +123,6 @@ static void *alpha_noop_alloc_coherent(struct device *dev, size_t size,
return ret;
}
-static void alpha_noop_free_coherent(struct device *dev, size_t size,
- void *cpu_addr, dma_addr_t dma_addr,
- struct dma_attrs *attrs)
-{
- free_pages((unsigned long)cpu_addr, get_order(size));
-}
-
-static dma_addr_t alpha_noop_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction dir,
- struct dma_attrs *attrs)
-{
- return page_to_pa(page) + offset;
-}
-
-static int alpha_noop_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
- enum dma_data_direction dir, struct dma_attrs *attrs)
-{
- int i;
- struct scatterlist *sg;
-
- for_each_sg(sgl, sg, nents, i) {
- void *va;
-
- BUG_ON(!sg_page(sg));
- va = sg_virt(sg);
- sg_dma_address(sg) = (dma_addr_t)virt_to_phys(va);
- sg_dma_len(sg) = sg->length;
- }
-
- return nents;
-}
-
-static int alpha_noop_mapping_error(struct device *dev, dma_addr_t dma_addr)
-{
- return 0;
-}
-
static int alpha_noop_supported(struct device *dev, u64 mask)
{
return mask < 0x00ffffffUL ? 0 : 1;
@@ -168,10 +130,10 @@ static int alpha_noop_supported(struct device *dev, u64 mask)
struct dma_map_ops alpha_noop_ops = {
.alloc = alpha_noop_alloc_coherent,
- .free = alpha_noop_free_coherent,
- .map_page = alpha_noop_map_page,
- .map_sg = alpha_noop_map_sg,
- .mapping_error = alpha_noop_mapping_error,
+ .free = dma_noop_free_coherent,
+ .map_page = dma_noop_map_page,
+ .map_sg = dma_noop_map_sg,
+ .mapping_error = dma_noop_mapping_error,
.dma_supported = alpha_noop_supported,
};
--
2.5.0
^ permalink raw reply related
* [PATCH v5 03/10] s390/dma: Allow per device dma ops
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel, xen-devel,
sparclinux, Paolo Bonzini, Linux Virtualization, David Woodhouse,
David S. Miller, Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
From: Christian Borntraeger <borntraeger@de.ibm.com>
As virtio-ccw will have dma ops, we can no longer default to the
zPCI ones. Make use of dev_archdata to keep the dma_ops per device.
The pci devices now use that to override the default, and the
default is changed to use the noop ops for everything that does not
specify a device specific one.
To compile without PCI support we will enable HAS_DMA all the time,
via the default config in lib/Kconfig.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
---
arch/s390/Kconfig | 6 ++----
arch/s390/include/asm/device.h | 6 +++++-
arch/s390/include/asm/dma-mapping.h | 6 ++++--
arch/s390/pci/pci.c | 1 +
arch/s390/pci/pci_dma.c | 4 ++--
5 files changed, 14 insertions(+), 9 deletions(-)
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 3be9c832dec1..5b22a26337b2 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -124,6 +124,8 @@ config S390
select HAVE_CMPXCHG_DOUBLE
select HAVE_CMPXCHG_LOCAL
select HAVE_DEBUG_KMEMLEAK
+ select HAVE_DMA_ATTRS
+ select HAVE_DMA_API_DEBUG
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_FTRACE_MCOUNT_RECORD
@@ -619,10 +621,6 @@ config HAS_IOMEM
config IOMMU_HELPER
def_bool PCI
-config HAS_DMA
- def_bool PCI
- select HAVE_DMA_API_DEBUG
-
config NEED_SG_DMA_LENGTH
def_bool PCI
diff --git a/arch/s390/include/asm/device.h b/arch/s390/include/asm/device.h
index d8f9872b0e2d..4a9f35e0973f 100644
--- a/arch/s390/include/asm/device.h
+++ b/arch/s390/include/asm/device.h
@@ -3,5 +3,9 @@
*
* This file is released under the GPLv2
*/
-#include <asm-generic/device.h>
+struct dev_archdata {
+ struct dma_map_ops *dma_ops;
+};
+struct pdev_archdata {
+};
diff --git a/arch/s390/include/asm/dma-mapping.h b/arch/s390/include/asm/dma-mapping.h
index e64bfcb9702f..3249b7464889 100644
--- a/arch/s390/include/asm/dma-mapping.h
+++ b/arch/s390/include/asm/dma-mapping.h
@@ -11,11 +11,13 @@
#define DMA_ERROR_CODE (~(dma_addr_t) 0x0)
-extern struct dma_map_ops s390_dma_ops;
+extern struct dma_map_ops s390_pci_dma_ops;
static inline struct dma_map_ops *get_dma_ops(struct device *dev)
{
- return &s390_dma_ops;
+ if (dev && dev->archdata.dma_ops)
+ return dev->archdata.dma_ops;
+ return &dma_noop_ops;
}
static inline void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 11d4f277e9f6..f5931135b9ae 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -649,6 +649,7 @@ int pcibios_add_device(struct pci_dev *pdev)
zdev->pdev = pdev;
pdev->dev.groups = zpci_attr_groups;
+ pdev->dev.archdata.dma_ops = &s390_pci_dma_ops;
zpci_map_resources(pdev);
for (i = 0; i < PCI_BAR_COUNT; i++) {
diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c
index 4638b93c7632..a79173ec54b9 100644
--- a/arch/s390/pci/pci_dma.c
+++ b/arch/s390/pci/pci_dma.c
@@ -544,7 +544,7 @@ static int __init dma_debug_do_init(void)
}
fs_initcall(dma_debug_do_init);
-struct dma_map_ops s390_dma_ops = {
+struct dma_map_ops s390_pci_dma_ops = {
.alloc = s390_dma_alloc,
.free = s390_dma_free,
.map_sg = s390_dma_map_sg,
@@ -555,7 +555,7 @@ struct dma_map_ops s390_dma_ops = {
.is_phys = 0,
/* dma_supported is unconditionally true without a callback */
};
-EXPORT_SYMBOL_GPL(s390_dma_ops);
+EXPORT_SYMBOL_GPL(s390_pci_dma_ops);
static int __init s390_iommu_setup(char *str)
{
--
2.5.0
^ permalink raw reply related
* [PATCH v5 04/10] vring: Introduce vring_use_dma_api()
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
This is a kludge, but no one has come up with a a better idea yet.
We'll introduce DMA API support guarded by vring_use_dma_api().
Eventually we may be able to return true on more and more systems,
and hopefully we can get rid of vring_use_dma_api() entirely some
day.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_ring.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index e12e385f7ac3..4b8dab4960bb 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -25,6 +25,30 @@
#include <linux/hrtimer.h>
#include <linux/kmemleak.h>
+/*
+ * The interaction between virtio and a possible IOMMU is a mess.
+ *
+ * On most systems with virtio, physical addresses match bus addresses,
+ * and it doesn't particularly matter whether we use the DMI API.
+ *
+ * On some sytems, including Xen and any system with a physical device
+ * that speaks virtio behind a physical IOMMU, we must use the DMA API
+ * for virtio DMA to work at all.
+ *
+ * On other systems, including SPARC and PPC64, virtio-pci devices are
+ * enumerated as though they are behind an IOMMU, but the virtio host
+ * ignores the IOMMU, so we must either pretend that the IOMMU isn't
+ * there or somehow map everything as the identity.
+ *
+ * For the time being, we preseve historic behavior and bypass the DMA
+ * API.
+ */
+
+static bool vring_use_dma_api(void)
+{
+ return false;
+}
+
#ifdef DEBUG
/* For development, we want to crash whenever the ring is screwed. */
#define BAD_RING(_vq, fmt, args...) \
--
2.5.0
^ permalink raw reply related
* [PATCH v5 05/10] virtio_ring: Support DMA APIs
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
virtio_ring currently sends the device (usually a hypervisor)
physical addresses of its I/O buffers. This is okay when DMA
addresses and physical addresses are the same thing, but this isn't
always the case. For example, this never works on Xen guests, and
it is likely to fail if a physical "virtio" device ever ends up
behind an IOMMU or swiotlb.
The immediate use case for me is to enable virtio on Xen guests.
For that to work, we need vring to support DMA address translation
as well as a corresponding change to virtio_pci or to another
driver.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/Kconfig | 2 +-
drivers/virtio/virtio_ring.c | 200 ++++++++++++++++++++++++++++++++-------
tools/virtio/linux/dma-mapping.h | 17 ++++
3 files changed, 183 insertions(+), 36 deletions(-)
create mode 100644 tools/virtio/linux/dma-mapping.h
diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index cab9f3f63a38..77590320d44c 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -60,7 +60,7 @@ config VIRTIO_INPUT
config VIRTIO_MMIO
tristate "Platform bus driver for memory mapped virtio devices"
- depends on HAS_IOMEM
+ depends on HAS_IOMEM && HAS_DMA
select VIRTIO
---help---
This drivers provides support for memory mapped virtio
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 4b8dab4960bb..ec2e65876b29 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -24,6 +24,7 @@
#include <linux/module.h>
#include <linux/hrtimer.h>
#include <linux/kmemleak.h>
+#include <linux/dma-mapping.h>
/*
* The interaction between virtio and a possible IOMMU is a mess.
@@ -78,6 +79,11 @@ static bool vring_use_dma_api(void)
#define END_USE(vq)
#endif
+struct vring_desc_state {
+ void *data; /* Data for callback. */
+ struct vring_desc *indir_desc; /* Indirect descriptor, if any. */
+};
+
struct vring_virtqueue {
struct virtqueue vq;
@@ -122,12 +128,85 @@ struct vring_virtqueue {
ktime_t last_add_time;
#endif
- /* Tokens for callbacks. */
- void *data[];
+ /* Per-descriptor state. */
+ struct vring_desc_state desc_state[];
};
#define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
+/*
+ * The DMA ops on various arches are rather gnarly right now, and
+ * making all of the arch DMA ops work on the vring device itself
+ * is a mess. For now, we use the parent device for DMA ops.
+ */
+struct device *vring_dma_dev(const struct vring_virtqueue *vq)
+{
+ return vq->vq.vdev->dev.parent;
+}
+
+/* Map one sg entry. */
+static dma_addr_t vring_map_one_sg(const struct vring_virtqueue *vq,
+ struct scatterlist *sg,
+ enum dma_data_direction direction)
+{
+ if (!vring_use_dma_api())
+ return (dma_addr_t)sg_phys(sg);
+
+ /*
+ * We can't use dma_map_sg, because we don't use scatterlists in
+ * the way it expects (we don't guarantee that the scatterlist
+ * will exist for the lifetime of the mapping).
+ */
+ return dma_map_page(vring_dma_dev(vq),
+ sg_page(sg), sg->offset, sg->length,
+ direction);
+}
+
+static dma_addr_t vring_map_single(const struct vring_virtqueue *vq,
+ void *cpu_addr, size_t size,
+ enum dma_data_direction direction)
+{
+ if (!vring_use_dma_api())
+ return (dma_addr_t)virt_to_phys(cpu_addr);
+
+ return dma_map_single(vring_dma_dev(vq),
+ cpu_addr, size, direction);
+}
+
+static void vring_unmap_one(const struct vring_virtqueue *vq,
+ struct vring_desc *desc)
+{
+ u16 flags;
+
+ if (!vring_use_dma_api())
+ return;
+
+ flags = virtio16_to_cpu(vq->vq.vdev, desc->flags);
+
+ if (flags & VRING_DESC_F_INDIRECT) {
+ dma_unmap_single(vring_dma_dev(vq),
+ virtio64_to_cpu(vq->vq.vdev, desc->addr),
+ virtio32_to_cpu(vq->vq.vdev, desc->len),
+ (flags & VRING_DESC_F_WRITE) ?
+ DMA_FROM_DEVICE : DMA_TO_DEVICE);
+ } else {
+ dma_unmap_page(vring_dma_dev(vq),
+ virtio64_to_cpu(vq->vq.vdev, desc->addr),
+ virtio32_to_cpu(vq->vq.vdev, desc->len),
+ (flags & VRING_DESC_F_WRITE) ?
+ DMA_FROM_DEVICE : DMA_TO_DEVICE);
+ }
+}
+
+static int vring_mapping_error(const struct vring_virtqueue *vq,
+ dma_addr_t addr)
+{
+ if (!vring_use_dma_api())
+ return 0;
+
+ return dma_mapping_error(vring_dma_dev(vq), addr);
+}
+
static struct vring_desc *alloc_indirect(struct virtqueue *_vq,
unsigned int total_sg, gfp_t gfp)
{
@@ -161,7 +240,7 @@ static inline int virtqueue_add(struct virtqueue *_vq,
struct vring_virtqueue *vq = to_vvq(_vq);
struct scatterlist *sg;
struct vring_desc *desc;
- unsigned int i, n, avail, descs_used, uninitialized_var(prev);
+ unsigned int i, n, avail, descs_used, uninitialized_var(prev), err_idx;
int head;
bool indirect;
@@ -201,21 +280,15 @@ static inline int virtqueue_add(struct virtqueue *_vq,
if (desc) {
/* Use a single buffer which doesn't continue */
- vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
- vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, virt_to_phys(desc));
- /* avoid kmemleak false positive (hidden by virt_to_phys) */
- kmemleak_ignore(desc);
- vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
-
+ indirect = true;
/* Set up rest to use this indirect table. */
i = 0;
descs_used = 1;
- indirect = true;
} else {
+ indirect = false;
desc = vq->vring.desc;
i = head;
descs_used = total_sg;
- indirect = false;
}
if (vq->vq.num_free < descs_used) {
@@ -230,13 +303,14 @@ static inline int virtqueue_add(struct virtqueue *_vq,
return -ENOSPC;
}
- /* We're about to use some buffers from the free list. */
- vq->vq.num_free -= descs_used;
-
for (n = 0; n < out_sgs; n++) {
for (sg = sgs[n]; sg; sg = sg_next(sg)) {
+ dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_TO_DEVICE);
+ if (vring_mapping_error(vq, addr))
+ goto unmap_release;
+
desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT);
- desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
+ desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
prev = i;
i = virtio16_to_cpu(_vq->vdev, desc[i].next);
@@ -244,8 +318,12 @@ static inline int virtqueue_add(struct virtqueue *_vq,
}
for (; n < (out_sgs + in_sgs); n++) {
for (sg = sgs[n]; sg; sg = sg_next(sg)) {
+ dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_FROM_DEVICE);
+ if (vring_mapping_error(vq, addr))
+ goto unmap_release;
+
desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT | VRING_DESC_F_WRITE);
- desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
+ desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
prev = i;
i = virtio16_to_cpu(_vq->vdev, desc[i].next);
@@ -254,14 +332,33 @@ static inline int virtqueue_add(struct virtqueue *_vq,
/* Last one doesn't continue. */
desc[prev].flags &= cpu_to_virtio16(_vq->vdev, ~VRING_DESC_F_NEXT);
+ if (indirect) {
+ /* Now that the indirect table is filled in, map it. */
+ dma_addr_t addr = vring_map_single(
+ vq, desc, total_sg * sizeof(struct vring_desc),
+ DMA_TO_DEVICE);
+ if (vring_mapping_error(vq, addr))
+ goto unmap_release;
+
+ vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
+ vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, addr);
+
+ vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
+ }
+
+ /* We're using some buffers from the free list. */
+ vq->vq.num_free -= descs_used;
+
/* Update free pointer */
if (indirect)
vq->free_head = virtio16_to_cpu(_vq->vdev, vq->vring.desc[head].next);
else
vq->free_head = i;
- /* Set token. */
- vq->data[head] = data;
+ /* Store token and indirect buffer state. */
+ vq->desc_state[head].data = data;
+ if (indirect)
+ vq->desc_state[head].indir_desc = desc;
/* Put entry in available array (but don't update avail->idx until they
* do sync). */
@@ -284,6 +381,24 @@ static inline int virtqueue_add(struct virtqueue *_vq,
virtqueue_kick(_vq);
return 0;
+
+unmap_release:
+ err_idx = i;
+ i = head;
+
+ for (n = 0; n < total_sg; n++) {
+ if (i == err_idx)
+ break;
+ vring_unmap_one(vq, &desc[i]);
+ i = vq->vring.desc[i].next;
+ }
+
+ vq->vq.num_free += total_sg;
+
+ if (indirect)
+ kfree(desc);
+
+ return -EIO;
}
/**
@@ -454,27 +569,43 @@ EXPORT_SYMBOL_GPL(virtqueue_kick);
static void detach_buf(struct vring_virtqueue *vq, unsigned int head)
{
- unsigned int i;
+ unsigned int i, j;
+ u16 nextflag = cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT);
/* Clear data ptr. */
- vq->data[head] = NULL;
+ vq->desc_state[head].data = NULL;
- /* Put back on free list: find end */
+ /* Put back on free list: unmap first-level descriptors and find end */
i = head;
- /* Free the indirect table */
- if (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT))
- kfree(phys_to_virt(virtio64_to_cpu(vq->vq.vdev, vq->vring.desc[i].addr)));
-
- while (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT)) {
+ while (vq->vring.desc[i].flags & nextflag) {
+ vring_unmap_one(vq, &vq->vring.desc[i]);
i = virtio16_to_cpu(vq->vq.vdev, vq->vring.desc[i].next);
vq->vq.num_free++;
}
+ vring_unmap_one(vq, &vq->vring.desc[i]);
vq->vring.desc[i].next = cpu_to_virtio16(vq->vq.vdev, vq->free_head);
vq->free_head = head;
+
/* Plus final descriptor */
vq->vq.num_free++;
+
+ /* Free the indirect table, if any, now that it's unmapped. */
+ if (vq->desc_state[head].indir_desc) {
+ struct vring_desc *indir_desc = vq->desc_state[head].indir_desc;
+ u32 len = virtio32_to_cpu(vq->vq.vdev, vq->vring.desc[head].len);
+
+ BUG_ON(!(vq->vring.desc[head].flags &
+ cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT)));
+ BUG_ON(len == 0 || len % sizeof(struct vring_desc));
+
+ for (j = 0; j < len / sizeof(struct vring_desc); j++)
+ vring_unmap_one(vq, &indir_desc[j]);
+
+ kfree(vq->desc_state[head].indir_desc);
+ vq->desc_state[head].indir_desc = NULL;
+ }
}
static inline bool more_used(const struct vring_virtqueue *vq)
@@ -529,13 +660,13 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)
BAD_RING(vq, "id %u out of range\n", i);
return NULL;
}
- if (unlikely(!vq->data[i])) {
+ if (unlikely(!vq->desc_state[i].data)) {
BAD_RING(vq, "id %u is not a head!\n", i);
return NULL;
}
/* detach_buf clears data, so grab it now. */
- ret = vq->data[i];
+ ret = vq->desc_state[i].data;
detach_buf(vq, i);
vq->last_used_idx++;
/* If we expect an interrupt for the next entry, tell host
@@ -709,10 +840,10 @@ void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
START_USE(vq);
for (i = 0; i < vq->vring.num; i++) {
- if (!vq->data[i])
+ if (!vq->desc_state[i].data)
continue;
/* detach_buf clears data, so grab it now. */
- buf = vq->data[i];
+ buf = vq->desc_state[i].data;
detach_buf(vq, i);
vq->avail_idx_shadow--;
vq->vring.avail->idx = cpu_to_virtio16(_vq->vdev, vq->avail_idx_shadow);
@@ -766,7 +897,8 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
return NULL;
}
- vq = kmalloc(sizeof(*vq) + sizeof(void *)*num, GFP_KERNEL);
+ vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
+ GFP_KERNEL);
if (!vq)
return NULL;
@@ -800,11 +932,9 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
/* Put everything in free lists. */
vq->free_head = 0;
- for (i = 0; i < num-1; i++) {
+ for (i = 0; i < num-1; i++)
vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
- vq->data[i] = NULL;
- }
- vq->data[i] = NULL;
+ memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
return &vq->vq;
}
diff --git a/tools/virtio/linux/dma-mapping.h b/tools/virtio/linux/dma-mapping.h
new file mode 100644
index 000000000000..4f93af89ae16
--- /dev/null
+++ b/tools/virtio/linux/dma-mapping.h
@@ -0,0 +1,17 @@
+#ifndef _LINUX_DMA_MAPPING_H
+#define _LINUX_DMA_MAPPING_H
+
+#ifdef CONFIG_HAS_DMA
+# error Virtio userspace code does not support CONFIG_HAS_DMA
+#endif
+
+#define PCI_DMA_BUS_IS_PHYS 1
+
+enum dma_data_direction {
+ DMA_BIDIRECTIONAL = 0,
+ DMA_TO_DEVICE = 1,
+ DMA_FROM_DEVICE = 2,
+ DMA_NONE = 3,
+};
+
+#endif
--
2.5.0
^ permalink raw reply related
* [PATCH v5 06/10] virtio: Add improved queue allocation API
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
This leaves vring_new_virtqueue alone for compatbility, but it
adds two new improved APIs:
vring_create_virtqueue: Creates a virtqueue backed by automatically
allocated coherent memory. (Some day it this could be extended to
support non-coherent memory, too, if there ends up being a platform
on which it's worthwhile.)
__vring_new_virtqueue: Creates a virtqueue with a manually-specified
layout. This should allow mic_virtio to work much more cleanly.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_ring.c | 178 +++++++++++++++++++++++++++++++++++--------
include/linux/virtio.h | 23 +++++-
include/linux/virtio_ring.h | 35 +++++++++
3 files changed, 204 insertions(+), 32 deletions(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index ec2e65876b29..c169c6444637 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -119,6 +119,11 @@ struct vring_virtqueue {
/* How to notify other side. FIXME: commonalize hcalls! */
bool (*notify)(struct virtqueue *vq);
+ /* DMA, allocation, and size information */
+ bool we_own_ring;
+ size_t queue_size_in_bytes;
+ dma_addr_t queue_dma_addr;
+
#ifdef DEBUG
/* They're supposed to lock for us. */
unsigned int in_use;
@@ -878,36 +883,31 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
}
EXPORT_SYMBOL_GPL(vring_interrupt);
-struct virtqueue *vring_new_virtqueue(unsigned int index,
- unsigned int num,
- unsigned int vring_align,
- struct virtio_device *vdev,
- bool weak_barriers,
- void *pages,
- bool (*notify)(struct virtqueue *),
- void (*callback)(struct virtqueue *),
- const char *name)
+struct virtqueue *__vring_new_virtqueue(unsigned int index,
+ struct vring vring,
+ struct virtio_device *vdev,
+ bool weak_barriers,
+ bool (*notify)(struct virtqueue *),
+ void (*callback)(struct virtqueue *),
+ const char *name)
{
- struct vring_virtqueue *vq;
unsigned int i;
+ struct vring_virtqueue *vq;
- /* We assume num is a power of 2. */
- if (num & (num - 1)) {
- dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
- return NULL;
- }
-
- vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
+ vq = kmalloc(sizeof(*vq) + vring.num * sizeof(struct vring_desc_state),
GFP_KERNEL);
if (!vq)
return NULL;
- vring_init(&vq->vring, num, pages, vring_align);
+ vq->vring = vring;
vq->vq.callback = callback;
vq->vq.vdev = vdev;
vq->vq.name = name;
- vq->vq.num_free = num;
+ vq->vq.num_free = vring.num;
vq->vq.index = index;
+ vq->we_own_ring = false;
+ vq->queue_dma_addr = 0;
+ vq->queue_size_in_bytes = 0;
vq->notify = notify;
vq->weak_barriers = weak_barriers;
vq->broken = false;
@@ -932,18 +932,105 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
/* Put everything in free lists. */
vq->free_head = 0;
- for (i = 0; i < num-1; i++)
+ for (i = 0; i < vring.num-1; i++)
vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
- memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
+ memset(vq->desc_state, 0, vring.num * sizeof(struct vring_desc_state));
return &vq->vq;
}
+EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
+
+struct virtqueue *vring_create_virtqueue(
+ unsigned int index,
+ unsigned int num,
+ unsigned int vring_align,
+ struct virtio_device *vdev,
+ bool weak_barriers,
+ bool may_reduce_num,
+ bool (*notify)(struct virtqueue *),
+ void (*callback)(struct virtqueue *),
+ const char *name)
+{
+ struct virtqueue *vq;
+ void *queue;
+ dma_addr_t dma_addr;
+ size_t queue_size_in_bytes;
+ struct vring vring;
+
+ /* We assume num is a power of 2. */
+ if (num & (num - 1)) {
+ dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
+ return NULL;
+ }
+
+ /* TODO: allocate each queue chunk individually */
+ for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
+ queue = dma_zalloc_coherent(
+ vdev->dev.parent, vring_size(num, vring_align),
+ &dma_addr, GFP_KERNEL|__GFP_NOWARN);
+ if (queue)
+ break;
+ }
+
+ if (!num)
+ return NULL;
+
+ if (!queue) {
+ /* Try to get a single page. You are my only hope! */
+ queue = dma_zalloc_coherent(
+ vdev->dev.parent, vring_size(num, vring_align),
+ &dma_addr, GFP_KERNEL);
+ }
+ if (!queue)
+ return NULL;
+
+ queue_size_in_bytes = vring_size(num, vring_align);
+ vring_init(&vring, num, queue, vring_align);
+
+ vq = __vring_new_virtqueue(index, vring, vdev, weak_barriers,
+ notify, callback, name);
+ if (!vq) {
+ dma_free_coherent(vdev->dev.parent,
+ queue_size_in_bytes, queue,
+ dma_addr);
+ return NULL;
+ }
+
+ to_vvq(vq)->queue_dma_addr = dma_addr;
+ to_vvq(vq)->queue_size_in_bytes = queue_size_in_bytes;
+ to_vvq(vq)->we_own_ring = true;
+
+ return vq;
+}
+EXPORT_SYMBOL_GPL(vring_create_virtqueue);
+
+struct virtqueue *vring_new_virtqueue(unsigned int index,
+ unsigned int num,
+ unsigned int vring_align,
+ struct virtio_device *vdev,
+ bool weak_barriers,
+ void *pages,
+ bool (*notify)(struct virtqueue *vq),
+ void (*callback)(struct virtqueue *vq),
+ const char *name)
+{
+ struct vring vring;
+ vring_init(&vring, num, pages, vring_align);
+ return __vring_new_virtqueue(index, vring, vdev, weak_barriers,
+ notify, callback, name);
+}
EXPORT_SYMBOL_GPL(vring_new_virtqueue);
-void vring_del_virtqueue(struct virtqueue *vq)
+void vring_del_virtqueue(struct virtqueue *_vq)
{
- list_del(&vq->list);
- kfree(to_vvq(vq));
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (vq->we_own_ring) {
+ dma_free_coherent(vring_dma_dev(vq), vq->queue_size_in_bytes,
+ vq->vring.desc, vq->queue_dma_addr);
+ }
+ list_del(&_vq->list);
+ kfree(vq);
}
EXPORT_SYMBOL_GPL(vring_del_virtqueue);
@@ -1007,20 +1094,51 @@ void virtio_break_device(struct virtio_device *dev)
}
EXPORT_SYMBOL_GPL(virtio_break_device);
-void *virtqueue_get_avail(struct virtqueue *_vq)
+dma_addr_t virtqueue_get_desc_addr(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ BUG_ON(!vq->we_own_ring);
+
+ if (vring_use_dma_api())
+ return vq->queue_dma_addr;
+ else
+ return virt_to_phys(vq->vring.desc);
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_desc_addr);
+
+dma_addr_t virtqueue_get_avail_addr(struct virtqueue *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
- return vq->vring.avail;
+ BUG_ON(!vq->we_own_ring);
+
+ if (vring_use_dma_api())
+ return vq->queue_dma_addr +
+ ((char *)vq->vring.avail - (char *)vq->vring.desc);
+ else
+ return virt_to_phys(vq->vring.avail);
}
-EXPORT_SYMBOL_GPL(virtqueue_get_avail);
+EXPORT_SYMBOL_GPL(virtqueue_get_avail_addr);
-void *virtqueue_get_used(struct virtqueue *_vq)
+dma_addr_t virtqueue_get_used_addr(struct virtqueue *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
- return vq->vring.used;
+ BUG_ON(!vq->we_own_ring);
+
+ if (vring_use_dma_api())
+ return vq->queue_dma_addr +
+ ((char *)vq->vring.used - (char *)vq->vring.desc);
+ else
+ return virt_to_phys(vq->vring.used);
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_used_addr);
+
+const struct vring *virtqueue_get_vring(struct virtqueue *vq)
+{
+ return &to_vvq(vq)->vring;
}
-EXPORT_SYMBOL_GPL(virtqueue_get_used);
+EXPORT_SYMBOL_GPL(virtqueue_get_vring);
MODULE_LICENSE("GPL");
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 8f4d4bfa6d46..d5eb5479a425 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -75,8 +75,27 @@ unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
bool virtqueue_is_broken(struct virtqueue *vq);
-void *virtqueue_get_avail(struct virtqueue *vq);
-void *virtqueue_get_used(struct virtqueue *vq);
+const struct vring *virtqueue_get_vring(struct virtqueue *vq);
+dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
+dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
+dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
+
+/*
+ * Legacy accessors -- in almost all cases, these are the wrong functions
+ * to use.
+ */
+static inline void *virtqueue_get_desc(struct virtqueue *vq)
+{
+ return virtqueue_get_vring(vq)->desc;
+}
+static inline void *virtqueue_get_avail(struct virtqueue *vq)
+{
+ return virtqueue_get_vring(vq)->avail;
+}
+static inline void *virtqueue_get_used(struct virtqueue *vq)
+{
+ return virtqueue_get_vring(vq)->used;
+}
/**
* virtio_device - representation of a device using virtio
diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h
index a156e2b6ccfe..e8d36938f09a 100644
--- a/include/linux/virtio_ring.h
+++ b/include/linux/virtio_ring.h
@@ -59,6 +59,35 @@ static inline void virtio_store_mb(bool weak_barriers,
struct virtio_device;
struct virtqueue;
+/*
+ * Creates a virtqueue and allocates the descriptor ring. If
+ * may_reduce_num is set, then this may allocate a smaller ring than
+ * expected. The caller should query virtqueue_get_ring_size to learn
+ * the actual size of the ring.
+ */
+struct virtqueue *vring_create_virtqueue(unsigned int index,
+ unsigned int num,
+ unsigned int vring_align,
+ struct virtio_device *vdev,
+ bool weak_barriers,
+ bool may_reduce_num,
+ bool (*notify)(struct virtqueue *vq),
+ void (*callback)(struct virtqueue *vq),
+ const char *name);
+
+/* Creates a virtqueue with a custom layout. */
+struct virtqueue *__vring_new_virtqueue(unsigned int index,
+ struct vring vring,
+ struct virtio_device *vdev,
+ bool weak_barriers,
+ bool (*notify)(struct virtqueue *),
+ void (*callback)(struct virtqueue *),
+ const char *name);
+
+/*
+ * Creates a virtqueue with a standard layout but a caller-allocated
+ * ring.
+ */
struct virtqueue *vring_new_virtqueue(unsigned int index,
unsigned int num,
unsigned int vring_align,
@@ -68,7 +97,13 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
bool (*notify)(struct virtqueue *vq),
void (*callback)(struct virtqueue *vq),
const char *name);
+
+/*
+ * Destroys a virtqueue. If created with vring_create_virtqueue, this
+ * also frees the ring.
+ */
void vring_del_virtqueue(struct virtqueue *vq);
+
/* Filter out transport-specific feature bits. */
void vring_transport_features(struct virtio_device *vdev);
--
2.5.0
^ permalink raw reply related
* [PATCH v5 07/10] virtio_mmio: Use the DMA API if enabled
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
This switches to vring_create_virtqueue, simplifying the driver and
adding DMA API support.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_mmio.c | 67 ++++++++++----------------------------------
1 file changed, 15 insertions(+), 52 deletions(-)
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 745c6ee1bb3e..48bfea91dbca 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -99,12 +99,6 @@ struct virtio_mmio_vq_info {
/* the actual virtqueue */
struct virtqueue *vq;
- /* the number of entries in the queue */
- unsigned int num;
-
- /* the virtual address of the ring queue */
- void *queue;
-
/* the list node for the virtqueues list */
struct list_head node;
};
@@ -322,15 +316,13 @@ static void vm_del_vq(struct virtqueue *vq)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vq->vdev);
struct virtio_mmio_vq_info *info = vq->priv;
- unsigned long flags, size;
+ unsigned long flags;
unsigned int index = vq->index;
spin_lock_irqsave(&vm_dev->lock, flags);
list_del(&info->node);
spin_unlock_irqrestore(&vm_dev->lock, flags);
- vring_del_virtqueue(vq);
-
/* Select and deactivate the queue */
writel(index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
if (vm_dev->version == 1) {
@@ -340,8 +332,8 @@ static void vm_del_vq(struct virtqueue *vq)
WARN_ON(readl(vm_dev->base + VIRTIO_MMIO_QUEUE_READY));
}
- size = PAGE_ALIGN(vring_size(info->num, VIRTIO_MMIO_VRING_ALIGN));
- free_pages_exact(info->queue, size);
+ vring_del_virtqueue(vq);
+
kfree(info);
}
@@ -356,8 +348,6 @@ static void vm_del_vqs(struct virtio_device *vdev)
free_irq(platform_get_irq(vm_dev->pdev, 0), vm_dev);
}
-
-
static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
void (*callback)(struct virtqueue *vq),
const char *name)
@@ -365,7 +355,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
struct virtio_mmio_vq_info *info;
struct virtqueue *vq;
- unsigned long flags, size;
+ unsigned long flags;
+ unsigned int num;
int err;
if (!name)
@@ -388,66 +379,40 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
goto error_kmalloc;
}
- /* Allocate pages for the queue - start with a queue as big as
- * possible (limited by maximum size allowed by device), drop down
- * to a minimal size, just big enough to fit descriptor table
- * and two rings (which makes it "alignment_size * 2")
- */
- info->num = readl(vm_dev->base + VIRTIO_MMIO_QUEUE_NUM_MAX);
-
- /* If the device reports a 0 entry queue, we won't be able to
- * use it to perform I/O, and vring_new_virtqueue() can't create
- * empty queues anyway, so don't bother to set up the device.
- */
- if (info->num == 0) {
+ num = readl(vm_dev->base + VIRTIO_MMIO_QUEUE_NUM_MAX);
+ if (num == 0) {
err = -ENOENT;
- goto error_alloc_pages;
- }
-
- while (1) {
- size = PAGE_ALIGN(vring_size(info->num,
- VIRTIO_MMIO_VRING_ALIGN));
- /* Did the last iter shrink the queue below minimum size? */
- if (size < VIRTIO_MMIO_VRING_ALIGN * 2) {
- err = -ENOMEM;
- goto error_alloc_pages;
- }
-
- info->queue = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
- if (info->queue)
- break;
-
- info->num /= 2;
+ goto error_new_virtqueue;
}
/* Create the vring */
- vq = vring_new_virtqueue(index, info->num, VIRTIO_MMIO_VRING_ALIGN, vdev,
- true, info->queue, vm_notify, callback, name);
+ vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
+ true, true, vm_notify, callback, name);
if (!vq) {
err = -ENOMEM;
goto error_new_virtqueue;
}
/* Activate the queue */
- writel(info->num, vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
+ writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
if (vm_dev->version == 1) {
writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_QUEUE_ALIGN);
- writel(virt_to_phys(info->queue) >> PAGE_SHIFT,
+ writel(virtqueue_get_desc_addr(vq) >> PAGE_SHIFT,
vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
} else {
u64 addr;
- addr = virt_to_phys(info->queue);
+ addr = virtqueue_get_desc_addr(vq);
writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_LOW);
writel((u32)(addr >> 32),
vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_HIGH);
- addr = virt_to_phys(virtqueue_get_avail(vq));
+ addr = virtqueue_get_avail_addr(vq);
writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_LOW);
writel((u32)(addr >> 32),
vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_HIGH);
- addr = virt_to_phys(virtqueue_get_used(vq));
+ addr = virtqueue_get_used_addr(vq);
writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_USED_LOW);
writel((u32)(addr >> 32),
vm_dev->base + VIRTIO_MMIO_QUEUE_USED_HIGH);
@@ -471,8 +436,6 @@ error_new_virtqueue:
writel(0, vm_dev->base + VIRTIO_MMIO_QUEUE_READY);
WARN_ON(readl(vm_dev->base + VIRTIO_MMIO_QUEUE_READY));
}
- free_pages_exact(info->queue, size);
-error_alloc_pages:
kfree(info);
error_kmalloc:
error_available:
--
2.5.0
^ permalink raw reply related
* [PATCH v5 08/10] virtio_pci: Use the DMA API if enabled
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
This switches to vring_create_virtqueue, simplifying the driver and
adding DMA API support.
This fixes virtio-pci on platforms and busses that have IOMMUs. This
will break the experimental QEMU Q35 IOMMU support until QEMU is
fixed. In exchange, it fixes physical virtio hardware as well as
virtio-pci running under Xen.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_pci_common.h | 6 ----
drivers/virtio/virtio_pci_legacy.c | 42 +++++++++++---------------
drivers/virtio/virtio_pci_modern.c | 61 ++++++++++----------------------------
3 files changed, 33 insertions(+), 76 deletions(-)
diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 2cc252270b2d..28263200ed42 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -35,12 +35,6 @@ struct virtio_pci_vq_info {
/* the actual virtqueue */
struct virtqueue *vq;
- /* the number of entries in the queue */
- int num;
-
- /* the virtual address of the ring queue */
- void *queue;
-
/* the list node for the virtqueues list */
struct list_head node;
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index 48bc9797e530..8c4e61783441 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -119,7 +119,6 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
u16 msix_vec)
{
struct virtqueue *vq;
- unsigned long size;
u16 num;
int err;
@@ -131,27 +130,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
if (!num || ioread32(vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN))
return ERR_PTR(-ENOENT);
- info->num = num;
info->msix_vector = msix_vec;
- size = PAGE_ALIGN(vring_size(num, VIRTIO_PCI_VRING_ALIGN));
- info->queue = alloc_pages_exact(size, GFP_KERNEL|__GFP_ZERO);
- if (info->queue == NULL)
+ /* create the vring */
+ vq = vring_create_virtqueue(index, num,
+ VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
+ true, false, vp_notify, callback, name);
+ if (!vq)
return ERR_PTR(-ENOMEM);
/* activate the queue */
- iowrite32(virt_to_phys(info->queue) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT,
+ iowrite32(virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT,
vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
- /* create the vring */
- vq = vring_new_virtqueue(index, info->num,
- VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
- true, info->queue, vp_notify, callback, name);
- if (!vq) {
- err = -ENOMEM;
- goto out_activate_queue;
- }
-
vq->priv = (void __force *)vp_dev->ioaddr + VIRTIO_PCI_QUEUE_NOTIFY;
if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
@@ -159,17 +150,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
msix_vec = ioread16(vp_dev->ioaddr + VIRTIO_MSI_QUEUE_VECTOR);
if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
err = -EBUSY;
- goto out_assign;
+ goto out_deactivate;
}
}
return vq;
-out_assign:
- vring_del_virtqueue(vq);
-out_activate_queue:
+out_deactivate:
iowrite32(0, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
- free_pages_exact(info->queue, size);
+ vring_del_virtqueue(vq);
return ERR_PTR(err);
}
@@ -177,7 +166,6 @@ static void del_vq(struct virtio_pci_vq_info *info)
{
struct virtqueue *vq = info->vq;
struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
- unsigned long size;
iowrite16(vq->index, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_SEL);
@@ -188,13 +176,10 @@ static void del_vq(struct virtio_pci_vq_info *info)
ioread8(vp_dev->ioaddr + VIRTIO_PCI_ISR);
}
- vring_del_virtqueue(vq);
-
/* Select and deactivate the queue */
iowrite32(0, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
- size = PAGE_ALIGN(vring_size(info->num, VIRTIO_PCI_VRING_ALIGN));
- free_pages_exact(info->queue, size);
+ vring_del_virtqueue(vq);
}
static const struct virtio_config_ops virtio_pci_config_ops = {
@@ -227,6 +212,13 @@ int virtio_pci_legacy_probe(struct virtio_pci_device *vp_dev)
return -ENODEV;
}
+ rc = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(64));
+ if (rc)
+ rc = dma_set_mask_and_coherent(&pci_dev->dev,
+ DMA_BIT_MASK(32));
+ if (rc)
+ dev_warn(&pci_dev->dev, "Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.\n");
+
rc = pci_request_region(pci_dev, 0, "virtio-pci-legacy");
if (rc)
return rc;
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index c0c11fad4611..0b4a4f440b85 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -287,31 +287,6 @@ static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
return vp_ioread16(&vp_dev->common->msix_config);
}
-static size_t vring_pci_size(u16 num)
-{
- /* We only need a cacheline separation. */
- return PAGE_ALIGN(vring_size(num, SMP_CACHE_BYTES));
-}
-
-static void *alloc_virtqueue_pages(int *num)
-{
- void *pages;
-
- /* TODO: allocate each queue chunk individually */
- for (; *num && vring_pci_size(*num) > PAGE_SIZE; *num /= 2) {
- pages = alloc_pages_exact(vring_pci_size(*num),
- GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
- if (pages)
- return pages;
- }
-
- if (!*num)
- return NULL;
-
- /* Try to get a single page. You are my only hope! */
- return alloc_pages_exact(vring_pci_size(*num), GFP_KERNEL|__GFP_ZERO);
-}
-
static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
struct virtio_pci_vq_info *info,
unsigned index,
@@ -343,29 +318,22 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
/* get offset of notification word for this vq */
off = vp_ioread16(&cfg->queue_notify_off);
- info->num = num;
info->msix_vector = msix_vec;
- info->queue = alloc_virtqueue_pages(&info->num);
- if (info->queue == NULL)
- return ERR_PTR(-ENOMEM);
-
/* create the vring */
- vq = vring_new_virtqueue(index, info->num,
- SMP_CACHE_BYTES, &vp_dev->vdev,
- true, info->queue, vp_notify, callback, name);
- if (!vq) {
- err = -ENOMEM;
- goto err_new_queue;
- }
+ vq = vring_create_virtqueue(index, num,
+ SMP_CACHE_BYTES, &vp_dev->vdev,
+ true, true, vp_notify, callback, name);
+ if (!vq)
+ return ERR_PTR(-ENOMEM);
/* activate the queue */
- vp_iowrite16(num, &cfg->queue_size);
- vp_iowrite64_twopart(virt_to_phys(info->queue),
+ vp_iowrite16(virtqueue_get_vring_size(vq), &cfg->queue_size);
+ vp_iowrite64_twopart(virtqueue_get_desc_addr(vq),
&cfg->queue_desc_lo, &cfg->queue_desc_hi);
- vp_iowrite64_twopart(virt_to_phys(virtqueue_get_avail(vq)),
+ vp_iowrite64_twopart(virtqueue_get_avail_addr(vq),
&cfg->queue_avail_lo, &cfg->queue_avail_hi);
- vp_iowrite64_twopart(virt_to_phys(virtqueue_get_used(vq)),
+ vp_iowrite64_twopart(virtqueue_get_used_addr(vq),
&cfg->queue_used_lo, &cfg->queue_used_hi);
if (vp_dev->notify_base) {
@@ -410,8 +378,6 @@ err_assign_vector:
pci_iounmap(vp_dev->pci_dev, (void __iomem __force *)vq->priv);
err_map_notify:
vring_del_virtqueue(vq);
-err_new_queue:
- free_pages_exact(info->queue, vring_pci_size(info->num));
return ERR_PTR(err);
}
@@ -456,8 +422,6 @@ static void del_vq(struct virtio_pci_vq_info *info)
pci_iounmap(vp_dev->pci_dev, (void __force __iomem *)vq->priv);
vring_del_virtqueue(vq);
-
- free_pages_exact(info->queue, vring_pci_size(info->num));
}
static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
@@ -641,6 +605,13 @@ int virtio_pci_modern_probe(struct virtio_pci_device *vp_dev)
return -EINVAL;
}
+ err = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(64));
+ if (err)
+ err = dma_set_mask_and_coherent(&pci_dev->dev,
+ DMA_BIT_MASK(32));
+ if (err)
+ dev_warn(&pci_dev->dev, "Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.\n");
+
/* Device capability is only mandatory for devices that have
* device-specific configuration.
*/
--
2.5.0
^ permalink raw reply related
* [PATCH v5 09/10] vring: Use the DMA API on Xen
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_ring.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c169c6444637..305c05cc249a 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -47,6 +47,18 @@
static bool vring_use_dma_api(void)
{
+#if defined(CONFIG_X86) && defined(CONFIG_XEN)
+ /*
+ * In theory, it's possible to have a buggy QEMU-supposed
+ * emulated Q35 IOMMU and Xen enabled at the same time. On
+ * such a configuration, virtio has never worked and will
+ * not work without an even larger kludge. Instead, enable
+ * the DMA API if we're a Xen guest, which at least allows
+ * all of the sensible Xen configurations to work correctly.
+ */
+ return static_cpu_has(X86_FEATURE_XENPV);
+#endif
+
return false;
}
--
2.5.0
^ permalink raw reply related
* [PATCH v5 10/10] vring: Add a module parameter to force-enable the DMA API
From: Andy Lutomirski @ 2016-01-29 2:31 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
Christoph Hellwig, Christian Borntraeger, David Vrabel,
Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
Linux Virtualization, David Woodhouse, David S. Miller,
Martin Schwidefsky
In-Reply-To: <cover.1454034075.git.luto@kernel.org>
This will be useful for testing.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
drivers/virtio/virtio_ring.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 305c05cc249a..46fb77d824e9 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -45,6 +45,10 @@
* API.
*/
+static bool force_dma_api = false;
+module_param(force_dma_api, bool, 0644);
+MODULE_PARM_DESC(force_dma_api, "force-enable the DMA API");
+
static bool vring_use_dma_api(void)
{
#if defined(CONFIG_X86) && defined(CONFIG_XEN)
@@ -59,7 +63,7 @@ static bool vring_use_dma_api(void)
return static_cpu_has(X86_FEATURE_XENPV);
#endif
- return false;
+ return force_dma_api;
}
#ifdef DEBUG
--
2.5.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox