* Re: [PATCH] powerpc: memcpy optimization for 64bit LE
From: Philippe Bergheaud @ 2013-11-06 10:21 UTC (permalink / raw)
To: Michael Neuling; +Cc: Linuxppc-dev
In-Reply-To: <11438.1383718966@ale.ozlabs.ibm.com>
Michael Neuling wrote:
> Philippe Bergheaud <felix@linux.vnet.ibm.com> wrote:
>
>
>>Unaligned stores take alignment exceptions on POWER7 running in little-endian.
>>This is a dumb little-endian base memcpy that prevents unaligned stores.
>>It is replaced by the VMX memcpy at boot.
>
>
> Is this any faster than the generic version?
The little-endian assembly code of the base memcpy is similar to the code emitted by gcc when compiling the generic memcpy in lib/string.c, and runs at the same speed.
However, a little-endian assembly version of the base memcpy is required (as opposed to a C version), in order to use the self-modifying code instrumentation system.
After the cpu feature CPU_FTR_ALTIVEC is detected at boot, the slow base memcpy is nop'ed out, and the fast memcpy_power7 is used instead.
Philippe
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Will Deacon @ 2013-11-06 11:00 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Michael Neuling, Mathieu Desnoyers, heiko.carstens@de.ibm.com,
Oleg Nesterov, LKML, Linux PPC dev, Anton Blanchard,
Frederic Weisbecker, Victor Kaplansky, linux@arm.linux.org.uk,
Paul E. McKenney, Linus Torvalds, schwidefsky@de.ibm.com
In-Reply-To: <20131105184943.GY16117@laptop.programming.kicks-ass.net>
On Tue, Nov 05, 2013 at 06:49:43PM +0000, Peter Zijlstra wrote:
> On Tue, Nov 05, 2013 at 02:05:48PM +0000, Will Deacon wrote:
> > > > +
> > > > +#define smp_store_release(p, v) \
> > > > +do { \
> > > > + smp_mb(); \
> > > > + ACCESS_ONCE(p) = (v); \
> > > > +} while (0)
> > > > +
> > > > +#define smp_load_acquire(p, v) \
> > > > +do { \
> > > > + typeof(p) ___p1 = ACCESS_ONCE(p); \
> > > > + smp_mb(); \
> > > > + return ___p1; \
> > > > +} while (0)
> >
> > What data sizes do these accessors operate on? Assuming that we want
> > single-copy atomicity (with respect to interrupts in the UP case), we
> > probably want a check to stop people passing in things like structs.
>
> Fair enough; I think we should restrict to native word sizes same as we
> do for atomics.
>
> Something like so perhaps:
>
> #ifdef CONFIG_64BIT
> #define __check_native_word(t) (sizeof(t) == 4 || sizeof(t) == 8)
Ok, if we want to support 32-bit accesses on 64-bit machines, that will
complicate some of your assembly (more below).
> #else
> #define __check_native_word(t) (sizeof(t) == 4)
> #endif
>
> #define smp_store_release(p, v) \
> do { \
> BUILD_BUG_ON(!__check_native_word(p)); \
> smp_mb(); \
> ACCESS_ONCE(p) = (v); \
> } while (0)
>
> > > > +#define smp_store_release(p, v) \
> > > > +do { \
> > > > + asm volatile ("stlr %w0 [%1]" : : "r" (v), "r" (&p) : "memory");\
> >
> > Missing comma between the operands. Also, that 'w' output modifier enforces
> > a 32-bit store (same early question about sizes). Finally, it might be more
> > efficient to use "=Q" for the addressing mode, rather than take the address
> > of p manually.
>
> so something like:
>
> asm volatile ("stlr %0, [%1]" : : "r" (v), "=Q" (p) : "memory");
>
> ?
>
> My inline asm foo is horrid and I mostly get by with copy paste from a
> semi similar existing form :/
Almost: you just need to drop the square brackets and make the memory
location an output operand:
asm volatile("stlr %1, %0" : "=Q" (p) : "r" (v) : "memory");
however, for a 32-bit access, you need to use an output modifier:
asm volatile("stlr %w1, %0" : "=Q" (p) : "r" (v) : "memory");
so I guess a switch on sizeof(p) is required.
> > Random other question: have you considered how these accessors should behave
> > when presented with __iomem pointers?
>
> A what? ;-)
Then let's go with Paul's suggestion of mandating __kernel, or the like
(unless we need to worry about __user addresses for things like futexes?).
Will
^ permalink raw reply
* Re: [PATCH 2/3] powerpc/kvm: fix rare but potential deadlock scene
From: Paul Mackerras @ 2013-11-06 11:18 UTC (permalink / raw)
To: Liu ping fan; +Cc: linuxppc-dev, Alexander Graf, kvm-ppc
In-Reply-To: <CAFgQCTsxEAqMDt=zfFMhEG2CZ6G=QpH+OsPvJkvjFU7GkDOcXQ@mail.gmail.com>
On Wed, Nov 06, 2013 at 02:02:07PM +0800, Liu ping fan wrote:
> On Wed, Nov 6, 2013 at 1:04 PM, Paul Mackerras <paulus@samba.org> wrote:
> > On Tue, Nov 05, 2013 at 03:42:43PM +0800, Liu Ping Fan wrote:
> >> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
> >> realmode, so it can trigger the deadlock.
> >
> > Good catch, we should have preemption disabled while ever we have a
> > HPTE locked.
> >
> >> @@ -474,8 +474,10 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
> >> }
> >>
> >> /* Find the HPTE in the hash table */
> >> + preempt_disable();
> >> index = kvmppc_hv_find_lock_hpte(kvm, eaddr, slb_v,
> >> HPTE_V_VALID | HPTE_V_ABSENT);
> >> + preempt_enable();
> >
> > Which means we need to add the preempt_enable after unlocking the
> > HPTE, not here.
> >
> Yes. Sorry, but I am not sure about whether we can call
> preempt_disable/enable() in realmode. I think since thread_info is
> allocated with linear address, so we can use preempt_disable/enable()
> inside kvmppc_hv_find_lock_hpte(), right?
Your analysis correctly pointed out that we can get a deadlock if we
can be preempted while holding a lock on a HPTE. That means that we
have to disable preemption before taking an HPTE lock and keep it
disabled until after we unlock the HPTE. Since the point of
kvmppc_hv_find_lock_hpte() is to lock the HPTE and return with it
locked, we can't have the preempt_enable() inside it. The
preempt_enable() has to come after we have unlocked the HPTE. That is
also why we can't have the preempt_enable() where your patch put it;
it needs to be about 9 lines further down, after the statement
hptep[0] = v. (We also need to make sure to re-enable preemption in
the index < 0 case.)
Regards,
Paul.
^ permalink raw reply
* Re: [PATCH 3/3] powerpc/kvm: remove redundant assignment
From: Alexander Graf @ 2013-11-06 11:24 UTC (permalink / raw)
To: Liu Ping Fan; +Cc: Paul Mackerras, linuxppc-dev, kvm-ppc
In-Reply-To: <1383637364-14691-3-git-send-email-pingfank@linux.vnet.ibm.com>
On 05.11.2013, at 08:42, Liu Ping Fan <kernelfans@gmail.com> wrote:
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
Patch description missing.
Please add Paul's ack in the next revision of this patch :).
Alex
> ---
> arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 -
> 1 file changed, 1 deletion(-)
>=20
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c =
b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index 28160ac..7682837 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -731,7 +731,6 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run =
*run, struct kvm_vcpu *vcpu,
> lock_rmap(rmap);
>=20
> /* Check if we might have been invalidated; let the guest retry =
if so */
> - ret =3D RESUME_GUEST;
> if (mmu_notifier_retry(vcpu->kvm, mmu_seq)) {
> unlock_rmap(rmap);
> goto out_unlock;
> --=20
> 1.8.1.4
>=20
> --
> To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Peter Zijlstra @ 2013-11-06 12:39 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Michael Neuling, tony.luck, Mathieu Desnoyers, heiko.carstens,
Oleg Nesterov, LKML, Linux PPC dev, Anton Blanchard,
Frederic Weisbecker, Victor Kaplansky, linux, Linus Torvalds,
schwidefsky
In-Reply-To: <20131104205344.GW3947@linux.vnet.ibm.com>
Subject: arch: Introduce smp_load_acquire(), smp_store_release()
From: Peter Zijlstra <peterz@infradead.org>
Date: Mon, 4 Nov 2013 20:18:11 +0100
A number of situations currently require the heavyweight smp_mb(),
even though there is no need to order prior stores against later
loads. Many architectures have much cheaper ways to handle these
situations, but the Linux kernel currently has no portable way
to make use of them.
This commit therefore supplies smp_load_acquire() and
smp_store_release() to remedy this situation. The new
smp_load_acquire() primitive orders the specified load against
any subsequent reads or writes, while the new smp_store_release()
primitive orders the specifed store against any prior reads or
writes. These primitives allow array-based circular FIFOs to be
implemented without an smp_mb(), and also allow a theoretical
hole in rcu_assign_pointer() to be closed at no additional
expense on most architectures.
In addition, the RCU experience transitioning from explicit
smp_read_barrier_depends() and smp_wmb() to rcu_dereference()
and rcu_assign_pointer(), respectively resulted in substantial
improvements in readability. It therefore seems likely that
replacing other explicit barriers with smp_load_acquire() and
smp_store_release() will provide similar benefits. It appears
that roughly half of the explicit barriers in core kernel code
might be so replaced.
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Cc: Michael Ellerman <michael@ellerman.id.au>
Cc: Michael Neuling <mikey@neuling.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Victor Kaplansky <VICTORK@il.ibm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
---
Documentation/memory-barriers.txt | 157 +++++++++++++++++-----------------
arch/alpha/include/asm/barrier.h | 15 +++
arch/arc/include/asm/barrier.h | 15 +++
arch/arm/include/asm/barrier.h | 15 +++
arch/arm64/include/asm/barrier.h | 50 ++++++++++
arch/avr32/include/asm/barrier.h | 14 +++
arch/blackfin/include/asm/barrier.h | 15 +++
arch/cris/include/asm/barrier.h | 15 +++
arch/frv/include/asm/barrier.h | 15 +++
arch/h8300/include/asm/barrier.h | 15 +++
arch/hexagon/include/asm/barrier.h | 15 +++
arch/ia64/include/asm/barrier.h | 49 ++++++++++
arch/m32r/include/asm/barrier.h | 15 +++
arch/m68k/include/asm/barrier.h | 15 +++
arch/metag/include/asm/barrier.h | 15 +++
arch/microblaze/include/asm/barrier.h | 15 +++
arch/mips/include/asm/barrier.h | 15 +++
arch/mn10300/include/asm/barrier.h | 15 +++
arch/parisc/include/asm/barrier.h | 15 +++
arch/powerpc/include/asm/barrier.h | 21 ++++
arch/s390/include/asm/barrier.h | 15 +++
arch/score/include/asm/barrier.h | 15 +++
arch/sh/include/asm/barrier.h | 15 +++
arch/sparc/include/asm/barrier_32.h | 15 +++
arch/sparc/include/asm/barrier_64.h | 15 +++
arch/tile/include/asm/barrier.h | 15 +++
arch/unicore32/include/asm/barrier.h | 15 +++
arch/x86/include/asm/barrier.h | 15 +++
arch/xtensa/include/asm/barrier.h | 15 +++
include/linux/compiler.h | 9 +
30 files changed, 581 insertions(+), 79 deletions(-)
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -371,33 +371,35 @@ VARIETIES OF MEMORY BARRIER
And a couple of implicit varieties:
- (5) LOCK operations.
+ (5) ACQUIRE operations.
This acts as a one-way permeable barrier. It guarantees that all memory
- operations after the LOCK operation will appear to happen after the LOCK
- operation with respect to the other components of the system.
+ operations after the ACQUIRE operation will appear to happen after the
+ ACQUIRE operation with respect to the other components of the system.
- Memory operations that occur before a LOCK operation may appear to happen
- after it completes.
+ Memory operations that occur before a ACQUIRE operation may appear to
+ happen after it completes.
- A LOCK operation should almost always be paired with an UNLOCK operation.
+ A ACQUIRE operation should almost always be paired with an RELEASE
+ operation.
- (6) UNLOCK operations.
+ (6) RELEASE operations.
This also acts as a one-way permeable barrier. It guarantees that all
- memory operations before the UNLOCK operation will appear to happen before
- the UNLOCK operation with respect to the other components of the system.
+ memory operations before the RELEASE operation will appear to happen
+ before the RELEASE operation with respect to the other components of the
+ system.
- Memory operations that occur after an UNLOCK operation may appear to
+ Memory operations that occur after an RELEASE operation may appear to
happen before it completes.
- LOCK and UNLOCK operations are guaranteed to appear with respect to each
- other strictly in the order specified.
+ ACQUIRE and RELEASE operations are guaranteed to appear with respect to
+ each other strictly in the order specified.
- The use of LOCK and UNLOCK operations generally precludes the need for
- other sorts of memory barrier (but note the exceptions mentioned in the
- subsection "MMIO write barrier").
+ The use of ACQUIRE and RELEASE operations generally precludes the need
+ for other sorts of memory barrier (but note the exceptions mentioned in
+ the subsection "MMIO write barrier").
Memory barriers are only required where there's a possibility of interaction
@@ -1135,7 +1137,7 @@ CPU from reordering them.
clear_bit( ... );
This prevents memory operations before the clear leaking to after it. See
- the subsection on "Locking Functions" with reference to UNLOCK operation
+ the subsection on "Locking Functions" with reference to RELEASE operation
implications.
See Documentation/atomic_ops.txt for more information. See the "Atomic
@@ -1181,65 +1183,66 @@ LOCKING FUNCTIONS
(*) R/W semaphores
(*) RCU
-In all cases there are variants on "LOCK" operations and "UNLOCK" operations
+In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations
for each construct. These operations all imply certain barriers:
- (1) LOCK operation implication:
+ (1) ACQUIRE operation implication:
- Memory operations issued after the LOCK will be completed after the LOCK
- operation has completed.
+ Memory operations issued after the ACQUIRE will be completed after the
+ ACQUIRE operation has completed.
- Memory operations issued before the LOCK may be completed after the LOCK
- operation has completed.
+ Memory operations issued before the ACQUIRE may be completed after the
+ ACQUIRE operation has completed.
- (2) UNLOCK operation implication:
+ (2) RELEASE operation implication:
- Memory operations issued before the UNLOCK will be completed before the
- UNLOCK operation has completed.
+ Memory operations issued before the RELEASE will be completed before the
+ RELEASE operation has completed.
- Memory operations issued after the UNLOCK may be completed before the
- UNLOCK operation has completed.
+ Memory operations issued after the RELEASE may be completed before the
+ RELEASE operation has completed.
- (3) LOCK vs LOCK implication:
+ (3) ACQUIRE vs ACQUIRE implication:
- All LOCK operations issued before another LOCK operation will be completed
- before that LOCK operation.
+ All ACQUIRE operations issued before another ACQUIRE operation will be
+ completed before that ACQUIRE operation.
- (4) LOCK vs UNLOCK implication:
+ (4) ACQUIRE vs RELEASE implication:
- All LOCK operations issued before an UNLOCK operation will be completed
- before the UNLOCK operation.
+ All ACQUIRE operations issued before an RELEASE operation will be
+ completed before the RELEASE operation.
- All UNLOCK operations issued before a LOCK operation will be completed
- before the LOCK operation.
+ All RELEASE operations issued before a ACQUIRE operation will be
+ completed before the ACQUIRE operation.
- (5) Failed conditional LOCK implication:
+ (5) Failed conditional ACQUIRE implication:
- Certain variants of the LOCK operation may fail, either due to being
+ Certain variants of the ACQUIRE operation may fail, either due to being
unable to get the lock immediately, or due to receiving an unblocked
signal whilst asleep waiting for the lock to become available. Failed
locks do not imply any sort of barrier.
-Therefore, from (1), (2) and (4) an UNLOCK followed by an unconditional LOCK is
-equivalent to a full barrier, but a LOCK followed by an UNLOCK is not.
+Therefore, from (1), (2) and (4) an RELEASE followed by an unconditional
+ACQUIRE is equivalent to a full barrier, but a ACQUIRE followed by an RELEASE
+is not.
[!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way
barriers is that the effects of instructions outside of a critical section
may seep into the inside of the critical section.
-A LOCK followed by an UNLOCK may not be assumed to be full memory barrier
-because it is possible for an access preceding the LOCK to happen after the
-LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the
-two accesses can themselves then cross:
+A ACQUIRE followed by an RELEASE may not be assumed to be full memory barrier
+because it is possible for an access preceding the ACQUIRE to happen after the
+ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and
+the two accesses can themselves then cross:
*A = a;
- LOCK
- UNLOCK
+ ACQUIRE
+ RELEASE
*B = b;
may occur as:
- LOCK, STORE *B, STORE *A, UNLOCK
+ ACQUIRE, STORE *B, STORE *A, RELEASE
Locks and semaphores may not provide any guarantee of ordering on UP compiled
systems, and so cannot be counted on in such a situation to actually achieve
@@ -1253,33 +1256,33 @@ See also the section on "Inter-CPU locki
*A = a;
*B = b;
- LOCK
+ ACQUIRE
*C = c;
*D = d;
- UNLOCK
+ RELEASE
*E = e;
*F = f;
The following sequence of events is acceptable:
- LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK
+ ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE
[+] Note that {*F,*A} indicates a combined access.
But none of the following are:
- {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E
- *A, *B, *C, LOCK, *D, UNLOCK, *E, *F
- *A, *B, LOCK, *C, UNLOCK, *D, *E, *F
- *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E
+ {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E
+ *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F
+ *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F
+ *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E
INTERRUPT DISABLING FUNCTIONS
-----------------------------
-Functions that disable interrupts (LOCK equivalent) and enable interrupts
-(UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O
+Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts
+(RELEASE equivalent) will act as compiler barriers only. So if memory or I/O
barriers are required in such a situation, they must be provided from some
other means.
@@ -1436,24 +1439,24 @@ Consider the following: the system has a
CPU 1 CPU 2
=============================== ===============================
*A = a; *E = e;
- LOCK M LOCK Q
+ ACQUIRE M ACQUIRE Q
*B = b; *F = f;
*C = c; *G = g;
- UNLOCK M UNLOCK Q
+ RELEASE M RELEASE Q
*D = d; *H = h;
Then there is no guarantee as to what order CPU 3 will see the accesses to *A
through *H occur in, other than the constraints imposed by the separate locks
on the separate CPUs. It might, for example, see:
- *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M
+ *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M
But it won't see any of:
- *B, *C or *D preceding LOCK M
- *A, *B or *C following UNLOCK M
- *F, *G or *H preceding LOCK Q
- *E, *F or *G following UNLOCK Q
+ *B, *C or *D preceding ACQUIRE M
+ *A, *B or *C following RELEASE M
+ *F, *G or *H preceding ACQUIRE Q
+ *E, *F or *G following RELEASE Q
However, if the following occurs:
@@ -1461,28 +1464,28 @@ through *H occur in, other than the cons
CPU 1 CPU 2
=============================== ===============================
*A = a;
- LOCK M [1]
+ ACQUIRE M [1]
*B = b;
*C = c;
- UNLOCK M [1]
+ RELEASE M [1]
*D = d; *E = e;
- LOCK M [2]
+ ACQUIRE M [2]
*F = f;
*G = g;
- UNLOCK M [2]
+ RELEASE M [2]
*H = h;
CPU 3 might see:
- *E, LOCK M [1], *C, *B, *A, UNLOCK M [1],
- LOCK M [2], *H, *F, *G, UNLOCK M [2], *D
+ *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1],
+ ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D
But assuming CPU 1 gets the lock first, CPU 3 won't see any of:
- *B, *C, *D, *F, *G or *H preceding LOCK M [1]
- *A, *B or *C following UNLOCK M [1]
- *F, *G or *H preceding LOCK M [2]
- *A, *B, *C, *E, *F or *G following UNLOCK M [2]
+ *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1]
+ *A, *B or *C following RELEASE M [1]
+ *F, *G or *H preceding ACQUIRE M [2]
+ *A, *B, *C, *E, *F or *G following RELEASE M [2]
LOCKS VS I/O ACCESSES
@@ -1702,13 +1705,13 @@ about the state (old or new) implies an
test_and_clear_bit();
test_and_change_bit();
-These are used for such things as implementing LOCK-class and UNLOCK-class
+These are used for such things as implementing ACQUIRE-class and RELEASE-class
operations and adjusting reference counters towards object destruction, and as
such the implicit memory barrier effects are necessary.
The following operations are potential problems as they do _not_ imply memory
-barriers, but might be used for implementing such things as UNLOCK-class
+barriers, but might be used for implementing such things as RELEASE-class
operations:
atomic_set();
@@ -1750,9 +1753,9 @@ barriers are needed or not.
clear_bit_unlock();
__clear_bit_unlock();
-These implement LOCK-class and UNLOCK-class operations. These should be used in
-preference to other operations when implementing locking primitives, because
-their implementations can be optimised on many architectures.
+These implement ACQUIRE-class and RELEASE-class operations. These should be
+used in preference to other operations when implementing locking primitives,
+because their implementations can be optimised on many architectures.
[!] Note that special memory barrier primitives are available for these
situations because on some CPUs the atomic instructions used imply full memory
--- a/arch/alpha/include/asm/barrier.h
+++ b/arch/alpha/include/asm/barrier.h
@@ -29,6 +29,21 @@ __asm__ __volatile__("mb": : :"memory")
#define smp_read_barrier_depends() do { } while (0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#define set_mb(var, value) \
do { var = value; mb(); } while (0)
--- a/arch/arc/include/asm/barrier.h
+++ b/arch/arc/include/asm/barrier.h
@@ -30,6 +30,21 @@
#define smp_wmb() barrier()
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#define smp_mb__before_atomic_dec() barrier()
#define smp_mb__after_atomic_dec() barrier()
#define smp_mb__before_atomic_inc() barrier()
--- a/arch/arm/include/asm/barrier.h
+++ b/arch/arm/include/asm/barrier.h
@@ -59,6 +59,21 @@
#define smp_wmb() dmb(ishst)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#define read_barrier_depends() do { } while(0)
#define smp_read_barrier_depends() do { } while(0)
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -35,11 +35,59 @@
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#else
+
#define smp_mb() asm volatile("dmb ish" : : : "memory")
#define smp_rmb() asm volatile("dmb ishld" : : : "memory")
#define smp_wmb() asm volatile("dmb ishst" : : : "memory")
-#endif
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("stlr %w1, [%0]" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("stlr %1, [%0]" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ } \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("ldar %w0, [%1]" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("ldar %0, [%1]" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ } \
+ ___p1; \
+})
#define read_barrier_depends() do { } while(0)
#define smp_read_barrier_depends() do { } while(0)
--- a/arch/avr32/include/asm/barrier.h
+++ b/arch/avr32/include/asm/barrier.h
@@ -25,5 +25,19 @@
# define smp_read_barrier_depends() do { } while(0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
#endif /* __ASM_AVR32_BARRIER_H */
--- a/arch/blackfin/include/asm/barrier.h
+++ b/arch/blackfin/include/asm/barrier.h
@@ -45,4 +45,19 @@
#define set_mb(var, value) do { var = value; mb(); } while (0)
#define smp_read_barrier_depends() read_barrier_depends()
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _BLACKFIN_BARRIER_H */
--- a/arch/cris/include/asm/barrier.h
+++ b/arch/cris/include/asm/barrier.h
@@ -22,4 +22,19 @@
#define smp_read_barrier_depends() do { } while(0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __ASM_CRIS_BARRIER_H */
--- a/arch/frv/include/asm/barrier.h
+++ b/arch/frv/include/asm/barrier.h
@@ -26,4 +26,19 @@
#define set_mb(var, value) \
do { var = (value); barrier(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_BARRIER_H */
--- a/arch/h8300/include/asm/barrier.h
+++ b/arch/h8300/include/asm/barrier.h
@@ -26,4 +26,19 @@
#define smp_read_barrier_depends() do { } while(0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _H8300_BARRIER_H */
--- a/arch/hexagon/include/asm/barrier.h
+++ b/arch/hexagon/include/asm/barrier.h
@@ -38,4 +38,19 @@
#define set_mb(var, value) \
do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_BARRIER_H */
--- a/arch/ia64/include/asm/barrier.h
+++ b/arch/ia64/include/asm/barrier.h
@@ -45,11 +45,60 @@
# define smp_rmb() rmb()
# define smp_wmb() wmb()
# define smp_read_barrier_depends() read_barrier_depends()
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("st4.rel [%0]=%1" \
+ : "=r" (p) : "r" (v) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("st8.rel [%0]=%1" \
+ : "=r" (p) : "r" (v) : "memory"); \
+ break; \
+ } \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("ld4.acq %0=[%1]" \
+ : "=r" (___p1) : "r" (p) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("ld8.acq %0=[%1]" \
+ : "=r" (___p1) : "r" (p) : "memory"); \
+ break; \
+ } \
+ ___p1; \
+})
+
#else
+
# define smp_mb() barrier()
# define smp_rmb() barrier()
# define smp_wmb() barrier()
# define smp_read_barrier_depends() do { } while(0)
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
#endif
/*
--- a/arch/m32r/include/asm/barrier.h
+++ b/arch/m32r/include/asm/barrier.h
@@ -91,4 +91,19 @@
#define set_mb(var, value) do { var = value; barrier(); } while (0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_M32R_BARRIER_H */
--- a/arch/m68k/include/asm/barrier.h
+++ b/arch/m68k/include/asm/barrier.h
@@ -17,4 +17,19 @@
#define smp_wmb() barrier()
#define smp_read_barrier_depends() ((void)0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _M68K_BARRIER_H */
--- a/arch/metag/include/asm/barrier.h
+++ b/arch/metag/include/asm/barrier.h
@@ -82,4 +82,19 @@ static inline void fence(void)
#define smp_read_barrier_depends() do { } while (0)
#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_METAG_BARRIER_H */
--- a/arch/microblaze/include/asm/barrier.h
+++ b/arch/microblaze/include/asm/barrier.h
@@ -24,4 +24,19 @@
#define smp_rmb() rmb()
#define smp_wmb() wmb()
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_MICROBLAZE_BARRIER_H */
--- a/arch/mips/include/asm/barrier.h
+++ b/arch/mips/include/asm/barrier.h
@@ -180,4 +180,19 @@
#define nudge_writes() mb()
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
--- a/arch/mn10300/include/asm/barrier.h
+++ b/arch/mn10300/include/asm/barrier.h
@@ -34,4 +34,19 @@
#define read_barrier_depends() do {} while (0)
#define smp_read_barrier_depends() do {} while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_BARRIER_H */
--- a/arch/parisc/include/asm/barrier.h
+++ b/arch/parisc/include/asm/barrier.h
@@ -32,4 +32,19 @@
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __PARISC_BARRIER_H */
--- a/arch/powerpc/include/asm/barrier.h
+++ b/arch/powerpc/include/asm/barrier.h
@@ -45,11 +45,15 @@
# define SMPWMB eieio
#endif
+#define __lwsync() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+
#define smp_mb() mb()
-#define smp_rmb() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+#define smp_rmb() __lwsync()
#define smp_wmb() __asm__ __volatile__ (stringify_in_c(SMPWMB) : : :"memory")
#define smp_read_barrier_depends() read_barrier_depends()
#else
+#define __lwsync() barrier()
+
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
@@ -65,4 +69,19 @@
#define data_barrier(x) \
asm volatile("twi 0,%0,0; isync" : : "r" (x) : "memory");
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ___p1; \
+})
+
#endif /* _ASM_POWERPC_BARRIER_H */
--- a/arch/s390/include/asm/barrier.h
+++ b/arch/s390/include/asm/barrier.h
@@ -32,4 +32,19 @@
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
--- a/arch/score/include/asm/barrier.h
+++ b/arch/score/include/asm/barrier.h
@@ -13,4 +13,19 @@
#define set_mb(var, value) do {var = value; wmb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_SCORE_BARRIER_H */
--- a/arch/sh/include/asm/barrier.h
+++ b/arch/sh/include/asm/barrier.h
@@ -51,4 +51,19 @@
#define set_mb(var, value) do { (void)xchg(&var, value); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __ASM_SH_BARRIER_H */
--- a/arch/sparc/include/asm/barrier_32.h
+++ b/arch/sparc/include/asm/barrier_32.h
@@ -12,4 +12,19 @@
#define smp_wmb() __asm__ __volatile__("":::"memory")
#define smp_read_barrier_depends() do { } while(0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* !(__SPARC_BARRIER_H) */
--- a/arch/sparc/include/asm/barrier_64.h
+++ b/arch/sparc/include/asm/barrier_64.h
@@ -53,4 +53,19 @@ do { __asm__ __volatile__("ba,pt %%xcc,
#define smp_read_barrier_depends() do { } while(0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* !(__SPARC64_BARRIER_H) */
--- a/arch/tile/include/asm/barrier.h
+++ b/arch/tile/include/asm/barrier.h
@@ -140,5 +140,20 @@ mb_incoherent(void)
#define set_mb(var, value) \
do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_TILE_BARRIER_H */
--- a/arch/unicore32/include/asm/barrier.h
+++ b/arch/unicore32/include/asm/barrier.h
@@ -25,4 +25,19 @@
#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __UNICORE_BARRIER_H__ */
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -100,6 +100,21 @@
#define set_mb(var, value) do { var = value; barrier(); } while (0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
/*
* Stop RDTSC speculation. This is needed when you need to use RDTSC
* (or get_cycles or vread that possibly accesses the TSC) in a defined
--- a/arch/xtensa/include/asm/barrier.h
+++ b/arch/xtensa/include/asm/barrier.h
@@ -26,4 +26,19 @@
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _XTENSA_SYSTEM_H */
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -298,6 +298,11 @@ void ftrace_likely_update(struct ftrace_
# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
#endif
+/* Is this type a native word size -- useful for atomic operations */
+#ifndef __native_word
+# define __native_word(t) (sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
+#endif
+
/* Compile time object size, -1 for unknown */
#ifndef __compiletime_object_size
# define __compiletime_object_size(obj) -1
@@ -337,6 +342,10 @@ void ftrace_likely_update(struct ftrace_
#define compiletime_assert(condition, msg) \
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
+#define compiletime_assert_atomic_type(t) \
+ compiletime_assert(__native_word(t), \
+ "Need native word sized stores/loads for atomicity.")
+
/*
* Prevent the compiler from merging or refetching accesses. The compiler
* is also forbidden from reordering successive instances of ACCESS_ONCE(),
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Geert Uytterhoeven @ 2013-11-06 12:51 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Michael Neuling, Tony Luck, Mathieu Desnoyers, Heiko Carstens,
Oleg Nesterov, LKML, Linux PPC dev, Anton Blanchard,
Frederic Weisbecker, Victor Kaplansky, Russell King,
Paul E. McKenney, Linus Torvalds, Martin Schwidefsky
In-Reply-To: <20131106123946.GJ10651@twins.programming.kicks-ass.net>
On Wed, Nov 6, 2013 at 1:39 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> Documentation/memory-barriers.txt | 157 +++++++++++++++++-----------------
> arch/alpha/include/asm/barrier.h | 15 +++
> arch/arc/include/asm/barrier.h | 15 +++
> arch/arm/include/asm/barrier.h | 15 +++
> arch/arm64/include/asm/barrier.h | 50 ++++++++++
> arch/avr32/include/asm/barrier.h | 14 +++
> arch/blackfin/include/asm/barrier.h | 15 +++
> arch/cris/include/asm/barrier.h | 15 +++
> arch/frv/include/asm/barrier.h | 15 +++
> arch/h8300/include/asm/barrier.h | 15 +++
> arch/hexagon/include/asm/barrier.h | 15 +++
> arch/ia64/include/asm/barrier.h | 49 ++++++++++
> arch/m32r/include/asm/barrier.h | 15 +++
> arch/m68k/include/asm/barrier.h | 15 +++
> arch/metag/include/asm/barrier.h | 15 +++
> arch/microblaze/include/asm/barrier.h | 15 +++
> arch/mips/include/asm/barrier.h | 15 +++
> arch/mn10300/include/asm/barrier.h | 15 +++
> arch/parisc/include/asm/barrier.h | 15 +++
> arch/powerpc/include/asm/barrier.h | 21 ++++
> arch/s390/include/asm/barrier.h | 15 +++
> arch/score/include/asm/barrier.h | 15 +++
> arch/sh/include/asm/barrier.h | 15 +++
> arch/sparc/include/asm/barrier_32.h | 15 +++
> arch/sparc/include/asm/barrier_64.h | 15 +++
> arch/tile/include/asm/barrier.h | 15 +++
> arch/unicore32/include/asm/barrier.h | 15 +++
> arch/x86/include/asm/barrier.h | 15 +++
> arch/xtensa/include/asm/barrier.h | 15 +++
> include/linux/compiler.h | 9 +
> 30 files changed, 581 insertions(+), 79 deletions(-)
This is screaming for a default implementation in asm-generic.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Peter Zijlstra @ 2013-11-06 13:57 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Michael Neuling, Tony Luck, Mathieu Desnoyers, Heiko Carstens,
Oleg Nesterov, LKML, Linux PPC dev, Anton Blanchard,
Frederic Weisbecker, Victor Kaplansky, Russell King,
Paul E. McKenney, Linus Torvalds, Martin Schwidefsky
In-Reply-To: <CAMuHMdXZjQJvT7R4ODbDNBgAVwPUBPjB6UoR66_sNuFx7opUDw@mail.gmail.com>
On Wed, Nov 06, 2013 at 01:51:10PM +0100, Geert Uytterhoeven wrote:
> This is screaming for a default implementation in asm-generic.
Right you are... how about a little something like this?
There's a few archs I didn't fully merge with the generic one because of
weird nop implementations.
asm volatile ("nop" :: ) vs asm volatile ("nop" ::: "memory") and the
like. They probably can (and should) use the regular asm volatile
("nop") but I misplaced the toolchains for many of the weird archs so I
didn't attempt.
Also fixed a silly mistake in the return type definition for most
smp_load_acquire() implementions: typeof(p) vs typeof(*p).
---
Subject: arch: Introduce smp_load_acquire(), smp_store_release()
From: Peter Zijlstra <peterz@infradead.org>
Date: Mon, 4 Nov 2013 20:18:11 +0100
A number of situations currently require the heavyweight smp_mb(),
even though there is no need to order prior stores against later
loads. Many architectures have much cheaper ways to handle these
situations, but the Linux kernel currently has no portable way
to make use of them.
This commit therefore supplies smp_load_acquire() and
smp_store_release() to remedy this situation. The new
smp_load_acquire() primitive orders the specified load against
any subsequent reads or writes, while the new smp_store_release()
primitive orders the specifed store against any prior reads or
writes. These primitives allow array-based circular FIFOs to be
implemented without an smp_mb(), and also allow a theoretical
hole in rcu_assign_pointer() to be closed at no additional
expense on most architectures.
In addition, the RCU experience transitioning from explicit
smp_read_barrier_depends() and smp_wmb() to rcu_dereference()
and rcu_assign_pointer(), respectively resulted in substantial
improvements in readability. It therefore seems likely that
replacing other explicit barriers with smp_load_acquire() and
smp_store_release() will provide similar benefits. It appears
that roughly half of the explicit barriers in core kernel code
might be so replaced.
Cc: Michael Ellerman <michael@ellerman.id.au>
Cc: Michael Neuling <mikey@neuling.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Victor Kaplansky <VICTORK@il.ibm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
---
Documentation/memory-barriers.txt | 157 +++++++++++++++++-----------------
arch/alpha/include/asm/barrier.h | 25 +----
arch/arc/include/asm/Kbuild | 1
arch/arc/include/asm/atomic.h | 5 +
arch/arc/include/asm/barrier.h | 42 ---------
arch/arm/include/asm/barrier.h | 15 +++
arch/arm64/include/asm/barrier.h | 50 ++++++++++
arch/avr32/include/asm/barrier.h | 17 +--
arch/blackfin/include/asm/barrier.h | 18 ---
arch/cris/include/asm/Kbuild | 1
arch/cris/include/asm/barrier.h | 25 -----
arch/frv/include/asm/barrier.h | 8 -
arch/h8300/include/asm/barrier.h | 21 ----
arch/hexagon/include/asm/Kbuild | 1
arch/hexagon/include/asm/barrier.h | 41 --------
arch/ia64/include/asm/barrier.h | 49 ++++++++++
arch/m32r/include/asm/barrier.h | 80 -----------------
arch/m68k/include/asm/barrier.h | 14 ---
arch/metag/include/asm/barrier.h | 15 +++
arch/microblaze/include/asm/Kbuild | 1
arch/microblaze/include/asm/barrier.h | 27 -----
arch/mips/include/asm/barrier.h | 15 +++
arch/mn10300/include/asm/Kbuild | 1
arch/mn10300/include/asm/barrier.h | 37 --------
arch/parisc/include/asm/Kbuild | 1
arch/parisc/include/asm/barrier.h | 35 -------
arch/powerpc/include/asm/barrier.h | 21 ++++
arch/s390/include/asm/barrier.h | 15 +++
arch/score/include/asm/Kbuild | 1
arch/score/include/asm/barrier.h | 16 ---
arch/sh/include/asm/barrier.h | 21 ----
arch/sparc/include/asm/barrier_32.h | 11 --
arch/sparc/include/asm/barrier_64.h | 15 +++
arch/tile/include/asm/barrier.h | 68 --------------
arch/unicore32/include/asm/barrier.h | 11 --
arch/x86/include/asm/barrier.h | 15 +++
arch/xtensa/include/asm/barrier.h | 9 -
include/asm-generic/barrier.h | 55 +++++++++--
include/linux/compiler.h | 9 +
39 files changed, 375 insertions(+), 594 deletions(-)
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -371,33 +371,35 @@ VARIETIES OF MEMORY BARRIER
And a couple of implicit varieties:
- (5) LOCK operations.
+ (5) ACQUIRE operations.
This acts as a one-way permeable barrier. It guarantees that all memory
- operations after the LOCK operation will appear to happen after the LOCK
- operation with respect to the other components of the system.
+ operations after the ACQUIRE operation will appear to happen after the
+ ACQUIRE operation with respect to the other components of the system.
- Memory operations that occur before a LOCK operation may appear to happen
- after it completes.
+ Memory operations that occur before a ACQUIRE operation may appear to
+ happen after it completes.
- A LOCK operation should almost always be paired with an UNLOCK operation.
+ A ACQUIRE operation should almost always be paired with an RELEASE
+ operation.
- (6) UNLOCK operations.
+ (6) RELEASE operations.
This also acts as a one-way permeable barrier. It guarantees that all
- memory operations before the UNLOCK operation will appear to happen before
- the UNLOCK operation with respect to the other components of the system.
+ memory operations before the RELEASE operation will appear to happen
+ before the RELEASE operation with respect to the other components of the
+ system.
- Memory operations that occur after an UNLOCK operation may appear to
+ Memory operations that occur after an RELEASE operation may appear to
happen before it completes.
- LOCK and UNLOCK operations are guaranteed to appear with respect to each
- other strictly in the order specified.
+ ACQUIRE and RELEASE operations are guaranteed to appear with respect to
+ each other strictly in the order specified.
- The use of LOCK and UNLOCK operations generally precludes the need for
- other sorts of memory barrier (but note the exceptions mentioned in the
- subsection "MMIO write barrier").
+ The use of ACQUIRE and RELEASE operations generally precludes the need
+ for other sorts of memory barrier (but note the exceptions mentioned in
+ the subsection "MMIO write barrier").
Memory barriers are only required where there's a possibility of interaction
@@ -1135,7 +1137,7 @@ CPU from reordering them.
clear_bit( ... );
This prevents memory operations before the clear leaking to after it. See
- the subsection on "Locking Functions" with reference to UNLOCK operation
+ the subsection on "Locking Functions" with reference to RELEASE operation
implications.
See Documentation/atomic_ops.txt for more information. See the "Atomic
@@ -1181,65 +1183,66 @@ LOCKING FUNCTIONS
(*) R/W semaphores
(*) RCU
-In all cases there are variants on "LOCK" operations and "UNLOCK" operations
+In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations
for each construct. These operations all imply certain barriers:
- (1) LOCK operation implication:
+ (1) ACQUIRE operation implication:
- Memory operations issued after the LOCK will be completed after the LOCK
- operation has completed.
+ Memory operations issued after the ACQUIRE will be completed after the
+ ACQUIRE operation has completed.
- Memory operations issued before the LOCK may be completed after the LOCK
- operation has completed.
+ Memory operations issued before the ACQUIRE may be completed after the
+ ACQUIRE operation has completed.
- (2) UNLOCK operation implication:
+ (2) RELEASE operation implication:
- Memory operations issued before the UNLOCK will be completed before the
- UNLOCK operation has completed.
+ Memory operations issued before the RELEASE will be completed before the
+ RELEASE operation has completed.
- Memory operations issued after the UNLOCK may be completed before the
- UNLOCK operation has completed.
+ Memory operations issued after the RELEASE may be completed before the
+ RELEASE operation has completed.
- (3) LOCK vs LOCK implication:
+ (3) ACQUIRE vs ACQUIRE implication:
- All LOCK operations issued before another LOCK operation will be completed
- before that LOCK operation.
+ All ACQUIRE operations issued before another ACQUIRE operation will be
+ completed before that ACQUIRE operation.
- (4) LOCK vs UNLOCK implication:
+ (4) ACQUIRE vs RELEASE implication:
- All LOCK operations issued before an UNLOCK operation will be completed
- before the UNLOCK operation.
+ All ACQUIRE operations issued before an RELEASE operation will be
+ completed before the RELEASE operation.
- All UNLOCK operations issued before a LOCK operation will be completed
- before the LOCK operation.
+ All RELEASE operations issued before a ACQUIRE operation will be
+ completed before the ACQUIRE operation.
- (5) Failed conditional LOCK implication:
+ (5) Failed conditional ACQUIRE implication:
- Certain variants of the LOCK operation may fail, either due to being
+ Certain variants of the ACQUIRE operation may fail, either due to being
unable to get the lock immediately, or due to receiving an unblocked
signal whilst asleep waiting for the lock to become available. Failed
locks do not imply any sort of barrier.
-Therefore, from (1), (2) and (4) an UNLOCK followed by an unconditional LOCK is
-equivalent to a full barrier, but a LOCK followed by an UNLOCK is not.
+Therefore, from (1), (2) and (4) an RELEASE followed by an unconditional
+ACQUIRE is equivalent to a full barrier, but a ACQUIRE followed by an RELEASE
+is not.
[!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way
barriers is that the effects of instructions outside of a critical section
may seep into the inside of the critical section.
-A LOCK followed by an UNLOCK may not be assumed to be full memory barrier
-because it is possible for an access preceding the LOCK to happen after the
-LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the
-two accesses can themselves then cross:
+A ACQUIRE followed by an RELEASE may not be assumed to be full memory barrier
+because it is possible for an access preceding the ACQUIRE to happen after the
+ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and
+the two accesses can themselves then cross:
*A = a;
- LOCK
- UNLOCK
+ ACQUIRE
+ RELEASE
*B = b;
may occur as:
- LOCK, STORE *B, STORE *A, UNLOCK
+ ACQUIRE, STORE *B, STORE *A, RELEASE
Locks and semaphores may not provide any guarantee of ordering on UP compiled
systems, and so cannot be counted on in such a situation to actually achieve
@@ -1253,33 +1256,33 @@ See also the section on "Inter-CPU locki
*A = a;
*B = b;
- LOCK
+ ACQUIRE
*C = c;
*D = d;
- UNLOCK
+ RELEASE
*E = e;
*F = f;
The following sequence of events is acceptable:
- LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK
+ ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE
[+] Note that {*F,*A} indicates a combined access.
But none of the following are:
- {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E
- *A, *B, *C, LOCK, *D, UNLOCK, *E, *F
- *A, *B, LOCK, *C, UNLOCK, *D, *E, *F
- *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E
+ {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E
+ *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F
+ *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F
+ *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E
INTERRUPT DISABLING FUNCTIONS
-----------------------------
-Functions that disable interrupts (LOCK equivalent) and enable interrupts
-(UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O
+Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts
+(RELEASE equivalent) will act as compiler barriers only. So if memory or I/O
barriers are required in such a situation, they must be provided from some
other means.
@@ -1436,24 +1439,24 @@ Consider the following: the system has a
CPU 1 CPU 2
=============================== ===============================
*A = a; *E = e;
- LOCK M LOCK Q
+ ACQUIRE M ACQUIRE Q
*B = b; *F = f;
*C = c; *G = g;
- UNLOCK M UNLOCK Q
+ RELEASE M RELEASE Q
*D = d; *H = h;
Then there is no guarantee as to what order CPU 3 will see the accesses to *A
through *H occur in, other than the constraints imposed by the separate locks
on the separate CPUs. It might, for example, see:
- *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M
+ *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M
But it won't see any of:
- *B, *C or *D preceding LOCK M
- *A, *B or *C following UNLOCK M
- *F, *G or *H preceding LOCK Q
- *E, *F or *G following UNLOCK Q
+ *B, *C or *D preceding ACQUIRE M
+ *A, *B or *C following RELEASE M
+ *F, *G or *H preceding ACQUIRE Q
+ *E, *F or *G following RELEASE Q
However, if the following occurs:
@@ -1461,28 +1464,28 @@ through *H occur in, other than the cons
CPU 1 CPU 2
=============================== ===============================
*A = a;
- LOCK M [1]
+ ACQUIRE M [1]
*B = b;
*C = c;
- UNLOCK M [1]
+ RELEASE M [1]
*D = d; *E = e;
- LOCK M [2]
+ ACQUIRE M [2]
*F = f;
*G = g;
- UNLOCK M [2]
+ RELEASE M [2]
*H = h;
CPU 3 might see:
- *E, LOCK M [1], *C, *B, *A, UNLOCK M [1],
- LOCK M [2], *H, *F, *G, UNLOCK M [2], *D
+ *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1],
+ ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D
But assuming CPU 1 gets the lock first, CPU 3 won't see any of:
- *B, *C, *D, *F, *G or *H preceding LOCK M [1]
- *A, *B or *C following UNLOCK M [1]
- *F, *G or *H preceding LOCK M [2]
- *A, *B, *C, *E, *F or *G following UNLOCK M [2]
+ *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1]
+ *A, *B or *C following RELEASE M [1]
+ *F, *G or *H preceding ACQUIRE M [2]
+ *A, *B, *C, *E, *F or *G following RELEASE M [2]
LOCKS VS I/O ACCESSES
@@ -1702,13 +1705,13 @@ about the state (old or new) implies an
test_and_clear_bit();
test_and_change_bit();
-These are used for such things as implementing LOCK-class and UNLOCK-class
+These are used for such things as implementing ACQUIRE-class and RELEASE-class
operations and adjusting reference counters towards object destruction, and as
such the implicit memory barrier effects are necessary.
The following operations are potential problems as they do _not_ imply memory
-barriers, but might be used for implementing such things as UNLOCK-class
+barriers, but might be used for implementing such things as RELEASE-class
operations:
atomic_set();
@@ -1750,9 +1753,9 @@ barriers are needed or not.
clear_bit_unlock();
__clear_bit_unlock();
-These implement LOCK-class and UNLOCK-class operations. These should be used in
-preference to other operations when implementing locking primitives, because
-their implementations can be optimised on many architectures.
+These implement ACQUIRE-class and RELEASE-class operations. These should be
+used in preference to other operations when implementing locking primitives,
+because their implementations can be optimised on many architectures.
[!] Note that special memory barrier primitives are available for these
situations because on some CPUs the atomic instructions used imply full memory
--- a/arch/alpha/include/asm/barrier.h
+++ b/arch/alpha/include/asm/barrier.h
@@ -3,33 +3,18 @@
#include <asm/compiler.h>
-#define mb() \
-__asm__ __volatile__("mb": : :"memory")
+#define mb() __asm__ __volatile__("mb": : :"memory")
+#define rmb() __asm__ __volatile__("mb": : :"memory")
+#define wmb() __asm__ __volatile__("wmb": : :"memory")
-#define rmb() \
-__asm__ __volatile__("mb": : :"memory")
-
-#define wmb() \
-__asm__ __volatile__("wmb": : :"memory")
-
-#define read_barrier_depends() \
-__asm__ __volatile__("mb": : :"memory")
+#define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
#ifdef CONFIG_SMP
#define __ASM_SMP_MB "\tmb\n"
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
#else
#define __ASM_SMP_MB
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
#endif
-#define set_mb(var, value) \
-do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* __BARRIER_H */
--- a/arch/arc/include/asm/Kbuild
+++ b/arch/arc/include/asm/Kbuild
@@ -47,3 +47,4 @@ generic-y += user.h
generic-y += vga.h
generic-y += xor.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/arc/include/asm/atomic.h
+++ b/arch/arc/include/asm/atomic.h
@@ -190,6 +190,11 @@ static inline void atomic_clear_mask(uns
#endif /* !CONFIG_ARC_HAS_LLSC */
+#define smp_mb__before_atomic_dec() barrier()
+#define smp_mb__after_atomic_dec() barrier()
+#define smp_mb__before_atomic_inc() barrier()
+#define smp_mb__after_atomic_inc() barrier()
+
/**
* __atomic_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
--- a/arch/arc/include/asm/barrier.h
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef __ASM_BARRIER_H
-#define __ASM_BARRIER_H
-
-#ifndef __ASSEMBLY__
-
-/* TODO-vineetg: Need to see what this does, don't we need sync anywhere */
-#define mb() __asm__ __volatile__ ("" : : : "memory")
-#define rmb() mb()
-#define wmb() mb()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-#define read_barrier_depends() mb()
-
-/* TODO-vineetg verify the correctness of macros here */
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#endif
-
-#define smp_mb__before_atomic_dec() barrier()
-#define smp_mb__after_atomic_dec() barrier()
-#define smp_mb__before_atomic_inc() barrier()
-#define smp_mb__after_atomic_inc() barrier()
-
-#define smp_read_barrier_depends() do { } while (0)
-
-#endif
-
-#endif
--- a/arch/arm/include/asm/barrier.h
+++ b/arch/arm/include/asm/barrier.h
@@ -59,6 +59,21 @@
#define smp_wmb() dmb(ishst)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#define read_barrier_depends() do { } while(0)
#define smp_read_barrier_depends() do { } while(0)
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -35,11 +35,59 @@
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#else
+
#define smp_mb() asm volatile("dmb ish" : : : "memory")
#define smp_rmb() asm volatile("dmb ishld" : : : "memory")
#define smp_wmb() asm volatile("dmb ishst" : : : "memory")
-#endif
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("stlr %w1, [%0]" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("stlr %1, [%0]" \
+ : "=Q" (*p) : "r" (v) : "memory"); \
+ break; \
+ } \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("ldar %w0, [%1]" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("ldar %0, [%1]" \
+ : "=r" (___p1) : "Q" (*p) : "memory"); \
+ break; \
+ } \
+ ___p1; \
+})
#define read_barrier_depends() do { } while(0)
#define smp_read_barrier_depends() do { } while(0)
--- a/arch/avr32/include/asm/barrier.h
+++ b/arch/avr32/include/asm/barrier.h
@@ -8,22 +8,15 @@
#ifndef __ASM_AVR32_BARRIER_H
#define __ASM_AVR32_BARRIER_H
-#define nop() asm volatile("nop")
-
-#define mb() asm volatile("" : : : "memory")
-#define rmb() mb()
-#define wmb() asm volatile("sync 0" : : : "memory")
-#define read_barrier_depends() do { } while(0)
-#define set_mb(var, value) do { var = value; mb(); } while(0)
+/*
+ * Weirdest thing ever.. no full barrier, but it has a write barrier!
+ */
+#define wmb() asm volatile("sync 0" : : : "memory")
#ifdef CONFIG_SMP
# error "The AVR32 port does not support SMP"
-#else
-# define smp_mb() barrier()
-# define smp_rmb() barrier()
-# define smp_wmb() barrier()
-# define smp_read_barrier_depends() do { } while(0)
#endif
+#include <asm-generic/barrier.h>
#endif /* __ASM_AVR32_BARRIER_H */
--- a/arch/blackfin/include/asm/barrier.h
+++ b/arch/blackfin/include/asm/barrier.h
@@ -23,26 +23,10 @@
# define rmb() do { barrier(); smp_check_barrier(); } while (0)
# define wmb() do { barrier(); smp_mark_barrier(); } while (0)
# define read_barrier_depends() do { barrier(); smp_check_barrier(); } while (0)
-#else
-# define mb() barrier()
-# define rmb() barrier()
-# define wmb() barrier()
-# define read_barrier_depends() do { } while (0)
#endif
-#else /* !CONFIG_SMP */
-
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define read_barrier_depends() do { } while (0)
-
#endif /* !CONFIG_SMP */
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define smp_read_barrier_depends() read_barrier_depends()
+#include <asm-generic/barrier.h>
#endif /* _BLACKFIN_BARRIER_H */
--- a/arch/cris/include/asm/Kbuild
+++ b/arch/cris/include/asm/Kbuild
@@ -12,3 +12,4 @@ generic-y += trace_clock.h
generic-y += vga.h
generic-y += xor.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/cris/include/asm/barrier.h
+++ /dev/null
@@ -1,25 +0,0 @@
-#ifndef __ASM_CRIS_BARRIER_H
-#define __ASM_CRIS_BARRIER_H
-
-#define nop() __asm__ __volatile__ ("nop");
-
-#define barrier() __asm__ __volatile__("": : :"memory")
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-#define read_barrier_depends() do { } while(0)
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while(0)
-#endif
-
-#endif /* __ASM_CRIS_BARRIER_H */
--- a/arch/frv/include/asm/barrier.h
+++ b/arch/frv/include/asm/barrier.h
@@ -17,13 +17,7 @@
#define mb() asm volatile ("membar" : : :"memory")
#define rmb() asm volatile ("membar" : : :"memory")
#define wmb() asm volatile ("membar" : : :"memory")
-#define read_barrier_depends() do { } while (0)
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do {} while(0)
-#define set_mb(var, value) \
- do { var = (value); barrier(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* _ASM_BARRIER_H */
--- a/arch/h8300/include/asm/barrier.h
+++ b/arch/h8300/include/asm/barrier.h
@@ -3,27 +3,8 @@
#define nop() asm volatile ("nop"::)
-/*
- * Force strict CPU ordering.
- * Not really required on H8...
- */
-#define mb() asm volatile ("" : : :"memory")
-#define rmb() asm volatile ("" : : :"memory")
-#define wmb() asm volatile ("" : : :"memory")
#define set_mb(var, value) do { xchg(&var, value); } while (0)
-#define read_barrier_depends() do { } while (0)
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while(0)
-#endif
+#include <asm-generic/barrier.h>
#endif /* _H8300_BARRIER_H */
--- a/arch/hexagon/include/asm/Kbuild
+++ b/arch/hexagon/include/asm/Kbuild
@@ -54,3 +54,4 @@ generic-y += ucontext.h
generic-y += unaligned.h
generic-y += xor.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/hexagon/include/asm/barrier.h
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
- * Memory barrier definitions for the Hexagon architecture
- *
- * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 and
- * only version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301, USA.
- */
-
-#ifndef _ASM_BARRIER_H
-#define _ASM_BARRIER_H
-
-#define rmb() barrier()
-#define read_barrier_depends() barrier()
-#define wmb() barrier()
-#define mb() barrier()
-#define smp_rmb() barrier()
-#define smp_read_barrier_depends() barrier()
-#define smp_wmb() barrier()
-#define smp_mb() barrier()
-#define smp_mb__before_atomic_dec() barrier()
-#define smp_mb__after_atomic_dec() barrier()
-#define smp_mb__before_atomic_inc() barrier()
-#define smp_mb__after_atomic_inc() barrier()
-
-/* Set a value and use a memory barrier. Used by the scheduler somewhere. */
-#define set_mb(var, value) \
- do { var = value; mb(); } while (0)
-
-#endif /* _ASM_BARRIER_H */
--- a/arch/ia64/include/asm/barrier.h
+++ b/arch/ia64/include/asm/barrier.h
@@ -45,11 +45,60 @@
# define smp_rmb() rmb()
# define smp_wmb() wmb()
# define smp_read_barrier_depends() read_barrier_depends()
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("st4.rel [%0]=%1" \
+ : "=r" (p) : "r" (v) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("st8.rel [%0]=%1" \
+ : "=r" (p) : "r" (v) : "memory"); \
+ break; \
+ } \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 4: \
+ asm volatile ("ld4.acq %0=[%1]" \
+ : "=r" (___p1) : "r" (p) : "memory"); \
+ break; \
+ case 8: \
+ asm volatile ("ld8.acq %0=[%1]" \
+ : "=r" (___p1) : "r" (p) : "memory"); \
+ break; \
+ } \
+ ___p1; \
+})
+
#else
+
# define smp_mb() barrier()
# define smp_rmb() barrier()
# define smp_wmb() barrier()
# define smp_read_barrier_depends() do { } while(0)
+
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
#endif
/*
--- a/arch/m32r/include/asm/barrier.h
+++ b/arch/m32r/include/asm/barrier.h
@@ -11,84 +11,6 @@
#define nop() __asm__ __volatile__ ("nop" : : )
-/*
- * Memory barrier.
- *
- * mb() prevents loads and stores being reordered across this point.
- * rmb() prevents loads being reordered across this point.
- * wmb() prevents stores being reordered across this point.
- */
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-
-/**
- * read_barrier_depends - Flush all pending reads that subsequents reads
- * depend on.
- *
- * No data-dependent reads from memory-like regions are ever reordered
- * over this barrier. All reads preceding this primitive are guaranteed
- * to access memory (but not necessarily other CPUs' caches) before any
- * reads following this primitive that depend on the data return by
- * any of the preceding reads. This primitive is much lighter weight than
- * rmb() on most CPUs, and is never heavier weight than is
- * rmb().
- *
- * These ordering constraints are respected by both the local CPU
- * and the compiler.
- *
- * Ordering is not guaranteed by anything other than these primitives,
- * not even by data dependencies. See the documentation for
- * memory_barrier() for examples and URLs to more information.
- *
- * For example, the following code would force ordering (the initial
- * value of "a" is zero, "b" is one, and "p" is "&a"):
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * b = 2;
- * memory_barrier();
- * p = &b; q = p;
- * read_barrier_depends();
- * d = *q;
- * </programlisting>
- *
- *
- * because the read of "*q" depends on the read of "p" and these
- * two reads are separated by a read_barrier_depends(). However,
- * the following code, with the same initial values for "a" and "b":
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * a = 2;
- * memory_barrier();
- * b = 3; y = b;
- * read_barrier_depends();
- * x = a;
- * </programlisting>
- *
- * does not enforce ordering, since there is no data dependency between
- * the read of "a" and the read of "b". Therefore, on some CPUs, such
- * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
- * in cases like this where there are no data dependencies.
- **/
-
-#define read_barrier_depends() do { } while (0)
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#define set_mb(var, value) do { (void) xchg(&var, value); } while (0)
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
-#define set_mb(var, value) do { var = value; barrier(); } while (0)
-#endif
+#include <asm-generic/barrier.h>
#endif /* _ASM_M32R_BARRIER_H */
--- a/arch/m68k/include/asm/barrier.h
+++ b/arch/m68k/include/asm/barrier.h
@@ -1,20 +1,8 @@
#ifndef _M68K_BARRIER_H
#define _M68K_BARRIER_H
-/*
- * Force strict CPU ordering.
- * Not really required on m68k...
- */
#define nop() do { asm volatile ("nop"); barrier(); } while (0)
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define read_barrier_depends() ((void)0)
-#define set_mb(var, value) ({ (var) = (value); wmb(); })
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() ((void)0)
+#include <asm-generic/barrier.h>
#endif /* _M68K_BARRIER_H */
--- a/arch/metag/include/asm/barrier.h
+++ b/arch/metag/include/asm/barrier.h
@@ -82,4 +82,19 @@ static inline void fence(void)
#define smp_read_barrier_depends() do { } while (0)
#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* _ASM_METAG_BARRIER_H */
--- a/arch/microblaze/include/asm/Kbuild
+++ b/arch/microblaze/include/asm/Kbuild
@@ -4,3 +4,4 @@ generic-y += exec.h
generic-y += trace_clock.h
generic-y += syscalls.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/microblaze/include/asm/barrier.h
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Copyright (C) 2006 Atmark Techno, Inc.
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#ifndef _ASM_MICROBLAZE_BARRIER_H
-#define _ASM_MICROBLAZE_BARRIER_H
-
-#define nop() asm volatile ("nop")
-
-#define smp_read_barrier_depends() do {} while (0)
-#define read_barrier_depends() do {} while (0)
-
-#define mb() barrier()
-#define rmb() mb()
-#define wmb() mb()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-
-#endif /* _ASM_MICROBLAZE_BARRIER_H */
--- a/arch/mips/include/asm/barrier.h
+++ b/arch/mips/include/asm/barrier.h
@@ -180,4 +180,19 @@
#define nudge_writes() mb()
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
--- a/arch/mn10300/include/asm/Kbuild
+++ b/arch/mn10300/include/asm/Kbuild
@@ -3,3 +3,4 @@ generic-y += clkdev.h
generic-y += exec.h
generic-y += trace_clock.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/mn10300/include/asm/barrier.h
+++ /dev/null
@@ -1,37 +0,0 @@
-/* MN10300 memory barrier definitions
- *
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public Licence
- * as published by the Free Software Foundation; either version
- * 2 of the Licence, or (at your option) any later version.
- */
-#ifndef _ASM_BARRIER_H
-#define _ASM_BARRIER_H
-
-#define nop() asm volatile ("nop")
-
-#define mb() asm volatile ("": : :"memory")
-#define rmb() mb()
-#define wmb() asm volatile ("": : :"memory")
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define set_mb(var, value) do { xchg(&var, value); } while (0)
-#else /* CONFIG_SMP */
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#endif /* CONFIG_SMP */
-
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
-
-#endif /* _ASM_BARRIER_H */
--- a/arch/parisc/include/asm/Kbuild
+++ b/arch/parisc/include/asm/Kbuild
@@ -5,3 +5,4 @@ generic-y += word-at-a-time.h auxvec.h u
poll.h xor.h clkdev.h exec.h
generic-y += trace_clock.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/parisc/include/asm/barrier.h
+++ /dev/null
@@ -1,35 +0,0 @@
-#ifndef __PARISC_BARRIER_H
-#define __PARISC_BARRIER_H
-
-/*
-** This is simply the barrier() macro from linux/kernel.h but when serial.c
-** uses tqueue.h uses smp_mb() defined using barrier(), linux/kernel.h
-** hasn't yet been included yet so it fails, thus repeating the macro here.
-**
-** PA-RISC architecture allows for weakly ordered memory accesses although
-** none of the processors use it. There is a strong ordered bit that is
-** set in the O-bit of the page directory entry. Operating systems that
-** can not tolerate out of order accesses should set this bit when mapping
-** pages. The O-bit of the PSW should also be set to 1 (I don't believe any
-** of the processor implemented the PSW O-bit). The PCX-W ERS states that
-** the TLB O-bit is not implemented so the page directory does not need to
-** have the O-bit set when mapping pages (section 3.1). This section also
-** states that the PSW Y, Z, G, and O bits are not implemented.
-** So it looks like nothing needs to be done for parisc-linux (yet).
-** (thanks to chada for the above comment -ggg)
-**
-** The __asm__ op below simple prevents gcc/ld from reordering
-** instructions across the mb() "call".
-*/
-#define mb() __asm__ __volatile__("":::"memory") /* barrier() */
-#define rmb() mb()
-#define wmb() mb()
-#define smp_mb() mb()
-#define smp_rmb() mb()
-#define smp_wmb() mb()
-#define smp_read_barrier_depends() do { } while(0)
-#define read_barrier_depends() do { } while(0)
-
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-
-#endif /* __PARISC_BARRIER_H */
--- a/arch/powerpc/include/asm/barrier.h
+++ b/arch/powerpc/include/asm/barrier.h
@@ -45,11 +45,15 @@
# define SMPWMB eieio
#endif
+#define __lwsync() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+
#define smp_mb() mb()
-#define smp_rmb() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
+#define smp_rmb() __lwsync()
#define smp_wmb() __asm__ __volatile__ (stringify_in_c(SMPWMB) : : :"memory")
#define smp_read_barrier_depends() read_barrier_depends()
#else
+#define __lwsync() barrier()
+
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
@@ -65,4 +69,19 @@
#define data_barrier(x) \
asm volatile("twi 0,%0,0; isync" : : "r" (x) : "memory");
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ __lwsync(); \
+ ___p1; \
+})
+
#endif /* _ASM_POWERPC_BARRIER_H */
--- a/arch/s390/include/asm/barrier.h
+++ b/arch/s390/include/asm/barrier.h
@@ -32,4 +32,19 @@
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* __ASM_BARRIER_H */
--- a/arch/score/include/asm/Kbuild
+++ b/arch/score/include/asm/Kbuild
@@ -5,3 +5,4 @@ generic-y += clkdev.h
generic-y += trace_clock.h
generic-y += xor.h
generic-y += preempt.h
+generic-y += barrier.h
--- a/arch/score/include/asm/barrier.h
+++ /dev/null
@@ -1,16 +0,0 @@
-#ifndef _ASM_SCORE_BARRIER_H
-#define _ASM_SCORE_BARRIER_H
-
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
-
-#define set_mb(var, value) do {var = value; wmb(); } while (0)
-
-#endif /* _ASM_SCORE_BARRIER_H */
--- a/arch/sh/include/asm/barrier.h
+++ b/arch/sh/include/asm/barrier.h
@@ -26,29 +26,14 @@
#if defined(CONFIG_CPU_SH4A) || defined(CONFIG_CPU_SH5)
#define mb() __asm__ __volatile__ ("synco": : :"memory")
#define rmb() mb()
-#define wmb() __asm__ __volatile__ ("synco": : :"memory")
+#define wmb() mb()
#define ctrl_barrier() __icbi(PAGE_OFFSET)
-#define read_barrier_depends() do { } while(0)
#else
-#define mb() __asm__ __volatile__ ("": : :"memory")
-#define rmb() mb()
-#define wmb() __asm__ __volatile__ ("": : :"memory")
#define ctrl_barrier() __asm__ __volatile__ ("nop;nop;nop;nop;nop;nop;nop;nop")
-#define read_barrier_depends() do { } while(0)
-#endif
-
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while(0)
#endif
#define set_mb(var, value) do { (void)xchg(&var, value); } while (0)
+#include <asm-generic/barrier.h>
+
#endif /* __ASM_SH_BARRIER_H */
--- a/arch/sparc/include/asm/barrier_32.h
+++ b/arch/sparc/include/asm/barrier_32.h
@@ -1,15 +1,6 @@
#ifndef __SPARC_BARRIER_H
#define __SPARC_BARRIER_H
-/* XXX Change this if we ever use a PSO mode kernel. */
-#define mb() __asm__ __volatile__ ("" : : : "memory")
-#define rmb() mb()
-#define wmb() mb()
-#define read_barrier_depends() do { } while(0)
-#define set_mb(__var, __value) do { __var = __value; mb(); } while(0)
-#define smp_mb() __asm__ __volatile__("":::"memory")
-#define smp_rmb() __asm__ __volatile__("":::"memory")
-#define smp_wmb() __asm__ __volatile__("":::"memory")
-#define smp_read_barrier_depends() do { } while(0)
+#include <asm-generic/barrier.h>
#endif /* !(__SPARC_BARRIER_H) */
--- a/arch/sparc/include/asm/barrier_64.h
+++ b/arch/sparc/include/asm/barrier_64.h
@@ -53,4 +53,19 @@ do { __asm__ __volatile__("ba,pt %%xcc,
#define smp_read_barrier_depends() do { } while(0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
#endif /* !(__SPARC64_BARRIER_H) */
--- a/arch/tile/include/asm/barrier.h
+++ b/arch/tile/include/asm/barrier.h
@@ -22,59 +22,6 @@
#include <arch/spr_def.h>
#include <asm/timex.h>
-/*
- * read_barrier_depends - Flush all pending reads that subsequents reads
- * depend on.
- *
- * No data-dependent reads from memory-like regions are ever reordered
- * over this barrier. All reads preceding this primitive are guaranteed
- * to access memory (but not necessarily other CPUs' caches) before any
- * reads following this primitive that depend on the data return by
- * any of the preceding reads. This primitive is much lighter weight than
- * rmb() on most CPUs, and is never heavier weight than is
- * rmb().
- *
- * These ordering constraints are respected by both the local CPU
- * and the compiler.
- *
- * Ordering is not guaranteed by anything other than these primitives,
- * not even by data dependencies. See the documentation for
- * memory_barrier() for examples and URLs to more information.
- *
- * For example, the following code would force ordering (the initial
- * value of "a" is zero, "b" is one, and "p" is "&a"):
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * b = 2;
- * memory_barrier();
- * p = &b; q = p;
- * read_barrier_depends();
- * d = *q;
- * </programlisting>
- *
- * because the read of "*q" depends on the read of "p" and these
- * two reads are separated by a read_barrier_depends(). However,
- * the following code, with the same initial values for "a" and "b":
- *
- * <programlisting>
- * CPU 0 CPU 1
- *
- * a = 2;
- * memory_barrier();
- * b = 3; y = b;
- * read_barrier_depends();
- * x = a;
- * </programlisting>
- *
- * does not enforce ordering, since there is no data dependency between
- * the read of "a" and the read of "b". Therefore, on some CPUs, such
- * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
- * in cases like this where there are no data dependencies.
- */
-#define read_barrier_depends() do { } while (0)
-
#define __sync() __insn_mf()
#include <hv/syscall_public.h>
@@ -125,20 +72,7 @@ mb_incoherent(void)
#define mb() fast_mb()
#define iob() fast_iob()
-#ifdef CONFIG_SMP
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
-#define smp_read_barrier_depends() read_barrier_depends()
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
-#endif
-
-#define set_mb(var, value) \
- do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_TILE_BARRIER_H */
--- a/arch/unicore32/include/asm/barrier.h
+++ b/arch/unicore32/include/asm/barrier.h
@@ -14,15 +14,6 @@
#define dsb() __asm__ __volatile__ ("" : : : "memory")
#define dmb() __asm__ __volatile__ ("" : : : "memory")
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define read_barrier_depends() do { } while (0)
-#define smp_read_barrier_depends() do { } while (0)
-
-#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* __UNICORE_BARRIER_H__ */
--- a/arch/x86/include/asm/barrier.h
+++ b/arch/x86/include/asm/barrier.h
@@ -100,6 +100,21 @@
#define set_mb(var, value) do { var = value; barrier(); } while (0)
#endif
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ barrier(); \
+ ___p1; \
+})
+
/*
* Stop RDTSC speculation. This is needed when you need to use RDTSC
* (or get_cycles or vread that possibly accesses the TSC) in a defined
--- a/arch/xtensa/include/asm/barrier.h
+++ b/arch/xtensa/include/asm/barrier.h
@@ -9,21 +9,14 @@
#ifndef _XTENSA_SYSTEM_H
#define _XTENSA_SYSTEM_H
-#define smp_read_barrier_depends() do { } while(0)
-#define read_barrier_depends() do { } while(0)
-
#define mb() ({ __asm__ __volatile__("memw" : : : "memory"); })
#define rmb() barrier()
#define wmb() mb()
#ifdef CONFIG_SMP
#error smp_* not defined
-#else
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
#endif
-#define set_mb(var, value) do { var = value; mb(); } while (0)
+#include <asm-generic/barrier.h>
#endif /* _XTENSA_SYSTEM_H */
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -1,4 +1,5 @@
-/* Generic barrier definitions, based on MN10300 definitions.
+/*
+ * Generic barrier definitions, based on MN10300 definitions.
*
* It should be possible to use these on really simple architectures,
* but it serves more as a starting point for new ports.
@@ -16,35 +17,67 @@
#ifndef __ASSEMBLY__
-#define nop() asm volatile ("nop")
+#include <asm/compiler.h>
+
+#ifndef nop
+#define nop() asm volatile ("nop")
+#endif
/*
- * Force strict CPU ordering.
- * And yes, this is required on UP too when we're talking
- * to devices.
+ * Force strict CPU ordering. And yes, this is required on UP too when we're
+ * talking to devices.
*
- * This implementation only contains a compiler barrier.
+ * Fall back to compiler barriers if nothing better is provided.
*/
-#define mb() asm volatile ("": : :"memory")
-#define rmb() mb()
-#define wmb() asm volatile ("": : :"memory")
+#ifndef mb
+#define mb() barrier()
+#endif
+
+#ifndef rmb
+#define rmb() barrier()
+#endif
+
+#ifndef wmb
+#define wmb() barrier()
+#endif
+
+#ifndef read_barrier_depends
+#define read_barrier_depends() do {} while (0)
+#endif
#ifdef CONFIG_SMP
#define smp_mb() mb()
#define smp_rmb() rmb()
#define smp_wmb() wmb()
+#define smp_read_barrier_depends() read_barrier_depends()
#else
#define smp_mb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
+#define smp_read_barrier_depends() do {} while (0)
#endif
+#ifndef set_mb
#define set_mb(var, value) do { var = value; mb(); } while (0)
+#endif
+
#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-#define read_barrier_depends() do {} while (0)
-#define smp_read_barrier_depends() do {} while (0)
+#define smp_store_release(p, v) \
+do { \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ACCESS_ONCE(*p) = (v); \
+} while (0)
+
+#define smp_load_acquire(p) \
+({ \
+ typeof(*p) ___p1 = ACCESS_ONCE(*p); \
+ compiletime_assert_atomic_type(*p); \
+ smp_mb(); \
+ ___p1; \
+})
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_GENERIC_BARRIER_H */
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -298,6 +298,11 @@ void ftrace_likely_update(struct ftrace_
# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
#endif
+/* Is this type a native word size -- useful for atomic operations */
+#ifndef __native_word
+# define __native_word(t) (sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
+#endif
+
/* Compile time object size, -1 for unknown */
#ifndef __compiletime_object_size
# define __compiletime_object_size(obj) -1
@@ -337,6 +342,10 @@ void ftrace_likely_update(struct ftrace_
#define compiletime_assert(condition, msg) \
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
+#define compiletime_assert_atomic_type(t) \
+ compiletime_assert(__native_word(t), \
+ "Need native word sized stores/loads for atomicity.")
+
/*
* Prevent the compiler from merging or refetching accesses. The compiler
* is also forbidden from reordering successive instances of ACCESS_ONCE(),
^ permalink raw reply
* Re: [RFC PATCH] ehci-platform: Merge ppc-of EHCI driver into the ehci-platform driver
From: Alan Stern @ 2013-11-06 16:14 UTC (permalink / raw)
To: Alistair Popple; +Cc: linux-usb, linuxppc-dev
In-Reply-To: <1690980.YlPWToa1rW@mexican>
On Wed, 6 Nov 2013, Alistair Popple wrote:
> Currently the ppc-of driver uses the compatibility string
> "usb-ehci". This means platforms that use device-tree and implement an
> EHCI compatible interface have to either use the ppc-of driver or add
> a compatible line to the ehci-platform driver. It would be more
> appropriate for the platform driver to be compatible with "usb-ehci"
> as non-powerpc platforms are also beginning to utilise device-tree.
>
> This patch merges the device tree property parsing from ehci-ppc-of
> into the platform driver and adds a "usb-ehci" compatibility
> string. The existing ehci-ppc-of driver is renamed to ehci-440epx as
> it contains platform specific work arounds for the 440EPX SoC.
>
> Signed-off-by: Alistair Popple <alistair@popple.id.au>
> ---
>
> So I could submit something like this that essentially merges the ppc-of
> driver into the platform driver instead of adding the "ibm,akebono-ehci"
> compatible line to the platform driver.
>
> However I'm still fairly new to device-tree so I'm not sure what (if any) the
> broader impact would be. A quick grep for "usb-ehci" turned up a couple of ARM
> device tree's using it however they all provided their own drivers and don't
> select CONFIG_USB_EHCI_HCD_PLATFORM so I'm guess they shouldn't be impacted.
>
> I have attempted to fix up any PowerPC device trees/configs, although I wasn't
> sure if "usb-ehci" should remain in sequoia.dts or not given that it needs to
> use the 440EPX specific driver.
>
> Also this hasn't been tested (beyond compilation) yet.
> diff --git a/drivers/usb/host/ehci-platform.c b/drivers/usb/host/ehci-
> platform.c
> index f6b790c..027f368 100644
> --- a/drivers/usb/host/ehci-platform.c
> +++ b/drivers/usb/host/ehci-platform.c
> @@ -77,6 +77,7 @@ static int ehci_platform_probe(struct platform_device *dev)
> struct usb_hcd *hcd;
> struct resource *res_mem;
> struct usb_ehci_pdata *pdata;
> + struct device_node *dn = dev->dev.of_node;
> int irq;
> int err = -ENOMEM;
>
> @@ -96,6 +97,18 @@ static int ehci_platform_probe(struct platform_device *dev)
>
> pdata = dev_get_platdata(&dev->dev);
>
> + /* Initialise platform data from device tree if available. */
> + if (!dn) {
Shouldn't this be "if (dn)"?
> + if (of_get_property(dn, "big-endian", NULL)) {
> + pdata->big_endian_mmio = 1;
> + pdata->big_endian_desc = 1;
> + }
> + if (of_get_property(dn, "big-endian-regs", NULL))
> + pdata->big_endian_mmio = 1;
> + if (of_get_property(dn, "big-endian-desc", NULL))
> + pdata->big_endian_desc = 1;
> + }
> +
This isn't good if there is more than one EHCI controller using
ehci-platform. To accomodate such cases, it would be necessary to
allocate a separate copy of ehci_platform_defaults for each controller.
Alan Stern
^ permalink raw reply
* Re: [PATCH 3/7] IBM Akebono: Add support for a new PHY to the IBM emac driver
From: Ben Hutchings @ 2013-11-06 16:40 UTC (permalink / raw)
To: Alistair Popple; +Cc: linuxppc-dev, David S. Miller, netdev
In-Reply-To: <3555701.fBbreSn0Lp@mexican>
On Wed, 2013-11-06 at 12:34 +1100, Alistair Popple wrote:
> On Tue, 5 Nov 2013 23:11:50 Ben Hutchings wrote:
> > On Wed, 2013-11-06 at 06:54 +1100, Benjamin Herrenschmidt wrote:
>
> [snip]
>
> > > It's an SoC bit so there's little point making it generally
> > > selectable by the user.
> >
> > I think a better way to do this is:
> >
> > config IBM_EMAC_RGMII_WOL
> > bool "IBM EMAC RGMII wake-on-LAN support"
> > depends on MY_WONDERFUL_NEW_SOC || COMPILE_TEST
> > default y if MY_WONDERFUL_NEW_SOC
> >
> > Then anyone making an API change that affects this driver can check that
> > it still complies.
>
> The method used in this patch is the same as what is currently used by the
> other IBM EMAC PHY interfaces (eg. config IBM_EMAC_ZMII etc). I'm happy to
> send a patch to update all of those as well for consistency but that would
> mean adding what each platform requires into EMACS Kconfig as well.
>
> Personally I think it is nicer to keep the definitions of what each platform
> requires in one place (ie. arch/powerpc/platforms/44x/Kconfig) as it is
> consistent with what we do for other 44x drivers, however I am happy to use
> the above method if people think it's better.
Yes, I see your point.
> Alternatively we could do something like this:
>
> config IBM_EMAC_RGMII_WOL
> bool
> default y if COMPILE_TEST
> default n
>
> This would leave the platform dependencies as they are currently but still
> allow compile testing.
It still shouldn't default to y in that case. Instead you can make the
symbol conditionally configurable:
config IBM_EMAC_RGMII_WOL
bool "IBM EMAC RGMII wake-on-LAN support" if COMPILE_TEST
and then select this from your platform Kconfig as you intended.
(There is no need to put 'default n' as that's implicit for a
configurable symbol. But it doesn't hurt either.)
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: [PATCH v3 1/1] powerpc/embedded6xx: Add support for Motorola/Emerson MVME5100
From: Alessio Igor Bogani @ 2013-11-06 15:49 UTC (permalink / raw)
To: Stephen Chivers; +Cc: linuxppc-dev
In-Reply-To: <20131103210748.E74FAE06C0@canberra.localdomain>
On lun, 2013-11-04 at 08:07 +1100, Stephen Chivers wrote:
> Add support for the Motorola/Emerson MVME5100 Single Board Computer.
>
> The MVME5100 is a 6U form factor VME64 computer with:
>
> - A single MPC7410 or MPC750 CPU
> - A HAWK Processor Host Bridge (CPU to PCI) and
> MultiProcessor Interrupt Controller (MPIC)
> - Up to 500Mb of onboard memory
> - A M48T37 Real Time Clock (RTC) and Non-Volatile Memory chip
> - Two 16550 compatible UARTS
> - Two Intel E100 Fast Ethernets
> - Two PCI Mezzanine Card (PMC) Slots
> - PPCBug Firmware
>
> The HAWK PHB/MPIC is compatible with the MPC10x devices.
>
> There is no onboard disk support. This is usually provided by
> installing a PMC in the first PMC slot.
>
> This patch revives the board support, it was present in early 2.6
> series kernels. The board support in those days was by Matt Porter of
> MontaVista Software.
>
> CSC Australia has around 31 of these boards in service. The kernel in use
> for the boards is based on 2.6.31. The boards are operated without disks
> from a file server.
>
> This patch is based on linux-3.12-rc7 and has been boot tested.
>
> V1->V2:
> Address comments by Kular Gama and Scott Wood.
> Minor adjustment to platforms/embedded6xx/Kconfig to ensure
> correct indentation where possible.
>
> V2->V3:
> Address comments by Scott Wood and Ben Herrenschmidt.
> Address errors reported by checkpatch.
>
> Signed-off-by: Stephen Chivers <schivers@csc.com>A
Tested-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
> ---
> arch/powerpc/boot/Makefile | 3 +-
> arch/powerpc/boot/dts/mvme5100.dts | 185 +++++++++++++++++++++
> arch/powerpc/boot/mvme5100.c | 27 +++
> arch/powerpc/boot/wrapper | 4 +
> arch/powerpc/configs/mvme5100_defconfig | 144 ++++++++++++++++
> arch/powerpc/platforms/embedded6xx/Kconfig | 13 ++-
> arch/powerpc/platforms/embedded6xx/Makefile | 1 +
> arch/powerpc/platforms/embedded6xx/mvme5100.c | 221 +++++++++++++++++++++++++
> 8 files changed, 596 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
> index 15ca225..a1c9c86 100644
> --- a/arch/powerpc/boot/Makefile
> +++ b/arch/powerpc/boot/Makefile
> @@ -94,7 +94,7 @@ src-plat-$(CONFIG_FSL_SOC_BOOKE) += cuboot-85xx.c cuboot-85xx-cpm2.c
> src-plat-$(CONFIG_EMBEDDED6xx) += cuboot-pq2.c cuboot-mpc7448hpc2.c \
> cuboot-c2k.c gamecube-head.S \
> gamecube.c wii-head.S wii.c holly.c \
> - prpmc2800.c
> + prpmc2800.c fixed-head.S mvme5100.c
> src-plat-$(CONFIG_AMIGAONE) += cuboot-amigaone.c
> src-plat-$(CONFIG_PPC_PS3) += ps3-head.S ps3-hvcall.S ps3.c
> src-plat-$(CONFIG_EPAPR_BOOT) += epapr.c epapr-wrapper.c
> @@ -285,6 +285,7 @@ image-$(CONFIG_MPC7448HPC2) += cuImage.mpc7448hpc2
> image-$(CONFIG_PPC_C2K) += cuImage.c2k
> image-$(CONFIG_GAMECUBE) += dtbImage.gamecube
> image-$(CONFIG_WII) += dtbImage.wii
> +image-$(CONFIG_MVME5100) += dtbImage.mvme5100
>
> # Board port in arch/powerpc/platform/amigaone/Kconfig
> image-$(CONFIG_AMIGAONE) += cuImage.amigaone
> diff --git a/arch/powerpc/boot/dts/mvme5100.dts b/arch/powerpc/boot/dts/mvme5100.dts
> new file mode 100644
> index 0000000..4cb2f05
> --- /dev/null
> +++ b/arch/powerpc/boot/dts/mvme5100.dts
> @@ -0,0 +1,185 @@
> +/*
> + * Device Tree Souce for Motorola/Emerson MVME5100.
> + *
> + * Copyright 2013 CSC Australia Pty. Ltd.
> + *
> + * This file is licensed under the terms of the GNU General Public
> + * License version 2. This program is licensed "as is" without
> + * any warranty of any kind, whether express or implied.
> + */
> +
> +/dts-v1/;
> +
> +/ {
> + model = "MVME5100";
> + compatible = "MVME5100";
> + #address-cells = <1>;
> + #size-cells = <1>;
> +
> + aliases {
> + serial0 = &serial0;
> + pci0 = &pci0;
> + };
> +
> + cpus {
> + #address-cells = <1>;
> + #size-cells = <0>;
> +
> + PowerPC,7410 {
> + device_type = "cpu";
> + reg = <0x0>;
> + /* Following required by dtc but not used */
> + d-cache-line-size = <32>;
> + i-cache-line-size = <32>;
> + i-cache-size = <32768>;
> + d-cache-size = <32768>;
> + timebase-frequency = <25000000>;
> + clock-frequency = <500000000>;
> + bus-frequency = <100000000>;
> + };
> + };
> +
> + memory {
> + device_type = "memory";
> + reg = <0x0 0x20000000>;
> + };
> +
> + hawk@fef80000 {
> + #address-cells = <1>;
> + #size-cells = <1>;
> + compatible = "hawk-bridge", "simple-bus";
> + ranges = <0x0 0xfef80000 0x10000>;
> + reg = <0xfef80000 0x10000>;
> +
> + serial0: serial@8000 {
> + device_type = "serial";
> + compatible = "ns16550";
> + reg = <0x8000 0x80>;
> + reg-shift = <4>;
> + clock-frequency = <1843200>;
> + current-speed = <9600>;
> + interrupts = <1 1>; // IRQ1 Level Active Low.
> + interrupt-parent = <&mpic>;
> + };
> +
> + serial1: serial@8200 {
> + device_type = "serial";
> + compatible = "ns16550";
> + reg = <0x8200 0x80>;
> + reg-shift = <4>;
> + clock-frequency = <1843200>;
> + current-speed = <9600>;
> + interrupts = <1 1>; // IRQ1 Level Active Low.
> + interrupt-parent = <&mpic>;
> + };
> +
> + mpic: interrupt-controller@f3f80000 {
> + #interrupt-cells = <2>;
> + #address-cells = <0>;
> + device_type = "open-pic";
> + compatible = "chrp,open-pic";
> + interrupt-controller;
> + reg = <0xf3f80000 0x40000>;
> + };
> + };
> +
> + pci0: pci@feff0000 {
> + #address-cells = <3>;
> + #size-cells = <2>;
> + #interrupt-cells = <1>;
> + device_type = "pci";
> + compatible = "hawk-pci";
> + reg = <0xfec00000 0x400000>;
> + 8259-interrupt-acknowledge = <0xfeff0030>;
> + ranges = <0x1000000 0x0 0x0 0xfe000000 0x0 0x800000
> + 0x2000000 0x0 0x80000000 0x80000000 0x0 0x74000000>;
> + bus-range = <0 255>;
> + clock-frequency = <33333333>;
> + interrupt-parent = <&mpic>;
> + interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
> + interrupt-map = <
> +
> + /*
> + * This definition (IDSEL 11) duplicates the
> + * interrupts definition in the i8259
> + * interrupt controller below.
> + *
> + * Do not change the interrupt sense/polarity from
> + * 0x2 to anything else, doing so will cause endless
> + * "spurious" i8259 interrupts to be fielded.
> + */
> + // IDSEL 11 - iPMC712 PCI/ISA Bridge
> + 0x5800 0x0 0x0 0x1 &mpic 0x0 0x2
> + 0x5800 0x0 0x0 0x2 &mpic 0x0 0x2
> + 0x5800 0x0 0x0 0x3 &mpic 0x0 0x2
> + 0x5800 0x0 0x0 0x4 &mpic 0x0 0x2
> +
> + /* IDSEL 12 - Not Used */
> +
> + /* IDSEL 13 - Universe VME Bridge */
> + 0x6800 0x0 0x0 0x1 &mpic 0x5 0x1
> + 0x6800 0x0 0x0 0x2 &mpic 0x6 0x1
> + 0x6800 0x0 0x0 0x3 &mpic 0x7 0x1
> + 0x6800 0x0 0x0 0x4 &mpic 0x8 0x1
> +
> + /* IDSEL 14 - ENET 1 */
> + 0x7000 0x0 0x0 0x1 &mpic 0x2 0x1
> +
> + /* IDSEL 15 - Not Used */
> +
> + /* IDSEL 16 - PMC Slot 1 */
> + 0x8000 0x0 0x0 0x1 &mpic 0x9 0x1
> + 0x8000 0x0 0x0 0x2 &mpic 0xa 0x1
> + 0x8000 0x0 0x0 0x3 &mpic 0xb 0x1
> + 0x8000 0x0 0x0 0x4 &mpic 0xc 0x1
> +
> + /* IDSEL 17 - PMC Slot 2 */
> + 0x8800 0x0 0x0 0x1 &mpic 0xc 0x1
> + 0x8800 0x0 0x0 0x2 &mpic 0x9 0x1
> + 0x8800 0x0 0x0 0x3 &mpic 0xa 0x1
> + 0x8800 0x0 0x0 0x4 &mpic 0xb 0x1
> +
> + /* IDSEL 18 - Not Used */
> +
> + /* IDSEL 19 - ENET 2 */
> + 0x9800 0x0 0x0 0x1 &mpic 0xd 0x1
> +
> + /* IDSEL 20 - PMCSPAN (PCI-X) */
> + 0xa000 0x0 0x0 0x1 &mpic 0x9 0x1
> + 0xa000 0x0 0x0 0x2 &mpic 0xa 0x1
> + 0xa000 0x0 0x0 0x3 &mpic 0xb 0x1
> + 0xa000 0x0 0x0 0x4 &mpic 0xc 0x1
> +
> + >;
> +
> + isa {
> + #address-cells = <2>;
> + #size-cells = <1>;
> + #interrupt-cells = <2>;
> + device_type = "isa";
> + compatible = "isa";
> + ranges = <0x00000001 0 0x01000000 0 0x00000000 0x00001000>;
> + interrupt-parent = <&i8259>;
> +
> + i8259: interrupt-controller@20 {
> + #interrupt-cells = <2>;
> + #address-cells = <0>;
> + interrupts = <0 2>;
> + device_type = "interrupt-controller";
> + compatible = "chrp,iic";
> + interrupt-controller;
> + reg = <1 0x00000020 0x00000002
> + 1 0x000000a0 0x00000002
> + 1 0x000004d0 0x00000002>;
> + interrupt-parent = <&mpic>;
> + };
> +
> + };
> +
> + };
> +
> + chosen {
> + linux,stdout-path = &serial0;
> + };
> +
> +};
> diff --git a/arch/powerpc/boot/mvme5100.c b/arch/powerpc/boot/mvme5100.c
> new file mode 100644
> index 0000000..cb865f8
> --- /dev/null
> +++ b/arch/powerpc/boot/mvme5100.c
> @@ -0,0 +1,27 @@
> +/*
> + * Motorola/Emerson MVME5100 with PPCBug firmware.
> + *
> + * Author: Stephen Chivers <schivers@csc.com>
> + *
> + * Copyright 2013 CSC Australia Pty. Ltd.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * version 2 as published by the Free Software Foundation.
> + *
> + */
> +#include "types.h"
> +#include "ops.h"
> +#include "io.h"
> +
> +BSS_STACK(4096);
> +
> +void platform_init(unsigned long r3, unsigned long r4, unsigned long r5)
> +{
> + u32 heapsize;
> +
> + heapsize = 0x8000000 - (u32)_end; /* 128M */
> + simple_alloc_init(_end, heapsize, 32, 64);
> + fdt_init(_dtb_start);
> + serial_console_init();
> +}
> diff --git a/arch/powerpc/boot/wrapper b/arch/powerpc/boot/wrapper
> index cd7af84..401c6a1 100755
> --- a/arch/powerpc/boot/wrapper
> +++ b/arch/powerpc/boot/wrapper
> @@ -257,6 +257,10 @@ epapr)
> link_address='0x20000000'
> pie=-pie
> ;;
> +mvme5100)
> + platformo="$object/fixed-head.o $object/mvme5100.o"
> + binary=y
> + ;;
> esac
>
> vmz="$tmpdir/`basename \"$kernel\"`.$ext"
> diff --git a/arch/powerpc/configs/mvme5100_defconfig b/arch/powerpc/configs/mvme5100_defconfig
> new file mode 100644
> index 0000000..93c7752
> --- /dev/null
> +++ b/arch/powerpc/configs/mvme5100_defconfig
> @@ -0,0 +1,144 @@
> +CONFIG_SYSVIPC=y
> +CONFIG_POSIX_MQUEUE=y
> +CONFIG_NO_HZ=y
> +CONFIG_HIGH_RES_TIMERS=y
> +CONFIG_IKCONFIG=y
> +CONFIG_IKCONFIG_PROC=y
> +CONFIG_LOG_BUF_SHIFT=14
> +# CONFIG_UTS_NS is not set
> +# CONFIG_IPC_NS is not set
> +# CONFIG_PID_NS is not set
> +# CONFIG_NET_NS is not set
> +CONFIG_CC_OPTIMIZE_FOR_SIZE=y
> +# CONFIG_COMPAT_BRK is not set
> +CONFIG_MODULES=y
> +CONFIG_MODULE_UNLOAD=y
> +# CONFIG_BLK_DEV_BSG is not set
> +# CONFIG_PPC_CHRP is not set
> +# CONFIG_PPC_PMAC is not set
> +CONFIG_EMBEDDED6xx=y
> +CONFIG_MVME5100=y
> +CONFIG_KVM_GUEST=y
> +CONFIG_HZ_100=y
> +# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
> +# CONFIG_COMPACTION is not set
> +CONFIG_CMDLINE_BOOL=y
> +CONFIG_CMDLINE="console=ttyS0,9600 ip=dhcp root=/dev/nfs"
> +CONFIG_NET=y
> +CONFIG_PACKET=y
> +CONFIG_UNIX=y
> +CONFIG_INET=y
> +CONFIG_IP_MULTICAST=y
> +CONFIG_IP_PNP=y
> +CONFIG_IP_PNP_DHCP=y
> +CONFIG_IP_PNP_BOOTP=y
> +# CONFIG_INET_LRO is not set
> +# CONFIG_IPV6 is not set
> +CONFIG_NETFILTER=y
> +CONFIG_NF_CONNTRACK=m
> +CONFIG_NF_CT_PROTO_SCTP=m
> +CONFIG_NF_CONNTRACK_AMANDA=m
> +CONFIG_NF_CONNTRACK_FTP=m
> +CONFIG_NF_CONNTRACK_H323=m
> +CONFIG_NF_CONNTRACK_IRC=m
> +CONFIG_NF_CONNTRACK_NETBIOS_NS=m
> +CONFIG_NF_CONNTRACK_PPTP=m
> +CONFIG_NF_CONNTRACK_SIP=m
> +CONFIG_NF_CONNTRACK_TFTP=m
> +CONFIG_NETFILTER_XT_MATCH_MAC=m
> +CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
> +CONFIG_NETFILTER_XT_MATCH_STATE=m
> +CONFIG_NF_CONNTRACK_IPV4=m
> +CONFIG_IP_NF_IPTABLES=m
> +CONFIG_IP_NF_FILTER=m
> +CONFIG_IP_NF_TARGET_REJECT=m
> +CONFIG_IP_NF_MANGLE=m
> +CONFIG_IP_NF_TARGET_ECN=m
> +CONFIG_IP_NF_TARGET_TTL=m
> +CONFIG_IP_NF_RAW=m
> +CONFIG_IP_NF_ARPTABLES=m
> +CONFIG_IP_NF_ARPFILTER=m
> +CONFIG_IP_NF_ARP_MANGLE=m
> +CONFIG_LAPB=m
> +CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
> +CONFIG_PROC_DEVICETREE=y
> +CONFIG_BLK_DEV_LOOP=y
> +CONFIG_BLK_DEV_RAM=y
> +CONFIG_BLK_DEV_RAM_COUNT=2
> +CONFIG_BLK_DEV_RAM_SIZE=8192
> +CONFIG_EEPROM_LEGACY=m
> +CONFIG_NETDEVICES=y
> +CONFIG_TUN=m
> +# CONFIG_NET_VENDOR_3COM is not set
> +CONFIG_E100=y
> +# CONFIG_WLAN is not set
> +# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
> +# CONFIG_INPUT_KEYBOARD is not set
> +# CONFIG_INPUT_MOUSE is not set
> +# CONFIG_SERIO is not set
> +CONFIG_SERIAL_8250=y
> +CONFIG_SERIAL_8250_CONSOLE=y
> +CONFIG_SERIAL_8250_NR_UARTS=10
> +CONFIG_SERIAL_8250_EXTENDED=y
> +CONFIG_SERIAL_8250_MANY_PORTS=y
> +CONFIG_SERIAL_8250_SHARE_IRQ=y
> +CONFIG_SERIAL_OF_PLATFORM=y
> +CONFIG_HW_RANDOM=y
> +CONFIG_I2C=y
> +CONFIG_I2C_CHARDEV=y
> +CONFIG_I2C_MPC=y
> +# CONFIG_HWMON is not set
> +CONFIG_VIDEO_OUTPUT_CONTROL=m
> +# CONFIG_VGA_CONSOLE is not set
> +# CONFIG_HID is not set
> +# CONFIG_USB_SUPPORT is not set
> +# CONFIG_IOMMU_SUPPORT is not set
> +CONFIG_VME_BUS=m
> +CONFIG_VME_CA91CX42=m
> +CONFIG_EXT2_FS=m
> +CONFIG_EXT3_FS=m
> +# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
> +CONFIG_XFS_FS=m
> +CONFIG_ISO9660_FS=m
> +CONFIG_JOLIET=y
> +CONFIG_ZISOFS=y
> +CONFIG_UDF_FS=m
> +CONFIG_MSDOS_FS=m
> +CONFIG_VFAT_FS=m
> +CONFIG_PROC_KCORE=y
> +CONFIG_TMPFS=y
> +CONFIG_NFS_FS=y
> +CONFIG_NFS_V3_ACL=y
> +CONFIG_NFS_V4=y
> +CONFIG_ROOT_NFS=y
> +CONFIG_NFSD=m
> +CONFIG_NFSD_V3=y
> +CONFIG_CIFS=m
> +CONFIG_NLS=y
> +CONFIG_NLS_CODEPAGE_437=m
> +CONFIG_NLS_CODEPAGE_932=m
> +CONFIG_NLS_ISO8859_1=m
> +CONFIG_NLS_UTF8=m
> +CONFIG_CRC_CCITT=m
> +CONFIG_CRC_T10DIF=y
> +CONFIG_XZ_DEC=y
> +CONFIG_XZ_DEC_X86=y
> +CONFIG_XZ_DEC_IA64=y
> +CONFIG_XZ_DEC_ARM=y
> +CONFIG_XZ_DEC_ARMTHUMB=y
> +CONFIG_XZ_DEC_SPARC=y
> +CONFIG_MAGIC_SYSRQ=y
> +CONFIG_DEBUG_KERNEL=y
> +CONFIG_DETECT_HUNG_TASK=y
> +CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=20
> +CONFIG_CRYPTO_CBC=y
> +CONFIG_CRYPTO_PCBC=m
> +CONFIG_CRYPTO_MD5=y
> +CONFIG_CRYPTO_MICHAEL_MIC=m
> +CONFIG_CRYPTO_SHA1=m
> +CONFIG_CRYPTO_BLOWFISH=m
> +CONFIG_CRYPTO_DES=y
> +CONFIG_CRYPTO_SERPENT=m
> +CONFIG_CRYPTO_TWOFISH=m
> +CONFIG_CRYPTO_DEFLATE=m
> +# CONFIG_CRYPTO_ANSI_CPRNG is not set
> diff --git a/arch/powerpc/platforms/embedded6xx/Kconfig b/arch/powerpc/platforms/embedded6xx/Kconfig
> index 302ba43..6d3c7a9 100644
> --- a/arch/powerpc/platforms/embedded6xx/Kconfig
> +++ b/arch/powerpc/platforms/embedded6xx/Kconfig
> @@ -67,6 +67,18 @@ config PPC_C2K
> This option enables support for the GE Fanuc C2K board (formerly
> an SBS board).
>
> +config MVME5100
> + bool "Motorola/Emerson MVME5100"
> + depends on EMBEDDED6xx
> + select MPIC
> + select PCI
> + select PPC_INDIRECT_PCI
> + select PPC_I8259
> + select PPC_NATIVE
> + help
> + This option enables support for the Motorola (now Emerson) MVME5100
> + board.
> +
> config TSI108_BRIDGE
> bool
> select PCI
> @@ -113,4 +125,3 @@ config WII
> help
> Select WII if configuring for the Nintendo Wii.
> More information at: <http://gc-linux.sourceforge.net/>
> -
> diff --git a/arch/powerpc/platforms/embedded6xx/Makefile b/arch/powerpc/platforms/embedded6xx/Makefile
> index 66c23e4..cdd48d4 100644
> --- a/arch/powerpc/platforms/embedded6xx/Makefile
> +++ b/arch/powerpc/platforms/embedded6xx/Makefile
> @@ -11,3 +11,4 @@ obj-$(CONFIG_USBGECKO_UDBG) += usbgecko_udbg.o
> obj-$(CONFIG_GAMECUBE_COMMON) += flipper-pic.o
> obj-$(CONFIG_GAMECUBE) += gamecube.o
> obj-$(CONFIG_WII) += wii.o hlwd-pic.o
> +obj-$(CONFIG_MVME5100) += mvme5100.o
> diff --git a/arch/powerpc/platforms/embedded6xx/mvme5100.c b/arch/powerpc/platforms/embedded6xx/mvme5100.c
> new file mode 100644
> index 0000000..25e3bfb
> --- /dev/null
> +++ b/arch/powerpc/platforms/embedded6xx/mvme5100.c
> @@ -0,0 +1,221 @@
> +/*
> + * Board setup routines for the Motorola/Emerson MVME5100.
> + *
> + * Copyright 2013 CSC Australia Pty. Ltd.
> + *
> + * Based on earlier code by:
> + *
> + * Matt Porter, MontaVista Software Inc.
> + * Copyright 2001 MontaVista Software Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License as published by the
> + * Free Software Foundation; either version 2 of the License, or (at your
> + * option) any later version.
> + *
> + * Author: Stephen Chivers <schivers@csc.com>
> + *
> + */
> +
> +#include <linux/of_platform.h>
> +
> +#include <asm/i8259.h>
> +#include <asm/pci-bridge.h>
> +#include <asm/mpic.h>
> +#include <asm/prom.h>
> +#include <mm/mmu_decl.h>
> +#include <asm/udbg.h>
> +
> +#define HAWK_MPIC_SIZE 0x00040000U
> +#define MVME5100_PCI_MEM_OFFSET 0x00000000
> +
> +/* Board register addresses. */
> +#define BOARD_STATUS_REG 0xfef88080
> +#define BOARD_MODFAIL_REG 0xfef88090
> +#define BOARD_MODRST_REG 0xfef880a0
> +#define BOARD_TBEN_REG 0xfef880c0
> +#define BOARD_SW_READ_REG 0xfef880e0
> +#define BOARD_GEO_ADDR_REG 0xfef880e8
> +#define BOARD_EXT_FEATURE1_REG 0xfef880f0
> +#define BOARD_EXT_FEATURE2_REG 0xfef88100
> +
> +static phys_addr_t pci_membase;
> +static u_char *restart;
> +
> +static void mvme5100_8259_cascade(unsigned int irq, struct irq_desc *desc)
> +{
> + struct irq_chip *chip = irq_desc_get_chip(desc);
> + unsigned int cascade_irq = i8259_irq();
> +
> + if (cascade_irq != NO_IRQ)
> + generic_handle_irq(cascade_irq);
> +
> + chip->irq_eoi(&desc->irq_data);
> +}
> +
> +static void __init mvme5100_pic_init(void)
> +{
> + struct mpic *mpic;
> + struct device_node *np;
> + struct device_node *cp = NULL;
> + unsigned int cirq;
> + unsigned long intack = 0;
> + const u32 *prop = NULL;
> +
> + np = of_find_node_by_type(NULL, "open-pic");
> + if (!np) {
> + pr_err("Could not find open-pic node\n");
> + return;
> + }
> +
> + mpic = mpic_alloc(np, pci_membase, 0, 16, 256, " OpenPIC ");
> +
> + BUG_ON(mpic == NULL);
> + of_node_put(np);
> +
> + mpic_assign_isu(mpic, 0, pci_membase + 0x10000);
> +
> + mpic_init(mpic);
> +
> + cp = of_find_compatible_node(NULL, NULL, "chrp,iic");
> + if (cp == NULL) {
> + pr_warn("mvme5100_pic_init: couldn't find i8259\n");
> + return;
> + }
> +
> + cirq = irq_of_parse_and_map(cp, 0);
> + if (cirq == NO_IRQ) {
> + pr_warn("mvme5100_pic_init: no cascade interrupt?\n");
> + return;
> + }
> +
> + np = of_find_compatible_node(NULL, "pci", "mpc10x-pci");
> + if (np) {
> + prop = of_get_property(np, "8259-interrupt-acknowledge", NULL);
> +
> + if (prop)
> + intack = prop[0];
> +
> + of_node_put(np);
> + }
> +
> + if (intack)
> + pr_debug("mvme5100_pic_init: PCI 8259 intack at 0x%016lx\n",
> + intack);
> +
> + i8259_init(cp, intack);
> + of_node_put(cp);
> + irq_set_chained_handler(cirq, mvme5100_8259_cascade);
> +}
> +
> +static int __init mvme5100_add_bridge(struct device_node *dev)
> +{
> + const int *bus_range;
> + int len;
> + struct pci_controller *hose;
> + unsigned short devid;
> +
> + pr_info("Adding PCI host bridge %s\n", dev->full_name);
> +
> + bus_range = of_get_property(dev, "bus-range", &len);
> +
> + hose = pcibios_alloc_controller(dev);
> + if (hose == NULL)
> + return -ENOMEM;
> +
> + hose->first_busno = bus_range ? bus_range[0] : 0;
> + hose->last_busno = bus_range ? bus_range[1] : 0xff;
> +
> + setup_indirect_pci(hose, 0xfe000cf8, 0xfe000cfc, 0);
> +
> + pci_process_bridge_OF_ranges(hose, dev, 1);
> +
> + early_read_config_word(hose, 0, 0, PCI_DEVICE_ID, &devid);
> +
> + if (devid != PCI_DEVICE_ID_MOTOROLA_HAWK) {
> + pr_err("HAWK PHB not present?\n");
> + return 0;
> + }
> +
> + early_read_config_dword(hose, 0, 0, PCI_BASE_ADDRESS_1, &pci_membase);
> +
> + if (pci_membase == 0) {
> + pr_err("HAWK PHB mibar not correctly set?\n");
> + return 0;
> + }
> +
> + pr_info("mvme5100_pic_init: pci_membase: %x\n", pci_membase);
> +
> + return 0;
> +}
> +
> +static struct of_device_id mvme5100_of_bus_ids[] __initdata = {
> + { .compatible = "hawk-bridge", },
> + {},
> +};
> +
> +/*
> + * Setup the architecture
> + */
> +static void __init mvme5100_setup_arch(void)
> +{
> + struct device_node *np;
> +
> + if (ppc_md.progress)
> + ppc_md.progress("mvme5100_setup_arch()", 0);
> +
> + for_each_compatible_node(np, "pci", "hawk-pci")
> + mvme5100_add_bridge(np);
> +
> + restart = ioremap(BOARD_MODRST_REG, 4);
> +}
> +
> +
> +static void mvme5100_show_cpuinfo(struct seq_file *m)
> +{
> + seq_puts(m, "Vendor\t\t: Motorola/Emerson\n");
> + seq_puts(m, "Machine\t\t: MVME5100\n");
> +}
> +
> +static void mvme5100_restart(char *cmd)
> +{
> +
> + local_irq_disable();
> + mtmsr(mfmsr() | MSR_IP);
> +
> + out_8((u_char *) restart, 0x01);
> +
> + while (1)
> + ;
> +}
> +
> +/*
> + * Called very early, device-tree isn't unflattened
> + */
> +static int __init mvme5100_probe(void)
> +{
> + unsigned long root = of_get_flat_dt_root();
> +
> + return of_flat_dt_is_compatible(root, "MVME5100");
> +}
> +
> +static int __init probe_of_platform_devices(void)
> +{
> +
> + of_platform_bus_probe(NULL, mvme5100_of_bus_ids, NULL);
> + return 0;
> +}
> +
> +machine_device_initcall(mvme5100, probe_of_platform_devices);
> +
> +define_machine(mvme5100) {
> + .name = "MVME5100",
> + .probe = mvme5100_probe,
> + .setup_arch = mvme5100_setup_arch,
> + .init_IRQ = mvme5100_pic_init,
> + .show_cpuinfo = mvme5100_show_cpuinfo,
> + .get_irq = mpic_get_irq,
> + .restart = mvme5100_restart,
> + .calibrate_decr = generic_calibrate_decr,
> + .progress = udbg_progress,
> +};
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Paul E. McKenney @ 2013-11-06 18:48 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Michael Neuling, Tony Luck, Mathieu Desnoyers, Heiko Carstens,
Oleg Nesterov, LKML, Linux PPC dev, Geert Uytterhoeven,
Anton Blanchard, Frederic Weisbecker, Victor Kaplansky,
Russell King, Linus Torvalds, Martin Schwidefsky
In-Reply-To: <20131106135736.GK10651@twins.programming.kicks-ass.net>
On Wed, Nov 06, 2013 at 02:57:36PM +0100, Peter Zijlstra wrote:
> On Wed, Nov 06, 2013 at 01:51:10PM +0100, Geert Uytterhoeven wrote:
> > This is screaming for a default implementation in asm-generic.
>
> Right you are... how about a little something like this?
>
> There's a few archs I didn't fully merge with the generic one because of
> weird nop implementations.
>
> asm volatile ("nop" :: ) vs asm volatile ("nop" ::: "memory") and the
> like. They probably can (and should) use the regular asm volatile
> ("nop") but I misplaced the toolchains for many of the weird archs so I
> didn't attempt.
>
> Also fixed a silly mistake in the return type definition for most
> smp_load_acquire() implementions: typeof(p) vs typeof(*p).
>
> ---
> Subject: arch: Introduce smp_load_acquire(), smp_store_release()
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Mon, 4 Nov 2013 20:18:11 +0100
>
> A number of situations currently require the heavyweight smp_mb(),
> even though there is no need to order prior stores against later
> loads. Many architectures have much cheaper ways to handle these
> situations, but the Linux kernel currently has no portable way
> to make use of them.
>
> This commit therefore supplies smp_load_acquire() and
> smp_store_release() to remedy this situation. The new
> smp_load_acquire() primitive orders the specified load against
> any subsequent reads or writes, while the new smp_store_release()
> primitive orders the specifed store against any prior reads or
> writes. These primitives allow array-based circular FIFOs to be
> implemented without an smp_mb(), and also allow a theoretical
> hole in rcu_assign_pointer() to be closed at no additional
> expense on most architectures.
>
> In addition, the RCU experience transitioning from explicit
> smp_read_barrier_depends() and smp_wmb() to rcu_dereference()
> and rcu_assign_pointer(), respectively resulted in substantial
> improvements in readability. It therefore seems likely that
> replacing other explicit barriers with smp_load_acquire() and
> smp_store_release() will provide similar benefits. It appears
> that roughly half of the explicit barriers in core kernel code
> might be so replaced.
>
>
> Cc: Michael Ellerman <michael@ellerman.id.au>
> Cc: Michael Neuling <mikey@neuling.org>
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Victor Kaplansky <VICTORK@il.ibm.com>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Anton Blanchard <anton@samba.org>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> Signed-off-by: Peter Zijlstra <peterz@infradead.org>
A few nits on Documentation/memory-barriers.txt and some pointless
comments elsewhere. With the suggested Documentation/memory-barriers.txt
fixes:
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
> Documentation/memory-barriers.txt | 157 +++++++++++++++++-----------------
> arch/alpha/include/asm/barrier.h | 25 +----
> arch/arc/include/asm/Kbuild | 1
> arch/arc/include/asm/atomic.h | 5 +
> arch/arc/include/asm/barrier.h | 42 ---------
> arch/arm/include/asm/barrier.h | 15 +++
> arch/arm64/include/asm/barrier.h | 50 ++++++++++
> arch/avr32/include/asm/barrier.h | 17 +--
> arch/blackfin/include/asm/barrier.h | 18 ---
> arch/cris/include/asm/Kbuild | 1
> arch/cris/include/asm/barrier.h | 25 -----
> arch/frv/include/asm/barrier.h | 8 -
> arch/h8300/include/asm/barrier.h | 21 ----
> arch/hexagon/include/asm/Kbuild | 1
> arch/hexagon/include/asm/barrier.h | 41 --------
> arch/ia64/include/asm/barrier.h | 49 ++++++++++
> arch/m32r/include/asm/barrier.h | 80 -----------------
> arch/m68k/include/asm/barrier.h | 14 ---
> arch/metag/include/asm/barrier.h | 15 +++
> arch/microblaze/include/asm/Kbuild | 1
> arch/microblaze/include/asm/barrier.h | 27 -----
> arch/mips/include/asm/barrier.h | 15 +++
> arch/mn10300/include/asm/Kbuild | 1
> arch/mn10300/include/asm/barrier.h | 37 --------
> arch/parisc/include/asm/Kbuild | 1
> arch/parisc/include/asm/barrier.h | 35 -------
> arch/powerpc/include/asm/barrier.h | 21 ++++
> arch/s390/include/asm/barrier.h | 15 +++
> arch/score/include/asm/Kbuild | 1
> arch/score/include/asm/barrier.h | 16 ---
> arch/sh/include/asm/barrier.h | 21 ----
> arch/sparc/include/asm/barrier_32.h | 11 --
> arch/sparc/include/asm/barrier_64.h | 15 +++
> arch/tile/include/asm/barrier.h | 68 --------------
> arch/unicore32/include/asm/barrier.h | 11 --
> arch/x86/include/asm/barrier.h | 15 +++
> arch/xtensa/include/asm/barrier.h | 9 -
> include/asm-generic/barrier.h | 55 +++++++++--
> include/linux/compiler.h | 9 +
> 39 files changed, 375 insertions(+), 594 deletions(-)
>
> --- a/Documentation/memory-barriers.txt
> +++ b/Documentation/memory-barriers.txt
> @@ -371,33 +371,35 @@ VARIETIES OF MEMORY BARRIER
>
> And a couple of implicit varieties:
>
> - (5) LOCK operations.
> + (5) ACQUIRE operations.
>
> This acts as a one-way permeable barrier. It guarantees that all memory
> - operations after the LOCK operation will appear to happen after the LOCK
> - operation with respect to the other components of the system.
> + operations after the ACQUIRE operation will appear to happen after the
> + ACQUIRE operation with respect to the other components of the system.
ACQUIRE operations include LOCK operations and smp_load_acquire()
operations.
>
> - Memory operations that occur before a LOCK operation may appear to happen
> - after it completes.
> + Memory operations that occur before a ACQUIRE operation may appear to
> + happen after it completes.
>
> - A LOCK operation should almost always be paired with an UNLOCK operation.
> + A ACQUIRE operation should almost always be paired with an RELEASE
> + operation.
>
>
> - (6) UNLOCK operations.
> + (6) RELEASE operations.
>
> This also acts as a one-way permeable barrier. It guarantees that all
> - memory operations before the UNLOCK operation will appear to happen before
> - the UNLOCK operation with respect to the other components of the system.
> + memory operations before the RELEASE operation will appear to happen
> + before the RELEASE operation with respect to the other components of the
> + system. Release operations include UNLOCK operations and
smp_store_release() operations.
> - Memory operations that occur after an UNLOCK operation may appear to
> + Memory operations that occur after an RELEASE operation may appear to
> happen before it completes.
>
> - LOCK and UNLOCK operations are guaranteed to appear with respect to each
> - other strictly in the order specified.
> + ACQUIRE and RELEASE operations are guaranteed to appear with respect to
> + each other strictly in the order specified.
>
> - The use of LOCK and UNLOCK operations generally precludes the need for
> - other sorts of memory barrier (but note the exceptions mentioned in the
> - subsection "MMIO write barrier").
> + The use of ACQUIRE and RELEASE operations generally precludes the need
> + for other sorts of memory barrier (but note the exceptions mentioned in
> + the subsection "MMIO write barrier").
>
>
> Memory barriers are only required where there's a possibility of interaction
> @@ -1135,7 +1137,7 @@ CPU from reordering them.
> clear_bit( ... );
>
> This prevents memory operations before the clear leaking to after it. See
> - the subsection on "Locking Functions" with reference to UNLOCK operation
> + the subsection on "Locking Functions" with reference to RELEASE operation
> implications.
>
> See Documentation/atomic_ops.txt for more information. See the "Atomic
> @@ -1181,65 +1183,66 @@ LOCKING FUNCTIONS
> (*) R/W semaphores
> (*) RCU
>
> -In all cases there are variants on "LOCK" operations and "UNLOCK" operations
> +In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations
> for each construct. These operations all imply certain barriers:
>
> - (1) LOCK operation implication:
> + (1) ACQUIRE operation implication:
>
> - Memory operations issued after the LOCK will be completed after the LOCK
> - operation has completed.
> + Memory operations issued after the ACQUIRE will be completed after the
> + ACQUIRE operation has completed.
>
> - Memory operations issued before the LOCK may be completed after the LOCK
> - operation has completed.
> + Memory operations issued before the ACQUIRE may be completed after the
> + ACQUIRE operation has completed.
>
> - (2) UNLOCK operation implication:
> + (2) RELEASE operation implication:
>
> - Memory operations issued before the UNLOCK will be completed before the
> - UNLOCK operation has completed.
> + Memory operations issued before the RELEASE will be completed before the
> + RELEASE operation has completed.
>
> - Memory operations issued after the UNLOCK may be completed before the
> - UNLOCK operation has completed.
> + Memory operations issued after the RELEASE may be completed before the
> + RELEASE operation has completed.
>
> - (3) LOCK vs LOCK implication:
> + (3) ACQUIRE vs ACQUIRE implication:
>
> - All LOCK operations issued before another LOCK operation will be completed
> - before that LOCK operation.
> + All ACQUIRE operations issued before another ACQUIRE operation will be
> + completed before that ACQUIRE operation.
>
> - (4) LOCK vs UNLOCK implication:
> + (4) ACQUIRE vs RELEASE implication:
>
> - All LOCK operations issued before an UNLOCK operation will be completed
> - before the UNLOCK operation.
> + All ACQUIRE operations issued before an RELEASE operation will be
> + completed before the RELEASE operation.
>
> - All UNLOCK operations issued before a LOCK operation will be completed
> - before the LOCK operation.
> + All RELEASE operations issued before a ACQUIRE operation will be
> + completed before the ACQUIRE operation.
>
> - (5) Failed conditional LOCK implication:
> + (5) Failed conditional ACQUIRE implication:
>
> - Certain variants of the LOCK operation may fail, either due to being
> + Certain variants of the ACQUIRE operation may fail, either due to being
> unable to get the lock immediately, or due to receiving an unblocked
> signal whilst asleep waiting for the lock to become available. Failed
> locks do not imply any sort of barrier.
I suggest adding "For example" to the beginning of the last sentence:
For example, failed lock acquisitions do not imply any sort of
barrier.
Otherwise, the transition from ACQUIRE to lock is strange.
> -Therefore, from (1), (2) and (4) an UNLOCK followed by an unconditional LOCK is
> -equivalent to a full barrier, but a LOCK followed by an UNLOCK is not.
> +Therefore, from (1), (2) and (4) an RELEASE followed by an unconditional
> +ACQUIRE is equivalent to a full barrier, but a ACQUIRE followed by an RELEASE
> +is not.
>
> [!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way
> barriers is that the effects of instructions outside of a critical section
> may seep into the inside of the critical section.
>
> -A LOCK followed by an UNLOCK may not be assumed to be full memory barrier
> -because it is possible for an access preceding the LOCK to happen after the
> -LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the
> -two accesses can themselves then cross:
> +A ACQUIRE followed by an RELEASE may not be assumed to be full memory barrier
> +because it is possible for an access preceding the ACQUIRE to happen after the
> +ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and
> +the two accesses can themselves then cross:
>
> *A = a;
> - LOCK
> - UNLOCK
> + ACQUIRE
> + RELEASE
> *B = b;
>
> may occur as:
>
> - LOCK, STORE *B, STORE *A, UNLOCK
> + ACQUIRE, STORE *B, STORE *A, RELEASE
>
> Locks and semaphores may not provide any guarantee of ordering on UP compiled
> systems, and so cannot be counted on in such a situation to actually achieve
> @@ -1253,33 +1256,33 @@ See also the section on "Inter-CPU locki
>
> *A = a;
> *B = b;
> - LOCK
> + ACQUIRE
> *C = c;
> *D = d;
> - UNLOCK
> + RELEASE
> *E = e;
> *F = f;
>
> The following sequence of events is acceptable:
>
> - LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK
> + ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE
>
> [+] Note that {*F,*A} indicates a combined access.
>
> But none of the following are:
>
> - {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E
> - *A, *B, *C, LOCK, *D, UNLOCK, *E, *F
> - *A, *B, LOCK, *C, UNLOCK, *D, *E, *F
> - *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E
> + {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E
> + *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F
> + *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F
> + *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E
>
>
>
> INTERRUPT DISABLING FUNCTIONS
> -----------------------------
>
> -Functions that disable interrupts (LOCK equivalent) and enable interrupts
> -(UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O
> +Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts
> +(RELEASE equivalent) will act as compiler barriers only. So if memory or I/O
> barriers are required in such a situation, they must be provided from some
> other means.
>
> @@ -1436,24 +1439,24 @@ Consider the following: the system has a
> CPU 1 CPU 2
> =============================== ===============================
> *A = a; *E = e;
> - LOCK M LOCK Q
> + ACQUIRE M ACQUIRE Q
> *B = b; *F = f;
> *C = c; *G = g;
> - UNLOCK M UNLOCK Q
> + RELEASE M RELEASE Q
> *D = d; *H = h;
>
> Then there is no guarantee as to what order CPU 3 will see the accesses to *A
> through *H occur in, other than the constraints imposed by the separate locks
> on the separate CPUs. It might, for example, see:
>
> - *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M
> + *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M
>
> But it won't see any of:
>
> - *B, *C or *D preceding LOCK M
> - *A, *B or *C following UNLOCK M
> - *F, *G or *H preceding LOCK Q
> - *E, *F or *G following UNLOCK Q
> + *B, *C or *D preceding ACQUIRE M
> + *A, *B or *C following RELEASE M
> + *F, *G or *H preceding ACQUIRE Q
> + *E, *F or *G following RELEASE Q
>
>
> However, if the following occurs:
> @@ -1461,28 +1464,28 @@ through *H occur in, other than the cons
> CPU 1 CPU 2
> =============================== ===============================
> *A = a;
> - LOCK M [1]
> + ACQUIRE M [1]
> *B = b;
> *C = c;
> - UNLOCK M [1]
> + RELEASE M [1]
> *D = d; *E = e;
> - LOCK M [2]
> + ACQUIRE M [2]
> *F = f;
> *G = g;
> - UNLOCK M [2]
> + RELEASE M [2]
> *H = h;
>
> CPU 3 might see:
>
> - *E, LOCK M [1], *C, *B, *A, UNLOCK M [1],
> - LOCK M [2], *H, *F, *G, UNLOCK M [2], *D
> + *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1],
> + ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D
>
> But assuming CPU 1 gets the lock first, CPU 3 won't see any of:
>
> - *B, *C, *D, *F, *G or *H preceding LOCK M [1]
> - *A, *B or *C following UNLOCK M [1]
> - *F, *G or *H preceding LOCK M [2]
> - *A, *B, *C, *E, *F or *G following UNLOCK M [2]
> + *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1]
> + *A, *B or *C following RELEASE M [1]
> + *F, *G or *H preceding ACQUIRE M [2]
> + *A, *B, *C, *E, *F or *G following RELEASE M [2]
>
>
> LOCKS VS I/O ACCESSES
> @@ -1702,13 +1705,13 @@ about the state (old or new) implies an
> test_and_clear_bit();
> test_and_change_bit();
>
> -These are used for such things as implementing LOCK-class and UNLOCK-class
> +These are used for such things as implementing ACQUIRE-class and RELEASE-class
> operations and adjusting reference counters towards object destruction, and as
> such the implicit memory barrier effects are necessary.
>
>
> The following operations are potential problems as they do _not_ imply memory
> -barriers, but might be used for implementing such things as UNLOCK-class
> +barriers, but might be used for implementing such things as RELEASE-class
> operations:
>
> atomic_set();
> @@ -1750,9 +1753,9 @@ barriers are needed or not.
> clear_bit_unlock();
> __clear_bit_unlock();
>
> -These implement LOCK-class and UNLOCK-class operations. These should be used in
> -preference to other operations when implementing locking primitives, because
> -their implementations can be optimised on many architectures.
> +These implement ACQUIRE-class and RELEASE-class operations. These should be
> +used in preference to other operations when implementing locking primitives,
> +because their implementations can be optimised on many architectures.
>
> [!] Note that special memory barrier primitives are available for these
> situations because on some CPUs the atomic instructions used imply full memory
> --- a/arch/alpha/include/asm/barrier.h
> +++ b/arch/alpha/include/asm/barrier.h
> @@ -3,33 +3,18 @@
>
> #include <asm/compiler.h>
>
> -#define mb() \
> -__asm__ __volatile__("mb": : :"memory")
> +#define mb() __asm__ __volatile__("mb": : :"memory")
> +#define rmb() __asm__ __volatile__("mb": : :"memory")
> +#define wmb() __asm__ __volatile__("wmb": : :"memory")
>
> -#define rmb() \
> -__asm__ __volatile__("mb": : :"memory")
> -
> -#define wmb() \
> -__asm__ __volatile__("wmb": : :"memory")
> -
> -#define read_barrier_depends() \
> -__asm__ __volatile__("mb": : :"memory")
> +#define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
>
> #ifdef CONFIG_SMP
> #define __ASM_SMP_MB "\tmb\n"
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> #else
> #define __ASM_SMP_MB
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while (0)
> #endif
>
> -#define set_mb(var, value) \
> -do { var = value; mb(); } while (0)
> +#include <asm-generic/barrier.h>
>
> #endif /* __BARRIER_H */
> --- a/arch/arc/include/asm/Kbuild
> +++ b/arch/arc/include/asm/Kbuild
> @@ -47,3 +47,4 @@ generic-y += user.h
> generic-y += vga.h
> generic-y += xor.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/arc/include/asm/atomic.h
> +++ b/arch/arc/include/asm/atomic.h
> @@ -190,6 +190,11 @@ static inline void atomic_clear_mask(uns
>
> #endif /* !CONFIG_ARC_HAS_LLSC */
>
> +#define smp_mb__before_atomic_dec() barrier()
> +#define smp_mb__after_atomic_dec() barrier()
> +#define smp_mb__before_atomic_inc() barrier()
> +#define smp_mb__after_atomic_inc() barrier()
> +
> /**
> * __atomic_add_unless - add unless the number is a given value
> * @v: pointer of type atomic_t
> --- a/arch/arc/include/asm/barrier.h
> +++ /dev/null
> @@ -1,42 +0,0 @@
> -/*
> - * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 as
> - * published by the Free Software Foundation.
> - */
> -
> -#ifndef __ASM_BARRIER_H
> -#define __ASM_BARRIER_H
> -
> -#ifndef __ASSEMBLY__
> -
> -/* TODO-vineetg: Need to see what this does, don't we need sync anywhere */
> -#define mb() __asm__ __volatile__ ("" : : : "memory")
> -#define rmb() mb()
> -#define wmb() mb()
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -#define set_wmb(var, value) do { var = value; wmb(); } while (0)
> -#define read_barrier_depends() mb()
> -
> -/* TODO-vineetg verify the correctness of macros here */
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#endif
> -
> -#define smp_mb__before_atomic_dec() barrier()
> -#define smp_mb__after_atomic_dec() barrier()
> -#define smp_mb__before_atomic_inc() barrier()
> -#define smp_mb__after_atomic_inc() barrier()
> -
> -#define smp_read_barrier_depends() do { } while (0)
> -
> -#endif
> -
> -#endif
I do like this take-no-prisoners approach! ;-)
> --- a/arch/arm/include/asm/barrier.h
> +++ b/arch/arm/include/asm/barrier.h
> @@ -59,6 +59,21 @@
> #define smp_wmb() dmb(ishst)
> #endif
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
> +
> #define read_barrier_depends() do { } while(0)
> #define smp_read_barrier_depends() do { } while(0)
>
> --- a/arch/arm64/include/asm/barrier.h
> +++ b/arch/arm64/include/asm/barrier.h
> @@ -35,11 +35,59 @@
> #define smp_mb() barrier()
> #define smp_rmb() barrier()
> #define smp_wmb() barrier()
> +
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
> +
> #else
> +
> #define smp_mb() asm volatile("dmb ish" : : : "memory")
> #define smp_rmb() asm volatile("dmb ishld" : : : "memory")
> #define smp_wmb() asm volatile("dmb ishst" : : : "memory")
> -#endif
> +
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 4: \
> + asm volatile ("stlr %w1, [%0]" \
> + : "=Q" (*p) : "r" (v) : "memory"); \
> + break; \
> + case 8: \
> + asm volatile ("stlr %1, [%0]" \
> + : "=Q" (*p) : "r" (v) : "memory"); \
> + break; \
> + } \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1; \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 4: \
> + asm volatile ("ldar %w0, [%1]" \
> + : "=r" (___p1) : "Q" (*p) : "memory"); \
> + break; \
> + case 8: \
> + asm volatile ("ldar %0, [%1]" \
> + : "=r" (___p1) : "Q" (*p) : "memory"); \
> + break; \
> + } \
> + ___p1; \
> +})
>
> #define read_barrier_depends() do { } while(0)
> #define smp_read_barrier_depends() do { } while(0)
> --- a/arch/avr32/include/asm/barrier.h
> +++ b/arch/avr32/include/asm/barrier.h
> @@ -8,22 +8,15 @@
> #ifndef __ASM_AVR32_BARRIER_H
> #define __ASM_AVR32_BARRIER_H
>
> -#define nop() asm volatile("nop")
> -
> -#define mb() asm volatile("" : : : "memory")
> -#define rmb() mb()
> -#define wmb() asm volatile("sync 0" : : : "memory")
> -#define read_barrier_depends() do { } while(0)
> -#define set_mb(var, value) do { var = value; mb(); } while(0)
> +/*
> + * Weirdest thing ever.. no full barrier, but it has a write barrier!
> + */
> +#define wmb() asm volatile("sync 0" : : : "memory")
Doesn't this mean that asm-generic/barrier.h needs to check for
definitions? Ah, I see below that you added these checks.
> #ifdef CONFIG_SMP
> # error "The AVR32 port does not support SMP"
> -#else
> -# define smp_mb() barrier()
> -# define smp_rmb() barrier()
> -# define smp_wmb() barrier()
> -# define smp_read_barrier_depends() do { } while(0)
> #endif
>
> +#include <asm-generic/barrier.h>
>
> #endif /* __ASM_AVR32_BARRIER_H */
> --- a/arch/blackfin/include/asm/barrier.h
> +++ b/arch/blackfin/include/asm/barrier.h
> @@ -23,26 +23,10 @@
> # define rmb() do { barrier(); smp_check_barrier(); } while (0)
> # define wmb() do { barrier(); smp_mark_barrier(); } while (0)
> # define read_barrier_depends() do { barrier(); smp_check_barrier(); } while (0)
> -#else
> -# define mb() barrier()
> -# define rmb() barrier()
> -# define wmb() barrier()
> -# define read_barrier_depends() do { } while (0)
> #endif
>
> -#else /* !CONFIG_SMP */
> -
> -#define mb() barrier()
> -#define rmb() barrier()
> -#define wmb() barrier()
> -#define read_barrier_depends() do { } while (0)
> -
> #endif /* !CONFIG_SMP */
>
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -#define smp_read_barrier_depends() read_barrier_depends()
> +#include <asm-generic/barrier.h>
>
> #endif /* _BLACKFIN_BARRIER_H */
> --- a/arch/cris/include/asm/Kbuild
> +++ b/arch/cris/include/asm/Kbuild
> @@ -12,3 +12,4 @@ generic-y += trace_clock.h
> generic-y += vga.h
> generic-y += xor.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/cris/include/asm/barrier.h
> +++ /dev/null
> @@ -1,25 +0,0 @@
> -#ifndef __ASM_CRIS_BARRIER_H
> -#define __ASM_CRIS_BARRIER_H
> -
> -#define nop() __asm__ __volatile__ ("nop");
> -
> -#define barrier() __asm__ __volatile__("": : :"memory")
> -#define mb() barrier()
> -#define rmb() mb()
> -#define wmb() mb()
> -#define read_barrier_depends() do { } while(0)
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while(0)
> -#endif
> -
> -#endif /* __ASM_CRIS_BARRIER_H */
> --- a/arch/frv/include/asm/barrier.h
> +++ b/arch/frv/include/asm/barrier.h
> @@ -17,13 +17,7 @@
> #define mb() asm volatile ("membar" : : :"memory")
> #define rmb() asm volatile ("membar" : : :"memory")
> #define wmb() asm volatile ("membar" : : :"memory")
> -#define read_barrier_depends() do { } while (0)
>
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do {} while(0)
> -#define set_mb(var, value) \
> - do { var = (value); barrier(); } while (0)
> +#include <asm-generic/barrier.h>
>
> #endif /* _ASM_BARRIER_H */
> --- a/arch/h8300/include/asm/barrier.h
> +++ b/arch/h8300/include/asm/barrier.h
> @@ -3,27 +3,8 @@
>
> #define nop() asm volatile ("nop"::)
>
> -/*
> - * Force strict CPU ordering.
> - * Not really required on H8...
> - */
> -#define mb() asm volatile ("" : : :"memory")
> -#define rmb() asm volatile ("" : : :"memory")
> -#define wmb() asm volatile ("" : : :"memory")
> #define set_mb(var, value) do { xchg(&var, value); } while (0)
>
> -#define read_barrier_depends() do { } while (0)
> -
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while(0)
> -#endif
> +#include <asm-generic/barrier.h>
>
> #endif /* _H8300_BARRIER_H */
> --- a/arch/hexagon/include/asm/Kbuild
> +++ b/arch/hexagon/include/asm/Kbuild
> @@ -54,3 +54,4 @@ generic-y += ucontext.h
> generic-y += unaligned.h
> generic-y += xor.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/hexagon/include/asm/barrier.h
> +++ /dev/null
> @@ -1,41 +0,0 @@
> -/*
> - * Memory barrier definitions for the Hexagon architecture
> - *
> - * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 and
> - * only version 2 as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> - * GNU General Public License for more details.
> - *
> - * You should have received a copy of the GNU General Public License
> - * along with this program; if not, write to the Free Software
> - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> - * 02110-1301, USA.
> - */
> -
> -#ifndef _ASM_BARRIER_H
> -#define _ASM_BARRIER_H
> -
> -#define rmb() barrier()
> -#define read_barrier_depends() barrier()
> -#define wmb() barrier()
> -#define mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_read_barrier_depends() barrier()
> -#define smp_wmb() barrier()
> -#define smp_mb() barrier()
> -#define smp_mb__before_atomic_dec() barrier()
> -#define smp_mb__after_atomic_dec() barrier()
> -#define smp_mb__before_atomic_inc() barrier()
> -#define smp_mb__after_atomic_inc() barrier()
> -
> -/* Set a value and use a memory barrier. Used by the scheduler somewhere. */
> -#define set_mb(var, value) \
> - do { var = value; mb(); } while (0)
> -
> -#endif /* _ASM_BARRIER_H */
> --- a/arch/ia64/include/asm/barrier.h
> +++ b/arch/ia64/include/asm/barrier.h
> @@ -45,11 +45,60 @@
> # define smp_rmb() rmb()
> # define smp_wmb() wmb()
> # define smp_read_barrier_depends() read_barrier_depends()
> +
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 4: \
> + asm volatile ("st4.rel [%0]=%1" \
> + : "=r" (p) : "r" (v) : "memory"); \
> + break; \
> + case 8: \
> + asm volatile ("st8.rel [%0]=%1" \
> + : "=r" (p) : "r" (v) : "memory"); \
> + break; \
> + } \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1; \
> + compiletime_assert_atomic_type(*p); \
> + switch (sizeof(*p)) { \
> + case 4: \
> + asm volatile ("ld4.acq %0=[%1]" \
> + : "=r" (___p1) : "r" (p) : "memory"); \
> + break; \
> + case 8: \
> + asm volatile ("ld8.acq %0=[%1]" \
> + : "=r" (___p1) : "r" (p) : "memory"); \
> + break; \
> + } \
> + ___p1; \
> +})
> +
> #else
> +
> # define smp_mb() barrier()
> # define smp_rmb() barrier()
> # define smp_wmb() barrier()
> # define smp_read_barrier_depends() do { } while(0)
> +
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
> #endif
>
> /*
> --- a/arch/m32r/include/asm/barrier.h
> +++ b/arch/m32r/include/asm/barrier.h
> @@ -11,84 +11,6 @@
>
> #define nop() __asm__ __volatile__ ("nop" : : )
>
> -/*
> - * Memory barrier.
> - *
> - * mb() prevents loads and stores being reordered across this point.
> - * rmb() prevents loads being reordered across this point.
> - * wmb() prevents stores being reordered across this point.
> - */
> -#define mb() barrier()
> -#define rmb() mb()
> -#define wmb() mb()
> -
> -/**
> - * read_barrier_depends - Flush all pending reads that subsequents reads
> - * depend on.
> - *
> - * No data-dependent reads from memory-like regions are ever reordered
> - * over this barrier. All reads preceding this primitive are guaranteed
> - * to access memory (but not necessarily other CPUs' caches) before any
> - * reads following this primitive that depend on the data return by
> - * any of the preceding reads. This primitive is much lighter weight than
> - * rmb() on most CPUs, and is never heavier weight than is
> - * rmb().
> - *
> - * These ordering constraints are respected by both the local CPU
> - * and the compiler.
> - *
> - * Ordering is not guaranteed by anything other than these primitives,
> - * not even by data dependencies. See the documentation for
> - * memory_barrier() for examples and URLs to more information.
> - *
> - * For example, the following code would force ordering (the initial
> - * value of "a" is zero, "b" is one, and "p" is "&a"):
> - *
> - * <programlisting>
> - * CPU 0 CPU 1
> - *
> - * b = 2;
> - * memory_barrier();
> - * p = &b; q = p;
> - * read_barrier_depends();
> - * d = *q;
> - * </programlisting>
> - *
> - *
> - * because the read of "*q" depends on the read of "p" and these
> - * two reads are separated by a read_barrier_depends(). However,
> - * the following code, with the same initial values for "a" and "b":
> - *
> - * <programlisting>
> - * CPU 0 CPU 1
> - *
> - * a = 2;
> - * memory_barrier();
> - * b = 3; y = b;
> - * read_barrier_depends();
> - * x = a;
> - * </programlisting>
> - *
> - * does not enforce ordering, since there is no data dependency between
> - * the read of "a" and the read of "b". Therefore, on some CPUs, such
> - * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
> - * in cases like this where there are no data dependencies.
> - **/
> -
> -#define read_barrier_depends() do { } while (0)
> -
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> -#define set_mb(var, value) do { (void) xchg(&var, value); } while (0)
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while (0)
> -#define set_mb(var, value) do { var = value; barrier(); } while (0)
> -#endif
> +#include <asm-generic/barrier.h>
>
> #endif /* _ASM_M32R_BARRIER_H */
> --- a/arch/m68k/include/asm/barrier.h
> +++ b/arch/m68k/include/asm/barrier.h
> @@ -1,20 +1,8 @@
> #ifndef _M68K_BARRIER_H
> #define _M68K_BARRIER_H
>
> -/*
> - * Force strict CPU ordering.
> - * Not really required on m68k...
> - */
> #define nop() do { asm volatile ("nop"); barrier(); } while (0)
> -#define mb() barrier()
> -#define rmb() barrier()
> -#define wmb() barrier()
> -#define read_barrier_depends() ((void)0)
> -#define set_mb(var, value) ({ (var) = (value); wmb(); })
>
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() ((void)0)
> +#include <asm-generic/barrier.h>
>
> #endif /* _M68K_BARRIER_H */
> --- a/arch/metag/include/asm/barrier.h
> +++ b/arch/metag/include/asm/barrier.h
> @@ -82,4 +82,19 @@ static inline void fence(void)
> #define smp_read_barrier_depends() do { } while (0)
> #define set_mb(var, value) do { var = value; smp_mb(); } while (0)
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
> +
> #endif /* _ASM_METAG_BARRIER_H */
> --- a/arch/microblaze/include/asm/Kbuild
> +++ b/arch/microblaze/include/asm/Kbuild
> @@ -4,3 +4,4 @@ generic-y += exec.h
> generic-y += trace_clock.h
> generic-y += syscalls.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/microblaze/include/asm/barrier.h
> +++ /dev/null
> @@ -1,27 +0,0 @@
> -/*
> - * Copyright (C) 2006 Atmark Techno, Inc.
> - *
> - * This file is subject to the terms and conditions of the GNU General Public
> - * License. See the file "COPYING" in the main directory of this archive
> - * for more details.
> - */
> -
> -#ifndef _ASM_MICROBLAZE_BARRIER_H
> -#define _ASM_MICROBLAZE_BARRIER_H
> -
> -#define nop() asm volatile ("nop")
> -
> -#define smp_read_barrier_depends() do {} while (0)
> -#define read_barrier_depends() do {} while (0)
> -
> -#define mb() barrier()
> -#define rmb() mb()
> -#define wmb() mb()
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -#define set_wmb(var, value) do { var = value; wmb(); } while (0)
> -
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -
> -#endif /* _ASM_MICROBLAZE_BARRIER_H */
> --- a/arch/mips/include/asm/barrier.h
> +++ b/arch/mips/include/asm/barrier.h
> @@ -180,4 +180,19 @@
> #define nudge_writes() mb()
> #endif
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
> +
> #endif /* __ASM_BARRIER_H */
> --- a/arch/mn10300/include/asm/Kbuild
> +++ b/arch/mn10300/include/asm/Kbuild
> @@ -3,3 +3,4 @@ generic-y += clkdev.h
> generic-y += exec.h
> generic-y += trace_clock.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/mn10300/include/asm/barrier.h
> +++ /dev/null
> @@ -1,37 +0,0 @@
> -/* MN10300 memory barrier definitions
> - *
> - * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
> - * Written by David Howells (dhowells@redhat.com)
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public Licence
> - * as published by the Free Software Foundation; either version
> - * 2 of the Licence, or (at your option) any later version.
> - */
> -#ifndef _ASM_BARRIER_H
> -#define _ASM_BARRIER_H
> -
> -#define nop() asm volatile ("nop")
> -
> -#define mb() asm volatile ("": : :"memory")
> -#define rmb() mb()
> -#define wmb() asm volatile ("": : :"memory")
> -
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define set_mb(var, value) do { xchg(&var, value); } while (0)
> -#else /* CONFIG_SMP */
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -#endif /* CONFIG_SMP */
> -
> -#define set_wmb(var, value) do { var = value; wmb(); } while (0)
> -
> -#define read_barrier_depends() do {} while (0)
> -#define smp_read_barrier_depends() do {} while (0)
> -
> -#endif /* _ASM_BARRIER_H */
> --- a/arch/parisc/include/asm/Kbuild
> +++ b/arch/parisc/include/asm/Kbuild
> @@ -5,3 +5,4 @@ generic-y += word-at-a-time.h auxvec.h u
> poll.h xor.h clkdev.h exec.h
> generic-y += trace_clock.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/parisc/include/asm/barrier.h
> +++ /dev/null
> @@ -1,35 +0,0 @@
> -#ifndef __PARISC_BARRIER_H
> -#define __PARISC_BARRIER_H
> -
> -/*
> -** This is simply the barrier() macro from linux/kernel.h but when serial.c
> -** uses tqueue.h uses smp_mb() defined using barrier(), linux/kernel.h
> -** hasn't yet been included yet so it fails, thus repeating the macro here.
> -**
> -** PA-RISC architecture allows for weakly ordered memory accesses although
> -** none of the processors use it. There is a strong ordered bit that is
> -** set in the O-bit of the page directory entry. Operating systems that
> -** can not tolerate out of order accesses should set this bit when mapping
> -** pages. The O-bit of the PSW should also be set to 1 (I don't believe any
> -** of the processor implemented the PSW O-bit). The PCX-W ERS states that
> -** the TLB O-bit is not implemented so the page directory does not need to
> -** have the O-bit set when mapping pages (section 3.1). This section also
> -** states that the PSW Y, Z, G, and O bits are not implemented.
> -** So it looks like nothing needs to be done for parisc-linux (yet).
> -** (thanks to chada for the above comment -ggg)
> -**
> -** The __asm__ op below simple prevents gcc/ld from reordering
> -** instructions across the mb() "call".
> -*/
> -#define mb() __asm__ __volatile__("":::"memory") /* barrier() */
> -#define rmb() mb()
> -#define wmb() mb()
> -#define smp_mb() mb()
> -#define smp_rmb() mb()
> -#define smp_wmb() mb()
> -#define smp_read_barrier_depends() do { } while(0)
> -#define read_barrier_depends() do { } while(0)
> -
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> -
> -#endif /* __PARISC_BARRIER_H */
> --- a/arch/powerpc/include/asm/barrier.h
> +++ b/arch/powerpc/include/asm/barrier.h
> @@ -45,11 +45,15 @@
> # define SMPWMB eieio
> #endif
>
> +#define __lwsync() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
> +
> #define smp_mb() mb()
> -#define smp_rmb() __asm__ __volatile__ (stringify_in_c(LWSYNC) : : :"memory")
> +#define smp_rmb() __lwsync()
> #define smp_wmb() __asm__ __volatile__ (stringify_in_c(SMPWMB) : : :"memory")
> #define smp_read_barrier_depends() read_barrier_depends()
> #else
> +#define __lwsync() barrier()
> +
> #define smp_mb() barrier()
> #define smp_rmb() barrier()
> #define smp_wmb() barrier()
> @@ -65,4 +69,19 @@
> #define data_barrier(x) \
> asm volatile("twi 0,%0,0; isync" : : "r" (x) : "memory");
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + __lwsync(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + __lwsync(); \
> + ___p1; \
> +})
> +
> #endif /* _ASM_POWERPC_BARRIER_H */
> --- a/arch/s390/include/asm/barrier.h
> +++ b/arch/s390/include/asm/barrier.h
> @@ -32,4 +32,19 @@
>
> #define set_mb(var, value) do { var = value; mb(); } while (0)
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ___p1; \
> +})
> +
> #endif /* __ASM_BARRIER_H */
> --- a/arch/score/include/asm/Kbuild
> +++ b/arch/score/include/asm/Kbuild
> @@ -5,3 +5,4 @@ generic-y += clkdev.h
> generic-y += trace_clock.h
> generic-y += xor.h
> generic-y += preempt.h
> +generic-y += barrier.h
> --- a/arch/score/include/asm/barrier.h
> +++ /dev/null
> @@ -1,16 +0,0 @@
> -#ifndef _ASM_SCORE_BARRIER_H
> -#define _ASM_SCORE_BARRIER_H
> -
> -#define mb() barrier()
> -#define rmb() barrier()
> -#define wmb() barrier()
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -
> -#define read_barrier_depends() do {} while (0)
> -#define smp_read_barrier_depends() do {} while (0)
> -
> -#define set_mb(var, value) do {var = value; wmb(); } while (0)
> -
> -#endif /* _ASM_SCORE_BARRIER_H */
> --- a/arch/sh/include/asm/barrier.h
> +++ b/arch/sh/include/asm/barrier.h
> @@ -26,29 +26,14 @@
> #if defined(CONFIG_CPU_SH4A) || defined(CONFIG_CPU_SH5)
> #define mb() __asm__ __volatile__ ("synco": : :"memory")
> #define rmb() mb()
> -#define wmb() __asm__ __volatile__ ("synco": : :"memory")
> +#define wmb() mb()
> #define ctrl_barrier() __icbi(PAGE_OFFSET)
> -#define read_barrier_depends() do { } while(0)
> #else
> -#define mb() __asm__ __volatile__ ("": : :"memory")
> -#define rmb() mb()
> -#define wmb() __asm__ __volatile__ ("": : :"memory")
> #define ctrl_barrier() __asm__ __volatile__ ("nop;nop;nop;nop;nop;nop;nop;nop")
> -#define read_barrier_depends() do { } while(0)
> -#endif
> -
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while(0)
> #endif
>
> #define set_mb(var, value) do { (void)xchg(&var, value); } while (0)
>
> +#include <asm-generic/barrier.h>
> +
> #endif /* __ASM_SH_BARRIER_H */
> --- a/arch/sparc/include/asm/barrier_32.h
> +++ b/arch/sparc/include/asm/barrier_32.h
> @@ -1,15 +1,6 @@
> #ifndef __SPARC_BARRIER_H
> #define __SPARC_BARRIER_H
>
> -/* XXX Change this if we ever use a PSO mode kernel. */
> -#define mb() __asm__ __volatile__ ("" : : : "memory")
> -#define rmb() mb()
> -#define wmb() mb()
> -#define read_barrier_depends() do { } while(0)
> -#define set_mb(__var, __value) do { __var = __value; mb(); } while(0)
> -#define smp_mb() __asm__ __volatile__("":::"memory")
> -#define smp_rmb() __asm__ __volatile__("":::"memory")
> -#define smp_wmb() __asm__ __volatile__("":::"memory")
> -#define smp_read_barrier_depends() do { } while(0)
> +#include <asm-generic/barrier.h>
>
> #endif /* !(__SPARC_BARRIER_H) */
> --- a/arch/sparc/include/asm/barrier_64.h
> +++ b/arch/sparc/include/asm/barrier_64.h
> @@ -53,4 +53,19 @@ do { __asm__ __volatile__("ba,pt %%xcc,
>
> #define smp_read_barrier_depends() do { } while(0)
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ___p1; \
> +})
> +
> #endif /* !(__SPARC64_BARRIER_H) */
> --- a/arch/tile/include/asm/barrier.h
> +++ b/arch/tile/include/asm/barrier.h
> @@ -22,59 +22,6 @@
> #include <arch/spr_def.h>
> #include <asm/timex.h>
>
> -/*
> - * read_barrier_depends - Flush all pending reads that subsequents reads
> - * depend on.
> - *
> - * No data-dependent reads from memory-like regions are ever reordered
> - * over this barrier. All reads preceding this primitive are guaranteed
> - * to access memory (but not necessarily other CPUs' caches) before any
> - * reads following this primitive that depend on the data return by
> - * any of the preceding reads. This primitive is much lighter weight than
> - * rmb() on most CPUs, and is never heavier weight than is
> - * rmb().
> - *
> - * These ordering constraints are respected by both the local CPU
> - * and the compiler.
> - *
> - * Ordering is not guaranteed by anything other than these primitives,
> - * not even by data dependencies. See the documentation for
> - * memory_barrier() for examples and URLs to more information.
> - *
> - * For example, the following code would force ordering (the initial
> - * value of "a" is zero, "b" is one, and "p" is "&a"):
> - *
> - * <programlisting>
> - * CPU 0 CPU 1
> - *
> - * b = 2;
> - * memory_barrier();
> - * p = &b; q = p;
> - * read_barrier_depends();
> - * d = *q;
> - * </programlisting>
> - *
> - * because the read of "*q" depends on the read of "p" and these
> - * two reads are separated by a read_barrier_depends(). However,
> - * the following code, with the same initial values for "a" and "b":
> - *
> - * <programlisting>
> - * CPU 0 CPU 1
> - *
> - * a = 2;
> - * memory_barrier();
> - * b = 3; y = b;
> - * read_barrier_depends();
> - * x = a;
> - * </programlisting>
> - *
> - * does not enforce ordering, since there is no data dependency between
> - * the read of "a" and the read of "b". Therefore, on some CPUs, such
> - * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
> - * in cases like this where there are no data dependencies.
> - */
> -#define read_barrier_depends() do { } while (0)
> -
> #define __sync() __insn_mf()
>
> #include <hv/syscall_public.h>
> @@ -125,20 +72,7 @@ mb_incoherent(void)
> #define mb() fast_mb()
> #define iob() fast_iob()
>
> -#ifdef CONFIG_SMP
> -#define smp_mb() mb()
> -#define smp_rmb() rmb()
> -#define smp_wmb() wmb()
> -#define smp_read_barrier_depends() read_barrier_depends()
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define smp_read_barrier_depends() do { } while (0)
> -#endif
> -
> -#define set_mb(var, value) \
> - do { var = value; mb(); } while (0)
> +#include <asm-generic/barrier.h>
>
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASM_TILE_BARRIER_H */
> --- a/arch/unicore32/include/asm/barrier.h
> +++ b/arch/unicore32/include/asm/barrier.h
> @@ -14,15 +14,6 @@
> #define dsb() __asm__ __volatile__ ("" : : : "memory")
> #define dmb() __asm__ __volatile__ ("" : : : "memory")
>
> -#define mb() barrier()
> -#define rmb() barrier()
> -#define wmb() barrier()
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> -#define read_barrier_depends() do { } while (0)
> -#define smp_read_barrier_depends() do { } while (0)
> -
> -#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
> +#include <asm-generic/barrier.h>
>
> #endif /* __UNICORE_BARRIER_H__ */
> --- a/arch/x86/include/asm/barrier.h
> +++ b/arch/x86/include/asm/barrier.h
> @@ -100,6 +100,21 @@
> #define set_mb(var, value) do { var = value; barrier(); } while (0)
> #endif
>
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + barrier(); \
> + ___p1; \
> +})
> +
> /*
> * Stop RDTSC speculation. This is needed when you need to use RDTSC
> * (or get_cycles or vread that possibly accesses the TSC) in a defined
> --- a/arch/xtensa/include/asm/barrier.h
> +++ b/arch/xtensa/include/asm/barrier.h
> @@ -9,21 +9,14 @@
> #ifndef _XTENSA_SYSTEM_H
> #define _XTENSA_SYSTEM_H
>
> -#define smp_read_barrier_depends() do { } while(0)
> -#define read_barrier_depends() do { } while(0)
> -
> #define mb() ({ __asm__ __volatile__("memw" : : : "memory"); })
> #define rmb() barrier()
> #define wmb() mb()
>
> #ifdef CONFIG_SMP
> #error smp_* not defined
> -#else
> -#define smp_mb() barrier()
> -#define smp_rmb() barrier()
> -#define smp_wmb() barrier()
> #endif
>
> -#define set_mb(var, value) do { var = value; mb(); } while (0)
> +#include <asm-generic/barrier.h>
>
> #endif /* _XTENSA_SYSTEM_H */
> --- a/include/asm-generic/barrier.h
> +++ b/include/asm-generic/barrier.h
> @@ -1,4 +1,5 @@
> -/* Generic barrier definitions, based on MN10300 definitions.
> +/*
> + * Generic barrier definitions, based on MN10300 definitions.
> *
> * It should be possible to use these on really simple architectures,
> * but it serves more as a starting point for new ports.
> @@ -16,35 +17,67 @@
>
> #ifndef __ASSEMBLY__
>
> -#define nop() asm volatile ("nop")
> +#include <asm/compiler.h>
> +
> +#ifndef nop
> +#define nop() asm volatile ("nop")
> +#endif
>
> /*
> - * Force strict CPU ordering.
> - * And yes, this is required on UP too when we're talking
> - * to devices.
> + * Force strict CPU ordering. And yes, this is required on UP too when we're
> + * talking to devices.
> *
> - * This implementation only contains a compiler barrier.
> + * Fall back to compiler barriers if nothing better is provided.
> */
>
> -#define mb() asm volatile ("": : :"memory")
> -#define rmb() mb()
> -#define wmb() asm volatile ("": : :"memory")
> +#ifndef mb
> +#define mb() barrier()
> +#endif
> +
> +#ifndef rmb
> +#define rmb() barrier()
> +#endif
> +
> +#ifndef wmb
> +#define wmb() barrier()
> +#endif
> +
> +#ifndef read_barrier_depends
> +#define read_barrier_depends() do {} while (0)
> +#endif
>
> #ifdef CONFIG_SMP
> #define smp_mb() mb()
> #define smp_rmb() rmb()
> #define smp_wmb() wmb()
> +#define smp_read_barrier_depends() read_barrier_depends()
> #else
> #define smp_mb() barrier()
> #define smp_rmb() barrier()
> #define smp_wmb() barrier()
> +#define smp_read_barrier_depends() do {} while (0)
> #endif
>
> +#ifndef set_mb
> #define set_mb(var, value) do { var = value; mb(); } while (0)
> +#endif
> +
> #define set_wmb(var, value) do { var = value; wmb(); } while (0)
>
> -#define read_barrier_depends() do {} while (0)
> -#define smp_read_barrier_depends() do {} while (0)
> +#define smp_store_release(p, v) \
> +do { \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ACCESS_ONCE(*p) = (v); \
> +} while (0)
> +
> +#define smp_load_acquire(p) \
> +({ \
> + typeof(*p) ___p1 = ACCESS_ONCE(*p); \
> + compiletime_assert_atomic_type(*p); \
> + smp_mb(); \
> + ___p1; \
> +})
>
> #endif /* !__ASSEMBLY__ */
> #endif /* __ASM_GENERIC_BARRIER_H */
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -298,6 +298,11 @@ void ftrace_likely_update(struct ftrace_
> # define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
> #endif
>
> +/* Is this type a native word size -- useful for atomic operations */
> +#ifndef __native_word
> +# define __native_word(t) (sizeof(t) == sizeof(int) || sizeof(t) == sizeof(long))
> +#endif
> +
> /* Compile time object size, -1 for unknown */
> #ifndef __compiletime_object_size
> # define __compiletime_object_size(obj) -1
> @@ -337,6 +342,10 @@ void ftrace_likely_update(struct ftrace_
> #define compiletime_assert(condition, msg) \
> _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
>
> +#define compiletime_assert_atomic_type(t) \
> + compiletime_assert(__native_word(t), \
> + "Need native word sized stores/loads for atomicity.")
> +
> /*
> * Prevent the compiler from merging or refetching accesses. The compiler
> * is also forbidden from reordering successive instances of ACCESS_ONCE(),
>
^ permalink raw reply
* Re: [RFC] arch: Introduce new TSO memory barrier smp_tmb()
From: Peter Zijlstra @ 2013-11-06 19:42 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Michael Neuling, Tony Luck, Mathieu Desnoyers, Heiko Carstens,
Oleg Nesterov, LKML, Linux PPC dev, Geert Uytterhoeven,
Anton Blanchard, Frederic Weisbecker, Victor Kaplansky,
Russell King, Linus Torvalds, Martin Schwidefsky
In-Reply-To: <20131106184848.GM18245@linux.vnet.ibm.com>
On Wed, Nov 06, 2013 at 10:48:48AM -0800, Paul E. McKenney wrote:
> A few nits on Documentation/memory-barriers.txt and some pointless
> comments elsewhere. With the suggested Documentation/memory-barriers.txt
> fixes:
>
> Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Thanks, I think I'll cut the thing into a number of smaller patches with
identical end result. Will (hopefully) post a full new series tomorrow
somewhere.
I was thinking like:
1 - aggressively employ asm-generic/barrier.h
2 - Reformulate _The_ document to ACQUIRE/RELEASE
3 - add the new store/load thingies
That should hopefully be slightly easier to look at.
^ permalink raw reply
* Re: [RFC PATCH] ehci-platform: Merge ppc-of EHCI driver into the ehci-platform driver
From: Benjamin Herrenschmidt @ 2013-11-06 19:57 UTC (permalink / raw)
To: Alistair Popple
Cc: linuxppc-dev, linux-usb, Alan Stern, Russell King - ARM Linux
In-Reply-To: <1690980.YlPWToa1rW@mexican>
On Wed, 2013-11-06 at 18:39 +1100, Alistair Popple wrote:
> diff --git a/arch/powerpc/boot/dts/sequoia.dts
> b/arch/powerpc/boot/dts/sequoia.dts
> index b1d3292..e28371e 100644
> --- a/arch/powerpc/boot/dts/sequoia.dts
> +++ b/arch/powerpc/boot/dts/sequoia.dts
> @@ -153,7 +153,7 @@
> };
>
> USB0: ehci@e0000300 {
> - compatible = "ibm,usb-ehci-440epx", "usb-ehci";
> + compatible = "ibm,usb-ehci-440epx";
Compatible properties are string lists so it *should* be ok to have all
of them in the list. However I do remember Russell mentioning a problem
with the current matching code so adding him on CC for comments.
> +/*
> + * 440EPx Errata USBH_3
> + * Fix: Enable Break Memory Transfer (BMT) in INSNREG3
> + */
> +#define PPC440EPX_EHCI0_INSREG_BMT (0x1 << 0)
> +static int
> +ppc44x_enable_bmt(struct device_node *dn)
> +{
> + __iomem u32 *insreg_virt;
> +
> + insreg_virt = of_iomap(dn, 1);
> + if (!insreg_virt)
> + return -EINVAL;
> +
> + out_be32(insreg_virt + 3, PPC440EPX_EHCI0_INSREG_BMT);
> +
> + iounmap(insreg_virt);
> + return 0;
> +}
> +
I would go even further and add the 44x workarounds to the normal
platform device, with a compatible check in there. That isn't the
first time we add quirks to an existing driver.
> + /* Initialise platform data from device tree if available. */
> + if (!dn) {
That was supposed to be if (dn) no ?
> + if (of_get_property(dn, "big-endian", NULL)) {
> + pdata->big_endian_mmio = 1;
> + pdata->big_endian_desc = 1;
> + }
> + if (of_get_property(dn, "big-endian-regs", NULL))
> + pdata->big_endian_mmio = 1;
> + if (of_get_property(dn, "big-endian-desc", NULL))
> + pdata->big_endian_desc = 1;
> + }
> +
> irq = platform_get_irq(dev, 0);
> if (irq < 0) {
> dev_err(&dev->dev, "no irq provided");
> @@ -203,9 +216,10 @@ static int ehci_platform_resume(struct device *dev)
> #define ehci_platform_resume NULL
> #endif /* CONFIG_PM */
>
> -static const struct of_device_id vt8500_ehci_ids[] = {
> +static const struct of_device_id ehci_platform_ids[] = {
> { .compatible = "via,vt8500-ehci", },
> { .compatible = "wm,prizm-ehci", },
> + { .compatible = "usb-ehci", },
> {}
> };
Cheers,
Ben.
^ permalink raw reply
* Re: [PATCH 3/3] powerpc/kvm: remove redundant assignment
From: Benjamin Herrenschmidt @ 2013-11-06 19:58 UTC (permalink / raw)
To: Alexander Graf; +Cc: Paul Mackerras, linuxppc-dev, kvm-ppc, Liu Ping Fan
In-Reply-To: <740542A8-47BC-4015-8675-D36C72F5837D@suse.de>
On Wed, 2013-11-06 at 12:24 +0100, Alexander Graf wrote:
> On 05.11.2013, at 08:42, Liu Ping Fan <kernelfans@gmail.com> wrote:
>
> > Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>
> Patch description missing.
Do you really need a description for trivial one-lines whose subject
is a perfectly complete description already ?
> Please add Paul's ack in the next revision of this patch :).
>
>
> Alex
>
> > ---
> > arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 -
> > 1 file changed, 1 deletion(-)
> >
> > diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> > index 28160ac..7682837 100644
> > --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> > +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> > @@ -731,7 +731,6 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
> > lock_rmap(rmap);
> >
> > /* Check if we might have been invalidated; let the guest retry if so */
> > - ret = RESUME_GUEST;
> > if (mmu_notifier_retry(vcpu->kvm, mmu_seq)) {
> > unlock_rmap(rmap);
> > goto out_unlock;
> > --
> > 1.8.1.4
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH V7 1/7] pseries/cpuidle: Move processor_idle.c to drivers/cpuidle.
From: Daniel Lezcano @ 2013-11-06 20:58 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110120.31547.77532.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> Move the file from arch specific pseries/processor_idle.c
> to drivers/cpuidle/cpuidle-ibm-power.c
> Make the relevant Makefile and Kconfig changes.
> This will enable having a common backend cpuidle driver
> for POWER platform going forward.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> arch/powerpc/include/asm/processor.h | 2
> arch/powerpc/platforms/pseries/Kconfig | 9 -
> arch/powerpc/platforms/pseries/Makefile | 1
> arch/powerpc/platforms/pseries/processor_idle.c | 364 -----------------------
> drivers/cpuidle/Kconfig | 5
> drivers/cpuidle/Kconfig.powerpc | 10 +
> drivers/cpuidle/Makefile | 4
> drivers/cpuidle/cpuidle-ibm-power.c | 364 +++++++++++++++++++++++
> 8 files changed, 384 insertions(+), 375 deletions(-)
> delete mode 100644 arch/powerpc/platforms/pseries/processor_idle.c
> create mode 100644 drivers/cpuidle/Kconfig.powerpc
> create mode 100644 drivers/cpuidle/cpuidle-ibm-power.c
>
> diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
> index ce4de5a..d53bb88 100644
> --- a/arch/powerpc/include/asm/processor.h
> +++ b/arch/powerpc/include/asm/processor.h
> @@ -428,7 +428,7 @@ enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
> extern int powersave_nap; /* set if nap mode can be used in idle loop */
> extern void power7_nap(void);
>
> -#ifdef CONFIG_PSERIES_IDLE
> +#ifdef CONFIG_CPU_IDLE_IBM_POWER
> extern void update_smt_snooze_delay(int cpu, int residency);
> #else
> static inline void update_smt_snooze_delay(int cpu, int residency) {}
> diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
> index 62b4f80..bb59bb0 100644
> --- a/arch/powerpc/platforms/pseries/Kconfig
> +++ b/arch/powerpc/platforms/pseries/Kconfig
> @@ -119,12 +119,3 @@ config DTL
> which are accessible through a debugfs file.
>
> Say N if you are unsure.
> -
> -config PSERIES_IDLE
> - bool "Cpuidle driver for pSeries platforms"
> - depends on CPU_IDLE
> - depends on PPC_PSERIES
> - default y
> - help
> - Select this option to enable processor idle state management
> - through cpuidle subsystem.
> diff --git a/arch/powerpc/platforms/pseries/Makefile b/arch/powerpc/platforms/pseries/Makefile
> index 6c61ec5..6f2500b 100644
> --- a/arch/powerpc/platforms/pseries/Makefile
> +++ b/arch/powerpc/platforms/pseries/Makefile
> @@ -21,7 +21,6 @@ obj-$(CONFIG_HCALL_STATS) += hvCall_inst.o
> obj-$(CONFIG_CMM) += cmm.o
> obj-$(CONFIG_DTL) += dtl.o
> obj-$(CONFIG_IO_EVENT_IRQ) += io_event_irq.o
> -obj-$(CONFIG_PSERIES_IDLE) += processor_idle.o
> obj-$(CONFIG_LPARCFG) += lparcfg.o
>
> ifeq ($(CONFIG_PPC_PSERIES),y)
> diff --git a/arch/powerpc/platforms/pseries/processor_idle.c b/arch/powerpc/platforms/pseries/processor_idle.c
> deleted file mode 100644
> index a166e38..0000000
> --- a/arch/powerpc/platforms/pseries/processor_idle.c
> +++ /dev/null
> @@ -1,364 +0,0 @@
> -/*
> - * processor_idle - idle state cpuidle driver.
> - * Adapted from drivers/idle/intel_idle.c and
> - * drivers/acpi/processor_idle.c
> - *
> - */
> -
> -#include <linux/kernel.h>
> -#include <linux/module.h>
> -#include <linux/init.h>
> -#include <linux/moduleparam.h>
> -#include <linux/cpuidle.h>
> -#include <linux/cpu.h>
> -#include <linux/notifier.h>
> -
> -#include <asm/paca.h>
> -#include <asm/reg.h>
> -#include <asm/machdep.h>
> -#include <asm/firmware.h>
> -#include <asm/runlatch.h>
> -#include <asm/plpar_wrappers.h>
> -
> -struct cpuidle_driver pseries_idle_driver = {
> - .name = "pseries_idle",
> - .owner = THIS_MODULE,
> -};
> -
> -#define MAX_IDLE_STATE_COUNT 2
> -
> -static int max_idle_state = MAX_IDLE_STATE_COUNT - 1;
> -static struct cpuidle_device __percpu *pseries_cpuidle_devices;
> -static struct cpuidle_state *cpuidle_state_table;
> -
> -static inline void idle_loop_prolog(unsigned long *in_purr)
> -{
> - *in_purr = mfspr(SPRN_PURR);
> - /*
> - * Indicate to the HV that we are idle. Now would be
> - * a good time to find other work to dispatch.
> - */
> - get_lppaca()->idle = 1;
> -}
> -
> -static inline void idle_loop_epilog(unsigned long in_purr)
> -{
> - u64 wait_cycles;
> -
> - wait_cycles = be64_to_cpu(get_lppaca()->wait_state_cycles);
> - wait_cycles += mfspr(SPRN_PURR) - in_purr;
> - get_lppaca()->wait_state_cycles = cpu_to_be64(wait_cycles);
> - get_lppaca()->idle = 0;
> -}
> -
> -static int snooze_loop(struct cpuidle_device *dev,
> - struct cpuidle_driver *drv,
> - int index)
> -{
> - unsigned long in_purr;
> - int cpu = dev->cpu;
> -
> - idle_loop_prolog(&in_purr);
> - local_irq_enable();
> - set_thread_flag(TIF_POLLING_NRFLAG);
> -
> - while ((!need_resched()) && cpu_online(cpu)) {
> - ppc64_runlatch_off();
> - HMT_low();
> - HMT_very_low();
> - }
> -
> - HMT_medium();
> - clear_thread_flag(TIF_POLLING_NRFLAG);
> - smp_mb();
> -
> - idle_loop_epilog(in_purr);
> -
> - return index;
> -}
> -
> -static void check_and_cede_processor(void)
> -{
> - /*
> - * Ensure our interrupt state is properly tracked,
> - * also checks if no interrupt has occurred while we
> - * were soft-disabled
> - */
> - if (prep_irq_for_idle()) {
> - cede_processor();
> -#ifdef CONFIG_TRACE_IRQFLAGS
> - /* Ensure that H_CEDE returns with IRQs on */
> - if (WARN_ON(!(mfmsr() & MSR_EE)))
> - __hard_irq_enable();
> -#endif
> - }
> -}
> -
> -static int dedicated_cede_loop(struct cpuidle_device *dev,
> - struct cpuidle_driver *drv,
> - int index)
> -{
> - unsigned long in_purr;
> -
> - idle_loop_prolog(&in_purr);
> - get_lppaca()->donate_dedicated_cpu = 1;
> -
> - ppc64_runlatch_off();
> - HMT_medium();
> - check_and_cede_processor();
> -
> - get_lppaca()->donate_dedicated_cpu = 0;
> -
> - idle_loop_epilog(in_purr);
> -
> - return index;
> -}
> -
> -static int shared_cede_loop(struct cpuidle_device *dev,
> - struct cpuidle_driver *drv,
> - int index)
> -{
> - unsigned long in_purr;
> -
> - idle_loop_prolog(&in_purr);
> -
> - /*
> - * Yield the processor to the hypervisor. We return if
> - * an external interrupt occurs (which are driven prior
> - * to returning here) or if a prod occurs from another
> - * processor. When returning here, external interrupts
> - * are enabled.
> - */
> - check_and_cede_processor();
> -
> - idle_loop_epilog(in_purr);
> -
> - return index;
> -}
> -
> -/*
> - * States for dedicated partition case.
> - */
> -static struct cpuidle_state dedicated_states[MAX_IDLE_STATE_COUNT] = {
> - { /* Snooze */
> - .name = "snooze",
> - .desc = "snooze",
> - .flags = CPUIDLE_FLAG_TIME_VALID,
> - .exit_latency = 0,
> - .target_residency = 0,
> - .enter = &snooze_loop },
> - { /* CEDE */
> - .name = "CEDE",
> - .desc = "CEDE",
> - .flags = CPUIDLE_FLAG_TIME_VALID,
> - .exit_latency = 10,
> - .target_residency = 100,
> - .enter = &dedicated_cede_loop },
> -};
> -
> -/*
> - * States for shared partition case.
> - */
> -static struct cpuidle_state shared_states[MAX_IDLE_STATE_COUNT] = {
> - { /* Shared Cede */
> - .name = "Shared Cede",
> - .desc = "Shared Cede",
> - .flags = CPUIDLE_FLAG_TIME_VALID,
> - .exit_latency = 0,
> - .target_residency = 0,
> - .enter = &shared_cede_loop },
> -};
> -
> -void update_smt_snooze_delay(int cpu, int residency)
> -{
> - struct cpuidle_driver *drv = cpuidle_get_driver();
> - struct cpuidle_device *dev = per_cpu(cpuidle_devices, cpu);
> -
> - if (cpuidle_state_table != dedicated_states)
> - return;
> -
> - if (residency < 0) {
> - /* Disable the Nap state on that cpu */
> - if (dev)
> - dev->states_usage[1].disable = 1;
> - } else
> - if (drv)
> - drv->states[1].target_residency = residency;
> -}
> -
> -static int pseries_cpuidle_add_cpu_notifier(struct notifier_block *n,
> - unsigned long action, void *hcpu)
> -{
> - int hotcpu = (unsigned long)hcpu;
> - struct cpuidle_device *dev =
> - per_cpu_ptr(pseries_cpuidle_devices, hotcpu);
> -
> - if (dev && cpuidle_get_driver()) {
> - switch (action) {
> - case CPU_ONLINE:
> - case CPU_ONLINE_FROZEN:
> - cpuidle_pause_and_lock();
> - cpuidle_enable_device(dev);
> - cpuidle_resume_and_unlock();
> - break;
> -
> - case CPU_DEAD:
> - case CPU_DEAD_FROZEN:
> - cpuidle_pause_and_lock();
> - cpuidle_disable_device(dev);
> - cpuidle_resume_and_unlock();
> - break;
> -
> - default:
> - return NOTIFY_DONE;
> - }
> - }
> - return NOTIFY_OK;
> -}
> -
> -static struct notifier_block setup_hotplug_notifier = {
> - .notifier_call = pseries_cpuidle_add_cpu_notifier,
> -};
> -
> -/*
> - * pseries_cpuidle_driver_init()
> - */
> -static int pseries_cpuidle_driver_init(void)
> -{
> - int idle_state;
> - struct cpuidle_driver *drv = &pseries_idle_driver;
> -
> - drv->state_count = 0;
> -
> - for (idle_state = 0; idle_state < MAX_IDLE_STATE_COUNT; ++idle_state) {
> -
> - if (idle_state > max_idle_state)
> - break;
> -
> - /* is the state not enabled? */
> - if (cpuidle_state_table[idle_state].enter == NULL)
> - continue;
> -
> - drv->states[drv->state_count] = /* structure copy */
> - cpuidle_state_table[idle_state];
> -
> - drv->state_count += 1;
> - }
> -
> - return 0;
> -}
> -
> -/* pseries_idle_devices_uninit(void)
> - * unregister cpuidle devices and de-allocate memory
> - */
> -static void pseries_idle_devices_uninit(void)
> -{
> - int i;
> - struct cpuidle_device *dev;
> -
> - for_each_possible_cpu(i) {
> - dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> - cpuidle_unregister_device(dev);
> - }
> -
> - free_percpu(pseries_cpuidle_devices);
> - return;
> -}
> -
> -/* pseries_idle_devices_init()
> - * allocate, initialize and register cpuidle device
> - */
> -static int pseries_idle_devices_init(void)
> -{
> - int i;
> - struct cpuidle_driver *drv = &pseries_idle_driver;
> - struct cpuidle_device *dev;
> -
> - pseries_cpuidle_devices = alloc_percpu(struct cpuidle_device);
> - if (pseries_cpuidle_devices == NULL)
> - return -ENOMEM;
> -
> - for_each_possible_cpu(i) {
> - dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> - dev->state_count = drv->state_count;
> - dev->cpu = i;
> - if (cpuidle_register_device(dev)) {
> - printk(KERN_DEBUG \
> - "cpuidle_register_device %d failed!\n", i);
> - return -EIO;
> - }
> - }
> -
> - return 0;
> -}
> -
> -/*
> - * pseries_idle_probe()
> - * Choose state table for shared versus dedicated partition
> - */
> -static int pseries_idle_probe(void)
> -{
> -
> - if (!firmware_has_feature(FW_FEATURE_SPLPAR))
> - return -ENODEV;
> -
> - if (cpuidle_disable != IDLE_NO_OVERRIDE)
> - return -ENODEV;
> -
> - if (max_idle_state == 0) {
> - printk(KERN_DEBUG "pseries processor idle disabled.\n");
> - return -EPERM;
> - }
> -
> - if (lppaca_shared_proc(get_lppaca()))
> - cpuidle_state_table = shared_states;
> - else
> - cpuidle_state_table = dedicated_states;
> -
> - return 0;
> -}
> -
> -static int __init pseries_processor_idle_init(void)
> -{
> - int retval;
> -
> - retval = pseries_idle_probe();
> - if (retval)
> - return retval;
> -
> - pseries_cpuidle_driver_init();
> - retval = cpuidle_register_driver(&pseries_idle_driver);
> - if (retval) {
> - printk(KERN_DEBUG "Registration of pseries driver failed.\n");
> - return retval;
> - }
> -
> - retval = pseries_idle_devices_init();
> - if (retval) {
> - pseries_idle_devices_uninit();
> - cpuidle_unregister_driver(&pseries_idle_driver);
> - return retval;
> - }
> -
> - register_cpu_notifier(&setup_hotplug_notifier);
> - printk(KERN_DEBUG "pseries_idle_driver registered\n");
> -
> - return 0;
> -}
> -
> -static void __exit pseries_processor_idle_exit(void)
> -{
> -
> - unregister_cpu_notifier(&setup_hotplug_notifier);
> - pseries_idle_devices_uninit();
> - cpuidle_unregister_driver(&pseries_idle_driver);
> -
> - return;
> -}
> -
> -module_init(pseries_processor_idle_init);
> -module_exit(pseries_processor_idle_exit);
> -
> -MODULE_AUTHOR("Deepthi Dharwar <deepthi@linux.vnet.ibm.com>");
> -MODULE_DESCRIPTION("Cpuidle driver for POWER");
> -MODULE_LICENSE("GPL");
> diff --git a/drivers/cpuidle/Kconfig b/drivers/cpuidle/Kconfig
> index b3fb81d..f04e25f 100644
> --- a/drivers/cpuidle/Kconfig
> +++ b/drivers/cpuidle/Kconfig
> @@ -35,6 +35,11 @@ depends on ARM
> source "drivers/cpuidle/Kconfig.arm"
> endmenu
>
> +menu "POWERPC CPU Idle Drivers"
> +depends on PPC
> +source "drivers/cpuidle/Kconfig.powerpc"
> +endmenu
> +
> endif
>
> config ARCH_NEEDS_CPU_IDLE_COUPLED
> diff --git a/drivers/cpuidle/Kconfig.powerpc b/drivers/cpuidle/Kconfig.powerpc
> new file mode 100644
> index 0000000..8f43dc2
> --- /dev/null
> +++ b/drivers/cpuidle/Kconfig.powerpc
> @@ -0,0 +1,10 @@
> +#
> +# POWERPC CPU Idle Drivers
> +#
> +config CPU_IDLE_IBM_POWER
> + bool "CPU Idle driver for IBM POWER platforms"
> + depends on PPC_PSERIES
> + default y
> + help
> + Select this option to enable processor idle state management
> + on IBM POWER platform.
> diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
> index cea5ef5..60e7a6c 100644
> --- a/drivers/cpuidle/Makefile
> +++ b/drivers/cpuidle/Makefile
> @@ -12,3 +12,7 @@ obj-$(CONFIG_ARM_KIRKWOOD_CPUIDLE) += cpuidle-kirkwood.o
> obj-$(CONFIG_ARM_ZYNQ_CPUIDLE) += cpuidle-zynq.o
> obj-$(CONFIG_ARM_U8500_CPUIDLE) += cpuidle-ux500.o
> obj-$(CONFIG_CPU_IDLE_BIG_LITTLE) += cpuidle-big_little.o
> +
> +###############################################################################
> +# POWERPC drivers
> +obj-$(CONFIG_CPU_IDLE_IBM_POWER) += cpuidle-ibm-power.o
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> new file mode 100644
> index 0000000..a166e38
> --- /dev/null
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -0,0 +1,364 @@
> +/*
> + * processor_idle - idle state cpuidle driver.
> + * Adapted from drivers/idle/intel_idle.c and
> + * drivers/acpi/processor_idle.c
> + *
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/init.h>
> +#include <linux/moduleparam.h>
> +#include <linux/cpuidle.h>
> +#include <linux/cpu.h>
> +#include <linux/notifier.h>
> +
> +#include <asm/paca.h>
> +#include <asm/reg.h>
> +#include <asm/machdep.h>
> +#include <asm/firmware.h>
> +#include <asm/runlatch.h>
> +#include <asm/plpar_wrappers.h>
> +
> +struct cpuidle_driver pseries_idle_driver = {
> + .name = "pseries_idle",
> + .owner = THIS_MODULE,
> +};
> +
> +#define MAX_IDLE_STATE_COUNT 2
> +
> +static int max_idle_state = MAX_IDLE_STATE_COUNT - 1;
> +static struct cpuidle_device __percpu *pseries_cpuidle_devices;
> +static struct cpuidle_state *cpuidle_state_table;
> +
> +static inline void idle_loop_prolog(unsigned long *in_purr)
> +{
> + *in_purr = mfspr(SPRN_PURR);
> + /*
> + * Indicate to the HV that we are idle. Now would be
> + * a good time to find other work to dispatch.
> + */
> + get_lppaca()->idle = 1;
> +}
> +
> +static inline void idle_loop_epilog(unsigned long in_purr)
> +{
> + u64 wait_cycles;
> +
> + wait_cycles = be64_to_cpu(get_lppaca()->wait_state_cycles);
> + wait_cycles += mfspr(SPRN_PURR) - in_purr;
> + get_lppaca()->wait_state_cycles = cpu_to_be64(wait_cycles);
> + get_lppaca()->idle = 0;
> +}
> +
> +static int snooze_loop(struct cpuidle_device *dev,
> + struct cpuidle_driver *drv,
> + int index)
> +{
> + unsigned long in_purr;
> + int cpu = dev->cpu;
> +
> + idle_loop_prolog(&in_purr);
> + local_irq_enable();
> + set_thread_flag(TIF_POLLING_NRFLAG);
> +
> + while ((!need_resched()) && cpu_online(cpu)) {
> + ppc64_runlatch_off();
> + HMT_low();
> + HMT_very_low();
> + }
> +
> + HMT_medium();
> + clear_thread_flag(TIF_POLLING_NRFLAG);
> + smp_mb();
> +
> + idle_loop_epilog(in_purr);
> +
> + return index;
> +}
> +
> +static void check_and_cede_processor(void)
> +{
> + /*
> + * Ensure our interrupt state is properly tracked,
> + * also checks if no interrupt has occurred while we
> + * were soft-disabled
> + */
> + if (prep_irq_for_idle()) {
> + cede_processor();
> +#ifdef CONFIG_TRACE_IRQFLAGS
> + /* Ensure that H_CEDE returns with IRQs on */
> + if (WARN_ON(!(mfmsr() & MSR_EE)))
> + __hard_irq_enable();
> +#endif
> + }
> +}
> +
> +static int dedicated_cede_loop(struct cpuidle_device *dev,
> + struct cpuidle_driver *drv,
> + int index)
> +{
> + unsigned long in_purr;
> +
> + idle_loop_prolog(&in_purr);
> + get_lppaca()->donate_dedicated_cpu = 1;
> +
> + ppc64_runlatch_off();
> + HMT_medium();
> + check_and_cede_processor();
> +
> + get_lppaca()->donate_dedicated_cpu = 0;
> +
> + idle_loop_epilog(in_purr);
> +
> + return index;
> +}
> +
> +static int shared_cede_loop(struct cpuidle_device *dev,
> + struct cpuidle_driver *drv,
> + int index)
> +{
> + unsigned long in_purr;
> +
> + idle_loop_prolog(&in_purr);
> +
> + /*
> + * Yield the processor to the hypervisor. We return if
> + * an external interrupt occurs (which are driven prior
> + * to returning here) or if a prod occurs from another
> + * processor. When returning here, external interrupts
> + * are enabled.
> + */
> + check_and_cede_processor();
> +
> + idle_loop_epilog(in_purr);
> +
> + return index;
> +}
> +
> +/*
> + * States for dedicated partition case.
> + */
> +static struct cpuidle_state dedicated_states[MAX_IDLE_STATE_COUNT] = {
> + { /* Snooze */
> + .name = "snooze",
> + .desc = "snooze",
> + .flags = CPUIDLE_FLAG_TIME_VALID,
> + .exit_latency = 0,
> + .target_residency = 0,
> + .enter = &snooze_loop },
> + { /* CEDE */
> + .name = "CEDE",
> + .desc = "CEDE",
> + .flags = CPUIDLE_FLAG_TIME_VALID,
> + .exit_latency = 10,
> + .target_residency = 100,
> + .enter = &dedicated_cede_loop },
> +};
> +
> +/*
> + * States for shared partition case.
> + */
> +static struct cpuidle_state shared_states[MAX_IDLE_STATE_COUNT] = {
> + { /* Shared Cede */
> + .name = "Shared Cede",
> + .desc = "Shared Cede",
> + .flags = CPUIDLE_FLAG_TIME_VALID,
> + .exit_latency = 0,
> + .target_residency = 0,
> + .enter = &shared_cede_loop },
> +};
> +
> +void update_smt_snooze_delay(int cpu, int residency)
> +{
> + struct cpuidle_driver *drv = cpuidle_get_driver();
> + struct cpuidle_device *dev = per_cpu(cpuidle_devices, cpu);
> +
> + if (cpuidle_state_table != dedicated_states)
> + return;
> +
> + if (residency < 0) {
> + /* Disable the Nap state on that cpu */
> + if (dev)
> + dev->states_usage[1].disable = 1;
> + } else
> + if (drv)
> + drv->states[1].target_residency = residency;
> +}
> +
> +static int pseries_cpuidle_add_cpu_notifier(struct notifier_block *n,
> + unsigned long action, void *hcpu)
> +{
> + int hotcpu = (unsigned long)hcpu;
> + struct cpuidle_device *dev =
> + per_cpu_ptr(pseries_cpuidle_devices, hotcpu);
> +
> + if (dev && cpuidle_get_driver()) {
> + switch (action) {
> + case CPU_ONLINE:
> + case CPU_ONLINE_FROZEN:
> + cpuidle_pause_and_lock();
> + cpuidle_enable_device(dev);
> + cpuidle_resume_and_unlock();
> + break;
> +
> + case CPU_DEAD:
> + case CPU_DEAD_FROZEN:
> + cpuidle_pause_and_lock();
> + cpuidle_disable_device(dev);
> + cpuidle_resume_and_unlock();
> + break;
> +
> + default:
> + return NOTIFY_DONE;
> + }
> + }
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block setup_hotplug_notifier = {
> + .notifier_call = pseries_cpuidle_add_cpu_notifier,
> +};
> +
> +/*
> + * pseries_cpuidle_driver_init()
> + */
> +static int pseries_cpuidle_driver_init(void)
> +{
> + int idle_state;
> + struct cpuidle_driver *drv = &pseries_idle_driver;
> +
> + drv->state_count = 0;
> +
> + for (idle_state = 0; idle_state < MAX_IDLE_STATE_COUNT; ++idle_state) {
> +
> + if (idle_state > max_idle_state)
> + break;
> +
> + /* is the state not enabled? */
> + if (cpuidle_state_table[idle_state].enter == NULL)
> + continue;
> +
> + drv->states[drv->state_count] = /* structure copy */
> + cpuidle_state_table[idle_state];
> +
> + drv->state_count += 1;
> + }
> +
> + return 0;
> +}
> +
> +/* pseries_idle_devices_uninit(void)
> + * unregister cpuidle devices and de-allocate memory
> + */
> +static void pseries_idle_devices_uninit(void)
> +{
> + int i;
> + struct cpuidle_device *dev;
> +
> + for_each_possible_cpu(i) {
> + dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> + cpuidle_unregister_device(dev);
> + }
> +
> + free_percpu(pseries_cpuidle_devices);
> + return;
> +}
> +
> +/* pseries_idle_devices_init()
> + * allocate, initialize and register cpuidle device
> + */
> +static int pseries_idle_devices_init(void)
> +{
> + int i;
> + struct cpuidle_driver *drv = &pseries_idle_driver;
> + struct cpuidle_device *dev;
> +
> + pseries_cpuidle_devices = alloc_percpu(struct cpuidle_device);
> + if (pseries_cpuidle_devices == NULL)
> + return -ENOMEM;
> +
> + for_each_possible_cpu(i) {
> + dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> + dev->state_count = drv->state_count;
> + dev->cpu = i;
> + if (cpuidle_register_device(dev)) {
> + printk(KERN_DEBUG \
> + "cpuidle_register_device %d failed!\n", i);
> + return -EIO;
> + }
> + }
> +
> + return 0;
> +}
> +
> +/*
> + * pseries_idle_probe()
> + * Choose state table for shared versus dedicated partition
> + */
> +static int pseries_idle_probe(void)
> +{
> +
> + if (!firmware_has_feature(FW_FEATURE_SPLPAR))
> + return -ENODEV;
> +
> + if (cpuidle_disable != IDLE_NO_OVERRIDE)
> + return -ENODEV;
> +
> + if (max_idle_state == 0) {
> + printk(KERN_DEBUG "pseries processor idle disabled.\n");
> + return -EPERM;
> + }
> +
> + if (lppaca_shared_proc(get_lppaca()))
> + cpuidle_state_table = shared_states;
> + else
> + cpuidle_state_table = dedicated_states;
> +
> + return 0;
> +}
> +
> +static int __init pseries_processor_idle_init(void)
> +{
> + int retval;
> +
> + retval = pseries_idle_probe();
> + if (retval)
> + return retval;
> +
> + pseries_cpuidle_driver_init();
> + retval = cpuidle_register_driver(&pseries_idle_driver);
> + if (retval) {
> + printk(KERN_DEBUG "Registration of pseries driver failed.\n");
> + return retval;
> + }
> +
> + retval = pseries_idle_devices_init();
> + if (retval) {
> + pseries_idle_devices_uninit();
> + cpuidle_unregister_driver(&pseries_idle_driver);
> + return retval;
> + }
> +
> + register_cpu_notifier(&setup_hotplug_notifier);
> + printk(KERN_DEBUG "pseries_idle_driver registered\n");
> +
> + return 0;
> +}
> +
> +static void __exit pseries_processor_idle_exit(void)
> +{
> +
> + unregister_cpu_notifier(&setup_hotplug_notifier);
> + pseries_idle_devices_uninit();
> + cpuidle_unregister_driver(&pseries_idle_driver);
> +
> + return;
> +}
> +
> +module_init(pseries_processor_idle_init);
> +module_exit(pseries_processor_idle_exit);
> +
> +MODULE_AUTHOR("Deepthi Dharwar <deepthi@linux.vnet.ibm.com>");
> +MODULE_DESCRIPTION("Cpuidle driver for POWER");
> +MODULE_LICENSE("GPL");
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 2/7] pseries/cpuidle: Use cpuidle_register() for initialisation.
From: Daniel Lezcano @ 2013-11-06 21:00 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110127.31547.9773.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> This patch replaces the cpuidle driver and devices initialisation
> calls with a single generic cpuidle_register() call
> and also includes minor refactoring of the code around it.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> drivers/cpuidle/cpuidle-ibm-power.c | 80 +++++------------------------------
> 1 file changed, 12 insertions(+), 68 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> index a166e38..faf8cc1 100644
> --- a/drivers/cpuidle/cpuidle-ibm-power.c
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -1,5 +1,5 @@
> /*
> - * processor_idle - idle state cpuidle driver.
> + * cpuidle-ibm-power - idle state cpuidle driver.
> * Adapted from drivers/idle/intel_idle.c and
> * drivers/acpi/processor_idle.c
> *
> @@ -28,7 +28,6 @@ struct cpuidle_driver pseries_idle_driver = {
> #define MAX_IDLE_STATE_COUNT 2
>
> static int max_idle_state = MAX_IDLE_STATE_COUNT - 1;
> -static struct cpuidle_device __percpu *pseries_cpuidle_devices;
> static struct cpuidle_state *cpuidle_state_table;
>
> static inline void idle_loop_prolog(unsigned long *in_purr)
> @@ -56,13 +55,12 @@ static int snooze_loop(struct cpuidle_device *dev,
> int index)
> {
> unsigned long in_purr;
> - int cpu = dev->cpu;
>
> idle_loop_prolog(&in_purr);
> local_irq_enable();
> set_thread_flag(TIF_POLLING_NRFLAG);
>
> - while ((!need_resched()) && cpu_online(cpu)) {
> + while (!need_resched()) {
> ppc64_runlatch_off();
> HMT_low();
> HMT_very_low();
> @@ -191,7 +189,7 @@ static int pseries_cpuidle_add_cpu_notifier(struct notifier_block *n,
> {
> int hotcpu = (unsigned long)hcpu;
> struct cpuidle_device *dev =
> - per_cpu_ptr(pseries_cpuidle_devices, hotcpu);
> + per_cpu(cpuidle_devices, hotcpu);
>
> if (dev && cpuidle_get_driver()) {
> switch (action) {
> @@ -248,50 +246,6 @@ static int pseries_cpuidle_driver_init(void)
> return 0;
> }
>
> -/* pseries_idle_devices_uninit(void)
> - * unregister cpuidle devices and de-allocate memory
> - */
> -static void pseries_idle_devices_uninit(void)
> -{
> - int i;
> - struct cpuidle_device *dev;
> -
> - for_each_possible_cpu(i) {
> - dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> - cpuidle_unregister_device(dev);
> - }
> -
> - free_percpu(pseries_cpuidle_devices);
> - return;
> -}
> -
> -/* pseries_idle_devices_init()
> - * allocate, initialize and register cpuidle device
> - */
> -static int pseries_idle_devices_init(void)
> -{
> - int i;
> - struct cpuidle_driver *drv = &pseries_idle_driver;
> - struct cpuidle_device *dev;
> -
> - pseries_cpuidle_devices = alloc_percpu(struct cpuidle_device);
> - if (pseries_cpuidle_devices == NULL)
> - return -ENOMEM;
> -
> - for_each_possible_cpu(i) {
> - dev = per_cpu_ptr(pseries_cpuidle_devices, i);
> - dev->state_count = drv->state_count;
> - dev->cpu = i;
> - if (cpuidle_register_device(dev)) {
> - printk(KERN_DEBUG \
> - "cpuidle_register_device %d failed!\n", i);
> - return -EIO;
> - }
> - }
> -
> - return 0;
> -}
> -
> /*
> * pseries_idle_probe()
> * Choose state table for shared versus dedicated partition
> @@ -299,9 +253,6 @@ static int pseries_idle_devices_init(void)
> static int pseries_idle_probe(void)
> {
>
> - if (!firmware_has_feature(FW_FEATURE_SPLPAR))
> - return -ENODEV;
> -
> if (cpuidle_disable != IDLE_NO_OVERRIDE)
> return -ENODEV;
>
> @@ -310,10 +261,13 @@ static int pseries_idle_probe(void)
> return -EPERM;
> }
>
> - if (lppaca_shared_proc(get_lppaca()))
> - cpuidle_state_table = shared_states;
> - else
> - cpuidle_state_table = dedicated_states;
> + if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
> + if (lppaca_shared_proc(get_lppaca()))
> + cpuidle_state_table = shared_states;
> + else
> + cpuidle_state_table = dedicated_states;
> + } else
> + return -ENODEV;
>
> return 0;
> }
> @@ -327,22 +281,14 @@ static int __init pseries_processor_idle_init(void)
> return retval;
>
> pseries_cpuidle_driver_init();
> - retval = cpuidle_register_driver(&pseries_idle_driver);
> + retval = cpuidle_register(&pseries_idle_driver, NULL);
> if (retval) {
> printk(KERN_DEBUG "Registration of pseries driver failed.\n");
> return retval;
> }
>
> - retval = pseries_idle_devices_init();
> - if (retval) {
> - pseries_idle_devices_uninit();
> - cpuidle_unregister_driver(&pseries_idle_driver);
> - return retval;
> - }
> -
> register_cpu_notifier(&setup_hotplug_notifier);
> printk(KERN_DEBUG "pseries_idle_driver registered\n");
> -
> return 0;
> }
>
> @@ -350,9 +296,7 @@ static void __exit pseries_processor_idle_exit(void)
> {
>
> unregister_cpu_notifier(&setup_hotplug_notifier);
> - pseries_idle_devices_uninit();
> - cpuidle_unregister_driver(&pseries_idle_driver);
> -
> + cpuidle_unregister(&pseries_idle_driver);
> return;
> }
>
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 3/7] pseries/cpuidle: Make pseries_idle backend driver a non-module.
From: Daniel Lezcano @ 2013-11-06 21:01 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110134.31547.86550.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> Currently pseries_idle cpuidle backend driver cannot be
> built as a module due to dependencies. Therefore the driver has
> to be built in. The dependency is around update_snooze_delay() defined
> in cpuidle driver and called from kernel/sysfs.c.
> This patch is removes all the module related code.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> drivers/cpuidle/cpuidle-ibm-power.c | 15 +--------------
> 1 file changed, 1 insertion(+), 14 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> index faf8cc1..8c9e42c 100644
> --- a/drivers/cpuidle/cpuidle-ibm-power.c
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -292,17 +292,4 @@ static int __init pseries_processor_idle_init(void)
> return 0;
> }
>
> -static void __exit pseries_processor_idle_exit(void)
> -{
> -
> - unregister_cpu_notifier(&setup_hotplug_notifier);
> - cpuidle_unregister(&pseries_idle_driver);
> - return;
> -}
> -
> -module_init(pseries_processor_idle_init);
> -module_exit(pseries_processor_idle_exit);
> -
> -MODULE_AUTHOR("Deepthi Dharwar <deepthi@linux.vnet.ibm.com>");
> -MODULE_DESCRIPTION("Cpuidle driver for POWER");
> -MODULE_LICENSE("GPL");
> +device_initcall(pseries_processor_idle_init);
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 4/7] pseries/cpuidle: Remove MAX_IDLE_STATE macro.
From: Daniel Lezcano @ 2013-11-06 21:02 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110141.31547.23622.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> This patch removes the usage of MAX_IDLE_STATE macro
> and dead code around it. The number of states
> are determined at run time based on the cpuidle
> state table selected on a given platform
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> drivers/cpuidle/cpuidle-ibm-power.c | 26 +++++++++-----------------
> 1 file changed, 9 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> index 8c9e42c..e81c207 100644
> --- a/drivers/cpuidle/cpuidle-ibm-power.c
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -25,9 +25,7 @@ struct cpuidle_driver pseries_idle_driver = {
> .owner = THIS_MODULE,
> };
>
> -#define MAX_IDLE_STATE_COUNT 2
> -
> -static int max_idle_state = MAX_IDLE_STATE_COUNT - 1;
> +static int max_idle_state;
> static struct cpuidle_state *cpuidle_state_table;
>
> static inline void idle_loop_prolog(unsigned long *in_purr)
> @@ -137,7 +135,7 @@ static int shared_cede_loop(struct cpuidle_device *dev,
> /*
> * States for dedicated partition case.
> */
> -static struct cpuidle_state dedicated_states[MAX_IDLE_STATE_COUNT] = {
> +static struct cpuidle_state dedicated_states[] = {
> { /* Snooze */
> .name = "snooze",
> .desc = "snooze",
> @@ -157,7 +155,7 @@ static struct cpuidle_state dedicated_states[MAX_IDLE_STATE_COUNT] = {
> /*
> * States for shared partition case.
> */
> -static struct cpuidle_state shared_states[MAX_IDLE_STATE_COUNT] = {
> +static struct cpuidle_state shared_states[] = {
> { /* Shared Cede */
> .name = "Shared Cede",
> .desc = "Shared Cede",
> @@ -227,11 +225,7 @@ static int pseries_cpuidle_driver_init(void)
> struct cpuidle_driver *drv = &pseries_idle_driver;
>
> drv->state_count = 0;
> -
> - for (idle_state = 0; idle_state < MAX_IDLE_STATE_COUNT; ++idle_state) {
> -
> - if (idle_state > max_idle_state)
> - break;
> + for (idle_state = 0; idle_state < max_idle_state; ++idle_state) {
>
> /* is the state not enabled? */
> if (cpuidle_state_table[idle_state].enter == NULL)
> @@ -256,16 +250,14 @@ static int pseries_idle_probe(void)
> if (cpuidle_disable != IDLE_NO_OVERRIDE)
> return -ENODEV;
>
> - if (max_idle_state == 0) {
> - printk(KERN_DEBUG "pseries processor idle disabled.\n");
> - return -EPERM;
> - }
> -
> if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
> - if (lppaca_shared_proc(get_lppaca()))
> + if (lppaca_shared_proc(get_lppaca())) {
> cpuidle_state_table = shared_states;
> - else
> + max_idle_state = ARRAY_SIZE(shared_states);
> + } else {
> cpuidle_state_table = dedicated_states;
> + max_idle_state = ARRAY_SIZE(dedicated_states);
> + }
> } else
> return -ENODEV;
>
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 5/7] POWER/cpuidle: Generic POWER CPUIDLE driver supporting PSERIES.
From: Daniel Lezcano @ 2013-11-06 21:05 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110148.31547.8944.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> This patch includes cleanup and refactoring of the
> existing code to make the driver POWER generic.
> * Re-naming the functions from pseries to generic power.
> * Re-naming the backend driver from pseries_idle to
> ibm-power-idle.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
... but I am wondering if 'power' name is not confusing with power
management and if 'powerpc' would suit better.
> ---
> drivers/cpuidle/cpuidle-ibm-power.c | 32 ++++++++++++++++----------------
> 1 file changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> index e81c207..5b92242 100644
> --- a/drivers/cpuidle/cpuidle-ibm-power.c
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -20,8 +20,8 @@
> #include <asm/runlatch.h>
> #include <asm/plpar_wrappers.h>
>
> -struct cpuidle_driver pseries_idle_driver = {
> - .name = "pseries_idle",
> +struct cpuidle_driver power_idle_driver = {
> + .name = "ibm_power_idle",
> .owner = THIS_MODULE,
> };
>
> @@ -182,7 +182,7 @@ void update_smt_snooze_delay(int cpu, int residency)
> drv->states[1].target_residency = residency;
> }
>
> -static int pseries_cpuidle_add_cpu_notifier(struct notifier_block *n,
> +static int power_cpuidle_add_cpu_notifier(struct notifier_block *n,
> unsigned long action, void *hcpu)
> {
> int hotcpu = (unsigned long)hcpu;
> @@ -213,16 +213,16 @@ static int pseries_cpuidle_add_cpu_notifier(struct notifier_block *n,
> }
>
> static struct notifier_block setup_hotplug_notifier = {
> - .notifier_call = pseries_cpuidle_add_cpu_notifier,
> + .notifier_call = power_cpuidle_add_cpu_notifier,
> };
>
> /*
> - * pseries_cpuidle_driver_init()
> + * power_cpuidle_driver_init()
> */
> -static int pseries_cpuidle_driver_init(void)
> +static int power_cpuidle_driver_init(void)
> {
> int idle_state;
> - struct cpuidle_driver *drv = &pseries_idle_driver;
> + struct cpuidle_driver *drv = &power_idle_driver;
>
> drv->state_count = 0;
> for (idle_state = 0; idle_state < max_idle_state; ++idle_state) {
> @@ -241,10 +241,10 @@ static int pseries_cpuidle_driver_init(void)
> }
>
> /*
> - * pseries_idle_probe()
> + * power_idle_probe()
> * Choose state table for shared versus dedicated partition
> */
> -static int pseries_idle_probe(void)
> +static int power_idle_probe(void)
> {
>
> if (cpuidle_disable != IDLE_NO_OVERRIDE)
> @@ -264,24 +264,24 @@ static int pseries_idle_probe(void)
> return 0;
> }
>
> -static int __init pseries_processor_idle_init(void)
> +static int __init power_processor_idle_init(void)
> {
> int retval;
>
> - retval = pseries_idle_probe();
> + retval = power_idle_probe();
> if (retval)
> return retval;
>
> - pseries_cpuidle_driver_init();
> - retval = cpuidle_register(&pseries_idle_driver, NULL);
> + power_cpuidle_driver_init();
> + retval = cpuidle_register(&power_idle_driver, NULL);
> if (retval) {
> - printk(KERN_DEBUG "Registration of pseries driver failed.\n");
> + printk(KERN_DEBUG "Registration of ibm_power_idle driver failed.\n");
> return retval;
> }
>
> register_cpu_notifier(&setup_hotplug_notifier);
> - printk(KERN_DEBUG "pseries_idle_driver registered\n");
> + printk(KERN_DEBUG "ibm_power_idle registered\n");
> return 0;
> }
>
> -device_initcall(pseries_processor_idle_init);
> +device_initcall(power_processor_idle_init);
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 6/7] POWER/cpuidle: Enable powernv cpuidle support.
From: Daniel Lezcano @ 2013-11-06 21:06 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110154.31547.94129.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:01 PM, Deepthi Dharwar wrote:
> The following patch extends the current power backend
> idle driver to the powernv platform.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> drivers/cpuidle/cpuidle-ibm-power.c | 39 ++++++++++++++++++++++++++++++++---
> 1 file changed, 36 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-ibm-power.c b/drivers/cpuidle/cpuidle-ibm-power.c
> index 5b92242..f790ea2 100644
> --- a/drivers/cpuidle/cpuidle-ibm-power.c
> +++ b/drivers/cpuidle/cpuidle-ibm-power.c
> @@ -52,9 +52,10 @@ static int snooze_loop(struct cpuidle_device *dev,
> struct cpuidle_driver *drv,
> int index)
> {
> - unsigned long in_purr;
> + unsigned long in_purr = 0;
>
> - idle_loop_prolog(&in_purr);
> + if (firmware_has_feature(FW_FEATURE_SPLPAR))
> + idle_loop_prolog(&in_purr);
> local_irq_enable();
> set_thread_flag(TIF_POLLING_NRFLAG);
>
> @@ -68,7 +69,8 @@ static int snooze_loop(struct cpuidle_device *dev,
> clear_thread_flag(TIF_POLLING_NRFLAG);
> smp_mb();
>
> - idle_loop_epilog(in_purr);
> + if (firmware_has_feature(FW_FEATURE_SPLPAR))
> + idle_loop_epilog(in_purr);
>
> return index;
> }
> @@ -132,6 +134,15 @@ static int shared_cede_loop(struct cpuidle_device *dev,
> return index;
> }
>
> +static int nap_loop(struct cpuidle_device *dev,
> + struct cpuidle_driver *drv,
> + int index)
> +{
> + ppc64_runlatch_off();
> + power7_idle();
> + return index;
> +}
> +
> /*
> * States for dedicated partition case.
> */
> @@ -165,6 +176,23 @@ static struct cpuidle_state shared_states[] = {
> .enter = &shared_cede_loop },
> };
>
> +static struct cpuidle_state powernv_states[] = {
> + { /* Snooze */
> + .name = "snooze",
> + .desc = "snooze",
> + .flags = CPUIDLE_FLAG_TIME_VALID,
> + .exit_latency = 0,
> + .target_residency = 0,
> + .enter = &snooze_loop },
> + { /* NAP */
> + .name = "NAP",
> + .desc = "NAP",
> + .flags = CPUIDLE_FLAG_TIME_VALID,
> + .exit_latency = 10,
> + .target_residency = 100,
> + .enter = &nap_loop },
> +};
> +
> void update_smt_snooze_delay(int cpu, int residency)
> {
> struct cpuidle_driver *drv = cpuidle_get_driver();
> @@ -258,6 +286,11 @@ static int power_idle_probe(void)
> cpuidle_state_table = dedicated_states;
> max_idle_state = ARRAY_SIZE(dedicated_states);
> }
> +
> + } else if (firmware_has_feature(FW_FEATURE_OPALv3)) {
> + cpuidle_state_table = powernv_states;
> + max_idle_state = ARRAY_SIZE(powernv_states);
> +
> } else
> return -ENODEV;
>
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH V7 7/7] powernv/cpuidle: Enable idle powernv cpu to call into the cpuidle framework.
From: Daniel Lezcano @ 2013-11-06 21:11 UTC (permalink / raw)
To: Deepthi Dharwar, benh, linuxppc-dev, linux-kernel
Cc: preeti, srivatsa.bhat, scottwood, linux-pm
In-Reply-To: <20131029110201.31547.55088.stgit@deepthi.in.ibm.com>
On 10/29/2013 12:02 PM, Deepthi Dharwar wrote:
> This patch enables idle cpu on the powernv platform to hook on to the cpuidle
> framework, if available, else call on to default idle platform
> code.
>
> Signed-off-by: Deepthi Dharwar <deepthi@linux.vnet.ibm.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> arch/powerpc/platforms/powernv/setup.c | 13 ++++++++++++-
> 1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
> index e239dcf..42a6ba0 100644
> --- a/arch/powerpc/platforms/powernv/setup.c
> +++ b/arch/powerpc/platforms/powernv/setup.c
> @@ -25,6 +25,7 @@
> #include <linux/of.h>
> #include <linux/interrupt.h>
> #include <linux/bug.h>
> +#include <linux/cpuidle.h>
>
> #include <asm/machdep.h>
> #include <asm/firmware.h>
> @@ -192,6 +193,16 @@ static void __init pnv_setup_machdep_rtas(void)
> }
> #endif /* CONFIG_PPC_POWERNV_RTAS */
>
> +void powernv_idle(void)
> +{
> + /* Hook to cpuidle framework if available, else
> + * call on default platform idle code
> + */
> + if (cpuidle_idle_call()) {
> + power7_idle();
> + }
> +}
> +
> static int __init pnv_probe(void)
> {
> unsigned long root = of_get_flat_dt_root();
> @@ -222,7 +233,7 @@ define_machine(powernv) {
> .show_cpuinfo = pnv_show_cpuinfo,
> .progress = pnv_progress,
> .machine_shutdown = pnv_shutdown,
> - .power_save = power7_idle,
> + .power_save = powernv_idle,
> .calibrate_decr = generic_calibrate_decr,
> #ifdef CONFIG_KEXEC
> .kexec_cpu_down = pnv_kexec_cpu_down,
>
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply
* Re: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20 state and altivec idle
From: Scott Wood @ 2013-11-07 1:20 UTC (permalink / raw)
To: Wang Dongsheng-B40534
Cc: Wood Scott-B07421, linuxppc-dev@lists.ozlabs.org,
Bhushan Bharat-R65777
In-Reply-To: <ABB05CD9C9F68C46A5CEDC7F15439259010BD119@039-SN2MPN1-021.039d.mgd.msft.net>
On Wed, 2013-11-06 at 01:50 -0600, Wang Dongsheng-B40534 wrote:
>
> > -----Original Message-----
> > From: Bhushan Bharat-R65777
> > Sent: Wednesday, November 06, 2013 1:25 PM
> > To: Wang Dongsheng-B40534; Wood Scott-B07421
> > Cc: linuxppc-dev@lists.ozlabs.org
> > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20 state and
> > altivec idle
> >
> >
> >
> > > -----Original Message-----
> > > From: Wang Dongsheng-B40534
> > > Sent: Tuesday, November 05, 2013 8:40 AM
> > > To: Wood Scott-B07421
> > > Cc: Bhushan Bharat-R65777; linuxppc-dev@lists.ozlabs.org
> > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20 state and
> > > altivec idle
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Wood Scott-B07421
> > > > Sent: Tuesday, November 05, 2013 5:52 AM
> > > > To: Wang Dongsheng-B40534
> > > > Cc: Wood Scott-B07421; Bhushan Bharat-R65777; linuxppc-
> > > > dev@lists.ozlabs.org
> > > > Subject: Re: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20 state
> > > > and altivec idle
> > > >
> > > > On Sun, 2013-11-03 at 22:04 -0600, Wang Dongsheng-B40534 wrote:
> > > > > > -----Original Message-----
> > > > > > From: Wang Dongsheng-B40534
> > > > > > Sent: Monday, October 21, 2013 11:11 AM
> > > > > > To: Wood Scott-B07421
> > > > > > Cc: Bhushan Bharat-R65777; linuxppc-dev@lists.ozlabs.org
> > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20
> > > > > > state and altivec idle
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Wood Scott-B07421
> > > > > > > Sent: Saturday, October 19, 2013 3:22 AM
> > > > > > > To: Wang Dongsheng-B40534
> > > > > > > Cc: Bhushan Bharat-R65777; Wood Scott-B07421; linuxppc-
> > > > > > > dev@lists.ozlabs.org
> > > > > > > Subject: Re: [PATCH v5 4/4] powerpc/85xx: add sysfs for pw20
> > > > > > > state and altivec idle
> > > > > > >
> > > > > > > On Thu, 2013-10-17 at 22:02 -0500, Wang Dongsheng-B40534 wrote:
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Bhushan Bharat-R65777
> > > > > > > > > Sent: Thursday, October 17, 2013 2:46 PM
> > > > > > > > > To: Wang Dongsheng-B40534; Wood Scott-B07421
> > > > > > > > > Cc: linuxppc-dev@lists.ozlabs.org
> > > > > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add sysfs for
> > > > > > > > > pw20 state and altivec idle
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > From: Wang Dongsheng-B40534
> > > > > > > > > > > > Sent: Thursday, October 17, 2013 11:22 AM
> > > > > > > > > > > > To: Bhushan Bharat-R65777; Wood Scott-B07421
> > > > > > > > > > > > Cc: linuxppc-dev@lists.ozlabs.org
> > > > > > > > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add sysfs
> > > > > > > > > > > > for
> > > > > > > > > > > > pw20 state and altivec idle
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > From: Bhushan Bharat-R65777
> > > > > > > > > > > > > Sent: Thursday, October 17, 2013 11:20 AM
> > > > > > > > > > > > > To: Wang Dongsheng-B40534; Wood Scott-B07421
> > > > > > > > > > > > > Cc: linuxppc-dev@lists.ozlabs.org
> > > > > > > > > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add
> > > > > > > > > > > > > sysfs for
> > > > > > > > > > > > > pw20 state and altivec idle
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: Wang Dongsheng-B40534
> > > > > > > > > > > > > > Sent: Thursday, October 17, 2013 8:16 AM
> > > > > > > > > > > > > > To: Bhushan Bharat-R65777; Wood Scott-B07421
> > > > > > > > > > > > > > Cc: linuxppc-dev@lists.ozlabs.org
> > > > > > > > > > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add
> > > > > > > > > > > > > > sysfs for
> > > > > > > > > > > > > > pw20 state and altivec idle
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > > From: Bhushan Bharat-R65777
> > > > > > > > > > > > > > > Sent: Thursday, October 17, 2013 1:01 AM
> > > > > > > > > > > > > > > To: Wang Dongsheng-B40534; Wood Scott-B07421
> > > > > > > > > > > > > > > Cc: linuxppc-dev@lists.ozlabs.org
> > > > > > > > > > > > > > > Subject: RE: [PATCH v5 4/4] powerpc/85xx: add
> > > > > > > > > > > > > > > sysfs for
> > > > > > > > > > > > > > > pw20 state and altivec idle
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > > > From: Wang Dongsheng-B40534
> > > > > > > > > > > > > > > > Sent: Tuesday, October 15, 2013 2:51 PM
> > > > > > > > > > > > > > > > To: Wood Scott-B07421
> > > > > > > > > > > > > > > > Cc: Bhushan Bharat-R65777;
> > > > > > > > > > > > > > > > linuxppc-dev@lists.ozlabs.org; Wang
> > > > > > > > > > > > > > > Dongsheng-B40534
> > > > > > > > > > > > > > > > Subject: [PATCH v5 4/4] powerpc/85xx: add
> > > > > > > > > > > > > > > > sysfs for
> > > > > > > > > > > > > > > > pw20 state and
> > > > > > > > > > > > > > > altivec idle
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > From: Wang Dongsheng
> > > > > > > > > > > > > > > > <dongsheng.wang@freescale.com>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Add a sys interface to enable/diable pw20
> > > > > > > > > > > > > > > > state or altivec idle, and
> > > > > > > > > > > > > > > control the
> > > > > > > > > > > > > > > > wait entry time.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Enable/Disable interface:
> > > > > > > > > > > > > > > > 0, disable. 1, enable.
> > > > > > > > > > > > > > > > /sys/devices/system/cpu/cpuX/pw20_state
> > > > > > > > > > > > > > > > /sys/devices/system/cpu/cpuX/altivec_idle
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Set wait time interface:(Nanosecond)
> > > > > > > > > > > > > > > > /sys/devices/system/cpu/cpuX/pw20_wait_time
> > > > > > > > > > > > > > > > /sys/devices/system/cpu/cpuX/altivec_idle_wa
> > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > _t
> > > > > > > > > > > > > > > > ime
> > > > > > > > > > > > > > > > Example: Base on TBfreq is 41MHZ.
> > > > > > > > > > > > > > > > 1~48(ns): TB[63]
> > > > > > > > > > > > > > > > 49~97(ns): TB[62]
> > > > > > > > > > > > > > > > 98~195(ns): TB[61]
> > > > > > > > > > > > > > > > 196~390(ns): TB[60]
> > > > > > > > > > > > > > > > 391~780(ns): TB[59]
> > > > > > > > > > > > > > > > 781~1560(ns): TB[58] ...
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Signed-off-by: Wang Dongsheng
> > > > > > > > > > > > > > > > <dongsheng.wang@freescale.com>
> > > > > > > > > > > > > > > > ---
> > > > > > > > > > > > > > > > *v5:
> > > > > > > > > > > > > > > > Change get_idle_ticks_bit function
> > implementation.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *v4:
> > > > > > > > > > > > > > > > Move code from 85xx/common.c to
> > kernel/sysfs.c.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Remove has_pw20_altivec_idle function.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Change wait "entry_bit" to wait time.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > diff --git a/arch/powerpc/kernel/sysfs.c
> > > > > > > > > > > > > > > > b/arch/powerpc/kernel/sysfs.c
> > > > > > > > > > > > > > > index
> > > > > > > > > > > > > > > > 27a90b9..10d1128 100644
> > > > > > > > > > > > > > > > --- a/arch/powerpc/kernel/sysfs.c
> > > > > > > > > > > > > > > > +++ b/arch/powerpc/kernel/sysfs.c
> > > > > > > > > > > > > > > > @@ -85,6 +85,284 @@
> > > > > > > > > > > > > > > > __setup("smt-snooze-delay=",
> > > > > > > > > > > > > > > setup_smt_snooze_delay);
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > #endif /* CONFIG_PPC64 */
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > +#ifdef CONFIG_FSL_SOC
> > > > > > > > > > > > > > > > +#define MAX_BIT 63
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static u64 pw20_wt; static u64
> > > > > > > > > > > > > > > > +altivec_idle_wt;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static unsigned int get_idle_ticks_bit(u64
> > ns) {
> > > > > > > > > > > > > > > > + u64 cycle;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (ns >= 10000)
> > > > > > > > > > > > > > > > + cycle = div_u64(ns + 500, 1000) *
> > > > > > > > > tb_ticks_per_usec;
> > > > > > > > > > > > > > > > + else
> > > > > > > > > > > > > > > > + cycle = div_u64(ns *
> > > > tb_ticks_per_usec,
> > > > > > > 1000);
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (!cycle)
> > > > > > > > > > > > > > > > + return 0;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + return ilog2(cycle); }
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static void do_show_pwrmgtcr0(void *val) {
> > > > > > > > > > > > > > > > + u32 *value = val;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + *value = mfspr(SPRN_PWRMGTCR0); }
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static ssize_t show_pw20_state(struct
> > > > > > > > > > > > > > > > +device
> > > > *dev,
> > > > > > > > > > > > > > > > + struct device_attribute
> > > > *attr,
> > > > > > > char
> > > > > > > > > *buf) {
> > > > > > > > > > > > > > > > + u32 value;
> > > > > > > > > > > > > > > > + unsigned int cpu = dev->id;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + smp_call_function_single(cpu,
> > > > > > > > > > > > > > > > +do_show_pwrmgtcr0, &value, 1);
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + value &= PWRMGTCR0_PW20_WAIT;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + return sprintf(buf, "%u\n", value ? 1 :
> > > > 0); }
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static void do_store_pw20_state(void *val) {
> > > > > > > > > > > > > > > > + u32 *value = val;
> > > > > > > > > > > > > > > > + u32 pw20_state;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + pw20_state = mfspr(SPRN_PWRMGTCR0);
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (*value)
> > > > > > > > > > > > > > > > + pw20_state |= PWRMGTCR0_PW20_WAIT;
> > > > > > > > > > > > > > > > + else
> > > > > > > > > > > > > > > > + pw20_state &= ~PWRMGTCR0_PW20_WAIT;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + mtspr(SPRN_PWRMGTCR0, pw20_state); }
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static ssize_t store_pw20_state(struct
> > > > > > > > > > > > > > > > +device
> > > > *dev,
> > > > > > > > > > > > > > > > + struct device_attribute
> > > > *attr,
> > > > > > > > > > > > > > > > + const char *buf, size_t
> > > > count)
> > > > > > > {
> > > > > > > > > > > > > > > > + u32 value;
> > > > > > > > > > > > > > > > + unsigned int cpu = dev->id;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (kstrtou32(buf, 0, &value))
> > > > > > > > > > > > > > > > + return -EINVAL;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (value > 1)
> > > > > > > > > > > > > > > > + return -EINVAL;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + smp_call_function_single(cpu,
> > > > > > > > > > > > > > > > +do_store_pw20_state, &value, 1);
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + return count; }
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > +static ssize_t show_pw20_wait_time(struct
> > > > > > > > > > > > > > > > +device
> > > > > > *dev,
> > > > > > > > > > > > > > > > + struct device_attribute
> > > > *attr,
> > > > > > > char
> > > > > > > > > *buf) {
> > > > > > > > > > > > > > > > + u32 value;
> > > > > > > > > > > > > > > > + u64 tb_cycle;
> > > > > > > > > > > > > > > > + s64 time;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + unsigned int cpu = dev->id;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + if (!pw20_wt) {
> > > > > > > > > > > > > > > > + smp_call_function_single(cpu,
> > > > > > > do_show_pwrmgtcr0,
> > > > > > > > > > > > > > > > +&value,
> > > > > > > > > > > > > 1);
> > > > > > > > > > > > > > > > + value = (value &
> > > > PWRMGTCR0_PW20_ENT) >>
> > > > > > > > > > > > > > > > +
> > > > PWRMGTCR0_PW20_ENT_SHIFT;
> > > > > > > > > > > > > > > > +
> > > > > > > > > > > > > > > > + tb_cycle = (1 << (MAX_BIT - value)) *
> > > > 2;
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Is value = 0 and value = 1 legal? These will
> > > > > > > > > > > > > > > make tb_cycle = 0,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > + time = div_u64(tb_cycle * 1000,
> > > > > > > tb_ticks_per_usec)
> > > > > > > > > - 1;
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > And time = -1;
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > Please look at the end of the function, :)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > "return sprintf(buf, "%llu\n", time > 0 ? time :
> > 0);"
> > > > > > > > > > > > >
> > > > > > > > > > > > > I know you return 0 if value = 0/1, my question
> > > > > > > > > > > > > was that, is this correct as per specification?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Ahh, also for "value" upto 7 you will return 0, no?
> > > > > > > > > > > > >
> > > > > > > > > > > > If value = 0, MAX_BIT - value = 63 tb_cycle =
> > > > > > > > > > > > 0xffffffff_ffffffff, tb_cycle * 1000 will overflow,
> > > > > > > > > > > > but this
> > > > > > > situation is not possible.
> > > > > > > > > > > > Because if the "value = 0" means this feature will
> > > > > > > > > > > > be
> > > > > > "disable".
> > > > > > > > > > > > Now The default wait bit is 50(MAX_BIT - value,
> > > > > > > > > > > > value = 13), the PW20/Altivec Idle wait entry time
> > > > > > > > > > > > is about 1ms, this time is very long for wait idle
> > > > > > > > > > > > time, and it's cannot be increased(means (MAX_BIT
> > > > > > > > > > > > - value)
> > > > > > > > > > > cannot greater than 50).
> > > > > > > > > > >
> > > > > > > > > > > What you said is not obvious from code and so at least
> > > > > > > > > > > write a comment that value will be always >= 13 or
> > > > > > > > > > > value will never be less than < 8 and below
> > > > > > > > > > > calculation will not overflow. may be error out if
> > value is less than 8.
> > > > > > > > > > >
> > > > > > > > > > The "value" less than 10, this will overflow.
> > > > > > > > > > There is not error, The code I knew it could not be less
> > > > > > > > > > than 10, that's why I use the following code. :)
> > > > > > > > >
> > > > > > > > > I am sorry to persist but this is not about what you know,
> > > > > > > > > this is about how code is read and code does not say what
> > > > > > > > > you know, so add a comment at least and error out/warn
> > > > > > > > > when "value" is less than a
> > > > > > > certain number.
> > > > > > > > >
> > > > > > > > Sorry for the late to response the mail. If it caused
> > > > > > > > confusion, we can
> > > > > > > add a comment.
> > > > > > > >
> > > > > > > > How about the following comment?
> > > > > > > > /*
> > > > > > > > * If the "value" less than 10, this will overflow.
> > > > > > > > * From benchmark test, the default wait bit will not be set
> > > > > > > > less than
> > > > > > > 10bit.
> > > > > > > > * Because 10 bit corresponds to the wait entry time is
> > > > > > > > 439375573401999609(ns),
> > > > > > > > * for wait-entry-idle time this value looks too long, and
> > > > > > > > we cannot use those
> > > > > > > > * "long" time as a default wait-entry time. So overflow
> > > > > > > > could not have happened
> > > > > > > > * and we use this calculation method to get wait-entry-idle
> > time.
> > > > > > > > */
> > > > > > >
> > > > > > > If there's to be a limit on the times we accept, make it
> > explicit.
> > > > > > > Check for it before doing any conversions, and return an error
> > > > > > > if userspace tries to set it.
> > > > > > >
> > > > > > The branch only use to read default wait-entry-time.
> > > > > > We have no limit the user's input, and we can't restrict. Once
> > > > > > the user set the wait-entry-time, the code will do another branch.
> > > > > >
> > > > >
> > > > > Hi scott,
> > > > > Do you have any comments about this patch?
> > > > > I will add the comment and send this patch again.
> > > >
> > > > What do you mean by "and we can't restrict"? Why not?
> > > >
> > > > Why is it only used to read the default, and not the current value?
> > > >
> > > We already have a variable which value is set by the user, as we have
> > > discussed before.
> > >
> > > When the system boot-up. Before user set the wait-entry-time, we need
> > > to return a default wait-entry-time, if the user read this
> > > sys-interface. The default wait-entry-time is converted by wait-bit.
> > >
> > > Once the user set the sys-interface, a variable will be used to save
> > > it. And when the user read sys-interface we will return back the
> > variable.
> >
> > While we are not "restricting user defined value" or "define same
> > restriction for user defined and default", can we have only one flow of
> > calculation and setting rather than conditional based on user have set or
> > not set?
> >
> Yes, we can do that.
> If we want to use one flow to handle it. We should do the following changes:
>
> #ifdef CONFIG_FSL_SOC
> #include <linux/slab.h>
> #endif
Don't ifdef headers.
> static u64 *pw20_wt;
> static u64 *altivec_idle_wt;
>
> static ssize_t show_pw20_wait_time(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> return sprintf(buf, "%llu\n", pw20_wt[dev->id]);
> }
>
> static ssize_t store_pw20_wait_time(struct device *dev,
> struct device_attribute *attr,
> const char *buf, size_t count)
> {
> ...
> pw20_wt[cpu] = value;
> ...
> }
>
> static ssize_t show_altivec_idle_wait_time(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> return sprintf(buf, "%llu\n", altivec_idle_wt[dev->id]);
> }
>
> static ssize_t store_altivec_idle_wait_time(struct device *dev,
> struct device_attribute *attr,
> const char *buf, size_t count)
> {
> ...
> altivec_idle_wt[cpu] = value;
> ...
> }
>
> static void register_cpu_online(unsigned int cpu)
> {
> ...
> #ifdef CONFIG_FSL_SOC
> u32 value, pw20_value, altivec_value;
> u64 tb_cycle;
> #endif
> ...
> #ifdef CONFIG_FSL_SOC
> if (PVR_VER(cur_cpu_spec->pvr_value) == PVR_VER_E6500) {
> device_create_file(s, &dev_attr_pw20_state);
> device_create_file(s, &dev_attr_pw20_wait_time);
>
> device_create_file(s, &dev_attr_altivec_idle);
> device_create_file(s, &dev_attr_altivec_idle_wait_time);
> }
>
> if (!pw20_wt)
> pw20_wt = kzalloc(nr_cpu_ids * sizeof(*pw20_wt), GFP_KERNEL);
>
> if (!altivec_idle_wt)
> altivec_idle_wt = kzalloc(nr_cpu_ids * sizeof(*altivec_idle_wt), GFP_KERNEL);
>
> smp_call_function_single(cpu, do_show_pwrmgtcr0, &value, 1);
>
> pw20_value = (value & PWRMGTCR0_PW20_ENT) >> PWRMGTCR0_PW20_ENT_SHIFT;
> tb_cycle = (1 << (MAX_BIT - pw20_value)) * 2;
> pw20_wt[cpu] = div_u64(tb_cycle * 1000, tb_ticks_per_usec) - 1;
>
> altivec_value = (value & PWRMGTCR0_AV_IDLE_CNT) >> PWRMGTCR0_AV_IDLE_CNT_SHIFT;
> tb_cycle = (1 << (MAX_BIT - altivec_value)) * 2;
> altivec_idle_wt[cpu] = div_u64(tb_cycle * 1000, tb_ticks_per_usec) - 1;
> #endif
> ...
> }
Move this stuff to its own function. The only ifdef should be in a
header that provides an inline stub when it's not available.
Could you explain what you're changing and why? A diff from your
previous patch would help.
-Scott
^ permalink raw reply
* [PATCH 3/4] powerpc: ELF2 binaries launched directly.
From: Rusty Russell @ 2013-11-07 1:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Rusty Russell, Rusty Russell
In-Reply-To: <1383787012-16455-1-git-send-email-rusty@au1.ibm.com>
No function descriptor, but we set r12 up and set TIF_RESTOREALL as it
normally isn't restored on return from syscall.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/powerpc/kernel/process.c | 50 ++++++++++++++++++++++++++++++-------------
1 file changed, 35 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 59967ea..6fa271a 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1078,25 +1078,45 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
regs->msr = MSR_USER;
#else
if (!is_32bit_task()) {
- unsigned long entry, toc;
+ unsigned long entry;
- /* start is a relocated pointer to the function descriptor for
- * the elf _start routine. The first entry in the function
- * descriptor is the entry address of _start and the second
- * entry is the TOC value we need to use.
- */
- __get_user(entry, (unsigned long __user *)start);
- __get_user(toc, (unsigned long __user *)start+1);
+ if (is_elf2_task()) {
+ /* Look ma, no function descriptors! */
+ entry = start;
- /* Check whether the e_entry function descriptor entries
- * need to be relocated before we can use them.
- */
- if (load_addr != 0) {
- entry += load_addr;
- toc += load_addr;
+ /*
+ * Ulrich says:
+ * The latest iteration of the ABI requires that when
+ * calling a function (at its global entry point),
+ * the caller must ensure r12 holds the entry point
+ * address (so that the function can quickly
+ * establish addressability).
+ */
+ regs->gpr[12] = start;
+ /* Make sure that's restored on entry to userspace. */
+ set_thread_flag(TIF_RESTOREALL);
+ } else {
+ unsigned long toc;
+
+ /* start is a relocated pointer to the function
+ * descriptor for the elf _start routine. The first
+ * entry in the function descriptor is the entry
+ * address of _start and the second entry is the TOC
+ * value we need to use.
+ */
+ __get_user(entry, (unsigned long __user *)start);
+ __get_user(toc, (unsigned long __user *)start+1);
+
+ /* Check whether the e_entry function descriptor entries
+ * need to be relocated before we can use them.
+ */
+ if (load_addr != 0) {
+ entry += load_addr;
+ toc += load_addr;
+ }
+ regs->gpr[2] = toc;
}
regs->nip = entry;
- regs->gpr[2] = toc;
regs->msr = MSR_USER64;
} else {
regs->nip = start;
--
1.8.3.2
^ permalink raw reply related
* [PATCH 4/4] powerpc: ELF2 binaries signal handling
From: Rusty Russell @ 2013-11-07 1:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Rusty Russell, Rusty Russell
In-Reply-To: <1383787012-16455-1-git-send-email-rusty@au1.ibm.com>
For the ELFv2 ABI, the hander is the entry point, not a function descriptor.
We also need to set up r12, and fortunately the fast_exception_return
exit path restores r12 for us so nothing else is required.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/powerpc/kernel/signal_64.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c
index f7e61e0..af1f7ad 100644
--- a/arch/powerpc/kernel/signal_64.c
+++ b/arch/powerpc/kernel/signal_64.c
@@ -694,12 +694,6 @@ badframe:
int handle_rt_signal64(int signr, struct k_sigaction *ka, siginfo_t *info,
sigset_t *set, struct pt_regs *regs)
{
- /* Handler is *really* a pointer to the function descriptor for
- * the signal routine. The first entry in the function
- * descriptor is the entry address of signal and the second
- * entry is the TOC value we need to use.
- */
- func_descr_t __user *funct_desc_ptr;
struct rt_sigframe __user *frame;
unsigned long newsp = 0;
long err = 0;
@@ -759,19 +753,32 @@ int handle_rt_signal64(int signr, struct k_sigaction *ka, siginfo_t *info,
goto badframe;
regs->link = (unsigned long) &frame->tramp[0];
}
- funct_desc_ptr = (func_descr_t __user *) ka->sa.sa_handler;
/* Allocate a dummy caller frame for the signal handler. */
newsp = ((unsigned long)frame) - __SIGNAL_FRAMESIZE;
err |= put_user(regs->gpr[1], (unsigned long __user *)newsp);
/* Set up "regs" so we "return" to the signal handler. */
- err |= get_user(regs->nip, &funct_desc_ptr->entry);
+ if (is_elf2_task()) {
+ regs->nip = (unsigned long) ka->sa.sa_handler;
+ regs->gpr[12] = regs->nip;
+ } else {
+ /* Handler is *really* a pointer to the function descriptor for
+ * the signal routine. The first entry in the function
+ * descriptor is the entry address of signal and the second
+ * entry is the TOC value we need to use.
+ */
+ func_descr_t __user *funct_desc_ptr =
+ (func_descr_t __user *) ka->sa.sa_handler;
+
+ err |= get_user(regs->nip, &funct_desc_ptr->entry);
+ err |= get_user(regs->gpr[2], &funct_desc_ptr->toc);
+ }
+
/* enter the signal handler in native-endian mode */
regs->msr &= ~MSR_LE;
regs->msr |= (MSR_KERNEL & MSR_LE);
regs->gpr[1] = newsp;
- err |= get_user(regs->gpr[2], &funct_desc_ptr->toc);
regs->gpr[3] = signr;
regs->result = 0;
if (ka->sa.sa_flags & SA_SIGINFO) {
--
1.8.3.2
^ permalink raw reply related
* [PATCH 1/4] powerpc: Add TIF_ELF2ABI flag.
From: Rusty Russell @ 2013-11-07 1:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Rusty Russell, Rusty Russell
Little endian ppc64 is getting an exciting new ABI. This is reflected
by the bottom two bits of e_flags in the ELF header:
0 == legacy binaries (v1 ABI)
1 == binaries using the old ABI (compiled with a new toolchain)
2 == binaries using the new ABI.
We store this in a thread flag, because we need to set it in core
dumps and for signal delivery. Our chief concern is that it doesn't
use function descriptors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/powerpc/include/asm/elf.h | 4 ++++
arch/powerpc/include/asm/thread_info.h | 9 +++++++++
2 files changed, 13 insertions(+)
diff --git a/arch/powerpc/include/asm/elf.h b/arch/powerpc/include/asm/elf.h
index ac6ec6e..54c7445 100644
--- a/arch/powerpc/include/asm/elf.h
+++ b/arch/powerpc/include/asm/elf.h
@@ -85,6 +85,8 @@ typedef elf_vrregset_t elf_fpxregset_t;
# ifdef CONFIG_COMPAT
# define SET_PERSONALITY(ex) \
do { \
+ if (((ex).e_flags & 0x3) == 2) \
+ set_thread_flag(TIF_ELF2ABI); \
if ((ex).e_ident[EI_CLASS] == ELFCLASS32) \
set_thread_flag(TIF_32BIT); \
else \
@@ -96,6 +98,8 @@ do { \
# else /* !COMPAT: */
# define SET_PERSONALITY(ex) \
do { \
+ if (((ex).e_flags & 0x3) == 2) \
+ set_thread_flag(TIF_ELF2ABI); \
if (personality(current->personality) != PER_LINUX32) \
set_personality(PER_LINUX | \
(current->personality & (~PER_MASK))); \
diff --git a/arch/powerpc/include/asm/thread_info.h b/arch/powerpc/include/asm/thread_info.h
index f66c2c1..460326f 100644
--- a/arch/powerpc/include/asm/thread_info.h
+++ b/arch/powerpc/include/asm/thread_info.h
@@ -109,6 +109,9 @@ static inline struct thread_info *current_thread_info(void)
#define TIF_EMULATE_STACK_STORE 16 /* Is an instruction emulation
for stack store? */
#define TIF_MEMDIE 17 /* is terminating due to OOM killer */
+#if defined(CONFIG_PPC64)
+#define TIF_ELF2ABI 18 /* function descriptors must die! */
+#endif
/* as above, but as bit values */
#define _TIF_SYSCALL_TRACE (1<<TIF_SYSCALL_TRACE)
@@ -193,6 +196,12 @@ static inline bool test_thread_local_flags(unsigned int flags)
#define is_32bit_task() (1)
#endif
+#if defined(CONFIG_PPC64)
+#define is_elf2_task() (test_thread_flag(TIF_ELF2ABI))
+#else
+#define is_elf2_task() (0)
+#endif
+
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
--
1.8.3.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox