From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3tXLTk1hWtzDw4P for ; Mon, 5 Dec 2016 21:24:26 +1100 (AEDT) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uB5AK4PL024574 for ; Mon, 5 Dec 2016 05:24:24 -0500 Received: from e28smtp01.in.ibm.com (e28smtp01.in.ibm.com [125.16.236.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 27541n6hw3-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 05 Dec 2016 05:24:23 -0500 Received: from localhost by e28smtp01.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 5 Dec 2016 15:54:21 +0530 Received: from d28relay10.in.ibm.com (d28relay10.in.ibm.com [9.184.220.161]) by d28dlp02.in.ibm.com (Postfix) with ESMTP id D3095394004E for ; Mon, 5 Dec 2016 15:54:18 +0530 (IST) Received: from d28av04.in.ibm.com (d28av04.in.ibm.com [9.184.220.66]) by d28relay10.in.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id uB5ANhpa18743472 for ; Mon, 5 Dec 2016 15:53:43 +0530 Received: from d28av04.in.ibm.com (localhost [127.0.0.1]) by d28av04.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id uB5AOCYP029241 for ; Mon, 5 Dec 2016 15:54:18 +0530 From: Pan Xinhui To: linux-kernel@vger.kernel.org Cc: linuxppc-dev@lists.ozlabs.org, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, peterz@infradead.org, mingo@redhat.com, paulmck@linux.vnet.ibm.com, waiman.long@hpe.com, xinhui.pan@linux.vnet.ibm.com, virtualization@lists.linux-foundation.org, boqun.feng@gmail.com Subject: [PATCH v8 6/6] powerpc/pv-qspinlock: Optimise native unlock path Date: Mon, 5 Dec 2016 10:19:26 -0500 In-Reply-To: <1480951166-44830-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> References: <1480951166-44830-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> Message-Id: <1480951166-44830-7-git-send-email-xinhui.pan@linux.vnet.ibm.com> List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Avoid a function call under native version of qspinlock. On powerNV, bafore applying this patch, every unlock is expensive. This small optimizes enhance the performance. We use static_key with jump_label which removes unnecessary loads of lppaca and its stuff. Signed-off-by: Pan Xinhui --- arch/powerpc/include/asm/qspinlock_paravirt.h | 18 +++++++++++++++++- arch/powerpc/kernel/paravirt.c | 4 ++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/qspinlock_paravirt.h b/arch/powerpc/include/asm/qspinlock_paravirt.h index d87cda0..8d39446 100644 --- a/arch/powerpc/include/asm/qspinlock_paravirt.h +++ b/arch/powerpc/include/asm/qspinlock_paravirt.h @@ -6,12 +6,14 @@ #define _ASM_QSPINLOCK_PARAVIRT_H #include +#include extern void pv_lock_init(void); extern void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); extern void __pv_init_lock_hash(void); extern void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); extern void __pv_queued_spin_unlock(struct qspinlock *lock); +extern struct static_key_true sharedprocessor_key; static inline void pv_queued_spin_lock(struct qspinlock *lock, u32 val) { @@ -20,7 +22,21 @@ static inline void pv_queued_spin_lock(struct qspinlock *lock, u32 val) static inline void pv_queued_spin_unlock(struct qspinlock *lock) { - pv_lock_op.unlock(lock); + /* + * on powerNV and pSeries with jump_label, code will be + * PowerNV: pSeries: + * nop; b 2f; + * native unlock 2: + * pv unlock; + * In this way, we can do unlock quick in native case. + * + * IF jump_label is not enabled, we fall back into + * if condition, IOW, ld && cmp && bne. + */ + if (static_branch_likely(&sharedprocessor_key)) + native_queued_spin_unlock(lock); + else + pv_lock_op.unlock(lock); } static inline void pv_wait(u8 *ptr, u8 val) diff --git a/arch/powerpc/kernel/paravirt.c b/arch/powerpc/kernel/paravirt.c index e697b17..a0a000e 100644 --- a/arch/powerpc/kernel/paravirt.c +++ b/arch/powerpc/kernel/paravirt.c @@ -140,6 +140,9 @@ struct pv_lock_ops pv_lock_op = { }; EXPORT_SYMBOL(pv_lock_op); +struct static_key_true sharedprocessor_key = STATIC_KEY_TRUE_INIT; +EXPORT_SYMBOL(sharedprocessor_key); + void __init pv_lock_init(void) { if (SHARED_PROCESSOR) { @@ -149,5 +152,6 @@ void __init pv_lock_init(void) pv_lock_op.unlock = __pv_queued_spin_unlock; pv_lock_op.wait = __pv_wait; pv_lock_op.kick = __pv_kick; + static_branch_disable(&sharedprocessor_key); } } -- 2.4.11