From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754776AbcEYPVF (ORCPT ); Wed, 25 May 2016 11:21:05 -0400 Received: from mail-by2on0107.outbound.protection.outlook.com ([207.46.100.107]:26748 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750856AbcEYPUu (ORCPT ); Wed, 25 May 2016 11:20:50 -0400 Authentication-Results: linux.vnet.ibm.com; dkim=none (message not signed) header.d=none;linux.vnet.ibm.com; dmarc=none action=none header.from=hpe.com; Message-ID: <5745C2CA.4040003@hpe.com> Date: Wed, 25 May 2016 11:20:42 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: CC: Peter Zijlstra , , , , , , , , , , , , , , Subject: Re: [RFC][PATCH 1/3] locking: Introduce smp_acquire__after_ctrl_dep References: <20160524142723.178148277@infradead.org> <20160524143649.523586684@infradead.org> <57451581.6000700@hpe.com> <20160525045329.GQ4148@linux.vnet.ibm.com> In-Reply-To: <20160525045329.GQ4148@linux.vnet.ibm.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [71.168.64.180] X-ClientProxiedBy: CY1PR18CA0027.namprd18.prod.outlook.com (10.163.31.37) To CS1PR84MB0312.NAMPRD84.PROD.OUTLOOK.COM (10.162.190.30) X-MS-Office365-Filtering-Correlation-Id: 3e718ace-b7c7-4d64-0d1e-08d384b0263e X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;2:0qNXnGsDJVwtLRSBkOJv1vrlzyBTD2P4Bc2hlqwfP7JOH1kjTpqVbIc/lVvKi0NphWk1EbzF09OPYTy0H6yz4xNDsXUwAwBz9TLsawOfiWealo0QG4U2NacD3Ms5alGVH01iUjZdKYYzchFXhP1+DcyXX9ml0TJwYZeA9bl2bhmVTbGEEw4E5hUhNHTOFUPv;3:SQz77ZJwfkXkmjuosMwziFsWnM+3EMa6U0e1AW8q88+ghCN6LarWTLPLG0v+GbFlM/k0qqHVSup7sLzn9fu2uRaGfeo5QwU/349bpuDOCLMVUmRZIKv9tH8nREkVne3b X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0312; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;25:eQMBcWB2asRCVR/eyOlJbDHwyfsY7R8Evrc+OaL5zMxxC8SQYON23+FG6yvVtFy/7YC4oYRiZuOsG8MYjeJk/sHBTFG2xR8jX5VtEbaBW6vmbihzyfCZ7EmCob/XGFm1u8ZpU0VlMDHxBVVtaj7/z61qC2zobGwrcZeeoSz2msd+1OtYgbBA6B9VSdvZDMMoYasNQWWggIhAuQ9HGq3JibEB/JSi9LC32EAyuEg4/pb7WQXhAnHZPqVeUTi7tlI5nV9qbg8x3RxO9tV4X9kD8FQ996HDo2s1FOK25VFxd1HIgU1fzP/x2t/TGc/0E5Ix24fyUKvuc0jPn/L7w2NV4PFlf1JnmemDQKuKGQmTfqxqmrZ6uqLI3wU7lbqf4TpJvkwIMt/L33KYXDHyFL15/bdBIMYt2IKw1u2M31uea2a6RfAmznn85bfQiYkPZ/xggZTr2BblTEsJS8ss/aWHqmt/kROi7PaFprwmjKodiINcsRIjxh1OyIWqQdYS/37vKkvJPwSs3QyYEXZto3SchI3TuoNRgOSqSNQO/OAFWKXe3fCwVbG/sH8sX9m9n+D6u3hYd5jbdmH/HUZF3ll+mlOG5kcNCh73nPQNTV6h4NMoEBZ4qaG8JGLoURe/8DDplhazXFB+rjtEW1bYa/gvG24GoXgCQJlO/yjvga30g6MF3hu6G/WESYTeRi/579ui9s6AKZoEG/pfqc+J+MzHcg== X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;20:WqSPoepkm8WUHtDfBJ3A09pBCunky2xSj7+dWXne8Y80UVMOGIUPntDHAvemf9XuUBZdvCxoKduRM3zBlrDbQdmDHKKBiO6uKbl4kxK46P5qsykuv8QKLRDxyjVGInYgTkYhwohm7j4QUeEgdKJc4sYjaVhZsdg2EEOpIYKjXh8SJu37a/G5oEEolmzOvITsooYeQehKvu4F4EYjVMa+ev78WznisAGmrOtXMii7VmQ3+ogYDx6DW4noo3Me718sU0aJY31DFvH4E9Z/i1yg4K58sRbRgzDJMcvJFQRXDOb+oDaf2KCN3qiaxX5NXwVIvqFj/cd8FCKS4FeYl6+/0Q==;4:+Wq/4TFdXykDmOpf6a7/0wbajlAojonCLMkeIYx+tdKQJv17XBRPWUnLMTZi7J8bSEYG4dMtcIXlef5L4Ka1kyh4ZGlDo7Jf3GgLhUTWnHcHZDibJp58dwtrzcdQ03QcsaMwZ70JWkUwIEKK9ZRReBMVWGFvCai9Zwv3KDsHqfiHq7JbPz4kRp2SFZyePr8RrX9vYQYLmv9wu0w5m2baEDZ8VFpiFXvLbDjfA4qmDMQRgeNJC/lKqEnrHUqcv4HZStFu8ZYk+Ksln2bHfcFU8crCIMeBEZjF9m1xZp5r/QgdhyTCPqThZ+L1GqHFB5THUZi95HWwmgI3z2XUT2Urzq8c3OhQRcFeVn72AJm2ds9O5+Z3rmjXu+sqCuczwkBx X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:CS1PR84MB0312;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0312; X-Forefront-PRVS: 09538D3531 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(377454003)(24454002)(86362001)(23756003)(47776003)(65806001)(64126003)(36756003)(110136002)(2950100001)(189998001)(65956001)(66066001)(33656002)(77096005)(50466002)(2351001)(4326007)(230700001)(93886004)(8676002)(81166006)(3846002)(6116002)(586003)(5004730100002)(92566002)(76176999)(117156001)(65816999)(5008740100001)(83506001)(54356999)(50986999)(19580405001)(2906002)(42186005)(80316001)(19580395003);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0312;H:[192.168.142.139];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CS1PR84MB0312;23:t8hHjadeun4rhcsTxWSDmHaLWzxnBYkeHQEp0pa?= =?iso-8859-1?Q?iLchDUOlq5ieya/uaLU0RBlq1DZZPE0FbhWUv/JFKSmP2ud4GUPAmVdr0P?= =?iso-8859-1?Q?Sh111Bdmqs9bWj8YrTWf7ZFf89xUSSaUT9I/4l4m1Fyqmle6FJzYX+bXZ+?= =?iso-8859-1?Q?wSUkslVfQ7sJu1kjmz0uRrqbb+crHaar27J08sRQC2EaKxYv85v2Dqhowc?= =?iso-8859-1?Q?mc7jH3ZQHIRSXc2pZOasZM1a+cd1owvyHYwcBVjSO2cDPk54EqY15DvO8G?= =?iso-8859-1?Q?Vu+v1b3CUlaxfcN4yI9rg1AhYb4rdSlNwWRCd9Hwf9IPljNhbZCzkHQPt5?= =?iso-8859-1?Q?EECKlG+WRboqMk3ulStu+RzRDoqm30teBFBw29zGO9fZUJJ4Y+0JagvC7j?= =?iso-8859-1?Q?z+Lx8CNAEChACsbR+eyDZQUcmzvdmqhvIG4av5CtHymLb6yeZpaPXvorc1?= =?iso-8859-1?Q?3FU0Zi8bE3TJcdH9dRtAVp6X9F16nqHZXMfww9hdNlc6vrJss9H4gcVIA1?= =?iso-8859-1?Q?Kg+ekEdYpnIXN3KkHG4BR0sHDb4dttaP9XzqDdgHVY7tz/OvBMZ1oGseVP?= =?iso-8859-1?Q?RUfxs8srA0ltsaqPOYLtnD5Y08UhZrrIwJvupwAzg0FW+3WUn0zvUCSduC?= =?iso-8859-1?Q?7O8HtMNv0J6MbItUh3FzjW9rbm63ZCLC2TiRWUpnb6c9oCoFzaJW8Nci8Z?= =?iso-8859-1?Q?gPtBsHSKYge9hgn/ZSukCz0l496HsvYTdtGLHbS4Gzn81Xu/3PvOJh+vzL?= =?iso-8859-1?Q?1UTuaq+pnDZqay1liweZzHyM70moDExLrw+KXWF+ZSNCP8GRaEiIlwTWEV?= =?iso-8859-1?Q?S6wt/6qdztto6H7hZxN3kstBW9qG+b+62r67sAbCkoHMS5Rr9tY/zaRLTP?= =?iso-8859-1?Q?aemKMj5fySL8YKauRfr9bE+XSkY6WQGJixBJAkdajrIdWmY3SoGIDbjiKn?= =?iso-8859-1?Q?XgsEZW/2LAvoIdiiJNEl/QpRSTenOEhjoL3+oqxBlMyXVpJPPLGVsI1V8E?= =?iso-8859-1?Q?NaPCAMgfM71c1sksKLkuFNNONKa80rpk/HKduBdrWyucgezy3QtK7KEZIq?= =?iso-8859-1?Q?wkbZ9E0VVpV3OlnCrBxmp28N+CDC+jooKMiU308gFFlBqgZV3WPGtgT8qx?= =?iso-8859-1?Q?nND3m09+qTO7gu51LEtqlHNRJnA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0312;5:HVkhaWdi7+c5dNcxpcY5M3VqL6z7ymvq+hPi0WwsaJwONX40QEH3ebdOQ50k8rJtzhPNeMBkT4S2CvNFDyhy0Q/rPdY4YzOjuexiNxJJEt+IhKGLdo2MxPeKeCBX6jBWAnttd40Q3yofUIZ0SbMedQ==;24:pBZTPgmkOopy2UaZ1sQ2Vj5JQF4/GccqL2BTEPO+ufn/dojoIsxkvwtGsAg+/AoXlMbLWGWgouSIcYUwv9AkFvmM2/H0DnB0NTFpoXhGaUo=;7:uOkiXpSn0MDlFKVa3t6V2d+N9MATnxFiCYykgHp+KhIE9lz+lVl3tKsOSdHNEsPMhmO4oI38tS/Ye/Q/yWMM9eQUTYnH2Ke1YhAEj7KmUvIHd8zod875fahZYpmvRPoaZfxC19wnrE09R5CBVr3D/C+YHfjKmq0l2JUqPm79chH+H/OPLGMjoXKODbYEKq3d SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2016 15:20:46.5864 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0312 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/25/2016 12:53 AM, Paul E. McKenney wrote: > On Tue, May 24, 2016 at 11:01:21PM -0400, Waiman Long wrote: >> On 05/24/2016 10:27 AM, Peter Zijlstra wrote: >>> Introduce smp_acquire__after_ctrl_dep(), this construct is not >>> uncommen, but the lack of this barrier is. >>> >>> Signed-off-by: Peter Zijlstra (Intel) >>> --- >>> include/linux/compiler.h | 14 ++++++++++---- >>> ipc/sem.c | 14 ++------------ >>> 2 files changed, 12 insertions(+), 16 deletions(-) >>> >>> --- a/include/linux/compiler.h >>> +++ b/include/linux/compiler.h >>> @@ -305,20 +305,26 @@ static __always_inline void __write_once >>> }) >>> >>> /** >>> + * smp_acquire__after_ctrl_dep() - Provide ACQUIRE ordering after a control dependency >>> + * >>> + * A control dependency provides a LOAD->STORE order, the additional RMB >>> + * provides LOAD->LOAD order, together they provide LOAD->{LOAD,STORE} order, >>> + * aka. ACQUIRE. >>> + */ >>> +#define smp_acquire__after_ctrl_dep() smp_rmb() >>> + >>> +/** >>> * smp_cond_acquire() - Spin wait for cond with ACQUIRE ordering >>> * @cond: boolean expression to wait for >>> * >>> * Equivalent to using smp_load_acquire() on the condition variable but employs >>> * the control dependency of the wait to reduce the barrier on many platforms. >>> * >>> - * The control dependency provides a LOAD->STORE order, the additional RMB >>> - * provides LOAD->LOAD order, together they provide LOAD->{LOAD,STORE} order, >>> - * aka. ACQUIRE. >>> */ >>> #define smp_cond_acquire(cond) do { \ >>> while (!(cond)) \ >>> cpu_relax(); \ >>> - smp_rmb(); /* ctrl + rmb := acquire */ \ >>> + smp_acquire__after_ctrl_dep(); \ >>> } while (0) >>> >>> >> I have a question about the claim that control dependence + rmb is >> equivalent to an acquire memory barrier. For example, >> >> S1: if (a) >> S2: b = 1; >> smp_rmb() >> S3: c = 2; >> >> Since c is independent of both a and b, is it possible that the cpu >> may reorder to execute store statement S3 first before S1 and S2? > The CPUs I know of won't do, nor should the compiler, at least assuming > "a" (AKA "cond") includes READ_ONCE(). Ditto "b" and WRITE_ONCE(). > Otherwise, the compiler could do quite a few "interesting" things, > especially if it knows the value of "b". For example, if the compiler > knows that b==1, without the volatile casts, the compiler could just > throw away both S1 and S2, eliminating any ordering. This can get > quite tricky -- see memory-barriers.txt for more mischief. > > The smp_rmb() is not needed in this example because S3 is a write, not > a read. Perhaps you meant something more like this: > > if (READ_ONCE(a)) > WRITE_ONCE(b, 1); > smp_rmb(); > r1 = READ_ONCE(c); > > This sequence would guarantee that "a" was read before "c". > > Thanx, Paul > The smp_rmb() in Linux should be a compiler barrier. So the compiler should not recorder it above smp_rmb. However, what I am wondering is whether a condition + rmb combination can be considered a real acquire memory barrier from the CPU point of view which requires that it cannot reorder the data store in S3 above S1 and S2. This is where I am not so sure about. Cheers, Longman