From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Biggers Subject: Re: [PATCH] tools/memory-model: document the "one-time init" pattern Date: Fri, 17 Jul 2020 19:00:09 -0700 Message-ID: <20200718020009.GE2183@sol.localdomain> References: <20200717044427.68747-1-ebiggers@kernel.org> <20200717205340.GR7625@magnolia> <20200718005857.GB2183@sol.localdomain> <20200718012555.GA1168834@rowland.harvard.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20200718012555.GA1168834@rowland.harvard.edu> Sender: linux-kernel-owner@vger.kernel.org To: Alan Stern Cc: "Darrick J. Wong" , linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, "Paul E . McKenney" , linux-fsdevel@vger.kernel.org, Akira Yokosawa , Andrea Parri , Boqun Feng , Daniel Lustig , Dave Chinner , David Howells , Jade Alglave , Luc Maranget , Nicholas Piggin , Peter Zijlstra , Will Deacon List-Id: linux-arch.vger.kernel.org On Fri, Jul 17, 2020 at 09:25:55PM -0400, Alan Stern wrote: > On Fri, Jul 17, 2020 at 05:58:57PM -0700, Eric Biggers wrote: > > On Fri, Jul 17, 2020 at 01:53:40PM -0700, Darrick J. Wong wrote: > > > > +There are also cases in which the smp_load_acquire() can be replaced by > > > > +the more lightweight READ_ONCE(). (smp_store_release() is still > > > > +required.) Specifically, if all initialized memory is transitively > > > > +reachable from the pointer itself, then there is no control dependency > > > > > > I don't quite understand what "transitively reachable from the pointer > > > itself" means? Does that describe the situation where all the objects > > > reachable through the object that the global struct foo pointer points > > > at are /only/ reachable via that global pointer? > > > > > > > The intent is that "transitively reachable" means that all initialized memory > > can be reached by dereferencing the pointer in some way, e.g. p->a->b[5]->c. > > > > It could also be the case that allocating the object initializes some global or > > static data, which isn't reachable in that way. Access to that data would then > > be a control dependency, which a data dependency barrier wouldn't work for. > > > > It's possible I misunderstood something. (Note the next paragraph does say that > > using READ_ONCE() is discouraged, exactly for this reason -- it can be hard to > > tell whether it's correct.) Suggestions of what to write here are appreciated. > > Perhaps something like this: > > Specifically, if the only way to reach the initialized memory > involves dereferencing the pointer itself then READ_ONCE() is > sufficient. This is because there will be an address dependency > between reading the pointer and accessing the memory, which will > ensure proper ordering. But if some of the initialized memory > is reachable some other way (for example, if it is global or > static data) then there need not be an address dependency, > merely a control dependency (checking whether the pointer is > non-NULL). Control dependencies do not always ensure ordering > -- certainly not for reads, and depending on the compiler, > possibly not for some writes -- and therefore a load-acquire is > necessary. > > Perhaps this is more wordy than you want, but it does get the important > ideas across. > How about: There are also cases in which the smp_load_acquire() can be replaced by the more lightweight READ_ONCE(). (smp_store_release() is still required.) Specifically, if the only way to reach the initialized memory involves dereferencing the pointer itself, then the data dependency barrier provided by READ_ONCE() is sufficient. However, if some of the initialized memory is reachable some other way (for example, if it is global or static data) then there need not be an address dependency, merely a control dependency (checking whether the pointer is non-NULL). READ_ONCE() is *not* sufficient in that case. The optimization of replacing smp_load_acquire() with READ_ONCE() is discouraged for nontrivial data structures, since it can be difficult to determine if it is correct. In particular, for complex data structures the correctness of the READ_ONCE() optimization may depend on internal implementation details of other kernel subsystems. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.kernel.org ([198.145.29.99]:53624 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726665AbgGRCAM (ORCPT ); Fri, 17 Jul 2020 22:00:12 -0400 Date: Fri, 17 Jul 2020 19:00:09 -0700 From: Eric Biggers Subject: Re: [PATCH] tools/memory-model: document the "one-time init" pattern Message-ID: <20200718020009.GE2183@sol.localdomain> References: <20200717044427.68747-1-ebiggers@kernel.org> <20200717205340.GR7625@magnolia> <20200718005857.GB2183@sol.localdomain> <20200718012555.GA1168834@rowland.harvard.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200718012555.GA1168834@rowland.harvard.edu> Sender: linux-arch-owner@vger.kernel.org List-ID: To: Alan Stern Cc: "Darrick J. Wong" , linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, "Paul E . McKenney" , linux-fsdevel@vger.kernel.org, Akira Yokosawa , Andrea Parri , Boqun Feng , Daniel Lustig , Dave Chinner , David Howells , Jade Alglave , Luc Maranget , Nicholas Piggin , Peter Zijlstra , Will Deacon Message-ID: <20200718020009._x2iatg1PJYu1IDHAqUheqbdzUP748cLGSA-JPcLxAI@z> On Fri, Jul 17, 2020 at 09:25:55PM -0400, Alan Stern wrote: > On Fri, Jul 17, 2020 at 05:58:57PM -0700, Eric Biggers wrote: > > On Fri, Jul 17, 2020 at 01:53:40PM -0700, Darrick J. Wong wrote: > > > > +There are also cases in which the smp_load_acquire() can be replaced by > > > > +the more lightweight READ_ONCE(). (smp_store_release() is still > > > > +required.) Specifically, if all initialized memory is transitively > > > > +reachable from the pointer itself, then there is no control dependency > > > > > > I don't quite understand what "transitively reachable from the pointer > > > itself" means? Does that describe the situation where all the objects > > > reachable through the object that the global struct foo pointer points > > > at are /only/ reachable via that global pointer? > > > > > > > The intent is that "transitively reachable" means that all initialized memory > > can be reached by dereferencing the pointer in some way, e.g. p->a->b[5]->c. > > > > It could also be the case that allocating the object initializes some global or > > static data, which isn't reachable in that way. Access to that data would then > > be a control dependency, which a data dependency barrier wouldn't work for. > > > > It's possible I misunderstood something. (Note the next paragraph does say that > > using READ_ONCE() is discouraged, exactly for this reason -- it can be hard to > > tell whether it's correct.) Suggestions of what to write here are appreciated. > > Perhaps something like this: > > Specifically, if the only way to reach the initialized memory > involves dereferencing the pointer itself then READ_ONCE() is > sufficient. This is because there will be an address dependency > between reading the pointer and accessing the memory, which will > ensure proper ordering. But if some of the initialized memory > is reachable some other way (for example, if it is global or > static data) then there need not be an address dependency, > merely a control dependency (checking whether the pointer is > non-NULL). Control dependencies do not always ensure ordering > -- certainly not for reads, and depending on the compiler, > possibly not for some writes -- and therefore a load-acquire is > necessary. > > Perhaps this is more wordy than you want, but it does get the important > ideas across. > How about: There are also cases in which the smp_load_acquire() can be replaced by the more lightweight READ_ONCE(). (smp_store_release() is still required.) Specifically, if the only way to reach the initialized memory involves dereferencing the pointer itself, then the data dependency barrier provided by READ_ONCE() is sufficient. However, if some of the initialized memory is reachable some other way (for example, if it is global or static data) then there need not be an address dependency, merely a control dependency (checking whether the pointer is non-NULL). READ_ONCE() is *not* sufficient in that case. The optimization of replacing smp_load_acquire() with READ_ONCE() is discouraged for nontrivial data structures, since it can be difficult to determine if it is correct. In particular, for complex data structures the correctness of the READ_ONCE() optimization may depend on internal implementation details of other kernel subsystems.