From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755275AbbCRPWH (ORCPT ); Wed, 18 Mar 2015 11:22:07 -0400 Received: from foss.arm.com ([217.140.101.70]:34200 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755169AbbCRPWD (ORCPT ); Wed, 18 Mar 2015 11:22:03 -0400 Date: Wed, 18 Mar 2015 15:21:30 +0000 From: Mark Rutland To: Joonsoo Kim Cc: LKML , Andrew Morton , Catalin Marinas , Christoph Lameter , David Rientjes , Jesper Dangaard Brouer , Joonsoo Kim , Linus Torvalds , Pekka Enberg , Steve Capper Subject: Re: [PATCHv2] mm/slub: fix lockups on PREEMPT && !SMP kernels Message-ID: <20150318152130.GA19814@leverpostej> References: <20150317120058.GC23340@leverpostej> <1426594533-18221-1-git-send-email-mark.rutland@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, > > do { > > tid = this_cpu_read(s->cpu_slab->tid); > > c = raw_cpu_ptr(s->cpu_slab); > > - } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid)); > > + } while (IS_ENABLED(CONFIG_PREEMPT) && > > + unlikely(tid != READ_ONCE(c->tid))); [...] > Could you show me generated code again? The code generated without this patch in !SMP && PREEMPT kernels is: /* Hoisted load of c->tid */ ffffffc00016d3c4: f9400404 ldr x4, [x0,#8] /* this_cpu_read(s->cpu_slab->tid)) -- buggy, see [1] */ ffffffc00016d3c8: f9400401 ldr x1, [x0,#8] ffffffc00016d3cc: eb04003f cmp x1, x4 ffffffc00016d3d0: 54ffffc1 b.ne ffffffc00016d3c8 The code generated with this patch in !SMP && PREEMPT kernels is: /* this_cpu_read(s->cpu_slab->tid)) -- buggy, see [1] */ ffffffc00016d3c4: f9400401 ldr x1, [x0,#8] /* load of c->tid */ ffffffc00016d3c8: f9400404 ldr x4, [x0,#8] ffffffc00016d3cc: eb04003f cmp x1, x4 ffffffc00016d3d0: 54ffffa1 b.ne ffffffc00016d3c4 Note that with the patch the branch results in both loads being performed again. Given that in !SMP kernels we know that the loads _must_ happen on the same CPU, I think we could go a bit further with the loop condition: while (IS_ENABLED(CONFIG_PREEMPT) && !IS_ENABLED(CONFIG_SMP) && unlikely(tid != READ_ONCE(c->tid))); The barrier afterwards should be sufficient to order the load of the tid against subsequent accesses to the other cpu_slab fields. > What we need to check is redoing whole things in the loop. > Previous attached code seems to me that it already did > refetching c->tid in the loop and this patch looks only handle > refetching c->tid. The refetch in the loop is this_cpu_read(s->cpu_slab->tid), not the load of c->tid (which is hoisted above the loop). > READ_ONCE(c->tid) will trigger redoing 'tid = this_cpu_read(s->cpu_slab->tid)'? I was under the impression that this_cpu operations would always result in an access, much like the *_ONCE accessors, so we should aways redo the access for this_cpu_read(s->cpu_slab->tid). Is that not the case? Mark. [1] The arm64 this_cpu * operations are currently buggy. We generate the percpu address into a register, then perform the access with separate instructions (and could be preempted between the two). Steve Capper is currently fixing this. However, the hoisting of the c->tid load could happen regardless, whenever raw_cpu_ptr(c) can be evaluated at compile time.