[PATCH] mm/slub: use unchecked percpu access within preemptible sections

[PATCH] mm/slub: use unchecked percpu access within preemptible sections

[Sasha Levin]

Commit "mm/slub: optimize alloc/free fastpath by removing preemption on/off"
has added access to percpu memory while the code is preemptible.

While those accesses are okay, this creates a huge amount of warnings from
the code that checks for that.

Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
---
 mm/slub.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 907fabe..9b72b9c 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2431,8 +2431,8 @@ redo:
 	 * to check if it is matched or not.
 	 */
 	do {
-		tid = this_cpu_read(s->cpu_slab->tid);
-		c = this_cpu_ptr(s->cpu_slab);
+		tid = raw_cpu_read(s->cpu_slab->tid);
+		c = raw_cpu_ptr(s->cpu_slab);
 	} while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
 
 	/*
@@ -2700,8 +2700,8 @@ redo:
 	 * during the cmpxchg then the free will succedd.
 	 */
 	do {
-		tid = this_cpu_read(s->cpu_slab->tid);
-		c = this_cpu_ptr(s->cpu_slab);
+		tid = raw_cpu_read(s->cpu_slab->tid);
+		c = raw_cpu_ptr(s->cpu_slab);
 	} while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
 
 	/* Same with comment on barrier() in slab_alloc_node() */
-- 
1.7.10.4

Re: [PATCH] mm/slub: use unchecked percpu access within preemptible sections

[Joonsoo Kim]

On Tue, Jan 20, 2015 at 11:31:43PM -0500, Sasha Levin wrote:
> Commit "mm/slub: optimize alloc/free fastpath by removing preemption on/off"
> has added access to percpu memory while the code is preemptible.
> 
> While those accesses are okay, this creates a huge amount of warnings from
> the code that checks for that.
> 
> Signed-off-by: Sasha Levin <sasha.levin@oracle.com>

Hello,

I already sent the patch to fix this issue and it is in mmotm, but,
not be released yet.

https://lkml.org/lkml/2015/1/19/17

Thanks.

Re: [PATCH] mm/slub: use unchecked percpu access within preemptible sections

[Sasha Levin]

On 01/21/2015 08:59 PM, Joonsoo Kim wrote:
> On Tue, Jan 20, 2015 at 11:31:43PM -0500, Sasha Levin wrote:
>> > Commit "mm/slub: optimize alloc/free fastpath by removing preemption on/off"
>> > has added access to percpu memory while the code is preemptible.
>> > 
>> > While those accesses are okay, this creates a huge amount of warnings from
>> > the code that checks for that.
>> > 
>> > Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
> Hello,
> 
> I already sent the patch to fix this issue and it is in mmotm, but,
> not be released yet.
> 
> https://lkml.org/lkml/2015/1/19/17

The patch you sent out still has the issue. The one I sent goes on top of
it.

+	do {
+		tid = this_cpu_read(s->cpu_slab->tid); <=== checked percpu access
+		c = raw_cpu_ptr(s->cpu_slab);
+	} while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));


Thanks,
Sasha

Re: [PATCH] mm/slub: use unchecked percpu access within preemptible sections

[Joonsoo Kim]

On Wed, Jan 21, 2015 at 09:27:41PM -0500, Sasha Levin wrote:
> On 01/21/2015 08:59 PM, Joonsoo Kim wrote:
> > On Tue, Jan 20, 2015 at 11:31:43PM -0500, Sasha Levin wrote:
> >> > Commit "mm/slub: optimize alloc/free fastpath by removing preemption on/off"
> >> > has added access to percpu memory while the code is preemptible.
> >> > 
> >> > While those accesses are okay, this creates a huge amount of warnings from
> >> > the code that checks for that.
> >> > 
> >> > Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
> > Hello,
> > 
> > I already sent the patch to fix this issue and it is in mmotm, but,
> > not be released yet.
> > 
> > https://lkml.org/lkml/2015/1/19/17
> 
> The patch you sent out still has the issue. The one I sent goes on top of
> it.
> 
> +	do {
> +		tid = this_cpu_read(s->cpu_slab->tid); <=== checked percpu access
> +		c = raw_cpu_ptr(s->cpu_slab);
> +	} while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
> 

Hello,

this_cpu_xxx() is designed to be called regardless of interrupts and
preemption.

In Documentation/this_cpu_ops.txt,

The following this_cpu() operations with implied preemption protection
are defined. These operations can be used without worrying about
preemption and interrupts.

        this_cpu_read(pcp)
        ...

And, for correctness of algorithm, tid should be fetched through
this_cpu_read() rather than raw_cpu_read(). Generic implementation of
raw_cpu_read() has a race window to fetch different cpu's tid instead
of the cpu where we are at now and this will cause algorithm broken.

Thanks.