Re: [GIT RFC] percpu: use dynamic percpu allocator as the default percpu allocator

Tejun Heo
2009-03-25T11:55:18Z

Hello, Martin.

Martin Schwidefsky wrote:
>>>> With the dynamic percpu allocator there is no need anymore to play
>>>> tricks with the GOTENT relocation for the access to the percpu
>>>> symbols. A simple RELOC_HIDE gets the job done.
>>> Hmm... I don't quite get it. The GOTENT was to work around large
>>> offsets for modules, right? Can you please explain what changed by
>>> the dynamic percpu allocator?
>
> Unfortunately it didn't change. The problem is still there, only with
> my particular configuration (and the correct patch) the system did
> work because the problematic modules were not in use. But in general it
> won't work.
>
> The reason for the GOTENT indirection are static per-cpu variables that
> are defined inside a module. The compiler considers these to be local.
> For locally defined per_cpu__#var symbols the compiler uses an
> instruction that is limited to the scope of a single object, which is
> +-4 GB. The trick with GOTENT introduced an indirection which hid the
> problem.
>
> Without the GOTENT indirection the access to a static per cpu variable
> will look like this:
>
> 0000000000000000 <test_fn>:
> 0: e3 30 03 30 00 04 lg %r3,816
> 6: c0 10 00 00 00 00 larl %r1,6 <test_fn+0x6>
> 8: R_390_PC32DBL .data.percpu+0x2
> c: e3 23 10 00 00 04 lg %r2,0(%r3,%r1)
>
> The R_390_PC32DBL relocation in the module relocation will fail if the
> per-cpu area is farther than 4GB away from the vmalloc area.

Okay, up to this point, I understand, so nothing really changed for
symbols (core or modules) by the dynamic percpu allocator and they
still need GOTENT, right?

> With your patches and a RELOC_HIDE version that uses the GOTENT
> indirection the kernel won't compile because the "X" constraint for
> the GOTENT access needs a symbol and there are quite a few users that
> pass a pointer. I do not see a simple solution for that problem yet.

Ah... okay. Now I get it. It wasn't expecting variables there. How
about doing the following?

#define SHIFT_PERCPU_PTR(ptr, offset) (({ \
if (__builtin_constant_p(ptr)) \
do GOTENT trick; \
else \
RELOC_HIDE(); \
}))

Thanks.

--
tejun