Re: [crash, bisected] Re: [PATCH 3/4] x86_64: Fold pda into per cpu area

[Jeremy Fitzhardinge <jeremy@goop.org>]

Eric W. Biederman wrote:
> Nope.  It achieves that affect with a magic set of relocations instead
> of linker magic.
>   

Well, the code gcc generates for -fstack-protector emits a literal 
"%gs:40", so there's no relocations at all.

>> At present, the x86-64 only uses %gs-relative addressing to reach the pda, which
>> are always small positive offsets.  It always accesses per-cpu data in a
>> two-step process of getting the base of per-cpu data, then offsetting to find
>> the particular variable.
>>
>> x86-32 has no pda, and arranges %fs so that %fs:variable gets the percpu variant
>> of variable.  The offsets are always quite large.
>>     
>
> As a practical matter I like that approach (except for extra code size
> of the offsets).
>   

Yes, and there's no reason we couldn't do the same on 64-bit, aside from 
the stack-protector's use of %gs:40.  There's no code-size cost in large 
offsets, since they're always 32-bits anyway (there's no short absolute 
addressing mode).

>> The powerpc guys tried using gcc-level thread-local storage, but it doesn't work
>> well.  per-cpu data and per-thread data have different constraints, and its hard
>> to tell gcc about them.  For example, if you have a section of preemptable code
>> in your function, it's hard to tell gcc not to cache a "thread-local" variable
>> across it, even though we could have switched CPUs in the meantime.
>>     
>
> Yes, I completely agree with that.  It doesn't mean however that we
> can't keep gcc ignorant and generate the same code manually.
>   

Yes, I see.  I haven't looked at that specifically, but I think both 
Rusty and Andi have, and it gets tricky with modules and -ve kernel 
addresses, or something.

> Well I was thinking threads switching on a cpu having the kinds of problems you
> described when it was tried on ppc.

Uh, I think we're having a nomenclature imprecision here.  Strictly 
speaking, the kernel doesn't have threads, only tasks and CPUs.  We only 
care about per-cpu data, not per-task data, so the concern is not 
"threads switching on a CPU" but "CPUs switching on (under) a task".  
But I think we understand each other regardless ;)

If we manually generate %gs-relative references to percpu data, then 
it's no different to what we do with 32-bit, whether it be a specific 
symbol address or using the TLS relocations.

    J