Re: [RFC PATCH] getcpu_cache system call: caching current CPU number (x86)

["Ondřej Bílka" <neleai-9Vj9tDbzfuSlVyrhU4qvOw@public.gmane.org>]

On Sat, Jul 18, 2015 at 03:34:33AM -0400, Rich Felker wrote:
> On Sat, Jul 18, 2015 at 01:28:36AM +0200, Ondřej Bílka wrote:
> > On Fri, Jul 17, 2015 at 11:48:14AM -0700, Linus Torvalds wrote:
> > > On Thu, Jul 16, 2015 at 12:27 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> > > >
> > > > If we actually bit the bullet and implemented per-cpu mappings
> > > 
> > > That's not ever going to happen.
> > > 
> > > The Linux VM model of "one page table per VM" is the right one.
> > > Anything else sucks, and makes threading a disaster.
> > > 
> > > So you can try to prove me wrong, but seriously, I doubt you'll succeed.
> > > 
> > > On x86, if you want per-cpu memory areas, you should basically plan on
> > > using segment registers instead (although other odd state has been
> > > used - there's been the people who use segment limits etc rather than
> > > the *pointer* itself, preferring to use "lsl" to get percpu data. You
> > > could also imaging hiding things in the vector state somewhere if you
> > > control your environment well enough).
> > >
> > Thats correct, problem is that you need some sort of hack like this on
> > archs that otherwise would need syscall to get tid/access tls variable.
> > 
> > On x64 and archs that have register for tls this could be implemented
> > relatively easily.
> > 
> > Kernel needs to allocate 
> > 
> > int running_cpu_for_tid[32768];
> 
> This does not scale. You're assuming the default task ("pid") number
> limit, but this can be raised up to 512k (beyond that is impossible
> because of PI/robust futex ABI).
>
Doesn't matter much, you will allocate 512k instead. As scaling if you
have simultaneously more than 32k threads running you have different
worries than how slow its to get cpu id.
 
> > On context switch it atomically writes to this table 
> > 
> > running_cpu_for_tid[tid] = cpu;
> > 
> > This table is read-only accessible from userspace as mmaped file.
> 
> There is a much simpler solution: use a per-cpu (rather than per-task)
> page that contains the right value for the cpu. I believe vdso already
> does something like this, no?
> 
Thats exactly what I suggested before to improve tls when you don't have
register for that.

If thats already done its just matter of abi for kernel map per-cpu page
to fixed virtual page and save/restore tcb in same way as it sets cpuid.

With that abi a tls access would cost just extra load more than of static
variable.