Re: [PATCH 0/3] timer: Improve itimers scalability

[Hideaki Kimura <hideaki.kimura@hpe.com>]

Sure, let me elaborate.

Executive summary:
  Yes, enabling a process-wide timer in such a large machine is not 
wise, but sometimes users/applications cannot avoid it.


The issue was observed actually not in a database itself but in a common 
library it links to; gperftools.

The database itself is optimized for many-cores/sockets, so surely it 
avoids putting a process-wide timer or other unscalable things. It just 
links to libprofiler for an optional feature to profile performance 
bottleneck only when the user turns it on. We of course avoid turning 
the feature on unless while we debug/tune the database.

However, libprofiler sets the timer even when the client program doesn't 
invoke any of its functions: libprofiler does it when the shared library 
is loaded. We requested the developer of libprofiler to change the 
behavior, but seems like there is a reason to keep that behavior:
   https://code.google.com/p/gperftools/issues/detail?id=133

Based on this, I think there are two reasons why we should ameliorate 
this issue in kernel layer.


1. In the particular case, it's hard to prevent or even detect the issue 
in user space.

We (a team of low-level database and kernel experts) in fact spent huge 
amount of time to just figure out what's the bottleneck there because 
nothing measurable happens in user space. I pulled out countless hairs.

Also, the user has to de-link the library from the application to 
prevent the itimer installation. Imagine a case where the software is 
proprietary. It won't fly.


2. This is just one example. There could be many other such 
binaries/libraries that do similar things somewhere in a complex 
software stack.

Today we haven't heard of many such cases, but people will start hitting 
it once 100s~1,000s of cores become common.


After applying this patchset, we have observed that the performance hit 
almost completely went away at least for 240 cores. So, it's quite 
beneficial in real world.

-- 
Hideaki Kimura