Re: [PATCH 0/3] timer: Improve itimers scalability

Hideaki Kimura
2015-08-26T23:10:30Z

Sure, let me elaborate.

Executive summary:
Yes, enabling a process-wide timer in such a large machine is not
wise, but sometimes users/applications cannot avoid it.

The issue was observed actually not in a database itself but in a common
library it links to; gperftools.

The database itself is optimized for many-cores/sockets, so surely it
avoids putting a process-wide timer or other unscalable things. It just
links to libprofiler for an optional feature to profile performance
bottleneck only when the user turns it on. We of course avoid turning
the feature on unless while we debug/tune the database.

However, libprofiler sets the timer even when the client program doesn't
invoke any of its functions: libprofiler does it when the shared library
is loaded. We requested the developer of libprofiler to change the
behavior, but seems like there is a reason to keep that behavior:
https://code.google.com/p/gperftools/issues/detail?id=133

Based on this, I think there are two reasons why we should ameliorate
this issue in kernel layer.

1. In the particular case, it's hard to prevent or even detect the issue
in user space.

We (a team of low-level database and kernel experts) in fact spent huge
amount of time to just figure out what's the bottleneck there because
nothing measurable happens in user space. I pulled out countless hairs.

Also, the user has to de-link the library from the application to
prevent the itimer installation. Imagine a case where the software is
proprietary. It won't fly.

2. This is just one example. There could be many other such
binaries/libraries that do similar things somewhere in a complex
software stack.

Today we haven't heard of many such cases, but people will start hitting
it once 100s~1,000s of cores become common.

After applying this patchset, we have observed that the performance hit
almost completely went away at least for 240 cores. So, it's quite
beneficial in real world.

--
Hideaki Kimura