Re: [PATCH 1/3] epoll: increase default max_user_instances to 1024

Bron Gondwana
2009-01-28T05:32:18Z

On Tue, Jan 27, 2009 at 08:00:30PM -0800, Davide Libenzi wrote:
> On Wed, 28 Jan 2009, Bron Gondwana wrote:
>
> > Both Postfix and Apache use an epoll instance per child, which
> > leads to significant scalability issues with max_user_instances
> > set so low. Bump the default to 1024 so medium sized sites are
> > not impacted.
>
> NACK. Epoll allocates globally about 100 to 160 bytes (32/64 bit) for each
> file added to the interface:
>
> for i 1..1024
> for j 1..1024
> if i!=j
> add j -> i
>
> That's (N^2 * {100, 160}) = 100MB to 160MB of pinned kernel memory,

Woah - that's serious.

This:

instances_uid 0 (root)
num_instances 142
max_instances 4096
watches_uid 107 (postfix)
num_watches 1097
max_watches 266555

isn't serious. It's pretty sane. 142 processes with an epoll watcher,
and fewer than 10 fds per epoll. Unfortunately, it wouldn't work on an
unpatched and un-specially-configured stock kernel. That's steady-state
too, not a peak. I just grabbed it off a running MX:

[brong@mx1 ~]$ free
total used free shared buffers
cached
Mem: 4151652 3113128 1038524 0 130808
2014152
-/+ buffers/cache: 968168 3183484
Swap: 2047992 50364 1997628
[brong@mx1 ~]$ uptime
00:31:05 up 2 days, 18:03, 2 users, load average: 0.86, 1.23, 1.08

Hardly looking stressed right now.

If I'm reading it right, your concern is the massively recursive case,
where every single epoll gets added to every other epoll as a chained
file descriptor?

That's clearly not happening here - so it seems that maybe our "happy
medium" is actually in closer inspection of what's going on rather than
a blanket low N to keep N^2 down.

Bron.