Re: Scalability of interface creation and deletion

Alex Bligh
2011-05-08T13:06:10Z

--On 8 May 2011 05:44:28 -0700 "Paul E. McKenney"
<paulmck@linux.vnet.ibm.com> wrote:

> OK, let's break it out...
>
> 4.022 seconds for 100 interfaces means about 40 milliseconds per
> interface.

It's 200 interfaces as they are veth pairs, so 20ms.

> My guess is that you have CONFIG_NO_HZ=y, which means that RCU needs to
> figure out that various CPUs are in dyntick-idle state, which is a minimum
> of 6 jiffies.

That sounds like the main cause.

> It could be longer if a given CPU happens to be in IRQ
> when RCU checks, so call it 9 jiffies.

That sounds unlikely to happen much of the time. Looking at /proc/interrupts
not much is going on.

> If you are doing the interfaces
> synchronously, you will likely have to wait for a prior grace period (due
> to background activity).

Not sure I understand that (there's nothing else going on), but...

> So I can easily imagine 18 milliseconds for
> HZ=1000. 40 milliseconds sounds a bit high, but perhaps not impossible.

Add to that the fact that there are 2 x synchronize_sched() and one
rcu_barrier() per interface removal. If I ignore your IRQ idea, that's
6 x 3 = 18ms per interface removal at CONFIG_HZ=10, 180ms at
CONFIG_HZ=100.

--
Alex Bligh