From: Serge Hallyn <serge.hallyn@ubuntu.com>
To: Benoit Lourdelet <blourdel@juniper.net>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>,
Stephen Hemminger <stephen@networkplumber.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [RFC][PATCH] iproute: Faster ip link add, set and delete
Date: Thu, 28 Mar 2013 10:04:10 -0500 [thread overview]
Message-ID: <20130328150410.GA22789@sergelap> (raw)
In-Reply-To: <CD7A05CE.7802%blourdel@juniper.net>
Quoting Benoit Lourdelet (blourdel@juniper.net):
> Hello,
>
> My test consists in starting small containers (10MB of RAM ) each. Each
> container has 2x physical VLAN interfaces attached.
Which commands were you using to create/start them?
> lxc.network.type = phys
> lxc.network.flags = up
> lxc.network.link = eth6.3
> lxc.network.name = eth2
> lxc.network.hwaddr = 00:50:56:a8:03:03
> lxc.network.ipv4 = 192.168.1.1/24
> lxc.network.type = phys
> lxc.network.flags = up
> lxc.network.link = eth7.3
> lxc.network.name = eth1
> lxc.network.ipv4 = 2.2.2.2/24
> lxc.network.hwaddr = 00:50:57:b8:00:01
>
>
>
> With initial iproute2 , when I reach around 1600 containers, container
> creation almost stops.It takes at least 20s per container to start.
> With patched iproutes2 , I have started 4000 containers at a rate of 1 per
> second w/o problem. I have 8000 clan interfaces configured on the host (2x
> 4000).
>
>
> Regards
>
> Benoit
>
> On 28/03/2013 14:36, "Serge Hallyn" <serge.hallyn@ubuntu.com> wrote:
>
> >Quoting Eric W. Biederman (ebiederm@xmission.com):
> >> Serge Hallyn <serge.hallyn@ubuntu.com> writes:
> >>
> >> > Quoting Eric W. Biederman (ebiederm@xmission.com):
> >> >> Serge Hallyn <serge.hallyn@ubuntu.com> writes:
> >> >>
> >> >> > Quoting Eric W. Biederman (ebiederm@xmission.com):
> >> >> >> Stephen Hemminger <stephen@networkplumber.org> writes:
> >> >> >>
> >> >> >> > If you need to do lots of operations the --batch mode will be
> >>significantly faster.
> >> >> >> > One command start and one link map.
> >> >> >>
> >> >> >> The problem in this case as I understand it is lots of independent
> >> >> >> operations. Now maybe lxc should not shell out to ip and perform
> >>the
> >> >> >> work itself.
> >> >> >
> >> >> > fwiw lxc uses netlink to create new veths, and picks random names
> >>with
> >> >> > mktemp() ahead of time.
> >> >>
> >> >> I am puzzled where does the slownes in iproute2 come into play?
> >> >
> >> > Benoit originally reported slowness when starting >1500 containers. I
> >> > asked him to run a few manual tests to figure out what was taking the
> >> > time. Manually creating a large # of veths was an obvious test, and
> >> > one which showed poorly scaling performance.
> >>
> >> Apparently iproute is involved somehwere as when he tested with a
> >> patched iproute (as you asked him to) the lxc startup slowdown was
> >> gone.
> >>
> >> > May well be there are other things slowing down lxc of course.
> >>
> >> The evidence indicates it was iproute being called somewhere...
> >
> >Benoit can you tell us exactly what test you were running when you saw
> >the slowdown was gone?
> >
> >-serge
> >
>
>
next prev parent reply other threads:[~2013-03-28 15:04 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-03-22 22:23 [RFC][PATCH] iproute: Faster ip link add, set and delete Eric W. Biederman
2013-03-22 22:27 ` Stephen Hemminger
2013-03-26 11:51 ` Benoit Lourdelet
2013-03-26 12:40 ` Eric W. Biederman
2013-03-26 14:17 ` Serge Hallyn
2013-03-26 14:33 ` Serge Hallyn
2013-03-27 13:37 ` Benoit Lourdelet
2013-03-27 15:11 ` Eric W. Biederman
2013-03-27 17:47 ` Stephen Hemminger
2013-03-28 0:46 ` Eric W. Biederman
2013-03-28 3:20 ` Serge Hallyn
2013-03-28 3:44 ` Eric W. Biederman
2013-03-28 4:28 ` Serge Hallyn
2013-03-28 5:00 ` Eric W. Biederman
2013-03-28 13:36 ` Serge Hallyn
2013-03-28 13:42 ` Benoit Lourdelet
2013-03-28 15:04 ` Serge Hallyn [this message]
2013-03-28 15:21 ` Benoit Lourdelet
2013-03-28 22:20 ` Stephen Hemminger
2013-03-28 23:52 ` Eric W. Biederman
2013-03-29 0:13 ` Eric Dumazet
2013-03-29 0:25 ` Eric W. Biederman
2013-03-29 0:43 ` Eric Dumazet
2013-03-29 1:06 ` Eric W. Biederman
2013-03-29 1:10 ` Eric Dumazet
2013-03-29 1:29 ` Eric W. Biederman
2013-03-29 1:38 ` Eric Dumazet
2013-03-30 10:09 ` Benoit Lourdelet
2013-03-30 14:44 ` Eric Dumazet
2013-03-30 16:07 ` Benoit Lourdelet
2013-03-28 20:27 ` Benoit Lourdelet
2013-03-26 15:31 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130328150410.GA22789@sergelap \
--to=serge.hallyn@ubuntu.com \
--cc=blourdel@juniper.net \
--cc=ebiederm@xmission.com \
--cc=netdev@vger.kernel.org \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).