Re: allocate specific port range for container?

From: ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org (Eric W. Biederman)
To: Hongbing Wang <hongbingwang-rphTv4pjVZMJGwgDXS7ZQA@public.gmane.org>
Cc: "containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org"
	<containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org>,
	Serge Hallyn
	<serge.hallyn-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org>
Subject: Re: allocate specific port range for container?
Date: Fri, 08 Nov 2013 15:40:17 -0800	[thread overview]
Message-ID: <87vc02qx7i.fsf@xmission.com> (raw)
In-Reply-To: <1383690907.99487.YahooMailNeo-abza1nB0wQu2Y7dhQGSVAJOW+3bF1jUfVpNB7YpNyf8@public.gmane.org> (Hongbing Wang's message of "Tue, 5 Nov 2013 14:35:07 -0800 (PST)")

Hongbing Wang <hongbingwang@sbcglobal.net> writes:

> ________________________________
>  From: Serge Hallyn <serge.hallyn@ubuntu.com>
> To: Hongbing Wang <hongbingwang@sbcglobal.net> 
> Cc: "containers@lists.linux-foundation.org" <containers@lists.linux-foundation.org> 
> Sent: Tuesday, November 5, 2013 1:53 PM
> Subject: Re: allocate specific port range for container?
>  
>
> Quoting Hongbing Wang (hongbingwang@sbcglobal.net):
>> Hello LXC experts:
>> 
>> Is it possible to allocate specific transport port range for the socket based applications inside one container? 
>
> How do you mean?  You want ports 50000-51000 of the host to be forwarded
> to container 1, and 60000-61000 to container 2?
>
> [HB] We want the apps inside LXC_a to use the port b/w 50k - 51k and LXC_b to be allocated the port b/w 60k - 61k.
>
>> Say I have two containers: LXC_a and LXC_b, and each has some socket
>> based applications I cannot modify or have no source code. If I need
>> port range 50000 - 51000 for LXC_a and 60000 - 61000 for LXC_b. Any
>> way to achieve this?
>> 
>> The /proc/sys/net/ipv4/ip_local_port_range is for the host to adjust
>> the ephemeral port range, and we do not have the per LXC based
>> setting on the local_port_range. Is this due to that the LXC network
>> namespace isolation is at the L3 level? How could I achieve this per
>> port range LXC?

> The network namespaces are actually at L2, not L3.  Each container has
> its own routing table.
>
> I think you can get what you want by simply giving each container a veth
> nic and using iptables on the host to forward the ports you want to the
> appropriate container.  That's how I co-locate web, mail, and other
> server containers on the same host.
>
> [HB] for my specific case we do not use iptables. We can route the
> packets to the LXC_a or LXC_b, but we want the ports to be within
> specific ranges.

I have been working on this recently.

I have recently made ip_local_port range per netns to help support this.

Beyond that what I am doing is using ingress traffic control to redirect
packets in the appopriate port range into the appropriate network
namespace.

Compared to nat with iptables and some of the other solutions it comes
out fairly clean and it at the very beginning stages at the packet
processing path so unnecessary work can be avoided.

There are a couple of cases you have to watch for icmp, arp and looping
packets back to other namespaces on the same box but it is pretty
straight forward to setup.  The trickiest bit is that you have to read
the tc source to find the tc commands that make the tc classifier easy
to use.

The primary restricition I currently have is that the port ranges need
to be a power of 2 in size, to make the u32 classifier easy to use.

Eric

_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers