Maximum MPLS labels on Linux network stack

Maximum MPLS labels on Linux network stack

[Алексей Болдырев]

I watched one forum, there is listed in the properties of one license for Cisco, it says:

Layer 3 VPN • Multi-VRF CE (VRF-lite); requires IP Services Feature license
• MPLS VPN; requires Advanced IP Feature license
• 26 VRFs
• 8192 MPLS labels 

Especially interested in the figure 8192 MPLS Labels.

As I understand it, is it either a limit on the number of labels on the stack or the total number of labels?

In Linux, for example, you can specify a common col- lection of labels through /proc/sys/net/mpls/platforms_labels

Also I would like to know if the openvswitch has a limit of 3 tags in the stack or the total number of MPLS labels that can send?

Re: Maximum MPLS labels on Linux network stack

[David Ahern]

On 5/3/17 11:33 AM, Алексей Болдырев wrote:
> I watched one forum, there is listed in the properties of one license for Cisco, it says:
> 
> Layer 3 VPN • Multi-VRF CE (VRF-lite); requires IP Services Feature license
> • MPLS VPN; requires Advanced IP Feature license
> • 26 VRFs

There is no direct limit on the number of VRFs the kernel allows you to
create. There are indirect ones -- total memory in the system and limits
such as /proc/sys/net/ipv6/route/max_size. By increasing the latter I
have created 4k VRFs in a system.


> • 8192 MPLS labels 
> 
> Especially interested in the figure 8192 MPLS Labels.

8192 labels added in one pass is absurd. There is no reason to support
such a number. With the latest version of the MPLS stack in the kernel
you can add up to 30 labels in a single route. If you want more you have
to either recirculate the packet using routes or recompile the kernel
and increase the memory limit and the number of labels limit.

> 
> As I understand it, is it either a limit on the number of labels on the stack or the total number of labels?
> 
> In Linux, for example, you can specify a common col- lection of labels through /proc/sys/net/mpls/platforms_labels

that just allocates the size of an array which dictates the max label
number for that namespace. The array needs to be converted to a hash
table at some point.

> 
> Also I would like to know if the openvswitch has a limit of 3 tags in the stack or the total number of MPLS labels that can send?
> 

someone familiar with OVS needs to answer that.

Re: Maximum MPLS labels on Linux network stack

[Joe Stringer]

On 3 May 2017 at 11:14, David Ahern <dsahern@gmail.com> wrote:
> On 5/3/17 11:33 AM, Алексей Болдырев wrote:
>> I watched one forum, there is listed in the properties of one license for Cisco, it says:
>>
>> Layer 3 VPN • Multi-VRF CE (VRF-lite); requires IP Services Feature license
>> • MPLS VPN; requires Advanced IP Feature license
>> • 26 VRFs
>
> There is no direct limit on the number of VRFs the kernel allows you to
> create. There are indirect ones -- total memory in the system and limits
> such as /proc/sys/net/ipv6/route/max_size. By increasing the latter I
> have created 4k VRFs in a system.
>
>
>> • 8192 MPLS labels
>>
>> Especially interested in the figure 8192 MPLS Labels.
>
> 8192 labels added in one pass is absurd. There is no reason to support
> such a number. With the latest version of the MPLS stack in the kernel
> you can add up to 30 labels in a single route. If you want more you have
> to either recirculate the packet using routes or recompile the kernel
> and increase the memory limit and the number of labels limit.
>
>>
>> As I understand it, is it either a limit on the number of labels on the stack or the total number of labels?
>>
>> In Linux, for example, you can specify a common col- lection of labels through /proc/sys/net/mpls/platforms_labels
>
> that just allocates the size of an array which dictates the max label
> number for that namespace. The array needs to be converted to a hash
> table at some point.
>
>>
>> Also I would like to know if the openvswitch has a limit of 3 tags in the stack or the total number of MPLS labels that can send?
>>
>
> someone familiar with OVS needs to answer that.

That would be 3 tags in a stack. When we spoke to people involved in
the design and usage of MPLS in practice, we got the impression that
it's very rare for anyone to configure a setup where more than that is
used concurrently on a packet. If you believe the contrary, then I
imagine it's not hard to bump that limit.

There is no limit on which labels can be used from OVS, it's just a
number in an action attached to a flow.

Re: Maximum MPLS labels on Linux network stack

[Алексей Болдырев]

Is it possible to increase this limit in OpenVswitch?

03.05.2017, 23:21, "Joe Stringer" <joe@ovn.org>:
> On 3 May 2017 at 11:14, David Ahern <dsahern@gmail.com> wrote:
>>  On 5/3/17 11:33 AM, Алексей Болдырев wrote:
>>>  I watched one forum, there is listed in the properties of one license for Cisco, it says:
>>>
>>>  Layer 3 VPN • Multi-VRF CE (VRF-lite); requires IP Services Feature license
>>>  • MPLS VPN; requires Advanced IP Feature license
>>>  • 26 VRFs
>>
>>  There is no direct limit on the number of VRFs the kernel allows you to
>>  create. There are indirect ones -- total memory in the system and limits
>>  such as /proc/sys/net/ipv6/route/max_size. By increasing the latter I
>>  have created 4k VRFs in a system.
>>
>>>  • 8192 MPLS labels
>>>
>>>  Especially interested in the figure 8192 MPLS Labels.
>>
>>  8192 labels added in one pass is absurd. There is no reason to support
>>  such a number. With the latest version of the MPLS stack in the kernel
>>  you can add up to 30 labels in a single route. If you want more you have
>>  to either recirculate the packet using routes or recompile the kernel
>>  and increase the memory limit and the number of labels limit.
>>
>>>  As I understand it, is it either a limit on the number of labels on the stack or the total number of labels?
>>>
>>>  In Linux, for example, you can specify a common col- lection of labels through /proc/sys/net/mpls/platforms_labels
>>
>>  that just allocates the size of an array which dictates the max label
>>  number for that namespace. The array needs to be converted to a hash
>>  table at some point.
>>
>>>  Also I would like to know if the openvswitch has a limit of 3 tags in the stack or the total number of MPLS labels that can send?
>>
>>  someone familiar with OVS needs to answer that.
>
> That would be 3 tags in a stack. When we spoke to people involved in
> the design and usage of MPLS in practice, we got the impression that
> it's very rare for anyone to configure a setup where more than that is
> used concurrently on a packet. If you believe the contrary, then I
> imagine it's not hard to bump that limit.
>
> There is no limit on which labels can be used from OVS, it's just a
> number in an action attached to a flow.

Re: Maximum MPLS labels on Linux network stack

[Joe Stringer]

On 3 May 2017 at 14:19, Алексей Болдырев
<devel-net-ne-vlezay80@yandex.ru> wrote:
> Is it possible to increase this limit in OpenVswitch?

Yes.

Re: Maximum MPLS labels on Linux network stack

[Алексей Болдырев]

As I understand it, it's enough to just set the variable in the source
#define FLOW_MAX_MPLS_LABELS 3
on
#define FLOW_MAX_MPLS_LABELS 7
Or is there somehow still pitfalls?

04.05.2017, 00:22, "Joe Stringer" <joe@ovn.org>:
> On 3 May 2017 at 14:19, Алексей Болдырев
> <devel-net-ne-vlezay80@yandex.ru> wrote:
>>  Is it possible to increase this limit in OpenVswitch?
>
> Yes.

Re: Maximum MPLS labels on Linux network stack

[Joe Stringer]

On 3 May 2017 at 14:29, Алексей Болдырев
<devel-net-ne-vlezay80@yandex.ru> wrote:
> As I understand it, it's enough to just set the variable in the source
> #define FLOW_MAX_MPLS_LABELS 3
> on
> #define FLOW_MAX_MPLS_LABELS 7
> Or is there somehow still pitfalls?

You'll probably need to update the BUILD_ASSERT_DECL for the offset of
igmp_group_ip4, and for extra safety increment FLOW_WC_SEQ and inspect
each other place that has BUILD_ASSERT_DECL statements relying on that
FLOW_WC_SEQ. Changing FLOW_WC_SEQ will introduce compiler
warnings/failures so you can step through each location, take a look
if it's relevant and update the BUILD_ASSERT_DECL.

At a glance it seems like even with that bump, OVS userspace is only
detecting support for 3 labels so you might need to update something
kernel-side as well.

Re: Maximum MPLS labels on Linux network stack

[David Ahern]

On 5/3/17 2:21 PM, Joe Stringer wrote:
>> • 8192 MPLS labels
>>
>> Especially interested in the figure 8192 MPLS Labels.

The 8k labels has to be 8k individual routes with a single label (or a
few labels in the stack for the route). In that case you can set
net.mpls.platforms_labels to 10001 and install routes with label values
up to 10000.