[LARTC] I know there must be a way ...

[LARTC] I know there must be a way ...

[George Bonser]

Looking through all the advanced routing docs I can find, I think I have
dug up enough parts to cobble something together to do what I need but I
am befuddled. Here is a description of my problem:

Two providers. A primary I will call provider-A and a backup that I will
call provider-B. I collect full routes from both by BGP. My aggregate
traffic output varies from about 130MB in the middle of the night up to
about 300MB during the day ... a little lower on the weekends. Provider-B
is more expensive and has a 50MB minimum. I have fiddled with my BGP so
that I end up sending about 45-50MB of traffic to provider-B during my
peak time of the day.  What I would like to do is pretty much nail
provider-B to 50MB at all times using a Linux box in the traffic path.

A bit more detail on what I am trying to do:

A packet arriving from inside my network has 4 possible dispositions.

1. There is a route to the destination from both providers (most likely).
2. There is a route only from Provider-A.
3. There is a route only from Provider-B.
4. There is no route from either provider.

I can make zebra put routes into realms. I can then check arriving packets
to see if a realm has a route to the destination. Packets in disposition 2
must go to provider-A, packets in disposition 3 must go to provider-B.
Packets in disposition 1 are what I call "the pool" and may go to either A
or B to get to their distination.

What I want to do is create three streams ... A, B, and Pool. I need to
mark A so that it gets routed to provider-A (with FWMARK or some other
means ... say TOS), mark stream B so that it is nailed to provider B, BUT
when stream B is below 50MB, I want to pull in packets from the pool to
bring it up to 50. I do NOT want to rate-limit at 50 because if I loose my
link to provider-A or they have a peering issue, more than 50MB might need
to go to B, I just want to stop pulling traffic from the pool at that
point.  Any traffic in the pool remaining after B has pulled what it wants
would be marked for provider-A.

What I end up with is something like this:

Traffic that can only be routed to A goes to A
Traffic that can only be routed to B goes to B
If B is less than 50MB, get some of the traffic that could go to either A
or B to make up the difference
Remaining traffic goes to A.

Is anyone here ever done this?




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] I know there must be a way ...

[bert hubert]

On Tue, Dec 11, 2001 at 02:10:12PM -0800, George Bonser wrote:

> Two providers. A primary I will call provider-A and a backup that I will
> call provider-B. I collect full routes from both by BGP. My aggregate
> traffic output varies from about 130MB in the middle of the night up to
> about 300MB during the day ... a little lower on the weekends. Provider-B
> is more expensive and has a 50MB minimum. I have fiddled with my BGP so
> that I end up sending about 45-50MB of traffic to provider-B during my
> peak time of the day.  What I would like to do is pretty much nail
> provider-B to 50MB at all times using a Linux box in the traffic path.

So you send 300mbit/s? Wow.

> A bit more detail on what I am trying to do:
> 
> A packet arriving from inside my network has 4 possible dispositions.
> 
> 1. There is a route to the destination from both providers (most likely).
> 2. There is a route only from Provider-A.
> 3. There is a route only from Provider-B.
> 4. There is no route from either provider.
> 
> I can make zebra put routes into realms. I can then check arriving packets
> to see if a realm has a route to the destination. Packets in disposition 2
> must go to provider-A, packets in disposition 3 must go to provider-B.
> Packets in disposition 1 are what I call "the pool" and may go to either A
> or B to get to their distination.

I get this.

> What I want to do is create three streams ... A, B, and Pool. I need to
> mark A so that it gets routed to provider-A (with FWMARK or some other
> means ... say TOS), mark stream B so that it is nailed to provider B, BUT
> when stream B is below 50MB, I want to pull in packets from the pool to
> bring it up to 50. I do NOT want to rate-limit at 50 because if I loose my
> link to provider-A or they have a peering issue, more than 50MB might need
> to go to B, I just want to stop pulling traffic from the pool at that
> point.  Any traffic in the pool remaining after B has pulled what it wants
> would be marked for provider-A.

Hmm. Hmmm. I think this can be done! 

You should attach an ingress shaper to all interfaces that receive traffic.
I hope this is only one, or you will be in trouble. To attach an ingress
shaper to multiple interfaces, you need it loaded as a module, btw,
otherwise Bad Things will happen.

Now, you must use a policing filter that will tag all traffic below
50mbit/s. Later on, you will use this tag to route with. All traffic above
gets no tag, or another tag.

The policing filter isn't the hard part, see the 'synflood' section in the
HOWTO. The hard part is placing the mark. I think DSMARK does what you want.
Later on, you must find a way to route based on the tc_index, as set by
DSMARK. Perhaps by matching on it in iptables FORWARD and replacing it with
an fwmark.

I'm not sure. But I think the answer lies in DSMARK. Otherwise I can whip up
a tc filter that does fwmark directly. Let me know.

[ Ok, I reread what you want and it's a bit different, but these are the
  tools available. You may need to create lots and lots of rules because you
  need to be able to tell at ingress time where a packet is going to be
  sent.  Tc filters can be hashed & are then lighting fast. ]

If you get this working you can give a talk about it on whatever routing
conference you choose, btw.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] I know there must be a way ...

[bert hubert]

On Tue, Dec 11, 2001 at 02:10:12PM -0800, George Bonser wrote:

> dug up enough parts to cobble something together to do what I need but I
> am befuddled. Here is a description of my problem:
> 
> Two providers. A primary I will call provider-A and a backup that I will
> call provider-B. I collect full routes from both by BGP. My aggregate
> traffic output varies from about 130MB in the middle of the night up to
> about 300MB during the day ... a little lower on the weekends. Provider-B
> is more expensive and has a 50MB minimum. I have fiddled with my BGP so
> that I end up sending about 45-50MB of traffic to provider-B during my
> peak time of the day.  What I would like to do is pretty much nail
> provider-B to 50MB at all times using a Linux box in the traffic path.

Ok - I missed this. So we have this
                                                       A
                                                     /  
  [ your network ]  - [ linux machine ] - [ router ] 
                                                     \ 
                                                       B

The Linux machine also has a full view and knows where traffic will go. 

But the router does its own routing? 

> A bit more detail on what I am trying to do:
> 
> A packet arriving from inside my network has 4 possible dispositions.
> 
> 1. There is a route to the destination from both providers (most likely).

Should be so yes. Your router routes on AS path length however.

> I can make zebra put routes into realms. I can then check arriving packets
> to see if a realm has a route to the destination. Packets in disposition 2
> must go to provider-A, packets in disposition 3 must go to provider-B.
> Packets in disposition 1 are what I call "the pool" and may go to either A
> or B to get to their distination.

Well, tricky! Even if the Linux box knows, it should communicate this to the
router. Perhaps you can do measurements in the FORWARD chain and reconfigure
your router based on those measurements?

The linux machine only knows at egress where traffic will go, by then it's
really too late to do anything about it, except possibly DSMARK stuff.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] I know there must be a way ...

[George Bonser]

> Ok - I missed this. So we have this
>                                                        A
>                                                      /
>   [ your network ]  - [ linux machine ] - [ router ]
>                                                      \

Or possibly this:                       A
                                      /
    [ my network ] - [ linux machine ]
                                      \
                                        B

> The Linux machine also has a full view and knows where traffic will go.
>
> But the router does its own routing?

I currently do not have a linux box in the path. I have two GigE links to
Provider-A and a GigE link to provider-B.  Each GigE terminates on an
Extreme Summit 1i layer 2/3 switch. What I currently do is have each of
these units peer with internal BGP to a linux box running Zebra. I use it
to set the policies and aggregate everything into one single routing table
that I pass to a router just outside the firewall.  The effect there is
to provide that router with the best next hop for traffic so it goes to
the  router associated with the egress link without having to peer with
all  three of the routers. The linux box acts as a route server but is not
in the traffic path.

But enough of that ...

What I really need is a different kind of policing filter. Rather than one
that sets a maximum bandwidth, I need one where I can set a MINIMUM
bandwidth and something that will pull traffic from another queue to keep
it at that minumim

OR

A filter that can query ANOTHER queue's rate monitor and reclassify
traffic into that queue whenever the target queue's traffic rate is below
some configurable number.

It is probably going to involve writing something new. I really do not see
a way to do it yet with the pieces I see.  I am going to do some more
reading over the weekend and see what I can come up with before I start
hacking on stuff, I really hate reinventing wheels.



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/