Re: Expiring flows in an XDP based router

["Toke Høiland-Jørgensen" <toke@redhat.com>]

Edvard Fagerholm <edvard.fagerholm@gmail.com> writes:

> Hi,
>
> I'm working on an in-house router and I'm looking for some advice on
> whether to try to implement some functionality directly in an XDP
> program or instead use AF_XDP sockets with the additional flexibility
> provided by userspace.
>
> We basically have a flow table, which contains entries of the form:
>
>   (source ip, source port) -> (action, timestamp)
>
> Every time we see a packet belonging to a flow, the timestamp is
> updated and if the flow has been inactive for e.g. 10 seconds, the
> flow is considered inactive and should be deleted. Actions are of the
> form:
>
>   "forward packet to a.b.c.d port X using source port Y"
>
> The challenge that I have is how to clean up expired flows. Built-in
> options would be BPF_MAP_TYPE_LRU_PERCPU_HASH. However, dropping an
> active flow would be unacceptable.
>
> I'm looking at at most 10k new flow entries being added per second per
> router with a maximum number of concurrent flows at around 256k. Each
> flow sends a packet at least every 5 seconds, but most every 50ms.
> Does this allow me to tune the table size in such a way that no active
> flows can be evicted? If not, are there any other reasonable
> approaches for cleaning up the flows?

Well, hard to tell with only the number of additions. I mean, 256.000 +
10.000 * 10 seconds means you'd need on the order of 356k entries in the
map, assuming flows expire at the same rate as they are added. Do you
assume any such guarantee?

Alternatively, you could have your XDP program start a BPF timer and
perform the periodic cleanup from that. There will be some overhead from
walking the map, and a potential for racy behaviour if the same flow is
expired and re-added frequently, but otherwise this should be quite
doable.

In any case, you'd need to size your map so that it doesn't fill up
before flows expire. So whether to use an LRU map is a case of whether
overflowing the map will result in overwriting existing entries, or
blocking the addition of new ones.

-Toke