eBPF: how to check the flow table

eBPF: how to check the flow table

[Eric Leblond]

Hello,

I'm currently testing a code implementing AF_PACKET bypass for
Suricata. The idea is that Suricata is updating a hash table containing
a list of flows it does not want to see anymore.

I want to check flow timeout from the userspace, so my current
algorithm is doing:

    while (bpf_get_next_key(mapfd, &key, &next_key) == 0) {
        bpf_lookup_elem(mapfd, &next_key, &value);
        FlowCallback(mapfd, &next_key, &value, data);
        key = next_key;
    }

In the FlowCallback, I check the timing in the flow entry and I remove
the key if the flow is timeout.

This is currently working well when there is only a few flows but on a
real system with log of insertion in the table, the loop is never
returning because we dequeue slower than we enqueue.

Is there a better algorithm or an other way to do it ? 

BR,
-- 
Eric Leblond <eric@regit.org>
Blog: https://home.regit.org/

Re: eBPF: how to check the flow table

[Eric Leblond]

Hello,

On Sun, 2016-09-18 at 14:02 +0200, Eric Leblond wrote:
> Hello,
> 
> I'm currently testing a code implementing AF_PACKET bypass for
> Suricata. The idea is that Suricata is updating a hash table
> containing
> a list of flows it does not want to see anymore.
> 
> I want to check flow timeout from the userspace, so my current
> algorithm is doing:
> 
>     while (bpf_get_next_key(mapfd, &key, &next_key) == 0) {
>         bpf_lookup_elem(mapfd, &next_key, &value);
>         FlowCallback(mapfd, &next_key, &value, data);
>         key = next_key;
>     }
> 
> In the FlowCallback, I check the timing in the flow entry and I
> remove
> the key if the flow is timeout.
> 
> This is currently working well when there is only a few flows but on
> a
> real system with log of insertion in the table, the loop is never
> returning because we dequeue slower than we enqueue.
> 
> Is there a better algorithm or an other way to do it ? 

It seems I missed an obvious race condition in my existing code. I'll
continue to test and relive this thread if necessary.

BR,
-- 
Eric Leblond <eric@regit.org>

Re: eBPF: how to check the flow table

[Alexei Starovoitov]

On Mon, Sep 19, 2016 at 08:56:55AM +0200, Eric Leblond wrote:
> Hello,
> 
> On Sun, 2016-09-18 at 14:02 +0200, Eric Leblond wrote:
> > Hello,
> > 
> > I'm currently testing a code implementing AF_PACKET bypass for
> > Suricata. The idea is that Suricata is updating a hash table
> > containing
> > a list of flows it does not want to see anymore.
> > 
> > I want to check flow timeout from the userspace, so my current
> > algorithm is doing:
> > 
> >     while (bpf_get_next_key(mapfd, &key, &next_key) == 0) {
> >         bpf_lookup_elem(mapfd, &next_key, &value);
> >         FlowCallback(mapfd, &next_key, &value, data);
> >         key = next_key;
> >     }
> > 
> > In the FlowCallback, I check the timing in the flow entry and I
> > remove
> > the key if the flow is timeout.
> > 
> > This is currently working well when there is only a few flows but on
> > a
> > real system with log of insertion in the table, the loop is never
> > returning because we dequeue slower than we enqueue.
> > 
> > Is there a better algorithm or an other way to do it ? 
> 
> It seems I missed an obvious race condition in my existing code. I'll
> continue to test and relive this thread if necessary.

On tracing side we do similar loops all the time.
yes they're racy by design, since kernel always have to proceed.
User space have to be more careful with iteration over the map
to make sure that get_next actually moves forward.
It's indeed possible to custom craft a situation where user space
will forever loop.
Here is one:
https://github.com/iovisor/bcc/issues/665