Re: [PATCH 1/2] bpf: add table ID to bpf_fib_lookup BPF helper

[Louis DeLosSantos <louis.delos.devel@gmail.com>]

On Thu, May 25, 2023 at 11:48:12PM -0700, John Fastabend wrote:
> Louis DeLosSantos wrote:
> > Add ability to specify routing table ID to the `bpf_fib_lookup` BPF
> > helper.
> > 
> > A new field `tbid` is added to `struct bpf_fib_lookup` used as
> > parameters to the `bpf_fib_lookup` BPF helper.
> > 
> > When the helper is called with the `BPF_FIB_LOOKUP_DIRECT` flag and the
> > `tbid` field in `struct bpf_fib_lookup` is greater then 0, the `tbid`
> > field will be used as the table ID for the fib lookup.
> > 
> > If the `tbid` does not exist the fib lookup will fail with
> > `BPF_FIB_LKUP_RET_NOT_FWDED`.
> > 
> > The `tbid` field becomes a union over the vlan related output fields in
> > `struct bpf_fib_lookup` and will be zeroed immediately after usage.
> > 
> > This functionality is useful in containerized environments.
> > 
> > For instance, if a CNI wants to dictate the next-hop for traffic leaving
> > a container it can create a container-specific routing table and perform
> > a fib lookup against this table in a "host-net-namespace-side" TC program.
> > 
> > This functionality also allows `ip rule` like functionality at the TC
> > layer, allowing an eBPF program to pick a routing table based on some
> > aspect of the sk_buff.
> > 
> > As a concrete use case, this feature will be used in Cilium's SRv6 L3VPN
> > datapath.
> > 
> > When egress traffic leaves a Pod an eBPF program attached by Cilium will
> > determine which VRF the egress traffic should target, and then perform a
> > FIB lookup in a specific table representing this VRF's FIB.
> > 
> > Signed-off-by: Louis DeLosSantos <louis.delos.devel@gmail.com>
> > ---
> >  include/uapi/linux/bpf.h       | 17 ++++++++++++++---
> >  net/core/filter.c              | 12 ++++++++++++
> >  tools/include/uapi/linux/bpf.h | 17 ++++++++++++++---
> >  3 files changed, 40 insertions(+), 6 deletions(-)
> > 
> 
> Looks good one question. Should we hide tbid behind a flag we have
> lots of room. Is there any concern a user could feed a bpf_fib_lookup
> into the helper without clearing the vlan fields? Perhaps by
> pulling the struct from a map or something where it had been
> previously used.
> 
> Thanks,
> John

This is a fair point. 

I could imagine a scenario where an individual is caching bpf_fib_lookup structs,
pulls in a kernel with this change, and is now accidentally feeding the stale vlan
fields as table ID's, since their code is using `BPF_FIB_LOOKUP_DIRECT` with
the old semantics. 

Guarding with a new flag like this (just a quick example, not a full diff)...

```
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 2096fbb328a9b..22095ccaaa64d 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -6823,6 +6823,7 @@ enum {
        BPF_FIB_LOOKUP_DIRECT  = (1U << 0),
        BPF_FIB_LOOKUP_OUTPUT  = (1U << 1),
        BPF_FIB_LOOKUP_SKIP_NEIGH = (1U << 2),
+       BPF_FIB_LOOKUP_TBID    = (1U << 3),
 };
 
 enum {
diff --git a/net/core/filter.c b/net/core/filter.c
index 6f710aa0a54b3..9b78460e39af2 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5803,7 +5803,7 @@ static int bpf_ipv4_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
                u32 tbid = l3mdev_fib_table_rcu(dev) ? : RT_TABLE_MAIN;
                struct fib_table *tb;
 
-               if (params->tbid) {
+               if (flags & BPF_FIB_LOOKUP_TBID) {
                        tbid = params->tbid;
                        /* zero out for vlan output */
                        params->tbid = 0;
```

Maybe a bit safer, you're right. 

In this case the semantics around `BPF_FIB_LOOKUP_DIRECT` remain exactly the same,
and if we do `flags = BPF_FIB_LOOKUP_DIRECT | BPF_FIB_LOOKUP_TBID`, only then will
the `tbid` field in the incoming params wil be considered. 

If I squint at this, it technically also allows us to consider `tbid=0` as a 
valid table id, since the caller now explicitly opts into it, where previously
table id 0 was not selectable, tho I don't know if there's a *real* use case 
for selecting the `all` table. 

I'm happy to make this change, what are your thoughts?