Re: XDP_REDIRECT not working in XDP_DRV_MODE with Intel xgbe driver

["Toke Høiland-Jørgensen" <toke@redhat.com>]

Sophia Yoo <sy6@princeton.edu> writes:

> On Mon, Feb 14, 2022 at 1:28 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>>
>> Sophia Yoo <sy6@princeton.edu> writes:
>>
>> > Hello,
>> >
>> > I am relatively new to BPF/XDP, and I’m currently trying to use a TC
>> > egress program to redirect a packet back to ingress, where I have an
>> > XDP program attached. When the XDP program is attached in generic mode
>> > (XDP_SKB_MODE), the redirect occurs properly and the packet is seen on
>> > the ingress of the interface, but when the program is attached in
>> > native mode (XDP_DRV_MODE), the packet never redirects and is just
>> > seen exiting the interface, even though the return code of the
>> > redirect function is “success”.
>>
>> The difference between XDP generic and driver mode is that the generic
>> mode is hooked into the core networking stack whereas driver mode runs
>> in the driver as the very first thing after packets are physically
>> received from the network.
>>
>> This also explains why what you're trying to do doesn't work: the TC
>> hook runs in the core networking stack, and when it does a redirect, the
>> packet does not actually pass through the network driver, it appears
>> further up in the stack, where only the generic XDP hook will see it.
>>
> Thanks for your quick response! I see, I didn't realize that the
> packet was being redirected before it passed through the network
> driver. However, in this scenario, shouldn't the packets "disappear"
> after being redirected from tc egress to xdp ingress? In other words,
> even if the packet doesn't actually show up on the xdp ingress hook in
> driver mode, it should at least not be seen exiting on the egress path
> (e.g., on tcpdump)?

Is it possible the packet is looping? I.e., it goes out eth0, you
redirect it to egress on eth1, it goes all the way up the stack and the
kernel ends up forwarding it back out eth0?

>> So this has nothing to do with the support in the driver; XDP is simply
>> not suitable for what you're trying to do. Why are you trying to do this
>> in the first place? I.e., what's the higher-level use case here?
>
> The higher-level design goal (simplified to what's relevant) is that I
> am trying to perform a TCP 3WHS between the kernel network stack and
> my xdp ingress/tc egress hooks, where some trigger on xdp ingress
> sends a SYN packet to the network stack, and when the network stack
> returns a SYN-ACK packet the egress program redirects the packet to
> the ingress interface, where the XDP program converts the packet to an
> ACK packet, then completing the handshake with the network stack.

It's trying to handshake with itself (on the same box)? Why? Is this a
real use case, or a test for something else? If the latter, have you
considered using a veth pair in going to a different namespace (that
gets the "direction" right)?

> Initially, I had wanted to use an XDP program instead of a TC program
> on the egress path to do the redirect to ingress (which I believe
> would bypass the current issue if both ingress and egress xdp programs
> were attached in driver mode), but as of now it seems that redirect
> from egress to ingress simply isn't supported in XDP, which is why I'm
> using a TC program.

Well, there's also an ingress hook for TC you could look at (or just
stay with generic XDP).

-Toke