nft tproxy without iproute2 rule

nft tproxy without iproute2 rule

[Norman Rasmussen]

If there was a way to set the output interface in the prerouting and
output hooks, would the meta mark action and ip rule & route still be
required?

The netdev fwd statement can be used to transmit packets via a given
interface, but is only supported in the ingress hook. Would supporting
setting the output interface it for inet hooks just be a matter of
supporting the oif and oifname keywords for the "meta set" statement?
(adding NFT_META_OIF and NFT_META_OIFNAME to nft_meta_set_init and
nft_meta_set_eval)

For example (taking from Documentation/networking/tproxy.txt) the
config becomes:

# nft add chain filter divert "{ type filter hook prerouting priority -150; }"
# nft add rule filter divert meta l4proto tcp socket transparent 1 fwd
to lo accept
# nft add rule filter divert tcp dport 80 tproxy to :50080 meta set
oif lo accept

and for local connections:
# nft add chain filter local "{ type filter hook output priority -150; }"
# nft add rule filter local tcp dport 80 tproxy to :50080 meta set oif
to lo accept

Is there a complication with supporting this in prerouting and output,
or would this mostly just work as expected? I assume it wouldn't need
to do the same as the netdev fwd path where the packet is immediately
sent to egress and xmit, and just doing the far simpler set skb->dev
and let the rest of the processing continue would work fine?

For example in nft_meta_set_eval's switch:

  case NFT_META_OIF:
    dev = dev_get_by_index_rcu(nft_net(pkt), value);
    if (!dev) {
      kfree_skb(pkt->skb);
      return;
    }
    // does this need updating?
    nft_out(pkt)->ifindex = value
    // is this the right way to change the output device?
    skb->dst = dev
    break;
  NFT_META_OIFNAME:
    // something similar to above but using dev_get_by_name_rcu?

Thoughts?

-- 
- Norman Rasmussen
 - Email: norman@rasmussen.co.za
 - Home page: http://norman.rasmussen.co.za/

Re: nft tproxy without iproute2 rule

[Florian Westphal]

Norman Rasmussen <norman@rasmussen.co.za> wrote:
> If there was a way to set the output interface in the prerouting and
> output hooks, would the meta mark action and ip rule & route still be
> required?

Yes.

> The netdev fwd statement can be used to transmit packets via a given
> interface, but is only supported in the ingress hook. Would supporting
> setting the output interface it for inet hooks just be a matter of
> supporting the oif and oifname keywords for the "meta set" statement?

No.

> For example (taking from Documentation/networking/tproxy.txt) the
> config becomes:
> 
> # nft add chain filter divert "{ type filter hook prerouting priority -150; }"
> # nft add rule filter divert meta l4proto tcp socket transparent 1 fwd
> to lo accept
> # nft add rule filter divert tcp dport 80 tproxy to :50080 meta set
> oif lo accept

The purpose of mark + rule + route is to influence the routing decision
made after the prerouting step, so packet gets passed to input path and
not forward.

skb->dev is the input device at this point.

The code that is being influenced by mark + route is
the call to ip_route_input_noref() in ip_rcv_finish_core which assigns
skb->dst, and the dst_entry describes how the packet will travel in the stack.

> and for local connections:
> # nft add chain filter local "{ type filter hook output priority -150; }"
> # nft add rule filter local tcp dport 80 tproxy to :50080 meta set oif
> to lo accept

> Is there a complication with supporting this in prerouting and output,
> or would this mostly just work as expected? I assume it wouldn't need
> to do the same as the netdev fwd path where the packet is immediately
> sent to egress and xmit, and just doing the far simpler set skb->dev
> and let the rest of the processing continue would work fine?

See above, skb->dev is still telling the stack where the packet is
coming from, not where its going.

Is this about ip rule scalability or keeping config 'private' to nftables,
i.e. bypass the FIB?

Re: nft tproxy without iproute2 rule

[Norman Rasmussen]

> See above, skb->dev is still telling the stack where the packet is
> coming from, not where its going.

I think I meant skb->dst, or skb->dst->dev (I seem to have gotten
confused about the variables in the code snippet).

> Is this about ip rule scalability or keeping config 'private' to nftables,
>
> i.e. bypass the FIB?

It's mostly to keep the config in one place. I was thinking for tools
like sshuttle it would be nice if they only had to mutate nftables,
and didn't have to worry about also setting up (or requiring the user
to set up) the ip rules and routes.

It also feels like marking the packets and creating a rule & route
just to route the packets to the local interface is a bit unnecessary.
It would be simpler to just have an action that could route them to
the local interface, or even directly to the listening socket.

The tproxy action calls nf_tproxy_assign_sock if the socket is
transparent, why not finish setting up delivery so that the rule and
route are not required? Is there something else that could happen to a
packet once the tproxy action has assigned skb->sk?

--
- Norman Rasmussen
- Email: norman@rasmussen.co.za
- Home page: http://norman.rasmussen.co.za/