From: Antonio Ojea <antonio.ojea.garcia@gmail.com>
To: Pablo Neira Ayuso <pablo@netfilter.org>
Cc: netfilter-devel@vger.kernel.org
Subject: Re: [PATCH nf] netfilter: nfnetlink_queue: reroute reinjected packets from postrouting
Date: Tue, 17 Sep 2024 23:01:31 +0100 [thread overview]
Message-ID: <CABhP=taUnE6nxQ1ZPradgk7iNt3M_LCcFoM251OhpEJsasCoSw@mail.gmail.com> (raw)
In-Reply-To: <CABhP=tY2ceRAiZd3UCN3LqU8ZSO1G1W236XW+2rC6QhpeA9dsw@mail.gmail.com>
On Fri, 13 Sept 2024 at 07:24, Antonio Ojea
<antonio.ojea.garcia@gmail.com> wrote:
>
> On Thu, 12 Sept 2024 at 20:58, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> >
> > 368982cd7d1b ("netfilter: nfnetlink_queue: resolve clash for unconfirmed
> > conntracks") adjusts NAT again in case that packet loses race to confirm
> > the conntrack entry.
> >
> > The reinject path triggers a route lookup again for the output hook, but
> > not for the postrouting hook where queue to userspace is also possible.
> >
> > Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> > Reported-by: Antonio Ojea <antonio.ojea.garcia@gmail.com>
> > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > ---
> > I tried but I am not managing to make a selftest that runs reliable.
> > I can reproduce it manually and validate that this works.
> >
> > ./nft_queue -d 1000 helps by introducing a delay of 1000ms in the
> > userspace queue processing which helps trigger the race more easily,
> > socat needs to send several packets in the same UDP flow.
> >
> > @Antonio: Could you try this patch meanwhile there is a testcase for
> > this.
>
> Let me test it and report back
>
Ok, I finally managed to get this tested, and it does not seem to
solve the problem, it keeps dnating twice after the packet is enqueued
by nfqueue
See trace obtained with pwru, origin 10.244.0.3, virtual IP of DNS
server 10.96.0.10 that DNATs to 10.244.0.2 and 10.244.0.4
21:44:13.066 0xffff97ff83662280 0 <empty>:3552 2007043994686
10.244.0.3:39492->10.96.0.10:53(udp) nf_checksum
21:44:13.066 0xffff97ff83662280 0 <empty>:3552 2007043995059
10.244.0.3:39492->10.96.0.10:53(udp) nf_ip_checksum
21:44:13.066 0xffff97ff83662280 0 <empty>:3552 2007043995538
10.244.0.3:39492->10.96.0.10:53(udp) nf_nat_ipv4_pre_routing
21:44:13.066 0xffff97ff83662280 0 <empty>:3552 2007043995957
10.244.0.3:39492->10.96.0.10:53(udp) nf_nat_inet_fn
21:44:13.066 0xffff97ff83662280 0 <empty>:3552 2007043996439
10.244.0.3:39492->10.96.0.10:53(udp) nft_nat_do_chain
21:44:13.067 0xffff97ff83662280 0 <empty>:3552 2007043999827
10.244.0.3:39492->10.96.0.10:53(udp) xt_dnat_target_v2
21:44:13.067 0xffff97ff83662280 0 <empty>:3552 2007044000721
10.244.0.3:39492->10.96.0.10:53(udp) nf_nat_manip_pkt
21:44:13.067 0xffff97ff83662280 0 <empty>:3552 2007044023444
10.244.0.3:39492->10.96.0.10:53(udp) nf_nat_ipv4_manip_pkt
21:44:13.067 0xffff97ff83662280 0 <empty>:3552 2007044024162
10.244.0.3:39492->10.96.0.10:53(udp) skb_ensure_writable
21:44:13.068 0xffff97ff83662280 0 <empty>:3552 2007044024819
10.244.0.3:39492->10.96.0.10:53(udp) l4proto_manip_pkt
21:44:13.068 0xffff97ff83662280 0 <empty>:3552 2007044025158
10.244.0.3:39492->10.96.0.10:53(udp) skb_ensure_writable
21:44:13.068 0xffff97ff83662280 0 <empty>:3552 2007044025711
10.244.0.3:39492->10.96.0.10:53(udp) nf_csum_update
21:44:13.068 0xffff97ff83662280 0 <empty>:3552 2007044026381
10.244.0.3:39492->10.96.0.10:53(udp) inet_proto_csum_replace4
21:44:13.068 0xffff97ff83662280 0 <empty>:3552 2007044026730
10.244.0.3:39492->10.96.0.10:53(udp) inet_proto_csum_replace4
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044027433
10.244.0.3:39492->10.244.0.2:53(udp) ip_rcv_finish
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044028188
10.244.0.3:39492->10.244.0.2:53(udp) udp_v4_early_demux
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044029235
10.244.0.3:39492->10.244.0.2:53(udp) ip_route_input_noref
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044029696
10.244.0.3:39492->10.244.0.2:53(udp) ip_route_input_slow
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044030986
10.244.0.3:39492->10.244.0.2:53(udp) fib_validate_source
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044031571
10.244.0.3:39492->10.244.0.2:53(udp) __fib_validate_source
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044032576
10.244.0.3:39492->10.244.0.2:53(udp) ip_forward
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044033236
10.244.0.3:39492->10.244.0.2:53(udp) nf_hook_slow
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044034004
10.244.0.3:39492->10.244.0.2:53(udp) selinux_ip_forward
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044034601
10.244.0.3:39492->10.244.0.2:53(udp) nft_do_chain_ipv4
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044037452
10.244.0.3:39492->10.244.0.2:53(udp) ip_output
21:44:13.069 0xffff97ff83662280 0 <empty>:3552 2007044037796
10.244.0.3:39492->10.244.0.2:53(udp) nf_hook_slow
21:44:13.070 0xffff97ff83662280 0 <empty>:3552 2007044038241
10.244.0.3:39492->10.244.0.2:53(udp) nft_do_chain_inet
21:44:13.070 0xffff97ff83662280 0 <empty>:3552 2007044040343
10.244.0.3:39492->10.244.0.2:53(udp) nf_queue
--- snipped other skbs ---
21:44:13.149 0xffff97ff83662280 0 <empty>:1463 2007052515236
10.244.0.3:39492->10.244.0.2:53(udp) nf_conntrack_update
21:44:13.149 0xffff97ff83662280 0 <empty>:1463 2007052538616
10.244.0.3:39492->10.244.0.2:53(udp) nf_nat_manip_pkt
21:44:13.149 0xffff97ff83662280 0 <empty>:1463 2007052539511
10.244.0.3:39492->10.244.0.2:53(udp) nf_nat_ipv4_manip_pkt
21:44:13.150 0xffff97ff83662280 0 <empty>:1463 2007052540123
10.244.0.3:39492->10.244.0.2:53(udp) skb_ensure_writable
21:44:13.150 0xffff97ff83662280 0 <empty>:1463 2007052540589
10.244.0.3:39492->10.244.0.2:53(udp) l4proto_manip_pkt
21:44:13.150 0xffff97ff83662280 0 <empty>:1463 2007052540875
10.244.0.3:39492->10.244.0.2:53(udp) skb_ensure_writable
21:44:13.150 0xffff97ff83662280 0 <empty>:1463 2007052541326
10.244.0.3:39492->10.244.0.2:53(udp) nf_csum_update
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052541944
10.244.0.3:39492->10.244.0.2:53(udp) inet_proto_csum_replace4
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052542259
10.244.0.3:39492->10.244.0.2:53(udp) inet_proto_csum_replace4 <<<<
DNATed twice
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052543321
10.244.0.3:39492->10.244.0.4:53(udp) ip_route_me_harder
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052545374
10.244.0.3:39492->10.244.0.4:53(udp) __xfrm_decode_session
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052546324
10.244.0.3:39492->10.244.0.4:53(udp) nf_nat_ipv4_out
21:44:13.151 0xffff97ff83662280 0 <empty>:1463 2007052546676
10.244.0.3:39492->10.244.0.4:53(udp) nf_nat_inet_fn
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052547186
10.244.0.3:39492->10.244.0.4:53(udp) selinux_ip_postroute
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052547732
10.244.0.3:39492->10.244.0.4:53(udp) selinux_ip_postroute_compat
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052548217
10.244.0.3:39492->10.244.0.4:53(udp) nf_confirm
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052548744
10.244.0.3:39492->10.244.0.4:53(udp) ip_finish_output
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052549162
10.244.0.3:39492->10.244.0.4:53(udp) __ip_finish_output
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052549614
10.244.0.3:39492->10.244.0.4:53(udp) ip_finish_output2
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052550159
10.244.0.3:39492->10.244.0.4:53(udp) __dev_queue_xmit
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052550656
10.244.0.3:39492->10.244.0.4:53(udp) netdev_core_pick_tx
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052551436
10.244.0.3:39492->10.244.0.4:53(udp) validate_xmit_skb
21:44:13.152 0xffff97ff83662280 0 <empty>:1463 2007052551882
10.244.0.3:39492->10.244.0.4:53(udp) netif_skb_features
21:44:13.153 0xffff97ff83662280 0 <empty>:1463 2007052552291
10.244.0.3:39492->10.244.0.4:53(udp) passthru_features_check
21:44:13.153 0xffff97ff83662280 0 <empty>:1463 2007052552672
10.244.0.3:39492->10.244.0.4:53(udp) skb_network_protocol
21:44:13.153 0xffff97ff83662280 0 <empty>:1463 2007052553191
10.244.0.3:39492->10.244.0.4:53(udp) skb_csum_hwoffload_help
21:44:13.154 0xffff97ff83662280 0 <empty>:1463 2007052553566
10.244.0.3:39492->10.244.0.4:53(udp) validate_xmit_xfrm
21:44:13.155 0xffff97ff83662280 0 <empty>:1463 2007052554026
10.244.0.3:39492->10.244.0.4:53(udp) dev_hard_start_xmit
21:44:13.155 0xffff97ff83662280 0 <empty>:1463 2007052554482
10.244.0.3:39492->10.244.0.4:53(udp) veth_xmit
21:44:13.155 0xffff97ff83662280 0 <empty>:1463 2007052555156
10.244.0.3:39492->10.244.0.4:53(udp) __dev_forward_skb
21:44:13.155 0xffff97ff83662280 0 <empty>:1463 2007052555604
10.244.0.3:39492->10.244.0.4:53(udp) __dev_forward_skb2
21:44:13.155 0xffff97ff83662280 0 <empty>:1463 2007052556045
10.244.0.3:39492->10.244.0.4:53(udp) skb_scrub_packet
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052556449
10.244.0.3:39492->10.244.0.4:53(udp) eth_type_trans
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052557536
10.244.0.3:39492->10.244.0.4:53(udp) __netif_rx
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052559424
10.244.0.3:39492->10.244.0.4:53(udp) netif_rx_internal
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052559872
10.244.0.3:39492->10.244.0.4:53(udp) enqueue_to_backlog
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052560827
10.244.0.3:39492->10.244.0.4:53(udp) __netif_receive_skb_one_core
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052561410
10.244.0.3:39492->10.244.0.4:53(udp) ip_rcv
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052561845
10.244.0.3:39492->10.244.0.4:53(udp) ip_rcv_core
21:44:13.156 0xffff97ff83662280 0 <empty>:1463 2007052564056
10.244.0.3:39492->10.244.0.4:53(udp)
kfree_skb_reason(SKB_DROP_REASON_OTHERHOST)
> > net/netfilter/nfnetlink_queue.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
> > index e0716da256bf..aeb354271e85 100644
> > --- a/net/netfilter/nfnetlink_queue.c
> > +++ b/net/netfilter/nfnetlink_queue.c
> > @@ -276,7 +276,8 @@ static int nf_ip_reroute(struct sk_buff *skb, const struct nf_queue_entry *entry
> > #ifdef CONFIG_INET
> > const struct ip_rt_info *rt_info = nf_queue_entry_reroute(entry);
> >
> > - if (entry->state.hook == NF_INET_LOCAL_OUT) {
> > + if (entry->state.hook == NF_INET_LOCAL_OUT ||
> > + entry->state.hook == NF_INET_POST_ROUTING) {
> > const struct iphdr *iph = ip_hdr(skb);
> >
> > if (!(iph->tos == rt_info->tos &&
> > --
> > 2.30.2
> >
next prev parent reply other threads:[~2024-09-17 22:02 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-12 18:58 [PATCH nf] netfilter: nfnetlink_queue: reroute reinjected packets from postrouting Pablo Neira Ayuso
2024-09-13 6:24 ` Antonio Ojea
2024-09-17 22:01 ` Antonio Ojea [this message]
2024-09-18 8:30 ` Pablo Neira Ayuso
2024-09-18 8:42 ` Florian Westphal
2024-09-18 9:51 ` Antonio Ojea
2024-09-18 9:54 ` Florian Westphal
2024-09-18 20:53 ` Pablo Neira Ayuso
2024-09-18 21:42 ` Antonio Ojea
2024-09-19 12:02 ` Pablo Neira Ayuso
2024-10-06 14:44 ` Antonio Ojea
2024-10-07 8:14 ` Antonio Ojea
2024-10-07 8:30 ` Pablo Neira Ayuso
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABhP=taUnE6nxQ1ZPradgk7iNt3M_LCcFoM251OhpEJsasCoSw@mail.gmail.com' \
--to=antonio.ojea.garcia@gmail.com \
--cc=netfilter-devel@vger.kernel.org \
--cc=pablo@netfilter.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).