[regression] Completely editing packet should be allowed.

[regression] Completely editing packet should be allowed.

[allfox wy]

Greetings netfilter people.

I have a homebrew cipher program using libnetfilter_queue, which would
intercept certain IP in IP packets via iptables' NFQUEUE target, edit
them ( completely ) then put them back to OS for further processing.

Received packets would be redirected via RAW table ( which would
happen before CONNTRACK ) , while sending packets would be redirected
via MANGLE table POSTROUTING chain ( they are leaving the host, no
longer need to be CONTRACKed ).

The edit is to transform the entire inner packet with a cipher. By
entire, I mean including the inner header hidden and the original
length would change.
The receiving host would decipher it back to normal IP in IP packet in
RAW table.

This would form a transparent tunnel and had worked for years.

However recently I tried to switch from Linux 6.18 LTS to 7.4-rc1 ( in
favour of tcp_roccet ), my homebrew stopped working.
There is no error message anywhere, all libnetfilter_queue functions
return success, I could see the packet, I could edit it, I could send
the verdict. Just after my edit, the packet would fade into void
somehow. Like the host never had it.

So I did a bisect, it comes to
112e447d17f78d08485b3076a8baa1dfb9794bd8 is the first bad commit:
- netfilter: validate L4 headers after userspace packet writes.

I don't really get the idea of the situation here:
- Freestyle packet via libnetfilter_queue is no longer supported?
- What is the rule of validation? What can be edited and what can't be touched?

I still feel making a cipher should be a valid use case for
libnetfilter_queue. ( Which would replace the entire inner packet of
IP in IP encapsulation, so any validation on the inner content would
fail )
The whole point of libnetfilter_queue is giving userspace a way to
inspect/edit the packet. When all function calls success and the
userspace say "verdict, allow" then the system silently dropping the
packet into void does not sound very logical to me.

So that
- I vote for reverting 112e447d . As in my view, it would kill a valid
use case of libnetfilter_queue and NFQUEUE target.
- If it would stay, please consider adding an obvious message into
https://git.netfilter.org/libnetfilter_queue/tree/examples/nf-queue.c
, which would early warning the user that implementing a cipher or any
attempt to completely editing the packet via this library is blocked
by kernel policy. Before someone actually try it.

Thank you for taking a look. Please let me know if any further details
would be helpful.

Yue Wu

Re: [regression] Completely editing packet should be allowed.

[Florian Westphal]

allfox wy <allfoxwy@gmail.com> wrote:
> I have a homebrew cipher program using libnetfilter_queue, which would
> intercept certain IP in IP packets via iptables' NFQUEUE target, edit
> them ( completely ) then put them back to OS for further processing.
> 
> Received packets would be redirected via RAW table ( which would
> happen before CONNTRACK ) , while sending packets would be redirected
> via MANGLE table POSTROUTING chain ( they are leaving the host, no
> longer need to be CONTRACKed ).
>
> The edit is to transform the entire inner packet with a cipher. By
> entire, I mean including the inner header hidden and the original
> length would change.
> The receiving host would decipher it back to normal IP in IP packet in
> RAW table.
> 
> This would form a transparent tunnel and had worked for years.
> 
> However recently I tried to switch from Linux 6.18 LTS to 7.4-rc1 ( in
> favour of tcp_roccet ), my homebrew stopped working.
> There is no error message anywhere, all libnetfilter_queue functions
> return success, I could see the packet, I could edit it, I could send
> the verdict. Just after my edit, the packet would fade into void
> somehow. Like the host never had it.
> 
> So I did a bisect, it comes to
> 112e447d17f78d08485b3076a8baa1dfb9794bd8 is the first bad commit:
> - netfilter: validate L4 headers after userspace packet writes.
>
> I don't really get the idea of the situation here:
> - Freestyle packet via libnetfilter_queue is no longer supported?

It has to be restricted.

> - What is the rule of validation? What can be edited and what can't be touched?

ATM: ip saddr/daddr can change, l4 port numbers can change.
No insertion of extension headers or add/removal of IP options.

Could be supported but requires recomputations of internal data in the
kernel.

> I still feel making a cipher should be a valid use case for
> libnetfilter_queue. ( Which would replace the entire inner packet of
> IP in IP encapsulation, so any validation on the inner content would
> fail )
> The whole point of libnetfilter_queue is giving userspace a way to
> inspect/edit the packet. When all function calls success and the
> userspace say "verdict, allow" then the system silently dropping the
> packet into void does not sound very logical to me.
> 
> So that
> - I vote for reverting 112e447d . As in my view, it would kill a valid
> use case of libnetfilter_queue and NFQUEUE target.

We really can't, no restrictions allow kernel crash.  But its likely
your use case can be made to work again.

Wild guess: you're changing the l4 protocol number and the postrouting
hook reinject fails because of proto mismatch wrt. what conntrack picked
up?

diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
index 90d887546f7d..d2f83004e156 100644
--- a/net/netfilter/nfnetlink_queue.c
+++ b/net/netfilter/nfnetlink_queue.c
@@ -1229,10 +1229,8 @@ static bool nfqnl_validate_l4(const u8 *data, unsigned int data_len,
        const struct nf_conn *ct;

        ct = nf_ct_get(e->skb, &ctinfo);
-       if (ct && !nf_ct_is_template(ct)) {
-               if (fragment || nf_ct_protonum(ct) != proto)
-                       return false;
-       }
+       if (ct && nf_ct_protonum(ct) != proto)
+               nf_reset_ct(e->skb);
 #endif

If that doesn't help, can you either see if you can figure out where the
validation fails, or, alternatively, describe what header modifications
your userspace program is doing?

Re: [regression] Completely editing packet should be allowed.

[Pablo Neira Ayuso]

On Fri, Nov 06, 2026 at 06:08:04AM +0100, Florian Westphal wrote:
> allfox wy <allfoxwy@gmail.com> wrote:
> > I have a homebrew cipher program using libnetfilter_queue, which would
> > intercept certain IP in IP packets via iptables' NFQUEUE target, edit
> > them ( completely ) then put them back to OS for further processing.
> > 
> > Received packets would be redirected via RAW table ( which would
> > happen before CONNTRACK ) , while sending packets would be redirected
> > via MANGLE table POSTROUTING chain ( they are leaving the host, no
> > longer need to be CONTRACKed ).
> >
> > The edit is to transform the entire inner packet with a cipher. By
> > entire, I mean including the inner header hidden and the original
> > length would change.
> > The receiving host would decipher it back to normal IP in IP packet in
> > RAW table.
> > 
> > This would form a transparent tunnel and had worked for years.
> > 
> > However recently I tried to switch from Linux 6.18 LTS to 7.4-rc1 ( in
> > favour of tcp_roccet ), my homebrew stopped working.
> > There is no error message anywhere, all libnetfilter_queue functions
> > return success, I could see the packet, I could edit it, I could send
> > the verdict. Just after my edit, the packet would fade into void
> > somehow. Like the host never had it.
> > 
> > So I did a bisect, it comes to
> > 112e447d17f78d08485b3076a8baa1dfb9794bd8 is the first bad commit:
> > - netfilter: validate L4 headers after userspace packet writes.
> >
> > I don't really get the idea of the situation here:
> > - Freestyle packet via libnetfilter_queue is no longer supported?
> 
> It has to be restricted.
> 
> > - What is the rule of validation? What can be edited and what can't be touched?
> 
> ATM: ip saddr/daddr can change, l4 port numbers can change.
> No insertion of extension headers or add/removal of IP options.
> 
> Could be supported but requires recomputations of internal data in the
> kernel.
> 
> > I still feel making a cipher should be a valid use case for
> > libnetfilter_queue. ( Which would replace the entire inner packet of
> > IP in IP encapsulation, so any validation on the inner content would
> > fail )
> > The whole point of libnetfilter_queue is giving userspace a way to
> > inspect/edit the packet. When all function calls success and the
> > userspace say "verdict, allow" then the system silently dropping the
> > packet into void does not sound very logical to me.
> > 
> > So that
> > - I vote for reverting 112e447d . As in my view, it would kill a valid
> > use case of libnetfilter_queue and NFQUEUE target.
> 
> We really can't, no restrictions allow kernel crash.  But its likely
> your use case can be made to work again.
> 
> Wild guess: you're changing the l4 protocol number and the postrouting
> hook reinject fails because of proto mismatch wrt. what conntrack picked
> up?
> 
> diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
> index 90d887546f7d..d2f83004e156 100644
> --- a/net/netfilter/nfnetlink_queue.c
> +++ b/net/netfilter/nfnetlink_queue.c
> @@ -1229,10 +1229,8 @@ static bool nfqnl_validate_l4(const u8 *data, unsigned int data_len,
>         const struct nf_conn *ct;
> 
>         ct = nf_ct_get(e->skb, &ctinfo);
> -       if (ct && !nf_ct_is_template(ct)) {
> -               if (fragment || nf_ct_protonum(ct) != proto)
> -                       return false;
> -       }
> +       if (ct && nf_ct_protonum(ct) != proto)
> +               nf_reset_ct(e->skb);

I agree ct should be dropping in such a change, because this ct does
not represent this skb anymore.

>  #endif
> 
> If that doesn't help, can you either see if you can figure out where the
> validation fails, or, alternatively, describe what header modifications
> your userspace program is doing?
> 

Re: [regression] Completely editing packet should be allowed.

[allfox wy]

Thank you for explanation. I did a test. It is indeed the protocol
number change in homebrew causing the block.

However, calling nf_reset_ct(e->skb) would still result in packet
being dropped. I didn't do a precise diagnose to find out where it got
dropped. But replacing nf_reset_ct(e->skb) with "return true" would
result in a working system.

My deduction: in the system's NAT table there is MASQUERADE in
POSTROUTING. As my understanding, MASQUERADE requires CONNTRACK
information to work. When the outgoing packet going through MANGLE
table and got nf_reset_ct(e->skb), later MASQUERADE in NAT table might
dropped it.

Is there a function that recalculate the CONNTRACK info rather than
deleting it please?

On Fri, Nov 6, 2026 at 1:08 PM Florian Westphal <fw@strlen.de> wrote:
>
> allfox wy <allfoxwy@gmail.com> wrote:
> > I have a homebrew cipher program using libnetfilter_queue, which would
> > intercept certain IP in IP packets via iptables' NFQUEUE target, edit
> > them ( completely ) then put them back to OS for further processing.
> >
> > Received packets would be redirected via RAW table ( which would
> > happen before CONNTRACK ) , while sending packets would be redirected
> > via MANGLE table POSTROUTING chain ( they are leaving the host, no
> > longer need to be CONTRACKed ).
> >
> > The edit is to transform the entire inner packet with a cipher. By
> > entire, I mean including the inner header hidden and the original
> > length would change.
> > The receiving host would decipher it back to normal IP in IP packet in
> > RAW table.
> >
> > This would form a transparent tunnel and had worked for years.
> >
> > However recently I tried to switch from Linux 6.18 LTS to 7.4-rc1 ( in
> > favour of tcp_roccet ), my homebrew stopped working.
> > There is no error message anywhere, all libnetfilter_queue functions
> > return success, I could see the packet, I could edit it, I could send
> > the verdict. Just after my edit, the packet would fade into void
> > somehow. Like the host never had it.
> >
> > So I did a bisect, it comes to
> > 112e447d17f78d08485b3076a8baa1dfb9794bd8 is the first bad commit:
> > - netfilter: validate L4 headers after userspace packet writes.
> >
> > I don't really get the idea of the situation here:
> > - Freestyle packet via libnetfilter_queue is no longer supported?
>
> It has to be restricted.
>
> > - What is the rule of validation? What can be edited and what can't be touched?
>
> ATM: ip saddr/daddr can change, l4 port numbers can change.
> No insertion of extension headers or add/removal of IP options.
>
> Could be supported but requires recomputations of internal data in the
> kernel.
>
> > I still feel making a cipher should be a valid use case for
> > libnetfilter_queue. ( Which would replace the entire inner packet of
> > IP in IP encapsulation, so any validation on the inner content would
> > fail )
> > The whole point of libnetfilter_queue is giving userspace a way to
> > inspect/edit the packet. When all function calls success and the
> > userspace say "verdict, allow" then the system silently dropping the
> > packet into void does not sound very logical to me.
> >
> > So that
> > - I vote for reverting 112e447d . As in my view, it would kill a valid
> > use case of libnetfilter_queue and NFQUEUE target.
>
> We really can't, no restrictions allow kernel crash.  But its likely
> your use case can be made to work again.
>
> Wild guess: you're changing the l4 protocol number and the postrouting
> hook reinject fails because of proto mismatch wrt. what conntrack picked
> up?
>
> diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
> index 90d887546f7d..d2f83004e156 100644
> --- a/net/netfilter/nfnetlink_queue.c
> +++ b/net/netfilter/nfnetlink_queue.c
> @@ -1229,10 +1229,8 @@ static bool nfqnl_validate_l4(const u8 *data, unsigned int data_len,
>         const struct nf_conn *ct;
>
>         ct = nf_ct_get(e->skb, &ctinfo);
> -       if (ct && !nf_ct_is_template(ct)) {
> -               if (fragment || nf_ct_protonum(ct) != proto)
> -                       return false;
> -       }
> +       if (ct && nf_ct_protonum(ct) != proto)
> +               nf_reset_ct(e->skb);
>  #endif
>
> If that doesn't help, can you either see if you can figure out where the
> validation fails, or, alternatively, describe what header modifications
> your userspace program is doing?

Re: [regression] Completely editing packet should be allowed.

[allfox wy]

On Sat, Nov 7, 2026 at 4:04 PM allfox wy <allfoxwy@gmail.com> wrote:
>
> Thank you for explanation. I did a test. It is indeed the protocol
> number change in homebrew causing the block.
>
> However, calling nf_reset_ct(e->skb) would still result in packet
> being dropped. I didn't do a precise diagnose to find out where it got
> dropped. But replacing nf_reset_ct(e->skb) with "return true" would
> result in a working system.
>
> My deduction: in the system's NAT table there is MASQUERADE in
> POSTROUTING. As my understanding, MASQUERADE requires CONNTRACK
> information to work. When the outgoing packet going through MANGLE
> table and got nf_reset_ct(e->skb), later MASQUERADE in NAT table might
> dropped it.
>
> Is there a function that recalculate the CONNTRACK info rather than
> deleting it please?
>
> On Fri, Nov 6, 2026 at 1:08 PM Florian Westphal <fw@strlen.de> wrote:
> >
> > allfox wy <allfoxwy@gmail.com> wrote:
> > > I have a homebrew cipher program using libnetfilter_queue, which would
> > > intercept certain IP in IP packets via iptables' NFQUEUE target, edit
> > > them ( completely ) then put them back to OS for further processing.
> > >
> > > Received packets would be redirected via RAW table ( which would
> > > happen before CONNTRACK ) , while sending packets would be redirected
> > > via MANGLE table POSTROUTING chain ( they are leaving the host, no
> > > longer need to be CONTRACKed ).
> > >
> > > The edit is to transform the entire inner packet with a cipher. By
> > > entire, I mean including the inner header hidden and the original
> > > length would change.
> > > The receiving host would decipher it back to normal IP in IP packet in
> > > RAW table.
> > >
> > > This would form a transparent tunnel and had worked for years.
> > >
> > > However recently I tried to switch from Linux 6.18 LTS to 7.4-rc1 ( in
> > > favour of tcp_roccet ), my homebrew stopped working.
> > > There is no error message anywhere, all libnetfilter_queue functions
> > > return success, I could see the packet, I could edit it, I could send
> > > the verdict. Just after my edit, the packet would fade into void
> > > somehow. Like the host never had it.
> > >
> > > So I did a bisect, it comes to
> > > 112e447d17f78d08485b3076a8baa1dfb9794bd8 is the first bad commit:
> > > - netfilter: validate L4 headers after userspace packet writes.
> > >
> > > I don't really get the idea of the situation here:
> > > - Freestyle packet via libnetfilter_queue is no longer supported?
> >
> > It has to be restricted.
> >
> > > - What is the rule of validation? What can be edited and what can't be touched?
> >
> > ATM: ip saddr/daddr can change, l4 port numbers can change.
> > No insertion of extension headers or add/removal of IP options.
> >
> > Could be supported but requires recomputations of internal data in the
> > kernel.
> >
> > > I still feel making a cipher should be a valid use case for
> > > libnetfilter_queue. ( Which would replace the entire inner packet of
> > > IP in IP encapsulation, so any validation on the inner content would
> > > fail )
> > > The whole point of libnetfilter_queue is giving userspace a way to
> > > inspect/edit the packet. When all function calls success and the
> > > userspace say "verdict, allow" then the system silently dropping the
> > > packet into void does not sound very logical to me.
> > >
> > > So that
> > > - I vote for reverting 112e447d . As in my view, it would kill a valid
> > > use case of libnetfilter_queue and NFQUEUE target.
> >
> > We really can't, no restrictions allow kernel crash.  But its likely
> > your use case can be made to work again.
> >
> > Wild guess: you're changing the l4 protocol number and the postrouting
> > hook reinject fails because of proto mismatch wrt. what conntrack picked
> > up?
> >
> > diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
> > index 90d887546f7d..d2f83004e156 100644
> > --- a/net/netfilter/nfnetlink_queue.c
> > +++ b/net/netfilter/nfnetlink_queue.c
> > @@ -1229,10 +1229,8 @@ static bool nfqnl_validate_l4(const u8 *data, unsigned int data_len,
> >         const struct nf_conn *ct;
> >
> >         ct = nf_ct_get(e->skb, &ctinfo);
> > -       if (ct && !nf_ct_is_template(ct)) {
> > -               if (fragment || nf_ct_protonum(ct) != proto)
> > -                       return false;
> > -       }
> > +       if (ct && nf_ct_protonum(ct) != proto)
> > +               nf_reset_ct(e->skb);
> >  #endif
> >
> > If that doesn't help, can you either see if you can figure out where the
> > validation fails, or, alternatively, describe what header modifications
> > your userspace program is doing?

Excuse me for that I should have used interleaved style quoting.

Re: [regression] Completely editing packet should be allowed.

[Pablo Neira Ayuso]

On Sat, Nov 07, 2026 at 04:04:49PM +0800, allfox wy wrote:
> Thank you for explanation. I did a test. It is indeed the protocol
> number change in homebrew causing the block.
> 
> However, calling nf_reset_ct(e->skb) would still result in packet
> being dropped. I didn't do a precise diagnose to find out where it got
> dropped. But replacing nf_reset_ct(e->skb) with "return true" would
> result in a working system.
> 
> My deduction: in the system's NAT table there is MASQUERADE in
> POSTROUTING. As my understanding, MASQUERADE requires CONNTRACK
> information to work. When the outgoing packet going through MANGLE
> table and got nf_reset_ct(e->skb), later MASQUERADE in NAT table might
> dropped it.
> 
> Is there a function that recalculate the CONNTRACK info rather than
> deleting it please?

That would entirely conntrack lookup.

I think your software should be doing this packet mangling from the
prerouting hook *before* the conntrack hook, so you have a chance to
create a conntrack object that really represents the flows after your
mangling.

Re: [regression] Completely editing packet should be allowed.

[Pablo Neira Ayuso]

On Mon, Nov 09, 2026 at 11:25:34AM +0100, Pablo Neira Ayuso wrote:
> On Sat, Nov 07, 2026 at 04:04:49PM +0800, allfox wy wrote:
> > Thank you for explanation. I did a test. It is indeed the protocol
> > number change in homebrew causing the block.
> > 
> > However, calling nf_reset_ct(e->skb) would still result in packet
> > being dropped. I didn't do a precise diagnose to find out where it got
> > dropped. But replacing nf_reset_ct(e->skb) with "return true" would
> > result in a working system.
> > 
> > My deduction: in the system's NAT table there is MASQUERADE in
> > POSTROUTING. As my understanding, MASQUERADE requires CONNTRACK
> > information to work. When the outgoing packet going through MANGLE
> > table and got nf_reset_ct(e->skb), later MASQUERADE in NAT table might
> > dropped it.
> > 
> > Is there a function that recalculate the CONNTRACK info rather than
> > deleting it please?
> 
> That would entirely conntrack lookup.

I meant to say: That would require a mistream conntrack re-lookup.

> I think your software should be doing this packet mangling from the
> prerouting hook *before* the conntrack hook, so you have a chance to
> create a conntrack object that really represents the flows after your
> mangling.
> 

Re: [regression] Completely editing packet should be allowed.

[Florian Westphal]

allfox wy <allfoxwy@gmail.com> wrote:
> Thank you for explanation. I did a test. It is indeed the protocol
> number change in homebrew causing the block.
> 
> However, calling nf_reset_ct(e->skb) would still result in packet
> being dropped. I didn't do a precise diagnose to find out where it got
> dropped. But replacing nf_reset_ct(e->skb) with "return true" would
> result in a working system.

Unfortunately that is the problematic part.

I'm not even sure how this scheme of yours works:

The NAT dispatch uses the l4proto value from the tuple to decide
what to do.

So as far as I can see NAT postrouting will write updated tcp/udp
port to the packet, which is no longer udp/tcp - corrupting
the l4 header.

> My deduction: in the system's NAT table there is MASQUERADE in
> POSTROUTING. As my understanding, MASQUERADE requires CONNTRACK
> information to work. When the outgoing packet going through MANGLE
> table and got nf_reset_ct(e->skb), later MASQUERADE in NAT table might
> dropped it.

The removal of the conntrack info causes the packet to be treated as
INVALID; MASQUERADE / SNAT will not work on it.

> Is there a function that recalculate the CONNTRACK info rather than
> deleting it please?

We would need to make a re-lookup (possibly creating a new conntrack
entry) that reflects the new/updated L4protocol + any other changes to
saddr/daddr.

Re: [regression] Completely editing packet should be allowed.

[allfox wy]

Florian Westphal wrote:
> The NAT dispatch uses the l4proto value from the tuple to decide
> what to do.
>
> So as far as I can see NAT postrouting will write updated tcp/udp
> port to the packet, which is no longer udp/tcp - corrupting
> the l4 header.

Excuse me for that I should be more clear about this part.
The scheme is working by IP-in-IP encapsulation. The layer 4 protocol
is another IP and it does not have a port number. TCP/UDP is in layer
5 in this scheme.
I would encipher the entire layer 4 contents and edited layer 3
header's L4-protocol number. The recent change in kernel refuses this
L4-protocol number change.

The design:
Incoming packets => got deciphered in PREROUTING of RAW => CONNTRACK
see real contents => all the normal
Outgoing packets => CONNTRACK see real contents => enciphered in
POSTROUTING of MANGLE => MASQUERADE based on earlier CONNTRACK
information

The problem:
I can no longer change L4-protocol number in layer 3 IP header during
encipher/decipher process because of Kernel commit 112e447



Florian Westphal wrote:
> We would need to make a re-lookup (possibly creating a new conntrack
> entry) that reflects the new/updated L4protocol + any other changes to
> saddr/daddr.

I rethinked about it, this won't work unfortunately.
Because NAT need to match outgoing starter and later incoming reply.
If we re-lookup / creating a different entity for the outgoing
starter, NAT would fail because it would think later incoming reply is
not RELATED nor ESTABLISHED, hence drop/mislead the reply.

I'm not sure how netfilter do SNAT for IPIP, in the past it was
working fine to rewrite saddr/daddr of the outer envelop header ( the
layer 3 ).
Inner header/contents ( the layer 4 ) would be masked by cipher. But
as CONNTRACK happens earlier than the cipher, it already has real
information to do SNAT, even later those real information would be
hidden when departure.



Pablo Neira Ayuso wrote:
> I think your software should be doing this packet mangling from the
> prerouting hook *before* the conntrack hook, so you have a chance to
> create a conntrack object that really represents the flows after your
> mangling.

I did a test of replacing the use of POSTROUTING chain of MANGLE table
by OUTPUT chain of RAW table, so that outgoing packets would be
enciphered first, then enter CONNTRACK.
The new kernel system doesn't work. The phenomenon is that I could see
packet-counter increasing in both PREROUTING chain and OUTPUT chain of
RAW table, which indicates both encipher and decipher is working and
remote system is replying. However the new kernel system still can not
see the ICMP ping reply, it would say 100% packet loss.

I _guess_ that:
- Outgoing pings are enciphered first, CONNTRACK could only see a
valid L3-header ( with the L4-protocol number edited ) and a black box
follows it, it would have created state based on these masked
information.
- Incoming replies are deciphered first, CONNTRACK would see their
real contents and create states based on that real information.

CONNTRACK would have a difficult time to match the real state with the
masked state to tell they are belong to the same ICMP conversation. (
Well, that's the whole point of a cipher )
By using POSTROUTING chain of MANGLE table, the outgoing encipher
process happens after CONNTRACK seeing real contents and creating its
status based on real information. But this is now blocked by the
recent L4-protocol number validation change. ( Kernel commit 112e447 )




Back to the very beginning when I wrote the cipher software, I wished
to do the encipher _after_ SNAT / CONNTRACK. But this design can't be
done because MASQUERADE is a final destination and there is no more
POSTROUTING chain in RAW table.

Re: [regression] Completely editing packet should be allowed.

[Florian Westphal]

allfox wy <allfoxwy@gmail.com> wrote:
> Florian Westphal wrote:
> > The NAT dispatch uses the l4proto value from the tuple to decide
> > what to do.
> >
> > So as far as I can see NAT postrouting will write updated tcp/udp
> > port to the packet, which is no longer udp/tcp - corrupting
> > the l4 header.
> 
> Excuse me for that I should be more clear about this part.
> The scheme is working by IP-in-IP encapsulation. The layer 4 protocol
> is another IP and it does not have a port number. TCP/UDP is in layer
> 5 in this scheme.

Yes, but kernel doesn't know that.

static bool l4proto_manip_pkt(struct sk_buff *skb,
                              unsigned int iphdroff, unsigned int hdroff,
                              const struct nf_conntrack_tuple *tuple,
                              enum nf_nat_manip_type maniptype)
{
        switch (tuple->dst.protonum) {  <- from BEFORE nfqueue rewrite 
        case IPPROTO_TCP:
                return tcp_manip_pkt(skb, iphdroff, hdroff,
                                     tuple, maniptype);
        case IPPROTO_UDP:
                return udp_manip_pkt(skb, iphdroff, hdroff,
                                     tuple, maniptype);

so, when the conntrack entry is subject to a port translation, the
nat engine will now write into the 2nd ip header that follows the
first one, this will corrupt ip->ihl, version, tos and/or tot_len
fields.

> The design:
> Incoming packets => got deciphered in PREROUTING of RAW => CONNTRACK
> see real contents => all the normal
> Outgoing packets => CONNTRACK see real contents => enciphered in
> POSTROUTING of MANGLE => MASQUERADE based on earlier CONNTRACK
> information
> 
> The problem:
> I can no longer change L4-protocol number in layer 3 IP header during
> encipher/decipher process because of Kernel commit 112e447

Yes.  Not yet sure how to make this work.  Changing this part of
NAT engine isn't sufficient either, conntrack entry could have
a helper attached, and those do not re-validate tcp header sizes
or ip options.

Maybe best we can do is to relax the check for initial user
namespace (or just init_net).  But it would still allow
CAP_NET_ADMIN process to re-gain all other capabilities...

Re: [regression] Completely editing packet should be allowed.

[allfox wy]

Florian Westphal wrote:
> Yes, but kernel doesn't know that.
>
> static bool l4proto_manip_pkt(struct sk_buff *skb,
>                               unsigned int iphdroff, unsigned int hdroff,
>                               const struct nf_conntrack_tuple *tuple,
>                               enum nf_nat_manip_type maniptype)
> {
>         switch (tuple->dst.protonum) {  <- from BEFORE nfqueue rewrite
>         case IPPROTO_TCP:
>                 return tcp_manip_pkt(skb, iphdroff, hdroff,
>                                      tuple, maniptype);
>         case IPPROTO_UDP:
>                 return udp_manip_pkt(skb, iphdroff, hdroff,
>                                      tuple, maniptype);
>
> so, when the conntrack entry is subject to a port translation, the
> nat engine will now write into the 2nd ip header that follows the
> first one, this will corrupt ip->ihl, version, tos and/or tot_len
> fields.

I think it would be fine. Before nfqueue rewrite, layer 3 header would
have a number 4 as L4-protocol, which would fall out of the switch
statement to the last
> /* If we don't know protocol -- no error, pass it unmodified. */
> return true;

The IPIP interface knows the remote address, it does not need NAT to
fill in daddr, the device would do it. And I'm _guessing_ that saddr
would be filled in by routing decision.



> Yes.  Not yet sure how to make this work.  Changing this part of
> NAT engine isn't sufficient either, conntrack entry could have
> a helper attached, and those do not re-validate tcp header sizes
> or ip options.
>
> Maybe best we can do is to relax the check for initial user
> namespace (or just init_net).  But it would still allow
> CAP_NET_ADMIN process to re-gain all other capabilities...

Thank you but I think there is no need to relax security line because
of a homebrew. It's all fine as I could cook another pie with
something different than NFQUEUE.
Also I could maybe expose the real protocol number in layer 3 so
CONNTRACK would always see a 4 there.
All fine if the security calls the change. I could try some different
framework / infrastructure to carry out the design.