[NFQUEUE] lack of UID/GID fields in fragmented packets

[NFQUEUE] lack of UID/GID fields in fragmented packets

[Piotr Sawicki]

Hi,

I'm developing a user-space firewall service for managing outgoing 
network traffic. I want to control the access to the Internet on a per 
user basis.
So far, I've made a simple skeleton application which works quite well. 
The NFQUEUE is attached to the output chain of the mangle table.

Iptables -t mangle -A OUTPUT -j NFQUEUE --queue-num 0

The queue is configured to capture only meta-data of outgoing packets. 
The application reads the UID and GID fields from a captured packet. 
After checking a permission, the application issues a verdict. The 
configuration of NFQUEUE looks as follows:

nfq_set_mode (qh, NFQNL_COPY_META, 0xffff);
nfq_set_queue_flags (qh, NFQA_CFG_F_UID_GID, NFQA_CFG_F_UID_GID)

Everything works fine until I try to send huge packets. When the size of 
these packets is larger than MTU then the fragmentation occurs. I've 
observed that the first fragment has valid UID and GID fields, but the 
rest of the fragments do not include them.

I've found that the remedy for this concern is to set NFQA_CFG_F_GSO flag.

nfq_set_queue_flags (qh, NFQA_CFG_F_GSO, NFQA_CFG_F_GSO)

 From the analysis of the Linux kernel source code, I've figured out 
that when the above flag is not set, the processed packet is not 
fragmented before pushing it into the queue. So, the problem is in 
fragmentation mechanism.

The UID and GID fields (credentials) are taken from the socket structure 
(sk) of a socket buffer (skb) (skb->sk->sk_socket->file->f_cred). The sk 
field of skb may be NULL only if the skb originates from some internal 
kernel subsystem.

A user space application which sends network traffic via a BSD socket 
always produces skb having valid sk and thus having valid credentials.

I've found that when the fragmentation procedure splits the packet into 
fragments, it keeps a valid sk only in the first fragment. Therefore, it 
is impossible to fetch valid UID and GID fields from the rest of the 
fragments.

Is it intended behavior, or is it a bug?

Re: [NFQUEUE] lack of UID/GID fields in fragmented packets

[Florian Westphal]

Piotr Sawicki <piotr.sawicki@gmail.com> wrote:
> Everything works fine until I try to send huge packets. When the size of
> these packets is larger than MTU then the fragmentation occurs. I've
> observed that the first fragment has valid UID and GID fields, but the rest
> of the fragments do not include them.
> 
> I've found that the remedy for this concern is to set NFQA_CFG_F_GSO flag.

All users should set this flag.

We can't make it default because it breaks old applications
that can't deal with large (offload) packets.

> I've found that when the fragmentation procedure splits the packet into
> fragments, it keeps a valid sk only in the first fragment. Therefore, it is
> impossible to fetch valid UID and GID fields from the rest of the fragments.
> 
> Is it intended behavior, or is it a bug?

Neither.  UID code was added later, so this wasn't a problem.
I would suggest to just set F_GSO flag; it has no disadvantages.