[PATCH 0/2] Add UID/GID info to NFQUEUE

[PATCH 0/2] Add UID/GID info to NFQUEUE

[valentina.giusti]

From: Valentina Giusti <valentina.giusti@bmw-carit.de>

Hi,

this patchset adds the possibility to get the UID/GID of the socket they belong
with the NFQUEUE target.

The feature is meant to be a helper for network statistics made on a per
application basis.
In fact, letting userspace being able to associate a packet to a UID and GID
couple helps narrowing down the traffic to the application it belongs to.

One could argue that we already have this feature by means of the owner match,
but this is not true for incoming traffic. It could actually be partially true
thanks to commits 41063e9 (ipv4: Early TCP socket demux) and 421b388 (udp: 
ipv4: Add udp early demux), which allow to obtain socket information also for 
incoming TCP and UDP connections. However, as Pablo Neira Ayuso already pointed
out (http://www.spinics.net/lists/netfilter-devel/msg27952.html), enabling the
owner match on INPUT wouldn't be semantically equivalent to when it is used on
OUTPUT, even with the aforementioned early demux commits.

At the Linux Plumbers Conference 2013, there have been quite interesting 
discussions on the topic of network statistics, and it was proposed that it 
would make more sense to use NFQUEUE for this purpose, letting therefore 
userspace use the UID/GID information for application-based statistics purposes.
This way the UID and GID information of the incoming TCP and UDP traffic is not
"wasted" and can be used for more refined statistics.

For more information on what has been said at LPC2013, have a look at:

https://www.youtube.com/watch?v=ulIqVzsC03g
(Updates on 'New Challenges for Linux Network Support',
 Daniel Wagner, BMW Car IT GmbH)
https://www.youtube.com/watch?v=Fi_iyaF7Gw0
(Android netfilter changes,
 John Stultz, Linaro.org)

And also at this article:

https://lwn.net/Articles/517358/

With this patchset I am proposing an implementation that follows up the
discussions that have been made.
Please have a look at it, any comments are welcome.

-- 
Best Regards,
Val

[PATCH 1/2] netfilter_queue: enable UID/GID socket info retrieval

[valentina.giusti]

From: Valentina Giusti <valentina.giusti@bmw-carit.de>

Thanks to commits 41063e9 (ipv4: Early TCP socket demux) and 421b388 (udp: 
ipv4: Add udp early demux) it is now possible to get UID and GID socket info
also for incoming TCP and UDP connections. Having this info available, it
is convenient to let NFQUEUE retrieve it in order to improve and refine the
traffic analysis in userspace.

Signed-off-by: Valentina Giusti <valentina.giusti@bmw-carit.de>
---
 include/uapi/linux/netfilter/nfnetlink_queue.h |    2 ++
 net/netfilter/nfnetlink_queue_core.c           |   23 ++++++++++++++++++++++-
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/netfilter/nfnetlink_queue.h b/include/uapi/linux/netfilter/nfnetlink_queue.h
index 0132bad..46ebbc4 100644
--- a/include/uapi/linux/netfilter/nfnetlink_queue.h
+++ b/include/uapi/linux/netfilter/nfnetlink_queue.h
@@ -47,6 +47,8 @@ enum nfqnl_attr_type {
 	NFQA_CAP_LEN,			/* __u32 length of captured packet */
 	NFQA_SKB_INFO,			/* __u32 skb meta information */
 	NFQA_EXP,			/* nf_conntrack_netlink.h */
+	NFQA_UID,			/* __u32 sk uid */
+	NFQA_GID,			/* __u32 sk gid */
 
 	__NFQA_MAX
 };
diff --git a/net/netfilter/nfnetlink_queue_core.c b/net/netfilter/nfnetlink_queue_core.c
index 21258cf..7257ddb 100644
--- a/net/netfilter/nfnetlink_queue_core.c
+++ b/net/netfilter/nfnetlink_queue_core.c
@@ -328,7 +328,9 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
 		+ nla_total_size(sizeof(u_int32_t))	/* mark */
 		+ nla_total_size(sizeof(struct nfqnl_msg_packet_hw))
 		+ nla_total_size(sizeof(u_int32_t))	/* skbinfo */
-		+ nla_total_size(sizeof(u_int32_t));	/* cap_len */
+		+ nla_total_size(sizeof(u_int32_t))	/* cap_len */
+		+ nla_total_size(sizeof(u_int32_t))	/* uid */
+		+ nla_total_size(sizeof(u_int32_t));	/* gid */
 
 	if (entskb->tstamp.tv64)
 		size += nla_total_size(sizeof(struct nfqnl_msg_packet_timestamp));
@@ -484,6 +486,25 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
 			goto nla_put_failure;
 	}
 
+	if (entskb->sk) {
+		struct sock *sk = entskb->sk;
+		if (sk && sk->sk_state != TCP_TIME_WAIT) {
+			read_lock_bh(&sk->sk_callback_lock);
+			if (sk->sk_socket && sk->sk_socket->file) {
+				struct file *file = sk->sk_socket->file;
+				const struct cred *cred = file->f_cred;
+				if (nla_put_u32(skb, NFQA_UID,
+				    htonl(cred->fsuid)))
+					goto nla_put_failure;
+				if (nla_put_u32(skb, NFQA_GID,
+				    htonl(cred->fsgid)))
+					goto nla_put_failure;
+				read_unlock_bh(&sk->sk_callback_lock);
+			} else
+				read_unlock_bh(&sk->sk_callback_lock);
+		}
+	}
+
 	if (ct && nfqnl_ct_put(skb, ct, ctinfo) < 0)
 		goto nla_put_failure;
 
-- 
1.7.10.4

[PATCH 2/2] libnetfilter_queue: add support for UID/GID socket info

[valentina.giusti]

From: Valentina Giusti <Valentina.Giusti@bmw-carit.de>

With this patch libnetfilter_queue is able to parse the UID/GID socket
information from nfnetlink_queue.

Signed-off-by: Valentina Giusti <Valentina.Giusti@bmw-carit.de>
---
 include/libnetfilter_queue/libnetfilter_queue.h    |  4 ++++
 include/libnetfilter_queue/linux_nfnetlink_queue.h |  2 ++
 include/linux/netfilter/nfnetlink_queue.h          |  3 +++
 src/libnetfilter_queue.c                           | 25 ++++++++++++++++++++++
 src/nlmsg.c                                        |  2 ++
 5 files changed, 36 insertions(+)

diff --git a/include/libnetfilter_queue/libnetfilter_queue.h b/include/libnetfilter_queue/libnetfilter_queue.h
index b9f16e2..b4e2679 100644
--- a/include/libnetfilter_queue/libnetfilter_queue.h
+++ b/include/libnetfilter_queue/libnetfilter_queue.h
@@ -103,6 +103,8 @@ extern u_int32_t nfq_get_indev(struct nfq_data *nfad);
 extern u_int32_t nfq_get_physindev(struct nfq_data *nfad);
 extern u_int32_t nfq_get_outdev(struct nfq_data *nfad);
 extern u_int32_t nfq_get_physoutdev(struct nfq_data *nfad);
+extern u_int32_t nfq_get_uid(struct nfq_data *nfad);
+extern u_int32_t nfq_get_gid(struct nfq_data *nfad);
 
 extern int nfq_get_indev_name(struct nlif_handle *nlif_handle,
 			      struct nfq_data *nfad, char *name);
@@ -125,6 +127,8 @@ enum {
 	NFQ_XML_PHYSDEV	= (1 << 3),
 	NFQ_XML_PAYLOAD	= (1 << 4),
 	NFQ_XML_TIME	= (1 << 5),
+	NFQ_XML_UID	= (1 << 6),
+	NFQ_XML_GID	= (1 << 7),
 	NFQ_XML_ALL	= ~0U,
 };
 
diff --git a/include/libnetfilter_queue/linux_nfnetlink_queue.h b/include/libnetfilter_queue/linux_nfnetlink_queue.h
index 81a485b..88fd0c0 100644
--- a/include/libnetfilter_queue/linux_nfnetlink_queue.h
+++ b/include/libnetfilter_queue/linux_nfnetlink_queue.h
@@ -50,6 +50,8 @@ enum nfqnl_attr_type {
 	NFQA_CAP_LEN,                   /* __u32 length of captured packet */
 	NFQA_SKB_INFO,                  /* __u32 skb meta information */
 
+	NFQA_UID,			/* __u32 sk uid */
+	NFQA_GID,			/* __u32 sk gid */
 	__NFQA_MAX
 };
 #define NFQA_MAX (__NFQA_MAX - 1)
diff --git a/include/linux/netfilter/nfnetlink_queue.h b/include/linux/netfilter/nfnetlink_queue.h
index a2308ae..dfbd1ad 100644
--- a/include/linux/netfilter/nfnetlink_queue.h
+++ b/include/linux/netfilter/nfnetlink_queue.h
@@ -46,6 +46,9 @@ enum nfqnl_attr_type {
 	NFQA_CT_INFO,			/* enum ip_conntrack_info */
 	NFQA_CAP_LEN,			/* __u32 length of captured packet */
 	NFQA_SKB_INFO,			/* __u32 skb meta information */
+	NFQA_EXP,			/* nf_conntrack_netlink.h */
+	NFQA_UID,			/* __u32 sk uid */
+	NFQA_GID,			/* __u32 sk gid */
 
 	__NFQA_MAX
 };
diff --git a/src/libnetfilter_queue.c b/src/libnetfilter_queue.c
index fa8efe7..52456db 100644
--- a/src/libnetfilter_queue.c
+++ b/src/libnetfilter_queue.c
@@ -1180,6 +1180,18 @@ struct nfqnl_msg_packet_hw *nfq_get_packet_hw(struct nfq_data *nfad)
 }
 EXPORT_SYMBOL(nfq_get_packet_hw);
 
+uint32_t nfq_get_uid(struct nfq_data *nfad)
+{
+	return ntohl(nfnl_get_data(nfad->data, NFQA_UID, u_int32_t));
+}
+EXPORT_SYMBOL(nfq_get_uid);
+
+uint32_t nfq_get_gid(struct nfq_data *nfad)
+{
+	return ntohl(nfnl_get_data(nfad->data, NFQA_GID, u_int32_t));
+}
+EXPORT_SYMBOL(nfq_get_gid);
+
 /**
  * nfq_get_payload - get payload 
  * \param nfad Netlink packet data handle passed to callback function
@@ -1250,6 +1262,7 @@ int nfq_snprintf_xml(char *buf, size_t rem, struct nfq_data *tb, int flags)
 	struct nfqnl_msg_packet_hdr *ph;
 	struct nfqnl_msg_packet_hw *hwph;
 	u_int32_t mark, ifi;
+	u_int32_t uid, gid;
 	int size, offset = 0, len = 0, ret;
 	unsigned char *data;
 
@@ -1365,6 +1378,18 @@ int nfq_snprintf_xml(char *buf, size_t rem, struct nfq_data *tb, int flags)
 		SNPRINTF_FAILURE(size, rem, offset, len);
 	}
 
+	uid = nfq_get_uid(tb);
+	if (uid && (flags & NFQ_XML_UID)) {
+		size = snprintf(buf + offset, rem, "<uid>%u</uid>", uid);
+		SNPRINTF_FAILURE(size, rem, offset, len);
+	}
+
+	gid = nfq_get_gid(tb);
+	if (gid && (flags & NFQ_XML_GID)) {
+		size = snprintf(buf + offset, rem, "<gid>%u</gid>", gid);
+		SNPRINTF_FAILURE(size, rem, offset, len);
+	}
+
 	ret = nfq_get_payload(tb, &data);
 	if (ret >= 0 && (flags & NFQ_XML_PAYLOAD)) {
 		int i;
diff --git a/src/nlmsg.c b/src/nlmsg.c
index e7a30e0..81e170e 100644
--- a/src/nlmsg.c
+++ b/src/nlmsg.c
@@ -134,6 +134,8 @@ static int nfq_pkt_parse_attr_cb(const struct nlattr *attr, void *data)
 	case NFQA_IFINDEX_PHYSOUTDEV:
 	case NFQA_CAP_LEN:
 	case NFQA_SKB_INFO:
+	case NFQA_UID:
+	case NFQA_GID:
 		if (mnl_attr_validate(attr, MNL_TYPE_U32) < 0)
 			return MNL_CB_ERROR;
 		break;
-- 
1.8.5.1

Re: [PATCH 1/2] netfilter_queue: enable UID/GID socket info retrieval

[Florian Westphal]

valentina.giusti@bmw-carit.de <valentina.giusti@bmw-carit.de> wrote:
> From: Valentina Giusti <valentina.giusti@bmw-carit.de>
> 
> Thanks to commits 41063e9 (ipv4: Early TCP socket demux) and 421b388 (udp: 
> ipv4: Add udp early demux) it is now possible to get UID and GID socket info
> also for incoming TCP and UDP connections. Having this info available, it
> is convenient to let NFQUEUE retrieve it in order to improve and refine the
> traffic analysis in userspace.

No objections from me, except a few comments below.

> diff --git a/net/netfilter/nfnetlink_queue_core.c b/net/netfilter/nfnetlink_queue_core.c
> index 21258cf..7257ddb 100644
> --- a/net/netfilter/nfnetlink_queue_core.c
> +++ b/net/netfilter/nfnetlink_queue_core.c
> @@ -328,7 +328,9 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
[..]
>  	if (entskb->tstamp.tv64)
>  		size += nla_total_size(sizeof(struct nfqnl_msg_packet_timestamp));
> @@ -484,6 +486,25 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
>  			goto nla_put_failure;
>  	}
>  
> +	if (entskb->sk) {
> +		struct sock *sk = entskb->sk;
> +		if (sk && sk->sk_state != TCP_TIME_WAIT) {
> +			read_lock_bh(&sk->sk_callback_lock);

nfqnl_build_packet_message is already a huge function, I think it
might be useful to add a small helper function for this,
e.g.  'nfqnl_skinfo_put(skb, entskb->sk)' or something like that.

Also, it might be a good idea to only dump the sk info
on userspace request (not sure how expensive the readlock is
compared to the nfqueue cost...), see NFQA_CFG_F_CONNTRACK flag for
instance.

> +			if (sk->sk_socket && sk->sk_socket->file) {
> +				struct file *file = sk->sk_socket->file;
> +				const struct cred *cred = file->f_cred;
> +				if (nla_put_u32(skb, NFQA_UID,

nla_put_be32

> +				    htonl(cred->fsuid)))
> +					goto nla_put_failure;

careful - the sk_callback_lock was taken.

Cheers,
Florian

Re: [PATCH 1/2] netfilter_queue: enable UID/GID socket info retrieval

[Valentina Giusti]

Hi Florian,

thank you very much for your review.

On 12/18/2013 10:24 PM, Florian Westphal wrote:
> valentina.giusti@bmw-carit.de <valentina.giusti@bmw-carit.de> wrote:
>> From: Valentina Giusti <valentina.giusti@bmw-carit.de>
>>
>> Thanks to commits 41063e9 (ipv4: Early TCP socket demux) and 421b388 (udp:
>> ipv4: Add udp early demux) it is now possible to get UID and GID socket info
>> also for incoming TCP and UDP connections. Having this info available, it
>> is convenient to let NFQUEUE retrieve it in order to improve and refine the
>> traffic analysis in userspace.
> No objections from me, except a few comments below.
>
>> diff --git a/net/netfilter/nfnetlink_queue_core.c b/net/netfilter/nfnetlink_queue_core.c
>> index 21258cf..7257ddb 100644
>> --- a/net/netfilter/nfnetlink_queue_core.c
>> +++ b/net/netfilter/nfnetlink_queue_core.c
>> @@ -328,7 +328,9 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
> [..]
>>   	if (entskb->tstamp.tv64)
>>   		size += nla_total_size(sizeof(struct nfqnl_msg_packet_timestamp));
>> @@ -484,6 +486,25 @@ nfqnl_build_packet_message(struct net *net, struct nfqnl_instance *queue,
>>   			goto nla_put_failure;
>>   	}
>>   
>> +	if (entskb->sk) {
>> +		struct sock *sk = entskb->sk;
>> +		if (sk && sk->sk_state != TCP_TIME_WAIT) {
>> +			read_lock_bh(&sk->sk_callback_lock);
> nfqnl_build_packet_message is already a huge function, I think it
> might be useful to add a small helper function for this,
> e.g.  'nfqnl_skinfo_put(skb, entskb->sk)' or something like that.
>
> Also, it might be a good idea to only dump the sk info
> on userspace request (not sure how expensive the readlock is
> compared to the nfqueue cost...), see NFQA_CFG_F_CONNTRACK flag for
> instance.

I was actually wondering if there could be a better solution instead of 
always sending UID/GID to userspace, thanks for your suggestion.

>> +			if (sk->sk_socket && sk->sk_socket->file) {
>> +				struct file *file = sk->sk_socket->file;
>> +				const struct cred *cred = file->f_cred;
>> +				if (nla_put_u32(skb, NFQA_UID,
> nla_put_be32
>
>> +				    htonl(cred->fsuid)))
>> +					goto nla_put_failure;
> careful - the sk_callback_lock was taken.

Sorry, I totally overlooked this one.
I will send soon a new patchset that addresses all your comments.

--
BR,
Val

>
> Cheers,
> Florian

Re: [PATCH 0/2] Add UID/GID info to NFQUEUE

[JP Abgrall]

On Wed, Dec 18, 2013 at 10:04 AM,  <valentina.giusti@bmw-carit.de> wrote:
> At the Linux Plumbers Conference 2013, there have been quite interesting
> discussions on the topic of network statistics, and it was proposed that it
> would make more sense to use NFQUEUE for this purpose, letting therefore
> userspace use the UID/GID information for application-based statistics purposes.
> This way the UID and GID information of the incoming TCP and UDP traffic is not
> "wasted" and can be used for more refined statistics.

Some things to keep in mind when doing these changes...
For mobile devices, having the UID/GID is not sufficient when services
are involved (download manager, http service, ...).
Currently, Android's xt_qtaguid netfilter module supports the notion
of "delegation" which is achieved by having the service "tag" a socket
(tracked internally based on the struct sock address), via the file
descriptor, with a UID of who should be blamed for the statistics.
It would be nice if NFQUEUE presented/tracked some data related to the
socket that could be used to that effect.
This tagging of the socket needs to have the following properties:
 - a socket can stay open and be re-tagged (happens when pools are
used or when a stream can be used for multiple reasons).
 - a socket can be tagged with a different UID only if the current
UID/GID/pcap allows for it.
 - an app can have multiple sockets, each one belonging to different
UIDs (so no UID to UID mapping).
There are a few more, but those can be managed outside of the NFQUEUE changes.

Concerning the comment about TCP_TIMEWAIT in some other thread: the sk
is not a "struct sock" but "struct inet_timewait_sock" which is
missing fields. So beware when manipulating it. xt_qtaguid had to
ignore those struct sock.