* Re: [PATCH net-next] xfrm: fix RCU bugs
From: Fan Du @ 2012-08-20 6:33 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, fengguang.wu, Priyanka Jain
In-Reply-To: <1345440800.5158.239.camel@edumazet-glaptop>
On 2012年08月20日 13:33, Eric Dumazet wrote:
> On Mon, 2012-08-20 at 12:40 +0800, Fan Du wrote:
>> Hi Eric
>>
>> Please correct me if I'm wrong about below comments.
>>
>> On 2012年08月19日 18:31, Eric Dumazet wrote:
>>> From: Eric Dumazet<edumazet@google.com>
>>>
>>> This patch reverts commit 56892261ed1a (xfrm: Use rcu_dereference_bh to
>>> deference pointer protected by rcu_read_lock_bh), and fixes bugs
>>> introduced in commit 418a99ac6ad ( Replace rwlock on xfrm_policy_afinfo
>>> with rcu )
>>>
>>> 1) We properly use RCU variant in this file, not a mix of RCU/RCU_BH
>>>
>>> 2) We must defer some writes after the synchronize_rcu() call or a reader
>>> can crash dereferencing NULL pointer.
>>
>> Not exactly.
>>
>> net/ipv4/xfrm4_policy.c
>> static void __exit xfrm4_policy_fini(void)
>> -> xfrm_policy_unregister_afinfo
>>
>> IMHO, ip stack can never be compiled as module, so is xfrm4_policy_fini
>> freed up after system bootup? which means xfrm4_policy_fini can never be
>> called.
>>
>> so an dereferencing NULL pointer by a reader could not happen.
>>
>
> Last famous words.
>
> Anyway xfrm_policy_unregister_afinfo() is also called from
> xfrm6_policy_fini(), and IPv6 is a module. The day we can rmmod it,
> we uncover this bug.
>
> RCU is complex (most people dont get it right, thats the truth),
> and we should make it rock solid, or I can guarantee you
> many patch attempts from future readers of this code.
>
> You wont tell them :
>
> "OK but dont worry we never call this function for real, why do you care
> at all"
>
You are correct!
And one out of topic question:
The usage of xfrm_state_afinfo_lock/xfrm_km_lock is extremely
similar with xfrm_policy_afinfo_lock, except the former is not so
frequently read than that of the later.
Is it justified to convert RW xfrm_state_afinfo_lock/xfrm_km_lock into
RCU?
>>>
>>> 3) Now we use the xfrm_policy_afinfo_lock spinlock only from process
>>> context, we no longer need to block BH in xfrm_policy_register_afinfo()
>>> and xfrm_policy_unregister_afinfo()
>>>
>> I don't think it's related to what kinds of locks we are using.
>> we call xfrm_policy_register_afinfo in process context, but actually
>> what xfrm_policy_afinfo_lock protected can be used in soft irq context.
>> that's why xx_bh is used in:
>
> You did an RCU conversion and obviously have little idea of what
> happened there.
>
> This _bh stuff was needed because _before_ RCU, an rwlock was used.
>
> And since read_lock() was used from BH handler, _all_ write_lock() had
> to use the write_lock_bh() variant to avoid a possible deadlock.
>
> But after RCU, this no longer is needed, as an rcu_read_lock() cannot
> block a writer anymore in the lock/unlock section.
>
> In fact, xfrm_policy_afinfo_lock could be replaced by a mutex. So _bh()
> is absolutely not needed anymore.
>
I indeed misunderstood the code a bit.
Your explanation is crystal clear, thanks :)
>
>
--
Love each day!
--fan
^ permalink raw reply
* [PATCH] iproute2: fix typo in help message.
From: Li Wei @ 2012-08-20 6:28 UTC (permalink / raw)
To: shemminger; +Cc: netdev, Li Wei
Signed-off-by: Li Wei <lw@cn.fujitsu.com>
---
bridge/fdb.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bridge/fdb.c b/bridge/fdb.c
index 2bbdaa6..c6aa08e 100644
--- a/bridge/fdb.c
+++ b/bridge/fdb.c
@@ -239,6 +239,6 @@ int do_fdb(int argc, char **argv)
} else
return fdb_show(0, NULL);
- fprintf(stderr, "Command \"%s\" is unknown, try \"ip neigh help\".\n", *argv);
+ fprintf(stderr, "Command \"%s\" is unknown, try \"bridge fdb help\".\n", *argv);
exit(-1);
}
--
1.7.10.1
^ permalink raw reply related
* [PATCH 09/11] nfnetlink: add support for memory mapped netlink
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netfilter/nfnetlink.h | 2 ++
net/netfilter/nfnetlink.c | 7 +++++++
net/netfilter/nfnetlink_log.c | 9 +++++----
net/netfilter/nfnetlink_queue_core.c | 2 +-
4 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/include/linux/netfilter/nfnetlink.h b/include/linux/netfilter/nfnetlink.h
index 18341cd..447d1a7 100644
--- a/include/linux/netfilter/nfnetlink.h
+++ b/include/linux/netfilter/nfnetlink.h
@@ -81,6 +81,8 @@ extern int nfnetlink_subsys_register(const struct nfnetlink_subsystem *n);
extern int nfnetlink_subsys_unregister(const struct nfnetlink_subsystem *n);
extern int nfnetlink_has_listeners(struct net *net, unsigned int group);
+extern struct sk_buff *nfnetlink_alloc_skb(struct net *net, unsigned int size,
+ u32 dst_pid, gfp_t gfp_mask);
extern int nfnetlink_send(struct sk_buff *skb, struct net *net, u32 pid, unsigned int group,
int echo, gfp_t flags);
extern int nfnetlink_set_err(struct net *net, u32 pid, u32 group, int error);
diff --git a/net/netfilter/nfnetlink.c b/net/netfilter/nfnetlink.c
index a265033..d0aa9fc 100644
--- a/net/netfilter/nfnetlink.c
+++ b/net/netfilter/nfnetlink.c
@@ -111,6 +111,13 @@ int nfnetlink_has_listeners(struct net *net, unsigned int group)
}
EXPORT_SYMBOL_GPL(nfnetlink_has_listeners);
+struct sk_buff *nfnetlink_alloc_skb(struct net *net, unsigned int size,
+ u32 dst_pid, gfp_t gfp_mask)
+{
+ return netlink_alloc_skb(net->nfnl, size, dst_pid, gfp_mask);
+}
+EXPORT_SYMBOL_GPL(nfnetlink_alloc_skb);
+
int nfnetlink_send(struct sk_buff *skb, struct net *net, u32 pid,
unsigned int group, int echo, gfp_t flags)
{
diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c
index 169ab59..6232ca8 100644
--- a/net/netfilter/nfnetlink_log.c
+++ b/net/netfilter/nfnetlink_log.c
@@ -296,7 +296,7 @@ nfulnl_set_flags(struct nfulnl_instance *inst, u_int16_t flags)
}
static struct sk_buff *
-nfulnl_alloc_skb(unsigned int inst_size, unsigned int pkt_size)
+nfulnl_alloc_skb(u32 peer_pid, unsigned int inst_size, unsigned int pkt_size)
{
struct sk_buff *skb;
unsigned int n;
@@ -305,13 +305,14 @@ nfulnl_alloc_skb(unsigned int inst_size, unsigned int pkt_size)
* message. WARNING: has to be <= 128k due to slab restrictions */
n = max(inst_size, pkt_size);
- skb = alloc_skb(n, GFP_ATOMIC);
+ skb = nfnetlink_alloc_skb(&init_net, n, peer_pid, GFP_ATOMIC);
if (!skb) {
if (n > pkt_size) {
/* try to allocate only as much as we need for current
* packet */
- skb = alloc_skb(pkt_size, GFP_ATOMIC);
+ skb = nfnetlink_alloc_skb(&init_net, pkt_size, peer_pid,
+ GFP_ATOMIC);
if (!skb)
pr_err("nfnetlink_log: can't even alloc %u bytes\n",
pkt_size);
@@ -658,7 +659,7 @@ nfulnl_log_packet(u_int8_t pf,
}
if (!inst->skb) {
- inst->skb = nfulnl_alloc_skb(inst->nlbufsiz, size);
+ inst->skb = nfulnl_alloc_skb(inst->peer_pid, inst->nlbufsiz, size);
if (!inst->skb)
goto alloc_failure;
}
diff --git a/net/netfilter/nfnetlink_queue_core.c b/net/netfilter/nfnetlink_queue_core.c
index c0496a5..6983b2e 100644
--- a/net/netfilter/nfnetlink_queue_core.c
+++ b/net/netfilter/nfnetlink_queue_core.c
@@ -272,7 +272,7 @@ nfqnl_build_packet_message(struct nfqnl_instance *queue,
if (queue->flags & NFQA_CFG_F_CONNTRACK)
ct = nfqnl_ct_get(entskb, &size, &ctinfo);
- skb = alloc_skb(size, GFP_ATOMIC);
+ skb = nfnetlink_alloc_skb(&init_net, size, queue->peer_pid, GFP_ATOMIC);
if (!skb)
return NULL;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 06/11] netlink: add mmap'ed netlink helper functions
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Add helper functions for looking up mmap'ed frame headers, reading and writing
their status, allocating skbs with mmap'ed data areas and a poll function.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netlink.h | 8 ++
net/netlink/af_netlink.c | 185 +++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 191 insertions(+), 2 deletions(-)
diff --git a/include/linux/netlink.h b/include/linux/netlink.h
index 3600461..98754e8 100644
--- a/include/linux/netlink.h
+++ b/include/linux/netlink.h
@@ -193,10 +193,18 @@ static inline struct nlmsghdr *nlmsg_hdr(const struct sk_buff *skb)
return (struct nlmsghdr *)skb->data;
}
+enum netlink_skb_flags {
+ NETLINK_SKB_MMAPED = 0x1, /* Packet data is mmapped */
+ NETLINK_SKB_TX = 0x2, /* Packet was sent by userspace */
+ NETLINK_SKB_DELIVERED = 0x4, /* Packet was delivered */
+};
+
struct netlink_skb_parms {
struct ucred creds; /* Skb credentials */
__u32 pid;
__u32 dst_group;
+ __u32 flags;
+ struct sock *sk; /* socket owning mmaped ring */
};
#define NETLINK_CB(skb) (*(struct netlink_skb_parms*)&((skb)->cb))
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index e20a918..3810911 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -56,6 +56,7 @@
#include <linux/audit.h>
#include <linux/mutex.h>
#include <linux/vmalloc.h>
+#include <asm/cacheflush.h>
#include <net/net_namespace.h>
#include <net/sock.h>
@@ -159,6 +160,7 @@ static struct netlink_table *nl_table;
static DECLARE_WAIT_QUEUE_HEAD(nl_table_wait);
static int netlink_dump(struct sock *sk);
+static void netlink_skb_destructor(struct sk_buff *skb);
static DEFINE_RWLOCK(nl_table_lock);
static atomic_t nl_table_users = ATOMIC_INIT(0);
@@ -176,6 +178,11 @@ static inline struct hlist_head *nl_pid_hashfn(struct nl_pid_hash *hash, u32 pid
}
#ifdef CONFIG_NETLINK_MMAP
+static bool netlink_skb_is_mmaped(const struct sk_buff *skb)
+{
+ return NETLINK_CB(skb).flags & NETLINK_SKB_MMAPED;
+}
+
static __pure struct page *pgvec_to_page(const void *addr)
{
if (is_vmalloc_addr(addr))
@@ -399,8 +406,154 @@ out:
mutex_unlock(&nlk->pg_vec_lock);
return 0;
}
+
+static void netlink_frame_flush_dcache(const struct nl_mmap_hdr *hdr)
+{
+#if ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE == 1
+ struct page *p_start, *p_end;
+
+ /* First page is flushed through netlink_{get,set}_status */
+ p_start = pgvec_to_page(hdr + PAGE_SIZE);
+ p_end = pgvec_to_page((void *)hdr + NL_MMAP_MSG_HDRLEN + hdr->nm_len - 1);
+ while (p_start <= p_end) {
+ flush_dcache_page(p_start);
+ p_start++;
+ }
+#endif
+}
+
+static enum nl_mmap_status netlink_get_status(const struct nl_mmap_hdr *hdr)
+{
+ smp_rmb();
+ flush_dcache_page(pgvec_to_page(hdr));
+ return hdr->nm_status;
+}
+
+static void netlink_set_status(struct nl_mmap_hdr *hdr,
+ enum nl_mmap_status status)
+{
+ hdr->nm_status = status;
+ flush_dcache_page(pgvec_to_page(hdr));
+ smp_wmb();
+}
+
+static struct nl_mmap_hdr *
+__netlink_lookup_frame(const struct netlink_ring *ring, unsigned int pos)
+{
+ unsigned int pg_vec_pos, frame_off;
+
+ pg_vec_pos = pos / ring->frames_per_block;
+ frame_off = pos % ring->frames_per_block;
+
+ return ring->pg_vec[pg_vec_pos] + (frame_off * ring->frame_size);
+}
+
+static struct nl_mmap_hdr *
+netlink_lookup_frame(const struct netlink_ring *ring, unsigned int pos,
+ enum nl_mmap_status status)
+{
+ struct nl_mmap_hdr *hdr;
+
+ hdr = __netlink_lookup_frame(ring, pos);
+ if (netlink_get_status(hdr) != status)
+ return NULL;
+
+ return hdr;
+}
+
+static struct nl_mmap_hdr *
+netlink_current_frame(const struct netlink_ring *ring,
+ enum nl_mmap_status status)
+{
+ return netlink_lookup_frame(ring, ring->head, status);
+}
+
+static struct nl_mmap_hdr *
+netlink_previous_frame(const struct netlink_ring *ring,
+ enum nl_mmap_status status)
+{
+ unsigned int prev;
+
+ prev = ring->head ? ring->head - 1 : ring->frame_max;
+ return netlink_lookup_frame(ring, prev, status);
+}
+
+static void netlink_increment_head(struct netlink_ring *ring)
+{
+ ring->head = ring->head != ring->frame_max ? ring->head + 1 : 0;
+}
+
+static void netlink_forward_ring(struct netlink_ring *ring)
+{
+ unsigned int head = ring->head, pos = head;
+ const struct nl_mmap_hdr *hdr;
+
+ do {
+ hdr = __netlink_lookup_frame(ring, pos);
+ if (hdr->nm_status == NL_MMAP_STATUS_UNUSED)
+ break;
+ if (hdr->nm_status != NL_MMAP_STATUS_SKIP)
+ break;
+ netlink_increment_head(ring);
+ } while (ring->head != head);
+}
+
+static unsigned int netlink_poll(struct file *file, struct socket *sock,
+ poll_table *wait)
+{
+ struct sock *sk = sock->sk;
+ struct netlink_sock *nlk = nlk_sk(sk);
+ unsigned int mask;
+
+ mask = datagram_poll(file, sock, wait);
+
+ spin_lock_bh(&sk->sk_receive_queue.lock);
+ if (nlk->rx_ring.pg_vec) {
+ netlink_forward_ring(&nlk->rx_ring);
+ if (!netlink_previous_frame(&nlk->rx_ring, NL_MMAP_STATUS_UNUSED))
+ mask |= POLLIN | POLLRDNORM;
+ }
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+
+ spin_lock_bh(&sk->sk_write_queue.lock);
+ if (nlk->tx_ring.pg_vec) {
+ if (netlink_current_frame(&nlk->tx_ring, NL_MMAP_STATUS_UNUSED))
+ mask |= POLLOUT | POLLWRNORM;
+ }
+ spin_unlock_bh(&sk->sk_write_queue.lock);
+
+ return mask;
+}
+
+static struct nl_mmap_hdr *netlink_mmap_hdr(struct sk_buff *skb)
+{
+ return (struct nl_mmap_hdr *)(skb->head - NL_MMAP_HDRLEN);
+}
+
+static void netlink_ring_setup_skb(struct sk_buff *skb, struct sock *sk,
+ struct netlink_ring *ring,
+ struct nl_mmap_hdr *hdr)
+{
+ unsigned int size;
+ void *data;
+
+ size = ring->frame_size - NL_MMAP_HDRLEN;
+ data = (void *)hdr + NL_MMAP_HDRLEN;
+
+ skb->head = data;
+ skb->data = data;
+ skb_reset_tail_pointer(skb);
+ skb->end = skb->tail + size;
+ skb->len = 0;
+
+ skb->destructor = netlink_skb_destructor;
+ NETLINK_CB(skb).flags |= NETLINK_SKB_MMAPED;
+ NETLINK_CB(skb).sk = sk;
+}
#else /* CONFIG_NETLINK_MMAP */
+#define netlink_skb_is_mmaped(skb) false
#define netlink_mmap sock_no_mmap
+#define netlink_poll datagram_poll
#endif /* CONFIG_NETLINK_MMAP */
static void netlink_destroy_callback(struct netlink_callback *cb)
@@ -417,7 +570,35 @@ static void netlink_consume_callback(struct netlink_callback *cb)
static void netlink_skb_destructor(struct sk_buff *skb)
{
- sock_rfree(skb);
+#ifdef CONFIG_NETLINK_MMAP
+ struct nl_mmap_hdr *hdr;
+ struct netlink_ring *ring;
+ struct sock *sk;
+
+ /* If a packet from the kernel to userspace was freed because of an
+ * error without being delivered to userspace, the kernel must reset
+ * the status. In the direction userspace to kernel, the status is
+ * always reset here after the packet was processed and freed.
+ */
+ if (netlink_skb_is_mmaped(skb)) {
+ hdr = netlink_mmap_hdr(skb);
+ sk = NETLINK_CB(skb).sk;
+
+ if (!(NETLINK_CB(skb).flags & NETLINK_SKB_DELIVERED)) {
+ hdr->nm_len = 0;
+ netlink_set_status(hdr, NL_MMAP_STATUS_VALID);
+ }
+ ring = &nlk_sk(sk)->rx_ring;
+
+ WARN_ON(atomic_read(&ring->pending) == 0);
+ atomic_dec(&ring->pending);
+ sock_put(sk);
+
+ skb->data = NULL;
+ }
+#endif
+ if (skb->sk != NULL)
+ sock_rfree(skb);
}
static void netlink_skb_set_owner_r(struct sk_buff *skb, struct sock *sk)
@@ -2400,7 +2581,7 @@ static const struct proto_ops netlink_ops = {
.socketpair = sock_no_socketpair,
.accept = sock_no_accept,
.getname = netlink_getname,
- .poll = datagram_poll,
+ .poll = netlink_poll,
.ioctl = sock_no_ioctl,
.listen = sock_no_listen,
.shutdown = sock_no_shutdown,
--
1.7.7.6
^ permalink raw reply related
* [PATCH 05/11] netlink: mmaped netlink: ring setup
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Add support for mmap'ed RX and TX ring setup and teardown based on the
af_packet.c code. The following patches will use this to add the real
mmap'ed receive and transmit functionality.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netlink.h | 32 +++++
net/Kconfig | 9 ++
net/netlink/af_netlink.c | 288 +++++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 327 insertions(+), 2 deletions(-)
diff --git a/include/linux/netlink.h b/include/linux/netlink.h
index f74dd13..3600461 100644
--- a/include/linux/netlink.h
+++ b/include/linux/netlink.h
@@ -1,6 +1,7 @@
#ifndef __LINUX_NETLINK_H
#define __LINUX_NETLINK_H
+#include <linux/kernel.h>
#include <linux/socket.h> /* for __kernel_sa_family_t */
#include <linux/types.h>
@@ -105,11 +106,42 @@ struct nlmsgerr {
#define NETLINK_PKTINFO 3
#define NETLINK_BROADCAST_ERROR 4
#define NETLINK_NO_ENOBUFS 5
+#define NETLINK_RX_RING 6
+#define NETLINK_TX_RING 7
struct nl_pktinfo {
__u32 group;
};
+struct nl_mmap_req {
+ unsigned int nm_block_size;
+ unsigned int nm_block_nr;
+ unsigned int nm_frame_size;
+ unsigned int nm_frame_nr;
+};
+
+struct nl_mmap_hdr {
+ unsigned int nm_status;
+ unsigned int nm_len;
+ __u32 nm_group;
+ /* credentials */
+ __u32 nm_pid;
+ __u32 nm_uid;
+ __u32 nm_gid;
+};
+
+enum nl_mmap_status {
+ NL_MMAP_STATUS_UNUSED,
+ NL_MMAP_STATUS_RESERVED,
+ NL_MMAP_STATUS_VALID,
+ NL_MMAP_STATUS_COPY,
+ NL_MMAP_STATUS_SKIP,
+};
+
+#define NL_MMAP_MSG_ALIGNMENT NLMSG_ALIGNTO
+#define NL_MMAP_MSG_ALIGN(sz) __ALIGN_KERNEL(sz, NL_MMAP_MSG_ALIGNMENT)
+#define NL_MMAP_HDRLEN NL_MMAP_MSG_ALIGN(sizeof(struct nl_mmap_hdr))
+
#define NET_MAJOR 36 /* Major 36 is reserved for networking */
enum {
diff --git a/net/Kconfig b/net/Kconfig
index 245831b..7f48691 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -23,6 +23,15 @@ menuconfig NET
if NET
+config NETLINK_MMAP
+ bool "Netlink: mmaped IO"
+ help
+ This option enables support for memory mapped netlink IO. This
+ reduces overhead by avoiding copying data between kernel- and
+ userspace.
+
+ If unsure, say N.
+
config WANT_COMPAT_NETLINK_MESSAGES
bool
help
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index e2d0177..e20a918 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -55,6 +55,7 @@
#include <linux/types.h>
#include <linux/audit.h>
#include <linux/mutex.h>
+#include <linux/vmalloc.h>
#include <net/net_namespace.h>
#include <net/sock.h>
@@ -64,6 +65,20 @@
#define NLGRPSZ(x) (ALIGN(x, sizeof(unsigned long) * 8) / 8)
#define NLGRPLONGS(x) (NLGRPSZ(x)/sizeof(unsigned long))
+struct netlink_ring {
+ void **pg_vec;
+ unsigned int head;
+ unsigned int frames_per_block;
+ unsigned int frame_size;
+ unsigned int frame_max;
+
+ unsigned int pg_vec_order;
+ unsigned int pg_vec_pages;
+ unsigned int pg_vec_len;
+
+ atomic_t pending;
+};
+
struct netlink_sock {
/* struct sock has to be the first member of netlink_sock */
struct sock sk;
@@ -82,6 +97,12 @@ struct netlink_sock {
void (*netlink_rcv)(struct sk_buff *skb);
void (*netlink_bind)(int group);
struct module *module;
+#ifdef CONFIG_NETLINK_MMAP
+ struct mutex pg_vec_lock;
+ struct netlink_ring rx_ring;
+ struct netlink_ring tx_ring;
+ atomic_t mapped;
+#endif /* CONFIG_NETLINK_MMAP */
};
struct listeners {
@@ -154,6 +175,234 @@ static inline struct hlist_head *nl_pid_hashfn(struct nl_pid_hash *hash, u32 pid
return &hash->table[jhash_1word(pid, hash->rnd) & hash->mask];
}
+#ifdef CONFIG_NETLINK_MMAP
+static __pure struct page *pgvec_to_page(const void *addr)
+{
+ if (is_vmalloc_addr(addr))
+ return vmalloc_to_page(addr);
+ else
+ return virt_to_page(addr);
+}
+
+static void free_pg_vec(void **pg_vec, unsigned int order, unsigned int len)
+{
+ unsigned int i;
+
+ for (i = 0; i < len; i++) {
+ if (pg_vec[i] != NULL) {
+ if (is_vmalloc_addr(pg_vec[i]))
+ vfree(pg_vec[i]);
+ else
+ free_pages((unsigned long)pg_vec[i], order);
+ }
+ }
+ kfree(pg_vec);
+}
+
+static void *alloc_one_pg_vec_page(unsigned long order)
+{
+ void *buffer;
+ gfp_t gfp_flags = GFP_KERNEL | __GFP_COMP | __GFP_ZERO |
+ __GFP_NOWARN | __GFP_NORETRY;
+
+ buffer = (void *)__get_free_pages(gfp_flags, order);
+ if (buffer != NULL)
+ return buffer;
+
+ buffer = vzalloc((1 << order) * PAGE_SIZE);
+ if (buffer != NULL)
+ return buffer;
+
+ gfp_flags &= ~__GFP_NORETRY;
+ return (void *)__get_free_pages(gfp_flags, order);
+}
+
+static void **alloc_pg_vec(struct netlink_sock *nlk,
+ struct nl_mmap_req *req, unsigned int order)
+{
+ unsigned int block_nr = req->nm_block_nr;
+ unsigned int i;
+ void **pg_vec, *ptr;
+
+ pg_vec = kcalloc(block_nr, sizeof(void *), GFP_KERNEL);
+ if (pg_vec == NULL)
+ return NULL;
+
+ for (i = 0; i < block_nr; i++) {
+ pg_vec[i] = ptr = alloc_one_pg_vec_page(order);
+ if (pg_vec[i] == NULL)
+ goto err1;
+ }
+
+ return pg_vec;
+err1:
+ free_pg_vec(pg_vec, order, block_nr);
+ return NULL;
+}
+
+static int netlink_set_ring(struct sock *sk, struct nl_mmap_req *req,
+ bool closing, bool tx_ring)
+{
+ struct netlink_sock *nlk = nlk_sk(sk);
+ struct netlink_ring *ring;
+ struct sk_buff_head *queue;
+ void **pg_vec = NULL;
+ unsigned int order = 0;
+ int err;
+
+ ring = tx_ring ? &nlk->tx_ring : &nlk->rx_ring;
+ queue = tx_ring ? &sk->sk_write_queue : &sk->sk_receive_queue;
+
+ if (!closing) {
+ if (atomic_read(&nlk->mapped))
+ return -EBUSY;
+ if (atomic_read(&ring->pending))
+ return -EBUSY;
+ }
+
+ if (req->nm_block_nr) {
+ if (ring->pg_vec != NULL)
+ return -EBUSY;
+
+ if ((int)req->nm_block_size <= 0)
+ return -EINVAL;
+ if (!IS_ALIGNED(req->nm_block_size, PAGE_SIZE))
+ return -EINVAL;
+ if (req->nm_frame_size < NL_MMAP_HDRLEN)
+ return -EINVAL;
+ if (!IS_ALIGNED(req->nm_frame_size, NL_MMAP_MSG_ALIGNMENT))
+ return -EINVAL;
+
+ ring->frames_per_block = req->nm_block_size /
+ req->nm_frame_size;
+ if (ring->frames_per_block == 0)
+ return -EINVAL;
+ if (ring->frames_per_block * req->nm_block_nr !=
+ req->nm_frame_nr)
+ return -EINVAL;
+
+ order = get_order(req->nm_block_size);
+ pg_vec = alloc_pg_vec(nlk, req, order);
+ if (pg_vec == NULL)
+ return -ENOMEM;
+ } else {
+ if (req->nm_frame_nr)
+ return -EINVAL;
+ }
+
+ err = -EBUSY;
+ mutex_lock(&nlk->pg_vec_lock);
+ if (closing || atomic_read(&nlk->mapped) == 0) {
+ err = 0;
+ spin_lock_bh(&queue->lock);
+
+ ring->frame_max = req->nm_frame_nr - 1;
+ ring->head = 0;
+ ring->frame_size = req->nm_frame_size;
+ ring->pg_vec_pages = req->nm_block_size / PAGE_SIZE;
+
+ swap(ring->pg_vec_len, req->nm_block_nr);
+ swap(ring->pg_vec_order, order);
+ swap(ring->pg_vec, pg_vec);
+
+ __skb_queue_purge(queue);
+ spin_unlock_bh(&queue->lock);
+
+ WARN_ON(atomic_read(&nlk->mapped));
+ }
+ mutex_unlock(&nlk->pg_vec_lock);
+
+ if (pg_vec)
+ free_pg_vec(pg_vec, order, req->nm_block_nr);
+ return err;
+}
+
+static void netlink_mm_open(struct vm_area_struct *vma)
+{
+ struct file *file = vma->vm_file;
+ struct socket *sock = file->private_data;
+ struct sock *sk = sock->sk;
+
+ if (sk)
+ atomic_inc(&nlk_sk(sk)->mapped);
+}
+
+static void netlink_mm_close(struct vm_area_struct *vma)
+{
+ struct file *file = vma->vm_file;
+ struct socket *sock = file->private_data;
+ struct sock *sk = sock->sk;
+
+ if (sk)
+ atomic_dec(&nlk_sk(sk)->mapped);
+}
+
+static const struct vm_operations_struct netlink_mmap_ops = {
+ .open = netlink_mm_open,
+ .close = netlink_mm_close,
+};
+
+static int netlink_mmap(struct file *file, struct socket *sock,
+ struct vm_area_struct *vma)
+{
+ struct sock *sk = sock->sk;
+ struct netlink_sock *nlk = nlk_sk(sk);
+ struct netlink_ring *ring;
+ unsigned long start, size, expected;
+ unsigned int i;
+ int err = -EINVAL;
+
+ if (vma->vm_pgoff)
+ return -EINVAL;
+
+ mutex_lock(&nlk->pg_vec_lock);
+
+ expected = 0;
+ for (ring = &nlk->rx_ring; ring <= &nlk->tx_ring; ring++) {
+ if (ring->pg_vec == NULL)
+ continue;
+ expected += ring->pg_vec_len * ring->pg_vec_pages * PAGE_SIZE;
+ }
+
+ if (expected == 0)
+ goto out;
+
+ size = vma->vm_end - vma->vm_start;
+ if (size != expected)
+ goto out;
+
+ start = vma->vm_start;
+ for (ring = &nlk->rx_ring; ring <= &nlk->tx_ring; ring++) {
+ if (ring->pg_vec == NULL)
+ continue;
+
+ for (i = 0; i < ring->pg_vec_len; i++) {
+ struct page *page;
+ void *kaddr = ring->pg_vec[i];
+ unsigned int pg_num;
+
+ for (pg_num = 0; pg_num < ring->pg_vec_pages; pg_num++) {
+ page = pgvec_to_page(kaddr);
+ err = vm_insert_page(vma, start, page);
+ if (err < 0)
+ goto out;
+ start += PAGE_SIZE;
+ kaddr += PAGE_SIZE;
+ }
+ }
+ }
+
+ atomic_inc(&nlk->mapped);
+ vma->vm_ops = &netlink_mmap_ops;
+ err = 0;
+out:
+ mutex_unlock(&nlk->pg_vec_lock);
+ return 0;
+}
+#else /* CONFIG_NETLINK_MMAP */
+#define netlink_mmap sock_no_mmap
+#endif /* CONFIG_NETLINK_MMAP */
+
static void netlink_destroy_callback(struct netlink_callback *cb)
{
kfree_skb(cb->skb);
@@ -191,6 +440,18 @@ static void netlink_sock_destruct(struct sock *sk)
}
skb_queue_purge(&sk->sk_receive_queue);
+#ifdef CONFIG_NETLINK_MMAP
+ if (1) {
+ struct nl_mmap_req req;
+
+ memset(&req, 0, sizeof(req));
+ if (nlk->rx_ring.pg_vec)
+ netlink_set_ring(sk, &req, true, false);
+ memset(&req, 0, sizeof(req));
+ if (nlk->tx_ring.pg_vec)
+ netlink_set_ring(sk, &req, true, true);
+ }
+#endif /* CONFIG_NETLINK_MMAP */
if (!sock_flag(sk, SOCK_DEAD)) {
printk(KERN_ERR "Freeing alive netlink socket %p\n", sk);
@@ -452,6 +713,9 @@ static int __netlink_create(struct net *net, struct socket *sock,
mutex_init(nlk->cb_mutex);
}
init_waitqueue_head(&nlk->wait);
+#ifdef CONFIG_NETLINK_MMAP
+ mutex_init(&nlk->pg_vec_lock);
+#endif
sk->sk_destruct = netlink_sock_destruct;
sk->sk_protocol = protocol;
@@ -1246,7 +1510,8 @@ static int netlink_setsockopt(struct socket *sock, int level, int optname,
if (level != SOL_NETLINK)
return -ENOPROTOOPT;
- if (optlen >= sizeof(int) &&
+ if (optname != NETLINK_RX_RING && optname != NETLINK_TX_RING &&
+ optlen >= sizeof(int) &&
get_user(val, (unsigned int __user *)optval))
return -EFAULT;
@@ -1295,6 +1560,25 @@ static int netlink_setsockopt(struct socket *sock, int level, int optname,
}
err = 0;
break;
+#ifdef CONFIG_NETLINK_MMAP
+ case NETLINK_RX_RING:
+ case NETLINK_TX_RING: {
+ struct nl_mmap_req req;
+
+ /* Rings might consume more memory than queue limits, require
+ * CAP_NET_ADMIN.
+ */
+ if (!capable(CAP_NET_ADMIN))
+ return -EPERM;
+ if (optlen < sizeof(req))
+ return -EINVAL;
+ if (copy_from_user(&req, optval, sizeof(req)))
+ return -EFAULT;
+ err = netlink_set_ring(sk, &req, false,
+ optname == NETLINK_TX_RING);
+ break;
+ }
+#endif /* CONFIG_NETLINK_MMAP */
default:
err = -ENOPROTOOPT;
}
@@ -2124,7 +2408,7 @@ static const struct proto_ops netlink_ops = {
.getsockopt = netlink_getsockopt,
.sendmsg = netlink_sendmsg,
.recvmsg = netlink_recvmsg,
- .mmap = sock_no_mmap,
+ .mmap = netlink_mmap,
.sendpage = sock_no_sendpage,
};
--
1.7.7.6
^ permalink raw reply related
* [PATCH 11/11] netlink: add documentation for memory mapped I/O
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
Documentation/networking/netlink_mmap.txt | 337 +++++++++++++++++++++++++++++
1 files changed, 337 insertions(+), 0 deletions(-)
create mode 100644 Documentation/networking/netlink_mmap.txt
diff --git a/Documentation/networking/netlink_mmap.txt b/Documentation/networking/netlink_mmap.txt
new file mode 100644
index 0000000..413c521
--- /dev/null
+++ b/Documentation/networking/netlink_mmap.txt
@@ -0,0 +1,337 @@
+This file documents how to use memory mapped I/O with netlink.
+
+Overview
+--------
+
+Memory mapped netlink I/O can be used to increase throughput and decrease
+overhead of unicast receive and transmit operations. Some netlink subsystems
+require high throughput, these are mainly the netfilter subsystems
+nfnetlink_queue and nfnetlink_log, but it can also help speed up large
+dump operations of f.i. the routing database.
+
+Memory mapped netlink I/O used two circular ring buffers for RX and TX which
+are mapped into the processes address space.
+
+The RX ring is used by the kernel to directly construct netlink messages into
+user-space memory without copying them as done with regular socket I/O,
+additionally as long as the ring contains messages no recvmsg() or poll()
+syscalls have to be issued by user-space to get more message.
+
+The TX ring is used to process messages directly from user-space memory, the
+kernel processes all messages contained in the ring using a single sendmsg()
+call.
+
+Usage overview
+--------------
+
+In order to use memory mapped netlink I/O, user-space needs three main changes:
+
+- ring setup
+- conversion of the RX path to get messages from the ring instead of recvmsg()
+- conversion of the TX path to construct messages into the ring
+
+Ring setup is done using setsockopt() to provide the ring parameters to the
+kernel, then a call to mmap() to map the ring into the processes address space:
+
+- setsockopt(fd, SOL_NETLINK, NETLINK_RX_RING, ¶ms, sizeof(params));
+- setsockopt(fd, SOL_NETLINK, NETLINK_TX_RING, ¶ms, sizeof(params));
+- ring = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)
+
+Usage of either ring is optional, but even if only the RX ring is used the
+mapping still needs to be writable in order to update the frame status after
+processing.
+
+Conversion of the reception path involves calling poll() on the file
+descriptor, once the socket is readable the frames from the ring are
+processsed in order until no more messages are available, as indicated by
+a status word in the frame header.
+
+On kernel side, in order to make use of memory mapped I/O on receive, the
+originating netlink subsystem needs to support memory mapped I/O, otherwise
+it will use an allocated socket buffer as usual and the contents will be
+ copied to the ring on transmission, nullifying most of the performance gains.
+Dumps of kernel databases automatically support memory mapped I/O.
+
+Conversion of the transmit path involves changing message contruction to
+use memory from the TX ring instead of (usually) a buffer declared on the
+stack and setting up the frame header approriately. Optionally poll() can
+be used to wait for free frames in the TX ring.
+
+Structured and definitions for using memory mapped I/O are contained in
+<linux/netlink.h>.
+
+RX and TX rings
+----------------
+
+Each ring contains a number of continous memory blocks, containing frames of
+fixed size dependant on the parameters used for ring setup.
+
+Ring: [ block 0 ]
+ [ frame 0 ]
+ [ frame 1 ]
+ [ block 1 ]
+ [ frame 2 ]
+ [ frame 3 ]
+ ...
+ [ block n ]
+ [ frame 2 * n ]
+ [ frame 2 * n + 1 ]
+
+The blocks are only visible to the kernel, from the point of view of user-space
+the ring just contains the frames in a continous memory zone.
+
+The ring parameters used for setting up the ring are defined as follows:
+
+struct nl_mmap_req {
+ unsigned int nm_block_size;
+ unsigned int nm_block_nr;
+ unsigned int nm_frame_size;
+ unsigned int nm_frame_nr;
+};
+
+Frames are grouped into blocks, where each block is a continous region of memory
+and holds nm_block_size / nm_frame_size frames. The total number of frames in
+the ring is nm_frame_nr. The following invariants hold:
+
+- frames_per_block = nm_block_size / nm_frame_size
+
+- nm_frame_nr = frames_per_block * nm_block_nr
+
+Some parameters are constrained, specifically:
+
+- nm_block_size must be a multiple of the architectures memory page size.
+ The getpagesize() function can be used to get the page size.
+
+- nm_frame_size must be equal or larger to NL_MMAP_HDRLEN, IOW a frame must be
+ able to hold at least the frame header
+
+- nm_frame_size must be smaller or equal to nm_block_size
+
+- nm_frame_size must be a multiple of NL_MMAP_MSG_ALIGNMENT
+
+- nm_frame_nr must equal the actual number of frames as specified above.
+
+When the kernel can't allocate phsyically continous memory for a ring block,
+it will fall back to use physically discontinous memory. This might affect
+performance negatively, in order to avoid this the nm_frame_size parameter
+should be chosen to be as small as possible for the required frame size and
+the number of blocks should be increased instead.
+
+Ring frames
+------------
+
+Each frames contain a frame header, consisting of a synchronization word and some
+meta-data, and the message itself.
+
+Frame: [ header message ]
+
+The frame header is defined as follows:
+
+struct nl_mmap_hdr {
+ unsigned int nm_status;
+ unsigned int nm_len;
+ __u32 nm_group;
+ /* credentials */
+ __u32 nm_pid;
+ __u32 nm_uid;
+ __u32 nm_gid;
+};
+
+- nm_status is used for synchronizing processing between the kernel and user-
+ space and specifies ownership of the frame as well as the operation to perform
+
+- nm_len contains the length of the message contained in the data area
+
+- nm_group specified the destination multicast group of message
+
+- nm_pid, nm_uid and nm_gid contain the netlink pid, UID and GID of the sending
+ process. These values correspond to the data available using SOCK_PASSCRED in
+ the SCM_CREDENTIALS cmsg.
+
+The possible values in the status word are:
+
+- NL_MMAP_STATUS_UNUSED:
+ RX ring: frame belongs to the kernel and contains no message
+ for user-space. Approriate action is to invoke poll()
+ to wait for new messages.
+
+ TX ring: frame belongs to user-space and can be used for
+ message construction.
+
+- NL_MMAP_STATUS_RESERVED:
+ RX ring only: frame is currently used by the kernel for message
+ construction and contains no valid message yet.
+ Appropriate action is to invoke poll() to wait for
+ new messages.
+
+- NL_MMAP_STATUS_VALID:
+ RX ring: frame contains a valid message. Approriate action is
+ to process the message and release the frame back to
+ the kernel by setting the status to
+ NL_MMAP_STATUS_UNUSED or queue the frame by setting the
+ status to NL_MMAP_STATUS_SKIP.
+
+ TX ring: the frame contains a valid message from user-space to
+ be processed by the kernel. After completing processing
+ the kernel will release the frame back to user-space by
+ setting the status to NL_MMAP_STATUS_UNUSED.
+
+- NL_MMAP_STATUS_COPY:
+ RX ring only: a message is ready to be processed but could not be
+ stored in the ring, either because it exceeded the
+ frame size or because the originating subsystem does
+ not support memory mapped I/O. Appropriate action is
+ to invoke recvmsg() to receive the message and release
+ the frame back to the kernel by setting the status to
+ NL_MMAP_STATUS_UNUSED.
+
+- NL_MMAP_STATUS_SKIP:
+ RX ring only: user-space queued the message for later processing, but
+ processed some messages following it in the ring. The
+ kernel should skip this frame when looking for unused
+ frames.
+
+The data area of a frame begins at a offset of NL_MMAP_HDRLEN relative to the
+frame header.
+
+TX limitations
+--------------
+
+Kernel processing usually involves validation of the message received by
+user-space, then processing its contents. The kernel must assure that
+userspace is not able to modify the message contents after they have been
+validated. In order to do so, the message is copied from the ring frame
+to an allocated buffer if either of these conditions is false:
+
+- only a single mapping of the ring exists
+- the file descriptor is not shared between processes
+
+This means that for threaded programs, the kernel will fall back to copying.
+
+Example
+-------
+
+Ring setup:
+
+ unsigned int block_size = 16 * getpagesize();
+ struct nl_mmap_req req = {
+ .nm_block_size = block_size,
+ .nm_block_nr = 64,
+ .nm_frame_size = 16384,
+ .nm_frame_nr = 64 * block_size / 16384,
+ };
+ unsigned int ring_size;
+ void *rx_ring, *tx_ring;
+
+ /* Configure ring parameters */
+ if (setsockopt(fd, NETLINK_RX_RING, &req, sizeof(req)) < 0)
+ exit(1);
+ if (setsockopt(fd, NETLINK_TX_RING, &req, sizeof(req)) < 0)
+ exit(1)
+
+ /* Calculate size of each invididual ring */
+ ring_size = req.nm_block_nr * req.nm_block_size;
+
+ /* Map RX/TX rings. The TX ring is located after the RX ring */
+ rx_ring = mmap(NULL, 2 * ring_size, PROT_READ | PROT_WRITE,
+ MAP_SHARED, fd, 0);
+ if ((long)rx_ring == -1L)
+ exit(1);
+ tx_ring = rx_ring + ring_size:
+
+Message reception:
+
+This example assumes some ring parameters of the ring setup are available.
+
+ unsigned int frame_offset = 0;
+ struct nl_mmap_hdr *hdr;
+ struct nlmsghdr *nlh;
+ unsigned char buf[16384];
+ ssize_t len;
+
+ while (1) {
+ struct pollfd pfds[1];
+
+ pfds[0].fd = fd;
+ pfds[0].events = POLLIN | POLLERR;
+ pfds[0].revents = 0;
+
+ if (poll(pfds, 1, -1) < 0 && errno != -EINTR)
+ exit(1);
+
+ /* Check for errors. Error handling omitted */
+ if (pfds[0].revents & POLLERR)
+ <handle error>
+
+ /* If no new messages, poll again */
+ if (!(pfds[0].revents & POLLIN))
+ continue;
+
+ /* Process all frames */
+ while (1) {
+ /* Get next frame header */
+ hdr = rx_ring + frame_offset;
+
+ if (hdr->nm_status == NL_MMAP_STATUS_VALID)
+ /* Regular memory mapped frame */
+ nlh = (void *hdr) + NL_MMAP_HDRLEN;
+ len = hdr->nm_len;
+
+ /* Release empty message immediately. May happen
+ * on error during message construction.
+ */
+ if (len == 0)
+ goto release;
+ } else if (hdr->nm_status == NL_MMAP_STATUS_COPY) {
+ /* Frame queued to socket receive queue */
+ len = recv(fd, buf, sizeof(buf), MSG_DONTWAIT);
+ if (len <= 0)
+ break;
+ nlh = buf;
+ } else
+ /* No more messages to process, continue polling */
+ break;
+
+ process_msg(nlh);
+release:
+ /* Release frame back to the kernel */
+ hdr->nm_status = NL_MMAP_STATUS_UNUSED;
+
+ /* Advance frame offset to next frame */
+ frame_offset = (frame_offset + frame_size) % ring_size;
+ }
+ }
+
+Message transmission:
+
+This example assumes some ring parameters of the ring setup are available.
+A single message is constructed and transmitted, to send multiple messages
+at once they would be constructed in consecutive frames before a final call
+to sendto().
+
+ unsigned int frame_offset = 0;
+ struct nl_mmap_hdr *hdr;
+ struct nlmsghdr *nlh;
+ struct sockaddr_nl addr = {
+ .nl_family = AF_NETLINK,
+ };
+
+ hdr = tx_ring + frame_offset;
+ if (hdr->nm_status != NL_MMAP_STATUS_UNUSED)
+ /* No frame available. Use poll() to avoid. */
+ exit(1);
+
+ nlh = (void *)hdr + NL_MMAP_HDRLEN;
+
+ /* Build message */
+ build_message(nlh);
+
+ /* Fill frame header: length and status need to be set */
+ hdr->nm_len = nlh->nlmsg_len;
+ hdr->nm_status = NL_MMAP_STATUS_VALID;
+
+ if (sendto(fd, NULL, 0, 0, &addr, sizeof(addr)) < 0)
+ exit(1);
+
+ /* Advance frame offset to next frame */
+ frame_offset = (frame_offset + frame_size) % ring_size;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 10/11] netlink: add flow control for memory mapped I/O
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Add flow control for memory mapped RX. Since user-space usually doesn't
invoke recvmsg() when using memory mapped I/O, flow control is performed
in netlink_poll(). Dumps are allowed to continue if at least half of the
ring frames are unused.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/netlink/af_netlink.c | 87 +++++++++++++++++++++++++++++++--------------
1 files changed, 60 insertions(+), 27 deletions(-)
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index c42a601..cc90aa8 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -177,6 +177,29 @@ static inline struct hlist_head *nl_pid_hashfn(struct nl_pid_hash *hash, u32 pid
return &hash->table[jhash_1word(pid, hash->rnd) & hash->mask];
}
+static void netlink_overrun(struct sock *sk)
+{
+ struct netlink_sock *nlk = nlk_sk(sk);
+
+ if (!(nlk->flags & NETLINK_RECV_NO_ENOBUFS)) {
+ if (!test_and_set_bit(NETLINK_CONGESTED, &nlk_sk(sk)->state)) {
+ sk->sk_err = ENOBUFS;
+ sk->sk_error_report(sk);
+ }
+ }
+ atomic_inc(&sk->sk_drops);
+}
+
+static void netlink_rcv_wake(struct sock *sk)
+{
+ struct netlink_sock *nlk = nlk_sk(sk);
+
+ if (skb_queue_empty(&sk->sk_receive_queue))
+ clear_bit(NETLINK_CONGESTED, &nlk->state);
+ if (!test_bit(NETLINK_CONGESTED, &nlk->state))
+ wake_up_interruptible(&nlk->wait);
+}
+
#ifdef CONFIG_NETLINK_MMAP
static bool netlink_skb_is_mmaped(const struct sk_buff *skb)
{
@@ -508,15 +531,48 @@ static void netlink_forward_ring(struct netlink_ring *ring)
} while (ring->head != head);
}
+static bool netlink_dump_space(struct netlink_sock *nlk)
+{
+ struct netlink_ring *ring = &nlk->rx_ring;
+ struct nl_mmap_hdr *hdr;
+ unsigned int n;
+
+ hdr = netlink_current_frame(ring, NL_MMAP_STATUS_UNUSED);
+ if (hdr == NULL)
+ return false;
+
+ n = ring->head + ring->frame_max / 2;
+ if (n > ring->frame_max)
+ n -= ring->frame_max;
+
+ hdr = __netlink_lookup_frame(ring, n);
+
+ return hdr->nm_status == NL_MMAP_STATUS_UNUSED;
+}
+
static unsigned int netlink_poll(struct file *file, struct socket *sock,
poll_table *wait)
{
struct sock *sk = sock->sk;
struct netlink_sock *nlk = nlk_sk(sk);
unsigned int mask;
+ int err;
- if (nlk->cb != NULL && nlk->rx_ring.pg_vec != NULL)
- netlink_dump(sk);
+ if (nlk->rx_ring.pg_vec != NULL) {
+ /* Memory mapped sockets don't call recvmsg(), so flow control
+ * for dumps is performed here. A dump is allowed to continue
+ * if at least half the ring is unused.
+ */
+ while (nlk->cb != NULL && netlink_dump_space(nlk)) {
+ err = netlink_dump(sk);
+ if (err < 0) {
+ sk->sk_err = err;
+ sk->sk_error_report(sk);
+ break;
+ }
+ }
+ netlink_rcv_wake(sk);
+ }
mask = datagram_poll(file, sock, wait);
@@ -688,8 +744,7 @@ static void netlink_ring_set_copied(struct sock *sk, struct sk_buff *skb)
if (hdr == NULL) {
spin_unlock_bh(&sk->sk_receive_queue.lock);
kfree_skb(skb);
- sk->sk_err = ENOBUFS;
- sk->sk_error_report(sk);
+ netlink_overrun(sk);
return;
}
netlink_increment_head(ring);
@@ -1382,19 +1437,6 @@ static int netlink_getname(struct socket *sock, struct sockaddr *addr,
return 0;
}
-static void netlink_overrun(struct sock *sk)
-{
- struct netlink_sock *nlk = nlk_sk(sk);
-
- if (!(nlk->flags & NETLINK_RECV_NO_ENOBUFS)) {
- if (!test_and_set_bit(NETLINK_CONGESTED, &nlk_sk(sk)->state)) {
- sk->sk_err = ENOBUFS;
- sk->sk_error_report(sk);
- }
- }
- atomic_inc(&sk->sk_drops);
-}
-
static struct sock *netlink_getsockbypid(struct sock *ssk, u32 pid)
{
struct sock *sock;
@@ -1537,16 +1579,6 @@ static struct sk_buff *netlink_trim(struct sk_buff *skb, gfp_t allocation)
return skb;
}
-static void netlink_rcv_wake(struct sock *sk)
-{
- struct netlink_sock *nlk = nlk_sk(sk);
-
- if (skb_queue_empty(&sk->sk_receive_queue))
- clear_bit(NETLINK_CONGESTED, &nlk->state);
- if (!test_bit(NETLINK_CONGESTED, &nlk->state))
- wake_up_interruptible(&nlk->wait);
-}
-
static int netlink_unicast_kernel(struct sock *sk, struct sk_buff *skb)
{
int ret;
@@ -1648,6 +1680,7 @@ struct sk_buff *netlink_alloc_skb(struct sock *ssk, unsigned int size,
err2:
kfree_skb(skb);
spin_unlock_bh(&sk->sk_receive_queue.lock);
+ netlink_overrun(sk);
err1:
sock_put(sk);
return NULL;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 08/11] netlink: implement memory mapped recvmsg()
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netlink.h | 2 +
net/netlink/af_netlink.c | 145 ++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 143 insertions(+), 4 deletions(-)
diff --git a/include/linux/netlink.h b/include/linux/netlink.h
index 98754e8..144ef3a 100644
--- a/include/linux/netlink.h
+++ b/include/linux/netlink.h
@@ -232,6 +232,8 @@ extern void __netlink_clear_multicast_users(struct sock *sk, unsigned int group)
extern void netlink_clear_multicast_users(struct sock *sk, unsigned int group);
extern void netlink_ack(struct sk_buff *in_skb, struct nlmsghdr *nlh, int err);
extern int netlink_has_listeners(struct sock *sk, unsigned int group);
+extern struct sk_buff *netlink_alloc_skb(struct sock *ssk, unsigned int size,
+ u32 dst_pid, gfp_t gfp_mask);
extern int netlink_unicast(struct sock *ssk, struct sk_buff *skb, __u32 pid, int nonblock);
extern int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, __u32 pid,
__u32 group, gfp_t allocation);
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 65867fd..c42a601 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -183,6 +183,11 @@ static bool netlink_skb_is_mmaped(const struct sk_buff *skb)
return NETLINK_CB(skb).flags & NETLINK_SKB_MMAPED;
}
+static bool netlink_rx_is_mmaped(struct sock *sk)
+{
+ return nlk_sk(sk)->rx_ring.pg_vec != NULL;
+}
+
static bool netlink_tx_is_mmaped(struct sock *sk)
{
return nlk_sk(sk)->tx_ring.pg_vec != NULL;
@@ -654,8 +659,54 @@ out:
mutex_unlock(&nlk->pg_vec_lock);
return err;
}
+
+static void netlink_queue_mmaped_skb(struct sock *sk, struct sk_buff *skb)
+{
+ struct nl_mmap_hdr *hdr;
+
+ hdr = netlink_mmap_hdr(skb);
+ hdr->nm_len = skb->len;
+ hdr->nm_group = NETLINK_CB(skb).dst_group;
+ hdr->nm_pid = NETLINK_CB(skb).creds.pid;
+ hdr->nm_uid = NETLINK_CB(skb).creds.uid;
+ hdr->nm_gid = NETLINK_CB(skb).creds.gid;
+ netlink_frame_flush_dcache(hdr);
+ netlink_set_status(hdr, NL_MMAP_STATUS_VALID);
+
+ NETLINK_CB(skb).flags |= NETLINK_SKB_DELIVERED;
+ kfree_skb(skb);
+}
+
+static void netlink_ring_set_copied(struct sock *sk, struct sk_buff *skb)
+{
+ struct netlink_sock *nlk = nlk_sk(sk);
+ struct netlink_ring *ring = &nlk->rx_ring;
+ struct nl_mmap_hdr *hdr;
+
+ spin_lock_bh(&sk->sk_receive_queue.lock);
+ hdr = netlink_current_frame(ring, NL_MMAP_STATUS_UNUSED);
+ if (hdr == NULL) {
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+ kfree_skb(skb);
+ sk->sk_err = ENOBUFS;
+ sk->sk_error_report(sk);
+ return;
+ }
+ netlink_increment_head(ring);
+ __skb_queue_tail(&sk->sk_receive_queue, skb);
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+
+ hdr->nm_len = skb->len;
+ hdr->nm_group = NETLINK_CB(skb).dst_group;
+ hdr->nm_pid = NETLINK_CB(skb).creds.pid;
+ hdr->nm_uid = NETLINK_CB(skb).creds.uid;
+ hdr->nm_gid = NETLINK_CB(skb).creds.gid;
+ netlink_set_status(hdr, NL_MMAP_STATUS_COPY);
+}
+
#else /* CONFIG_NETLINK_MMAP */
#define netlink_skb_is_mmaped(skb) false
+#define netlink_rx_is_mmaped(sk) false
#define netlink_tx_is_mmaped(sk) false
#define netlink_mmap sock_no_mmap
#define netlink_poll datagram_poll
@@ -1434,7 +1485,14 @@ static int __netlink_sendskb(struct sock *sk, struct sk_buff *skb)
{
int len = skb->len;
- skb_queue_tail(&sk->sk_receive_queue, skb);
+#ifdef CONFIG_NETLINK_MMAP
+ if (netlink_skb_is_mmaped(skb))
+ netlink_queue_mmaped_skb(sk, skb);
+ else if (netlink_rx_is_mmaped(sk))
+ netlink_ring_set_copied(sk, skb);
+ else
+#endif /* CONFIG_NETLINK_MMAP */
+ skb_queue_tail(&sk->sk_receive_queue, skb);
sk->sk_data_ready(sk, len);
return len;
}
@@ -1543,6 +1601,68 @@ retry:
}
EXPORT_SYMBOL(netlink_unicast);
+struct sk_buff *netlink_alloc_skb(struct sock *ssk, unsigned int size,
+ u32 dst_pid, gfp_t gfp_mask)
+{
+#ifdef CONFIG_NETLINK_MMAP
+ struct sock *sk = NULL;
+ struct sk_buff *skb;
+ struct netlink_ring *ring;
+ struct nl_mmap_hdr *hdr;
+ unsigned int maxlen;
+
+ sk = netlink_getsockbypid(ssk, dst_pid);
+ if (IS_ERR(sk))
+ goto out;
+
+ ring = &nlk_sk(sk)->rx_ring;
+ /* fast-path without atomic ops for common case: non-mmaped receiver */
+ if (ring->pg_vec == NULL)
+ goto out_put;
+
+ skb = alloc_skb_head(gfp_mask);
+ if (skb == NULL)
+ goto err1;
+
+ spin_lock_bh(&sk->sk_receive_queue.lock);
+ /* check again under lock */
+ if (ring->pg_vec == NULL)
+ goto out_free;
+
+ maxlen = ring->frame_size - NL_MMAP_HDRLEN;
+ if (maxlen < size)
+ goto out_free;
+
+ netlink_forward_ring(ring);
+ hdr = netlink_current_frame(ring, NL_MMAP_STATUS_UNUSED);
+ if (hdr == NULL)
+ goto err2;
+ netlink_ring_setup_skb(skb, sk, ring, hdr);
+ netlink_set_status(hdr, NL_MMAP_STATUS_RESERVED);
+ atomic_inc(&ring->pending);
+ netlink_increment_head(ring);
+
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+ return skb;
+
+err2:
+ kfree_skb(skb);
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+err1:
+ sock_put(sk);
+ return NULL;
+
+out_free:
+ kfree_skb(skb);
+ spin_unlock_bh(&sk->sk_receive_queue.lock);
+out_put:
+ sock_put(sk);
+out:
+#endif
+ return alloc_skb(size, gfp_mask);
+}
+EXPORT_SYMBOL_GPL(netlink_alloc_skb);
+
int netlink_has_listeners(struct sock *sk, unsigned int group)
{
int res = 0;
@@ -2328,9 +2448,13 @@ static int netlink_dump(struct sock *sk)
alloc_size = max_t(int, cb->min_dump_alloc, NLMSG_GOODSIZE);
- skb = sock_rmalloc(sk, alloc_size, 0, GFP_KERNEL);
+ if (!netlink_rx_is_mmaped(sk) &&
+ atomic_read(&sk->sk_rmem_alloc) >= sk->sk_rcvbuf)
+ goto errout_skb;
+ skb = netlink_alloc_skb(sk, alloc_size, nlk->pid, GFP_KERNEL);
if (!skb)
goto errout_skb;
+ netlink_skb_set_owner_r(skb, sk);
len = cb->dump(skb, cb);
@@ -2384,12 +2508,24 @@ int netlink_dump_start(struct sock *ssk, struct sk_buff *skb,
if (cb == NULL)
return -ENOBUFS;
+ /* Memory mapped dump requests need to be copied to avoid looping
+ * on the pending state in netlink_mmap_sendmsg() while the cb holds
+ * a reference to the skb.
+ */
+ if (netlink_skb_is_mmaped(skb)) {
+ skb = skb_copy(skb, GFP_KERNEL);
+ if (skb == NULL) {
+ kfree(cb);
+ return -ENOBUFS;
+ }
+ } else
+ atomic_inc(&skb->users);
+
cb->dump = control->dump;
cb->done = control->done;
cb->nlh = nlh;
cb->data = control->data;
cb->min_dump_alloc = control->min_dump_alloc;
- atomic_inc(&skb->users);
cb->skb = skb;
sk = netlink_lookup(sock_net(ssk), ssk->sk_protocol, NETLINK_CB(skb).pid);
@@ -2434,7 +2570,8 @@ void netlink_ack(struct sk_buff *in_skb, struct nlmsghdr *nlh, int err)
if (err)
payload += nlmsg_len(nlh);
- skb = nlmsg_new(payload, GFP_KERNEL);
+ skb = netlink_alloc_skb(in_skb->sk, nlmsg_total_size(payload),
+ NETLINK_CB(in_skb).pid, GFP_KERNEL);
if (!skb) {
struct sock *sk;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 07/11] netlink: implement memory mapped sendmsg()
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Add support for mmap'ed sendmsg() to netlink.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/netlink/af_netlink.c | 132 +++++++++++++++++++++++++++++++++++++++++++--
1 files changed, 126 insertions(+), 6 deletions(-)
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 3810911..65867fd 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -183,6 +183,11 @@ static bool netlink_skb_is_mmaped(const struct sk_buff *skb)
return NETLINK_CB(skb).flags & NETLINK_SKB_MMAPED;
}
+static bool netlink_tx_is_mmaped(struct sock *sk)
+{
+ return nlk_sk(sk)->tx_ring.pg_vec != NULL;
+}
+
static __pure struct page *pgvec_to_page(const void *addr)
{
if (is_vmalloc_addr(addr))
@@ -505,6 +510,9 @@ static unsigned int netlink_poll(struct file *file, struct socket *sock,
struct netlink_sock *nlk = nlk_sk(sk);
unsigned int mask;
+ if (nlk->cb != NULL && nlk->rx_ring.pg_vec != NULL)
+ netlink_dump(sk);
+
mask = datagram_poll(file, sock, wait);
spin_lock_bh(&sk->sk_receive_queue.lock);
@@ -550,10 +558,108 @@ static void netlink_ring_setup_skb(struct sk_buff *skb, struct sock *sk,
NETLINK_CB(skb).flags |= NETLINK_SKB_MMAPED;
NETLINK_CB(skb).sk = sk;
}
+
+static int netlink_mmap_sendmsg(struct sock *sk, struct msghdr *msg,
+ u32 dst_pid, u32 dst_group,
+ struct sock_iocb *siocb)
+{
+ struct netlink_sock *nlk = nlk_sk(sk);
+ struct netlink_ring *ring;
+ struct nl_mmap_hdr *hdr;
+ struct sk_buff *skb;
+ unsigned int maxlen;
+ bool excl = true;
+ int err = 0, len = 0;
+
+ /* Netlink messages are validated by the receiver before processing.
+ * In order to avoid userspace changing the contents of the message
+ * after validation, the socket and the ring may only be used by a
+ * single process, otherwise we fall back to copying.
+ */
+ if (atomic_long_read(&sk->sk_socket->file->f_count) > 2 ||
+ atomic_read(&nlk->mapped) > 1)
+ excl = false;
+
+ mutex_lock(&nlk->pg_vec_lock);
+
+ ring = &nlk->tx_ring;
+ maxlen = ring->frame_size - NL_MMAP_HDRLEN;
+
+ do {
+ hdr = netlink_current_frame(ring, NL_MMAP_STATUS_VALID);
+ if (hdr == NULL) {
+ if (!(msg->msg_flags & MSG_DONTWAIT) &&
+ atomic_read(&nlk->tx_ring.pending))
+ schedule();
+ continue;
+ }
+ if (hdr->nm_len > maxlen) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ netlink_frame_flush_dcache(hdr);
+
+ if (likely(dst_pid == 0 && dst_group == 0 && excl)) {
+ skb = alloc_skb_head(GFP_KERNEL);
+ if (skb == NULL) {
+ err = -ENOBUFS;
+ goto out;
+ }
+ sock_hold(sk);
+ netlink_ring_setup_skb(skb, sk, ring, hdr);
+ NETLINK_CB(skb).flags |= NETLINK_SKB_TX;
+ __skb_put(skb, hdr->nm_len);
+ netlink_set_status(hdr, NL_MMAP_STATUS_RESERVED);
+ atomic_inc(&ring->pending);
+ } else {
+ skb = alloc_skb(hdr->nm_len, GFP_KERNEL);
+ if (skb == NULL) {
+ err = -ENOBUFS;
+ goto out;
+ }
+ __skb_put(skb, hdr->nm_len);
+ memcpy(skb->data, (void *)hdr + NL_MMAP_HDRLEN, hdr->nm_len);
+ netlink_set_status(hdr, NL_MMAP_STATUS_UNUSED);
+ }
+
+ netlink_increment_head(ring);
+
+ NETLINK_CB(skb).pid = nlk->pid;
+ NETLINK_CB(skb).dst_group = dst_group;
+ NETLINK_CB(skb).creds = siocb->scm->creds;
+
+ err = security_netlink_send(sk, skb);
+ if (err) {
+ kfree_skb(skb);
+ goto out;
+ }
+
+ if (unlikely(dst_group)) {
+ atomic_inc(&skb->users);
+ netlink_broadcast(sk, skb, dst_pid, dst_group, GFP_KERNEL);
+ }
+ err = netlink_unicast(sk, skb, dst_pid, msg->msg_flags & MSG_DONTWAIT);
+ if (err < 0)
+ goto out;
+ len += err;
+
+ } while (hdr != NULL ||
+ (!(msg->msg_flags & MSG_DONTWAIT) &&
+ atomic_read(&nlk->tx_ring.pending)));
+
+ if (len > 0)
+ err = len;
+out:
+ mutex_unlock(&nlk->pg_vec_lock);
+ return err;
+}
#else /* CONFIG_NETLINK_MMAP */
#define netlink_skb_is_mmaped(skb) false
+#define netlink_tx_is_mmaped(sk) false
#define netlink_mmap sock_no_mmap
#define netlink_poll datagram_poll
+#define netlink_mmap_sendmsg(sk, msg, dst_pid, dst_group, siocb) 0
#endif /* CONFIG_NETLINK_MMAP */
static void netlink_destroy_callback(struct netlink_callback *cb)
@@ -584,11 +690,16 @@ static void netlink_skb_destructor(struct sk_buff *skb)
hdr = netlink_mmap_hdr(skb);
sk = NETLINK_CB(skb).sk;
- if (!(NETLINK_CB(skb).flags & NETLINK_SKB_DELIVERED)) {
- hdr->nm_len = 0;
- netlink_set_status(hdr, NL_MMAP_STATUS_VALID);
+ if (NETLINK_CB(skb).flags & NETLINK_SKB_TX) {
+ netlink_set_status(hdr, NL_MMAP_STATUS_UNUSED);
+ ring = &nlk_sk(sk)->tx_ring;
+ } else {
+ if (!(NETLINK_CB(skb).flags & NETLINK_SKB_DELIVERED)) {
+ hdr->nm_len = 0;
+ netlink_set_status(hdr, NL_MMAP_STATUS_VALID);
+ }
+ ring = &nlk_sk(sk)->rx_ring;
}
- ring = &nlk_sk(sk)->rx_ring;
WARN_ON(atomic_read(&ring->pending) == 0);
atomic_dec(&ring->pending);
@@ -1285,8 +1396,9 @@ int netlink_attachskb(struct sock *sk, struct sk_buff *skb,
nlk = nlk_sk(sk);
- if (atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
- test_bit(NETLINK_CONGESTED, &nlk->state)) {
+ if ((atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
+ test_bit(NETLINK_CONGESTED, &nlk->state)) &&
+ !netlink_skb_is_mmaped(skb)) {
DECLARE_WAITQUEUE(wait, current);
if (!*timeo) {
if (!ssk || netlink_is_kernel(ssk))
@@ -1346,6 +1458,8 @@ static struct sk_buff *netlink_trim(struct sk_buff *skb, gfp_t allocation)
int delta;
WARN_ON(skb->sk != NULL);
+ if (netlink_skb_is_mmaped(skb))
+ return skb;
delta = skb->end - skb->tail;
if (delta * 2 < skb->truesize)
@@ -1869,6 +1983,12 @@ static int netlink_sendmsg(struct kiocb *kiocb, struct socket *sock,
goto out;
}
+ if (netlink_tx_is_mmaped(sk) &&
+ msg->msg_iov->iov_base == NULL) {
+ err = netlink_mmap_sendmsg(sk, msg, dst_pid, dst_group, siocb);
+ goto out;
+ }
+
err = -EMSGSIZE;
if (len > sk->sk_sndbuf - 32)
goto out;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 04/11] netlink: add netlink_skb_set_owner_r()
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
For mmap'ed I/O a netlink specific skb destructor needs to be invoked after the
final kfree_skb() to clean up state. This doesn't work currently since the skb's
ownership is transfered to the receiving socket using skb_set_owner_r(), which
orphans the skb, thereby invoking the destructor prematurely.
Since netlink doesn't account skbs to the originating socket, there's no need
to orphan the skb. Add a netlink specific skb_set_owner_r() variant that does
not orphan the skb and use a netlink specific destructor to call sock_rfree().
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/netlink/af_netlink.c | 18 ++++++++++++++++--
1 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 3233601..e2d0177 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -166,6 +166,20 @@ static void netlink_consume_callback(struct netlink_callback *cb)
kfree(cb);
}
+static void netlink_skb_destructor(struct sk_buff *skb)
+{
+ sock_rfree(skb);
+}
+
+static void netlink_skb_set_owner_r(struct sk_buff *skb, struct sock *sk)
+{
+ WARN_ON(skb->sk != NULL);
+ skb->sk = sk;
+ skb->destructor = netlink_skb_destructor;
+ atomic_add(skb->truesize, &sk->sk_rmem_alloc);
+ sk_mem_charge(sk, skb->truesize);
+}
+
static void netlink_sock_destruct(struct sock *sk)
{
struct netlink_sock *nlk = nlk_sk(sk);
@@ -855,7 +869,7 @@ int netlink_attachskb(struct sock *sk, struct sk_buff *skb,
}
return 1;
}
- skb_set_owner_r(skb, sk);
+ netlink_skb_set_owner_r(skb, sk);
return 0;
}
@@ -924,7 +938,7 @@ static int netlink_unicast_kernel(struct sock *sk, struct sk_buff *skb)
ret = -ECONNREFUSED;
if (nlk->netlink_rcv != NULL) {
ret = skb->len;
- skb_set_owner_r(skb, sk);
+ netlink_skb_set_owner_r(skb, sk);
nlk->netlink_rcv(skb);
consume_skb(skb);
} else {
--
1.7.7.6
^ permalink raw reply related
* [PATCH 03/11] netlink: don't orphan skb in netlink_trim()
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Netlink doesn't account skbs to the sending socket, so the there's no
need to orphan the skb before trimming it.
Removing the skb_orphan() call is required for mmap'ed netlink, which uses
a netlink specific skb destructor that must not be invoked before the
final freeing of the skb.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/netlink/af_netlink.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 1bdfa52..3233601 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -886,7 +886,7 @@ static struct sk_buff *netlink_trim(struct sk_buff *skb, gfp_t allocation)
{
int delta;
- skb_orphan(skb);
+ WARN_ON(skb->sk != NULL);
delta = skb->end - skb->tail;
if (delta * 2 < skb->truesize)
--
1.7.7.6
^ permalink raw reply related
* [PATCH 02/11] net: add function to allocate skbuff head without data area
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Add a function to allocate a skbuff head without any data. This will be
used by memory mapped netlink to attached data from the mmaped area to
the skb.
Additionally change skb_release_all() to check whether the skb has a
data area to allow the skb destructor to clear the data pointer in
case only a head has been allocated.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/skbuff.h | 6 ++++++
net/core/skbuff.c | 31 ++++++++++++++++++++++++++++++-
2 files changed, 36 insertions(+), 1 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index b33a3a1..93fe80f 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -589,6 +589,12 @@ static inline struct sk_buff *alloc_skb_fclone(unsigned int size,
return __alloc_skb(size, priority, SKB_ALLOC_FCLONE, NUMA_NO_NODE);
}
+extern struct sk_buff *__alloc_skb_head(gfp_t priority, int node);
+static inline struct sk_buff *alloc_skb_head(gfp_t priority)
+{
+ return __alloc_skb_head(priority, -1);
+}
+
extern void skb_recycle(struct sk_buff *skb);
extern bool skb_recycle_check(struct sk_buff *skb, int skb_size);
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index fe00d12..4046f6e 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -188,6 +188,34 @@ out:
*
*/
+struct sk_buff *__alloc_skb_head(gfp_t gfp_mask, int node)
+{
+ struct sk_buff *skb;
+
+ /* Get the HEAD */
+ skb = kmem_cache_alloc_node(skbuff_head_cache,
+ gfp_mask & ~__GFP_DMA, node);
+ if (!skb)
+ goto out;
+ prefetchw(skb);
+
+ /*
+ * Only clear those fields we need to clear, not those that we will
+ * actually initialise below. Hence, don't put any more fields after
+ * the tail pointer in struct sk_buff!
+ */
+ memset(skb, 0, offsetof(struct sk_buff, tail));
+ skb->data = NULL;
+ skb->truesize = sizeof(struct sk_buff);
+ atomic_set(&skb->users, 1);
+
+#ifdef NET_SKBUFF_DATA_USES_OFFSET
+ skb->mac_header = ~0U;
+#endif
+out:
+ return skb;
+}
+
/**
* __alloc_skb - allocate a network buffer
* @size: size to allocate
@@ -581,7 +609,8 @@ static void skb_release_head_state(struct sk_buff *skb)
static void skb_release_all(struct sk_buff *skb)
{
skb_release_head_state(skb);
- skb_release_data(skb);
+ if (likely(skb->data))
+ skb_release_data(skb);
}
/**
--
1.7.7.6
^ permalink raw reply related
* [PATCH 01/11] netlink: add symbolic value for congested state
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
In-Reply-To: <1345443532-3707-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/netlink/af_netlink.c | 18 +++++++++++-------
1 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 5463969..1bdfa52 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -89,6 +89,10 @@ struct listeners {
unsigned long masks[0];
};
+/* state bits */
+#define NETLINK_CONGESTED 0x0
+
+/* flags */
#define NETLINK_KERNEL_SOCKET 0x1
#define NETLINK_RECV_PKTINFO 0x2
#define NETLINK_BROADCAST_SEND_ERROR 0x4
@@ -762,7 +766,7 @@ static void netlink_overrun(struct sock *sk)
struct netlink_sock *nlk = nlk_sk(sk);
if (!(nlk->flags & NETLINK_RECV_NO_ENOBUFS)) {
- if (!test_and_set_bit(0, &nlk_sk(sk)->state)) {
+ if (!test_and_set_bit(NETLINK_CONGESTED, &nlk_sk(sk)->state)) {
sk->sk_err = ENOBUFS;
sk->sk_error_report(sk);
}
@@ -823,7 +827,7 @@ int netlink_attachskb(struct sock *sk, struct sk_buff *skb,
nlk = nlk_sk(sk);
if (atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
- test_bit(0, &nlk->state)) {
+ test_bit(NETLINK_CONGESTED, &nlk->state)) {
DECLARE_WAITQUEUE(wait, current);
if (!*timeo) {
if (!ssk || netlink_is_kernel(ssk))
@@ -837,7 +841,7 @@ int netlink_attachskb(struct sock *sk, struct sk_buff *skb,
add_wait_queue(&nlk->wait, &wait);
if ((atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
- test_bit(0, &nlk->state)) &&
+ test_bit(NETLINK_CONGESTED, &nlk->state)) &&
!sock_flag(sk, SOCK_DEAD))
*timeo = schedule_timeout(*timeo);
@@ -907,8 +911,8 @@ static void netlink_rcv_wake(struct sock *sk)
struct netlink_sock *nlk = nlk_sk(sk);
if (skb_queue_empty(&sk->sk_receive_queue))
- clear_bit(0, &nlk->state);
- if (!test_bit(0, &nlk->state))
+ clear_bit(NETLINK_CONGESTED, &nlk->state);
+ if (!test_bit(NETLINK_CONGESTED, &nlk->state))
wake_up_interruptible(&nlk->wait);
}
@@ -990,7 +994,7 @@ static int netlink_broadcast_deliver(struct sock *sk, struct sk_buff *skb)
struct netlink_sock *nlk = nlk_sk(sk);
if (atomic_read(&sk->sk_rmem_alloc) <= sk->sk_rcvbuf &&
- !test_bit(0, &nlk->state)) {
+ !test_bit(NETLINK_CONGESTED, &nlk->state)) {
skb_set_owner_r(skb, sk);
__netlink_sendskb(sk, skb);
return atomic_read(&sk->sk_rmem_alloc) > (sk->sk_rcvbuf >> 1);
@@ -1270,7 +1274,7 @@ static int netlink_setsockopt(struct socket *sock, int level, int optname,
case NETLINK_NO_ENOBUFS:
if (val) {
nlk->flags |= NETLINK_RECV_NO_ENOBUFS;
- clear_bit(0, &nlk->state);
+ clear_bit(NETLINK_CONGESTED, &nlk->state);
wake_up_interruptible(&nlk->wait);
} else {
nlk->flags &= ~NETLINK_RECV_NO_ENOBUFS;
--
1.7.7.6
^ permalink raw reply related
* [PATCH 00/11] netlink: memory mapped I/O
From: Patrick McHardy @ 2012-08-20 6:18 UTC (permalink / raw)
To: Florian.Westphal; +Cc: netdev, netfilter-devel
The following patches contain an implementation of memory mapped I/O for
netlink, rebased onto the current net-next tree. The implementation is
modelled after AF_PACKET memory mapped I/O with a few differences:
- In order to perform memory mapped I/O to userspace, the kernel allocates
skbs with the data area pointing to the data area of the mapped frames.
All netlink subsystems assume a linear data area, so for the sake of
simplicity, the mapped data area is not attached to the paged area but
to skb->data. This requires introduction of a special skb alloction
function that just allocates an skb head without the data area. Since this
is a quite rare use case, I introduced a new function based on __alloc_skb
instead of splitting it up into head and data alloction. The alternative
would be to introduce an __alloc_skb_head and __alloc_skb_data function,
which would actually be useful for a specific error case in memory mapped
netlink, but would require a couple of extra instructions for the common
skb allocation case, so it doesn't really seem worth it.
In order to get the destination memory area for skb->data before message
construction, memory mapped netlink I/O needs to look up the destination
socket during allocation instead of during transmission because the
ring is owned by the receiveing socket/process. A special skb allocation
function (netlink_alloc_skb) taking the destination pid as an argument is
used for this, all subsystems that want to support memory mapped I/O need
to use this function, automatic fallback to the receive queue happens
for unconverted subsystems. Dumps automatically use memory mapped I/O if
the receiving socket has enabled it.
The visible effect of looking up the destination socket during allocation
instead of transmission is that message ordering in userspace might
change in case allocation and transmission aren't performed atomically.
This usually doesn't matter since most subsystems have a BKL-like lock
like the rtnl mutex, to my knowledge the currently only existing case
where it might matter is nfnetlink_queue combined with the recently
introduced batched verdicts, but a) that subsystem already includes
sequence numbers which allow userspace to reorder messages in case it
cares to, also the reodering window is quite small and b) with memory
mapped transmission batching can be performed in a subsystem indepandant
manner.
- AF_NETLINK contains flow control for database dumps, with regular I/O
dump continuation are triggered based on the sockets receive queue space
and by recvmsg() calls. Since with memory mapped I/O there are no
recvmsg() calls under normal operation, this is done in netlink_poll(),
under the assumption that userspace has processed all pending frames
before invoking poll(), thus the ring is expected to have room for new
messages. Dumps currently don't benefit as much as they could from
memory mapped I/O because each single continuation requires a poll()
call. A more agressive approach seems like a good idea to me, especially
in case the socket is not subscribed to any multicast groups (IOW only
receiving explicitly requested data).
Besides that, the memory mapped netlink implementation extends the states
defined by AF_PACKET between userspace and the kernel by a SKIP status, this
is intended for the case that userspace wants to queue frames (specifically
when using nfnetlink_queue, an IDS and stream reassembly, requested by
Eric Leblond) for a longer period of time. The kernel skips over all frames
marked with SKIP when looking or unused frames and only fails when not finding
a free frame or when having skipped the entire ring.
Also noteworthy is memory mapped sendmsg: the kernel performs validation
of messages before accepting and processing them, in order to prevent
userspace from changing the messages contents after validation, the
kernel checks that the ring is only mapped once and the file descriptor
is not shared (in order to avoid having userspace set up another mapping
after the first mentioned check). If either of both is not true, the
message copied to an allocated skb and processed as with regular I/O.
As an example, nfnetlink_queue is convererted to support memory mapped
I/O. Other subsystems that would probably benefit are nfnetlink_log,
audit and maybe ISCSI.
Since the last posting only a minor bug in ring teardown has been fixed.
I'm still working with Florian on solving the nfnetlink_queue ordering
issue, besides that there are no known issues.
Some older performance numbers with nfnetlink_queue from Florian:
nfq recv: regular netlink I/O
mnl recv: mmap'ed netlink I/O
batch: number of batched verdicts
1400 byte UDP packets, 8 cores, NFQUEUE balancing using 4 queues, only
mmap'ed RX, regular TX, userspace running inline Snort with all rules
and preprocessors enabled:
nfq recv, batch 0 1250 MBit total rx
mnl recv, batch 0 1230 MBit total rx
nfq recv, batch 10 1590 MBit total rx
mnl recv, batch 10 1770 MBit total rx
I'll try to get some new numbers soon, including TX and dumps.
The patches are available in a git tree at:
git://github.com/kaber/netlink-mmap master
once git push has finished.
Patrick McHardy (11):
netlink: add symbolic value for congested state
net: add function to allocate skbuff head without data area
netlink: don't orphan skb in netlink_trim()
netlink: add netlink_skb_set_owner_r()
netlink: mmaped netlink: ring setup
netlink: add mmap'ed netlink helper functions
netlink: implement memory mapped sendmsg()
netlink: implement memory mapped recvmsg()
nfnetlink: add support for memory mapped netlink
netlink: add flow control for memory mapped I/O
netlink: add documentation for memory mapped I/O
Documentation/networking/netlink_mmap.txt | 337 ++++++++++++
include/linux/netfilter/nfnetlink.h | 2 +
include/linux/netlink.h | 42 ++
include/linux/skbuff.h | 6 +
net/Kconfig | 9 +
net/core/skbuff.c | 31 +-
net/netfilter/nfnetlink.c | 7 +
net/netfilter/nfnetlink_log.c | 9 +-
net/netfilter/nfnetlink_queue_core.c | 2 +-
net/netlink/af_netlink.c | 849 +++++++++++++++++++++++++++--
10 files changed, 1250 insertions(+), 44 deletions(-)
^ permalink raw reply
* Re: [PATCH net-next] xfrm: fix RCU bugs
From: Eric Dumazet @ 2012-08-20 5:33 UTC (permalink / raw)
To: Fan Du; +Cc: David Miller, netdev, fengguang.wu, Priyanka Jain
In-Reply-To: <5031BFB1.200@windriver.com>
On Mon, 2012-08-20 at 12:40 +0800, Fan Du wrote:
> Hi Eric
>
> Please correct me if I'm wrong about below comments.
>
> On 2012年08月19日 18:31, Eric Dumazet wrote:
> > From: Eric Dumazet<edumazet@google.com>
> >
> > This patch reverts commit 56892261ed1a (xfrm: Use rcu_dereference_bh to
> > deference pointer protected by rcu_read_lock_bh), and fixes bugs
> > introduced in commit 418a99ac6ad ( Replace rwlock on xfrm_policy_afinfo
> > with rcu )
> >
> > 1) We properly use RCU variant in this file, not a mix of RCU/RCU_BH
> >
> > 2) We must defer some writes after the synchronize_rcu() call or a reader
> > can crash dereferencing NULL pointer.
>
> Not exactly.
>
> net/ipv4/xfrm4_policy.c
> static void __exit xfrm4_policy_fini(void)
> -> xfrm_policy_unregister_afinfo
>
> IMHO, ip stack can never be compiled as module, so is xfrm4_policy_fini
> freed up after system bootup? which means xfrm4_policy_fini can never be
> called.
>
> so an dereferencing NULL pointer by a reader could not happen.
>
Last famous words.
Anyway xfrm_policy_unregister_afinfo() is also called from
xfrm6_policy_fini(), and IPv6 is a module. The day we can rmmod it,
we uncover this bug.
RCU is complex (most people dont get it right, thats the truth),
and we should make it rock solid, or I can guarantee you
many patch attempts from future readers of this code.
You wont tell them :
"OK but dont worry we never call this function for real, why do you care
at all"
> >
> > 3) Now we use the xfrm_policy_afinfo_lock spinlock only from process
> > context, we no longer need to block BH in xfrm_policy_register_afinfo()
> > and xfrm_policy_unregister_afinfo()
> >
> I don't think it's related to what kinds of locks we are using.
> we call xfrm_policy_register_afinfo in process context, but actually
> what xfrm_policy_afinfo_lock protected can be used in soft irq context.
> that's why xx_bh is used in:
You did an RCU conversion and obviously have little idea of what
happened there.
This _bh stuff was needed because _before_ RCU, an rwlock was used.
And since read_lock() was used from BH handler, _all_ write_lock() had
to use the write_lock_bh() variant to avoid a possible deadlock.
But after RCU, this no longer is needed, as an rcu_read_lock() cannot
block a writer anymore in the lock/unlock section.
In fact, xfrm_policy_afinfo_lock could be replaced by a mutex. So _bh()
is absolutely not needed anymore.
^ permalink raw reply
* Re: [PATCH net-next] xfrm: fix RCU bugs
From: Fan Du @ 2012-08-20 4:40 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, fengguang.wu, Priyanka Jain
In-Reply-To: <1345372308.5158.54.camel@edumazet-glaptop>
Hi Eric
Please correct me if I'm wrong about below comments.
On 2012年08月19日 18:31, Eric Dumazet wrote:
> From: Eric Dumazet<edumazet@google.com>
>
> This patch reverts commit 56892261ed1a (xfrm: Use rcu_dereference_bh to
> deference pointer protected by rcu_read_lock_bh), and fixes bugs
> introduced in commit 418a99ac6ad ( Replace rwlock on xfrm_policy_afinfo
> with rcu )
>
> 1) We properly use RCU variant in this file, not a mix of RCU/RCU_BH
>
> 2) We must defer some writes after the synchronize_rcu() call or a reader
> can crash dereferencing NULL pointer.
Not exactly.
net/ipv4/xfrm4_policy.c
static void __exit xfrm4_policy_fini(void)
-> xfrm_policy_unregister_afinfo
IMHO, ip stack can never be compiled as module, so is xfrm4_policy_fini
freed up after system bootup? which means xfrm4_policy_fini can never be
called.
so an dereferencing NULL pointer by a reader could not happen.
>
> 3) Now we use the xfrm_policy_afinfo_lock spinlock only from process
> context, we no longer need to block BH in xfrm_policy_register_afinfo()
> and xfrm_policy_unregister_afinfo()
>
I don't think it's related to what kinds of locks we are using.
we call xfrm_policy_register_afinfo in process context, but actually
what xfrm_policy_afinfo_lock protected can be used in soft irq context.
that's why xx_bh is used in:
e959d812 " [XFRM]: fix incorrect xfrm_policy_afinfo_lock use"
Is such scenario still valid?
> 4) Can use RCU_INIT_POINTER() instead of rcu_assign_pointer() in
> xfrm_policy_unregister_afinfo()
>
> 5) Remove a forward inline declaration (xfrm_policy_put_afinfo()),
> and also move xfrm_policy_get_afinfo() declaration.
>
> Signed-off-by: Eric Dumazet<edumazet@google.com>
> Cc: Fan Du<fan.du@windriver.com>
> Cc: Priyanka Jain<Priyanka.Jain@freescale.com>
> ---
> net/xfrm/xfrm_policy.c | 76 ++++++++++++++++++++-------------------
> 1 file changed, 39 insertions(+), 37 deletions(-)
>
> diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> index 6405764..e52f50f 100644
> --- a/net/xfrm/xfrm_policy.c
> +++ b/net/xfrm/xfrm_policy.c
> @@ -48,8 +48,6 @@ static struct xfrm_policy_afinfo __rcu *xfrm_policy_afinfo[NPROTO]
>
> static struct kmem_cache *xfrm_dst_cache __read_mostly;
>
> -static struct xfrm_policy_afinfo *xfrm_policy_get_afinfo(unsigned short family);
> -static inline void xfrm_policy_put_afinfo(struct xfrm_policy_afinfo *afinfo);
> static void xfrm_init_pmtu(struct dst_entry *dst);
> static int stale_bundle(struct dst_entry *dst);
> static int xfrm_bundle_ok(struct xfrm_dst *xdst);
> @@ -96,6 +94,24 @@ bool xfrm_selector_match(const struct xfrm_selector *sel, const struct flowi *fl
> return false;
> }
>
> +static struct xfrm_policy_afinfo *xfrm_policy_get_afinfo(unsigned short family)
> +{
> + struct xfrm_policy_afinfo *afinfo;
> +
> + if (unlikely(family>= NPROTO))
> + return NULL;
> + rcu_read_lock();
> + afinfo = rcu_dereference(xfrm_policy_afinfo[family]);
> + if (unlikely(!afinfo))
> + rcu_read_unlock();
> + return afinfo;
> +}
> +
> +static void xfrm_policy_put_afinfo(struct xfrm_policy_afinfo *afinfo)
> +{
> + rcu_read_unlock();
> +}
> +
> static inline struct dst_entry *__xfrm_dst_lookup(struct net *net, int tos,
> const xfrm_address_t *saddr,
> const xfrm_address_t *daddr,
> @@ -2419,7 +2435,7 @@ int xfrm_policy_register_afinfo(struct xfrm_policy_afinfo *afinfo)
> return -EINVAL;
> if (unlikely(afinfo->family>= NPROTO))
> return -EAFNOSUPPORT;
> - spin_lock_bh(&xfrm_policy_afinfo_lock);
> + spin_lock(&xfrm_policy_afinfo_lock);
> if (unlikely(xfrm_policy_afinfo[afinfo->family] != NULL))
> err = -ENOBUFS;
> else {
> @@ -2442,7 +2458,7 @@ int xfrm_policy_register_afinfo(struct xfrm_policy_afinfo *afinfo)
> afinfo->garbage_collect = xfrm_garbage_collect_deferred;
> rcu_assign_pointer(xfrm_policy_afinfo[afinfo->family], afinfo);
> }
> - spin_unlock_bh(&xfrm_policy_afinfo_lock);
> + spin_unlock(&xfrm_policy_afinfo_lock);
>
> rtnl_lock();
> for_each_net(net) {
> @@ -2475,23 +2491,26 @@ int xfrm_policy_unregister_afinfo(struct xfrm_policy_afinfo *afinfo)
> return -EINVAL;
> if (unlikely(afinfo->family>= NPROTO))
> return -EAFNOSUPPORT;
> - spin_lock_bh(&xfrm_policy_afinfo_lock);
> + spin_lock(&xfrm_policy_afinfo_lock);
> if (likely(xfrm_policy_afinfo[afinfo->family] != NULL)) {
> if (unlikely(xfrm_policy_afinfo[afinfo->family] != afinfo))
> err = -EINVAL;
> - else {
> - struct dst_ops *dst_ops = afinfo->dst_ops;
> - rcu_assign_pointer(xfrm_policy_afinfo[afinfo->family],
> - NULL);
> - dst_ops->kmem_cachep = NULL;
> - dst_ops->check = NULL;
> - dst_ops->negative_advice = NULL;
> - dst_ops->link_failure = NULL;
> - afinfo->garbage_collect = NULL;
> - }
> + else
> + RCU_INIT_POINTER(xfrm_policy_afinfo[afinfo->family],
> + NULL);
> + }
> + spin_unlock(&xfrm_policy_afinfo_lock);
> + if (!err) {
> + struct dst_ops *dst_ops = afinfo->dst_ops;
> +
> + synchronize_rcu();
> +
> + dst_ops->kmem_cachep = NULL;
> + dst_ops->check = NULL;
> + dst_ops->negative_advice = NULL;
> + dst_ops->link_failure = NULL;
> + afinfo->garbage_collect = NULL;
> }
> - spin_unlock_bh(&xfrm_policy_afinfo_lock);
> - synchronize_rcu();
> return err;
> }
> EXPORT_SYMBOL(xfrm_policy_unregister_afinfo);
> @@ -2500,32 +2519,15 @@ static void __net_init xfrm_dst_ops_init(struct net *net)
> {
> struct xfrm_policy_afinfo *afinfo;
>
> - rcu_read_lock_bh();
> - afinfo = rcu_dereference_bh(xfrm_policy_afinfo[AF_INET]);
> + rcu_read_lock();
> + afinfo = rcu_dereference(xfrm_policy_afinfo[AF_INET]);
> if (afinfo)
> net->xfrm.xfrm4_dst_ops = *afinfo->dst_ops;
> #if IS_ENABLED(CONFIG_IPV6)
> - afinfo = rcu_dereference_bh(xfrm_policy_afinfo[AF_INET6]);
> + afinfo = rcu_dereference(xfrm_policy_afinfo[AF_INET6]);
> if (afinfo)
> net->xfrm.xfrm6_dst_ops = *afinfo->dst_ops;
> #endif
> - rcu_read_unlock_bh();
> -}
> -
> -static struct xfrm_policy_afinfo *xfrm_policy_get_afinfo(unsigned short family)
> -{
> - struct xfrm_policy_afinfo *afinfo;
> - if (unlikely(family>= NPROTO))
> - return NULL;
> - rcu_read_lock();
> - afinfo = rcu_dereference(xfrm_policy_afinfo[family]);
> - if (unlikely(!afinfo))
> - rcu_read_unlock();
> - return afinfo;
> -}
> -
> -static inline void xfrm_policy_put_afinfo(struct xfrm_policy_afinfo *afinfo)
> -{
> rcu_read_unlock();
> }
>
>
>
>
--
Love each day!
--fan
^ permalink raw reply
* [PATCH 17/18] netfilter: nf_nat: support IPv6 in TFTP NAT helper
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
From: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/ipv4/netfilter/Kconfig | 5 -----
net/ipv4/netfilter/Makefile | 1 -
net/netfilter/Kconfig | 5 +++++
net/netfilter/Makefile | 1 +
net/netfilter/nf_conntrack_tftp.c | 3 +--
net/{ipv4 => }/netfilter/nf_nat_tftp.c | 0
6 files changed, 7 insertions(+), 8 deletions(-)
rename net/{ipv4 => }/netfilter/nf_nat_tftp.c (100%)
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 4770103..4f1ff53 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -220,11 +220,6 @@ config NF_NAT_PROTO_GRE
tristate
depends on NF_NAT_IPV4 && NF_CT_PROTO_GRE
-config NF_NAT_TFTP
- tristate
- depends on NF_CONNTRACK && NF_NAT_IPV4
- default NF_NAT_IPV4 && NF_CONNTRACK_TFTP
-
config NF_NAT_PPTP
tristate
depends on NF_CONNTRACK && NF_NAT_IPV4
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index 17e649b..b7dd189 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -23,7 +23,6 @@ obj-$(CONFIG_NF_DEFRAG_IPV4) += nf_defrag_ipv4.o
obj-$(CONFIG_NF_NAT_H323) += nf_nat_h323.o
obj-$(CONFIG_NF_NAT_PPTP) += nf_nat_pptp.o
obj-$(CONFIG_NF_NAT_SNMP_BASIC) += nf_nat_snmp_basic.o
-obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o
# NAT protocols (nf_nat)
obj-$(CONFIG_NF_NAT_PROTO_GRE) += nf_nat_proto_gre.o
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index cabe4da..052836e 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -400,6 +400,11 @@ config NF_NAT_SIP
depends on NF_CONNTRACK && NF_NAT
default NF_NAT && NF_CONNTRACK_SIP
+config NF_NAT_TFTP
+ tristate
+ depends on NF_CONNTRACK && NF_NAT
+ default NF_NAT && NF_CONNTRACK_TFTP
+
endif # NF_CONNTRACK
# transparent proxy support
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 0dd7929..403ea81 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -59,6 +59,7 @@ obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_SIP) += nf_nat_sip.o
+obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o
# transparent proxy support
obj-$(CONFIG_NETFILTER_TPROXY) += nf_tproxy_core.o
diff --git a/net/netfilter/nf_conntrack_tftp.c b/net/netfilter/nf_conntrack_tftp.c
index 9363e1c..81fc61c 100644
--- a/net/netfilter/nf_conntrack_tftp.c
+++ b/net/netfilter/nf_conntrack_tftp.c
@@ -72,8 +72,7 @@ static int tftp_help(struct sk_buff *skb,
nf_ct_dump_tuple(&exp->tuple);
nf_nat_tftp = rcu_dereference(nf_nat_tftp_hook);
- if (nf_nat_tftp && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_tftp && ct->status & IPS_NAT_MASK)
ret = nf_nat_tftp(skb, ctinfo, exp);
else if (nf_ct_expect_related(exp) != 0)
ret = NF_DROP;
diff --git a/net/ipv4/netfilter/nf_nat_tftp.c b/net/netfilter/nf_nat_tftp.c
similarity index 100%
rename from net/ipv4/netfilter/nf_nat_tftp.c
rename to net/netfilter/nf_nat_tftp.c
--
1.7.1
^ permalink raw reply related
* [PATCH 16/18] netfilter: nf_nat: support IPv6 in IRC NAT helper
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
From: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/ipv4/netfilter/Kconfig | 5 -----
net/ipv4/netfilter/Makefile | 1 -
net/netfilter/Kconfig | 5 +++++
net/netfilter/Makefile | 1 +
net/netfilter/nf_conntrack_irc.c | 3 +--
net/{ipv4 => }/netfilter/nf_nat_irc.c | 6 ------
6 files changed, 7 insertions(+), 14 deletions(-)
rename net/{ipv4 => }/netfilter/nf_nat_irc.c (92%)
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 27f9574..4770103 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -220,11 +220,6 @@ config NF_NAT_PROTO_GRE
tristate
depends on NF_NAT_IPV4 && NF_CT_PROTO_GRE
-config NF_NAT_IRC
- tristate
- depends on NF_CONNTRACK && NF_NAT_IPV4
- default NF_NAT_IPV4 && NF_CONNTRACK_IRC
-
config NF_NAT_TFTP
tristate
depends on NF_CONNTRACK && NF_NAT_IPV4
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index 8914abf..17e649b 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -21,7 +21,6 @@ obj-$(CONFIG_NF_DEFRAG_IPV4) += nf_defrag_ipv4.o
# NAT helpers (nf_conntrack)
obj-$(CONFIG_NF_NAT_H323) += nf_nat_h323.o
-obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_PPTP) += nf_nat_pptp.o
obj-$(CONFIG_NF_NAT_SNMP_BASIC) += nf_nat_snmp_basic.o
obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index bf3e464..cabe4da 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -390,6 +390,11 @@ config NF_NAT_FTP
depends on NF_CONNTRACK && NF_NAT
default NF_NAT && NF_CONNTRACK_FTP
+config NF_NAT_IRC
+ tristate
+ depends on NF_CONNTRACK && NF_NAT
+ default NF_NAT && NF_CONNTRACK_IRC
+
config NF_NAT_SIP
tristate
depends on NF_CONNTRACK && NF_NAT
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 7d6d1a0..0dd7929 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -57,6 +57,7 @@ obj-$(CONFIG_NF_NAT_PROTO_SCTP) += nf_nat_proto_sctp.o
# NAT helpers
obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
+obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_SIP) += nf_nat_sip.o
# transparent proxy support
diff --git a/net/netfilter/nf_conntrack_irc.c b/net/netfilter/nf_conntrack_irc.c
index 95d097c..3b20aa7 100644
--- a/net/netfilter/nf_conntrack_irc.c
+++ b/net/netfilter/nf_conntrack_irc.c
@@ -205,8 +205,7 @@ static int help(struct sk_buff *skb, unsigned int protoff,
IPPROTO_TCP, NULL, &port);
nf_nat_irc = rcu_dereference(nf_nat_irc_hook);
- if (nf_nat_irc && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_irc && ct->status & IPS_NAT_MASK)
ret = nf_nat_irc(skb, ctinfo, protoff,
addr_beg_p - ib_ptr,
addr_end_p - addr_beg_p,
diff --git a/net/ipv4/netfilter/nf_nat_irc.c b/net/netfilter/nf_nat_irc.c
similarity index 92%
rename from net/ipv4/netfilter/nf_nat_irc.c
rename to net/netfilter/nf_nat_irc.c
index 1ce37f8..1fedee6 100644
--- a/net/ipv4/netfilter/nf_nat_irc.c
+++ b/net/netfilter/nf_nat_irc.c
@@ -34,7 +34,6 @@ static unsigned int help(struct sk_buff *skb,
struct nf_conntrack_expect *exp)
{
char buffer[sizeof("4294967296 65635")];
- u_int32_t ip;
u_int16_t port;
unsigned int ret;
@@ -60,11 +59,6 @@ static unsigned int help(struct sk_buff *skb,
if (port == 0)
return NF_DROP;
- ip = ntohl(exp->master->tuplehash[IP_CT_DIR_REPLY].tuple.dst.u3.ip);
- sprintf(buffer, "%u %u", ip, port);
- pr_debug("nf_nat_irc: inserting '%s' == %pI4, port %u\n",
- buffer, &ip, port);
-
ret = nf_nat_mangle_tcp_packet(skb, exp->master, ctinfo,
protoff, matchoff, matchlen, buffer,
strlen(buffer));
--
1.7.1
^ permalink raw reply related
* [PATCH 15/18] netfilter: nf_nat: support IPv6 in SIP NAT helper
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Add IPv6 support to the SIP NAT helper. There are no functional differences
to IPv4 NAT, just different formats for addresses.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netfilter/nf_conntrack_sip.h | 9 +-
net/ipv4/netfilter/Kconfig | 5 -
net/ipv4/netfilter/Makefile | 1 -
net/netfilter/Kconfig | 5 +
net/netfilter/Makefile | 1 +
net/netfilter/nf_conntrack_sip.c | 68 ++++++------
net/{ipv4 => }/netfilter/nf_nat_sip.c | 171 ++++++++++++++++------------
7 files changed, 144 insertions(+), 116 deletions(-)
rename net/{ipv4 => }/netfilter/nf_nat_sip.c (76%)
diff --git a/include/linux/netfilter/nf_conntrack_sip.h b/include/linux/netfilter/nf_conntrack_sip.h
index 1afc669..387bdd0 100644
--- a/include/linux/netfilter/nf_conntrack_sip.h
+++ b/include/linux/netfilter/nf_conntrack_sip.h
@@ -99,10 +99,8 @@ enum sip_header_types {
enum sdp_header_types {
SDP_HDR_UNSPEC,
SDP_HDR_VERSION,
- SDP_HDR_OWNER_IP4,
- SDP_HDR_CONNECTION_IP4,
- SDP_HDR_OWNER_IP6,
- SDP_HDR_CONNECTION_IP6,
+ SDP_HDR_OWNER,
+ SDP_HDR_CONNECTION,
SDP_HDR_MEDIA,
};
@@ -111,7 +109,8 @@ extern unsigned int (*nf_nat_sip_hook)(struct sk_buff *skb,
unsigned int dataoff,
const char **dptr,
unsigned int *datalen);
-extern void (*nf_nat_sip_seq_adjust_hook)(struct sk_buff *skb, s16 off);
+extern void (*nf_nat_sip_seq_adjust_hook)(struct sk_buff *skb,
+ unsigned int protoff, s16 off);
extern unsigned int (*nf_nat_sip_expect_hook)(struct sk_buff *skb,
unsigned int protoff,
unsigned int dataoff,
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index df42031..27f9574 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -241,11 +241,6 @@ config NF_NAT_H323
depends on NF_CONNTRACK && NF_NAT_IPV4
default NF_NAT_IPV4 && NF_CONNTRACK_H323
-config NF_NAT_SIP
- tristate
- depends on NF_CONNTRACK && NF_NAT_IPV4
- default NF_NAT_IPV4 && NF_CONNTRACK_SIP
-
# mangle + specific targets
config IP_NF_MANGLE
tristate "Packet mangling"
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index 8baa496..8914abf 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -23,7 +23,6 @@ obj-$(CONFIG_NF_DEFRAG_IPV4) += nf_defrag_ipv4.o
obj-$(CONFIG_NF_NAT_H323) += nf_nat_h323.o
obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_PPTP) += nf_nat_pptp.o
-obj-$(CONFIG_NF_NAT_SIP) += nf_nat_sip.o
obj-$(CONFIG_NF_NAT_SNMP_BASIC) += nf_nat_snmp_basic.o
obj-$(CONFIG_NF_NAT_TFTP) += nf_nat_tftp.o
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 2eee9f1..bf3e464 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -390,6 +390,11 @@ config NF_NAT_FTP
depends on NF_CONNTRACK && NF_NAT
default NF_NAT && NF_CONNTRACK_FTP
+config NF_NAT_SIP
+ tristate
+ depends on NF_CONNTRACK && NF_NAT
+ default NF_NAT && NF_CONNTRACK_SIP
+
endif # NF_CONNTRACK
# transparent proxy support
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 7d6e1ea..7d6d1a0 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -57,6 +57,7 @@ obj-$(CONFIG_NF_NAT_PROTO_SCTP) += nf_nat_proto_sctp.o
# NAT helpers
obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
+obj-$(CONFIG_NF_NAT_SIP) += nf_nat_sip.o
# transparent proxy support
obj-$(CONFIG_NETFILTER_TPROXY) += nf_tproxy_core.o
diff --git a/net/netfilter/nf_conntrack_sip.c b/net/netfilter/nf_conntrack_sip.c
index d517490..df8f4f2 100644
--- a/net/netfilter/nf_conntrack_sip.c
+++ b/net/netfilter/nf_conntrack_sip.c
@@ -57,7 +57,8 @@ unsigned int (*nf_nat_sip_hook)(struct sk_buff *skb, unsigned int protoff,
unsigned int *datalen) __read_mostly;
EXPORT_SYMBOL_GPL(nf_nat_sip_hook);
-void (*nf_nat_sip_seq_adjust_hook)(struct sk_buff *skb, s16 off) __read_mostly;
+void (*nf_nat_sip_seq_adjust_hook)(struct sk_buff *skb, unsigned int protoff,
+ s16 off) __read_mostly;
EXPORT_SYMBOL_GPL(nf_nat_sip_seq_adjust_hook);
unsigned int (*nf_nat_sip_expect_hook)(struct sk_buff *skb,
@@ -742,13 +743,18 @@ static int sdp_addr_len(const struct nf_conn *ct, const char *dptr,
* be tolerant and also accept records terminated with a single newline
* character". We handle both cases.
*/
-static const struct sip_header ct_sdp_hdrs[] = {
- [SDP_HDR_VERSION] = SDP_HDR("v=", NULL, digits_len),
- [SDP_HDR_OWNER_IP4] = SDP_HDR("o=", "IN IP4 ", sdp_addr_len),
- [SDP_HDR_CONNECTION_IP4] = SDP_HDR("c=", "IN IP4 ", sdp_addr_len),
- [SDP_HDR_OWNER_IP6] = SDP_HDR("o=", "IN IP6 ", sdp_addr_len),
- [SDP_HDR_CONNECTION_IP6] = SDP_HDR("c=", "IN IP6 ", sdp_addr_len),
- [SDP_HDR_MEDIA] = SDP_HDR("m=", NULL, media_len),
+static const struct sip_header ct_sdp_hdrs_v4[] = {
+ [SDP_HDR_VERSION] = SDP_HDR("v=", NULL, digits_len),
+ [SDP_HDR_OWNER] = SDP_HDR("o=", "IN IP4 ", sdp_addr_len),
+ [SDP_HDR_CONNECTION] = SDP_HDR("c=", "IN IP4 ", sdp_addr_len),
+ [SDP_HDR_MEDIA] = SDP_HDR("m=", NULL, media_len),
+};
+
+static const struct sip_header ct_sdp_hdrs_v6[] = {
+ [SDP_HDR_VERSION] = SDP_HDR("v=", NULL, digits_len),
+ [SDP_HDR_OWNER] = SDP_HDR("o=", "IN IP6 ", sdp_addr_len),
+ [SDP_HDR_CONNECTION] = SDP_HDR("c=", "IN IP6 ", sdp_addr_len),
+ [SDP_HDR_MEDIA] = SDP_HDR("m=", NULL, media_len),
};
/* Linear string search within SDP header values */
@@ -774,11 +780,14 @@ int ct_sip_get_sdp_header(const struct nf_conn *ct, const char *dptr,
enum sdp_header_types term,
unsigned int *matchoff, unsigned int *matchlen)
{
- const struct sip_header *hdr = &ct_sdp_hdrs[type];
- const struct sip_header *thdr = &ct_sdp_hdrs[term];
+ const struct sip_header *hdrs, *hdr, *thdr;
const char *start = dptr, *limit = dptr + datalen;
int shift = 0;
+ hdrs = nf_ct_l3num(ct) == NFPROTO_IPV4 ? ct_sdp_hdrs_v4 : ct_sdp_hdrs_v6;
+ hdr = &hdrs[type];
+ thdr = &hdrs[term];
+
for (dptr += dataoff; dptr < limit; dptr++) {
/* Find beginning of line */
if (*dptr != '\r' && *dptr != '\n')
@@ -945,12 +954,12 @@ static int set_expected_rtp_rtcp(struct sk_buff *skb, unsigned int protoff,
exp->class != class)
break;
#ifdef CONFIG_NF_NAT_NEEDED
- if (exp->tuple.src.l3num == AF_INET && !direct_rtp &&
- (exp->saved_addr.ip != exp->tuple.dst.u3.ip ||
+ if (!direct_rtp &&
+ (!nf_inet_addr_cmp(&exp->saved_addr, &exp->tuple.dst.u3) ||
exp->saved_proto.udp.port != exp->tuple.dst.u.udp.port) &&
ct->status & IPS_NAT_MASK) {
- daddr->ip = exp->saved_addr.ip;
- tuple.dst.u3.ip = exp->saved_addr.ip;
+ *daddr = exp->saved_addr;
+ tuple.dst.u3 = exp->saved_addr;
tuple.dst.u.udp.port = exp->saved_proto.udp.port;
direct_rtp = 1;
} else
@@ -987,8 +996,7 @@ static int set_expected_rtp_rtcp(struct sk_buff *skb, unsigned int protoff,
IPPROTO_UDP, NULL, &rtcp_port);
nf_nat_sdp_media = rcu_dereference(nf_nat_sdp_media_hook);
- if (nf_nat_sdp_media && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK && !direct_rtp)
+ if (nf_nat_sdp_media && ct->status & IPS_NAT_MASK && !direct_rtp)
ret = nf_nat_sdp_media(skb, protoff, dataoff, dptr, datalen,
rtp_exp, rtcp_exp,
mediaoff, medialen, daddr);
@@ -1044,15 +1052,12 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
unsigned int i;
union nf_inet_addr caddr, maddr, rtp_addr;
unsigned int port;
- enum sdp_header_types c_hdr;
const struct sdp_media_type *t;
int ret = NF_ACCEPT;
typeof(nf_nat_sdp_addr_hook) nf_nat_sdp_addr;
typeof(nf_nat_sdp_session_hook) nf_nat_sdp_session;
nf_nat_sdp_addr = rcu_dereference(nf_nat_sdp_addr_hook);
- c_hdr = nf_ct_l3num(ct) == AF_INET ? SDP_HDR_CONNECTION_IP4 :
- SDP_HDR_CONNECTION_IP6;
/* Find beginning of session description */
if (ct_sip_get_sdp_header(ct, *dptr, 0, *datalen,
@@ -1066,7 +1071,7 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
* the end of the session description. */
caddr_len = 0;
if (ct_sip_parse_sdp_addr(ct, *dptr, sdpoff, *datalen,
- c_hdr, SDP_HDR_MEDIA,
+ SDP_HDR_CONNECTION, SDP_HDR_MEDIA,
&matchoff, &matchlen, &caddr) > 0)
caddr_len = matchlen;
@@ -1096,7 +1101,7 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
/* The media description overrides the session description. */
maddr_len = 0;
if (ct_sip_parse_sdp_addr(ct, *dptr, mediaoff, *datalen,
- c_hdr, SDP_HDR_MEDIA,
+ SDP_HDR_CONNECTION, SDP_HDR_MEDIA,
&matchoff, &matchlen, &maddr) > 0) {
maddr_len = matchlen;
memcpy(&rtp_addr, &maddr, sizeof(rtp_addr));
@@ -1113,11 +1118,10 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
return ret;
/* Update media connection address if present */
- if (maddr_len && nf_nat_sdp_addr &&
- nf_ct_l3num(ct) == NFPROTO_IPV4 && ct->status & IPS_NAT_MASK) {
+ if (maddr_len && nf_nat_sdp_addr && ct->status & IPS_NAT_MASK) {
ret = nf_nat_sdp_addr(skb, protoff, dataoff,
- dptr, datalen,
- mediaoff, c_hdr, SDP_HDR_MEDIA,
+ dptr, datalen, mediaoff,
+ SDP_HDR_CONNECTION, SDP_HDR_MEDIA,
&rtp_addr);
if (ret != NF_ACCEPT)
return ret;
@@ -1127,8 +1131,7 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
/* Update session connection and owner addresses */
nf_nat_sdp_session = rcu_dereference(nf_nat_sdp_session_hook);
- if (nf_nat_sdp_session && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_sdp_session && ct->status & IPS_NAT_MASK)
ret = nf_nat_sdp_session(skb, protoff, dataoff,
dptr, datalen, sdpoff, &rtp_addr);
@@ -1293,8 +1296,7 @@ static int process_register_request(struct sk_buff *skb, unsigned int protoff,
exp->flags = NF_CT_EXPECT_PERMANENT | NF_CT_EXPECT_INACTIVE;
nf_nat_sip_expect = rcu_dereference(nf_nat_sip_expect_hook);
- if (nf_nat_sip_expect && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_sip_expect && ct->status & IPS_NAT_MASK)
ret = nf_nat_sip_expect(skb, protoff, dataoff, dptr, datalen,
exp, matchoff, matchlen);
else {
@@ -1476,8 +1478,7 @@ static int process_sip_msg(struct sk_buff *skb, struct nf_conn *ct,
else
ret = process_sip_response(skb, protoff, dataoff, dptr, datalen);
- if (ret == NF_ACCEPT && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK) {
+ if (ret == NF_ACCEPT && ct->status & IPS_NAT_MASK) {
nf_nat_sip = rcu_dereference(nf_nat_sip_hook);
if (nf_nat_sip && !nf_nat_sip(skb, protoff, dataoff,
dptr, datalen))
@@ -1560,11 +1561,10 @@ static int sip_help_tcp(struct sk_buff *skb, unsigned int protoff,
datalen = datalen + diff - msglen;
}
- if (ret == NF_ACCEPT && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK) {
+ if (ret == NF_ACCEPT && ct->status & IPS_NAT_MASK) {
nf_nat_sip_seq_adjust = rcu_dereference(nf_nat_sip_seq_adjust_hook);
if (nf_nat_sip_seq_adjust)
- nf_nat_sip_seq_adjust(skb, tdiff);
+ nf_nat_sip_seq_adjust(skb, protoff, tdiff);
}
return ret;
diff --git a/net/ipv4/netfilter/nf_nat_sip.c b/net/netfilter/nf_nat_sip.c
similarity index 76%
rename from net/ipv4/netfilter/nf_nat_sip.c
rename to net/netfilter/nf_nat_sip.c
index 47a4718..f4db3a7 100644
--- a/net/ipv4/netfilter/nf_nat_sip.c
+++ b/net/netfilter/nf_nat_sip.c
@@ -3,7 +3,7 @@
* (C) 2005 by Christian Hentschel <chentschel@arnet.com.ar>
* based on RR's ip_nat_ftp.c and other modules.
* (C) 2007 United Security Providers
- * (C) 2007, 2008 Patrick McHardy <kaber@trash.net>
+ * (C) 2007, 2008, 2011, 2012 Patrick McHardy <kaber@trash.net>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -12,8 +12,7 @@
#include <linux/module.h>
#include <linux/skbuff.h>
-#include <linux/ip.h>
-#include <net/ip.h>
+#include <linux/inet.h>
#include <linux/udp.h>
#include <linux/tcp.h>
@@ -41,8 +40,8 @@ static unsigned int mangle_packet(struct sk_buff *skb, unsigned int protoff,
unsigned int baseoff;
if (nf_ct_protonum(ct) == IPPROTO_TCP) {
- th = (struct tcphdr *)(skb->data + ip_hdrlen(skb));
- baseoff = ip_hdrlen(skb) + th->doff * 4;
+ th = (struct tcphdr *)(skb->data + protoff);
+ baseoff = protoff + th->doff * 4;
matchoff += dataoff - baseoff;
if (!__nf_nat_mangle_tcp_packet(skb, ct, ctinfo,
@@ -50,7 +49,7 @@ static unsigned int mangle_packet(struct sk_buff *skb, unsigned int protoff,
buffer, buflen, false))
return 0;
} else {
- baseoff = ip_hdrlen(skb) + sizeof(struct udphdr);
+ baseoff = protoff + sizeof(struct udphdr);
matchoff += dataoff - baseoff;
if (!nf_nat_mangle_udp_packet(skb, ct, ctinfo,
@@ -65,6 +64,28 @@ static unsigned int mangle_packet(struct sk_buff *skb, unsigned int protoff,
return 1;
}
+static int sip_sprintf_addr(const struct nf_conn *ct, char *buffer,
+ const union nf_inet_addr *addr, bool delim)
+{
+ if (nf_ct_l3num(ct) == NFPROTO_IPV4)
+ return sprintf(buffer, "%pI4", &addr->ip);
+ else {
+ if (delim)
+ return sprintf(buffer, "[%pI6c]", &addr->ip6);
+ else
+ return sprintf(buffer, "%pI6c", &addr->ip6);
+ }
+}
+
+static int sip_sprintf_addr_port(const struct nf_conn *ct, char *buffer,
+ const union nf_inet_addr *addr, u16 port)
+{
+ if (nf_ct_l3num(ct) == NFPROTO_IPV4)
+ return sprintf(buffer, "%pI4:%u", &addr->ip, port);
+ else
+ return sprintf(buffer, "[%pI6c]:%u", &addr->ip6, port);
+}
+
static int map_addr(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
@@ -74,27 +95,26 @@ static int map_addr(struct sk_buff *skb, unsigned int protoff,
enum ip_conntrack_info ctinfo;
struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo);
- char buffer[sizeof("nnn.nnn.nnn.nnn:nnnnn")];
+ char buffer[INET6_ADDRSTRLEN + sizeof("[]:nnnnn")];
unsigned int buflen;
- __be32 newaddr;
+ union nf_inet_addr newaddr;
__be16 newport;
- if (ct->tuplehash[dir].tuple.src.u3.ip == addr->ip &&
+ if (nf_inet_addr_cmp(&ct->tuplehash[dir].tuple.src.u3, addr) &&
ct->tuplehash[dir].tuple.src.u.udp.port == port) {
- newaddr = ct->tuplehash[!dir].tuple.dst.u3.ip;
+ newaddr = ct->tuplehash[!dir].tuple.dst.u3;
newport = ct->tuplehash[!dir].tuple.dst.u.udp.port;
- } else if (ct->tuplehash[dir].tuple.dst.u3.ip == addr->ip &&
+ } else if (nf_inet_addr_cmp(&ct->tuplehash[dir].tuple.dst.u3, addr) &&
ct->tuplehash[dir].tuple.dst.u.udp.port == port) {
- newaddr = ct->tuplehash[!dir].tuple.src.u3.ip;
+ newaddr = ct->tuplehash[!dir].tuple.src.u3;
newport = ct->tuplehash[!dir].tuple.src.u.udp.port;
} else
return 1;
- if (newaddr == addr->ip && newport == port)
+ if (nf_inet_addr_cmp(&newaddr, addr) && newport == port)
return 1;
- buflen = sprintf(buffer, "%pI4:%u", &newaddr, ntohs(newport));
-
+ buflen = sip_sprintf_addr_port(ct, buffer, &newaddr, ntohs(newport));
return mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen);
}
@@ -117,7 +137,7 @@ static int map_sip_addr(struct sk_buff *skb, unsigned int protoff,
matchoff, matchlen, &addr, port);
}
-static unsigned int ip_nat_sip(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen)
{
@@ -152,16 +172,18 @@ static unsigned int ip_nat_sip(struct sk_buff *skb, unsigned int protoff,
hdr, NULL, &matchoff, &matchlen,
&addr, &port) > 0) {
unsigned int olen, matchend, poff, plen, buflen, n;
- char buffer[sizeof("nnn.nnn.nnn.nnn:nnnnn")];
+ char buffer[INET6_ADDRSTRLEN + sizeof("[]:nnnnn")];
/* We're only interested in headers related to this
* connection */
if (request) {
- if (addr.ip != ct->tuplehash[dir].tuple.src.u3.ip ||
+ if (!nf_inet_addr_cmp(&addr,
+ &ct->tuplehash[dir].tuple.src.u3) ||
port != ct->tuplehash[dir].tuple.src.u.udp.port)
goto next;
} else {
- if (addr.ip != ct->tuplehash[dir].tuple.dst.u3.ip ||
+ if (!nf_inet_addr_cmp(&addr,
+ &ct->tuplehash[dir].tuple.dst.u3) ||
port != ct->tuplehash[dir].tuple.dst.u.udp.port)
goto next;
}
@@ -178,10 +200,11 @@ static unsigned int ip_nat_sip(struct sk_buff *skb, unsigned int protoff,
if (ct_sip_parse_address_param(ct, *dptr, matchend, *datalen,
"maddr=", &poff, &plen,
&addr, true) > 0 &&
- addr.ip == ct->tuplehash[dir].tuple.src.u3.ip &&
- addr.ip != ct->tuplehash[!dir].tuple.dst.u3.ip) {
- buflen = sprintf(buffer, "%pI4",
- &ct->tuplehash[!dir].tuple.dst.u3.ip);
+ nf_inet_addr_cmp(&addr, &ct->tuplehash[dir].tuple.src.u3) &&
+ !nf_inet_addr_cmp(&addr, &ct->tuplehash[!dir].tuple.dst.u3)) {
+ buflen = sip_sprintf_addr(ct, buffer,
+ &ct->tuplehash[!dir].tuple.dst.u3,
+ true);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
poff, plen, buffer, buflen))
return NF_DROP;
@@ -192,10 +215,11 @@ static unsigned int ip_nat_sip(struct sk_buff *skb, unsigned int protoff,
if (ct_sip_parse_address_param(ct, *dptr, matchend, *datalen,
"received=", &poff, &plen,
&addr, false) > 0 &&
- addr.ip == ct->tuplehash[dir].tuple.dst.u3.ip &&
- addr.ip != ct->tuplehash[!dir].tuple.src.u3.ip) {
- buflen = sprintf(buffer, "%pI4",
- &ct->tuplehash[!dir].tuple.src.u3.ip);
+ nf_inet_addr_cmp(&addr, &ct->tuplehash[dir].tuple.dst.u3) &&
+ !nf_inet_addr_cmp(&addr, &ct->tuplehash[!dir].tuple.src.u3)) {
+ buflen = sip_sprintf_addr(ct, buffer,
+ &ct->tuplehash[!dir].tuple.src.u3,
+ false);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
poff, plen, buffer, buflen))
return NF_DROP;
@@ -237,7 +261,8 @@ next:
return NF_ACCEPT;
}
-static void ip_nat_sip_seq_adjust(struct sk_buff *skb, s16 off)
+static void nf_nat_sip_seq_adjust(struct sk_buff *skb, unsigned int protoff,
+ s16 off)
{
enum ip_conntrack_info ctinfo;
struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
@@ -246,12 +271,12 @@ static void ip_nat_sip_seq_adjust(struct sk_buff *skb, s16 off)
if (nf_ct_protonum(ct) != IPPROTO_TCP || off == 0)
return;
- th = (struct tcphdr *)(skb->data + ip_hdrlen(skb));
+ th = (struct tcphdr *)(skb->data + protoff);
nf_nat_set_seq_adjust(ct, ctinfo, th->seq, off);
}
/* Handles expected signalling connections and media streams */
-static void ip_nat_sip_expected(struct nf_conn *ct,
+static void nf_nat_sip_expected(struct nf_conn *ct,
struct nf_conntrack_expect *exp)
{
struct nf_nat_range range;
@@ -267,8 +292,8 @@ static void ip_nat_sip_expected(struct nf_conn *ct,
/* Change src to where master sends to, but only if the connection
* actually came from the same source. */
- if (ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3.ip ==
- ct->master->tuplehash[exp->dir].tuple.src.u3.ip) {
+ if (nf_inet_addr_cmp(&ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3,
+ &ct->master->tuplehash[exp->dir].tuple.src.u3)) {
range.flags = NF_NAT_RANGE_MAP_IPS;
range.min_addr = range.max_addr
= ct->master->tuplehash[!exp->dir].tuple.dst.u3;
@@ -276,7 +301,7 @@ static void ip_nat_sip_expected(struct nf_conn *ct,
}
}
-static unsigned int ip_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
struct nf_conntrack_expect *exp,
@@ -286,16 +311,17 @@ static unsigned int ip_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
enum ip_conntrack_info ctinfo;
struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo);
- __be32 newip;
+ union nf_inet_addr newaddr;
u_int16_t port;
- char buffer[sizeof("nnn.nnn.nnn.nnn:nnnnn")];
+ char buffer[INET6_ADDRSTRLEN + sizeof("[]:nnnnn")];
unsigned int buflen;
/* Connection will come from reply */
- if (ct->tuplehash[dir].tuple.src.u3.ip == ct->tuplehash[!dir].tuple.dst.u3.ip)
- newip = exp->tuple.dst.u3.ip;
+ if (nf_inet_addr_cmp(&ct->tuplehash[dir].tuple.src.u3,
+ &ct->tuplehash[!dir].tuple.dst.u3))
+ newaddr = exp->tuple.dst.u3;
else
- newip = ct->tuplehash[!dir].tuple.dst.u3.ip;
+ newaddr = ct->tuplehash[!dir].tuple.dst.u3;
/* If the signalling port matches the connection's source port in the
* original direction, try to use the destination port in the opposite
@@ -307,10 +333,10 @@ static unsigned int ip_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
port = ntohs(exp->tuple.dst.u.udp.port);
exp->saved_addr = exp->tuple.dst.u3;
- exp->tuple.dst.u3.ip = newip;
+ exp->tuple.dst.u3 = newaddr;
exp->saved_proto.udp.port = exp->tuple.dst.u.udp.port;
exp->dir = !dir;
- exp->expectfn = ip_nat_sip_expected;
+ exp->expectfn = nf_nat_sip_expected;
for (; port != 0; port++) {
int ret;
@@ -328,9 +354,9 @@ static unsigned int ip_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
if (port == 0)
return NF_DROP;
- if (exp->tuple.dst.u3.ip != exp->saved_addr.ip ||
+ if (!nf_inet_addr_cmp(&exp->tuple.dst.u3, &exp->saved_addr) ||
exp->tuple.dst.u.udp.port != exp->saved_proto.udp.port) {
- buflen = sprintf(buffer, "%pI4:%u", &newip, port);
+ buflen = sip_sprintf_addr_port(ct, buffer, &newaddr, port);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen))
goto err;
@@ -388,7 +414,7 @@ static int mangle_sdp_packet(struct sk_buff *skb, unsigned int protoff,
matchoff, matchlen, buffer, buflen) ? 0 : -EINVAL;
}
-static unsigned int ip_nat_sdp_addr(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sdp_addr(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
unsigned int sdpoff,
@@ -396,10 +422,12 @@ static unsigned int ip_nat_sdp_addr(struct sk_buff *skb, unsigned int protoff,
enum sdp_header_types term,
const union nf_inet_addr *addr)
{
- char buffer[sizeof("nnn.nnn.nnn.nnn")];
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
+ char buffer[INET6_ADDRSTRLEN];
unsigned int buflen;
- buflen = sprintf(buffer, "%pI4", &addr->ip);
+ buflen = sip_sprintf_addr(ct, buffer, addr, false);
if (mangle_sdp_packet(skb, protoff, dataoff, dptr, datalen,
sdpoff, type, term, buffer, buflen))
return 0;
@@ -407,7 +435,7 @@ static unsigned int ip_nat_sdp_addr(struct sk_buff *skb, unsigned int protoff,
return mangle_content_len(skb, protoff, dataoff, dptr, datalen);
}
-static unsigned int ip_nat_sdp_port(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sdp_port(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
unsigned int matchoff,
@@ -425,24 +453,25 @@ static unsigned int ip_nat_sdp_port(struct sk_buff *skb, unsigned int protoff,
return mangle_content_len(skb, protoff, dataoff, dptr, datalen);
}
-static unsigned int ip_nat_sdp_session(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sdp_session(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
unsigned int sdpoff,
const union nf_inet_addr *addr)
{
- char buffer[sizeof("nnn.nnn.nnn.nnn")];
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
+ char buffer[INET6_ADDRSTRLEN];
unsigned int buflen;
/* Mangle session description owner and contact addresses */
- buflen = sprintf(buffer, "%pI4", &addr->ip);
+ buflen = sip_sprintf_addr(ct, buffer, addr, false);
if (mangle_sdp_packet(skb, protoff, dataoff, dptr, datalen, sdpoff,
- SDP_HDR_OWNER_IP4, SDP_HDR_MEDIA,
- buffer, buflen))
+ SDP_HDR_OWNER, SDP_HDR_MEDIA, buffer, buflen))
return 0;
switch (mangle_sdp_packet(skb, protoff, dataoff, dptr, datalen, sdpoff,
- SDP_HDR_CONNECTION_IP4, SDP_HDR_MEDIA,
+ SDP_HDR_CONNECTION, SDP_HDR_MEDIA,
buffer, buflen)) {
case 0:
/*
@@ -463,7 +492,7 @@ static unsigned int ip_nat_sdp_session(struct sk_buff *skb, unsigned int protoff
/* So, this packet has hit the connection tracking matching code.
Mangle it, and change the expectation to match the new version. */
-static unsigned int ip_nat_sdp_media(struct sk_buff *skb, unsigned int protoff,
+static unsigned int nf_nat_sdp_media(struct sk_buff *skb, unsigned int protoff,
unsigned int dataoff,
const char **dptr, unsigned int *datalen,
struct nf_conntrack_expect *rtp_exp,
@@ -478,23 +507,23 @@ static unsigned int ip_nat_sdp_media(struct sk_buff *skb, unsigned int protoff,
u_int16_t port;
/* Connection will come from reply */
- if (ct->tuplehash[dir].tuple.src.u3.ip ==
- ct->tuplehash[!dir].tuple.dst.u3.ip)
- rtp_addr->ip = rtp_exp->tuple.dst.u3.ip;
+ if (nf_inet_addr_cmp(&ct->tuplehash[dir].tuple.src.u3,
+ &ct->tuplehash[!dir].tuple.dst.u3))
+ *rtp_addr = rtp_exp->tuple.dst.u3;
else
- rtp_addr->ip = ct->tuplehash[!dir].tuple.dst.u3.ip;
+ *rtp_addr = ct->tuplehash[!dir].tuple.dst.u3;
rtp_exp->saved_addr = rtp_exp->tuple.dst.u3;
- rtp_exp->tuple.dst.u3.ip = rtp_addr->ip;
+ rtp_exp->tuple.dst.u3 = *rtp_addr;
rtp_exp->saved_proto.udp.port = rtp_exp->tuple.dst.u.udp.port;
rtp_exp->dir = !dir;
- rtp_exp->expectfn = ip_nat_sip_expected;
+ rtp_exp->expectfn = nf_nat_sip_expected;
rtcp_exp->saved_addr = rtcp_exp->tuple.dst.u3;
- rtcp_exp->tuple.dst.u3.ip = rtp_addr->ip;
+ rtcp_exp->tuple.dst.u3 = *rtp_addr;
rtcp_exp->saved_proto.udp.port = rtcp_exp->tuple.dst.u.udp.port;
rtcp_exp->dir = !dir;
- rtcp_exp->expectfn = ip_nat_sip_expected;
+ rtcp_exp->expectfn = nf_nat_sip_expected;
/* Try to get same pair of ports: if not, try to change them. */
for (port = ntohs(rtp_exp->tuple.dst.u.udp.port);
@@ -525,7 +554,7 @@ static unsigned int ip_nat_sdp_media(struct sk_buff *skb, unsigned int protoff,
/* Update media port. */
if (rtp_exp->tuple.dst.u.udp.port != rtp_exp->saved_proto.udp.port &&
- !ip_nat_sdp_port(skb, protoff, dataoff, dptr, datalen,
+ !nf_nat_sdp_port(skb, protoff, dataoff, dptr, datalen,
mediaoff, medialen, port))
goto err2;
@@ -539,8 +568,8 @@ err1:
}
static struct nf_ct_helper_expectfn sip_nat = {
- .name = "sip",
- .expectfn = ip_nat_sip_expected,
+ .name = "sip",
+ .expectfn = nf_nat_sip_expected,
};
static void __exit nf_nat_sip_fini(void)
@@ -565,13 +594,13 @@ static int __init nf_nat_sip_init(void)
BUG_ON(nf_nat_sdp_port_hook != NULL);
BUG_ON(nf_nat_sdp_session_hook != NULL);
BUG_ON(nf_nat_sdp_media_hook != NULL);
- RCU_INIT_POINTER(nf_nat_sip_hook, ip_nat_sip);
- RCU_INIT_POINTER(nf_nat_sip_seq_adjust_hook, ip_nat_sip_seq_adjust);
- RCU_INIT_POINTER(nf_nat_sip_expect_hook, ip_nat_sip_expect);
- RCU_INIT_POINTER(nf_nat_sdp_addr_hook, ip_nat_sdp_addr);
- RCU_INIT_POINTER(nf_nat_sdp_port_hook, ip_nat_sdp_port);
- RCU_INIT_POINTER(nf_nat_sdp_session_hook, ip_nat_sdp_session);
- RCU_INIT_POINTER(nf_nat_sdp_media_hook, ip_nat_sdp_media);
+ RCU_INIT_POINTER(nf_nat_sip_hook, nf_nat_sip);
+ RCU_INIT_POINTER(nf_nat_sip_seq_adjust_hook, nf_nat_sip_seq_adjust);
+ RCU_INIT_POINTER(nf_nat_sip_expect_hook, nf_nat_sip_expect);
+ RCU_INIT_POINTER(nf_nat_sdp_addr_hook, nf_nat_sdp_addr);
+ RCU_INIT_POINTER(nf_nat_sdp_port_hook, nf_nat_sdp_port);
+ RCU_INIT_POINTER(nf_nat_sdp_session_hook, nf_nat_sdp_session);
+ RCU_INIT_POINTER(nf_nat_sdp_media_hook, nf_nat_sdp_media);
nf_ct_helper_expectfn_register(&sip_nat);
return 0;
}
--
1.7.1
^ permalink raw reply related
* [PATCH 14/18] netfilter: nf_nat: support IPv6 in amanda NAT helper
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/ipv4/netfilter/Kconfig | 5 -----
net/ipv4/netfilter/Makefile | 1 -
net/netfilter/Kconfig | 5 +++++
net/netfilter/Makefile | 1 +
net/netfilter/nf_conntrack_amanda.c | 3 +--
net/{ipv4 => }/netfilter/nf_nat_amanda.c | 0
6 files changed, 7 insertions(+), 8 deletions(-)
rename net/{ipv4 => }/netfilter/nf_nat_amanda.c (100%)
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 80272e7..df42031 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -230,11 +230,6 @@ config NF_NAT_TFTP
depends on NF_CONNTRACK && NF_NAT_IPV4
default NF_NAT_IPV4 && NF_CONNTRACK_TFTP
-config NF_NAT_AMANDA
- tristate
- depends on NF_CONNTRACK && NF_NAT_IPV4
- default NF_NAT_IPV4 && NF_CONNTRACK_AMANDA
-
config NF_NAT_PPTP
tristate
depends on NF_CONNTRACK && NF_NAT_IPV4
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index 4d8a4ad..8baa496 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -20,7 +20,6 @@ obj-$(CONFIG_NF_NAT_IPV4) += nf_nat_ipv4.o
obj-$(CONFIG_NF_DEFRAG_IPV4) += nf_defrag_ipv4.o
# NAT helpers (nf_conntrack)
-obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
obj-$(CONFIG_NF_NAT_H323) += nf_nat_h323.o
obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_PPTP) += nf_nat_pptp.o
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 3104494..2eee9f1 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -380,6 +380,11 @@ config NF_NAT_PROTO_SCTP
depends on NF_NAT && NF_CT_PROTO_SCTP
select LIBCRC32C
+config NF_NAT_AMANDA
+ tristate
+ depends on NF_CONNTRACK && NF_NAT
+ default NF_NAT && NF_CONNTRACK_AMANDA
+
config NF_NAT_FTP
tristate
depends on NF_CONNTRACK && NF_NAT
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 16592b1..7d6e1ea 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -55,6 +55,7 @@ obj-$(CONFIG_NF_NAT_PROTO_UDPLITE) += nf_nat_proto_udplite.o
obj-$(CONFIG_NF_NAT_PROTO_SCTP) += nf_nat_proto_sctp.o
# NAT helpers
+obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
# transparent proxy support
diff --git a/net/netfilter/nf_conntrack_amanda.c b/net/netfilter/nf_conntrack_amanda.c
index e0212b5..c514fe6 100644
--- a/net/netfilter/nf_conntrack_amanda.c
+++ b/net/netfilter/nf_conntrack_amanda.c
@@ -155,8 +155,7 @@ static int amanda_help(struct sk_buff *skb,
IPPROTO_TCP, NULL, &port);
nf_nat_amanda = rcu_dereference(nf_nat_amanda_hook);
- if (nf_nat_amanda && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_amanda && ct->status & IPS_NAT_MASK)
ret = nf_nat_amanda(skb, ctinfo, protoff,
off - dataoff, len, exp);
else if (nf_ct_expect_related(exp) != 0)
diff --git a/net/ipv4/netfilter/nf_nat_amanda.c b/net/netfilter/nf_nat_amanda.c
similarity index 100%
rename from net/ipv4/netfilter/nf_nat_amanda.c
rename to net/netfilter/nf_nat_amanda.c
--
1.7.1
^ permalink raw reply related
* [PATCH 11/18] netfilter: ip6tables: add REDIRECT target
From: Patrick McHardy @ 2012-08-20 3:39 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/ipv6/netfilter/Kconfig | 11 ++++
net/ipv6/netfilter/Makefile | 1 +
net/ipv6/netfilter/ip6t_REDIRECT.c | 98 ++++++++++++++++++++++++++++++++++++
3 files changed, 110 insertions(+), 0 deletions(-)
create mode 100644 net/ipv6/netfilter/ip6t_REDIRECT.c
diff --git a/net/ipv6/netfilter/Kconfig b/net/ipv6/netfilter/Kconfig
index 54a5032..585590f 100644
--- a/net/ipv6/netfilter/Kconfig
+++ b/net/ipv6/netfilter/Kconfig
@@ -156,6 +156,17 @@ config IP6_NF_TARGET_MASQUERADE
To compile it as a module, choose M here. If unsure, say N.
+config IP6_NF_TARGET_REDIRECT
+ tristate "REDIRECT target support"
+ depends on NF_NAT_IPV6
+ help
+ REDIRECT is a special case of NAT: all incoming connections are
+ mapped onto the incoming interface's address, causing the packets to
+ come to the local machine instead of passing through. This is
+ useful for transparent proxies.
+
+ To compile it as a module, choose M here. If unsure, say N.
+
config IP6_NF_FILTER
tristate "Packet filtering"
default m if NETFILTER_ADVANCED=n
diff --git a/net/ipv6/netfilter/Makefile b/net/ipv6/netfilter/Makefile
index 068bad1..e30a531 100644
--- a/net/ipv6/netfilter/Makefile
+++ b/net/ipv6/netfilter/Makefile
@@ -35,4 +35,5 @@ obj-$(CONFIG_IP6_NF_MATCH_RT) += ip6t_rt.o
# targets
obj-$(CONFIG_IP6_NF_TARGET_MASQUERADE) += ip6t_MASQUERADE.o
+obj-$(CONFIG_IP6_NF_TARGET_REDIRECT) += ip6t_REDIRECT.o
obj-$(CONFIG_IP6_NF_TARGET_REJECT) += ip6t_REJECT.o
diff --git a/net/ipv6/netfilter/ip6t_REDIRECT.c b/net/ipv6/netfilter/ip6t_REDIRECT.c
new file mode 100644
index 0000000..60497a3
--- /dev/null
+++ b/net/ipv6/netfilter/ip6t_REDIRECT.c
@@ -0,0 +1,98 @@
+/*
+ * Copyright (c) 2011 Patrick McHardy <kaber@trash.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * Based on Rusty Russell's IPv4 REDIRECT target. Development of IPv6
+ * NAT funded by Astaro.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/netfilter.h>
+#include <linux/netfilter_ipv6.h>
+#include <linux/netfilter/x_tables.h>
+#include <net/addrconf.h>
+#include <net/netfilter/nf_nat.h>
+
+static const struct in6_addr loopback_addr = IN6ADDR_LOOPBACK_INIT;
+
+static unsigned int
+redirect_tg6(struct sk_buff *skb, const struct xt_action_param *par)
+{
+ const struct nf_nat_range *range = par->targinfo;
+ struct nf_nat_range newrange;
+ struct in6_addr newdst;
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct;
+
+ ct = nf_ct_get(skb, &ctinfo);
+ if (par->hooknum == NF_INET_LOCAL_OUT)
+ newdst = loopback_addr;
+ else {
+ struct inet6_dev *idev;
+ struct inet6_ifaddr *ifa;
+ bool addr = false;
+
+ rcu_read_lock();
+ idev = __in6_dev_get(skb->dev);
+ if (idev != NULL) {
+ list_for_each_entry(ifa, &idev->addr_list, if_list) {
+ newdst = ifa->addr;
+ addr = true;
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ if (!addr)
+ return NF_DROP;
+ }
+
+ newrange.flags = range->flags | NF_NAT_RANGE_MAP_IPS;
+ newrange.min_addr.in6 = newdst;
+ newrange.max_addr.in6 = newdst;
+ newrange.min_proto = range->min_proto;
+ newrange.max_proto = range->max_proto;
+
+ return nf_nat_setup_info(ct, &newrange, NF_NAT_MANIP_DST);
+}
+
+static int redirect_tg6_checkentry(const struct xt_tgchk_param *par)
+{
+ const struct nf_nat_range *range = par->targinfo;
+
+ if (range->flags & NF_NAT_RANGE_MAP_IPS)
+ return -EINVAL;
+ return 0;
+}
+
+static struct xt_target redirect_tg6_reg __read_mostly = {
+ .name = "REDIRECT",
+ .family = NFPROTO_IPV6,
+ .checkentry = redirect_tg6_checkentry,
+ .target = redirect_tg6,
+ .targetsize = sizeof(struct nf_nat_range),
+ .table = "nat",
+ .hooks = (1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_OUT),
+ .me = THIS_MODULE,
+};
+
+static int __init redirect_tg6_init(void)
+{
+ return xt_register_target(&redirect_tg6_reg);
+}
+
+static void __exit redirect_tg6_exit(void)
+{
+ xt_unregister_target(&redirect_tg6_reg);
+}
+
+module_init(redirect_tg6_init);
+module_exit(redirect_tg6_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Patrick McHardy <kaber@trash.net>");
+MODULE_DESCRIPTION("Xtables: Connection redirection to localhost");
--
1.7.1
^ permalink raw reply related
* [PATCH 10/18] netfilter: ip6tables: add MASQUERADE target
From: Patrick McHardy @ 2012-08-20 3:39 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/net/addrconf.h | 2 +-
include/net/netfilter/nf_nat.h | 4 +-
net/ipv4/netfilter/ipt_MASQUERADE.c | 3 +-
net/ipv6/addrconf.c | 2 +-
net/ipv6/netfilter/Kconfig | 12 +++
net/ipv6/netfilter/Makefile | 1 +
net/ipv6/netfilter/ip6t_MASQUERADE.c | 135 ++++++++++++++++++++++++++++++++++
7 files changed, 155 insertions(+), 4 deletions(-)
create mode 100644 net/ipv6/netfilter/ip6t_MASQUERADE.c
diff --git a/include/net/addrconf.h b/include/net/addrconf.h
index 089a09d..9e63e76 100644
--- a/include/net/addrconf.h
+++ b/include/net/addrconf.h
@@ -78,7 +78,7 @@ extern struct inet6_ifaddr *ipv6_get_ifaddr(struct net *net,
int strict);
extern int ipv6_dev_get_saddr(struct net *net,
- struct net_device *dev,
+ const struct net_device *dev,
const struct in6_addr *daddr,
unsigned int srcprefs,
struct in6_addr *saddr);
diff --git a/include/net/netfilter/nf_nat.h b/include/net/netfilter/nf_nat.h
index 1752f13..bd8eea7 100644
--- a/include/net/netfilter/nf_nat.h
+++ b/include/net/netfilter/nf_nat.h
@@ -43,7 +43,9 @@ struct nf_conn_nat {
struct nf_conn *ct;
union nf_conntrack_nat_help help;
#if defined(CONFIG_IP_NF_TARGET_MASQUERADE) || \
- defined(CONFIG_IP_NF_TARGET_MASQUERADE_MODULE)
+ defined(CONFIG_IP_NF_TARGET_MASQUERADE_MODULE) || \
+ defined(CONFIG_IP6_NF_TARGET_MASQUERADE) || \
+ defined(CONFIG_IP6_NF_TARGET_MASQUERADE_MODULE)
int masq_index;
#endif
};
diff --git a/net/ipv4/netfilter/ipt_MASQUERADE.c b/net/ipv4/netfilter/ipt_MASQUERADE.c
index 1c3aa28..5d5d4d1 100644
--- a/net/ipv4/netfilter/ipt_MASQUERADE.c
+++ b/net/ipv4/netfilter/ipt_MASQUERADE.c
@@ -99,7 +99,8 @@ device_cmp(struct nf_conn *i, void *ifindex)
if (!nat)
return 0;
-
+ if (nf_ct_l3num(i) != NFPROTO_IPV4)
+ return 0;
return nat->masq_index == (int)(long)ifindex;
}
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index 7918181..6536404 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1095,7 +1095,7 @@ out:
return ret;
}
-int ipv6_dev_get_saddr(struct net *net, struct net_device *dst_dev,
+int ipv6_dev_get_saddr(struct net *net, const struct net_device *dst_dev,
const struct in6_addr *daddr, unsigned int prefs,
struct in6_addr *saddr)
{
diff --git a/net/ipv6/netfilter/Kconfig b/net/ipv6/netfilter/Kconfig
index b27e0ad..54a5032 100644
--- a/net/ipv6/netfilter/Kconfig
+++ b/net/ipv6/netfilter/Kconfig
@@ -144,6 +144,18 @@ config IP6_NF_TARGET_HL
(e.g. when running oldconfig). It selects
CONFIG_NETFILTER_XT_TARGET_HL.
+config IP6_NF_TARGET_MASQUERADE
+ tristate "MASQUERADE target support"
+ depends on NF_NAT_IPV6
+ help
+ Masquerading is a special case of NAT: all outgoing connections are
+ changed to seem to come from a particular interface's address, and
+ if the interface goes down, those connections are lost. This is
+ only useful for dialup accounts with dynamic IP address (ie. your IP
+ address will be different on next dialup).
+
+ To compile it as a module, choose M here. If unsure, say N.
+
config IP6_NF_FILTER
tristate "Packet filtering"
default m if NETFILTER_ADVANCED=n
diff --git a/net/ipv6/netfilter/Makefile b/net/ipv6/netfilter/Makefile
index 7677937..068bad1 100644
--- a/net/ipv6/netfilter/Makefile
+++ b/net/ipv6/netfilter/Makefile
@@ -34,4 +34,5 @@ obj-$(CONFIG_IP6_NF_MATCH_RPFILTER) += ip6t_rpfilter.o
obj-$(CONFIG_IP6_NF_MATCH_RT) += ip6t_rt.o
# targets
+obj-$(CONFIG_IP6_NF_TARGET_MASQUERADE) += ip6t_MASQUERADE.o
obj-$(CONFIG_IP6_NF_TARGET_REJECT) += ip6t_REJECT.o
diff --git a/net/ipv6/netfilter/ip6t_MASQUERADE.c b/net/ipv6/netfilter/ip6t_MASQUERADE.c
new file mode 100644
index 0000000..60e9053
--- /dev/null
+++ b/net/ipv6/netfilter/ip6t_MASQUERADE.c
@@ -0,0 +1,135 @@
+/*
+ * Copyright (c) 2011 Patrick McHardy <kaber@trash.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * Based on Rusty Russell's IPv6 MASQUERADE target. Development of IPv6
+ * NAT funded by Astaro.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/ipv6.h>
+#include <linux/netfilter.h>
+#include <linux/netfilter_ipv6.h>
+#include <linux/netfilter/x_tables.h>
+#include <net/netfilter/nf_nat.h>
+#include <net/addrconf.h>
+#include <net/ipv6.h>
+
+static unsigned int
+masquerade_tg6(struct sk_buff *skb, const struct xt_action_param *par)
+{
+ const struct nf_nat_range *range = par->targinfo;
+ enum ip_conntrack_info ctinfo;
+ struct in6_addr src;
+ struct nf_conn *ct;
+ struct nf_nat_range newrange;
+
+ ct = nf_ct_get(skb, &ctinfo);
+ NF_CT_ASSERT(ct && (ctinfo == IP_CT_NEW || ctinfo == IP_CT_RELATED ||
+ ctinfo == IP_CT_RELATED_REPLY));
+
+ if (ipv6_dev_get_saddr(dev_net(par->out), par->out,
+ &ipv6_hdr(skb)->daddr, 0, &src) < 0)
+ return NF_DROP;
+
+ nfct_nat(ct)->masq_index = par->out->ifindex;
+
+ newrange.flags = range->flags | NF_NAT_RANGE_MAP_IPS;
+ newrange.min_addr.in6 = src;
+ newrange.max_addr.in6 = src;
+ newrange.min_proto = range->min_proto;
+ newrange.max_proto = range->max_proto;
+
+ return nf_nat_setup_info(ct, &newrange, NF_NAT_MANIP_SRC);
+}
+
+static int masquerade_tg6_checkentry(const struct xt_tgchk_param *par)
+{
+ const struct nf_nat_range *range = par->targinfo;
+
+ if (range->flags & NF_NAT_RANGE_MAP_IPS)
+ return -EINVAL;
+ return 0;
+}
+
+static int device_cmp(struct nf_conn *ct, void *ifindex)
+{
+ const struct nf_conn_nat *nat = nfct_nat(ct);
+
+ if (!nat)
+ return 0;
+ if (nf_ct_l3num(ct) != NFPROTO_IPV6)
+ return 0;
+ return nat->masq_index == (int)(long)ifindex;
+}
+
+static int masq_device_event(struct notifier_block *this,
+ unsigned long event, void *ptr)
+{
+ const struct net_device *dev = ptr;
+ struct net *net = dev_net(dev);
+
+ if (event == NETDEV_DOWN)
+ nf_ct_iterate_cleanup(net, device_cmp,
+ (void *)(long)dev->ifindex);
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block masq_dev_notifier = {
+ .notifier_call = masq_device_event,
+};
+
+static int masq_inet_event(struct notifier_block *this,
+ unsigned long event, void *ptr)
+{
+ struct inet6_ifaddr *ifa = ptr;
+
+ return masq_device_event(this, event, ifa->idev->dev);
+}
+
+static struct notifier_block masq_inet_notifier = {
+ .notifier_call = masq_inet_event,
+};
+
+static struct xt_target masquerade_tg6_reg __read_mostly = {
+ .name = "MASQUERADE",
+ .family = NFPROTO_IPV6,
+ .checkentry = masquerade_tg6_checkentry,
+ .target = masquerade_tg6,
+ .targetsize = sizeof(struct nf_nat_range),
+ .table = "nat",
+ .hooks = 1 << NF_INET_POST_ROUTING,
+ .me = THIS_MODULE,
+};
+
+static int __init masquerade_tg6_init(void)
+{
+ int err;
+
+ err = xt_register_target(&masquerade_tg6_reg);
+ if (err == 0) {
+ register_netdevice_notifier(&masq_dev_notifier);
+ register_inet6addr_notifier(&masq_inet_notifier);
+ }
+
+ return err;
+}
+static void __exit masquerade_tg6_exit(void)
+{
+ unregister_inet6addr_notifier(&masq_inet_notifier);
+ unregister_netdevice_notifier(&masq_dev_notifier);
+ xt_unregister_target(&masquerade_tg6_reg);
+}
+
+module_init(masquerade_tg6_init);
+module_exit(masquerade_tg6_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Patrick McHardy <kaber@trash.net>");
+MODULE_DESCRIPTION("Xtables: automatic address SNAT");
--
1.7.1
^ permalink raw reply related
* [PATCH 01/18] ipv4: fix path MTU discovery with connection tracking
From: Patrick McHardy @ 2012-08-20 3:39 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
IPv4 conntrack defragments incoming packet at the PRE_ROUTING hook and
(in case of forwarded packets) refragments them at POST_ROUTING
independant of the IP_DF flag. Refragmentation uses the dst_mtu() of
the local route without caring about the original fragment sizes,
thereby breaking PMTUD.
This patch fixes this by keeping track of the largest received fragment
with IP_DF set and generates an ICMP fragmentation required error during
refragmentation if that size exceeds the MTU.
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/net/inet_frag.h | 2 ++
include/net/ip.h | 2 ++
net/ipv4/ip_fragment.c | 8 +++++++-
net/ipv4/ip_output.c | 4 +++-
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index 2431cf8..5098ee7 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -29,6 +29,8 @@ struct inet_frag_queue {
#define INET_FRAG_COMPLETE 4
#define INET_FRAG_FIRST_IN 2
#define INET_FRAG_LAST_IN 1
+
+ u16 max_size;
};
#define INETFRAGS_HASHSZ 64
diff --git a/include/net/ip.h b/include/net/ip.h
index 5a5d84d..0707fb9 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -42,6 +42,8 @@ struct inet_skb_parm {
#define IPSKB_XFRM_TRANSFORMED 4
#define IPSKB_FRAG_COMPLETE 8
#define IPSKB_REROUTED 16
+
+ u16 frag_max_size;
};
static inline unsigned int ip_hdrlen(const struct sk_buff *skb)
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index 8d07c97..fa6a12c 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -523,6 +523,10 @@ found:
if (offset == 0)
qp->q.last_in |= INET_FRAG_FIRST_IN;
+ if (ip_hdr(skb)->frag_off & htons(IP_DF) &&
+ skb->len + ihl > qp->q.max_size)
+ qp->q.max_size = skb->len + ihl;
+
if (qp->q.last_in == (INET_FRAG_FIRST_IN | INET_FRAG_LAST_IN) &&
qp->q.meat == qp->q.len)
return ip_frag_reasm(qp, prev, dev);
@@ -646,9 +650,11 @@ static int ip_frag_reasm(struct ipq *qp, struct sk_buff *prev,
head->next = NULL;
head->dev = dev;
head->tstamp = qp->q.stamp;
+ IPCB(head)->frag_max_size = qp->q.max_size;
iph = ip_hdr(head);
- iph->frag_off = 0;
+ /* max_size != 0 implies at least one fragment had IP_DF set */
+ iph->frag_off = qp->q.max_size ? htons(IP_DF) : 0;
iph->tot_len = htons(len);
iph->tos |= ecn;
IP_INC_STATS_BH(net, IPSTATS_MIB_REASMOKS);
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index 147ccc3..95a9d72 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -467,7 +467,9 @@ int ip_fragment(struct sk_buff *skb, int (*output)(struct sk_buff *))
iph = ip_hdr(skb);
- if (unlikely((iph->frag_off & htons(IP_DF)) && !skb->local_df)) {
+ if (unlikely(((iph->frag_off & htons(IP_DF)) && !skb->local_df) ||
+ (IPCB(skb)->frag_max_size &&
+ IPCB(skb)->frag_max_size > dst_mtu(&rt->dst)))) {
IP_INC_STATS(dev_net(dev), IPSTATS_MIB_FRAGFAILS);
icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
htonl(ip_skb_dst_mtu(skb)));
--
1.7.1
^ permalink raw reply related
* [PATCH 18/18] netfilter: ip6tables: add stateless IPv6-to-IPv6 Network Prefix Translation target
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
include/linux/netfilter_ipv6/Kbuild | 1 +
include/linux/netfilter_ipv6/ip6t_NPT.h | 16 +++
net/ipv6/netfilter/Kconfig | 9 ++
net/ipv6/netfilter/Makefile | 1 +
net/ipv6/netfilter/ip6t_NPT.c | 165 +++++++++++++++++++++++++++++++
5 files changed, 192 insertions(+), 0 deletions(-)
create mode 100644 include/linux/netfilter_ipv6/ip6t_NPT.h
create mode 100644 net/ipv6/netfilter/ip6t_NPT.c
diff --git a/include/linux/netfilter_ipv6/Kbuild b/include/linux/netfilter_ipv6/Kbuild
index bd095bc..b88c005 100644
--- a/include/linux/netfilter_ipv6/Kbuild
+++ b/include/linux/netfilter_ipv6/Kbuild
@@ -1,6 +1,7 @@
header-y += ip6_tables.h
header-y += ip6t_HL.h
header-y += ip6t_LOG.h
+header-y += ip6t_NPT.h
header-y += ip6t_REJECT.h
header-y += ip6t_ah.h
header-y += ip6t_frag.h
diff --git a/include/linux/netfilter_ipv6/ip6t_NPT.h b/include/linux/netfilter_ipv6/ip6t_NPT.h
new file mode 100644
index 0000000..f763355
--- /dev/null
+++ b/include/linux/netfilter_ipv6/ip6t_NPT.h
@@ -0,0 +1,16 @@
+#ifndef __NETFILTER_IP6T_NPT
+#define __NETFILTER_IP6T_NPT
+
+#include <linux/types.h>
+#include <linux/netfilter.h>
+
+struct ip6t_npt_tginfo {
+ union nf_inet_addr src_pfx;
+ union nf_inet_addr dst_pfx;
+ __u8 src_pfx_len;
+ __u8 dst_pfx_len;
+ /* Used internally by the kernel */
+ __sum16 adjustment;
+};
+
+#endif /* __NETFILTER_IP6T_NPT */
diff --git a/net/ipv6/netfilter/Kconfig b/net/ipv6/netfilter/Kconfig
index 7bdf73b..3b73254 100644
--- a/net/ipv6/netfilter/Kconfig
+++ b/net/ipv6/netfilter/Kconfig
@@ -177,6 +177,15 @@ config IP6_NF_TARGET_REDIRECT
To compile it as a module, choose M here. If unsure, say N.
+config IP6_NF_TARGET_NPT
+ tristate "NPT (Network Prefix translation) target support"
+ depends on NETFILTER_ADVANCED
+ help
+ This option adds the `SNPT' and `DNPT' target, which perform
+ stateless IPv6-to-IPv6 Network Prefix Translation per RFC 6296.
+
+ To compile it as a module, choose M here. If unsure, say N.
+
config IP6_NF_FILTER
tristate "Packet filtering"
default m if NETFILTER_ADVANCED=n
diff --git a/net/ipv6/netfilter/Makefile b/net/ipv6/netfilter/Makefile
index 0864ce6..5752132 100644
--- a/net/ipv6/netfilter/Makefile
+++ b/net/ipv6/netfilter/Makefile
@@ -36,5 +36,6 @@ obj-$(CONFIG_IP6_NF_MATCH_RT) += ip6t_rt.o
# targets
obj-$(CONFIG_IP6_NF_TARGET_MASQUERADE) += ip6t_MASQUERADE.o
obj-$(CONFIG_IP6_NF_TARGET_NETMAP) += ip6t_NETMAP.o
+obj-$(CONFIG_IP6_NF_TARGET_NPT) += ip6t_NPT.o
obj-$(CONFIG_IP6_NF_TARGET_REDIRECT) += ip6t_REDIRECT.o
obj-$(CONFIG_IP6_NF_TARGET_REJECT) += ip6t_REJECT.o
diff --git a/net/ipv6/netfilter/ip6t_NPT.c b/net/ipv6/netfilter/ip6t_NPT.c
new file mode 100644
index 0000000..e948691
--- /dev/null
+++ b/net/ipv6/netfilter/ip6t_NPT.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2011, 2012 Patrick McHardy <kaber@trash.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/ipv6.h>
+#include <linux/netfilter.h>
+#include <linux/netfilter_ipv6.h>
+#include <linux/netfilter_ipv6/ip6t_NPT.h>
+#include <linux/netfilter/x_tables.h>
+
+static __sum16 csum16_complement(__sum16 a)
+{
+ return (__force __sum16)(0xffff - (__force u16)a);
+}
+
+static __sum16 csum16_add(__sum16 a, __sum16 b)
+{
+ u16 sum;
+
+ sum = (__force u16)a + (__force u16)b;
+ sum += (__force u16)a < (__force u16)b;
+ return (__force __sum16)sum;
+}
+
+static __sum16 csum16_sub(__sum16 a, __sum16 b)
+{
+ return csum16_add(a, csum16_complement(b));
+}
+
+static int ip6t_npt_checkentry(const struct xt_tgchk_param *par)
+{
+ struct ip6t_npt_tginfo *npt = par->targinfo;
+ __sum16 src_sum = 0, dst_sum = 0;
+ unsigned int i;
+
+ if (npt->src_pfx_len > 64 || npt->dst_pfx_len > 64)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(npt->src_pfx.in6.s6_addr16); i++) {
+ src_sum = csum16_add(src_sum,
+ (__force __sum16)npt->src_pfx.in6.s6_addr16[i]);
+ dst_sum = csum16_add(dst_sum,
+ (__force __sum16)npt->dst_pfx.in6.s6_addr16[i]);
+ }
+
+ npt->adjustment = csum16_sub(src_sum, dst_sum);
+ return 0;
+}
+
+static bool ip6t_npt_map_pfx(const struct ip6t_npt_tginfo *npt,
+ struct in6_addr *addr)
+{
+ unsigned int pfx_len;
+ unsigned int i, idx;
+ __be32 mask;
+ __sum16 sum;
+
+ pfx_len = max(npt->src_pfx_len, npt->dst_pfx_len);
+ for (i = 0; i < pfx_len; i += 32) {
+ if (pfx_len - i >= 32)
+ mask = 0;
+ else
+ mask = htonl(~((1 << (pfx_len - i)) - 1));
+
+ idx = i / 32;
+ addr->s6_addr32[idx] &= mask;
+ addr->s6_addr32[idx] |= npt->dst_pfx.in6.s6_addr32[idx];
+ }
+
+ if (pfx_len <= 48)
+ idx = 3;
+ else {
+ for (idx = 4; idx < ARRAY_SIZE(addr->s6_addr16); idx++) {
+ if ((__force __sum16)addr->s6_addr16[idx] !=
+ CSUM_MANGLED_0)
+ break;
+ }
+ if (idx == ARRAY_SIZE(addr->s6_addr16))
+ return false;
+ }
+
+ sum = csum16_add((__force __sum16)addr->s6_addr16[idx],
+ npt->adjustment);
+ if (sum == CSUM_MANGLED_0)
+ sum = 0;
+ *(__force __sum16 *)&addr->s6_addr16[idx] = sum;
+
+ return true;
+}
+
+static unsigned int
+ip6t_snpt_tg(struct sk_buff *skb, const struct xt_action_param *par)
+{
+ const struct ip6t_npt_tginfo *npt = par->targinfo;
+
+ if (!ip6t_npt_map_pfx(npt, &ipv6_hdr(skb)->saddr)) {
+ icmpv6_send(skb, ICMPV6_PARAMPROB, ICMPV6_HDR_FIELD,
+ offsetof(struct ipv6hdr, saddr));
+ return NF_DROP;
+ }
+ return XT_CONTINUE;
+}
+
+static unsigned int
+ip6t_dnpt_tg(struct sk_buff *skb, const struct xt_action_param *par)
+{
+ const struct ip6t_npt_tginfo *npt = par->targinfo;
+
+ if (!ip6t_npt_map_pfx(npt, &ipv6_hdr(skb)->daddr)) {
+ icmpv6_send(skb, ICMPV6_PARAMPROB, ICMPV6_HDR_FIELD,
+ offsetof(struct ipv6hdr, daddr));
+ return NF_DROP;
+ }
+ return XT_CONTINUE;
+}
+
+static struct xt_target ip6t_npt_target_reg[] __read_mostly = {
+ {
+ .name = "SNPT",
+ .target = ip6t_snpt_tg,
+ .targetsize = sizeof(struct ip6t_npt_tginfo),
+ .checkentry = ip6t_npt_checkentry,
+ .family = NFPROTO_IPV6,
+ .hooks = (1 << NF_INET_LOCAL_IN) |
+ (1 << NF_INET_POST_ROUTING),
+ .me = THIS_MODULE,
+ },
+ {
+ .name = "DNPT",
+ .target = ip6t_dnpt_tg,
+ .targetsize = sizeof(struct ip6t_npt_tginfo),
+ .checkentry = ip6t_npt_checkentry,
+ .family = NFPROTO_IPV6,
+ .hooks = (1 << NF_INET_PRE_ROUTING) |
+ (1 << NF_INET_LOCAL_OUT),
+ .me = THIS_MODULE,
+ },
+};
+
+static int __init ip6t_npt_init(void)
+{
+ return xt_register_targets(ip6t_npt_target_reg,
+ ARRAY_SIZE(ip6t_npt_target_reg));
+}
+
+static void __exit ip6t_npt_exit(void)
+{
+ xt_unregister_targets(ip6t_npt_target_reg,
+ ARRAY_SIZE(ip6t_npt_target_reg));
+}
+
+module_init(ip6t_npt_init);
+module_exit(ip6t_npt_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("IPv6-to-IPv6 Network Prefix Translation (RFC 6296)");
+MODULE_AUTHOR("Patrick McHardy <kaber@trash.net>");
+MODULE_ALIAS("ip6t_SNPT");
+MODULE_ALIAS("ip6t_DNPT");
--
1.7.1
^ permalink raw reply related
* [PATCH 13/18] netfilter: nf_nat: support IPv6 in FTP NAT helper
From: Patrick McHardy @ 2012-08-20 3:40 UTC (permalink / raw)
To: netfilter-devel; +Cc: netdev
In-Reply-To: <1345434006-16549-1-git-send-email-kaber@trash.net>
Signed-off-by: Patrick McHardy <kaber@trash.net>
---
net/ipv4/netfilter/Kconfig | 5 -----
net/ipv4/netfilter/Makefile | 1 -
net/netfilter/Kconfig | 5 +++++
net/netfilter/Makefile | 3 +++
net/netfilter/nf_conntrack_ftp.c | 3 +--
net/{ipv4 => }/netfilter/nf_nat_ftp.c | 30 ++++++++++++++++++------------
6 files changed, 27 insertions(+), 20 deletions(-)
rename net/{ipv4 => }/netfilter/nf_nat_ftp.c (82%)
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 33372a1..80272e7 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -220,11 +220,6 @@ config NF_NAT_PROTO_GRE
tristate
depends on NF_NAT_IPV4 && NF_CT_PROTO_GRE
-config NF_NAT_FTP
- tristate
- depends on NF_CONNTRACK && NF_NAT_IPV4
- default NF_NAT_IPV4 && NF_CONNTRACK_FTP
-
config NF_NAT_IRC
tristate
depends on NF_CONNTRACK && NF_NAT_IPV4
diff --git a/net/ipv4/netfilter/Makefile b/net/ipv4/netfilter/Makefile
index 0ea3acc..4d8a4ad 100644
--- a/net/ipv4/netfilter/Makefile
+++ b/net/ipv4/netfilter/Makefile
@@ -21,7 +21,6 @@ obj-$(CONFIG_NF_DEFRAG_IPV4) += nf_defrag_ipv4.o
# NAT helpers (nf_conntrack)
obj-$(CONFIG_NF_NAT_AMANDA) += nf_nat_amanda.o
-obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
obj-$(CONFIG_NF_NAT_H323) += nf_nat_h323.o
obj-$(CONFIG_NF_NAT_IRC) += nf_nat_irc.o
obj-$(CONFIG_NF_NAT_PPTP) += nf_nat_pptp.o
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 91addda..3104494 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -380,6 +380,11 @@ config NF_NAT_PROTO_SCTP
depends on NF_NAT && NF_CT_PROTO_SCTP
select LIBCRC32C
+config NF_NAT_FTP
+ tristate
+ depends on NF_CONNTRACK && NF_NAT
+ default NF_NAT && NF_CONNTRACK_FTP
+
endif # NF_CONNTRACK
# transparent proxy support
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 09c9451..16592b1 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -54,6 +54,9 @@ obj-$(CONFIG_NF_NAT_PROTO_DCCP) += nf_nat_proto_dccp.o
obj-$(CONFIG_NF_NAT_PROTO_UDPLITE) += nf_nat_proto_udplite.o
obj-$(CONFIG_NF_NAT_PROTO_SCTP) += nf_nat_proto_sctp.o
+# NAT helpers
+obj-$(CONFIG_NF_NAT_FTP) += nf_nat_ftp.o
+
# transparent proxy support
obj-$(CONFIG_NETFILTER_TPROXY) += nf_tproxy_core.o
diff --git a/net/netfilter/nf_conntrack_ftp.c b/net/netfilter/nf_conntrack_ftp.c
index c0f4a5b..f8cc26a 100644
--- a/net/netfilter/nf_conntrack_ftp.c
+++ b/net/netfilter/nf_conntrack_ftp.c
@@ -488,8 +488,7 @@ static int help(struct sk_buff *skb,
/* Now, NAT might want to mangle the packet, and register the
* (possibly changed) expectation itself. */
nf_nat_ftp = rcu_dereference(nf_nat_ftp_hook);
- if (nf_nat_ftp && nf_ct_l3num(ct) == NFPROTO_IPV4 &&
- ct->status & IPS_NAT_MASK)
+ if (nf_nat_ftp && ct->status & IPS_NAT_MASK)
ret = nf_nat_ftp(skb, ctinfo, search[dir][i].ftptype,
protoff, matchoff, matchlen, exp);
else {
diff --git a/net/ipv4/netfilter/nf_nat_ftp.c b/net/netfilter/nf_nat_ftp.c
similarity index 82%
rename from net/ipv4/netfilter/nf_nat_ftp.c
rename to net/netfilter/nf_nat_ftp.c
index dd5e387..e839b97 100644
--- a/net/ipv4/netfilter/nf_nat_ftp.c
+++ b/net/netfilter/nf_nat_ftp.c
@@ -10,7 +10,7 @@
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/ip.h>
+#include <linux/inet.h>
#include <linux/tcp.h>
#include <linux/netfilter_ipv4.h>
#include <net/netfilter/nf_nat.h>
@@ -26,22 +26,27 @@ MODULE_ALIAS("ip_nat_ftp");
/* FIXME: Time out? --RR */
-static int nf_nat_ftp_fmt_cmd(enum nf_ct_ftp_type type,
+static int nf_nat_ftp_fmt_cmd(struct nf_conn *ct, enum nf_ct_ftp_type type,
char *buffer, size_t buflen,
- __be32 addr, u16 port)
+ union nf_inet_addr *addr, u16 port)
{
switch (type) {
case NF_CT_FTP_PORT:
case NF_CT_FTP_PASV:
return snprintf(buffer, buflen, "%u,%u,%u,%u,%u,%u",
- ((unsigned char *)&addr)[0],
- ((unsigned char *)&addr)[1],
- ((unsigned char *)&addr)[2],
- ((unsigned char *)&addr)[3],
+ ((unsigned char *)&addr->ip)[0],
+ ((unsigned char *)&addr->ip)[1],
+ ((unsigned char *)&addr->ip)[2],
+ ((unsigned char *)&addr->ip)[3],
port >> 8,
port & 0xFF);
case NF_CT_FTP_EPRT:
- return snprintf(buffer, buflen, "|1|%pI4|%u|", &addr, port);
+ if (nf_ct_l3num(ct) == NFPROTO_IPV4)
+ return snprintf(buffer, buflen, "|1|%pI4|%u|",
+ &addr->ip, port);
+ else
+ return snprintf(buffer, buflen, "|2|%pI6|%u|",
+ &addr->ip6, port);
case NF_CT_FTP_EPSV:
return snprintf(buffer, buflen, "|||%u|", port);
}
@@ -59,17 +64,17 @@ static unsigned int nf_nat_ftp(struct sk_buff *skb,
unsigned int matchlen,
struct nf_conntrack_expect *exp)
{
- __be32 newip;
+ union nf_inet_addr newaddr;
u_int16_t port;
int dir = CTINFO2DIR(ctinfo);
struct nf_conn *ct = exp->master;
- char buffer[sizeof("|1|255.255.255.255|65535|")];
+ char buffer[sizeof("|1||65535|") + INET6_ADDRSTRLEN];
unsigned int buflen;
pr_debug("FTP_NAT: type %i, off %u len %u\n", type, matchoff, matchlen);
/* Connection will come from wherever this packet goes, hence !dir */
- newip = ct->tuplehash[!dir].tuple.dst.u3.ip;
+ newaddr = ct->tuplehash[!dir].tuple.dst.u3;
exp->saved_proto.tcp.port = exp->tuple.dst.u.tcp.port;
exp->dir = !dir;
@@ -94,7 +99,8 @@ static unsigned int nf_nat_ftp(struct sk_buff *skb,
if (port == 0)
return NF_DROP;
- buflen = nf_nat_ftp_fmt_cmd(type, buffer, sizeof(buffer), newip, port);
+ buflen = nf_nat_ftp_fmt_cmd(ct, type, buffer, sizeof(buffer),
+ &newaddr, port);
if (!buflen)
goto out;
--
1.7.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox