[RFC PATCH 0/4] net: Add port mirroring support to the kernel

[RFC PATCH 0/4] net: Add port mirroring support to the kernel

[Neil Horman]

Hey all-
	One of the features that many multi-port Networking devices offer as a
debug facility is port mirroring.  The Linux kernels currently offers this
feature in only a very limited fashion,  Frames can be mangled and resent out
various devices using netfilter rules, but the reach of those rules is limited,
as they are largely protocol specific.  Ebtables is an option, but it applies
only to devices in a bridge configuration.  I'd like to propose a solution which
is protocol agnostic, simple to configure, and always available to debug.  by
doing this we can offer the ability to capture frames on a tertiary device when
the system under test is too heavily loaded to support capturing the data on a
network card without adversely affecting the performance or behavior of the
system

My proposed solution is:

1) Insert tracepoints at the start of the receive path in the network stack
(netif_receive_skb) and at the end of the send path (dev_hard_start_xmit).  By
using tracepoints performance impact can be minimized when mirroring is not in
use, and it lets us avoid adding additional data to the net_device or sk_buff
structures in support of this feature.

2) Add mirroring code which hooks those tracepoints (when activated), and for
each frame sent or received, inspects a table of configured devices.  matching
the sent/received skb to each source skb in the table, the buffer is cloned to
each configured destination device and sent there as well

3) Add a sysfs attribute called mirror_to.  This file holds the names of devices
that the addressed device should mirror skbs to.  eg, this commmand:
echo -n eth1 > /sys/class/net/eth0/mirror_to
would cause all frames sent and received on eth0 to be sent out of eth1
likewise, this command:
echo -n -eth1 > /sys/class/net/eth0/mirror_to
would remove eth1 from the mirroring table

I've tested this code out here and it works fairly well.  I think there lots of
room for optimizations, but this is functional, and IMHO a good stake in the
ground from which incremental improvements can be made.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Neil Horman]

Add net_dev_xmit & net_dev_receive tracepoints

Add tracepoints at the end of the network stack xmit path and the start of the
stack receive path.  Among other uses, these tracepoints can be used to tap the
raw input and output streams for any given network device for the purposes of
mirroring that traffic to other ports.

I should note that this patch relies on the patch here:
http://lkml.org/lkml/2009/12/7/403
As it fixes a bug in the tracing infrastructure that doesn't allow for the
definition of tracepoints from multiple header files in a single C file.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>


 include/trace/events/net.h |   59 +++++++++++++++++++++++++++++++++++++++++++++
 net/core/dev.c             |   10 +++++++
 net/core/net-traces.c      |    4 ++-
 3 files changed, 72 insertions(+), 1 deletion(-)

diff --git a/include/trace/events/net.h b/include/trace/events/net.h
new file mode 100644
index 0000000..09d26cf
--- /dev/null
+++ b/include/trace/events/net.h
@@ -0,0 +1,59 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM net 
+
+#if !defined(_TRACE_NET_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_NET_H
+
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(net_dev_xmit,
+
+	TP_PROTO(struct sk_buff *skb, int rc),
+
+	TP_ARGS(skb, rc),
+
+	TP_STRUCT__entry(
+		__field(	void *,	skbaddr		)
+		__field(	int, 	rc		)
+	),
+
+	TP_fast_assign(
+		__entry->skbaddr = skb;
+		__entry->rc = rc;
+	),
+
+	TP_printk("skbaddr=%p transmitted with rc=%d",
+		__entry->skbaddr, __entry->rc)
+);
+
+TRACE_EVENT(net_dev_receive,
+
+	TP_PROTO(struct sk_buff *skb),
+
+	TP_ARGS(skb),
+
+	TP_STRUCT__entry(
+		__field(	void *, skbaddr 		)
+		__field(	struct net_device *, dev	)
+	),
+
+	TP_fast_assign(
+		__entry->skbaddr = skb;
+		if (skb->dev) {
+			__entry->dev = skb->dev;
+		} else {
+			__entry->dev = NULL;
+		}
+	),
+
+	TP_printk("skbaddr=%p received on interface %s",
+		__entry->skbaddr, (__entry->dev) ? __entry->dev->name : "Unknown")
+);
+
+		
+#endif
+
+#include <trace/define_trace.h>
+
diff --git a/net/core/dev.c b/net/core/dev.c
index c36a17a..303d56f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -129,6 +129,7 @@
 #include <linux/jhash.h>
 #include <linux/random.h>
 #include <trace/events/napi.h>
+#include <trace/events/net.h>
 
 #include "net-sysfs.h"
 
@@ -1828,6 +1829,9 @@ int dev_hard_start_xmit(struct sk_buff *skb, struct net_device *dev,
 			skb_dst_drop(skb);
 
 		rc = ops->ndo_start_xmit(skb, dev);
+
+		trace_net_dev_xmit(skb, rc);
+
 		if (rc == NETDEV_TX_OK)
 			txq_trans_update(txq);
 		/*
@@ -1853,7 +1857,11 @@ gso:
 
 		skb->next = nskb->next;
 		nskb->next = NULL;
+
 		rc = ops->ndo_start_xmit(nskb, dev);
+
+		trace_net_dev_xmit(nskb, rc);
+
 		if (unlikely(rc != NETDEV_TX_OK)) {
 			if (rc & ~NETDEV_TX_MASK)
 				goto out_kfree_gso_skb;
@@ -2428,6 +2436,8 @@ int netif_receive_skb(struct sk_buff *skb)
 	if (!skb->tstamp.tv64)
 		net_timestamp(skb);
 
+	trace_net_dev_receive(skb);
+
 	if (vlan_tx_tag_present(skb) && vlan_hwaccel_do_receive(skb))
 		return NET_RX_SUCCESS;
 
diff --git a/net/core/net-traces.c b/net/core/net-traces.c
index f1e982c..da23ec8 100644
--- a/net/core/net-traces.c
+++ b/net/core/net-traces.c
@@ -25,8 +25,10 @@
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/skb.h>
+#include <trace/events/net.h>
 #include <trace/events/napi.h>
 
 EXPORT_TRACEPOINT_SYMBOL_GPL(kfree_skb);
-
 EXPORT_TRACEPOINT_SYMBOL_GPL(napi_poll);
+EXPORT_TRACEPOINT_SYMBOL_GPL(net_dev_xmit);
+EXPORT_TRACEPOINT_SYMBOL_GPL(net_dev_receive);

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Stephen Hemminger]

On Tue, 15 Dec 2009 11:36:04 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:

> Add net_dev_xmit & net_dev_receive tracepoints
> 
> Add tracepoints at the end of the network stack xmit path and the start of the
> stack receive path.  Among other uses, these tracepoints can be used to tap the
> raw input and output streams for any given network device for the purposes of
> mirroring that traffic to other ports.
> 

There already is a mirroring solution but people don't know how to use it.
Using mirred on ingress qdisc does that.

Why would we want to add another hook to the already complex networking
code path?

https://svn.openfabrics.org/svn/openib/gen2/branches/1.1/src/userspace/ipoibtools/iproute2/doc/actions/mirred-usage

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Neil Horman]

On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> On Tue, 15 Dec 2009 11:36:04 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > Add net_dev_xmit & net_dev_receive tracepoints
> > 
> > Add tracepoints at the end of the network stack xmit path and the start of the
> > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > raw input and output streams for any given network device for the purposes of
> > mirroring that traffic to other ports.
> > 
> 
> There already is a mirroring solution but people don't know how to use it.
> Using mirred on ingress qdisc does that.
> 
I wish that had been easier to find.  That only appears to mirror inbound frames
though, what about outbound frames? can you attach mirred to an outbound qdisc?

> Why would we want to add another hook to the already complex networking
> code path?
> 
Because tracepoints offer a low/zero impact hook (when not in use) that is
extensible to future debug needs.  Given that mirroring is largely a feature to
enable debugging, I thought tracepoints appropriate.

> https://svn.openfabrics.org/svn/openib/gen2/branches/1.1/src/userspace/ipoibtools/iproute2/doc/actions/mirred-usage
Reading over this, I'm still left wondering if this ony mirrors inbound frames.
Is the expectation that outbound frames can be mirrored using some other method?
Or can you just attach this tc action to an outbound qdisc?

Regards
Neil

> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Stephen Hemminger]

On Tue, 15 Dec 2009 12:02:49 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:

> On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > On Tue, 15 Dec 2009 11:36:04 -0500
> > Neil Horman <nhorman@tuxdriver.com> wrote:
> > 
> > > Add net_dev_xmit & net_dev_receive tracepoints
> > > 
> > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > raw input and output streams for any given network device for the purposes of
> > > mirroring that traffic to other ports.
> > > 
> > 
> > There already is a mirroring solution but people don't know how to use it.
> > Using mirred on ingress qdisc does that.
> > 
> I wish that had been easier to find.  That only appears to mirror inbound frames
> though, what about outbound frames? can you attach mirred to an outbound qdisc?

It works for outbound frames as well.

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Neil Horman]

On Tue, Dec 15, 2009 at 09:22:22AM -0800, Stephen Hemminger wrote:
> On Tue, 15 Dec 2009 12:02:49 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > > On Tue, 15 Dec 2009 11:36:04 -0500
> > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > 
> > > > Add net_dev_xmit & net_dev_receive tracepoints
> > > > 
> > > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > > raw input and output streams for any given network device for the purposes of
> > > > mirroring that traffic to other ports.
> > > > 
> > > 
> > > There already is a mirroring solution but people don't know how to use it.
> > > Using mirred on ingress qdisc does that.
> > > 
> > I wish that had been easier to find.  That only appears to mirror inbound frames
> > though, what about outbound frames? can you attach mirred to an outbound qdisc?
> 
> It works for outbound frames as well.
*Sigh*, well there goes a weeks worth of tinkering.  Stupid of me.  Rescinded
Thanks
Neil

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Stephen Hemminger]

On Tue, 15 Dec 2009 12:49:56 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:

> On Tue, Dec 15, 2009 at 09:22:22AM -0800, Stephen Hemminger wrote:
> > On Tue, 15 Dec 2009 12:02:49 -0500
> > Neil Horman <nhorman@tuxdriver.com> wrote:
> > 
> > > On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > > > On Tue, 15 Dec 2009 11:36:04 -0500
> > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > 
> > > > > Add net_dev_xmit & net_dev_receive tracepoints
> > > > > 
> > > > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > > > raw input and output streams for any given network device for the purposes of
> > > > > mirroring that traffic to other ports.
> > > > > 
> > > > 
> > > > There already is a mirroring solution but people don't know how to use it.
> > > > Using mirred on ingress qdisc does that.
> > > > 
> > > I wish that had been easier to find.  That only appears to mirror inbound frames
> > > though, what about outbound frames? can you attach mirred to an outbound qdisc?
> > 
> > It works for outbound frames as well.
> *Sigh*, well there goes a weeks worth of tinkering.  Stupid of me.  Rescinded
> Thanks
> Neil

The tracepoints are still useful, and all is not wasted.

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Neil Horman]

On Tue, Dec 15, 2009 at 09:54:06AM -0800, Stephen Hemminger wrote:
> On Tue, 15 Dec 2009 12:49:56 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Tue, Dec 15, 2009 at 09:22:22AM -0800, Stephen Hemminger wrote:
> > > On Tue, 15 Dec 2009 12:02:49 -0500
> > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > 
> > > > On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > > > > On Tue, 15 Dec 2009 11:36:04 -0500
> > > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > > 
> > > > > > Add net_dev_xmit & net_dev_receive tracepoints
> > > > > > 
> > > > > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > > > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > > > > raw input and output streams for any given network device for the purposes of
> > > > > > mirroring that traffic to other ports.
> > > > > > 
> > > > > 
> > > > > There already is a mirroring solution but people don't know how to use it.
> > > > > Using mirred on ingress qdisc does that.
> > > > > 
> > > > I wish that had been easier to find.  That only appears to mirror inbound frames
> > > > though, what about outbound frames? can you attach mirred to an outbound qdisc?
> > > 
> > > It works for outbound frames as well.
> > *Sigh*, well there goes a weeks worth of tinkering.  Stupid of me.  Rescinded
> > Thanks
> > Neil
> 
> The tracepoints are still useful, and all is not wasted.
Yeah, but it would be nice to have a recognized feature to go with them, so we
don't just have random tracepoints laying about.  Can you think of a good
alternative use for them?
Regards
Neil

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Stephen Hemminger]

On Tue, 15 Dec 2009 15:41:55 -0500
Neil Horman <nhorman@tuxdriver.com> wrote:

> On Tue, Dec 15, 2009 at 09:54:06AM -0800, Stephen Hemminger wrote:
> > On Tue, 15 Dec 2009 12:49:56 -0500
> > Neil Horman <nhorman@tuxdriver.com> wrote:
> > 
> > > On Tue, Dec 15, 2009 at 09:22:22AM -0800, Stephen Hemminger wrote:
> > > > On Tue, 15 Dec 2009 12:02:49 -0500
> > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > 
> > > > > On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > > > > > On Tue, 15 Dec 2009 11:36:04 -0500
> > > > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > > > 
> > > > > > > Add net_dev_xmit & net_dev_receive tracepoints
> > > > > > > 
> > > > > > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > > > > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > > > > > raw input and output streams for any given network device for the purposes of
> > > > > > > mirroring that traffic to other ports.
> > > > > > > 
> > > > > > 
> > > > > > There already is a mirroring solution but people don't know how to use it.
> > > > > > Using mirred on ingress qdisc does that.
> > > > > > 
> > > > > I wish that had been easier to find.  That only appears to mirror inbound frames
> > > > > though, what about outbound frames? can you attach mirred to an outbound qdisc?
> > > > 
> > > > It works for outbound frames as well.
> > > *Sigh*, well there goes a weeks worth of tinkering.  Stupid of me.  Rescinded
> > > Thanks
> > > Neil
> > 
> > The tracepoints are still useful, and all is not wasted.
> Yeah, but it would be nice to have a recognized feature to go with them, so we
> don't just have random tracepoints laying about.  Can you think of a good
> alternative use for them?
> Regards
> Neil
> 

Harald had some stuff with tracking packets through the system that
seems like a natural for using tracepoints.

-- 

Re: [RFC PATCH 1/4] net: port mirroring: add tracepoints to appropriate network paths

[Neil Horman]

On Tue, Dec 15, 2009 at 01:50:53PM -0800, Stephen Hemminger wrote:
> On Tue, 15 Dec 2009 15:41:55 -0500
> Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Tue, Dec 15, 2009 at 09:54:06AM -0800, Stephen Hemminger wrote:
> > > On Tue, 15 Dec 2009 12:49:56 -0500
> > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > 
> > > > On Tue, Dec 15, 2009 at 09:22:22AM -0800, Stephen Hemminger wrote:
> > > > > On Tue, 15 Dec 2009 12:02:49 -0500
> > > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > > 
> > > > > > On Tue, Dec 15, 2009 at 08:44:24AM -0800, Stephen Hemminger wrote:
> > > > > > > On Tue, 15 Dec 2009 11:36:04 -0500
> > > > > > > Neil Horman <nhorman@tuxdriver.com> wrote:
> > > > > > > 
> > > > > > > > Add net_dev_xmit & net_dev_receive tracepoints
> > > > > > > > 
> > > > > > > > Add tracepoints at the end of the network stack xmit path and the start of the
> > > > > > > > stack receive path.  Among other uses, these tracepoints can be used to tap the
> > > > > > > > raw input and output streams for any given network device for the purposes of
> > > > > > > > mirroring that traffic to other ports.
> > > > > > > > 
> > > > > > > 
> > > > > > > There already is a mirroring solution but people don't know how to use it.
> > > > > > > Using mirred on ingress qdisc does that.
> > > > > > > 
> > > > > > I wish that had been easier to find.  That only appears to mirror inbound frames
> > > > > > though, what about outbound frames? can you attach mirred to an outbound qdisc?
> > > > > 
> > > > > It works for outbound frames as well.
> > > > *Sigh*, well there goes a weeks worth of tinkering.  Stupid of me.  Rescinded
> > > > Thanks
> > > > Neil
> > > 
> > > The tracepoints are still useful, and all is not wasted.
> > Yeah, but it would be nice to have a recognized feature to go with them, so we
> > don't just have random tracepoints laying about.  Can you think of a good
> > alternative use for them?
> > Regards
> > Neil
> > 
> 
> Harald had some stuff with tracking packets through the system that
> seems like a natural for using tracepoints.
> 
That might be a good idea.  Some marking to validate skb integrity might be nice
too.  Back to the drawing board.....

Thanks!
Neil

Re: [RFC PATCH 2/4] net: port mirroring: add port mirroring core code to kernel

[Neil Horman]

Add port mirroring module to kernel

This patch adds the code to support port mirroring to the kernel. Implementation
uses tracepoints to catch frames being received and sent on an interface, and
copies those frames to a destination interface.  configuration is preformed via
sysfs, by echoing a target net interface to the source net interfaces mirror_to
attribute

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>


 include/linux/portmirror.h |   50 ++++++
 net/core/mirror.c          |  344 +++++++++++++++++++++++++++++++++++++++++++++
 net/core/net-sysfs.c       |   77 ++++++++++
 3 files changed, 471 insertions(+)

diff --git a/include/linux/portmirror.h b/include/linux/portmirror.h
new file mode 100644
index 0000000..454ca03
--- /dev/null
+++ b/include/linux/portmirror.h
@@ -0,0 +1,50 @@
+/*
+ * Generic Net Device Port Mirroring support 
+ *
+ * Authors:	Neil Horman <nhorman@tuxdriver.com>
+ *
+ *		This program is free software; you can redistribute it and/or
+ *		modify it under the terms of the GNU General Public License
+ *		as published by the Free Software Foundation; either version
+ *		2 of the License, or (at your option) any later version.
+ *
+ */
+#ifndef _LINUX_PORTMIRROR_H
+#define _LINUX_PORTMIRROR_H
+
+#include <linux/netdevice.h>
+
+/*
+ * mirror_dev
+ * Params:
+ *   src_dev: the net_device struct to monitor for frames
+ *   dst_dev: the net_device struct to copy src_dev frames to
+ * Returns:
+ *  0 on success
+ *  -ERRNO on failure
+ */
+extern int mirror_dev(struct net_device *src_dev, struct net_device *dst_dev);
+
+/*
+ * unmirror_dev
+ * Params:
+ *   src_dev: the net_device struct to monitor for frames
+ *   dst_dev: the net_device struct to copy src_dev frames to
+ * Returns:
+ *  0 on success
+ *  -ERRNO on failure
+ */
+extern int unmirror_dev(struct net_device *src_dev, struct net_device *dst_dev);
+
+/*
+ * list_mirror_devs
+ * Params:
+ *   src_dev: the net_device you are monitoring frames from
+ *   rc: a pointer to the number of elements in the returned array,
+ * 	or -ERRNO
+ * Returns:
+ *   an array of string pointers whos length is deonted by rc
+ */
+extern char ** list_mirror_devs(struct net_device *src_dev, int *rc);
+
+#endif	/* _LINUX_PORTMIRROR_H */
diff --git a/net/core/mirror.c b/net/core/mirror.c
new file mode 100644
index 0000000..ced7a05
--- /dev/null
+++ b/net/core/mirror.c
@@ -0,0 +1,344 @@
+/*
+ * Generic port mirroring for Network interfaces 
+ *
+ * Copyright (C) 2009 Neil Horman <nhorman@tuxdriver.com>
+ */
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/string.h>
+#include <linux/if_arp.h>
+#include <linux/inetdevice.h>
+#include <linux/inet.h>
+#include <linux/interrupt.h>
+#include <linux/netpoll.h>
+#include <linux/sched.h>
+#include <linux/delay.h>
+#include <linux/types.h>
+#include <linux/workqueue.h>
+#include <linux/netlink.h>
+#include <linux/net_dropmon.h>
+#include <linux/percpu.h>
+#include <linux/timer.h>
+#include <linux/bitops.h>
+#include <linux/portmirror.h>
+#include <net/netevent.h>
+
+#include <trace/events/net.h>
+
+struct mirror_entry {
+	struct list_head list;
+	struct rcu_head rcu;
+	struct net_device *src_dev;
+	struct net_device *dst_dev;
+};
+
+
+static LIST_HEAD(mirror_list);
+static DEFINE_SPINLOCK(mirror_list_lock);
+
+/*
+ * Helper to free a mirror_entry after rcu
+ * passes a quiescent point
+ */
+static void free_mirror_ent(struct rcu_head *head)
+{
+	struct mirror_entry *n;
+	n = container_of(head, struct mirror_entry, rcu);
+	dev_put(n->src_dev);
+	dev_put(n->dst_dev);
+	kfree(n);
+}
+
+/*
+ * find_mirror_entry
+ * Params:
+ *   src: The source net_device we're looking for
+ *   dst: The destination net_device we're looking for
+ * Returns:
+ *   a struct mirror_entry * on success
+ *   NULL on failure (not found)
+ */
+static struct mirror_entry *find_mirror_entry(struct net_device *src,
+					      struct net_device *dst)
+{
+	struct mirror_entry *ent;
+
+	rcu_read_lock();
+
+	list_for_each_entry_rcu(ent, &mirror_list, list) {
+		if ((ent->src_dev == src) &&
+		    (ent->dst_dev == dst)) {
+			goto found;
+		}
+	}
+	ent = NULL;
+found:
+	rcu_read_unlock();
+	return ent;
+}
+
+
+/*
+ * find_mirror_next_src
+ * Params:
+ *   src: The source net_device we're looking for
+ *   ent: The last found list entry 
+ * Returns:
+ *   a struct mirror_entry * on success
+ *   NULL on failure (not found)
+ * Notes:
+ *   Must be called with rcu_read_lock held
+ */
+static struct mirror_entry *find_mirror_next_src(struct net_device *src,
+						 struct mirror_entry *ent)
+{
+	struct mirror_entry *pos;
+
+	if (unlikely(list_empty(&mirror_list)))
+		return NULL;
+
+	if (ent == NULL) {
+		pos = list_entry(mirror_list.next, struct mirror_entry, list);
+		goto check;
+	} else
+		pos = ent;
+
+	list_for_each_entry_continue_rcu(pos, &mirror_list, list) {
+check:
+		if (pos->src_dev == src)
+			goto found;
+	}
+	pos = NULL;
+found:
+	return pos;
+}
+
+/*
+ * common code for the tracepoint hooks
+ */
+static void mirror_skb(struct sk_buff *skb)
+{
+	struct sk_buff *copy;
+	int rc;
+	struct net_device *src_dev = skb->dev;
+	struct mirror_entry *ent = NULL;
+
+
+
+	
+	if (unlikely(src_dev == NULL)) {
+		if (net_ratelimit())
+			printk(KERN_ERR "No device found in mirror\n");
+		return;
+	}
+
+	/*
+	 * we need to grab the rcu read lock here and
+	 * find every entry in the mirror list that
+	 * lists this source device
+	 */ 
+	rcu_read_lock();
+	while((ent = find_mirror_next_src(src_dev, ent)) != NULL) {
+
+		/*
+		 * We don't want to mirror frames that are bound for
+		 * an interface we are mirroring to, thats an infinite
+		 * loop
+		 */ 
+		if (src_dev == ent->dst_dev)
+			continue;
+
+		/*
+		 * for each found entry, clone the skb
+		 */
+		copy = skb_clone(skb, GFP_ATOMIC);
+
+		/*
+		 * change the skb->dev to the destination device
+		 */
+		copy->dev = ent->dst_dev;		
+
+		rc = dev_queue_xmit(copy);
+	}
+	rcu_read_unlock();
+}
+
+/*
+ * Tracepoint hooks to forward frames to appropriate output devs
+ */
+static void net_tx_hook(struct sk_buff *skb, int rc)
+{
+	/*
+	 * Don't mirror the frame if the original send failed
+	 */
+	if (rc != NETDEV_TX_OK)
+		return;
+	mirror_skb(skb);
+}
+
+static void net_rx_hook(struct sk_buff *skb)
+{
+	mirror_skb(skb);
+}
+
+/*
+ * Helper functions to enable and disable the appropriate tracepoints
+ */
+static int enable_mirror_tracepoints(void)
+{
+	int rc = 0;
+
+	rc |= register_trace_net_dev_xmit(net_tx_hook);
+	rc |= register_trace_net_dev_receive(net_rx_hook);
+
+	if (rc)
+		rc = -EFAULT;
+
+	return rc;
+}
+
+static void disable_mirror_tracepoints(void)
+{
+
+	unregister_trace_net_dev_xmit(net_tx_hook);
+	unregister_trace_net_dev_receive(net_rx_hook);
+	tracepoint_synchronize_unregister();
+}
+
+/*
+ * mirror_dev
+ * Params:
+ *   src_dev: the net_device struct to monitor for frames
+ *   dst_dev: the net_device struct to copy src_dev frames to
+ * Returns:
+ *  0 on success
+ *  -ERRNO on failure
+ */
+int mirror_dev(struct net_device *src_dev, struct net_device *dst_dev)
+{
+	struct mirror_entry *newm;
+	int rc = -EEXIST; 
+
+
+
+	if (find_mirror_entry(src_dev, dst_dev))
+		goto out;
+
+	rc = -ENOMEM;
+	newm = kmalloc(sizeof(struct mirror_entry), GFP_KERNEL);
+	if (!newm)
+		goto out;
+
+	if (list_empty(&mirror_list))
+		rc = enable_mirror_tracepoints();
+	if (rc)
+		goto out_free;
+
+
+	newm->src_dev = src_dev;
+	newm->dst_dev = dst_dev;
+	dev_hold(src_dev);
+	dev_hold(dst_dev);
+	printk(KERN_CRIT "Adding src dev %p and dst dev %p\n",src_dev, dst_dev);
+	spin_lock(&mirror_list_lock);
+	list_add_rcu(&newm->list, &mirror_list);
+	spin_unlock(&mirror_list_lock);
+
+	rc = 0;	
+
+
+out:
+	return rc;
+
+out_free:
+	kfree(newm);
+	goto out;
+	
+}
+EXPORT_SYMBOL_GPL(mirror_dev);
+
+/*
+ * unmirror_dev
+ * Params:
+ *   src_dev: the net_device struct to monitor for frames
+ *   dst_dev: the net_device struct to copy src_dev frames to
+ * Returns:
+ *  0 on success
+ *  -ERRNO on failure
+ */
+int unmirror_dev(struct net_device *src_dev, struct net_device *dst_dev)
+{
+	struct mirror_entry *ent;
+	int rc = -ENOENT;
+
+	spin_lock(&mirror_list_lock);
+
+	ent = find_mirror_entry(src_dev, dst_dev);
+
+	if (!ent)
+		goto out_unlock;
+
+	rc = 0;
+
+	list_del_rcu(&ent->list);	
+	call_rcu(&ent->rcu, free_mirror_ent);
+
+	if (list_empty(&mirror_list))
+		disable_mirror_tracepoints();
+		
+out_unlock:
+	spin_unlock(&mirror_list_lock);
+	return rc;
+}
+EXPORT_SYMBOL_GPL(unmirror_dev);
+
+/*
+ * list_mirror_devs
+ * Params:
+ *   src_dev: the net_device you are monitoring frames from
+ *   rc:  pointer to an integer for the return code
+ * Returns:
+ *	rc holds the number of elements in the array or -ERRNO
+ *	on failure
+ */
+char ** list_mirror_devs(struct net_device *src_dev, int *rc)
+{
+	struct mirror_entry *ent;
+	char **names = NULL;
+	int count = 0;
+	int i = 0;
+
+	*rc = -ENOENT;
+
+	rcu_read_lock();
+
+rescan:
+	list_for_each_entry_rcu(ent, &mirror_list, list) {
+		if (ent->src_dev == src_dev) { 
+			if (*rc == -ENOENT)
+				count++;
+			else
+				names[i++] = ent->dst_dev->name;
+		}
+	}
+
+	if (*rc == count)
+		goto out;
+	else if (count != 0) {
+		*rc = count;
+		names = kmalloc(sizeof(char *)*count, GFP_KERNEL);
+		if (names == NULL) {
+			*rc = -ENOMEM;
+			goto out;
+		}
+		goto rescan;
+	}
+
+out:
+	rcu_read_unlock();
+	return names;
+
+}
+EXPORT_SYMBOL_GPL(list_mirror_devs);
+
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index fbc1c74..c20415d 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -16,6 +16,7 @@
 #include <net/sock.h>
 #include <linux/rtnetlink.h>
 #include <linux/wireless.h>
+#include <linux/portmirror.h>
 #include <net/wext.h>
 
 #include "net-sysfs.h"
@@ -289,6 +290,78 @@ static ssize_t show_ifalias(struct device *dev,
 	return ret;
 }
 
+#ifdef CONFIG_NET_PORT_MIRRORING
+static ssize_t show_port_mirror(struct device *dev,
+			    struct device_attribute *attr, char *buf)
+{
+	int rc;
+	struct net_device  *src_dev = to_net_dev(dev);
+	char **names = NULL;
+	int i;
+	ssize_t ret = 0;
+	
+	names = list_mirror_devs(src_dev, &rc);
+	if (rc < 0)
+		return ret;
+
+	for(i=0 ; i<rc; i++) {
+		ret += sprintf(&buf[ret], "%s ", names[i]);
+	}
+	ret += sprintf(&buf[ret], "\n");
+	/*
+	 * need to free the names passed to us here
+	 */
+	kfree(names);
+	return ret;
+	
+}
+
+static ssize_t store_port_mirror(struct device *dev, struct device_attribute *attr,
+			    const char *buf, size_t len)
+{
+	int rc = -EINVAL;
+	struct net_device *dst_dev;
+	struct net_device *src_dev = to_net_dev(dev);
+	char removing = 0;
+
+	/*
+	 * Abort if theres not enough data
+	 */
+	if (len <= 1)
+		goto out;
+
+	/*
+	 * check to see if we're doing a removal
+	 * and get the appropriate destination device
+	 */
+	printk(KERN_CRIT "buf[0] is %c (0x%x)\n", buf[0], buf[0]);
+	if (buf[0] == '-') {
+		removing = 1;
+		printk(KERN_CRIT "REMOVING DEV %s\n", &buf[1]);
+		dst_dev = dev_get_by_name(dev_net(src_dev), &buf[1]);
+	} else
+		dst_dev = dev_get_by_name(dev_net(src_dev), buf);
+
+	rc = -ENOENT;
+	if (!dst_dev)
+		goto out;
+
+	/*
+	 * now we just need to insert or remove the mirror device
+	 */
+	if (removing)
+		rc = unmirror_dev(src_dev, dst_dev);
+	else
+		rc = mirror_dev(src_dev, dst_dev);
+
+	dev_put(dst_dev);
+
+	rc = (rc == 0) ? len : rc;
+out:
+	return rc;
+}
+#endif
+
 static struct device_attribute net_class_attributes[] = {
 	__ATTR(addr_len, S_IRUGO, show_addr_len, NULL),
 	__ATTR(dev_id, S_IRUGO, show_dev_id, NULL),
@@ -309,6 +382,10 @@ static struct device_attribute net_class_attributes[] = {
 	__ATTR(flags, S_IRUGO | S_IWUSR, show_flags, store_flags),
 	__ATTR(tx_queue_len, S_IRUGO | S_IWUSR, show_tx_queue_len,
 	       store_tx_queue_len),
+#ifdef CONFIG_NET_PORT_MIRRORING
+	__ATTR(mirror_to, S_IRUGO | S_IWUSR, show_port_mirror,
+	       store_port_mirror),
+#endif
 	{}
 };
 

Re: [RFC PATCH 3/4] net: port mirroring: add config options to enable port mirroring

[Neil Horman]

Add configuration to enable port mirroring feature.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>


 Kconfig       |   14 ++++++++++++++
 core/Makefile |    1 +
 2 files changed, 15 insertions(+)

diff --git a/net/Kconfig b/net/Kconfig
index 041c35e..b94f950 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -246,6 +246,20 @@ config NET_DROP_MONITOR
 	just checking the various proc files and other utilities for
 	drop statistics, say N here.
 
+config NET_PORT_MIRRORING
+	boolean "Network port mirroring"
+	depends on EXPERIMENTAL && TRACEPOINTS
+	---help---
+	When debugging network connectivity problems, it can be useful (and
+	sometimes necessecary) to record frames from an interface without using
+	any of the systems under test (e.g. if running a capture on the system
+	in question affects it in such a way that the problem then cannot be 
+	reproduced).  Port mirroring allows for the traffic to and from a given
+	port to be resent unaltered to a secondary port, so that a tertiary
+	system can record the traffic).  If you need to be able to see monitor
+	network traffic to/from a system without running software on the system
+	in question, say Y here.
+
 endmenu
 
 endmenu
diff --git a/net/core/Makefile b/net/core/Makefile
index 08791ac..305d88d 100644
--- a/net/core/Makefile
+++ b/net/core/Makefile
@@ -18,4 +18,5 @@ obj-$(CONFIG_NET_DMA) += user_dma.o
 obj-$(CONFIG_FIB_RULES) += fib_rules.o
 obj-$(CONFIG_TRACEPOINTS) += net-traces.o
 obj-$(CONFIG_NET_DROP_MONITOR) += drop_monitor.o
+obj-$(CONFIG_NET_PORT_MIRRORING) += mirror.o
 

Re: [RFC PATCH 4/4] net: port mirroring: Add maintainer

[Neil Horman]

Add myself as maintainer for port mirroring.

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>

diff --git a/MAINTAINERS b/MAINTAINERS
index ea781c1..3f73a90 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3737,6 +3737,14 @@ S:	Maintained
 W:	https://fedorahosted.org/dropwatch/
 F:	net/core/drop_monitor.c
 
+NETWORK PORT MIRRORING
+M:	Neil Horman <nhorman@tuxdriver.com>
+L:	netdev@vger.kernel.org
+S:	Maintained
+F:	net/core/mirror.c
+F:	include/linux/portmirror.h
+F:	net/core/net-sysfs.c
+
 NETWORKING [GENERAL]
 M:	"David S. Miller" <davem@davemloft.net>
 L:	netdev@vger.kernel.org