Proposed interface for per-packet mesh-ttl

Proposed interface for per-packet mesh-ttl

[Javier Cardona]

David Woodhouse suggested that this list is a more appropriate forum
for my message...

---------- Forwarded message ----------
From: Javier Cardona <javier@cozybit.com>
Date: Jul 3, 2007 11:49 AM
Subject: Proposed interface for per-packet mesh-ttl
To: libertas-dev@lists.infradead.org


Libertas-dev,

I'm currently working on per-packet mesh ttl.  My plan is to register
new mesh sockopts through netfilter.  The user interface will be:

#include <sys/types.h>
#include <sys/socket.h>

/* in mesh.h ? */
#define MESH_SO_SET_TTL         77
#define MESH_SO_GET_TTL         77

int main()
{
    int sock;
    int optlen;
    unsigned char ttl;

    ttl = 7;
    optlen = sizeof(ttl);

    sock = socket (PF_INET, SOCK_STREAM, 0);
    setsockopt(sock, SOL_IP, MESH_SO_SET_TTL, &ttl, optlen);
    getsockopt(sock, SOL_IP, MESH_SO_GET_TTL, &ttl, &optlen);

    /* from here on, all traffic from sock will be sent to the mesh
with ttl=7 */

    return 0;
}


Pros:

* it is non-intrusive (only need to reserve the socket option values,
no other changes to the net stack are needed)
* runtime configurable (this options may be supported only when a
mesh_opts module is loaded).
* familiar and intuitive (at least to me :)

Cons:

* netfilter only has hooks for IPv4 and IPv6.  If we want to make mesh
parameters configurable at other layers (e.g. packet) we'll have to
create the hooks.
* It is debatable whether a mesh option should be set at SOL_IP, as it
is a layer 2 protocol.    We do want (some) mesh options to be applied
to flows a that level, and the {g,s}etsockopt interface does not
support setting lower layer options to higher layer sockets.

Speak up if you would like to see this done in a different way.

Cheers,

Javier

Re: Proposed interface for per-packet mesh-ttl

[Dan Williams]

On Tue, 2007-07-03 at 12:29 -0700, Javier Cardona wrote:
> David Woodhouse suggested that this list is a more appropriate forum
> for my message...

Attached is Javier's proposed patch for this.  Please flame away.

Dan

-----------------------------------
Resent per Dan's request.

Support for using setsockpt() to change the mesh-ttl on a network flow, i.e.

<snip>
    sock = socket (PF_INET, SOCK_STREAM, 0);
    setsockopt(sock, SOL_IP, MESH_SO_SET_TTL, &ttl, optlen);
    ttl = 0;
    getsockopt(sock, SOL_IP, MESH_SO_GET_TTL, &ttl, &optlen);
</snip>

Signed-off-by: Javier Cardona <javier@cozybit.com>
---
 drivers/net/wireless/Kconfig              |    7 +
 drivers/net/wireless/libertas/Makefile    |    1 +
 drivers/net/wireless/libertas/decl.h      |    3 +
 drivers/net/wireless/libertas/hostcmd.h   |    6 +
 drivers/net/wireless/libertas/mesh_opts.c |  174 +++++++++++++++++++++++++++++
 drivers/net/wireless/libertas/mesh_opts.h |    5 +
 drivers/net/wireless/libertas/tx.c        |   46 +++++++-
 include/linux/in.h                        |    3 +
 8 files changed, 241 insertions(+), 4 deletions(-)

diff --git a/drivers/net/wireless/Kconfig b/drivers/net/wireless/Kconfig
index 1146f3d..f4123c3 100644
--- a/drivers/net/wireless/Kconfig
+++ b/drivers/net/wireless/Kconfig
@@ -294,6 +294,13 @@ config LIBERTAS_USB
 	---help---
 	  A driver for Marvell Libertas 8388 USB devices.
 
+config LIBERTAS_MESH_OPTS
+	tristate "Mesh Configuration Options for Libertas USB 802.11b/g cards"
+	depends on LIBERTAS_USB && NETFILTER
+	---help---
+	  This module enables the configuration of mesh parameters on a
+	  per-socket basis, via setsockopt() calls. 
+
 config LIBERTAS_DEBUG
 	bool "Enable full debugging output in the Libertas module."
 	depends on LIBERTAS
diff --git a/drivers/net/wireless/libertas/Makefile b/drivers/net/wireless/libertas/Makefile
index 71c5a25..a31d4f7 100644
--- a/drivers/net/wireless/libertas/Makefile
+++ b/drivers/net/wireless/libertas/Makefile
@@ -18,3 +18,4 @@ usb8xxx-objs += if_usb.o
 
 obj-$(CONFIG_LIBERTAS)     += libertas.o
 obj-$(CONFIG_LIBERTAS_USB) += usb8xxx.o
+obj-m			   += mesh_opts.o
diff --git a/drivers/net/wireless/libertas/decl.h b/drivers/net/wireless/libertas/decl.h
index 4d553da..2cbc137 100644
--- a/drivers/net/wireless/libertas/decl.h
+++ b/drivers/net/wireless/libertas/decl.h
@@ -14,6 +14,7 @@
 struct wlan_private;
 struct sk_buff;
 struct net_device;
+struct mesh_options;
 
 extern char *libertas_fw_name;
 
@@ -86,6 +87,8 @@ int libertas_activate_card(wlan_private *priv, char *fw_name);
 int libertas_remove_card(wlan_private *priv);
 int libertas_add_mesh(wlan_private *priv, struct device *dev);
 void libertas_remove_mesh(wlan_private *priv);
+int libertas_register_mesh_opts(struct mesh_options *);
+int libertas_unregister_mesh_opts(struct mesh_options *);
 
 
 #endif				/* _WLAN_DECL_H_ */
diff --git a/drivers/net/wireless/libertas/hostcmd.h b/drivers/net/wireless/libertas/hostcmd.h
index 0f67cba..bc86ed0 100644
--- a/drivers/net/wireless/libertas/hostcmd.h
+++ b/drivers/net/wireless/libertas/hostcmd.h
@@ -34,6 +34,12 @@ struct txpd {
 	u8 reserved1;
 };
 
+struct txpd_mesh {
+	__le16 reserved;
+	/* mesh ttl */
+	u8 ttl;
+} __attribute__ ((packed));
+
 /* RxPD Descriptor */
 struct rxpd {
 	/* Current Rx packet status */
diff --git a/drivers/net/wireless/libertas/mesh_opts.c b/drivers/net/wireless/libertas/mesh_opts.c
new file mode 100644
index 0000000..118eaed
--- /dev/null
+++ b/drivers/net/wireless/libertas/mesh_opts.c
@@ -0,0 +1,174 @@
+/*
+ * mesh_opts
+ *
+ * Author: Javier Cardona <javier@cozybit.com>
+ * Copyright: Marvell Semiconductors Inc., 2007
+ *
+ * Apply mesh-layer specific configuration to network flows.  Currently this
+ * only supports the mesh TTL parameter.
+ *
+ * Users call setsockopt on sockets to configure mesh parameters.  This module
+ * maintains a list of sockets (mesh_sks) that have different mesh parameters
+ * than the per-interface defaults.  The driver will modify the mesh
+ * configuration for each outgoing frame that belongs to one of the sockets in
+ * the mesh_sks list.
+ */
+
+#include <linux/module.h>
+#include <linux/list.h>
+#include <linux/net.h>
+#include <linux/in.h>
+#include <linux/netfilter.h>
+#include <linux/netfilter_ipv4.h>
+#include <linux/netfilter_ipv6.h>
+#include <linux/spinlock.h>
+#include <net/sock.h>
+
+#include <asm/uaccess.h>
+
+#include "mesh_opts.h"
+
+#define MESH_SO_BASE_CTL 	MESH_SO_SET_TTL
+
+static struct list_head mesh_sks = LIST_HEAD_INIT(mesh_sks);
+static DEFINE_RWLOCK(mesh_sks_lock);
+
+struct mesh_sock {
+	struct list_head list;
+
+	struct sock *sk;
+	unsigned char ttl;
+	void (*orig_sk_destruct) (struct sock *sk);
+};
+
+static struct mesh_sock * lookup_socket(struct sock *sk)
+{
+	struct mesh_sock *mesh_sk;
+	struct mesh_sock *found_sk = NULL;
+
+	read_lock(&mesh_sks_lock);
+	list_for_each_entry(mesh_sk, &mesh_sks, list) 
+		if (mesh_sk->sk == sk) {
+			found_sk = mesh_sk;
+			break;
+		}
+	read_unlock(&mesh_sks_lock);
+	return found_sk;
+}
+
+static void mesh_sk_destruct(struct sock *sk)
+{
+	struct mesh_sock *mesh_sk;
+	void (*orig_sk_destruct) (struct sock *sk);
+
+	mesh_sk = lookup_socket(sk);
+
+	if (mesh_sk) {
+		orig_sk_destruct = mesh_sk->orig_sk_destruct;
+		write_lock(&mesh_sks_lock);
+		list_del(&mesh_sk->list);
+		write_unlock(&mesh_sks_lock);
+		kfree(mesh_sk);
+		(*orig_sk_destruct)(sk);
+	}
+}
+
+static int do_mesh_set_mesh_ttl(struct sock *sk, void __user *user, unsigned int len)
+{
+	struct mesh_sock *mesh_sk;
+	unsigned char ttl;
+
+
+	if (len) {
+		if (get_user(ttl, (unsigned char *) user))
+			return -EFAULT;
+	} else
+		return -EINVAL;
+
+	mesh_sk = (struct mesh_sock*) kmalloc(sizeof(struct mesh_sock), GFP_KERNEL);
+	mesh_sk->ttl = ttl;
+	mesh_sk->sk = sk;
+	mesh_sk->orig_sk_destruct = sk->sk_destruct;
+	sk->sk_destruct = mesh_sk_destruct;
+	write_lock(&mesh_sks_lock);
+	list_add(&mesh_sk->list, &mesh_sks);
+	write_unlock(&mesh_sks_lock);
+
+	return 0;
+}
+
+static int do_mesh_get_mesh_ttl(struct sock *sk, void __user *user, int *len)
+{
+	struct mesh_sock *mesh_sk;
+	int rc = 0;
+
+	if ((mesh_sk = lookup_socket(sk)) == NULL)
+		return -ENODATA;	
+
+	if (put_user(mesh_sk->ttl, (unsigned char*) user))
+		return -EFAULT;
+
+	/* netfilter wrapper does the copy to user of len */
+	*len = sizeof(unsigned char);
+
+	return rc;
+}
+
+static int do_mesh_set_ctl(struct sock *sk, int optval, void __user *user,
+		unsigned int len) 
+{
+	return do_mesh_set_mesh_ttl(sk, user, len);
+}
+
+static int do_mesh_get_ctl(struct sock *sk, int optval, void __user *user, 
+		int *len) 
+{
+	return do_mesh_get_mesh_ttl(sk, user, len);
+}
+
+static unsigned char get_sock_mesh_ttl(struct sock *sk)
+{
+	struct mesh_sock *mesh_sk;
+
+	mesh_sk = lookup_socket(sk);
+
+	/* zero ttl results in using the network interface default */
+	return mesh_sk ? mesh_sk->ttl : 0;	
+}
+
+static struct mesh_options mesh_opts =
+{
+	.get_sock_ttl = get_sock_mesh_ttl,
+};
+
+static struct nf_sockopt_ops mesh_sockopt_ops =
+{
+	.pf		= PF_INET,
+	.set_optmin     = MESH_SO_BASE_CTL,
+	.set_optmax     = MESH_SO_BASE_CTL + 1,
+	.set		= do_mesh_set_ctl,
+	.get_optmin     = MESH_SO_BASE_CTL,
+	.get_optmax     = MESH_SO_BASE_CTL + 1,
+	.get		= do_mesh_get_ctl,
+};
+
+static int __init mesh_opts_init(void)
+{
+	int ret;
+
+	if ((ret = nf_register_sockopt(&mesh_sockopt_ops)) < 0) {
+		return ret;
+	}
+	libertas_register_mesh_opts(&mesh_opts);
+	return ret;
+}
+
+static void __exit mesh_opts_fini(void)
+{
+	nf_unregister_sockopt(&mesh_sockopt_ops);
+	libertas_unregister_mesh_opts(&mesh_opts);
+}
+
+module_init(mesh_opts_init);
+module_exit(mesh_opts_fini);
+MODULE_LICENSE("GPL");
diff --git a/drivers/net/wireless/libertas/mesh_opts.h b/drivers/net/wireless/libertas/mesh_opts.h
new file mode 100644
index 0000000..cd18fb1
--- /dev/null
+++ b/drivers/net/wireless/libertas/mesh_opts.h
@@ -0,0 +1,5 @@
+
+struct mesh_options {
+	unsigned char (*get_sock_ttl)(struct sock*);
+};
+
diff --git a/drivers/net/wireless/libertas/tx.c b/drivers/net/wireless/libertas/tx.c
index 194f033..497c227 100644
--- a/drivers/net/wireless/libertas/tx.c
+++ b/drivers/net/wireless/libertas/tx.c
@@ -9,6 +9,10 @@
 #include "defs.h"
 #include "dev.h"
 #include "wext.h"
+#include "mesh_opts.h"
+
+
+static struct mesh_options mesh_opts = { NULL };
 
 /**
  *  @brief This function converts Tx/Rx rates from IEEE80211_RADIOTAP_RATE 
@@ -122,15 +126,34 @@ static int SendSinglePacket(wlan_private * priv, struct sk_buff *skb)
 	lbs_dbg_hex("txpd", (u8 *) plocaltxpd, sizeof(struct txpd));
 	
 	if (IS_MESH_FRAME(skb)) {
-		plocaltxpd->tx_control |= TxPD_MESH_FRAME;
+		struct txpd_mesh msh_txpd;
+ 		plocaltxpd->tx_control |= cpu_to_le32(TxPD_MESH_FRAME);
+
+		memset(&msh_txpd, 0, sizeof(msh_txpd));
+
+		if (mesh_opts.get_sock_ttl)
+			msh_txpd.ttl = (*mesh_opts.get_sock_ttl)(skb->sk);
+
+		/* ttl of zero is never sent to the hardware */
+		if (msh_txpd.ttl) {
+			/* make room for the mesh descriptor that follows */
+			plocaltxpd->tx_packet_location = cpu_to_le32(
+				sizeof(struct txpd) + sizeof(msh_txpd));
+
+			memcpy(ptr + sizeof(struct txpd), &msh_txpd, 
+					sizeof(msh_txpd));
+			lbs_dbg_hex("txpd_mesh", (u8 *) &msh_txpd, 
+					sizeof(msh_txpd));
+		}
 	}
 
 	memcpy(ptr, plocaltxpd, sizeof(struct txpd));
 
-	ptr += sizeof(struct txpd);
+	ptr += le32_to_cpu(plocaltxpd->tx_packet_location);
 
-	lbs_dbg_hex("Tx Data", (u8 *) p802x_hdr, plocaltxpd->tx_packet_length);
-	memcpy(ptr, p802x_hdr, plocaltxpd->tx_packet_length);
+	lbs_dbg_hex("Tx Data", (u8 *) p802x_hdr, 
+			le16_to_cpu(plocaltxpd->tx_packet_length));
+	memcpy(ptr, p802x_hdr, le16_to_cpu(plocaltxpd->tx_packet_length));
 	ret = priv->hw_host_to_card(priv, MVMS_DAT,
 		priv->adapter->tmptxbuf,
 		plocaltxpd->tx_packet_length +
@@ -287,4 +310,19 @@ void libertas_send_tx_feedback(wlan_private * priv)
 		netif_wake_queue(priv->mesh_dev);
 	}
 }
+
+int libertas_register_mesh_opts(struct mesh_options *opts)
+{
+	mesh_opts.get_sock_ttl = opts->get_sock_ttl;
+	return 0;
+}
+
+int libertas_unregister_mesh_opts(struct mesh_options *opts)
+{
+	mesh_opts.get_sock_ttl = NULL;
+	return 0;
+}
+
+EXPORT_SYMBOL_GPL(libertas_register_mesh_opts);
+EXPORT_SYMBOL_GPL(libertas_unregister_mesh_opts);
 EXPORT_SYMBOL_GPL(libertas_send_tx_feedback);
diff --git a/include/linux/in.h b/include/linux/in.h
index 1912e7c..a69e2ab 100644
--- a/include/linux/in.h
+++ b/include/linux/in.h
@@ -102,6 +102,9 @@ struct in_addr {
 #define MCAST_LEAVE_SOURCE_GROUP	47
 #define MCAST_MSFILTER			48
 
+#define MESH_SO_SET_TTL 		50
+#define MESH_SO_GET_TTL 		50	
+
 #define MCAST_EXCLUDE	0
 #define MCAST_INCLUDE	1

Re: Proposed interface for per-packet mesh-ttl

[Christoph Hellwig]

On Tue, Jul 03, 2007 at 12:29:24PM -0700, Javier Cardona wrote:
> I'm currently working on per-packet mesh ttl.  My plan is to register
> new mesh sockopts through netfilter.  The user interface will be:

NACK.  Drivers should never add sockopt, and drivers should not abuse
netfilter hooks.

Re: Proposed interface for per-packet mesh-ttl

[Dan Williams]

On Fri, 2007-07-27 at 20:56 +0100, Christoph Hellwig wrote:
> On Tue, Jul 03, 2007 at 12:29:24PM -0700, Javier Cardona wrote:
> > I'm currently working on per-packet mesh ttl.  My plan is to register
> > new mesh sockopts through netfilter.  The user interface will be:
> 
> NACK.  Drivers should never add sockopt, and drivers should not abuse
> netfilter hooks.

Do you have a suggestion for a better way to approach per-packet options
that userspace programs can set?

Thanks,
Dan

Re: Proposed interface for per-packet mesh-ttl

[Stephen Hemminger]

On Fri, 27 Jul 2007 18:22:22 -0400
Dan Williams <dcbw@redhat.com> wrote:

> On Fri, 2007-07-27 at 20:56 +0100, Christoph Hellwig wrote:
> > On Tue, Jul 03, 2007 at 12:29:24PM -0700, Javier Cardona wrote:
> > > I'm currently working on per-packet mesh ttl.  My plan is to register
> > > new mesh sockopts through netfilter.  The user interface will be:
> > 
> > NACK.  Drivers should never add sockopt, and drivers should not abuse
> > netfilter hooks.
> 
> Do you have a suggestion for a better way to approach per-packet options
> that userspace programs can set?
> 
> Thanks,
> Dan

In this case perhaps you can have a table that maps skb->priority to
mesh ttl? priorty can already by handled by existing setsockopt calls,
and modified by netfilter and QoS managements.

Re: Proposed interface for per-packet mesh-ttl

[Javier Cardona]

Hi Stephen,

On 7/27/07, Stephen Hemminger <shemminger@linux-foundation.org> wrote:
> In this case perhaps you can have a table that maps skb->priority to
> mesh ttl? priorty can already by handled by existing setsockopt calls,
> and modified by netfilter and QoS managements.

Thanks for the feedback.  IMHO overloading the [SOL_SOCKET]
SO_PRIORITY  sockoption for something as different as changing the
mesh-ttl is quite a stretch.   But I admit that it meets the driver
requirements of "never add sockopt" and "not [ab]use netfilter hooks".
 If that's acceptable to all, I'll modify the patch in that direction.

Cheers,

Javier

Re: Proposed interface for per-packet mesh-ttl

[Stephen Hemminger]

On Mon, 30 Jul 2007 13:37:20 -0700
"Javier Cardona" <javier@cozybit.com> wrote:

> Hi Stephen,
> 
> On 7/27/07, Stephen Hemminger <shemminger@linux-foundation.org> wrote:
> > In this case perhaps you can have a table that maps skb->priority to
> > mesh ttl? priorty can already by handled by existing setsockopt calls,
> > and modified by netfilter and QoS managements.
> 
> Thanks for the feedback.  IMHO overloading the [SOL_SOCKET]
> SO_PRIORITY  sockoption for something as different as changing the
> mesh-ttl is quite a stretch.   But I admit that it meets the driver
> requirements of "never add sockopt" and "not [ab]use netfilter hooks".
>  If that's acceptable to all, I'll modify the patch in that direction.
> 
> Cheers,
> 
> Javier

Take it as idea not a proscription.  You can use TC rules to rewrite
the priority.  It is kind of like how priority is being mapped to TOS already
in softmac. Alternatively, the TTL can be set with sockopt, but it is a IP so
it would need an IP ttl to mesh mapping. The fundamental thing is to try
and avoid topology specific options bleeding all the way up the socket layer,
especially since the network layer is involved and may need to multipath.

Re: Proposed interface for per-packet mesh-ttl

[Luis Carlos Cobo]

On 7/30/07, Stephen Hemminger <shemminger@linux-foundation.org> wrote:
> it would need an IP ttl to mesh mapping. The fundamental thing is to try
> and avoid topology specific options bleeding all the way up the socket layer,
> especially since the network layer is involved and may need to multipath.

I think the cleanest way would be to add a ll_ttl (ll for link layer)
field to struct sock and a SO_LL_TTL socket option that sets both the
field and a flag in sk->flags. This way it is useful for any driver
that can do mesh or any other protocol that involves a ttl at link
layer (not that I'm aware of any).

However I guess you are not supposed to add new socket options nor
modify struct socket too often so I'd appreciate feedback on whether
this would be considered a good approach.

-- 
Luis Carlos Cobo Rus       GnuPG ID: 44019B60
cozybit Inc.

Re: Proposed interface for per-packet mesh-ttl

[Stephen Hemminger]

On Thu, 16 Aug 2007 12:21:14 -0700
"Luis Carlos Cobo" <luisca@cozybit.com> wrote:

> On 7/30/07, Stephen Hemminger <shemminger@linux-foundation.org> wrote:
> > it would need an IP ttl to mesh mapping. The fundamental thing is to try
> > and avoid topology specific options bleeding all the way up the socket layer,
> > especially since the network layer is involved and may need to multipath.
> 
> I think the cleanest way would be to add a ll_ttl (ll for link layer)
> field to struct sock and a SO_LL_TTL socket option that sets both the
> field and a flag in sk->flags. This way it is useful for any driver
> that can do mesh or any other protocol that involves a ttl at link
> layer (not that I'm aware of any).
> 
> However I guess you are not supposed to add new socket options nor
> modify struct socket too often so I'd appreciate feedback on whether
> this would be considered a good approach.
> 
> -- 
> Luis Carlos Cobo Rus       GnuPG ID: 44019B60
> cozybit Inc.

The problem with socket options is how does the application know
the correct policy? Pushing configuration to application is just deferring
the problem, not solving it.  You want some policy to be done by the
infrastructure; that means kernel, libraries, daemons, etc. Doing it in
the application is often inflexible and unusable.

Re: Proposed interface for per-packet mesh-ttl

[Luis Carlos Cobo]

On 8/16/07, Stephen Hemminger <shemminger@linux-foundation.org> wrote:
> The problem with socket options is how does the application know
> the correct policy? Pushing configuration to application is just deferring
> the problem, not solving it.  You want some policy to be done by the
> infrastructure; that means kernel, libraries, daemons, etc. Doing it in
> the application is often inflexible and unusable.

The policy will usually be done by the kernel. If the flag is not set,
which will happen most of the time, the driver will use a sensible
default ttl. The socket option would only be used by applications that
are specifically interested in a configurable ttl (like an application
to plot neighbors within an specific diameter). A per-interface
setting is not good enough, since we do not want the neighbor-plotting
tool to affect the ttl of other connections (e.g. a ssh session) that
might be going on at the same time.

-- 
Luis Carlos Cobo Rus       GnuPG ID: 44019B60
cozybit Inc.