RFC: Redirect-Device

RFC: Redirect-Device

[Ben Greear]

[-- Attachment #1: Type: text/plain, Size: 1651 bytes --]

I created a new virtual ethernet device that solves a problem
I faced.  I thought I'd see if anyone else sees a use for this,
and if so, I'll work to polish the patch so that it can be
accepted into the kernel.

The redirect device is fairly simple:

You create a pair of them, each associated with each other.
When you transmit on one, the other one receives a packet.
The devices are presented as regular ethernet devices, and they
have MACs, will do ARP, and other such things.

I use these in the following manner:

I have a software bridge that works by registering a protocol for all
packets on, for example, eth1, and it will forward these packets to another
device.  In my case, the other device will be one of the re-direct interfaces
(rdd0).  Both eth1 and rdd0 will NOT have any IP configuration.  This keeps
the stacks from interfering with my bridge.  rdd1 will have the IP address that
would normally be associated with eth1.

This allows me to have total control of the packets flowing between the physical
ethernet interface and the ethernet and higher level protocols on the rdd1 interface.
I believe this is the only way to have such total control of packets, for both inspection
and arbitrary modification, before the packets hit the protocol handlers.

So, I'm attaching the bulk of the module.  There are a few other patches to
tie in the ioctls and such that is not included here.  Please let me know
what you think of the module.  If it's deemed worthy, I'll dis-entangle the other
small patches from my big patch set.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com


[-- Attachment #2: redirdev.c --]
[-- Type: text/plain, Size: 24412 bytes --]

/* -*- linux-c -*-
#######################################################################
#
# (C) Copyright 2005
# Ben Greear <greearb@candelatech.com>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307 USA
#######################################################################
# Notes:
# 
# This file implements the Redirect-net-device module.  A pair of
# redir devices linked to each other act like two ethernet interfaces
# connected with a cross-over cable.
#
# This provides an IOCTL interface which allows you to
# It uses an IOCTL interface which allows you to
#
# 1. create redirect device
# 2. delete redirect device
# 
#######################################################################
*/

#include <linux/config.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/kernel.h> 
#include <linux/fs.h>	  
#include <linux/errno.h>  
#include <linux/delay.h>  
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/poll.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/if_redirdev.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
#include <net/arp.h>
#include <linux/rtnetlink.h>
#include <linux/notifier.h>

#include <asm/io.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
#include <asm/semaphore.h>
#include <net/dst.h>

#ifdef CONFIG_PROC_FS
#include <linux/proc_fs.h>
#define RDD_PROC_DIR "redirdev"
#define RDD_PROC_CFG "config"
static struct proc_dir_entry *rdd_proc_dir;
static struct proc_dir_entry *rdd_proc_cfg;
#endif

#include "redirdev.h"

/* Defined in socket.c */
void redirdev_ioctl_set(int (*hook)(void*));
static int redirdev_device_event(struct notifier_block *unused,
				 unsigned long event, void *ptr);

static struct notifier_block redirdev_notifier_block = {
	.notifier_call = redirdev_device_event,
};

/*********************************************************/
/*			defines				 */
/*********************************************************/

#if 0
#define DEBUG(format,args...) printk(KERN_ERR format, ##args);
#else
#define DEBUG(format,args...)
#endif


#undef RDD_USE_RW_LOCKS
#ifdef RDD_USE_RW_LOCKS
/*  Must hold this lock to make any changes to the macvlan structures.
 */
static rwlock_t rdd_cfg_lock = RW_LOCK_UNLOCKED;

#define RDD_READ_LOCK /* printk("%i: read-lock port list\n", __LINE__); */ \
                            BUG_ON(in_interrupt()); \
			    read_lock(&rdd_cfg_lock);
#define RDD_READ_UNLOCK /* printk("%i: read-unlock port list\n", __LINE__); */ \
                            BUG_ON(in_interrupt()); \
			    read_unlock(&rdd_cfg_lock);

#define RDD_WRITE_LOCK /* printk("%i: write-lock port list\n", __LINE__); */ \
                            BUG_ON(in_interrupt()); \
			    write_lock(&rdd_cfg_lock);
#define RDD_WRITE_UNLOCK /* printk("%i: write-unlock port list\n", __LINE__); */ \
                            BUG_ON(in_interrupt()); \
			    write_unlock(&rdd_cfg_lock);


#define RDD_IRQ_RLOCK(a) /* printk("%i: read-unlock port list\n", __LINE__); */ { \
                            __u64 now = getCurUs(); \
	                    __u64 later; \
			    read_lock_irqsave(&rdd_cfg_lock, a); \
                            later = getCurUs(); \
                            if ((later - now) > 100) { \
	                       printk("took: %lluus to acquire read lock, line: %i\n", \
				      later - now, __LINE__); \
                            }}

#define RDD_IRQ_RUNLOCK(a) /* printk("%i: read-unlock port list\n", __LINE__); */ \
			    read_unlock_irqrestore(&rdd_cfg_lock, a);
#else
/*  Must hold this lock to make any changes to the macvlan structures.
 */
static spinlock_t rdd_cfg_lock = SPIN_LOCK_UNLOCKED;

#define RDD_READ_LOCK(a) RDD_WRITE_LOCK(a)
#define RDD_READ_UNLOCK(a) RDD_WRITE_UNLOCK(a)

#define RDD_WRITE_LOCK(a) /* printk("%i: write-lock port list\n", __LINE__); */	\
		           spin_lock_irqsave(&rdd_cfg_lock, a);
#define RDD_WRITE_UNLOCK(a) /* printk("%i: write-unlock port list\n", __LINE__); */ \
                           spin_unlock_irqrestore(&rdd_cfg_lock, a); \


#define RDD_IRQ_RLOCK(a) /* printk("%i: read-unlock port list\n", __LINE__); */ \
			    spin_lock_irqsave(&rdd_cfg_lock, a); \

#define RDD_IRQ_RUNLOCK(a) /* printk("%i: read-unlock port list\n", __LINE__); */ \
			    spin_unlock_irqrestore(&rdd_cfg_lock, a);
#endif


/*********************************************************/
/*		       file scope variables		 */
/*********************************************************/

static struct redirdev* rdds = NULL;

static atomic_t rdd_dev_counter;

static int debug_lvl = 0;


/*********************************************************/
/*		   forward declarations			 */
/*********************************************************/

#ifdef RDD_CONFIG_PROC_FS
static int read_rdd_glbl(char *page, char **start, off_t off,
			 int count, int *eof, void *data);
static int write_rdd_glbl(struct file *file, const char *buffer,
			  unsigned long count, void *data);
#endif



/*********************************************************/
/*		   function definitions			 */
/*********************************************************/

/** Convert to micro-seconds */
static inline __u64 tv_to_us(const struct timeval* tv) {
        __u64 us = tv->tv_usec;
        us += (__u64)tv->tv_sec * (__u64)1000000;
        return us;
}


/* Since the epoc.  More precise over long periods of time than
 * getRelativeCurMs
 */
static inline __u64 getCurUs(void) {
        struct timeval tv;
        do_gettimeofday(&tv);
        return tv_to_us(&tv);
}


char toupper(char in) {
	if ((in >= 'a') && (in <= 'z')) {
		in -= ('a' - 'A');
	}
	return in;
}

#define iswhitespace(x)\
	((x) == ' ' || (x) == '\n' || (x) == '\r' || (x) == '\r' )

#define skip_whitespace(x) 	{ while (iswhitespace(*x)) (x)++; }

static int copy_next_word(char *dst, char *src, int len) {
	char *p;
	for (p=src; p < src + len ; p++) {
		if ( iswhitespace(*p))
			break;
		*dst++ = *p;
	}
	return p - src;
}

/*  Grab the RDD lock before calling this method. */
struct redirdev* rdd_find_dev_by_name(const char* ifname) {
	struct redirdev* d;
	//printk("finding port for underlying ifname: %s\n", ifname);
	for (d = rdds; d; d = d->next) {
		//printk("Testing port: %p name: %s\n", port, port->dev->name);
		if (strcmp(d->dev->name, ifname) == 0) {
			break;
		}
	}
	//printk("done finding port: %p\n", port);
	return d;
}

/*  Grab the RDD lock before calling this method. */
struct redirdev* rdd_find_dev_by_txdev_name(const char* ifname) {
	struct redirdev* d;
	for (d = rdds; d; d = d->next) {
		if (d->tx_dev) {
			if (strcmp(d->tx_dev->name, ifname) == 0) {
				break;
			}
		}
	}
	return d;
}	 


static struct net_device_stats *redirdev_get_stats(struct net_device *dev)
{
	struct redirdev* rdd = dev->priv;

	return &rdd->statistics;
}

/** Bump our tx counters and then act as if this was received from
 * the network on the tx_dev device.  Since we don't do any CSUM
 * activity in this driver, make sure SKB as marked as not checksummed
 * yet.
 */
static int redirdev_xmit(struct sk_buff *iskb, struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	struct net_device_stats* txs;
	
	if (unlikely(!rdd->tx_dev)) {
		printk("ERROR:  tx_dev null in redirdev_xmit.\n");
		kfree_skb(iskb);
		rdd->statistics.tx_errors++;
		return 0;
	}

	//printk("%s: dev: %s tx_dev: %s\n",
	//       __PRETTY_FUNCTION__, dev->name, rdd->tx_dev->name);

	if (netif_running(rdd->tx_dev)) {

		/* We need to free the old skb so that the socket
		 * account works correctly.  We'll make a copy and
		 * then forward that to the other device.
		 */

		struct sk_buff* skb = skb_clone(iskb, GFP_ATOMIC);

		kfree_skb(iskb); //Let the sending socket reclaim it's memory
		
		if (!skb) {
			rdd->statistics.tx_dropped++;
		}
		else {
			int rv;
			struct ethhdr *eth;
			skb->dev = rdd->tx_dev;

			/* We didn't calculate the csum, so mark as such. */
			skb->ip_summed = CHECKSUM_UNNECESSARY;//NONE;
	
			rdd->statistics.tx_packets++;
			rdd->statistics.tx_bytes += skb->len;

			txs = rdd->tx_dev->get_stats(rdd->tx_dev);
			txs->rx_packets++;
			txs->rx_bytes += skb->len;

			/* Call this on the receiving net device.  This assumes
			 * that all devices are ethernet or ethernet-like.  Valid
			 * for now.  TODO:  Generalize tx_dev ??
			 */
			skb->pkt_type = PACKET_HOST; //Reset this to default.

			skb->protocol = eth_type_trans(skb, skb->dev);

			eth = eth_hdr(skb);
			
			/*printk("dev: %s dst: %02hx:%02hx:%02hx:%02hx:%02hx:%02hx src: %02hx:%02hx:%02hx:%02hx:%02hx:%02hx adr: %02hx:%02hx:%02hx:%02hx:%02hx:%02hx\n  proto: %x  type: %u sum: %i cloned: %i shared: %i dst: %p\n",
			       skb->dev->name,
			       eth->h_dest[0], eth->h_dest[1], eth->h_dest[2],
			       eth->h_dest[3], eth->h_dest[4], eth->h_dest[5],
			       eth->h_source[0], eth->h_source[1], eth->h_source[2],
			       eth->h_source[3], eth->h_source[4], eth->h_source[5],
			       skb->dev->dev_addr[0], skb->dev->dev_addr[1],
			       skb->dev->dev_addr[2], skb->dev->dev_addr[3],
			       skb->dev->dev_addr[4], skb->dev->dev_addr[5],
			       (unsigned int)(skb->protocol),
			       (unsigned int)(skb->pkt_type),
			       (int)(skb->ip_summed), (int)(skb->cloned),
			       skb_shared(skb), skb->dst);
			*/
			
			if (skb->dst) {
				dst_release(skb->dst);
				skb->dst = NULL;
			}
			
			//printk("skb->protocol: %x  pkt_type: %u\n",
			//       (unsigned int)(skb->protocol),
			//       (unsigned int)(skb->pkt_type));
			rv = netif_rx(skb);
			if (rv != NET_RX_SUCCESS) {
				// TODO:  Remove
				//printk("netif_rx rv: %i\n", (int)(rv));
			}
			rdd->tx_dev->last_rx = jiffies;
			rdd->dev->trans_start = jiffies;
		}
	}
	else {
		/* Chunk the packet and log some errors */
		rdd->statistics.tx_errors++;
		kfree_skb(iskb);
	}
	return 0;
}/* redir xmit */

static int redirdev_open(struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	if (!rdd->tx_dev) {
		rdd->tx_dev = dev_get_by_name(rdd->tx_dev_name);
	}
	if (!rdd->tx_dev) {
		printk("redir: Could not start device %s because tx_dev: %s is not found.\n",
		       dev->name, rdd->tx_dev_name);
		return -ENODEV;
	}
	else {
		printk("redirdev:  Starting device: %s\n", dev->name);
		netif_start_queue(dev);
		return 0;
	}
}

//static void redirdev_set_multicast_list(struct net_device *dev) {
	/* TODO ??? */
//}

static int redirdev_stop(struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	printk("redirdev:  stopping device: %s\n", dev->name);
	netif_stop_queue(dev);
	if (rdd->tx_dev) {
		struct net_device* tmp = rdd->tx_dev;
		rdd->tx_dev = NULL;
		printk("  releasing reference to dev: %s\n", tmp->name);
		dev_put(tmp);
	}
	printk("   done stopping %s\n", dev->name);
	return 0;
}


void redirdev_dev_destructor(struct net_device *dev) {
	atomic_dec(&rdd_dev_counter);
	if (dev->priv) {
		//printk("dst: %s", dev->name);
		kfree(dev->priv);
		dev->priv = NULL;
	}
	else {
		//printk("dst2: %s", dev->name);
	}
}

int redirdev_change_mtu(struct net_device *dev, int new_mtu) {
	dev->mtu = new_mtu;
	return 0;
}

static int redirdev_create(const char* newifname,
			   const char* txdevname) {
	struct redirdev *rdd = NULL;
	struct net_device* td = NULL;
	struct net_device* nnd = NULL;
	struct net_device* txd = NULL;
	unsigned long flags;
	int rv;

	if ((strlen(txdevname) == 0) ||
	    (strlen(newifname) == 0)) {
		printk("redirdev: ERROR:  Must specify ifname and txifname"
		       " when creating redirect devices!\n");
		rv = -ENODEV;
		goto out;
	}	   

	printk("redirdev: creating interface: -:%s:- with tx_dev: -:%s:-\n",
	       newifname, txdevname);
	

	//printk("malloc ");
	if ((rdd = kmalloc(sizeof(*rdd), GFP_KERNEL)) == NULL) {
		DEBUG("redirdev: kmalloc failure\n");
		rv = -ENOMEM;
		goto outfree;
	}
	memset(rdd, 0, sizeof(*rdd));

	//printk("4 ");
	if ((nnd = kmalloc(sizeof(struct net_device), GFP_KERNEL)) == NULL) {
		DEBUG("redirdev: kmalloc net_device failure\n");
		rv = -ENOMEM;
		goto outfree;
	}
	memset(nnd, 0, sizeof(struct net_device));	

	if ((td = dev_get_by_name(newifname)) != NULL) {
		DEBUG("redirdev: device by that name already exists\n");
		rv = -EEXIST;
		goto outfree;
	}

	/* If it's not here yet, no problem, will associate later */
	txd = dev_get_by_name(txdevname);
	strncpy(rdd->tx_dev_name, txdevname, IFNAMSIZ);

	//printk("4 ");
	rdd->dev = nnd;
	
	//printk("5 ");
	strncpy(rdd->dev->name, newifname, IFNAMSIZ-1);
	rdd->dev->name[IFNAMSIZ-1] = 0; //Ensure null termination.
	ether_setup(rdd->dev);
	
	dev_hold(rdd->dev); /* RDD code holds reference */
	
	rdd->dev->priv = rdd;
	rdd->tx_dev = txd;
	
	//printk("6 ");
	/* dev->do_ioctl = macvlan_do_ioctl; */
	rdd->dev->get_stats = redirdev_get_stats;
	rdd->dev->hard_start_xmit = redirdev_xmit;
	rdd->dev->change_mtu = redirdev_change_mtu;
	rdd->dev->open = redirdev_open;
	rdd->dev->stop = redirdev_stop;
	rdd->dev->destructor = redirdev_dev_destructor;

	// Defaults are fine for these
	//rdd->dev->rebuild_header = redirdev_dev_rebuild_header;
	//rdd->dev->set_multicast_list = redirdev_set_multicast_list;
	//rdd->dev->hard_header = redirdev_hard_header;	

	rdd->dev->dev_addr[0] = 0;
	rdd->dev->dev_addr[1] = net_random();
	rdd->dev->dev_addr[2] = net_random();
	rdd->dev->dev_addr[3] = net_random();
	rdd->dev->dev_addr[4] = net_random();
	rdd->dev->dev_addr[5] = net_random();

	/* No qdisc for us */
	rdd->dev->qdisc = NULL;
	rdd->dev->tx_queue_len = 0;
	
	DEBUG("redirdev: created redirect-device %p\n", vlan);

	/* link to list */
	//printk("8 ");
	RDD_WRITE_LOCK(flags);
	rdd->next = rdds;
	rdds = rdd;
	RDD_WRITE_UNLOCK(flags);

	//printk("End of redirdev_create, registering rdd->dev: %p (%s)\n",
	//       rdd->dev, rdd->dev->name);
	
	register_netdev(rdd->dev);

	//printk("End of mac_vlan create2\n");

	atomic_inc(&rdd_dev_counter);
	//printk("9\n");
	rv = 0;
	goto out;

	/* Error case, clean up vlan memory */
 outfree:
	if (rdd) {
		kfree(rdd);
	}
	if (nnd) {
		kfree(nnd);
	}
	if (td) {
		dev_put(td);
	}
	if (txd) {
		dev_put(txd);
	}
 out:
	return rv;
} /* redirdev_create */

static int redirdev_device_event(struct notifier_block *unused,
				 unsigned long event, void *ptr) {
	struct net_device* dev = ptr;
	struct redirdev* rdd;
	unsigned long flags;
	
	RDD_READ_LOCK(flags);
	rdd = rdd_find_dev_by_txdev_name(dev->name);
	RDD_READ_UNLOCK(flags);

	if (!rdd) {
		//printk("redirdev: Ignoring event: %lu for device: %s\n",
		//       event, dev->name);
		goto out;
	}


	/* It is OK that we do not hold the group lock right now,
	 * as we run under the RTNL lock.
	 */

	switch (event) {
	case NETDEV_CHANGE:
	case NETDEV_UP:
	case NETDEV_DOWN:
		DEBUG("redirdev: Ignoring change/up/down for device: %s\n",
		      dev->name);
		/* Ignore for now */
		break;

	case NETDEV_UNREGISTER:
		/* Stop the redir-dev too */
		printk("Device: %s is going away, closing redir-device: %s too.\n",
		       dev->name, rdd->dev->name);
		dev_close(rdd->dev);
		break;

	};

out:
	return NOTIFY_DONE;
}

/* Has locking internally */
int redirdev_cleanup(const char* ifname, int force) {
	struct redirdev* d; //walker
	struct redirdev* prev = NULL;
	unsigned long flags;
	int rv;
	
	DEBUG(__FUNCTION__"(%p)\n",vlan);
	//printk("rdd_cln: %s", ifname);

	RDD_WRITE_LOCK(flags);
	for (d = rdds; d; d = d->next) {
		if (strcmp(d->dev->name, ifname) == 0) {			
			if ((d->dev->flags & IFF_UP) && (!force)) {
				rv = -EBUSY;
				goto unlockout;
			}

			// Un-link from the list.
			if (prev) {
				prev->next = d->next;
				d->next = NULL;
			}
			else {
				// This means we're first in line
				rdds = d->next;
				d->next = NULL;
			}

			break;
		}
		prev = d;
	}

	RDD_WRITE_UNLOCK(flags);

	if (d) {
		if (d->dev->flags & IFF_UP) {
			BUG_ON(!force);

			rtnl_lock();
			dev_close(d->dev);
			rtnl_unlock();
		}

		if (d->tx_dev) {
			dev_put(d->tx_dev);
		}
		
		dev_put(d->dev);
		unregister_netdev(d->dev);
		rv = 0;
	}
	else {
		rv = -ENODEV;
	}
	goto out;

 unlockout:
	RDD_WRITE_UNLOCK(flags);

 out:
	return rv;	
} /* redirdev cleanup */


static int redirdev_ioctl_deviceless_stub(void* arg) {
	int err = 0;
	struct redirdev_ioctl req;
	unsigned long flags;
	
	if (!capable(CAP_NET_ADMIN))
		return -EPERM;

	if (copy_from_user(&req, arg, sizeof(req)))
		return -EFAULT;

	switch (req.cmd) {
	case REDIRDEV_ADD: {
		/* 
		 * create a new redirect device
		 */
		req.txifname[IFNAMSIZ-1] = '\0';
		req.ifname[IFNAMSIZ-1] = '\0';
		printk("Creating redir via ioctl, ifname: %s  txifname: %s\n",
		       req.ifname, req.txifname);

		/* Has internal locking. */
		err = redirdev_create(req.ifname, req.txifname);
		break;
	}
	case REDIRDEV_DEL: {
		/*
		 * destroy a redirect device
		 */
		req.ifname[IFNAMSIZ-1] = '\0';

		/* Has internal locking */
		err = redirdev_cleanup(req.ifname, 0);
		break;
	}

	case REDIRDEV_IS_REDIRDEV: {
		/* 
		 * Give user-space a chance of determining if we are a redirect-device
		 * or not.
		 *  (If the IOCTL fails, we are not, otherwise we are.)
		 */
		struct redirdev* rdd;
		req.ifname[IFNAMSIZ-1] = '\0';
	    
		RDD_READ_LOCK(flags);
		/* find the port in question */
		rdd = rdd_find_dev_by_name(req.ifname);
		RDD_READ_UNLOCK(flags);

		if (!rdd) {
			/* printk("device: %s is NOT a REDIR device\n", ifname); */
			err = -ENODEV;
		}
		else {
			/* printk("device: %s IS a MAC-VLAN\n", ifname); */
			err = 0;
		}
		break;
	}
	case REDIRDEV_GET_BY_IDX: {
		/* 
		 * get the nth redirdev name
		 */
		struct redirdev *rdd;
		int n = req.ifidx;

		RDD_READ_LOCK(flags);
		/* find the port in question */
		for (rdd = rdds; rdd && n; rdd = rdd->next, n--);
		if (!rdd) {
			err = -ENODEV;
			RDD_READ_UNLOCK(flags);
		}
		else {
			memcpy(req.ifname, rdd->dev->name, IFNAMSIZ);
			memcpy(req.txifname, rdd->tx_dev_name, IFNAMSIZ);
			if (rdd->tx_dev) {
				req.flags |= RDD_ASSOCIATED;
			}
			else {
				req.flags &= ~RDD_ASSOCIATED;
			}
			RDD_READ_UNLOCK(flags);
					
			if (copy_to_user(arg, &req, sizeof(req))) {
				err = -EFAULT;
			}
		}
		break;
	}
	case REDIRDEV_GET_BY_NAME: {
		/* 
		 * get info on the specified redirect device
		 */
		struct redirdev *rdd;
		req.ifname[IFNAMSIZ-1] = '\0';
	    
		RDD_READ_LOCK(flags);
		/* find the port in question */
		rdd = rdd_find_dev_by_name(req.ifname);

		if (!rdd) {
			err = -ENODEV;
			RDD_READ_UNLOCK(flags);
		}
		else {
			memcpy(req.ifname, rdd->dev->name, IFNAMSIZ);
			memcpy(req.txifname, rdd->tx_dev_name, IFNAMSIZ);
			if (rdd->tx_dev) {
				req.flags |= RDD_ASSOCIATED;
			}
			else {
				req.flags &= ~RDD_ASSOCIATED;
			}
			RDD_READ_UNLOCK(flags);
					
			if (copy_to_user(arg, &req, sizeof(req))) {
				err = -EFAULT;
			}
		}
		break;
	}
	default:
		printk("ERROR:  Un-supported redirdev ioctl command: %u\n",
		       (unsigned int)(req.cmd));
		send_sig(SIGSEGV, current, 1); // TODO:  Remove
		err = -EOPNOTSUPP;
		break;
	}//switch

	/* printk("Returning err: %i\n", err); */
	return err;
}/* ioctl handler */


#ifdef RDD_CONFIG_PROC_FS

static int read_rdd_glbl(char *page, char **start, off_t off,
			 int count, int *eof, void *data) {
	int	ret = -1;
	char	*p = page;
	int mx_len = (4096 - (p - page));
	
	if (! *eof ) {
		struct redirdev* rdd;
		int cnt;
		unsigned long flags;
		
                /* Global counts here... */
		p += sprintf(p, "Redirect-Device module:\n");

		p += sprintf(p, " redirect-devices: %i\n",
			     atomic_read(&rdd_dev_counter));

		RDD_READ_LOCK(flags);
		rdd = rdds;
		while (rdd) {
			if (rdd->tx_dev) {
				p += sprintf(p, " %s  tx-dev: %s\n",
					     rdd->dev->name, rdd->tx_dev->name);
			}
			else {
				p += sprintf(p, " %s  tx-dev: [%s]\n",
					     rdd->dev->name, rdd->tx_dev_name);
			}

			/* catch overflow */
			cnt = p - page;
			if (cnt > (mx_len - 60)) {
				if (mx_len - cnt >= 20) {
					p += sprintf(p, "OUT_OF_SPACE!\n");
				}
				break;
			}

			rdd = rdd->next;
		}

		ret = p - page;
		RDD_READ_UNLOCK(flags);
	}
	return ret;
} /* read_rdd_glbl */

static int write_rdd_glbl(struct file *file, const char *buffer,
			  unsigned long count, void *data) {
	char		*p;
	const char	*end;
	int		ret=count;
	int		len;
	char		dev_name[2][IFNAMSIZ];
        char* tmps = NULL;
        char ss[50];
	end = buffer + count;

	snprintf(ss, 50, "redir proc cmd: %%.%lus", count);

	printk(ss, buffer);
	
	for (p= (char *) buffer; p< end ; ) {
		if (iswhitespace(*p)) {
			p++;
			continue;
		}

		memset(dev_name[0], 0 ,IFNAMSIZ);
		memset(dev_name[1], 0 ,IFNAMSIZ);

		len = strlen("add_rdd ");
		if (strncmp(p, "add_rdd ", len)==0)
		{
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

			skip_whitespace(p);

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[1], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[1], p, end-p );

			skip_whitespace(p);

                        /* This can fail, but not sure how to return failure
                         * to user-space here.
			 * NOTE:  Does it's own internal locking.
                         */
                        redirdev_create(dev_name[0], dev_name[1]);
                        goto forend;
		}
                
		len = strlen("remove_rdd ");
		if (strncmp(p,"remove_rdd ", len)==0) {
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

                        skip_whitespace(p);

                        redirdev_cleanup(dev_name[0], 0);
                        goto forend;
		}

                len = strlen("debug_lvl ");
		if (strncmp(p,"debug_lvl ",len)==0)
		{
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

                        skip_whitespace(p);

                        debug_lvl = simple_strtoul(dev_name[0], &tmps, 10);
                        goto forend;
		}

                printk("ERROR:  Unsupported command\n");

        forend:
		p++;
	}

	return ret;
} /* write_rdd_glbl */

#endif


static int __init redirdev_init(void) {
	int err;
	printk(KERN_INFO "Redirect-Network-Device: 1.0 <greearb@candelatech.com>\n");

	rdds = NULL;

	redirdev_ioctl_set(redirdev_ioctl_deviceless_stub);

#ifdef RDD_CONFIG_PROC_FS

        rdd_proc_dir = proc_mkdir(RDD_PROC_DIR, proc_net);
        if (rdd_proc_dir) {
		rdd_proc_cfg = create_proc_read_entry(RDD_PROC_CFG, S_IRUGO, rdd_proc_dir,
						      read_rdd_glbl, NULL);
		if (rdd_proc_cfg) {
			rdd_proc_cfg->write_proc = write_rdd_glbl;
			rdd_proc_cfg->owner = THIS_MODULE;
		}
	}
#endif

	/* Register us to receive netdevice events */
	err = register_netdevice_notifier(&redirdev_notifier_block);
	if (err < 0) {
		printk("ERROR:  redirdev: Failed to register netdevice notifier callback!\n");
	}	

	return 0;
}

static void redirdev_module_cleanup(void) {
	char nm[IFNAMSIZ+1];
	unsigned long flags;
	
	redirdev_ioctl_set(NULL);

	RDD_WRITE_LOCK(flags);
	/* destroy all redirect devices */
	while (rdds) {
		strncpy(nm, rdds->dev->name, IFNAMSIZ);
		RDD_WRITE_UNLOCK(flags);
		if (redirdev_cleanup(nm, 1) < 0) {
			printk("redirdev: ERROR:  Failed redir_cleanup in redir_module_cleanup\n");
			
		}
		RDD_WRITE_LOCK(flags);
	}
	RDD_WRITE_UNLOCK(flags);

	/* Un-register us from receiving netdevice events */
	unregister_netdevice_notifier(&redirdev_notifier_block);
	
#ifdef RDD_CONFIG_PROC_FS
	if (rdd_proc_cfg) {
		remove_proc_entry(RDD_PROC_CFG, rdd_proc_dir);
		rdd_proc_cfg = NULL;
	}
	if (rdd_proc_dir) {
		remove_proc_entry(RDD_PROC_DIR, proc_net);
		rdd_proc_dir = NULL;
	}
#endif

}/* redirdev_cleanup */


module_init(redirdev_init);
module_exit(redirdev_module_cleanup);
MODULE_LICENSE("GPL");

[-- Attachment #3: redirdev.h --]
[-- Type: text/plain, Size: 906 bytes --]

/* -*- linux-c -*-

# (C) Copyright 2005
# Ben Greear <greearb@candelatech.com>
# Released under the GPL version 2
*/

#ifndef REDIRDEV_KERNEL_H_FILE__
#define REDIRDEV_KERNEL_H_FILE__


/* Proc file related */
#define RDD_MX_ARG_LEN 80

#ifdef CONFIG_PROC_FS

/* To use or not to use the PROC-FS */
#define RDD_CONFIG_PROC_FS

#endif


/*********************************************************/
/*		       types				 */
/*********************************************************/
/* a macvlan_vlan represents an upper layer interface */
struct redirdev {
	/* Can be NULL if not yet associated */
	struct net_device* tx_dev; /* Call rx on this device when a packet
				    * is _transmitted_ on this redirect
				    * device.
				    */
	struct net_device* dev; /* the device struct this belongs too */
	struct redirdev *next;
	char tx_dev_name[IFNAMSIZ];
	struct net_device_stats	statistics;
};

#endif


[-- Attachment #4: if_redirdev.h --]
[-- Type: text/plain, Size: 762 bytes --]

/* -*- linux-c -*- */
#ifndef _LINUX_IF_REDIRDEV_H
#define _LINUX_IF_REDIRDEV_H

/* the ioctl commands */

#define REDIRDEV_ADD           2090
#define REDIRDEV_DEL           2091
/* If this IOCTL succeedes, we are a MAC-VLAN interface, otherwise, we are not. */
#define REDIRDEV_IS_REDIRDEV   2092
#define REDIRDEV_GET_BY_IDX    2093
#define REDIRDEV_GET_BY_NAME   2094

#ifdef __KERNEL__
#include <linux/if.h>
#include <linux/netdevice.h>
extern int (*redirdev_ioctl_hook)(void*);

#endif

/* Request and response */
struct redirdev_ioctl {
	u32  cmd;
	u32 ifidx; /* when getting info by idx */

#define RDD_ASSOCIATED (1<<0)
	u32 flags; /* 1<<0:  Is the interface associated with tx-dev or not */
	
	char ifname[IFNAMSIZ];
	char txifname[IFNAMSIZ];
};

#endif

Re: RFC: Redirect-Device

[David S. Miller]

My only comment is to either use something other than ioctls or realize
that fs/compat_ioctl.c entries for your new ioctls will be needed.

Re: RFC: Redirect-Device

[Stephen Hemminger]

You need to cleanup the style.  
	Lots of dead commented out code.
	Encapsulating locking in macros is ugly and makes maintenance a nuisance.
       CamelCaps
        Use prdebug instead of DEBUG
	RCU instead of rwlock's or just use spinlock
	Adding new /proc interfaces is frowned on
	Adding new ioctl's are very frowned on
	global namespace issues with toupper et a..

Re: RFC: Redirect-Device

[Ben Greear]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

Stephen Hemminger wrote:
> You need to cleanup the style.  
> 	Lots of dead commented out code.

Ok, think I got rid of most of the dead code and commented code.  I left
in the commented printks in case I need to do more debugging.

> 	Encapsulating locking in macros is ugly and makes maintenance a nuisance.

Changed to just support spin locks and got rid of the macros.

>        CamelCaps

No idea what you mean here.  Do you want JavaStyleCapitalization?

>         Use prdebug instead of DEBUG

Just changed the DEBUG to printk and commented them out.

> 	RCU instead of rwlock's or just use spinlock

Used spin-lock.  The rwlock code was never tested or enabled anyway.

> 	Adding new /proc interfaces is frowned on

I want to keep this.  I want quick script access to configure the devices so I don't
have to maintain a rddconfig c program that uses ioctls.

> 	Adding new ioctl's are very frowned on

Bummer.  I want these as well, makes programatic reading of the information
much easier than trying to parse some proc file.  I will look into what it
takes to make them compat with 64-bit as DaveM suggested.

> 	global namespace issues with toupper et a..

Purged it, it was dead code.

Please see if the attached code looks better to you.

Thanks!
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com


[-- Attachment #2: redirdev.c --]
[-- Type: text/plain, Size: 21257 bytes --]

/* -*- linux-c -*-
#######################################################################
#
# (C) Copyright 2005
# Ben Greear <greearb@candelatech.com>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307 USA
#######################################################################
# Notes:
# 
# This file implements the Redirect-net-device module.  A pair of
# redir devices linked to each other act like two ethernet interfaces
# connected with a cross-over cable.
#
# This provides an IOCTL interface which allows you to
# It uses an IOCTL interface which allows you to
#
# 1. create redirect device
# 2. delete redirect device
# 
#######################################################################
*/

#include <linux/config.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/kernel.h> 
#include <linux/fs.h>	  
#include <linux/errno.h>  
#include <linux/delay.h>  
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/poll.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/if_redirdev.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
#include <net/arp.h>
#include <linux/rtnetlink.h>
#include <linux/notifier.h>

#include <asm/io.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
#include <asm/semaphore.h>
#include <net/dst.h>

#ifdef CONFIG_PROC_FS
#include <linux/proc_fs.h>
#define RDD_PROC_DIR "redirdev"
#define RDD_PROC_CFG "config"
static struct proc_dir_entry *rdd_proc_dir;
static struct proc_dir_entry *rdd_proc_cfg;
#endif

#include "redirdev.h"

/* Defined in socket.c */
void redirdev_ioctl_set(int (*hook)(void*));
static int redirdev_device_event(struct notifier_block *unused,
				 unsigned long event, void *ptr);

static struct notifier_block redirdev_notifier_block = {
	.notifier_call = redirdev_device_event,
};

/*********************************************************/
/*			defines				 */
/*********************************************************/

/*  Must hold this lock to make any changes to the Redirect-Device structures.
 */
static spinlock_t rdd_cfg_lock = SPIN_LOCK_UNLOCKED;


/*********************************************************/
/*		       file scope variables		 */
/*********************************************************/

static struct redirdev* rdds = NULL;

static atomic_t rdd_dev_counter;

static int debug_lvl = 0;


/*********************************************************/
/*		   forward declarations			 */
/*********************************************************/

#ifdef RDD_CONFIG_PROC_FS
static int read_rdd_glbl(char *page, char **start, off_t off,
			 int count, int *eof, void *data);
static int write_rdd_glbl(struct file *file, const char *buffer,
			  unsigned long count, void *data);
#endif



/*********************************************************/
/*		   function definitions			 */
/*********************************************************/


#define iswhitespace(x)\
	((x) == ' ' || (x) == '\n' || (x) == '\r' || (x) == '\r' )

#define skip_whitespace(x) 	{ while (iswhitespace(*x)) (x)++; }

static int copy_next_word(char *dst, char *src, int len) {
	char *p;
	for (p=src; p < src + len ; p++) {
		if ( iswhitespace(*p))
			break;
		*dst++ = *p;
	}
	return p - src;
}

/*  Grab the RDD lock before calling this method. */
struct redirdev* rdd_find_dev_by_name(const char* ifname) {
	struct redirdev* d;
	//printk("finding port for underlying ifname: %s\n", ifname);
	for (d = rdds; d; d = d->next) {
		//printk("Testing port: %p name: %s\n", port, port->dev->name);
		if (strcmp(d->dev->name, ifname) == 0) {
			break;
		}
	}
	//printk("done finding port: %p\n", port);
	return d;
}

/*  Grab the RDD lock before calling this method. */
struct redirdev* rdd_find_dev_by_txdev_name(const char* ifname) {
	struct redirdev* d;
	for (d = rdds; d; d = d->next) {
		if (d->tx_dev) {
			if (strcmp(d->tx_dev->name, ifname) == 0) {
				break;
			}
		}
	}
	return d;
}	 


static struct net_device_stats *redirdev_get_stats(struct net_device *dev)
{
	struct redirdev* rdd = dev->priv;

	return &rdd->statistics;
}

/** Bump our tx counters and then act as if this was received from
 * the network on the tx_dev device.  Since we don't do any CSUM
 * activity in this driver, make sure SKB as marked as not checksummed
 * yet.
 */
static int redirdev_xmit(struct sk_buff *iskb, struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	struct net_device_stats* txs;
	
	if (unlikely(!rdd->tx_dev)) {
		printk("ERROR:  tx_dev null in redirdev_xmit.\n");
		kfree_skb(iskb);
		rdd->statistics.tx_errors++;
		return 0;
	}

	//printk("%s: dev: %s tx_dev: %s\n",
	//       __PRETTY_FUNCTION__, dev->name, rdd->tx_dev->name);

	if (netif_running(rdd->tx_dev)) {

		/* We need to free the old skb so that the socket
		 * account works correctly.  We'll make a copy and
		 * then forward that to the other device.
		 */

		struct sk_buff* skb = skb_clone(iskb, GFP_ATOMIC);

		kfree_skb(iskb); //Let the sending socket reclaim it's memory
		
		if (!skb) {
			rdd->statistics.tx_dropped++;
		}
		else {
			int rv;
			struct ethhdr *eth;
			skb->dev = rdd->tx_dev;

			/* We didn't calculate the csum, so mark as such. */
			skb->ip_summed = CHECKSUM_UNNECESSARY;//NONE;
	
			rdd->statistics.tx_packets++;
			rdd->statistics.tx_bytes += skb->len;

			txs = rdd->tx_dev->get_stats(rdd->tx_dev);
			txs->rx_packets++;
			txs->rx_bytes += skb->len;

			/* Call this on the receiving net device.  This assumes
			 * that all devices are ethernet or ethernet-like.  Valid
			 * for now.  TODO:  Generalize tx_dev ??
			 */
			skb->pkt_type = PACKET_HOST; //Reset this to default.

			skb->protocol = eth_type_trans(skb, skb->dev);

			eth = eth_hdr(skb);
			
			if (skb->dst) {
				dst_release(skb->dst);
				skb->dst = NULL;
			}
			
			//printk("skb->protocol: %x  pkt_type: %u\n",
			//       (unsigned int)(skb->protocol),
			//       (unsigned int)(skb->pkt_type));
			rv = netif_rx(skb);
			if (rv != NET_RX_SUCCESS) {
				// TODO:  Remove
				//printk("netif_rx rv: %i\n", (int)(rv));
			}
			rdd->tx_dev->last_rx = jiffies;
			rdd->dev->trans_start = jiffies;
		}
	}
	else {
		/* Chunk the packet and log some errors */
		rdd->statistics.tx_errors++;
		kfree_skb(iskb);
	}
	return 0;
}/* redir xmit */

static int redirdev_open(struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	if (!rdd->tx_dev) {
		rdd->tx_dev = dev_get_by_name(rdd->tx_dev_name);
	}
	if (!rdd->tx_dev) {
		printk("redir: Could not start device %s because tx_dev: %s is not found.\n",
		       dev->name, rdd->tx_dev_name);
		return -ENODEV;
	}
	else {
		printk("redirdev:  Starting device: %s\n", dev->name);
		netif_start_queue(dev);
		return 0;
	}
}

//static void redirdev_set_multicast_list(struct net_device *dev) {
	/* TODO ??? */
//}

static int redirdev_stop(struct net_device *dev) {
	struct redirdev* rdd = dev->priv;
	printk("redirdev:  stopping device: %s\n", dev->name);
	netif_stop_queue(dev);
	if (rdd->tx_dev) {
		struct net_device* tmp = rdd->tx_dev;
		rdd->tx_dev = NULL;
		printk("  releasing reference to dev: %s\n", tmp->name);
		dev_put(tmp);
	}
	printk("   done stopping %s\n", dev->name);
	return 0;
}


void redirdev_dev_destructor(struct net_device *dev) {
	atomic_dec(&rdd_dev_counter);
	if (dev->priv) {
		//printk("dst: %s", dev->name);
		kfree(dev->priv);
		dev->priv = NULL;
	}
	else {
		//printk("dst2: %s", dev->name);
	}
}

int redirdev_change_mtu(struct net_device *dev, int new_mtu) {
	dev->mtu = new_mtu;
	return 0;
}

static int redirdev_create(const char* newifname,
			   const char* txdevname) {
	struct redirdev *rdd = NULL;
	struct net_device* td = NULL;
	struct net_device* nnd = NULL;
	struct net_device* txd = NULL;
	unsigned long flags;
	int rv;

	if ((strlen(txdevname) == 0) ||
	    (strlen(newifname) == 0)) {
		printk("redirdev: ERROR:  Must specify ifname and txifname"
		       " when creating redirect devices!\n");
		rv = -ENODEV;
		goto out;
	}	   

	printk("redirdev: creating interface: -:%s:- with tx_dev: -:%s:-\n",
	       newifname, txdevname);
	

	//printk("malloc ");
	if ((rdd = kmalloc(sizeof(*rdd), GFP_KERNEL)) == NULL) {
		//printk("redirdev: kmalloc failure\n");
		rv = -ENOMEM;
		goto outfree;
	}
	memset(rdd, 0, sizeof(*rdd));

	//printk("4 ");
	if ((nnd = kmalloc(sizeof(struct net_device), GFP_KERNEL)) == NULL) {
		//printk("redirdev: kmalloc net_device failure\n");
		rv = -ENOMEM;
		goto outfree;
	}
	memset(nnd, 0, sizeof(struct net_device));	

	if ((td = dev_get_by_name(newifname)) != NULL) {
		//printk("redirdev: device by that name already exists\n");
		rv = -EEXIST;
		goto outfree;
	}

	/* If it's not here yet, no problem, will associate later */
	txd = dev_get_by_name(txdevname);
	strncpy(rdd->tx_dev_name, txdevname, IFNAMSIZ);

	//printk("4 ");
	rdd->dev = nnd;
	
	//printk("5 ");
	strncpy(rdd->dev->name, newifname, IFNAMSIZ-1);
	rdd->dev->name[IFNAMSIZ-1] = 0; //Ensure null termination.
	ether_setup(rdd->dev);
	
	dev_hold(rdd->dev); /* RDD code holds reference */
	
	rdd->dev->priv = rdd;
	rdd->tx_dev = txd;
	
	//printk("6 ");
	rdd->dev->get_stats = redirdev_get_stats;
	rdd->dev->hard_start_xmit = redirdev_xmit;
	rdd->dev->change_mtu = redirdev_change_mtu;
	rdd->dev->open = redirdev_open;
	rdd->dev->stop = redirdev_stop;
	rdd->dev->destructor = redirdev_dev_destructor;

	// Defaults are fine for these
	//rdd->dev->rebuild_header = redirdev_dev_rebuild_header;
	//rdd->dev->set_multicast_list = redirdev_set_multicast_list;
	//rdd->dev->hard_header = redirdev_hard_header;	

	rdd->dev->dev_addr[0] = 0;
	rdd->dev->dev_addr[1] = net_random();
	rdd->dev->dev_addr[2] = net_random();
	rdd->dev->dev_addr[3] = net_random();
	rdd->dev->dev_addr[4] = net_random();
	rdd->dev->dev_addr[5] = net_random();

	/* No qdisc for us */
	rdd->dev->qdisc = NULL;
	rdd->dev->tx_queue_len = 0;
	
	//printk("redirdev: created redirect-device %p\n", vlan);

	/* link to list */
	//printk("8 ");
	spin_lock_irqsave(&rdd_cfg_lock, flags);
	rdd->next = rdds;
	rdds = rdd;
	spin_unlock_irqrestore(&rdd_cfg_lock, flags);

	//printk("End of redirdev_create, registering rdd->dev: %p (%s)\n",
	//       rdd->dev, rdd->dev->name);
	
	register_netdev(rdd->dev);

	//printk("End of mac_vlan create2\n");

	atomic_inc(&rdd_dev_counter);
	//printk("9\n");
	rv = 0;
	goto out;

	/* Error case, clean up vlan memory */
 outfree:
	if (rdd) {
		kfree(rdd);
	}
	if (nnd) {
		kfree(nnd);
	}
	if (td) {
		dev_put(td);
	}
	if (txd) {
		dev_put(txd);
	}
 out:
	return rv;
} /* redirdev_create */

static int redirdev_device_event(struct notifier_block *unused,
				 unsigned long event, void *ptr) {
	struct net_device* dev = ptr;
	struct redirdev* rdd;
	unsigned long flags;
	
	spin_lock_irqsave(&rdd_cfg_lock, flags);
	rdd = rdd_find_dev_by_txdev_name(dev->name);
	spin_unlock_irqrestore(&rdd_cfg_lock, flags);

	if (!rdd) {
		//printk("redirdev: Ignoring event: %lu for device: %s\n",
		//       event, dev->name);
		goto out;
	}


	/* It is OK that we do not hold the group lock right now,
	 * as we run under the RTNL lock.
	 */

	switch (event) {
	case NETDEV_CHANGE:
	case NETDEV_UP:
	case NETDEV_DOWN:
		//printk("redirdev: Ignoring change/up/down for device: %s\n",
		//       dev->name);
		/* Ignore for now */
		break;

	case NETDEV_UNREGISTER:
		/* Stop the redir-dev too */
		printk("Device: %s is going away, closing redir-device: %s too.\n",
		       dev->name, rdd->dev->name);
		dev_close(rdd->dev);
		break;

	};

out:
	return NOTIFY_DONE;
}

/* Has locking internally */
int redirdev_cleanup(const char* ifname, int force) {
	struct redirdev* d; //walker
	struct redirdev* prev = NULL;
	unsigned long flags;
	int rv;
	
	//printk(__FUNCTION__"(%p)\n",vlan);
	//printk("rdd_cln: %s", ifname);

	spin_lock_irqsave(&rdd_cfg_lock, flags);
	for (d = rdds; d; d = d->next) {
		if (strcmp(d->dev->name, ifname) == 0) {			
			if ((d->dev->flags & IFF_UP) && (!force)) {
				rv = -EBUSY;
				goto unlockout;
			}

			// Un-link from the list.
			if (prev) {
				prev->next = d->next;
				d->next = NULL;
			}
			else {
				// This means we're first in line
				rdds = d->next;
				d->next = NULL;
			}

			break;
		}
		prev = d;
	}

	spin_unlock_irqrestore(&rdd_cfg_lock, flags);

	if (d) {
		if (d->dev->flags & IFF_UP) {
			BUG_ON(!force);

			rtnl_lock();
			dev_close(d->dev);
			rtnl_unlock();
		}

		if (d->tx_dev) {
			dev_put(d->tx_dev);
		}
		
		dev_put(d->dev);
		unregister_netdev(d->dev);
		rv = 0;
	}
	else {
		rv = -ENODEV;
	}
	goto out;

 unlockout:
	spin_unlock_irqrestore(&rdd_cfg_lock, flags);

 out:
	return rv;	
} /* redirdev cleanup */


static int redirdev_ioctl_deviceless_stub(void* arg) {
	int err = 0;
	struct redirdev_ioctl req;
	unsigned long flags;
	
	if (!capable(CAP_NET_ADMIN))
		return -EPERM;

	if (copy_from_user(&req, arg, sizeof(req)))
		return -EFAULT;

	switch (req.cmd) {
	case REDIRDEV_ADD: {
		/* 
		 * create a new redirect device
		 */
		req.txifname[IFNAMSIZ-1] = '\0';
		req.ifname[IFNAMSIZ-1] = '\0';
		printk("Creating redir via ioctl, ifname: %s  txifname: %s\n",
		       req.ifname, req.txifname);

		/* Has internal locking. */
		err = redirdev_create(req.ifname, req.txifname);
		break;
	}
	case REDIRDEV_DEL: {
		/*
		 * destroy a redirect device
		 */
		req.ifname[IFNAMSIZ-1] = '\0';

		/* Has internal locking */
		err = redirdev_cleanup(req.ifname, 0);
		break;
	}

	case REDIRDEV_IS_REDIRDEV: {
		/* 
		 * Give user-space a chance of determining if we are a redirect-device
		 * or not.
		 *  (If the IOCTL fails, we are not, otherwise we are.)
		 */
		struct redirdev* rdd;
		req.ifname[IFNAMSIZ-1] = '\0';
	    
		spin_lock_irqsave(&rdd_cfg_lock, flags);
		/* find the port in question */
		rdd = rdd_find_dev_by_name(req.ifname);
		spin_unlock_irqrestore(&rdd_cfg_lock, flags);

		if (!rdd) {
			/* printk("device: %s is NOT a REDIR device\n", ifname); */
			err = -ENODEV;
		}
		else {
			/* printk("device: %s IS a MAC-VLAN\n", ifname); */
			err = 0;
		}
		break;
	}
	case REDIRDEV_GET_BY_IDX: {
		/* 
		 * get the nth redirdev name
		 */
		struct redirdev *rdd;
		int n = req.ifidx;

		spin_lock_irqsave(&rdd_cfg_lock, flags);
		/* find the port in question */
		for (rdd = rdds; rdd && n; rdd = rdd->next, n--);
		if (!rdd) {
			err = -ENODEV;
			spin_unlock_irqrestore(&rdd_cfg_lock, flags);
		}
		else {
			memcpy(req.ifname, rdd->dev->name, IFNAMSIZ);
			memcpy(req.txifname, rdd->tx_dev_name, IFNAMSIZ);
			if (rdd->tx_dev) {
				req.flags |= RDD_ASSOCIATED;
			}
			else {
				req.flags &= ~RDD_ASSOCIATED;
			}
			spin_unlock_irqrestore(&rdd_cfg_lock, flags);
					
			if (copy_to_user(arg, &req, sizeof(req))) {
				err = -EFAULT;
			}
		}
		break;
	}
	case REDIRDEV_GET_BY_NAME: {
		/* 
		 * get info on the specified redirect device
		 */
		struct redirdev *rdd;
		req.ifname[IFNAMSIZ-1] = '\0';
	    
		spin_lock_irqsave(&rdd_cfg_lock, flags);
		/* find the port in question */
		rdd = rdd_find_dev_by_name(req.ifname);

		if (!rdd) {
			err = -ENODEV;
			spin_unlock_irqrestore(&rdd_cfg_lock, flags);
		}
		else {
			memcpy(req.ifname, rdd->dev->name, IFNAMSIZ);
			memcpy(req.txifname, rdd->tx_dev_name, IFNAMSIZ);
			if (rdd->tx_dev) {
				req.flags |= RDD_ASSOCIATED;
			}
			else {
				req.flags &= ~RDD_ASSOCIATED;
			}
			spin_unlock_irqrestore(&rdd_cfg_lock, flags);
					
			if (copy_to_user(arg, &req, sizeof(req))) {
				err = -EFAULT;
			}
		}
		break;
	}
	default:
		printk("ERROR:  Un-supported redirdev ioctl command: %u\n",
		       (unsigned int)(req.cmd));
		send_sig(SIGSEGV, current, 1); // TODO:  Remove
		err = -EOPNOTSUPP;
		break;
	}//switch

	/* printk("Returning err: %i\n", err); */
	return err;
}/* ioctl handler */


#ifdef RDD_CONFIG_PROC_FS

static int read_rdd_glbl(char *page, char **start, off_t off,
			 int count, int *eof, void *data) {
	int	ret = -1;
	char	*p = page;
	int mx_len = (4096 - (p - page));
	
	if (! *eof ) {
		struct redirdev* rdd;
		int cnt;
		unsigned long flags;
		
                /* Global counts here... */
		p += sprintf(p, "Redirect-Device module:\n");

		p += sprintf(p, " redirect-devices: %i\n",
			     atomic_read(&rdd_dev_counter));

		spin_lock_irqsave(&rdd_cfg_lock, flags);
		rdd = rdds;
		while (rdd) {
			if (rdd->tx_dev) {
				p += sprintf(p, " %s  tx-dev: %s\n",
					     rdd->dev->name, rdd->tx_dev->name);
			}
			else {
				p += sprintf(p, " %s  tx-dev: [%s]\n",
					     rdd->dev->name, rdd->tx_dev_name);
			}

			/* catch overflow */
			cnt = p - page;
			if (cnt > (mx_len - 60)) {
				if (mx_len - cnt >= 20) {
					p += sprintf(p, "OUT_OF_SPACE!\n");
				}
				break;
			}

			rdd = rdd->next;
		}

		ret = p - page;
		spin_unlock_irqrestore(&rdd_cfg_lock, flags);
	}
	return ret;
} /* read_rdd_glbl */

static int write_rdd_glbl(struct file *file, const char *buffer,
			  unsigned long count, void *data) {
	char		*p;
	const char	*end;
	int		ret=count;
	int		len;
	char		dev_name[2][IFNAMSIZ];
        char* tmps = NULL;
        char ss[50];
	end = buffer + count;

	snprintf(ss, 50, "redir proc cmd: %%.%lus", count);

	printk(ss, buffer);
	
	for (p= (char *) buffer; p< end ; ) {
		if (iswhitespace(*p)) {
			p++;
			continue;
		}

		memset(dev_name[0], 0 ,IFNAMSIZ);
		memset(dev_name[1], 0 ,IFNAMSIZ);

		len = strlen("add_rdd ");
		if (strncmp(p, "add_rdd ", len)==0)
		{
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

			skip_whitespace(p);

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[1], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[1], p, end-p );

			skip_whitespace(p);

                        /* This can fail, but not sure how to return failure
                         * to user-space here.
			 * NOTE:  Does it's own internal locking.
                         */
                        redirdev_create(dev_name[0], dev_name[1]);
                        goto forend;
		}
                
		len = strlen("remove_rdd ");
		if (strncmp(p,"remove_rdd ", len)==0) {
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

                        skip_whitespace(p);

                        redirdev_cleanup(dev_name[0], 0);
                        goto forend;
		}

                len = strlen("debug_lvl ");
		if (strncmp(p,"debug_lvl ",len)==0)
		{
			p += len;

			if ( (p + IFNAMSIZ) <= end)
				p += copy_next_word(dev_name[0], p, IFNAMSIZ);
			else
				p += copy_next_word(dev_name[0], p, end-p );

                        skip_whitespace(p);

                        debug_lvl = simple_strtoul(dev_name[0], &tmps, 10);
                        goto forend;
		}

                printk("ERROR:  Unsupported command\n");

        forend:
		p++;
	}

	return ret;
} /* write_rdd_glbl */

#endif


static int __init redirdev_init(void) {
	int err;
	printk(KERN_INFO "Redirect-Network-Device: 1.0 <greearb@candelatech.com>\n");

	rdds = NULL;

	redirdev_ioctl_set(redirdev_ioctl_deviceless_stub);

#ifdef RDD_CONFIG_PROC_FS

        rdd_proc_dir = proc_mkdir(RDD_PROC_DIR, proc_net);
        if (rdd_proc_dir) {
		rdd_proc_cfg = create_proc_read_entry(RDD_PROC_CFG, S_IRUGO, rdd_proc_dir,
						      read_rdd_glbl, NULL);
		if (rdd_proc_cfg) {
			rdd_proc_cfg->write_proc = write_rdd_glbl;
			rdd_proc_cfg->owner = THIS_MODULE;
		}
	}
#endif

	/* Register us to receive netdevice events */
	err = register_netdevice_notifier(&redirdev_notifier_block);
	if (err < 0) {
		printk("ERROR:  redirdev: Failed to register netdevice notifier callback!\n");
	}	

	return 0;
}

static void redirdev_module_cleanup(void) {
	char nm[IFNAMSIZ+1];
	unsigned long flags;
	
	redirdev_ioctl_set(NULL);

	spin_lock_irqsave(&rdd_cfg_lock, flags);
	/* destroy all redirect devices */
	while (rdds) {
		strncpy(nm, rdds->dev->name, IFNAMSIZ);
		spin_unlock_irqrestore(&rdd_cfg_lock, flags);
		if (redirdev_cleanup(nm, 1) < 0) {
			printk("redirdev: ERROR:  Failed redir_cleanup in redir_module_cleanup\n");
			
		}
		spin_lock_irqsave(&rdd_cfg_lock, flags);
	}
	spin_unlock_irqrestore(&rdd_cfg_lock, flags);

	/* Un-register us from receiving netdevice events */
	unregister_netdevice_notifier(&redirdev_notifier_block);
	
#ifdef RDD_CONFIG_PROC_FS
	if (rdd_proc_cfg) {
		remove_proc_entry(RDD_PROC_CFG, rdd_proc_dir);
		rdd_proc_cfg = NULL;
	}
	if (rdd_proc_dir) {
		remove_proc_entry(RDD_PROC_DIR, proc_net);
		rdd_proc_dir = NULL;
	}
#endif

}/* redirdev_cleanup */


module_init(redirdev_init);
module_exit(redirdev_module_cleanup);
MODULE_LICENSE("GPL");

[-- Attachment #3: redirdev.h --]
[-- Type: text/plain, Size: 849 bytes --]

/* -*- linux-c -*-

# (C) Copyright 2005
# Ben Greear <greearb@candelatech.com>
# Released under the GPL version 2
*/

#ifndef REDIRDEV_KERNEL_H_FILE__
#define REDIRDEV_KERNEL_H_FILE__


/* Proc file related */
#define RDD_MX_ARG_LEN 80

#ifdef CONFIG_PROC_FS

/* To use or not to use the PROC-FS */
#define RDD_CONFIG_PROC_FS

#endif


/*********************************************************/
/*		       types				 */
/*********************************************************/
struct redirdev {
	/* Can be NULL if not yet associated */
	struct net_device* tx_dev; /* Call rx on this device when a packet
				    * is _transmitted_ on this redirect
				    * device.
				    */
	struct net_device* dev; /* the device struct this belongs too */
	struct redirdev *next;
	char tx_dev_name[IFNAMSIZ];
	struct net_device_stats	statistics;
};

#endif

Re: RFC: Redirect-Device

[David S. Miller]

On Thu, 31 Mar 2005 13:45:38 -0800
Ben Greear <greearb@candelatech.com> wrote:

> > 	Adding new ioctl's are very frowned on
> 
> Bummer.  I want these as well, makes programatic reading of the information
> much easier than trying to parse some proc file.  I will look into what it
> takes to make them compat with 64-bit as DaveM suggested.

You could extend and use ethtool.  That is what it's there for.

Re: RFC: Redirect-Device

[Ben Greear]

David S. Miller wrote:
> On Thu, 31 Mar 2005 13:45:38 -0800
> Ben Greear <greearb@candelatech.com> wrote:
> 
> 
>>>	Adding new ioctl's are very frowned on
>>
>>Bummer.  I want these as well, makes programatic reading of the information
>>much easier than trying to parse some proc file.  I will look into what it
>>takes to make them compat with 64-bit as DaveM suggested.
> 
> 
> You could extend and use ethtool.  That is what it's there for.

The IOCTLs I need are very specific and will only be used for
the redirect module.  I don't think it would be worth adding
a function pointer to the ethtool-ops, but if that is what you
prefer I can do so.

With regard to the compat:  I did a quick search for 'vlan' in that
file, thinking I'd mimic whatever was done for the .1q module.  There
is no mention of vlan anywhere in there.  Does that mean that so long
as I don't use pointers, strictly define my data types (u32, etc),
and make sure the ioctl structs are padded to 32-bit (or maybe 64-bit?)
boundaries, I don't need any 64-bit compat code?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[David S. Miller]

On Thu, 31 Mar 2005 14:04:11 -0800
Ben Greear <greearb@candelatech.com> wrote:

> > You could extend and use ethtool.  That is what it's there for.
> 
> The IOCTLs I need are very specific and will only be used for
> the redirect module.  I don't think it would be worth adding
> a function pointer to the ethtool-ops, but if that is what you
> prefer I can do so.

I'll let Jeff make this call.

> With regard to the compat:  I did a quick search for 'vlan' in that
> file, thinking I'd mimic whatever was done for the .1q module.  There
> is no mention of vlan anywhere in there.

fs/compat_ioctl.c includes some files, perhaps you should check
some of those out.  In particular, what you seek might, just might,
be discoverd in include/linux/compat_ioctl.h

Also see Stephen's commentary wrt. sysfs

Re: RFC: Redirect-Device

[Stephen Hemminger]

On Thu, 31 Mar 2005 13:52:29 -0800
"David S. Miller" <davem@davemloft.net> wrote:

> On Thu, 31 Mar 2005 13:45:38 -0800
> Ben Greear <greearb@candelatech.com> wrote:
> 
> > > 	Adding new ioctl's are very frowned on
> > 
> > Bummer.  I want these as well, makes programatic reading of the information
> > much easier than trying to parse some proc file.  I will look into what it
> > takes to make them compat with 64-bit as DaveM suggested.
> 
> You could extend and use ethtool.  That is what it's there for.

Other possiblity is adding additional attributes onto the device with sysfs.
That is what net/bridge/br_sysfs_if.c does. I'm not claiming it is a pretty
kernel programming model, but the user side API is simpler.

Re: RFC: Redirect-Device

[Ben Greear]

Stephen Hemminger wrote:

> Other possiblity is adding additional attributes onto the device with sysfs.
> That is what net/bridge/br_sysfs_if.c does. I'm not claiming it is a pretty
> kernel programming model, but the user side API is simpler.

I prefer to stay away from sysfs.  procfs may have it's faults, but at
least in this case, it is quite straight-forward and easily does what
I need.  And procfs is less encumbered by flame wars :)

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[David S. Miller]

On Thu, 31 Mar 2005 14:26:49 -0800
Ben Greear <greearb@candelatech.com> wrote:

> Stephen Hemminger wrote:
> 
> > Other possiblity is adding additional attributes onto the device with sysfs.
> > That is what net/bridge/br_sysfs_if.c does. I'm not claiming it is a pretty
> > kernel programming model, but the user side API is simpler.
> 
> I prefer to stay away from sysfs.  procfs may have it's faults, but at
> least in this case, it is quite straight-forward and easily does what
> I need.  And procfs is less encumbered by flame wars :)

Are you shying away from sysfs merely for encumberance reasons?

You've dissed Jamal's excellent TC action infrastructure, you're
now dissing sysfs as well which can also solve your problems.

All new interfaces are being done through sysfs, that's simply a
fact of life in the kernel.  If you want to buck this trend, you
have to come up with a much better argument than "procfs is more
straight-forward".  Well if sysfs isn't as straightforward, suggest
a layer of interfaces that might make it so.

Re: RFC: Redirect-Device

[Ben Greear]

David S. Miller wrote:
> On Thu, 31 Mar 2005 14:26:49 -0800
> Ben Greear <greearb@candelatech.com> wrote:
> 
> 
>>Stephen Hemminger wrote:
>>
>>
>>>Other possiblity is adding additional attributes onto the device with sysfs.
>>>That is what net/bridge/br_sysfs_if.c does. I'm not claiming it is a pretty
>>>kernel programming model, but the user side API is simpler.
>>
>>I prefer to stay away from sysfs.  procfs may have it's faults, but at
>>least in this case, it is quite straight-forward and easily does what
>>I need.  And procfs is less encumbered by flame wars :)
> 
> 
> Are you shying away from sysfs merely for encumberance reasons?

Primarily laziness:  I know procfs and it works well for me.

> You've dissed Jamal's excellent TC action infrastructure, you're
> now dissing sysfs as well which can also solve your problems.

I don't think the TC stuff can do what I need.  However, there are
a lot less constraints on the text configuration interface, so I imagine
I can get sysfs to do what I need.

> All new interfaces are being done through sysfs, that's simply a
> fact of life in the kernel.  If you want to buck this trend, you
> have to come up with a much better argument than "procfs is more
> straight-forward".  Well if sysfs isn't as straightforward, suggest
> a layer of interfaces that might make it so.

I was not aware that procfs was off limits.  Does sysfs exist in the
2.4 kernel?  (I'll also want to backport the redirect device code to
2.4, even if just for my own kernel builds.)

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[David S. Miller]

On Thu, 31 Mar 2005 14:42:54 -0800
Ben Greear <greearb@candelatech.com> wrote:

> Does sysfs exist in the 2.4 kernel?

No, it does not.

Re: RFC: Redirect-Device

[jamal]

On Thu, 2005-03-31 at 15:41, Ben Greear wrote:
> I created a new virtual ethernet device that solves a problem
> I faced.  I thought I'd see if anyone else sees a use for this,
> and if so, I'll work to polish the patch so that it can be
> accepted into the kernel.
> 
> The redirect device is fairly simple:
> 
> You create a pair of them, each associated with each other.
> When you transmit on one, the other one receives a packet.
> The devices are presented as regular ethernet devices, and they
> have MACs, will do ARP, and other such things.
> 
> I use these in the following manner:
> 
> I have a software bridge that works by registering a protocol for all
> packets on, for example, eth1, and it will forward these packets to another
> device.  In my case, the other device will be one of the re-direct interfaces
> (rdd0).  Both eth1 and rdd0 will NOT have any IP configuration.  This keeps
> the stacks from interfering with my bridge.  rdd1 will have the IP address that
> would normally be associated with eth1.
> 
> This allows me to have total control of the packets flowing between the physical
> ethernet interface and the ethernet and higher level protocols on the rdd1 interface.
> I believe this is the only way to have such total control of packets, for both inspection
> and arbitrary modification, before the packets hit the protocol handlers.
> 

I must be missing something: What is it that this device can do that the
mirred action cant do? 
Or in general the action framework on the ingress side?
We can redirect to any arbitrary device; we can mirror to any arbitray
device; we can drop, mangle packets identified via classification rules
in any arbitrary way etc


cheers,
jamal

Re: RFC: Redirect-Device

[Ben Greear]

jamal wrote:
> On Thu, 2005-03-31 at 15:41, Ben Greear wrote:

> I must be missing something: What is it that this device can do that the
> mirred action cant do? 
> Or in general the action framework on the ingress side?
> We can redirect to any arbitrary device; we can mirror to any arbitray
> device; we can drop, mangle packets identified via classification rules
> in any arbitrary way etc

I can operate on these devices with normal socket calls from user-space, and
can treat them as normal net_devices from kernel modules.  I do not have to
parse or manage the mirrored action logic, and I know that I absolutely have
total control over packets with my user-space language of choice.  (I am not
sure how easy it is to use your classification rules and mangling operations
in an arbitrary manner.)

I can also create a nice little set of virtual interfaces
and connections  rdd0 <-> rdd1  |bridge|  rdd2 <-> rdd3.  I can then send traffic
from rdd0 to rdd3 across the bridge, etc.  Now, this last bit is fairly
contrived, but it happens to help me with some testing on my laptop which
lacks a lot of external ethernet interfaces :)

To be honest, I didn't dig into the actions.  It would be much harder for
me to manage things in that manner, whereas virtual interfaces just work
for me.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[jamal]

On Thu, 2005-03-31 at 16:26, Ben Greear wrote:
> jamal wrote:
> > On Thu, 2005-03-31 at 15:41, Ben Greear wrote:
> 
> > I must be missing something: What is it that this device can do that the
> > mirred action cant do? 
> > Or in general the action framework on the ingress side?
> > We can redirect to any arbitrary device; we can mirror to any arbitray
> > device; we can drop, mangle packets identified via classification rules
> > in any arbitrary way etc
> 
> I can operate on these devices with normal socket calls from user-space, and
> can treat them as normal net_devices from kernel modules.  I do not have to
> parse or manage the mirrored action logic, and I know that I absolutely have
> total control over packets with my user-space language of choice.  (I am not
> sure how easy it is to use your classification rules and mangling operations
> in an arbitrary manner.)
> 

You can configure actions using sockets (netlink sockets).
Thats what tc does for example. As a matter of fact you already have a
"language" - its what tc is, sample BNF grammar of the language:

---
[root@jzny root]# tc
Usage: tc [ OPTIONS ] OBJECT { COMMAND | help }
where  OBJECT := { qdisc | class | filter }
       OPTIONS := { -s[tatistics] | -d[etails] | -r[aw] | -b[atch] file
}
---

You can write a little API on top of netlink that could ease the
intricacies of netlink. Thats what libnetlink in iproute2(that tc use)
and what Thomas is writting are attempting to do.

So - you have a lot of control ..

A few people have written "languages" to improve on tc. The most
prominent being Werner Almesbergers tcng http://tcng.sourceforge.net/

> I can also create a nice little set of virtual interfaces
> and connections  rdd0 <-> rdd1  |bridge|  rdd2 <-> rdd3.  I can then send traffic
> from rdd0 to rdd3 across the bridge, etc.  Now, this last bit is fairly
> contrived, but it happens to help me with some testing on my laptop which
> lacks a lot of external ethernet interfaces :)
> 

So your goal is to define a path that the packet takes inside the kernel
across multiple devices? i.e some form of loose source routing?

> To be honest, I didn't dig into the actions.  It would be much harder for
> me to manage things in that manner, whereas virtual interfaces just work
> for me.

Understood - just not empathized ;->

cheers,
jamal

Re: RFC: Redirect-Device

[Ben Greear]

jamal wrote:
> On Thu, 2005-03-31 at 16:26, Ben Greear wrote:

> You can configure actions using sockets (netlink sockets).
> Thats what tc does for example. As a matter of fact you already have a
> "language" - its what tc is, sample BNF grammar of the language:
> 
> ---
> [root@jzny root]# tc
> Usage: tc [ OPTIONS ] OBJECT { COMMAND | help }
> where  OBJECT := { qdisc | class | filter }
>        OPTIONS := { -s[tatistics] | -d[etails] | -r[aw] | -b[atch] file
> }

My personal opinion is that netlink sockets are a pain in the ass to deal
with, and there is no way I want to try to programatically parse the tc
input or output.

And probably not so easy to manipulate from a kernel module.

And BNF cannot be more powerful than a c/c++ program with a byte-buffer
representing the entire ethernet frame.

>>I can also create a nice little set of virtual interfaces
>>and connections  rdd0 <-> rdd1  |bridge|  rdd2 <-> rdd3.  I can then send traffic
>>from rdd0 to rdd3 across the bridge, etc.  Now, this last bit is fairly
>>contrived, but it happens to help me with some testing on my laptop which
>>lacks a lot of external ethernet interfaces :)
> 
> So your goal is to define a path that the packet takes inside the kernel
> across multiple devices? i.e some form of loose source routing?

Well, I have already hacked the kernel to allow sending traffic from eth0 to
eth3 across a loopback cable.  In my example above, you can think of
rdd0 - rdd3 as eth0 - eth3:  eth0 has an IP 10.1.1.2, connected to
eth1 with a loopback cable.  eth1 has NO ip and is bridged to eth2 in
software.  eth2 has NO ip and is connected via loopback cable to eth3.
eth3 has an IP 10.1.1.3.  As far as the networking protocol stacks are
concerned, eth0 is connected via loopback cable to eth3.

My primary goal is to enable this bridge to not require so many physical
ethernet ports.  This 'bridge' is actually my network emulator that I sell
for a living, so if I can cram twice as much functionality into the same number of
physical interfaces, I stand to gain.  Since I remove the IP addresses from the bridged interfaces,
I can do the bridging without hacking another hook into the dev.c pkt receive
routines, and since my redirect devices are net_devices sending & receiving
ethernet packets, they just work with my existing code, ethereal, and every
other tool I've tried.  Heck, I could probably run 802.1Q VLANs across them
as well if I wanted :)

>>To be honest, I didn't dig into the actions.  It would be much harder for
>>me to manage things in that manner, whereas virtual interfaces just work
>>for me.
> 
> 
> Understood - just not empathized ;->

Fair enough :)

Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[David S. Miller]

On Thu, 31 Mar 2005 14:22:11 -0800
Ben Greear <greearb@candelatech.com> wrote:

> > [root@jzny root]# tc
> > Usage: tc [ OPTIONS ] OBJECT { COMMAND | help }
> > where  OBJECT := { qdisc | class | filter }
> >        OPTIONS := { -s[tatistics] | -d[etails] | -r[aw] | -b[atch] file
> > }
> 
> My personal opinion is that netlink sockets are a pain in the ass to deal
> with, and there is no way I want to try to programatically parse the tc
> input or output.
> 
> And probably not so easy to manipulate from a kernel module.
> 
> And BNF cannot be more powerful than a c/c++ program with a byte-buffer
> representing the entire ethernet frame.

So you're not even going to give Jamal's suggestion a try?

If we have the infrastructure to do what you want, we should use it,
not add "yet another way" to do something we can do already.

If adding some clean abstraction layer helps out your cause, that's
fine too.  We could even hack tc to output things in a format that
you might find easier to parse.

'tc' is very powerful, and very shamefully under-utilizied.  You're task
seems the perfect match for it's use.

Re: RFC: Redirect-Device

[Ben Greear]

David S. Miller wrote:
> On Thu, 31 Mar 2005 14:22:11 -0800
> Ben Greear <greearb@candelatech.com> wrote:
> 
> 
>>>[root@jzny root]# tc
>>>Usage: tc [ OPTIONS ] OBJECT { COMMAND | help }
>>>where  OBJECT := { qdisc | class | filter }
>>>       OPTIONS := { -s[tatistics] | -d[etails] | -r[aw] | -b[atch] file
>>>}
>>
>>My personal opinion is that netlink sockets are a pain in the ass to deal
>>with, and there is no way I want to try to programatically parse the tc
>>input or output.
>>
>>And probably not so easy to manipulate from a kernel module.
>>
>>And BNF cannot be more powerful than a c/c++ program with a byte-buffer
>>representing the entire ethernet frame.
> 
> 
> So you're not even going to give Jamal's suggestion a try?

No.  I can't imagine a way to make it work with my application.

I don't think I can overstate the benefits I see from having
net_devices and their attending standard interfaces
to utilize in both kernel and user-space.

> If we have the infrastructure to do what you want, we should use it,
> not add "yet another way" to do something we can do already.
>
> If adding some clean abstraction layer helps out your cause, that's
> fine too.  We could even hack tc to output things in a format that
> you might find easier to parse.
> 
> 'tc' is very powerful, and very shamefully under-utilizied.  You're task
> seems the perfect match for it's use.

I obviously can't force you to accept the redirect module, so
if no one else sees any reason for it, then we can simply
drop the matter and I'll carry it in my own patch set like
I do my other stuff.  No hard feelings, and if someone decides
they could use something like it in the future, then perhaps
we can take another look at it.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[jamal]

On Thu, 2005-03-31 at 17:54, Ben Greear wrote:

> No.  I can't imagine a way to make it work with my application.
> 

I think you are more comfortable with using netdevices and ioctls and
/proc. 
If the action stuff cant do what you need i will make a donation
to the EFF on your behalf;->



> I obviously can't force you to accept the redirect module, so
> if no one else sees any reason for it, then we can simply
> drop the matter and I'll carry it in my own patch set like
> I do my other stuff.  No hard feelings, and if someone decides
> they could use something like it in the future, then perhaps
> we can take another look at it.
> 

Why dont we try to help you so you migrate from the approach you are
currently taking?

cheers,
jamal

Re: RFC: Redirect-Device

[Ben Greear]

jamal wrote:
> On Thu, 2005-03-31 at 17:54, Ben Greear wrote:
> 
> 
>>No.  I can't imagine a way to make it work with my application.
>>
> I think you are more comfortable with using netdevices and ioctls and
> /proc. 

Definately.  Ever tried to sniff a socket with ethereal? :)

> Why dont we try to help you so you migrate from the approach you are
> currently taking?

Our approaches to the problem are fundamentally different.  I choose
virtual net devices because all my user-space and GUI code that deals
with ethernet & .1q interfaces works with virtual devices.  AND, my
kernel module that can bridge at 990Mbps+ in both directions on a
couple of GigE interfaces also works with any ethernet-like net-device.
(Using raw sockets in user-space I cannot achieve this high of a performance.)

I can also stack virtual devices on other devices (.1q on re-direct devices,
for example).  You may very well be able to do all of these things with
tc, but it would not be worth my time to try to figure out how to change my
entire application over to support a new interface that *might* do what I
need when I can easily add a new virtual net-device that I have already proved
does what I need.

So, I'm not claiming that my code does what you want better than your
code, but I am quite certain that my code does what *I* want better
than your code.

Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[jamal]

On Thu, 2005-03-31 at 18:56, Ben Greear wrote:

> Our approaches to the problem are fundamentally different.  

The framework i described gives you the leggo to build the constructs
you want. Its like building a house using bricks. If you decide that you
need to build a house by moulding steel (because youve been using steel
for 80 years dammit and it works) - its your call.

OTOH, if you need help feel free to ask. Whether that code goes in is
really up to Dave. 

cheers,
jamal

Re: RFC: Redirect-Device

[bert hubert]

On Thu, Mar 31, 2005 at 03:56:28PM -0800, Ben Greear wrote:

> >I think you are more comfortable with using netdevices and ioctls and
> >/proc. 
> 
> Definately.  Ever tried to sniff a socket with ethereal? :)

On loopback, all the time. I'm probably dense but I don't understand what
problem you've solved with this interface. Could you elaborate a bit?

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

Re: RFC: Redirect-Device

[Ben Greear]

bert hubert wrote:
> On Thu, Mar 31, 2005 at 03:56:28PM -0800, Ben Greear wrote:
> 
> 
>>>I think you are more comfortable with using netdevices and ioctls and
>>>/proc. 
>>
>>Definately.  Ever tried to sniff a socket with ethereal? :)
> 
> 
> On loopback, all the time. I'm probably dense but I don't understand what
> problem you've solved with this interface. Could you elaborate a bit?

It allows me to place a software bridge that can intercept all packets
from user-space via raw packet sockets, and kernel space via registering
an 'all' protocol on the device.  Please note that to bridge in this manner
I have to remove the IP protocol (set IP to 0.0.0.0), otherwise the IP stack
can interfere with the bridging behaviour.

By using a virtual pair of interfaces that are looped back, I can add an
IP to the second virtual network interface that does not interfere with
the two bridged interfaces (one physical, one redirect, both with 0.0.0.0 IP
addresses).

If there were an API to register handlers dynamically that act like the
netpoll hook (ie, with ability to consume frames), then I would not have to
remove the IP from the physical interface and I probably would not have had
to create these redirect devices.  But, when I was suggesting such a hook
in the past, it was shot down because it could allow someone to write their
own TCP stack, and the network guys did not want to allow this possibility.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[Thomas Graf]

* Ben Greear <424C7813.4000101@candelatech.com> 2005-03-31 14:22
> My personal opinion is that netlink sockets are a pain in the ass to deal
> with, and there is no way I want to try to programatically parse the tc
> input or output.

libnl will make your life a lot easier, the tc support is not yet
finished but I'll be releasing a version this week which at least
fully support all the basics (link, neigh, address, routes, rules).
It's only a matter of time until one can use the tc interface with
only a few library calls.

Re: RFC: Redirect-Device

[Ben Greear]

Thomas Graf wrote:
> * Ben Greear <424C7813.4000101@candelatech.com> 2005-03-31 14:22
> 
>>My personal opinion is that netlink sockets are a pain in the ass to deal
>>with, and there is no way I want to try to programatically parse the tc
>>input or output.
> 
> 
> libnl will make your life a lot easier, the tc support is not yet
> finished but I'll be releasing a version this week which at least
> fully support all the basics (link, neigh, address, routes, rules).
> It's only a matter of time until one can use the tc interface with
> only a few library calls.

Sometime I will revisit netlink.  There are some things I want to do
like allowing more than 8 bits of routing tables, and perhaps watching
for network link up/down events and the like.  Please don't bust ass to support
tc on my account as it's quite unlikely I'll use it any time soon.

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[jamal]

On Thu, 2005-03-31 at 17:22, Ben Greear wrote:
> jamal wrote:
> > On Thu, 2005-03-31 at 16:26, Ben Greear wrote:

> 
> My personal opinion is that netlink sockets are a pain in the ass to deal
> with, and there is no way I want to try to programatically parse the tc
> input or output.
> 

Take a look at the libraries i mentioned. 

> And probably not so easy to manipulate from a kernel module.
> 
> And BNF cannot be more powerful than a c/c++ program with a byte-buffer
> representing the entire ethernet frame.
> 

For that level you write a program. In any language you want;->
I dont think you can beat the u32 classifier interface on how to
describe a packet. 

> >>I can also create a nice little set of virtual interfaces
> >>and connections  rdd0 <-> rdd1  |bridge|  rdd2 <-> rdd3.  I can then send traffic
> >>from rdd0 to rdd3 across the bridge, etc.  Now, this last bit is fairly
> >>contrived, but it happens to help me with some testing on my laptop which
> >>lacks a lot of external ethernet interfaces :)
> > 
> > So your goal is to define a path that the packet takes inside the kernel
> > across multiple devices? i.e some form of loose source routing?
> 
> Well, I have already hacked the kernel to allow sending traffic from eth0 to
> eth3 across a loopback cable.  In my example above, you can think of
> rdd0 - rdd3 as eth0 - eth3:  eth0 has an IP 10.1.1.2, connected to
> eth1 with a loopback cable.  eth1 has NO ip and is bridged to eth2 in
> software.  eth2 has NO ip and is connected via loopback cable to eth3.
> eth3 has an IP 10.1.1.3.  As far as the networking protocol stacks are
> concerned, eth0 is connected via loopback cable to eth3.
> 
>
> My primary goal is to enable this bridge to not require so many physical
> ethernet ports.  This 'bridge' is actually my network emulator that I sell
> for a living, so if I can cram twice as much functionality into the same number of
> physical interfaces, I stand to gain.  Since I remove the IP addresses from the bridged interfaces,
> I can do the bridging without hacking another hook into the dev.c pkt receive
> routines, and since my redirect devices are net_devices sending & receiving
> ethernet packets, they just work with my existing code, ethereal, and every
> other tool I've tried.  Heck, I could probably run 802.1Q VLANs across them
> as well if I wanted :)
> 

Like i said _all this_ is doable via actions. I dont know what your
bridge does but infact all the rest of the functionality exists.And your
bridge could be written as an action with very little code (and you get
the netlink interface for free. 
One thing you probably havent understood is that all the action stuff
that happens on ingress happens before dev.c pkt receive.

cheers,
jamal

Re: RFC: Redirect-Device

[Ben Greear]

jamal wrote:

> One thing you probably havent understood is that all the action stuff
> that happens on ingress happens before dev.c pkt receive.

Could you point me to where this is, or give me something to
search for?  I'm curious how/where it does the hook, and if I
understand that better, maybe I can start thinking about how
to make use of it for future hacks...

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[jamal]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

On Thu, 2005-03-31 at 18:35, Ben Greear wrote:
> jamal wrote:
> 
> > One thing you probably havent understood is that all the action stuff
> > that happens on ingress happens before dev.c pkt receive.
> 
> Could you point me to where this is, or give me something to
> search for?  I'm curious how/where it does the hook, and if I
> understand that better, maybe I can start thinking about how
> to make use of it for future hacks...

dev.c
search for CONFIG_NET_CLS_ACT

To see how simple an action can look like, take a look at
net/sched/gact.c (which does simple drop/accept etc); I have some
patches i need to submit to Dave that would make it even simpler to use
(and write less code). Attached is an example. 

Actually gact may have gotten a little bit complex because it now allows
you to add randomness to accepting, dropping, going to next action etc.

The one thing you have to understand is filters and actions are
separate. So what i am pointing to you is a simple action that is
executed after a packet matches a specified filter. Thomas has been
working on providing what are know as ematches which are very simple
filters that you could program.
You can match a packet and pass it through a series of actions of your
choice.

Also note: This stuff can be done at egress (not just restricted to
ingress).

cheers,
jamal

[-- Attachment #2: p16 --]
[-- Type: text/plain, Size: 2911 bytes --]

--- /dev/null	2004-01-29 13:33:32.773091056 -0500
+++ 2611-rc3+bk3/net/sched/simple.c	2005-02-06 15:55:57.000000000 -0500
@@ -0,0 +1,107 @@
+/*
+ * net/sched/simp.c	Simple example of an action
+ *
+ *		This program is free software; you can redistribute it and/or
+ *		modify it under the terms of the GNU General Public License
+ *		as published by the Free Software Foundation; either version
+ *		2 of the License, or (at your option) any later version.
+ *
+ * Authors:	Jamal Hadi Salim (2005)
+ *
+ */
+
+#include <asm/uaccess.h>
+#include <asm/system.h>
+#include <asm/bitops.h>
+#include <linux/config.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/string.h>
+#include <linux/mm.h>
+#include <linux/socket.h>
+#include <linux/sockios.h>
+#include <linux/in.h>
+#include <linux/errno.h>
+#include <linux/interrupt.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/rtnetlink.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/proc_fs.h>
+#include <net/sock.h>
+#include <net/pkt_sched.h>
+
+#define TCA_ACT_SIMP 22
+
+/* XXX: Hide all these common elements under some macro 
+ * probably
+*/
+#include <linux/tc_act/tc_defact.h>
+#include <net/tc_act/tc_defact.h>
+
+/* use generic hash table with 8 buckets */
+#define MY_TAB_SIZE     8
+#define MY_TAB_MASK     (MY_TAB_SIZE - 1)
+static u32 idx_gen;
+static struct tcf_defact *tcf_simp_ht[MY_TAB_SIZE];
+static DEFINE_RWLOCK(simp_lock);
+
+/* override the defaults */
+#define tcf_st		tcf_defact
+#define tc_st		tc_defact
+#define tcf_t_lock	simp_lock
+#define tcf_ht		tcf_simp_ht
+
+#define CONFIG_NET_ACT_INIT 1
+#include <net/pkt_act.h>
+#include <net/act_generic.h>
+
+static int tcf_simp(struct sk_buff **pskb, struct tc_action *a)
+{
+	struct sk_buff *skb = *pskb;
+	struct tcf_defact *p = PRIV(a, defact);
+
+	spin_lock(&p->lock);
+	p->tm.lastuse = jiffies;
+	p->bstats.bytes += skb->len;
+	p->bstats.packets++;
+
+	/* print policy string followed by _ then packet count 
+	 * Example if this was the 3rd packet and the string was "hello" 
+	 * then it would look like "hello_3" (without quotes) 
+	 **/
+	printk("simple: %s_%d\n", (char *)p->defdata, p->bstats.packets);
+	spin_unlock(&p->lock);
+	return p->action;
+}
+
+static struct tc_action_ops act_simp_ops = {
+	.kind = "simple",
+	.type = TCA_ACT_SIMP,
+	.capab = TCA_CAP_NONE,
+	.owner = THIS_MODULE,
+	.act = tcf_simp,
+	tca_use_default_ops
+};
+
+MODULE_AUTHOR("Jamal Hadi Salim(2005)");
+MODULE_DESCRIPTION("Simple example action");
+MODULE_LICENSE("GPL");
+
+static int __init simp_init_module(void)
+{
+	int ret = tcf_register_action(&act_simp_ops);
+	if (!ret)
+		printk("Simple TC action Loaded\n");
+	return ret;
+}
+
+static void __exit simp_cleanup_module(void)
+{
+	tcf_unregister_action(&act_simp_ops);
+}
+
+module_init(simp_init_module);
+module_exit(simp_cleanup_module);

Re: RFC: Redirect-Device

[Meelis Roos]

j> I must be missing something: What is it that this device can do that the
j> mirred action cant do? 

I know what I am missing here: documentation. There is very basic
documentation about tc qdisc+class+filter level and almost nothing on the
newer features. Without good documentation only some developers
understand it.

-- 
Meelis Roos

Re: RFC: Redirect-Device

[jamal]

 Sat, 2005-04-02 at 03:41, Meelis Roos wrote:
> j> I must be missing something: What is it that this device can do that the
> j> mirred action cant do? 
> 
> I know what I am missing here: documentation. There is very basic
> documentation about tc qdisc+class+filter level and almost nothing on the
> newer features. Without good documentation only some developers
> understand it.

Have you tried looking at iproute2 doc/examples? Theres some new stuff
in there. Over time more stuff will be added - and contributions 
welcome as well. 

cheers,
jamal

Re: RFC: Redirect-Device

[Pekka Savola]

On Thu, 31 Mar 2005, Ben Greear wrote:
> I created a new virtual ethernet device that solves a problem
> I faced.  I thought I'd see if anyone else sees a use for this,
> and if so, I'll work to polish the patch so that it can be
> accepted into the kernel.

Unless I'm misunderstanding..

You seem to have pretty much duplicated the work by Lennert (see 
thread "[PATCH][RFC] etherip: Ethernet-in-IPv4 tunneling" from 
January).

That particular thread seemed to conclude that instead of creating 
something new, you can just use GRE tunneling to bridge Ethernet, and 
it works in a multi-vendor environment as well.

Is there something in your problem statement I'm missing?

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Re: RFC: Redirect-Device

[Ben Greear]

Pekka Savola wrote:
> On Thu, 31 Mar 2005, Ben Greear wrote:
> 
>> I created a new virtual ethernet device that solves a problem
>> I faced.  I thought I'd see if anyone else sees a use for this,
>> and if so, I'll work to polish the patch so that it can be
>> accepted into the kernel.
> 
> 
> Unless I'm misunderstanding..
> 
> You seem to have pretty much duplicated the work by Lennert (see thread 
> "[PATCH][RFC] etherip: Ethernet-in-IPv4 tunneling" from January).
> 
> That particular thread seemed to conclude that instead of creating 
> something new, you can just use GRE tunneling to bridge Ethernet, and it 
> works in a multi-vendor environment as well.
> 
> Is there something in your problem statement I'm missing?

That would be similar to what I'm doing, but I'm not really trying
to tunnel anything.  I am trying to duplicate the behaviour of two
ethernet interfaces connected by an external cross-over cable, and I'm
trying to duplicate it at the network-device interface level so that
common tools (and my own tools) can treat these virtual interfaces
just like ethernet interfaces.

I think I may be the only one that needs such a thing, and since I'm
reluctant to remove the /proc interface, I don't think this will be
going into the official kernel soon.  If anyone does end up wanting
this later, I can email them my consolidated patch....

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: RFC: Redirect-Device

[Pekka Savola]

On Thu, 31 Mar 2005, Ben Greear wrote:
>> Is there something in your problem statement I'm missing?
>
> That would be similar to what I'm doing, but I'm not really trying
> to tunnel anything.  I am trying to duplicate the behaviour of two
> ethernet interfaces connected by an external cross-over cable, and I'm
> trying to duplicate it at the network-device interface level so that
> common tools (and my own tools) can treat these virtual interfaces
> just like ethernet interfaces.

Oh ok, what you seem to want is some kind of "Ethernet loopback++", 
but the "looped" packets should come back from a virtual interface 
instead of the same interface?

Btw, does the kernel support traditional loopback, so that at the last 
stage, just before sending a packet on the wire, it would be pushed 
back.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Re: RFC: Redirect-Device

[Ben Greear]

Pekka Savola wrote:
> On Thu, 31 Mar 2005, Ben Greear wrote:
> 
>>> Is there something in your problem statement I'm missing?
>>
>>
>> That would be similar to what I'm doing, but I'm not really trying
>> to tunnel anything.  I am trying to duplicate the behaviour of two
>> ethernet interfaces connected by an external cross-over cable, and I'm
>> trying to duplicate it at the network-device interface level so that
>> common tools (and my own tools) can treat these virtual interfaces
>> just like ethernet interfaces.
> 
> 
> Oh ok, what you seem to want is some kind of "Ethernet loopback++", but 
> the "looped" packets should come back from a virtual interface instead 
> of the same interface?

Yes.  In practice, I use a pair of virtual interfaces, so I send on one
virtual and receive on the other.  I use separate software to bridge,
or the normal linux stacks to route, the packets to other interfaces,
including real interfaces.

> Btw, does the kernel support traditional loopback, so that at the last 
> stage, just before sending a packet on the wire, it would be pushed back.

Not that I'm aware of.


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com