[ULOGD PATCH 0/8] Allow reuse of plugin instance (and misc fixes)

[ULOGD PATCH 0/8] Allow reuse of plugin instance (and misc fixes)

[Eric Leblond]

Hello,

This patchset starts with some cleaning:
 - Add hook to list of fields in database for packet logging.
 - Introduce RAWSTR type to avoid confusion in future developement.
 - Fix indentation in ulogd_inppkt_NFLOG.c

Latest patches introduce a modification of plugin behaviour. It was not
possible with existing code to reuse a plugin instance. For example, it
is not possible to do :
stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU
stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu2:LOGEMU

This kind of setup is useful to be able to log in a file and in a database for
example. To be able to have such a setup it is necessary:
 - Not to call twice or more start function to avoid problem (it is not possible
 to listen to the same nfgroup more than once)
 - Duplicate the message in the INPUT module.
This patchset contains ulogd necessary modifications and modifications of NFLOG,
ULOG and NFCT INPUT plugins.

The patches are :
 - Don't call start function multiple time for a single plugin instance.
 - Add a list of used plugininstance.
 - Duplicate message to all existing instance of NFLOG.
 - Add code for duplication of message in ULOG
 - Duplication of message in NFCT input plugin

Global stats:
 doc/mysql-ulogd2.sql              |   17 +++++++++++------
 doc/pgsql-ulogd2.sql              |   20 ++++++++++++--------
 filter/ulogd_filter_IP2BIN.c      |   12 ++++++------
 include/ulogd/ulogd.h             |    3 +++
 input/flow/ulogd_inpflow_NFCT.c   |    7 +++++++
 input/packet/ulogd_inppkt_NFLOG.c |   12 ++++++++++--
 input/packet/ulogd_inppkt_ULOG.c  |    5 +++++
 src/ulogd.c                       |   35 ++++++++++++++++++++++++++++++-----
 util/db.c                         |    5 ++++-
 9 files changed, 88 insertions(+), 28 deletions(-)

BR,
--
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/

[ULOGD PATCH 1/8] Add hook to list of fields in database for packet logging.

[Eric Leblond]

This patch adds oob.hook to the list of fields export to the databases. This
adds the capability to know where the packet has been logged and will be used
to make a link between connection and logged packets.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 doc/mysql-ulogd2.sql |   17 +++++++++++------
 doc/pgsql-ulogd2.sql |   20 ++++++++++++--------
 2 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/doc/mysql-ulogd2.sql b/doc/mysql-ulogd2.sql
index b154c74..1265eba 100644
--- a/doc/mysql-ulogd2.sql
+++ b/doc/mysql-ulogd2.sql
@@ -14,7 +14,7 @@ CREATE TABLE `_format` (
   `version` int(4) NOT NULL
 ) ENGINE=INNODB;
 
-INSERT INTO _format (version) VALUES (5);
+INSERT INTO _format (version) VALUES (6);
 
 -- this table could be used to know which user-defined tables are linked
 -- to ulog
@@ -42,6 +42,7 @@ CREATE TABLE `ulog2` (
   `_id` bigint unsigned NOT NULL auto_increment,
   `oob_time_sec` int(10) unsigned default NULL,
   `oob_time_usec` int(10) unsigned default NULL,
+  `oob_hook` tinyint(3) unsigned default NULL,
   `oob_prefix` varchar(32) default NULL,
   `oob_mark` int(10) unsigned default NULL,
   `oob_in` varchar(32) default NULL,
@@ -171,6 +172,7 @@ CREATE SQL SECURITY INVOKER VIEW `ulog` AS
         SELECT _id,
         oob_time_sec,
         oob_time_usec,
+        oob_hook,
         oob_prefix,
         oob_mark,
         oob_in,
@@ -417,6 +419,7 @@ DROP FUNCTION IF EXISTS INSERT_IP_PACKET;
 CREATE FUNCTION INSERT_IP_PACKET(
 		_oob_time_sec int(10) unsigned,
 		_oob_time_usec int(10) unsigned,
+		_oob_hook tinyint(3) unsigned,
 		_oob_prefix varchar(32),
 		_oob_mark int(10) unsigned,
 		_oob_in varchar(32),
@@ -430,9 +433,9 @@ SQL SECURITY INVOKER
 NOT DETERMINISTIC
 READS SQL DATA
 BEGIN
-	INSERT INTO ulog2 (oob_time_sec, oob_time_usec, oob_prefix, oob_mark, oob_in, oob_out, oob_family,
+	INSERT INTO ulog2 (oob_time_sec, oob_time_usec, oob_hook, oob_prefix, oob_mark, oob_in, oob_out, oob_family,
 			   ip_saddr, ip_daddr, ip_protocol) VALUES 
-		(_oob_time_sec, _oob_time_usec, _oob_prefix, _oob_mark, _oob_in, _oob_out, _oob_family,
+		(_oob_time_sec, _oob_time_usec, _oob_hook, _oob_prefix, _oob_mark, _oob_in, _oob_out, _oob_family,
 		 _ip_saddr, _ip_daddr, _ip_protocol);
 	RETURN LAST_INSERT_ID();
 END
@@ -443,6 +446,7 @@ DROP FUNCTION IF EXISTS INSERT_IP_PACKET_FULL;
 CREATE FUNCTION INSERT_IP_PACKET_FULL(
 		_oob_time_sec int(10) unsigned,
 		_oob_time_usec int(10) unsigned,
+		_oob_hook tinyint(3) unsigned,
 		_oob_prefix varchar(32),
 		_oob_mark int(10) unsigned,
 		_oob_in varchar(32),
@@ -463,10 +467,10 @@ SQL SECURITY INVOKER
 NOT DETERMINISTIC
 READS SQL DATA
 BEGIN
-	INSERT INTO ulog2 (oob_time_sec, oob_time_usec, oob_prefix, oob_mark, oob_in, oob_out, oob_family,
+	INSERT INTO ulog2 (oob_time_sec, oob_time_usec, oob_hook, oob_prefix, oob_mark, oob_in, oob_out, oob_family,
 			   ip_saddr, ip_daddr, ip_protocol, ip_tos, ip_ttl, ip_totlen, ip_ihl,
 		 	   ip_csum, ip_id, ip_fragoff ) VALUES 
-		(_oob_time_sec, _oob_time_usec, _oob_prefix, _oob_mark, _oob_in, _oob_out, _oob_family,
+		(_oob_time_sec, _oob_time_usec, _oob_hook, _oob_prefix, _oob_mark, _oob_in, _oob_out, _oob_family,
 		 _ip_saddr, _ip_daddr, _ip_protocol, _ip_tos, _ip_ttl, _ip_totlen, _ip_ihl,
 		 _ip_csum, _ip_id, _ip_fragoff);
 	RETURN LAST_INSERT_ID();
@@ -582,6 +586,7 @@ DROP FUNCTION IF EXISTS INSERT_PACKET_FULL;
 CREATE FUNCTION INSERT_PACKET_FULL(
 		_oob_time_sec int(10) unsigned,
 		_oob_time_usec int(10) unsigned,
+		_oob_hook tinyint(3) unsigned,
 		_oob_prefix varchar(32),
 		_oob_mark int(10) unsigned,
 		_oob_in varchar(32),
@@ -629,7 +634,7 @@ CREATE FUNCTION INSERT_PACKET_FULL(
 		) RETURNS bigint unsigned
 READS SQL DATA
 BEGIN
-	SET @lastid = INSERT_IP_PACKET_FULL(_oob_time_sec, _oob_time_usec, _oob_prefix,
+	SET @lastid = INSERT_IP_PACKET_FULL(_oob_time_sec, _oob_time_usec, _oob_hook, _oob_prefix,
 					   _oob_mark, _oob_in, _oob_out, _oob_family, 
 					   _ip_saddr, _ip_daddr, _ip_protocol, _ip_tos,
 					   _ip_ttl, _ip_totlen, _ip_ihl, _ip_csum, _ip_id,
diff --git a/doc/pgsql-ulogd2.sql b/doc/pgsql-ulogd2.sql
index 87a85fb..f4359ed 100644
--- a/doc/pgsql-ulogd2.sql
+++ b/doc/pgsql-ulogd2.sql
@@ -13,7 +13,7 @@ CREATE TABLE _format (
   version integer
 ) WITH (OIDS=FALSE);
 
-INSERT INTO _format (version) VALUES (5);
+INSERT INTO _format (version) VALUES (6);
 
 -- this table could be used to know which user-defined tables are linked
 -- to ulog
@@ -41,6 +41,7 @@ CREATE TABLE ulog2 (
   _id bigint PRIMARY KEY UNIQUE NOT NULL DEFAULT nextval('ulog2__id_seq'),
   oob_time_sec integer default NULL,
   oob_time_usec integer default NULL,
+  oob_hook smallint default NULL,
   oob_prefix varchar(32) default NULL,
   oob_mark integer default NULL,
   oob_in varchar(32) default NULL,
@@ -149,6 +150,7 @@ CREATE OR REPLACE VIEW ulog AS
         SELECT _id,
         oob_time_sec,
         oob_time_usec,
+        oob_hook,
         oob_prefix,
         oob_mark,
         oob_in,
@@ -343,6 +345,7 @@ $$ LANGUAGE SQL SECURITY INVOKER;
 CREATE OR REPLACE FUNCTION INSERT_IP_PACKET_FULL(
                 IN oob_time_sec integer,
                 IN oob_time_usec integer,
+                IN oob_hook integer,
                 IN oob_prefix varchar(32),
                 IN oob_mark integer,
                 IN oob_in varchar(32),
@@ -360,10 +363,10 @@ CREATE OR REPLACE FUNCTION INSERT_IP_PACKET_FULL(
                 IN ip_fragoff integer
         )
 RETURNS bigint AS $$
-        INSERT INTO ulog2 (oob_time_sec,oob_time_usec,oob_prefix,oob_mark,
+        INSERT INTO ulog2 (oob_time_sec,oob_time_usec,oob_hook,oob_prefix,oob_mark,
                         oob_in,oob_out,oob_family,ip_saddr_str,ip_daddr_str,ip_protocol,
                         ip_tos,ip_ttl,ip_totlen,ip_ihl,ip_csum,ip_id,ip_fragoff)
-                VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17);
+                VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18);
         SELECT currval('ulog2__id_seq');
 $$ LANGUAGE SQL SECURITY INVOKER;
 
@@ -436,6 +439,7 @@ $$ LANGUAGE SQL SECURITY INVOKER;
 CREATE OR REPLACE FUNCTION INSERT_PACKET_FULL(
                 IN oob_time_sec integer,
                 IN oob_time_usec integer,
+                IN oob_hook integer,
                 IN oob_prefix varchar(32),
                 IN oob_mark integer,
                 IN oob_in varchar(32),
@@ -482,15 +486,15 @@ RETURNS bigint AS $$
 DECLARE
         _id bigint;
 BEGIN
-        _id := INSERT_IP_PACKET_FULL($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17) ;
+        _id := INSERT_IP_PACKET_FULL($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18) ;
         IF (ip_protocol = 6) THEN
-                PERFORM INSERT_TCP_FULL(_id,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29);
+                PERFORM INSERT_TCP_FULL(_id,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30);
         ELSIF (ip_protocol = 17) THEN
-                PERFORM INSERT_UDP(_id,$30,$31,$32);
+                PERFORM INSERT_UDP(_id,$31,$32,$33);
         ELSIF (ip_protocol = 1) THEN
-                PERFORM INSERT_ICMP(_id,$33,$34,$35,$36,$37,$38);
+                PERFORM INSERT_ICMP(_id,$34,$35,$36,$37,$38,$39);
         ELSIF (ip_protocol = 58) THEN
-                PERFORM INSERT_ICMPV6(_id,$39,$40,$41,$42,$43);
+                PERFORM INSERT_ICMPV6(_id,$40,$41,$42,$43,$44);
         END IF;
         RETURN _id;
 END
-- 
1.5.4.3

Re: [ULOGD PATCH 1/8] Add hook to list of fields in database for packet logging.

[Pablo Neira Ayuso]

Eric Leblond wrote:
> This patch adds oob.hook to the list of fields export to the databases. This
> adds the capability to know where the packet has been logged and will be used
> to make a link between connection and logged packets.

Applied. Thanks.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 2/8] Introduce RAWSTR type to avoid confusion in future developement.

[Eric Leblond]

IP2BIN filter convert IP address from host storage to a "binary" string which
can be use by MySQL. This is not strictly speaking raw data but it was of type
RAW.

Following remark from Hugo Mildenberger, I introduce in this patch a dedicated
type ULOGD_RET_RAWSTR. The main reason not to use a ULOGD_RET_STRING parameter
is that the paramater is not human readable.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 filter/ulogd_filter_IP2BIN.c |   12 ++++++------
 include/ulogd/ulogd.h        |    1 +
 util/db.c                    |    5 ++++-
 3 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/filter/ulogd_filter_IP2BIN.c b/filter/ulogd_filter_IP2BIN.c
index 9b0110e..33db6a2 100644
--- a/filter/ulogd_filter_IP2BIN.c
+++ b/filter/ulogd_filter_IP2BIN.c
@@ -82,32 +82,32 @@ static struct ulogd_key ip2bin_inp[] = {
 
 static struct ulogd_key ip2bin_keys[] = {
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "ip.saddr.bin",
 	},
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "ip.daddr.bin",
 	},
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "orig.ip.saddr.bin",
 	},
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "orig.ip.daddr.bin",
 	},
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "reply.ip.saddr.bin",
 	},
 	{
-		.type = ULOGD_RET_RAW,
+		.type = ULOGD_RET_RAWSTR,
 		.flags = ULOGD_RETF_FREE,
 		.name = "reply.ip.daddr.bin",
 	},
diff --git a/include/ulogd/ulogd.h b/include/ulogd/ulogd.h
index 8c052f2..39ac464 100644
--- a/include/ulogd/ulogd.h
+++ b/include/ulogd/ulogd.h
@@ -47,6 +47,7 @@
 /* types with length field */
 #define ULOGD_RET_STRING	0x8020
 #define ULOGD_RET_RAW		0x8030
+#define ULOGD_RET_RAWSTR	0x8040
 
 
 /* FLAGS */
diff --git a/util/db.c b/util/db.c
index d57ab6a..49f6c29 100644
--- a/util/db.c
+++ b/util/db.c
@@ -294,9 +294,12 @@ static int __interp_db(struct ulogd_pluginstance *upi)
 			}
 			sprintf(di->stmt_ins, "',");
 			break;
-		case ULOGD_RET_RAW:
+		case ULOGD_RET_RAWSTR:
 			sprintf(di->stmt_ins, "%s,", res->u.value.ptr);
 			break;
+		case ULOGD_RET_RAW:
+			ulogd_log(ULOGD_NOTICE,
+				"Unsupported RAW type is unsupported in SQL output");
 		default:
 			ulogd_log(ULOGD_NOTICE,
 				"unknown type %d for %s\n",
-- 
1.5.4.3

Re: [ULOGD PATCH 2/8] Introduce RAWSTR type to avoid confusion in future developement.

[Pablo Neira Ayuso]

Eric Leblond wrote:
> IP2BIN filter convert IP address from host storage to a "binary" string which
> can be use by MySQL. This is not strictly speaking raw data but it was of type
> RAW.
> 
> Following remark from Hugo Mildenberger, I introduce in this patch a dedicated
> type ULOGD_RET_RAWSTR. The main reason not to use a ULOGD_RET_STRING parameter
> is that the paramater is not human readable.

Applied. Thanks.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 3/8] Fix indentation in ulogd_inppkt_NFLOG.c

[Eric Leblond]

Minor indentation fix in ulogd_inppkt_NFLOG.c.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 input/packet/ulogd_inppkt_NFLOG.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/input/packet/ulogd_inppkt_NFLOG.c b/input/packet/ulogd_inppkt_NFLOG.c
index 3f013cc..e0e8554 100644
--- a/input/packet/ulogd_inppkt_NFLOG.c
+++ b/input/packet/ulogd_inppkt_NFLOG.c
@@ -406,7 +406,7 @@ static int start(struct ulogd_pluginstance *upi)
 				    af_ce(upi->config_kset).u.value) < 0) {
 			ulogd_log(ULOGD_ERROR, "unable to force-unbind "
 				  "existing log handler for protocol %d\n",
-			  	  af_ce(upi->config_kset).u.value);
+				  af_ce(upi->config_kset).u.value);
 			goto out_handle;
 		}
 	}
-- 
1.5.4.3

Re: [ULOGD PATCH 3/8] Fix indentation in ulogd_inppkt_NFLOG.c

[Pablo Neira Ayuso]

Eric Leblond wrote:
> Minor indentation fix in ulogd_inppkt_NFLOG.c.

Applied. Thanks.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 4/8] Don't call start function multiple time for a single plugin instance.

[Eric Leblond]

When a plugin instance is used in multiple stack it is not necessary to
call the start function for each stack.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 src/ulogd.c |   33 ++++++++++++++++++++++++++++-----
 1 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/src/ulogd.c b/src/ulogd.c
index da821ee..b69d25d 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -710,6 +710,25 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
 	return 0;
 }
 
+/* iterate on already defined stack to find if a plugininstance
+ * has already been instance (matching is on id) */
+static int pluginstance_started(struct ulogd_pluginstance *npi)
+{
+	struct ulogd_pluginstance_stack *stack;
+	struct ulogd_pluginstance *pi;
+
+	llist_for_each_entry(stack, &ulogd_pi_stacks, stack_list) {
+		llist_for_each_entry(pi, &stack->list, list) {
+			/* OK, pluginstance already exists in stack list */
+			if (! strcmp(pi->id, npi->id)) {
+				ulogd_log(ULOGD_INFO, "%s instance already loaded\n", pi->id);
+				return 1;
+			}
+		}
+	}
+	return 0;
+}
+
 static int create_stack_start_instances(struct ulogd_pluginstance_stack *stack)
 {
 	int ret;
@@ -720,11 +739,15 @@ static int create_stack_start_instances(struct ulogd_pluginstance_stack *stack)
 		if (!pi->plugin->start)
 			continue;
 
-		ret = pi->plugin->start(pi);
-		if (ret < 0) {
-			ulogd_log(ULOGD_ERROR, "error during start of `%s'\n",
-				  pi->id);
-			return ret;
+		/* we only call start if a plugin with same ID has not yet be
+		 * started */
+		if (! pluginstance_started(pi)) {
+			ret = pi->plugin->start(pi);
+			if (ret < 0) {
+				ulogd_log(ULOGD_ERROR, "error during start of `%s'\n",
+						pi->id);
+				return ret;
+			}
 		}
 	}
 	return 0;
-- 
1.5.4.3

Re: [ULOGD PATCH 4/8] Don't call start function multiple time for a single plugin instance.

[Pablo Neira Ayuso]

Eric Leblond wrote:
> When a plugin instance is used in multiple stack it is not necessary to
> call the start function for each stack.

Also applied. Thanks.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 5/8] Add a list of used plugininstance.

[Eric Leblond]

This patch adds plist a linked list to the pluginstance
structure. It can be used by input modules to duplicate an
entry. This solves the issue of not being able to use the same
plugin instance twice.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 include/ulogd/ulogd.h |    2 ++
 src/ulogd.c           |    2 ++
 2 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/include/ulogd/ulogd.h b/include/ulogd/ulogd.h
index 39ac464..73a1711 100644
--- a/include/ulogd/ulogd.h
+++ b/include/ulogd/ulogd.h
@@ -168,6 +168,8 @@ struct ulogd_plugin {
 struct ulogd_pluginstance {
 	/* local list of plugins in this stack */
 	struct llist_head list;
+	/* local list of plugininstance in other stacks */
+	struct llist_head plist;
 	/* plugin */
 	struct ulogd_plugin *plugin;
 	/* stack that we're part of */
diff --git a/src/ulogd.c b/src/ulogd.c
index b69d25d..a6356f2 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -529,6 +529,7 @@ pluginstance_alloc_init(struct ulogd_plugin *pl, char *pi_id,
 	/* initialize */
 	memset(pi, 0, size);
 	INIT_LLIST_HEAD(&pi->list);
+	INIT_LLIST_HEAD(&pi->plist);
 	pi->plugin = pl;
 	pi->stack = stack;
 	memcpy(pi->id, pi_id, sizeof(pi->id));
@@ -722,6 +723,7 @@ static int pluginstance_started(struct ulogd_pluginstance *npi)
 			/* OK, pluginstance already exists in stack list */
 			if (! strcmp(pi->id, npi->id)) {
 				ulogd_log(ULOGD_INFO, "%s instance already loaded\n", pi->id);
+				llist_add(&pi->plist, &npi->plist);
 				return 1;
 			}
 		}
-- 
1.5.4.3

Re: [ULOGD PATCH 5/8] Add a list of used plugininstance.

[Pablo Neira Ayuso]

Eric Leblond wrote:
> This patch adds plist a linked list to the pluginstance
> structure. It can be used by input modules to duplicate an
> entry. This solves the issue of not being able to use the same
> plugin instance twice.

Applied. Thanks.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 6/8] Duplicate message to all existing instance of NFLOG.

[Eric Leblond]

A specific instance of NFLOG can now be use in multiple stacks. This is done
by duplicating the interpretation of the message.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 input/packet/ulogd_inppkt_NFLOG.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/input/packet/ulogd_inppkt_NFLOG.c b/input/packet/ulogd_inppkt_NFLOG.c
index e0e8554..166e048 100644
--- a/input/packet/ulogd_inppkt_NFLOG.c
+++ b/input/packet/ulogd_inppkt_NFLOG.c
@@ -370,7 +370,15 @@ static int msg_cb(struct nflog_g_handle *gh, struct nfgenmsg *nfmsg,
 		  struct nflog_data *nfa, void *data)
 {
 	struct ulogd_pluginstance *upi = data;
-
+	struct ulogd_pluginstance *npi = NULL;
+	int ret = 0;
+
+	/* ok let's feed the bird (other instance) */
+	llist_for_each_entry(npi, &upi->plist, plist) {
+		ret = interp_packet(npi, nfa);
+		if (ret != 0)
+			return ret;
+	}
 	return interp_packet(upi, nfa);
 }
 
-- 
1.5.4.3

Re: [ULOGD PATCH 6/8] Duplicate message to all existing instance of NFLOG.

[Pablo Neira Ayuso]

Eric Leblond wrote:
> A specific instance of NFLOG can now be use in multiple stacks. This is done
> by duplicating the interpretation of the message.

Applied. Thanks Eric.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 7/8] Add code for duplication of message in ULOG

[Eric Leblond]

In case of multiple usage of the same instance, this patch adds support for duplication
of the message to be able to have multiple use of the same ULOG instance.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 input/packet/ulogd_inppkt_ULOG.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/input/packet/ulogd_inppkt_ULOG.c b/input/packet/ulogd_inppkt_ULOG.c
index cf44474..c49345d 100644
--- a/input/packet/ulogd_inppkt_ULOG.c
+++ b/input/packet/ulogd_inppkt_ULOG.c
@@ -215,6 +215,7 @@ static int interp_packet(struct ulogd_pluginstance *ip, ulog_packet_msg_t *pkt)
 static int ulog_read_cb(int fd, unsigned int what, void *param)
 {
 	struct ulogd_pluginstance *upi = (struct ulogd_pluginstance *)param;
+	struct ulogd_pluginstance *npi = NULL;
 	struct ulog_input *u = (struct ulog_input *) &upi->private;
 	ulog_packet_msg_t *upkt;
 	int len;
@@ -239,6 +240,10 @@ static int ulog_read_cb(int fd, unsigned int what, void *param)
 		while ((upkt = ipulog_get_packet(u->libulog_h,
 						 u->libulog_buf, len))) {
 			ulogd_log(ULOGD_DEBUG, "==> ulog packet received\n");
+			/* ok let's feed the bird (other instance) */
+			llist_for_each_entry(npi, &upi->plist, plist) {
+				interp_packet(npi, upkt);
+			}
 			interp_packet(upi, upkt);
 		}
 	}
-- 
1.5.4.3

Re: [ULOGD PATCH 7/8] Add code for duplication of message in ULOG

[Pablo Neira Ayuso]

Eric Leblond wrote:
> In case of multiple usage of the same instance, this patch adds support for duplication
> of the message to be able to have multiple use of the same ULOG instance.

Applied. Thanks Eric. I have changed the comment on "feeding the birds"
whatever with one more descriptive :)

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

[ULOGD PATCH 8/8] Duplication of message in NFCT input plugin

[Eric Leblond]

This patch adds support for duplication of the message to be
able to use multiple time the same instance of NFCT.

Signed-off-by: Eric Leblond <eric@inl.fr>
---
 input/flow/ulogd_inpflow_NFCT.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/input/flow/ulogd_inpflow_NFCT.c b/input/flow/ulogd_inpflow_NFCT.c
index 7d3aa91..1843acb 100644
--- a/input/flow/ulogd_inpflow_NFCT.c
+++ b/input/flow/ulogd_inpflow_NFCT.c
@@ -559,8 +559,10 @@ static int event_handler(void *arg, unsigned int flags, int type,
 {
 	struct nfct_conntrack *ct = arg;
 	struct ulogd_pluginstance *upi = data;
+	struct ulogd_pluginstance *npi = NULL;
 	struct nfct_pluginstance *cpi = 
 				(struct nfct_pluginstance *) upi->private;
+	int ret = 0;
 
 	if (type == NFCT_MSG_NEW) {
 		if (usehash_ce(upi->config_kset).u.value != 0)
@@ -571,6 +573,11 @@ static int event_handler(void *arg, unsigned int flags, int type,
 		if (usehash_ce(upi->config_kset).u.value != 0)
 			ts = ct_hash_get(cpi->ct_active, ct->id);
 
+		llist_for_each_entry(npi, &upi->plist, plist) {
+			ret = propagate_ct(npi, ct, flags, ts);
+			if (ret != 0)
+				return ret;
+		}
 		return propagate_ct(upi, ct, flags, ts);
 	}
 	return 0;
-- 
1.5.4.3

Re: [ULOGD PATCH 8/8] Duplication of message in NFCT input plugin

[Pablo Neira Ayuso]

Eric Leblond wrote:
> This patch adds support for duplication of the message to be
> able to use multiple time the same instance of NFCT.

Applied. Thanks. I'll keep iterating over your patches along today.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [ULOGD PATCH 0/8] Allow reuse of plugin instance (and misc fixes)

[Pablo Neira Ayuso]

Eric Leblond wrote:
> Hello,
> 
> This patchset starts with some cleaning:
>  - Add hook to list of fields in database for packet logging.
>  - Introduce RAWSTR type to avoid confusion in future developement.
>  - Fix indentation in ulogd_inppkt_NFLOG.c
> 
> Latest patches introduce a modification of plugin behaviour. It was not
> possible with existing code to reuse a plugin instance. For example, it
> is not possible to do :
> stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU
> stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu2:LOGEMU

I'd appreciate a patch with a more realistic example that I can apply to
SVN.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [ULOGD PATCH 0/8] Allow reuse of plugin instance (and misc fixes)

[Eric Leblond]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

Hi,

On Tuesday, 2008 March 25 at 11:03:33 +0100, Pablo Neira Ayuso wrote:
> Eric Leblond wrote:
> > Hello,
> > 
> > This patchset starts with some cleaning:
> >  - Add hook to list of fields in database for packet logging.
> >  - Introduce RAWSTR type to avoid confusion in future developement.
> >  - Fix indentation in ulogd_inppkt_NFLOG.c
> > 
> > Latest patches introduce a modification of plugin behaviour. It was not
> > possible with existing code to reuse a plugin instance. For example, it
> > is not possible to do :
> > stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu1:LOGEMU
> > stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu2:LOGEMU
> 
> I'd appreciate a patch with a more realistic example that I can apply to
> SVN.

I've tried to add an example in the example configuration file but I've
found that some matching examples seems already available. Follow an excerpt of 
ulogd.conf.in (taken from my last patchset):

# this is a stack for NFLOG packet-based logging to PCAP
#stack=log1:NFLOG,base1:BASE,pcap1:PCAP

# this is a stack for logging packet to MySQL
#stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2bin1:IP2BIN,mac2str1:MAC2STR,mysql1:MYSQL

Furthermore, I think the behaviour obtained with that patchset is what users are
waiting from ulogd2. IMHO, it is not necessary to add long explanation
in the configuration file.

But, just to be sure it is written somewhere, I've added a more detailled explanation
of multiple usage of an instance in my ulogd configuration page:
	http://software.inl.fr//trac/wiki/ulogd2/user

BR,
-- 
Eric Leblond
INL: http://www.inl.fr/
NuFW: http://www.nufw.org/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]