[PATCH 0/1] libv4l: Add plugin support

[PATCH 0/1] libv4l: Add plugin support

[Yordan Kamenov]

Hi Hans,

Here is initial version of plugin support for libv4l, based on your RFC.

It is provided by functions v4l2_plugin_[open,close,etc]. When open() is
called libv4l dlopens files in /usr/lib/libv4l/plugins 1 at a time and call
open() callback passing through the applications parameters unmodified.
If a plugin is relevant for the specified device node, it can indicate so by
returning a value other then -1 (the actual file descriptor).

As soon as a plugin returns another value then -1 plugin loading stops and
information about it (fd and corresponding library handle) is stored.
For each function v4l2_[ioctl,read,close,etc] is called corresponding 
v4l2_plugin_* function which looks if there is loaded plugin for that file
and call it's callbacks. v4l2_plugin_* functions indicate by their first 
argument if plugin was used, and if it was not then v4l2_* functions proceed 
with their usual behavior.

Yordan Kamenov (1):
  Add plugin support to libv4l

 lib/include/libv4l2-plugin.h |   74 ++++++++
 lib/include/libv4l2.h        |   15 ++
 lib/libv4l2/Makefile         |    4 +-
 lib/libv4l2/libv4l2-priv.h   |    9 +
 lib/libv4l2/libv4l2.c        |   56 ++++++-
 lib/libv4l2/v4l2-plugin.c    |  399 ++++++++++++++++++++++++++++++++++++++++++
 lib/libv4l2/v4l2convert.c    |   20 ++-
 7 files changed, 568 insertions(+), 9 deletions(-)
 create mode 100644 lib/include/libv4l2-plugin.h
 create mode 100644 lib/libv4l2/v4l2-plugin.c

-- 
1.7.3.1

[PATCH 1/1] Add plugin support to libv4l

[Yordan Kamenov]

A libv4l2 plugin will sit in between libv4l2 itself and the
actual /dev/video device node a fd refers to. It will be called each time
libv4l2 wants to do an operation (read/write/ioctl/mmap/munmap) on the
actual /dev/video node in question.

Signed-off-by: Yordan Kamenov <ykamenov@mm-sol.com>
---
 lib/include/libv4l2-plugin.h |   74 ++++++++
 lib/include/libv4l2.h        |   15 ++
 lib/libv4l2/Makefile         |    4 +-
 lib/libv4l2/libv4l2-priv.h   |    9 +
 lib/libv4l2/libv4l2.c        |   56 ++++++-
 lib/libv4l2/v4l2-plugin.c    |  399 ++++++++++++++++++++++++++++++++++++++++++
 lib/libv4l2/v4l2convert.c    |   20 ++-
 7 files changed, 568 insertions(+), 9 deletions(-)
 create mode 100644 lib/include/libv4l2-plugin.h
 create mode 100644 lib/libv4l2/v4l2-plugin.c

diff --git a/lib/include/libv4l2-plugin.h b/lib/include/libv4l2-plugin.h
new file mode 100644
index 0000000..881b55d
--- /dev/null
+++ b/lib/include/libv4l2-plugin.h
@@ -0,0 +1,74 @@
+/*
+* Copyright (C) 2010 Nokia Corporation <multimedia@maemo.org>
+
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU Lesser General Public License as published by
+* the Free Software Foundation; either version 2.1 of the License, or
+* (at your option) any later version.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+* Lesser General Public License for more details.
+*
+* You should have received a copy of the GNU Lesser General Public License
+* along with this program; if not, write to the Free Software
+* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+
+#ifndef __LIBV4L2_PLUGIN_H
+#define __LIBV4L2_PLUGIN_H
+
+#include <sys/types.h>
+
+/* A libv4l2 plugin will sit in between libv4l2 itself and the
+   actual /dev/video device node a fd refers to. It will be called each time
+   libv4l2 wants to do an operation (read/write/ioctl/mmap/munmap) on the
+   actual /dev/video node in question. When called the plugin can then choose
+   to do one of the following:
+   1. Pass the call unmodified to the fd, and return the return value unmodifed
+   2. Modify some arguments in the call and pass it through
+   3. Modify the return(ed) value(s) of a passed through call
+   4. Not do any operation on the fd at all but instead completely fake it
+       (which opens the possibility for "fake" v4l devices)
+
+   libv4l2 plugins should *never* use any global variables. All data should be
+   bound to the specific fd to which the plugin is bound. This ensures that for
+   example a plugin for a specific type of usb webcam will also work when 2
+   identical cameras are plugged into a system (and both are used from the same
+   process).
+
+   A libv4l2 plugin can register plugin private data using:
+   void libv4l2_set_plugindata(int fd, void *plugin_data);
+
+   And can get this data out of libv4l2 again inside a callback using:
+   void *libv4l2_get_plugindata(int fd);
+
+   Note that a plugin should call libv4l2_set_plugindata only once per fd !
+   Calling it a second time will overwrite the previous value. The logical
+   place to use libv4l2_set_plugindata is from the plugin's open callback.
+*/
+
+/* Plugin callback function struct */
+struct libv4l2_plugin_data {
+	int (*open)(const char *file, int oflag, ...);
+	int (*close)(int fd);
+	int (*ioctl)(int fd, unsigned long int request, ...);
+	ssize_t (*read)(int fd, void *buffer, size_t n);
+	void *(*mmap)(void *start, size_t length, int prot, int flags,
+			int fd, int64_t offset);
+	/* Note as munmap has no fd argument, defining a callback for munmap
+		will result in it getting called for *any* call to v4l2_munmap.
+		So if a plugin defines a callback for munmap (because for
+		example it returns fake mmap buffers from its mmap callback).
+		Then it must keep track of the addresses at which these buffers
+		live and their size and check the munmap arguments to see if the
+		munmap call was meant for it. */
+	int (*munmap)(void *_start, size_t length);
+};
+
+/* Plugin utility functions */
+void libv4l2_set_plugindata(int fd, void *plugin_data);
+void *libv4l2_get_plugindata(int fd);
+
+#endif
diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
index cc0ab4a..2123546 100644
--- a/lib/include/libv4l2.h
+++ b/lib/include/libv4l2.h
@@ -22,6 +22,7 @@
 #include <stdio.h>
 #include <unistd.h>
 #include <stdint.h>
+#include "libv4l2-plugin.h"
 
 #ifdef __cplusplus
 extern "C" {
@@ -108,6 +109,20 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
    (note the fd is left open in this case). */
 LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
 
+
+LIBV4L_PUBLIC int v4l2_plugin_open(int *plugin_used, const char *file,
+					int oflag, ...);
+LIBV4L_PUBLIC int v4l2_plugin_close(int *plugin_used, int fd);
+LIBV4L_PUBLIC int v4l2_plugin_ioctl(int *plugin_used, int fd,
+					unsigned long int request, ...);
+LIBV4L_PUBLIC ssize_t v4l2_plugin_read(int *plugin_used, int fd, void *dest,
+					size_t n);
+LIBV4L_PUBLIC void *v4l2_plugin_mmap(int *plugin_used, void *start,
+					size_t length, int prot, int flags,
+					int fd, int64_t offset);
+LIBV4L_PUBLIC int v4l2_plugin_munmap(int *plugin_used, void *_start,
+					size_t length);
+
 #ifdef __cplusplus
 }
 #endif /* __cplusplus */
diff --git a/lib/libv4l2/Makefile b/lib/libv4l2/Makefile
index d78632f..eb1c019 100644
--- a/lib/libv4l2/Makefile
+++ b/lib/libv4l2/Makefile
@@ -1,8 +1,8 @@
 override CPPFLAGS += -I../include -fvisibility=hidden
 
-LIBS_libv4l2  = -lpthread
+LIBS_libv4l2  = -lpthread -ldl
 
-V4L2_OBJS     = libv4l2.o log.o
+V4L2_OBJS     = libv4l2.o v4l2-plugin.o log.o
 V4L2CONVERT   = v4l2convert.so
 V4L2CONVERT_O = v4l2convert.o libv4l2.so
 TARGETS       = $(V4L2_LIB) libv4l2.pc
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 46d6103..3873b1d 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -87,6 +87,15 @@ struct v4l2_dev_info {
 	unsigned char *readbuf;
 };
 
+struct v4l2_plugin_info {
+	int fd;
+	void *plugin_library;
+	struct libv4l2_plugin_data *libv4l2_plugin;
+	unsigned char *frame_pointers[V4L2_MAX_NO_FRAMES];
+	int frame_sizes[V4L2_MAX_NO_FRAMES];
+	void *plugin_data;
+};
+
 /* From log.c */
 void v4l2_log_ioctl(unsigned long int request, void *arg, int result);
 
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index ab85ea7..8696c68 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -67,6 +67,7 @@
 #include <sys/stat.h>
 #include "libv4l2.h"
 #include "libv4l2-priv.h"
+#include "libv4l2-plugin.h"
 
 /* Note these flags are stored together with the flags passed to v4l2_fd_open()
    in v4l2_dev_info's flags member, so care should be taken that the do not
@@ -522,7 +523,24 @@ static int v4l2_buffers_mapped(int index)
 
 int v4l2_open(const char *file, int oflag, ...)
 {
-	int fd;
+	int fd, plugin_used;
+
+	if (oflag & O_CREAT) {
+		va_list ap;
+		mode_t mode;
+
+		va_start(ap, oflag);
+		mode = va_arg(ap, mode_t);
+
+		fd = v4l2_plugin_open(&plugin_used, file, oflag, mode);
+
+		va_end(ap);
+	} else {
+		fd = v4l2_plugin_open(&plugin_used, file, oflag, 0);
+	}
+
+	if (plugin_used)
+		return fd;
 
 	/* original open code */
 	if (oflag & O_CREAT) {
@@ -684,7 +702,12 @@ static int v4l2_get_index(int fd)
 
 int v4l2_close(int fd)
 {
-	int index, result;
+	int index, result, plugin_used;
+
+	result = v4l2_plugin_close(&plugin_used, fd);
+
+	if (plugin_used)
+		return result;
 
 	index = v4l2_get_index(fd);
 	if (index == -1)
@@ -806,13 +829,18 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
 	va_list ap;
-	int result, index, saved_err;
+	int result, index, saved_err, plugin_used;
 	int is_capture_request = 0, stream_needs_locking = 0;
 
 	va_start(ap, request);
 	arg = va_arg(ap, void *);
 	va_end(ap);
 
+	result = v4l2_plugin_ioctl(&plugin_used, fd, request, arg);
+
+	if (plugin_used)
+		return result;
+
 	index = v4l2_get_index(fd);
 	if (index == -1)
 		return SYS_IOCTL(fd, request, arg);
@@ -1205,9 +1233,14 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 ssize_t v4l2_read(int fd, void *dest, size_t n)
 {
 	ssize_t result;
-	int saved_errno;
+	int saved_errno, plugin_used;
 	int index = v4l2_get_index(fd);
 
+	result = v4l2_plugin_read(&plugin_used, fd, dest, n);
+
+	if (plugin_used)
+		return result;
+
 	if (index == -1)
 		return SYS_READ(fd, dest, n);
 
@@ -1264,10 +1297,16 @@ leave:
 void *v4l2_mmap(void *start, size_t length, int prot, int flags, int fd,
 		int64_t offset)
 {
-	int index;
+	int index, plugin_used;
 	unsigned int buffer_index;
 	void *result;
 
+	result = v4l2_plugin_mmap(&plugin_used, start, length, prot,
+							flags, fd, offset);
+
+	if (plugin_used)
+		return result;
+
 	index = v4l2_get_index(fd);
 	if (index == -1 ||
 			/* Check if the mmap data matches our answer to QUERY_BUF, if it doesn't
@@ -1329,10 +1368,15 @@ leave:
 
 int v4l2_munmap(void *_start, size_t length)
 {
-	int index;
+	int index, result, plugin_used;
 	unsigned int buffer_index;
 	unsigned char *start = _start;
 
+	result = v4l2_plugin_munmap(&plugin_used, _start, length);
+
+	if (plugin_used)
+		return result;
+
 	/* Is this memory ours? */
 	if (start != MAP_FAILED && length == V4L2_FRAME_BUF_SIZE) {
 		for (index = 0; index < devices_used; index++)
diff --git a/lib/libv4l2/v4l2-plugin.c b/lib/libv4l2/v4l2-plugin.c
new file mode 100644
index 0000000..3efd533
--- /dev/null
+++ b/lib/libv4l2/v4l2-plugin.c
@@ -0,0 +1,399 @@
+/*
+* Copyright (C) 2010 Nokia Corporation <multimedia@maemo.org>
+
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU Lesser General Public License as published by
+* the Free Software Foundation; either version 2.1 of the License, or
+* (at your option) any later version.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+* Lesser General Public License for more details.
+*
+* You should have received a copy of the GNU Lesser General Public License
+* along with this program; if not, write to the Free Software
+* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+
+#include <stdarg.h>
+#include <dlfcn.h>
+#include <fcntl.h>
+#include <glob.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include "libv4l2.h"
+#include "libv4l2-priv.h"
+#include "libv4l2-plugin.h"
+
+/* libv4l plugin support:
+   it is provided by functions v4l2_plugin_[open,close,etc].
+
+   When open() is called libv4l dlopens files in /usr/lib[64]/libv4l/plugins
+   1 at a time and call open callback passing through the applications
+   parameters unmodified.
+
+   If a plugin is relevant for the specified device node, it can indicate so
+   by returning a value other then -1 (the actual file descriptor).
+   As soon as a plugin returns another value then -1 plugin loading stops and
+   information about it (fd and corresponding library handle) is stored. For
+   each function v4l2_[ioctl,read,close,etc] is called corresponding
+   v4l2_plugin_* function which looks if there is loaded plugin for that file
+   and call it's callbacks.
+
+   v4l2_plugin_* function indicates by it's first argument if plugin was used,
+   and if it was not then v4l2_* functions proceed with their usual behavior.
+*/
+
+#define PLUGINS_PATTERN "/usr/lib/libv4l/plugins/*.so"
+
+static pthread_mutex_t v4l2_plugin_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+static struct v4l2_plugin_info plugins[V4L2_MAX_DEVICES];
+
+int v4l2_plugin_open(int *plugin_used, const char *file, int oflag, ...)
+{
+	char *error;
+	int index, fd = -1, glob_ret, plugin_num, i;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+	glob_t globbuf;
+	static int structs_initialized = 0;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	if (!structs_initialized) {
+		for (index = 0; index < V4L2_MAX_DEVICES; index++)
+			plugins[index].fd = -1;
+
+		structs_initialized = 1;
+	}
+
+	/* Check if there is empty slot for plugin */
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == -1)
+			break;
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	if (index == V4L2_MAX_DEVICES) {
+		V4L2_LOG_ERR("attempting to open more than %d libv4l plugins\n",
+			V4L2_MAX_DEVICES);
+		*plugin_used = 0;
+		return -1;
+	}
+
+	glob_ret = glob(PLUGINS_PATTERN, 0, NULL, &globbuf);
+
+	if (glob_ret == GLOB_NOSPACE) {
+		*plugin_used = 0;
+		return -1;
+	}
+
+	if (glob_ret == GLOB_ABORTED || glob_ret == GLOB_NOMATCH) {
+		*plugin_used = 0;
+		goto leave;
+	}
+
+	for (plugin_num = 0; plugin_num < globbuf.gl_pathc; plugin_num++) {
+
+		V4L2_LOG("PLUGIN: dlopen(%s);\n", globbuf.gl_pathv[plugin_num]);
+
+		plugin_library = dlopen(globbuf.gl_pathv[plugin_num], RTLD_LAZY);
+
+		if (!plugin_library)
+			continue;
+
+		dlerror();    /* Clear any existing error */
+		libv4l2_plugin = (struct libv4l2_plugin_data *)
+					dlsym(plugin_library, "libv4l2_plugin");
+
+		error = dlerror();
+		if (error != NULL)  {
+			V4L2_LOG_ERR("PLUGIN: dlsym failed: %s\n", error);
+			dlclose(plugin_library);
+
+			continue;
+		}
+
+		if (libv4l2_plugin->open == NULL) {
+			fd = -1;
+		} else {
+			if (oflag & O_CREAT) {
+				va_list ap;
+				mode_t mode;
+
+				va_start(ap, oflag);
+				mode = va_arg(ap, mode_t);
+
+				fd = libv4l2_plugin->open(file, oflag, mode);
+
+				va_end(ap);
+			} else {
+				fd = libv4l2_plugin->open(file, oflag, 0);
+			}
+		}
+
+		if (fd != -1) {
+			V4L2_LOG("PLUGIN: plugin open() returned %d\n", fd);
+			pthread_mutex_lock(&v4l2_plugin_mutex);
+			/* The plugin may have called libv4l2_set_plugindata()
+			   and there is already reserved slot with that fd */
+			for (index = 0; index < V4L2_MAX_DEVICES; index++)
+				if (plugins[index].fd == fd)
+					break;
+
+			/* There is no such fd */
+			if (index == V4L2_MAX_DEVICES)
+				for (index = 0; index < V4L2_MAX_DEVICES; index++)
+					if (plugins[index].fd == -1)
+						break;
+
+			plugins[index].fd = fd;
+			plugins[index].plugin_library = plugin_library;
+			plugins[index].libv4l2_plugin = libv4l2_plugin;
+			for (i = 0; i < V4L2_MAX_NO_FRAMES; i++) {
+				plugins[index].frame_pointers[i] = MAP_FAILED;
+				plugins[index].frame_sizes[i] = 0;
+			}
+			pthread_mutex_unlock(&v4l2_plugin_mutex);
+			break;
+		} else {
+			V4L2_LOG("PLUGIN: plugin open() returned -1\n");
+			dlclose(plugin_library);
+			plugin_library = NULL;
+		}
+
+	}
+
+leave:
+	globfree(&globbuf);
+
+	if (fd == -1)
+		*plugin_used = 0;
+	else
+		*plugin_used = 1;
+
+	return fd;
+}
+
+int v4l2_plugin_close(int *plugin_used, int fd)
+{
+	int index, result = -1;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd) {
+			plugin_library = plugins[index].plugin_library;
+			libv4l2_plugin = plugins[index].libv4l2_plugin;
+			break;
+		}
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	if (index == V4L2_MAX_DEVICES) {
+		*plugin_used = 0;
+	} else {
+		if (libv4l2_plugin->close == NULL) {
+			*plugin_used = 0;
+		} else {
+			result = libv4l2_plugin->close(fd);
+			*plugin_used = 1;
+		}
+
+		dlclose(plugin_library);
+
+		pthread_mutex_lock(&v4l2_plugin_mutex);
+		plugins[index].fd = -1;
+		pthread_mutex_unlock(&v4l2_plugin_mutex);
+	}
+
+	return result;
+}
+
+int v4l2_plugin_ioctl(int *plugin_used, int fd, unsigned long int request, ...)
+{
+	void *arg;
+	va_list ap;
+	int index, result = -1;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd) {
+			plugin_library = plugins[index].plugin_library;
+			libv4l2_plugin = plugins[index].libv4l2_plugin;
+			break;
+		}
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	if (index == V4L2_MAX_DEVICES || libv4l2_plugin->ioctl == NULL) {
+		*plugin_used = 0;
+	} else {
+		va_start(ap, request);
+		arg = va_arg(ap, void *);
+		va_end(ap);
+
+		result = libv4l2_plugin->ioctl(fd, request, arg);
+		*plugin_used = 1;
+	}
+
+	return result;
+}
+
+ssize_t v4l2_plugin_read(int *plugin_used, int fd, void *dest, size_t n)
+{
+	int index, result = -1;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd) {
+			plugin_library = plugins[index].plugin_library;
+			libv4l2_plugin = plugins[index].libv4l2_plugin;
+			break;
+		}
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	if (index == V4L2_MAX_DEVICES || libv4l2_plugin->read == NULL) {
+		*plugin_used = 0;
+	} else {
+		result = libv4l2_plugin->read(fd, dest, n);
+		*plugin_used = 1;
+	}
+
+	return result;
+}
+
+void *v4l2_plugin_mmap(int *plugin_used, void *start, size_t length, int prot,
+					int flags, int fd, int64_t offset)
+{
+	int index, i;
+	void *result = NULL;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd) {
+			plugin_library = plugins[index].plugin_library;
+			libv4l2_plugin = plugins[index].libv4l2_plugin;
+			break;
+		}
+
+	if (fd == -1 || index == V4L2_MAX_DEVICES
+		|| libv4l2_plugin->mmap == NULL) {
+		*plugin_used = 0;
+	} else {
+		for (i = 0; i < V4L2_MAX_NO_FRAMES; i++)
+			if (plugins[index].frame_pointers[i] == MAP_FAILED)
+				break;
+
+		if (i == V4L2_MAX_NO_FRAMES) {
+			*plugin_used = 0;
+			result = NULL;
+			goto leave;
+		}
+
+		result = libv4l2_plugin->mmap(start, length, prot,
+						flags, fd, offset);
+		if (result) {
+			plugins[index].frame_pointers[i] = result;
+			plugins[index].frame_sizes[i] = length;
+			*plugin_used = 1;
+		} else {
+			*plugin_used = 0;
+		}
+	}
+
+leave:
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	return result;
+}
+
+int v4l2_plugin_munmap(int *plugin_used, void *_start, size_t length)
+{
+	int index, map, result = 0;
+	void *plugin_library = NULL;
+	struct libv4l2_plugin_data *libv4l2_plugin = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++) {
+		for (map = 0; map < V4L2_MAX_NO_FRAMES; map++) {
+			if (plugins[index].frame_pointers[map] == _start &&
+				plugins[index].frame_sizes[map] == length) {
+
+				plugin_library = plugins[index].plugin_library;
+				libv4l2_plugin = plugins[index].libv4l2_plugin;
+				plugins[index].frame_pointers[map] = MAP_FAILED;
+				plugins[index].frame_sizes[map] = 0;
+				break;
+			}
+		}
+		if (plugin_library)
+			break;
+	}
+
+	if (plugin_library) {
+		result = libv4l2_plugin->munmap(_start, length);
+
+		if (result)
+			*plugin_used = 0;
+		else
+			*plugin_used = 1;
+
+	} else {
+		*plugin_used = 0;
+	}
+
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	return result;
+}
+
+LIBV4L_PUBLIC void libv4l2_set_plugindata(int fd, void *plugin_data)
+{
+	int index;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd)
+			break;
+
+	/* We have no info about this fd - reserve an empty slot */
+	if (index == V4L2_MAX_DEVICES) {
+		for (index = 0; index < V4L2_MAX_DEVICES; index++)
+			if (plugins[index].fd == -1)
+				break;
+	}
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	if (index == V4L2_MAX_DEVICES) {
+		V4L2_LOG_ERR("store private data for more than %d plugins\n",
+			V4L2_MAX_DEVICES);
+		return;
+	}
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	plugins[index].fd = fd;
+	plugins[index].plugin_data = plugin_data;
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+}
+
+LIBV4L_PUBLIC void *libv4l2_get_plugindata(int fd)
+{
+	int index;
+	void *result = NULL;
+
+	pthread_mutex_lock(&v4l2_plugin_mutex);
+	for (index = 0; index < V4L2_MAX_DEVICES; index++)
+		if (plugins[index].fd == fd) {
+			result = plugins[index].plugin_data;
+			break;
+		}
+	pthread_mutex_unlock(&v4l2_plugin_mutex);
+
+	return result;
+}
diff --git a/lib/libv4l2/v4l2convert.c b/lib/libv4l2/v4l2convert.c
index e251085..9f28490 100644
--- a/lib/libv4l2/v4l2convert.c
+++ b/lib/libv4l2/v4l2convert.c
@@ -31,6 +31,7 @@
 #include "../libv4lconvert/libv4lsyscall-priv.h"
 #include <linux/videodev2.h>
 #include <libv4l2.h>
+#include "libv4l2-plugin.h"
 
 /* Check that open/read/mmap is not a define */
 #if defined open || defined read || defined mmap
@@ -45,10 +46,27 @@
 
 LIBV4L_PUBLIC int open(const char *file, int oflag, ...)
 {
-	int fd;
+	int fd, plugin_used;
 	struct v4l2_capability cap;
 	int v4l_device = 0;
 
+	if (oflag & O_CREAT) {
+		va_list ap;
+		mode_t mode;
+
+		va_start(ap, oflag);
+		mode = va_arg(ap, mode_t);
+
+		fd = v4l2_plugin_open(&plugin_used, file, oflag, mode);
+
+		va_end(ap);
+	} else {
+		fd = v4l2_plugin_open(&plugin_used, file, oflag, 0);
+	}
+
+	if (plugin_used)
+		return fd;
+
 	/* check if we're opening a video4linux2 device */
 	if (!strncmp(file, "/dev/video", 10) || !strncmp(file, "/dev/v4l/", 9)) {
 		/* Some apps open the device read only, but we need rw rights as the
-- 
1.7.3.1

Re: [PATCH 1/1] Add plugin support to libv4l

[Hans de Goede]

Hi,

First of all many thanks for working on this! I've several remarks
which I would like to see addressed before merging this.

Since most remarks are rather high level remarks I've opted to
just make a bulleted list of them rather then inserting them inline.

* The biggest problem with your current implementation is that for
each existing libv4l2_foo function you check if there is a plugin attached
to the fd passed in and if that plugin wants to handle the call. Now lets
assume that there is a plugin and that it wants to handle all calls. That
means that you've now effectively replaced all libv4l2_foo calls
with calling the corresponding foo function from the plugin and returning
its result. This means that for this fd / device you've achieved the
same result as completely replacing libv4l2.so.0 with a new library
containing the plugin code.

IOW you've not placed then plugin between libv4l2 and the device (as
intended) but completely short-circuited / replaced libv4l2. This means
for example for a device which only supports yuv output, that libv4l2 will
no longer do format emulation and conversion and an app which only supports
devices which deliver rgb data will no longer work.

To actually place the plugin between libv4l2 (and libv4lconvert) and the
device, you should replace all the SYS_FOO calls in libv4l2. The SYS_FOO
calls are the calls to the actual device, so be replacing those with calls
to the plugin you actual place the plugin between libv4l and the device as
intended.

* Currently you add a loop much like the one in the v4l2_get_index
function to each libv4l2_plugin function. Basically you add an array of
v4l2_plugin_info structs in libv4l2-plugin. Which gets searched by fd,
much like the v4l2_dev_info struct array. Including needing similar
locking. I would like you to instead just store the plugin info for
a certain fd directly into the v4l2_dev_info struct. This way the
separate array, looping and locking can go away.

* Next I would also like to see all the libv4l2_plugin_foo functions
except for libv4l2_plugin_open go away. Instead libv4l2.c can call
the plugin functions directly. Let me try to explain what I have in
mind. Lets say we store the struct libv4l2_plugin_data pointer to the
active plugin in the v4l2_dev_info struct and name it dev_ops
(short for device operations).

Then we can replace all SYS_FOO calls inside libv4l2 (except the ones
were v4l2_get_index returns -1), with a call to the relevant devop
functions, for example:
                 result = SYS_IOCTL(devices[index].fd, VIDIOC_REQBUFS, req);
Would become:
                 result = devices[index].dev_ops->v4l2_plugin_ioctl(
                                     devices[index].fd, VIDIOC_REQBUFS, req);

Note that the plugin_used parameter of the v4l2_plugin_ioctl is gone,
it should simply do a normal SYS_IOCTL and return the return value
of that if it is not interested in intercepting the ioctl (you could move
the definition of the SYS_FOO macros to libv4l2-plugin.h to make them
availables to plugins).

Also I think it would be better to rename the function pointers inside
the libv4l2_plugin_data struct from v4l2_plugin_foo to just foo, so
that the above code would become:
                 result = devices[index].dev_ops->v4l2_plugin_ioctl(
                                     devices[index].fd, VIDIOC_REQBUFS, req);

* The above means that need to always have a dev_ops pointer, so we
need to have a default_dev_ops struct to use when no plugin wants to
talk to the device.

* You've put the v4l2_plugin_foo functions (of which only
v4l2_plugin_foo will remain in my vision) in lib/include/libv4l2.h
I don't think these functions should be public, their prototypes should
be moved to lib/libv4l2/libv4l2-priv.h, and they should not be declared
LIBV4L_PUBLIC.

* There is one special case in all this, files under libv4lconvert also
use SYS_IOCTL in various places. Since this now need to go through the
plugin we need to take some special measures here. There are 2 options:
1) Break the libv4lconvert ABI (very few programs use it) and pass a
    struct libv4l2_plugin_data pointer to the v4lconvert_create function.
    *And* export the default_dev_ops struct from libv4l2.
2) Add a libv4l2_raw_ioctl method, which just gets the index and then
    does devices[index].dev_ops->v4l2_plugin_ioctl
    Except that this is not really an option as libv4lconvert should not
    depend on libv4l2
My vote personally goes to 1.

* I think that once we do 1) from above it would be good to rename
libv4l2_plugin_data to libv4l2_dev_ops, as that makes the public API
more clear and dev_ops is in essence what a plugin provides.

* Note that were I wrote: "like to see all the libv4l2_plugin_foo
functions except for libv4l2_plugin_open go away" I did so for
simplicity, in reality the wrappers around mmap and munmap need to
stay too, but they should use data directly stored inside the
v4l2_dev_info struct. This means that we need to either:
mv the mmap and munmap code to libv4l2.c; or export the v4l2_dev_info
struct array. I vote for exporting the v4l2_dev_info struct array
(through libv4l2-priv.h, so it won't be visible to the outside
world, but it will be usable outside libv4l2.c).

Thanks & Regards,

Hans

Re: [PATCH 1/1] Add plugin support to libv4l

[Yordan Kamenov]

Hi Hans,

Thanks for your comments.

Hans de Goede wrote:
> Hi,
>
> First of all many thanks for working on this! I've several remarks
> which I would like to see addressed before merging this.
>
> Since most remarks are rather high level remarks I've opted to
> just make a bulleted list of them rather then inserting them inline.
>
> * The biggest problem with your current implementation is that for
> each existing libv4l2_foo function you check if there is a plugin 
> attached
> to the fd passed in and if that plugin wants to handle the call. Now lets
> assume that there is a plugin and that it wants to handle all calls. That
> means that you've now effectively replaced all libv4l2_foo calls
> with calling the corresponding foo function from the plugin and returning
> its result. This means that for this fd / device you've achieved the
> same result as completely replacing libv4l2.so.0 with a new library
> containing the plugin code.
>
> IOW you've not placed then plugin between libv4l2 and the device (as
> intended) but completely short-circuited / replaced libv4l2. This means
> for example for a device which only supports yuv output, that libv4l2 
> will
> no longer do format emulation and conversion and an app which only 
> supports
> devices which deliver rgb data will no longer work.
>
> To actually place the plugin between libv4l2 (and libv4lconvert) and the
> device, you should replace all the SYS_FOO calls in libv4l2. The SYS_FOO
> calls are the calls to the actual device, so be replacing those with 
> calls
> to the plugin you actual place the plugin between libv4l and the 
> device as
> intended.
I agree with that, currently the plugin can replace the libv4l2, but if
we replace SYS_FOO calls it will actually sit between library and the
video node. I will do that.
>
> * Currently you add a loop much like the one in the v4l2_get_index
> function to each libv4l2_plugin function. Basically you add an array of
> v4l2_plugin_info structs in libv4l2-plugin. Which gets searched by fd,
> much like the v4l2_dev_info struct array. Including needing similar
> locking. I would like you to instead just store the plugin info for
> a certain fd directly into the v4l2_dev_info struct. This way the
> separate array, looping and locking can go away.
I have put separate array, because the array of v4l2_dev_info is declared
static in libv4l2.c and is not visible to v4l2-plugin.c (I did't want to
change it's declaration). But with changes that you suggest below, there
is no problem to add plugin data to v4l2_dev_info.
>
> * Next I would also like to see all the libv4l2_plugin_foo functions
> except for libv4l2_plugin_open go away. Instead libv4l2.c can call
> the plugin functions directly. Let me try to explain what I have in
> mind. Lets say we store the struct libv4l2_plugin_data pointer to the
> active plugin in the v4l2_dev_info struct and name it dev_ops
> (short for device operations).
>
> Then we can replace all SYS_FOO calls inside libv4l2 (except the ones
> were v4l2_get_index returns -1), with a call to the relevant devop
> functions, for example:
>                 result = SYS_IOCTL(devices[index].fd, VIDIOC_REQBUFS, 
> req);
> Would become:
>                 result = devices[index].dev_ops->v4l2_plugin_ioctl(
>                                     devices[index].fd, VIDIOC_REQBUFS, 
> req);
>
> Note that the plugin_used parameter of the v4l2_plugin_ioctl is gone,
> it should simply do a normal SYS_IOCTL and return the return value
> of that if it is not interested in intercepting the ioctl (you could move
> the definition of the SYS_FOO macros to libv4l2-plugin.h to make them
> availables to plugins).
>
> Also I think it would be better to rename the function pointers inside
> the libv4l2_plugin_data struct from v4l2_plugin_foo to just foo, so
> that the above code would become:
>                 result = devices[index].dev_ops->v4l2_plugin_ioctl(
>                                     devices[index].fd, VIDIOC_REQBUFS, 
> req);
>
> * The above means that need to always have a dev_ops pointer, so we
> need to have a default_dev_ops struct to use when no plugin wants to
> talk to the device.
Ok, I will replace SYS_FOO calls with dev_ops structure.
>
> * You've put the v4l2_plugin_foo functions (of which only
> v4l2_plugin_foo will remain in my vision) in lib/include/libv4l2.h
> I don't think these functions should be public, their prototypes should
> be moved to lib/libv4l2/libv4l2-priv.h, and they should not be declared
> LIBV4L_PUBLIC.
>
> * There is one special case in all this, files under libv4lconvert also
> use SYS_IOCTL in various places. Since this now need to go through the
> plugin we need to take some special measures here. There are 2 options:
> 1) Break the libv4lconvert ABI (very few programs use it) and pass a
>    struct libv4l2_plugin_data pointer to the v4lconvert_create function.
>    *And* export the default_dev_ops struct from libv4l2.
> 2) Add a libv4l2_raw_ioctl method, which just gets the index and then
>    does devices[index].dev_ops->v4l2_plugin_ioctl
>    Except that this is not really an option as libv4lconvert should not
>    depend on libv4l2
> My vote personally goes to 1.
>
> * I think that once we do 1) from above it would be good to rename
> libv4l2_plugin_data to libv4l2_dev_ops, as that makes the public API
> more clear and dev_ops is in essence what a plugin provides.
>
> * Note that were I wrote: "like to see all the libv4l2_plugin_foo
> functions except for libv4l2_plugin_open go away" I did so for
> simplicity, in reality the wrappers around mmap and munmap need to
> stay too, but they should use data directly stored inside the
> v4l2_dev_info struct. This means that we need to either:
> mv the mmap and munmap code to libv4l2.c; or export the v4l2_dev_info
> struct array. I vote for exporting the v4l2_dev_info struct array
> (through libv4l2-priv.h, so it won't be visible to the outside
> world, but it will be usable outside libv4l2.c).
Ok, I will make 1) and export v4l2_dev_info.
>
> Thanks & Regards,
>
> Hans
>
Best Regards
Yordan