[PATCH v4 0/3] Media Device Allocator API

[PATCH v4 0/3] Media Device Allocator API

[Shuah Khan]

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Patches 0001 and 0002 are rebased to 4.9-rc4. Patch 0003 for snd-usb-audio
is a rebase of the patch that was tested with the original Media Device
Allocator patch series.

snd-usb-audio patch includes the fixes found during 4.7-rc1 time in the
original snd-usb-audio patch.

Changes to patch 0001 since v3:
- Fixed undefined reference to `__media_device_usb_init compile error when
  CONFIG_USB is disabled.
- Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
  module that owns the media_device. The fix bumps the reference count for
  the owner when second driver comes along to share the media_device. If
  au0828 owns the media_device, then snd_usb_audio will bump the refcount
  for au0828, so it won't get deleted and vice versa.

Changes to patch 0002 since v2:
- Updated media_device_delete() to pass in module name.

Changes to patch 0003 since the last version in 4.7-rc1:
- Included fixes to bugs found during testing. 
- Updated to use the Media Allocator API.

This patch series has been tested with au0828 and snd-usb-audio drivers.
Ran bind and unbind loop tests on each driver with mc_nextgen_test and
media_device_test app loop tests while checking lsmod and dmesg.

Please refer to tools/testing/selftests/media_tests/regression_test.txt
for testing done on this series.

Shuah Khan (3):
  media: Media Device Allocator API
  media: change au0828 to use Media Device Allocator API
  sound/usb: Use Media Controller API to share media resources

 drivers/media/Makefile                 |   3 +-
 drivers/media/media-dev-allocator.c    | 146 +++++++++++++++
 drivers/media/usb/au0828/au0828-core.c |  12 +-
 drivers/media/usb/au0828/au0828.h      |   1 +
 include/media/media-dev-allocator.h    |  87 +++++++++
 sound/usb/Kconfig                      |   4 +
 sound/usb/Makefile                     |   2 +
 sound/usb/card.c                       |  14 ++
 sound/usb/card.h                       |   3 +
 sound/usb/media.c                      | 314 +++++++++++++++++++++++++++++++++
 sound/usb/media.h                      |  73 ++++++++
 sound/usb/mixer.h                      |   3 +
 sound/usb/pcm.c                        |  28 ++-
 sound/usb/quirks-table.h               |   1 +
 sound/usb/stream.c                     |   2 +
 sound/usb/usbaudio.h                   |   6 +
 16 files changed, 685 insertions(+), 14 deletions(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

-- 
2.7.4

[PATCH v4 1/3] media: Media Device Allocator API

[Shuah Khan]

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
Changes since v3:
- Fixed undefined reference to `__media_device_usb_init compile error when
  CONFIG_USB is disabled.
- Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
  module that owns the media_device. The fix bumps the reference count for
  the owner when second driver comes along to share the media_device. If
  au0828 owns the media_device, then snd_usb_audio will bump the refcount
  for au0828, so it won't get deleted and vice versa.

 drivers/media/Makefile              |   3 +-
 drivers/media/media-dev-allocator.c | 146 ++++++++++++++++++++++++++++++++++++
 include/media/media-dev-allocator.h |  87 +++++++++++++++++++++
 3 files changed, 235 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h

diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 0deaa93..7c0701d 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -6,7 +6,8 @@ ifeq ($(CONFIG_MEDIA_CEC_EDID),y)
   obj-$(CONFIG_MEDIA_SUPPORT) += cec-edid.o
 endif
 
-media-objs	:= media-device.o media-devnode.o media-entity.o
+media-objs	:= media-device.o media-devnode.o media-entity.o \
+		   media-dev-allocator.o
 
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
new file mode 100644
index 0000000..014a317
--- /dev/null
+++ b/drivers/media/media-dev-allocator.c
@@ -0,0 +1,146 @@
+/*
+ * media-dev-allocator.c - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global refcounted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ *
+ */
+
+#include <linux/slab.h>
+#include <linux/kref.h>
+#include <linux/usb.h>
+#include <linux/module.h>
+#include <media/media-device.h>
+
+static LIST_HEAD(media_device_list);
+static DEFINE_MUTEX(media_device_lock);
+
+struct media_device_instance {
+	struct media_device mdev;
+	struct module *owner;
+	struct list_head list;
+	struct device *dev;
+	struct kref refcount;
+};
+
+static inline struct media_device_instance *
+to_media_device_instance(struct media_device *mdev)
+{
+	return container_of(mdev, struct media_device_instance, mdev);
+}
+
+static void media_device_instance_release(struct kref *kref)
+{
+	struct media_device_instance *mdi =
+		container_of(kref, struct media_device_instance, refcount);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
+
+	mutex_lock(&media_device_lock);
+
+	media_device_unregister(&mdi->mdev);
+	media_device_cleanup(&mdi->mdev);
+
+	list_del(&mdi->list);
+	mutex_unlock(&media_device_lock);
+
+	kfree(mdi);
+}
+
+/* Callers should hold media_device_lock when calling this function */
+static struct media_device *__media_device_get(struct device *dev,
+					       char *module_name)
+{
+	struct media_device_instance *mdi;
+
+	list_for_each_entry(mdi, &media_device_list, list) {
+		if (mdi->dev == dev) {
+			kref_get(&mdi->refcount);
+			/* get module reference for the media_device owner */
+			if (find_module(module_name) != mdi->owner &&
+			    !try_module_get(mdi->owner))
+				dev_err(dev, "%s: try_module_get() error\n",
+					__func__);
+			dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
+				__func__, &mdi->mdev, module_name);
+			return &mdi->mdev;
+		}
+	}
+
+	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
+	if (!mdi)
+		return NULL;
+
+	mdi->dev = dev;
+	mdi->owner = find_module(module_name);
+	kref_init(&mdi->refcount);
+	list_add_tail(&mdi->list, &media_device_list);
+
+	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
+		&mdi->mdev, module_name);
+	return &mdi->mdev;
+}
+
+#if IS_ENABLED(CONFIG_USB)
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name)
+{
+	struct media_device *mdev;
+
+	if (!module_name) {
+		dev_err(&udev->dev, "%s Module Name is null\n", __func__);
+		return ERR_PTR(-EINVAL);
+	}
+
+	mutex_lock(&media_device_lock);
+	mdev = __media_device_get(&udev->dev, module_name);
+	if (!mdev) {
+		mutex_unlock(&media_device_lock);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	/* check if media device is already initialized */
+	if (!mdev->dev)
+		__media_device_usb_init(mdev, udev, udev->product,
+					module_name);
+	mutex_unlock(&media_device_lock);
+
+	dev_dbg(&udev->dev, "%s\n", __func__);
+	return mdev;
+}
+EXPORT_SYMBOL_GPL(media_device_usb_allocate);
+#endif
+
+void media_device_delete(struct media_device *mdev, char *module_name)
+{
+	struct media_device_instance *mdi = to_media_device_instance(mdev);
+
+	if (!module_name) {
+		dev_err(mdi->mdev.dev, "%s Module Name is null\n", __func__);
+		return;
+	}
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
+		__func__, &mdi->mdev, module_name);
+
+	mutex_lock(&media_device_lock);
+	/* put module reference if media_device owner is not THIS_MODULE */
+	if (mdi->owner != find_module(module_name)) {
+		module_put(mdi->owner);
+		dev_dbg(mdi->mdev.dev,
+			"%s decremented owner module reference\n", __func__);
+	}
+	mutex_unlock(&media_device_lock);
+	kref_put(&mdi->refcount, media_device_instance_release);
+}
+EXPORT_SYMBOL_GPL(media_device_delete);
diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
new file mode 100644
index 0000000..45c437d
--- /dev/null
+++ b/include/media/media-dev-allocator.h
@@ -0,0 +1,87 @@
+/*
+ * media-dev-allocator.h - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global ref-counted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ */
+
+#ifndef _MEDIA_DEV_ALLOCTOR_H
+#define _MEDIA_DEV_ALLOCTOR_H
+
+struct usb_device;
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+/**
+ * DOC: Media Controller Device Allocator API
+ *
+ * When media device belongs to more than one driver, the shared media device
+ * is allocated with the shared struct device as the key for look ups.
+ *
+ * Shared media device should stay in registered state until the last driver
+ * unregisters it. In addition, media device should be released when all the
+ * references are released. Each driver gets a reference to the media device
+ * during probe, when it allocates the media device. If media device is already
+ * allocated, allocate API bumps up the refcount and return the existing media
+ * device. Driver puts the reference back from its disconnect routine when it
+ * calls media_device_delete().
+ *
+ * Media device is unregistered and cleaned up from the kref put handler to
+ * ensure that the media device stays in registered state until the last driver
+ * unregisters the media device.
+ *
+ * Driver Usage:
+ *
+ * Drivers should use the media-core routines to get register reference and
+ * call media_device_delete() routine to make sure the shared media device
+ * delete is handled correctly.
+ *
+ * driver probe:
+ *	Call media_device_usb_allocate() to allocate or get a reference
+ *	Call media_device_register(), if media devnode isn't registered
+ *
+ * driver disconnect:
+ *	Call media_device_delete() to free the media_device. Free'ing is
+ *	handled by the put handler.
+ *
+ */
+
+/**
+ * media_device_usb_allocate() - Allocate and return media device
+ *
+ * @udev - struct usb_device pointer
+ * @module_name
+ *
+ * This interface should be called to allocate a media device when multiple
+ * drivers share usb_device and the media device. This interface allocates
+ * media device and calls media_device_usb_init() to initialize it.
+ *
+ */
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name);
+/**
+ * media_device_delete() - Release media device. Calls kref_put().
+ *
+ * @mdev - struct media_device pointer
+ * @module_name
+ *
+ * This interface should be called to put Media Device Instance kref.
+ */
+void media_device_delete(struct media_device *mdev, char *driver_name);
+#else
+static inline struct media_device *media_device_usb_allocate(
+			struct usb_device *udev, char *driver_name)
+			{ return NULL; }
+static inline void media_device_delete(
+			struct media_device *mdev, char *module_name) { }
+#endif /* CONFIG_MEDIA_CONTROLLER */
+#endif
-- 
2.7.4

Re: [PATCH v4 1/3] media: Media Device Allocator API

[Shuah Khan]

Sorry for the duplicates. Replying to catch bounced emails on the original
patch emails from git send-email

On 11/16/2016 07:29 AM, Shuah Khan wrote:
> Media Device Allocator API to allows multiple drivers share a media device.
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
> Changes since v3:
> - Fixed undefined reference to `__media_device_usb_init compile error when
>   CONFIG_USB is disabled.
> - Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
>   module that owns the media_device. The fix bumps the reference count for
>   the owner when second driver comes along to share the media_device. If
>   au0828 owns the media_device, then snd_usb_audio will bump the refcount
>   for au0828, so it won't get deleted and vice versa.
> 
>  drivers/media/Makefile              |   3 +-
>  drivers/media/media-dev-allocator.c | 146 ++++++++++++++++++++++++++++++++++++
>  include/media/media-dev-allocator.h |  87 +++++++++++++++++++++
>  3 files changed, 235 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/media-dev-allocator.c
>  create mode 100644 include/media/media-dev-allocator.h
> 
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index 0deaa93..7c0701d 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -6,7 +6,8 @@ ifeq ($(CONFIG_MEDIA_CEC_EDID),y)
>    obj-$(CONFIG_MEDIA_SUPPORT) += cec-edid.o
>  endif
>  
> -media-objs	:= media-device.o media-devnode.o media-entity.o
> +media-objs	:= media-device.o media-devnode.o media-entity.o \
> +		   media-dev-allocator.o
>  
>  #
>  # I2C drivers should come before other drivers, otherwise they'll fail
> diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
> new file mode 100644
> index 0000000..014a317
> --- /dev/null
> +++ b/drivers/media/media-dev-allocator.c
> @@ -0,0 +1,146 @@
> +/*
> + * media-dev-allocator.c - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global refcounted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + *
> + */
> +
> +#include <linux/slab.h>
> +#include <linux/kref.h>
> +#include <linux/usb.h>
> +#include <linux/module.h>
> +#include <media/media-device.h>
> +
> +static LIST_HEAD(media_device_list);
> +static DEFINE_MUTEX(media_device_lock);
> +
> +struct media_device_instance {
> +	struct media_device mdev;
> +	struct module *owner;
> +	struct list_head list;
> +	struct device *dev;
> +	struct kref refcount;
> +};
> +
> +static inline struct media_device_instance *
> +to_media_device_instance(struct media_device *mdev)
> +{
> +	return container_of(mdev, struct media_device_instance, mdev);
> +}
> +
> +static void media_device_instance_release(struct kref *kref)
> +{
> +	struct media_device_instance *mdi =
> +		container_of(kref, struct media_device_instance, refcount);
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
> +
> +	mutex_lock(&media_device_lock);
> +
> +	media_device_unregister(&mdi->mdev);
> +	media_device_cleanup(&mdi->mdev);
> +
> +	list_del(&mdi->list);
> +	mutex_unlock(&media_device_lock);
> +
> +	kfree(mdi);
> +}
> +
> +/* Callers should hold media_device_lock when calling this function */
> +static struct media_device *__media_device_get(struct device *dev,
> +					       char *module_name)
> +{
> +	struct media_device_instance *mdi;
> +
> +	list_for_each_entry(mdi, &media_device_list, list) {
> +		if (mdi->dev == dev) {
> +			kref_get(&mdi->refcount);
> +			/* get module reference for the media_device owner */
> +			if (find_module(module_name) != mdi->owner &&
> +			    !try_module_get(mdi->owner))
> +				dev_err(dev, "%s: try_module_get() error\n",
> +					__func__);
> +			dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
> +				__func__, &mdi->mdev, module_name);
> +			return &mdi->mdev;
> +		}
> +	}
> +
> +	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
> +	if (!mdi)
> +		return NULL;
> +
> +	mdi->dev = dev;
> +	mdi->owner = find_module(module_name);
> +	kref_init(&mdi->refcount);
> +	list_add_tail(&mdi->list, &media_device_list);
> +
> +	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
> +		&mdi->mdev, module_name);
> +	return &mdi->mdev;
> +}
> +
> +#if IS_ENABLED(CONFIG_USB)
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name)
> +{
> +	struct media_device *mdev;
> +
> +	if (!module_name) {
> +		dev_err(&udev->dev, "%s Module Name is null\n", __func__);
> +		return ERR_PTR(-EINVAL);
> +	}
> +
> +	mutex_lock(&media_device_lock);
> +	mdev = __media_device_get(&udev->dev, module_name);
> +	if (!mdev) {
> +		mutex_unlock(&media_device_lock);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	/* check if media device is already initialized */
> +	if (!mdev->dev)
> +		__media_device_usb_init(mdev, udev, udev->product,
> +					module_name);
> +	mutex_unlock(&media_device_lock);
> +
> +	dev_dbg(&udev->dev, "%s\n", __func__);
> +	return mdev;
> +}
> +EXPORT_SYMBOL_GPL(media_device_usb_allocate);
> +#endif
> +
> +void media_device_delete(struct media_device *mdev, char *module_name)
> +{
> +	struct media_device_instance *mdi = to_media_device_instance(mdev);
> +
> +	if (!module_name) {
> +		dev_err(mdi->mdev.dev, "%s Module Name is null\n", __func__);
> +		return;
> +	}
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
> +		__func__, &mdi->mdev, module_name);
> +
> +	mutex_lock(&media_device_lock);
> +	/* put module reference if media_device owner is not THIS_MODULE */
> +	if (mdi->owner != find_module(module_name)) {
> +		module_put(mdi->owner);
> +		dev_dbg(mdi->mdev.dev,
> +			"%s decremented owner module reference\n", __func__);
> +	}
> +	mutex_unlock(&media_device_lock);
> +	kref_put(&mdi->refcount, media_device_instance_release);
> +}
> +EXPORT_SYMBOL_GPL(media_device_delete);
> diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
> new file mode 100644
> index 0000000..45c437d
> --- /dev/null
> +++ b/include/media/media-dev-allocator.h
> @@ -0,0 +1,87 @@
> +/*
> + * media-dev-allocator.h - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global ref-counted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + */
> +
> +#ifndef _MEDIA_DEV_ALLOCTOR_H
> +#define _MEDIA_DEV_ALLOCTOR_H
> +
> +struct usb_device;
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +/**
> + * DOC: Media Controller Device Allocator API
> + *
> + * When media device belongs to more than one driver, the shared media device
> + * is allocated with the shared struct device as the key for look ups.
> + *
> + * Shared media device should stay in registered state until the last driver
> + * unregisters it. In addition, media device should be released when all the
> + * references are released. Each driver gets a reference to the media device
> + * during probe, when it allocates the media device. If media device is already
> + * allocated, allocate API bumps up the refcount and return the existing media
> + * device. Driver puts the reference back from its disconnect routine when it
> + * calls media_device_delete().
> + *
> + * Media device is unregistered and cleaned up from the kref put handler to
> + * ensure that the media device stays in registered state until the last driver
> + * unregisters the media device.
> + *
> + * Driver Usage:
> + *
> + * Drivers should use the media-core routines to get register reference and
> + * call media_device_delete() routine to make sure the shared media device
> + * delete is handled correctly.
> + *
> + * driver probe:
> + *	Call media_device_usb_allocate() to allocate or get a reference
> + *	Call media_device_register(), if media devnode isn't registered
> + *
> + * driver disconnect:
> + *	Call media_device_delete() to free the media_device. Free'ing is
> + *	handled by the put handler.
> + *
> + */
> +
> +/**
> + * media_device_usb_allocate() - Allocate and return media device
> + *
> + * @udev - struct usb_device pointer
> + * @module_name
> + *
> + * This interface should be called to allocate a media device when multiple
> + * drivers share usb_device and the media device. This interface allocates
> + * media device and calls media_device_usb_init() to initialize it.
> + *
> + */
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name);
> +/**
> + * media_device_delete() - Release media device. Calls kref_put().
> + *
> + * @mdev - struct media_device pointer
> + * @module_name
> + *
> + * This interface should be called to put Media Device Instance kref.
> + */
> +void media_device_delete(struct media_device *mdev, char *driver_name);
> +#else
> +static inline struct media_device *media_device_usb_allocate(
> +			struct usb_device *udev, char *driver_name)
> +			{ return NULL; }
> +static inline void media_device_delete(
> +			struct media_device *mdev, char *module_name) { }
> +#endif /* CONFIG_MEDIA_CONTROLLER */
> +#endif
> 

Re: [PATCH v4 1/3] media: Media Device Allocator API

[Sakari Ailus]

Hi Shuah,

On Wed, Nov 16, 2016 at 07:29:09AM -0700, Shuah Khan wrote:
> Media Device Allocator API to allows multiple drivers share a media device.
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
> Changes since v3:
> - Fixed undefined reference to `__media_device_usb_init compile error when
>   CONFIG_USB is disabled.
> - Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
>   module that owns the media_device. The fix bumps the reference count for
>   the owner when second driver comes along to share the media_device. If
>   au0828 owns the media_device, then snd_usb_audio will bump the refcount
>   for au0828, so it won't get deleted and vice versa.
> 
>  drivers/media/Makefile              |   3 +-
>  drivers/media/media-dev-allocator.c | 146 ++++++++++++++++++++++++++++++++++++
>  include/media/media-dev-allocator.h |  87 +++++++++++++++++++++
>  3 files changed, 235 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/media-dev-allocator.c
>  create mode 100644 include/media/media-dev-allocator.h
> 
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index 0deaa93..7c0701d 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -6,7 +6,8 @@ ifeq ($(CONFIG_MEDIA_CEC_EDID),y)
>    obj-$(CONFIG_MEDIA_SUPPORT) += cec-edid.o
>  endif
>  
> -media-objs	:= media-device.o media-devnode.o media-entity.o
> +media-objs	:= media-device.o media-devnode.o media-entity.o \
> +		   media-dev-allocator.o
>  
>  #
>  # I2C drivers should come before other drivers, otherwise they'll fail
> diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
> new file mode 100644
> index 0000000..014a317
> --- /dev/null
> +++ b/drivers/media/media-dev-allocator.c
> @@ -0,0 +1,146 @@
> +/*
> + * media-dev-allocator.c - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global refcounted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + *
> + */
> +
> +#include <linux/slab.h>
> +#include <linux/kref.h>
> +#include <linux/usb.h>
> +#include <linux/module.h>
> +#include <media/media-device.h>

Alphabetical order, please.

> +
> +static LIST_HEAD(media_device_list);
> +static DEFINE_MUTEX(media_device_lock);
> +
> +struct media_device_instance {
> +	struct media_device mdev;
> +	struct module *owner;
> +	struct list_head list;
> +	struct device *dev;

We've got a struct device pointer in struct media_device. Should we have the
same here, or use the value in struct media_device?

> +	struct kref refcount;
> +};
> +
> +static inline struct media_device_instance *
> +to_media_device_instance(struct media_device *mdev)
> +{
> +	return container_of(mdev, struct media_device_instance, mdev);
> +}
> +
> +static void media_device_instance_release(struct kref *kref)
> +{
> +	struct media_device_instance *mdi =
> +		container_of(kref, struct media_device_instance, refcount);
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
> +
> +	mutex_lock(&media_device_lock);

Do you need the lock during unregister / cleanup? Isn't it just to serialise
the access to the linked list?

> +
> +	media_device_unregister(&mdi->mdev);
> +	media_device_cleanup(&mdi->mdev);
> +
> +	list_del(&mdi->list);
> +	mutex_unlock(&media_device_lock);
> +
> +	kfree(mdi);
> +}
> +
> +/* Callers should hold media_device_lock when calling this function */
> +static struct media_device *__media_device_get(struct device *dev,
> +					       char *module_name)
> +{
> +	struct media_device_instance *mdi;
> +
> +	list_for_each_entry(mdi, &media_device_list, list) {
> +		if (mdi->dev == dev) {

You could make this look nicer if you continue here if mdi->dev != dev.

> +			kref_get(&mdi->refcount);
> +			/* get module reference for the media_device owner */
> +			if (find_module(module_name) != mdi->owner &&
> +			    !try_module_get(mdi->owner))
> +				dev_err(dev, "%s: try_module_get() error\n",
> +					__func__);
> +			dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
> +				__func__, &mdi->mdev, module_name);
> +			return &mdi->mdev;
> +		}
> +	}
> +
> +	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
> +	if (!mdi)
> +		return NULL;
> +
> +	mdi->dev = dev;
> +	mdi->owner = find_module(module_name);
> +	kref_init(&mdi->refcount);
> +	list_add_tail(&mdi->list, &media_device_list);
> +
> +	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,

Does the pointer value provide useful information here? The media device is
specific to a device and there's just one per device. The device name is
already printed anyway.

> +		&mdi->mdev, module_name);
> +	return &mdi->mdev;
> +}
> +
> +#if IS_ENABLED(CONFIG_USB)
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name)

How about adding a macro bearing this name, and using KBUILD_MODNAME there?
It'd look cleaner in drivers. That's what e.g. media_device_register() does,
for instance.

> +{
> +	struct media_device *mdev;
> +
> +	if (!module_name) {
> +		dev_err(&udev->dev, "%s Module Name is null\n", __func__);

You use colon (":") elsewhere after the function name, how about here as
well? "Name" could begin with lowercase letter, too.

> +		return ERR_PTR(-EINVAL);
> +	}
> +
> +	mutex_lock(&media_device_lock);
> +	mdev = __media_device_get(&udev->dev, module_name);
> +	if (!mdev) {
> +		mutex_unlock(&media_device_lock);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	/* check if media device is already initialized */
> +	if (!mdev->dev)

I think I would look up a device explicitly, and allocate one if needed.
This would mean splitting __media_device_get(), you could return the device
early if found, and then initialise without a need to check anything.

> +		__media_device_usb_init(mdev, udev, udev->product,
> +					module_name);

When is the media device registered and by which party?

I guess you can register entities once the media device is registered.
Nothing I can think of should prevent that, but I don't think it's been done
before. There could be bugs.

> +	mutex_unlock(&media_device_lock);
> +
> +	dev_dbg(&udev->dev, "%s\n", __func__);

This was probably nice to have during development time but should be removed
now.

> +	return mdev;
> +}
> +EXPORT_SYMBOL_GPL(media_device_usb_allocate);
> +#endif
> +
> +void media_device_delete(struct media_device *mdev, char *module_name)
> +{
> +	struct media_device_instance *mdi = to_media_device_instance(mdev);
> +
> +	if (!module_name) {
> +		dev_err(mdi->mdev.dev, "%s Module Name is null\n", __func__);

Can this happen? If so, why? What if the driver is linked to the kernel
instead?

> +		return;
> +	}
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
> +		__func__, &mdi->mdev, module_name);
> +
> +	mutex_lock(&media_device_lock);
> +	/* put module reference if media_device owner is not THIS_MODULE */
> +	if (mdi->owner != find_module(module_name)) {
> +		module_put(mdi->owner);
> +		dev_dbg(mdi->mdev.dev,
> +			"%s decremented owner module reference\n", __func__);
> +	}
> +	mutex_unlock(&media_device_lock);
> +	kref_put(&mdi->refcount, media_device_instance_release);
> +}
> +EXPORT_SYMBOL_GPL(media_device_delete);
> diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
> new file mode 100644
> index 0000000..45c437d
> --- /dev/null
> +++ b/include/media/media-dev-allocator.h
> @@ -0,0 +1,87 @@
> +/*
> + * media-dev-allocator.h - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global ref-counted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + */
> +
> +#ifndef _MEDIA_DEV_ALLOCTOR_H
> +#define _MEDIA_DEV_ALLOCTOR_H
> +
> +struct usb_device;
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +/**
> + * DOC: Media Controller Device Allocator API
> + *
> + * When media device belongs to more than one driver, the shared media device
> + * is allocated with the shared struct device as the key for look ups.
> + *
> + * Shared media device should stay in registered state until the last driver
> + * unregisters it. In addition, media device should be released when all the

The last or the first?

Presumably any driver that acquires the media device this way can create
entities, and these entities would be removed when that driver releases the
media device. We don't refcount graph objects at the moment, so if these
objects are referred to elsewhere in the kernel while that driver goes away,
bad things will follow.

It's not the only place this kind of problems exist at the moment though, so
the solution could perhaps be left for later as well. The patchset I've been
pushing solves, as far as I understand, the ones that pre-exist w/o this
patchset. The solutions need some reworking if this set is applied, as
there's no longer a single driver responsible for a media device.

> + * references are released. Each driver gets a reference to the media device
> + * during probe, when it allocates the media device. If media device is already
> + * allocated, allocate API bumps up the refcount and return the existing media
> + * device. Driver puts the reference back from its disconnect routine when it
> + * calls media_device_delete().
> + *
> + * Media device is unregistered and cleaned up from the kref put handler to
> + * ensure that the media device stays in registered state until the last driver
> + * unregisters the media device.
> + *
> + * Driver Usage:
> + *
> + * Drivers should use the media-core routines to get register reference and
> + * call media_device_delete() routine to make sure the shared media device
> + * delete is handled correctly.
> + *
> + * driver probe:
> + *	Call media_device_usb_allocate() to allocate or get a reference
> + *	Call media_device_register(), if media devnode isn't registered
> + *
> + * driver disconnect:
> + *	Call media_device_delete() to free the media_device. Free'ing is
> + *	handled by the put handler.
> + *
> + */
> +
> +/**
> + * media_device_usb_allocate() - Allocate and return media device
> + *
> + * @udev - struct usb_device pointer
> + * @module_name
> + *
> + * This interface should be called to allocate a media device when multiple
> + * drivers share usb_device and the media device. This interface allocates
> + * media device and calls media_device_usb_init() to initialize it.
> + *
> + */
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name);
> +/**
> + * media_device_delete() - Release media device. Calls kref_put().
> + *
> + * @mdev - struct media_device pointer
> + * @module_name
> + *
> + * This interface should be called to put Media Device Instance kref.
> + */
> +void media_device_delete(struct media_device *mdev, char *driver_name);
> +#else
> +static inline struct media_device *media_device_usb_allocate(
> +			struct usb_device *udev, char *driver_name)
> +			{ return NULL; }
> +static inline void media_device_delete(
> +			struct media_device *mdev, char *module_name) { }
> +#endif /* CONFIG_MEDIA_CONTROLLER */
> +#endif

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v4 1/3] media: Media Device Allocator API

[Shuah Khan]

Hi Sakari,

Thanks for a quick review.

On 11/17/2016 04:12 PM, Sakari Ailus wrote:
> Hi Shuah,
> 
> On Wed, Nov 16, 2016 at 07:29:09AM -0700, Shuah Khan wrote:
>> Media Device Allocator API to allows multiple drivers share a media device.
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
>>
>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>> Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>> Changes since v3:
>> - Fixed undefined reference to `__media_device_usb_init compile error when
>>   CONFIG_USB is disabled.
>> - Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
>>   module that owns the media_device. The fix bumps the reference count for
>>   the owner when second driver comes along to share the media_device. If
>>   au0828 owns the media_device, then snd_usb_audio will bump the refcount
>>   for au0828, so it won't get deleted and vice versa.
>>
>>  drivers/media/Makefile              |   3 +-
>>  drivers/media/media-dev-allocator.c | 146 ++++++++++++++++++++++++++++++++++++
>>  include/media/media-dev-allocator.h |  87 +++++++++++++++++++++
>>  3 files changed, 235 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/media/media-dev-allocator.c
>>  create mode 100644 include/media/media-dev-allocator.h
>>
>> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
>> index 0deaa93..7c0701d 100644
>> --- a/drivers/media/Makefile
>> +++ b/drivers/media/Makefile
>> @@ -6,7 +6,8 @@ ifeq ($(CONFIG_MEDIA_CEC_EDID),y)
>>    obj-$(CONFIG_MEDIA_SUPPORT) += cec-edid.o
>>  endif
>>  
>> -media-objs	:= media-device.o media-devnode.o media-entity.o
>> +media-objs	:= media-device.o media-devnode.o media-entity.o \
>> +		   media-dev-allocator.o
>>  
>>  #
>>  # I2C drivers should come before other drivers, otherwise they'll fail
>> diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
>> new file mode 100644
>> index 0000000..014a317
>> --- /dev/null
>> +++ b/drivers/media/media-dev-allocator.c
>> @@ -0,0 +1,146 @@
>> +/*
>> + * media-dev-allocator.c - Media Controller Device Allocator API
>> + *
>> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
>> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
>> + *
>> + * This file is released under the GPLv2.
>> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> + */
>> +
>> +/*
>> + * This file adds a global refcounted Media Controller Device Instance API.
>> + * A system wide global media device list is managed and each media device
>> + * includes a kref count. The last put on the media device releases the media
>> + * device instance.
>> + *
>> + */
>> +
>> +#include <linux/slab.h>
>> +#include <linux/kref.h>
>> +#include <linux/usb.h>
>> +#include <linux/module.h>
>> +#include <media/media-device.h>
> 
> Alphabetical order, please.
> 

Will do. It gets out of hand as I keep adding headers :)

>> +
>> +static LIST_HEAD(media_device_list);
>> +static DEFINE_MUTEX(media_device_lock);
>> +
>> +struct media_device_instance {
>> +	struct media_device mdev;
>> +	struct module *owner;
>> +	struct list_head list;
>> +	struct device *dev;
> 
> We've got a struct device pointer in struct media_device. Should we have the
> same here, or use the value in struct media_device?

I think it is good to have it independent of media_device.

> 
>> +	struct kref refcount;
>> +};
>> +
>> +static inline struct media_device_instance *
>> +to_media_device_instance(struct media_device *mdev)
>> +{
>> +	return container_of(mdev, struct media_device_instance, mdev);
>> +}
>> +
>> +static void media_device_instance_release(struct kref *kref)
>> +{
>> +	struct media_device_instance *mdi =
>> +		container_of(kref, struct media_device_instance, refcount);
>> +
>> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
>> +
>> +	mutex_lock(&media_device_lock);
> 
> Do you need the lock during unregister / cleanup? Isn't it just to serialise
> the access to the linked list?

It is for serializing the list. Think of a case where two drivers are running
through their release sequence and call media_device_delete(). We want only one
of them to do unregister.

> 
>> +
>> +	media_device_unregister(&mdi->mdev);
>> +	media_device_cleanup(&mdi->mdev);
>> +
>> +	list_del(&mdi->list);
>> +	mutex_unlock(&media_device_lock);
>> +
>> +	kfree(mdi);
>> +}
>> +
>> +/* Callers should hold media_device_lock when calling this function */
>> +static struct media_device *__media_device_get(struct device *dev,
>> +					       char *module_name)
>> +{
>> +	struct media_device_instance *mdi;
>> +
>> +	list_for_each_entry(mdi, &media_device_list, list) {
>> +		if (mdi->dev == dev) {
> 
> You could make this look nicer if you continue here if mdi->dev != dev.

I can do that.

> 
>> +			kref_get(&mdi->refcount);
>> +			/* get module reference for the media_device owner */
>> +			if (find_module(module_name) != mdi->owner &&
>> +			    !try_module_get(mdi->owner))
>> +				dev_err(dev, "%s: try_module_get() error\n",
>> +					__func__);
>> +			dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
>> +				__func__, &mdi->mdev, module_name);
>> +			return &mdi->mdev;
>> +		}
>> +	}
>> +
>> +	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
>> +	if (!mdi)
>> +		return NULL;
>> +
>> +	mdi->dev = dev;
>> +	mdi->owner = find_module(module_name);
>> +	kref_init(&mdi->refcount);
>> +	list_add_tail(&mdi->list, &media_device_list);
>> +
>> +	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
> 
> Does the pointer value provide useful information here? The media device is
> specific to a device and there's just one per device. The device name is
> already printed anyway.

It is very useful in making sure there is no leak. That is when second
driver calls media_device_usb_allocate(), we want to see that the same
pointer is handed out. It came in handy for me during testing.

> 
>> +		&mdi->mdev, module_name);
>> +	return &mdi->mdev;
>> +}
>> +
>> +#if IS_ENABLED(CONFIG_USB)
>> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
>> +					       char *module_name)
> 
> How about adding a macro bearing this name, and using KBUILD_MODNAME there?
> It'd look cleaner in drivers. That's what e.g. media_device_register() does,
> for instance.
> 

Yes. I can do it the way media_device_register() does.

>> +{
>> +	struct media_device *mdev;
>> +
>> +	if (!module_name) {
>> +		dev_err(&udev->dev, "%s Module Name is null\n", __func__);
> 
> You use colon (":") elsewhere after the function name, how about here as
> well? "Name" could begin with lowercase letter, too.

Yeah. I added this later, that explains the inconsistency. Will fix it.

> 
>> +		return ERR_PTR(-EINVAL);
>> +	}
>> +
>> +	mutex_lock(&media_device_lock);
>> +	mdev = __media_device_get(&udev->dev, module_name);
>> +	if (!mdev) {
>> +		mutex_unlock(&media_device_lock);
>> +		return ERR_PTR(-ENOMEM);
>> +	}
>> +
>> +	/* check if media device is already initialized */
>> +	if (!mdev->dev)
> 
> I think I would look up a device explicitly, and allocate one if needed.
> This would mean splitting __media_device_get(), you could return the device
> early if found, and then initialise without a need to check anything.

Yeah that could work. I didn't want to add yet another interface and wanted
to _get to do both. I will play with it and see how it comes together. There
is a reason why I kept them the same - but can't recall why.

> 
>> +		__media_device_usb_init(mdev, udev, udev->product,
>> +					module_name);
> 
> When is the media device registered and by which party?

Media device init and register steps are now separate. Drivers first call
media_device_init() and then at a later time call media_device_register().
This split happened during media controller v2 work. This split helps bail
out early if media_device init fails. Here is the sequence:

media_device_init() or media_device_usb_allocate() if using shared media-device
v4l2 and dvb register
media_device_register()

> 
> I guess you can register entities once the media device is registered.
> Nothing I can think of should prevent that, but I don't think it's been done
> before. There could be bugs.
> 
>> +	mutex_unlock(&media_device_lock);
>> +
>> +	dev_dbg(&udev->dev, "%s\n", __func__);
> 
> This was probably nice to have during development time but should be removed
> now.

Yeah this can go.

> 
>> +	return mdev;
>> +}
>> +EXPORT_SYMBOL_GPL(media_device_usb_allocate);
>> +#endif
>> +
>> +void media_device_delete(struct media_device *mdev, char *module_name)
>> +{
>> +	struct media_device_instance *mdi = to_media_device_instance(mdev);
>> +
>> +	if (!module_name) {
>> +		dev_err(mdi->mdev.dev, "%s Module Name is null\n", __func__);
> 
> Can this happen? If so, why? What if the driver is linked to the kernel
> instead?

I don't believe this should happen. I can remove it.

> 
>> +		return;
>> +	}
>> +
>> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
>> +		__func__, &mdi->mdev, module_name);
>> +
>> +	mutex_lock(&media_device_lock);
>> +	/* put module reference if media_device owner is not THIS_MODULE */
>> +	if (mdi->owner != find_module(module_name)) {
>> +		module_put(mdi->owner);
>> +		dev_dbg(mdi->mdev.dev,
>> +			"%s decremented owner module reference\n", __func__);
>> +	}
>> +	mutex_unlock(&media_device_lock);
>> +	kref_put(&mdi->refcount, media_device_instance_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_device_delete);
>> diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
>> new file mode 100644
>> index 0000000..45c437d
>> --- /dev/null
>> +++ b/include/media/media-dev-allocator.h
>> @@ -0,0 +1,87 @@
>> +/*
>> + * media-dev-allocator.h - Media Controller Device Allocator API
>> + *
>> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
>> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
>> + *
>> + * This file is released under the GPLv2.
>> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> + */
>> +
>> +/*
>> + * This file adds a global ref-counted Media Controller Device Instance API.
>> + * A system wide global media device list is managed and each media device
>> + * includes a kref count. The last put on the media device releases the media
>> + * device instance.
>> + */
>> +
>> +#ifndef _MEDIA_DEV_ALLOCTOR_H
>> +#define _MEDIA_DEV_ALLOCTOR_H
>> +
>> +struct usb_device;
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +/**
>> + * DOC: Media Controller Device Allocator API
>> + *
>> + * When media device belongs to more than one driver, the shared media device
>> + * is allocated with the shared struct device as the key for look ups.
>> + *
>> + * Shared media device should stay in registered state until the last driver
>> + * unregisters it. In addition, media device should be released when all the
> 
> The last or the first?
> 
> Presumably any driver that acquires the media device this way can create
> entities, and these entities would be removed when that driver releases the
> media device. We don't refcount graph objects at the moment, so if these
> objects are referred to elsewhere in the kernel while that driver goes away,
> bad things will follow.

Two scenarios:

Say if one of the drivers is unbound. The second driver still has its graph
that can be accessed. When the first driver comes back with a subsequent bind,
then it can add its graph nodes. This works very well at the moment. I ran
au0828 unbind/bind loops while running media-ctl and media_device_tests
accessing the graph.

I do think last is necessary and makes things simpler. Otherwise, we will
have to address cases when a driver goes away taking media_device with it,
the other driver might still have references to it. We have to do some kind
of handshake or callbacks to make sure the other driver knows about its
counterpart disappearing. They do have kind of split brain now with their
own devices i.e video, dvb, audio devices.

The second scenario is when the driver that owns the media_device (module
ownership) get removed via rmmod, then without the second driver holding
the reference to the first module, bad things happen. This is why I added
the logic to keep track of the module ownership and have the second driver
get module reference for the owning module.

The implication of holding module reference is:

rmmmod on the owner module will fail with "module in use error". This isn't
unusual anyway, when the device or module is in use, we do this now.
e.g: if au08282 is the owner the, it can't be removed until
snd_usb_audio is removed and vice versa.

thanks,
-- Shuah

> 
> It's not the only place this kind of problems exist at the moment though, so
> the solution could perhaps be left for later as well. The patchset I've been
> pushing solves, as far as I understand, the ones that pre-exist w/o this
> patchset. The solutions need some reworking if this set is applied, as
> there's no longer a single driver responsible for a media device.
> 
>> + * references are released. Each driver gets a reference to the media device
>> + * during probe, when it allocates the media device. If media device is already
>> + * allocated, allocate API bumps up the refcount and return the existing media
>> + * device. Driver puts the reference back from its disconnect routine when it
>> + * calls media_device_delete().
>> + *
>> + * Media device is unregistered and cleaned up from the kref put handler to
>> + * ensure that the media device stays in registered state until the last driver
>> + * unregisters the media device.
>> + *
>> + * Driver Usage:
>> + *
>> + * Drivers should use the media-core routines to get register reference and
>> + * call media_device_delete() routine to make sure the shared media device
>> + * delete is handled correctly.
>> + *
>> + * driver probe:
>> + *	Call media_device_usb_allocate() to allocate or get a reference
>> + *	Call media_device_register(), if media devnode isn't registered
>> + *
>> + * driver disconnect:
>> + *	Call media_device_delete() to free the media_device. Free'ing is
>> + *	handled by the put handler.
>> + *
>> + */
>> +
>> +/**
>> + * media_device_usb_allocate() - Allocate and return media device
>> + *
>> + * @udev - struct usb_device pointer
>> + * @module_name
>> + *
>> + * This interface should be called to allocate a media device when multiple
>> + * drivers share usb_device and the media device. This interface allocates
>> + * media device and calls media_device_usb_init() to initialize it.
>> + *
>> + */
>> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
>> +					       char *module_name);
>> +/**
>> + * media_device_delete() - Release media device. Calls kref_put().
>> + *
>> + * @mdev - struct media_device pointer
>> + * @module_name
>> + *
>> + * This interface should be called to put Media Device Instance kref.
>> + */
>> +void media_device_delete(struct media_device *mdev, char *driver_name);
>> +#else
>> +static inline struct media_device *media_device_usb_allocate(
>> +			struct usb_device *udev, char *driver_name)
>> +			{ return NULL; }
>> +static inline void media_device_delete(
>> +			struct media_device *mdev, char *module_name) { }
>> +#endif /* CONFIG_MEDIA_CONTROLLER */
>> +#endif
> 

[PATCH v4 2/3] media: change au0828 to use Media Device Allocator API

[Shuah Khan]

Change au0828 to use Media Device Allocator API to allocate media device
with the parent usb struct device as the key, so it can be shared with the
snd_usb_audio driver.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
Changes since v2:
- Updated media_device_delete() to pass in module name.

 drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
 drivers/media/usb/au0828/au0828.h      |  1 +
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index bf53553..582f31f 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -157,9 +157,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
 	dev->media_dev->enable_source = NULL;
 	dev->media_dev->disable_source = NULL;
 
-	media_device_unregister(dev->media_dev);
-	media_device_cleanup(dev->media_dev);
-	kfree(dev->media_dev);
+	media_device_delete(dev->media_dev, KBUILD_MODNAME);
 	dev->media_dev = NULL;
 #endif
 }
@@ -212,14 +210,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
 #ifdef CONFIG_MEDIA_CONTROLLER
 	struct media_device *mdev;
 
-	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
+	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
 	if (!mdev)
 		return -ENOMEM;
 
-	/* check if media device is already initialized */
-	if (!mdev->dev)
-		media_device_usb_init(mdev, udev, udev->product);
-
 	dev->media_dev = mdev;
 #endif
 	return 0;
@@ -487,6 +481,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
 		/* register media device */
 		ret = media_device_register(dev->media_dev);
 		if (ret) {
+			media_device_delete(dev->media_dev, KBUILD_MODNAME);
+			dev->media_dev = NULL;
 			dev_err(&udev->dev,
 				"Media Device Register Error: %d\n", ret);
 			return ret;
diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
index dd7b378..4bf1b0c 100644
--- a/drivers/media/usb/au0828/au0828.h
+++ b/drivers/media/usb/au0828/au0828.h
@@ -35,6 +35,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-fh.h>
 #include <media/media-device.h>
+#include <media/media-dev-allocator.h>
 
 /* DVB */
 #include "demux.h"
-- 
2.7.4

Re: [PATCH v4 2/3] media: change au0828 to use Media Device Allocator API

[Shuah Khan]

Sorry for the duplicates. Replying to catch bounced emails on the original
patch emails from git send-email

On 11/16/2016 07:29 AM, Shuah Khan wrote:
> Change au0828 to use Media Device Allocator API to allocate media device
> with the parent usb struct device as the key, so it can be shared with the
> snd_usb_audio driver.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
> Changes since v2:
> - Updated media_device_delete() to pass in module name.
> 
>  drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
>  drivers/media/usb/au0828/au0828.h      |  1 +
>  2 files changed, 5 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
> index bf53553..582f31f 100644
> --- a/drivers/media/usb/au0828/au0828-core.c
> +++ b/drivers/media/usb/au0828/au0828-core.c
> @@ -157,9 +157,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
>  	dev->media_dev->enable_source = NULL;
>  	dev->media_dev->disable_source = NULL;
>  
> -	media_device_unregister(dev->media_dev);
> -	media_device_cleanup(dev->media_dev);
> -	kfree(dev->media_dev);
> +	media_device_delete(dev->media_dev, KBUILD_MODNAME);
>  	dev->media_dev = NULL;
>  #endif
>  }
> @@ -212,14 +210,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
>  #ifdef CONFIG_MEDIA_CONTROLLER
>  	struct media_device *mdev;
>  
> -	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
> +	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
>  	if (!mdev)
>  		return -ENOMEM;
>  
> -	/* check if media device is already initialized */
> -	if (!mdev->dev)
> -		media_device_usb_init(mdev, udev, udev->product);
> -
>  	dev->media_dev = mdev;
>  #endif
>  	return 0;
> @@ -487,6 +481,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
>  		/* register media device */
>  		ret = media_device_register(dev->media_dev);
>  		if (ret) {
> +			media_device_delete(dev->media_dev, KBUILD_MODNAME);
> +			dev->media_dev = NULL;
>  			dev_err(&udev->dev,
>  				"Media Device Register Error: %d\n", ret);
>  			return ret;
> diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
> index dd7b378..4bf1b0c 100644
> --- a/drivers/media/usb/au0828/au0828.h
> +++ b/drivers/media/usb/au0828/au0828.h
> @@ -35,6 +35,7 @@
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-fh.h>
>  #include <media/media-device.h>
> +#include <media/media-dev-allocator.h>
>  
>  /* DVB */
>  #include "demux.h"
> 

Re: [PATCH v4 2/3] media: change au0828 to use Media Device Allocator API

[Mauro Carvalho Chehab]

Em Wed, 16 Nov 2016 07:29:10 -0700
Shuah Khan <shuahkh@osg.samsung.com> escreveu:

> Change au0828 to use Media Device Allocator API to allocate media device
> with the parent usb struct device as the key, so it can be shared with the
> snd_usb_audio driver.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>

I missed a v5 for this patch. This one looks OK.

If you don't change this anymore, please add on the v6:

Reviewed-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

> ---
> Changes since v2:
> - Updated media_device_delete() to pass in module name.
> 
>  drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
>  drivers/media/usb/au0828/au0828.h      |  1 +
>  2 files changed, 5 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
> index bf53553..582f31f 100644
> --- a/drivers/media/usb/au0828/au0828-core.c
> +++ b/drivers/media/usb/au0828/au0828-core.c
> @@ -157,9 +157,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
>  	dev->media_dev->enable_source = NULL;
>  	dev->media_dev->disable_source = NULL;
>  
> -	media_device_unregister(dev->media_dev);
> -	media_device_cleanup(dev->media_dev);
> -	kfree(dev->media_dev);
> +	media_device_delete(dev->media_dev, KBUILD_MODNAME);
>  	dev->media_dev = NULL;
>  #endif
>  }
> @@ -212,14 +210,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
>  #ifdef CONFIG_MEDIA_CONTROLLER
>  	struct media_device *mdev;
>  
> -	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
> +	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
>  	if (!mdev)
>  		return -ENOMEM;
>  
> -	/* check if media device is already initialized */
> -	if (!mdev->dev)
> -		media_device_usb_init(mdev, udev, udev->product);
> -
>  	dev->media_dev = mdev;
>  #endif
>  	return 0;
> @@ -487,6 +481,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
>  		/* register media device */
>  		ret = media_device_register(dev->media_dev);
>  		if (ret) {
> +			media_device_delete(dev->media_dev, KBUILD_MODNAME);
> +			dev->media_dev = NULL;
>  			dev_err(&udev->dev,
>  				"Media Device Register Error: %d\n", ret);
>  			return ret;
> diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
> index dd7b378..4bf1b0c 100644
> --- a/drivers/media/usb/au0828/au0828.h
> +++ b/drivers/media/usb/au0828/au0828.h
> @@ -35,6 +35,7 @@
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-fh.h>
>  #include <media/media-device.h>
> +#include <media/media-dev-allocator.h>
>  
>  /* DVB */
>  #include "demux.h"



Thanks,
Mauro

[PATCH v4 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Change ALSA driver to use Media Controller API to share media resources
with DVB, and V4L2 drivers on a AU0828 media device.

Media Controller specific initialization is done after sound card is
registered.  ALSA creates Media interface and entity function graph
nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler
interface to request the media resource. If resource request is granted,
it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
Changes to patch 0003 since the last version in 4.7-rc1:
- Included fixes to bugs found during testing. 
- Updated to use the Media Allocator API.

 sound/usb/Kconfig        |   4 +
 sound/usb/Makefile       |   2 +
 sound/usb/card.c         |  14 +++
 sound/usb/card.h         |   3 +
 sound/usb/media.c        | 314 +++++++++++++++++++++++++++++++++++++++++++++++
 sound/usb/media.h        |  73 +++++++++++
 sound/usb/mixer.h        |   3 +
 sound/usb/pcm.c          |  28 ++++-
 sound/usb/quirks-table.h |   1 +
 sound/usb/stream.c       |   2 +
 sound/usb/usbaudio.h     |   6 +
 11 files changed, 445 insertions(+), 5 deletions(-)
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
index a452ad7..d14bf41 100644
--- a/sound/usb/Kconfig
+++ b/sound/usb/Kconfig
@@ -15,6 +15,7 @@ config SND_USB_AUDIO
 	select SND_RAWMIDI
 	select SND_PCM
 	select BITREVERSE
+	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
 	help
 	  Say Y here to include support for USB audio and USB MIDI
 	  devices.
@@ -22,6 +23,9 @@ config SND_USB_AUDIO
 	  To compile this driver as a module, choose M here: the module
 	  will be called snd-usb-audio.
 
+config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+	bool
+
 config SND_USB_UA101
 	tristate "Edirol UA-101/UA-1000 driver"
 	select SND_PCM
diff --git a/sound/usb/Makefile b/sound/usb/Makefile
index 2d2d122..8dca3c4 100644
--- a/sound/usb/Makefile
+++ b/sound/usb/Makefile
@@ -15,6 +15,8 @@ snd-usb-audio-objs := 	card.o \
 			quirks.o \
 			stream.o
 
+snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
+
 snd-usbmidi-lib-objs := midi.o
 
 # Toplevel Module Dependency
diff --git a/sound/usb/card.c b/sound/usb/card.c
index 9e5276d..2750e56 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -66,6 +66,7 @@
 #include "format.h"
 #include "power.h"
 #include "stream.h"
+#include "media.h"
 
 MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
 MODULE_DESCRIPTION("USB Audio");
@@ -615,6 +616,11 @@ static int usb_audio_probe(struct usb_interface *intf,
 	if (err < 0)
 		goto __error;
 
+	if (quirk && quirk->media_device) {
+		/* don't want to fail when media_snd_device_create() fails */
+		media_snd_device_create(chip, intf);
+	}
+
 	usb_chip[chip->index] = chip;
 	chip->num_interfaces++;
 	usb_set_intfdata(intf, chip);
@@ -671,6 +677,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
 		list_for_each(p, &chip->midi_list) {
 			snd_usbmidi_disconnect(p);
 		}
+		/*
+		 * Nice to check quirk && quirk->media_device and
+		 * then call media_snd_device_delete(). Don't have
+		 * access to quirk here. media_snd_device_delete()
+		 * acceses mixer_list
+		 */
+		media_snd_device_delete(chip);
+
 		/* release mixer resources */
 		list_for_each_entry(mixer, &chip->mixer_list, list) {
 			snd_usb_mixer_disconnect(mixer);
diff --git a/sound/usb/card.h b/sound/usb/card.h
index 111b0f0..6ef8e88 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -105,6 +105,8 @@ struct snd_usb_endpoint {
 	struct list_head list;
 };
 
+struct media_ctl;
+
 struct snd_usb_substream {
 	struct snd_usb_stream *stream;
 	struct usb_device *dev;
@@ -156,6 +158,7 @@ struct snd_usb_substream {
 	} dsd_dop;
 
 	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
+	struct media_ctl *media_ctl;
 };
 
 struct snd_usb_stream {
diff --git a/sound/usb/media.c b/sound/usb/media.c
new file mode 100644
index 0000000..ccdc4d9
--- /dev/null
+++ b/sound/usb/media.c
@@ -0,0 +1,314 @@
+/*
+ * media.c - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <sound/pcm.h>
+#include <sound/core.h>
+
+#include "usbaudio.h"
+#include "card.h"
+#include "mixer.h"
+#include "media.h"
+
+static int media_snd_enable_source(struct media_ctl *mctl)
+{
+	if (mctl && mctl->media_dev->enable_source)
+		return mctl->media_dev->enable_source(&mctl->media_entity,
+						      &mctl->media_pipe);
+	return 0;
+}
+
+static void media_snd_disable_source(struct media_ctl *mctl)
+{
+	if (mctl && mctl->media_dev->disable_source)
+		mctl->media_dev->disable_source(&mctl->media_entity);
+}
+
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			int stream)
+{
+	struct media_device *mdev;
+	struct media_ctl *mctl;
+	struct device *pcm_dev = &pcm->streams[stream].dev;
+	u32 intf_type;
+	int ret = 0;
+	u16 mixer_pad;
+	struct media_entity *entity;
+
+	mdev = subs->stream->chip->media_dev;
+	if (!mdev)
+		return 0;
+
+	if (subs->media_ctl)
+		return 0;
+
+	/* allocate media_ctl */
+	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+	if (!mctl)
+		return -ENOMEM;
+
+	mctl->media_dev = mdev;
+	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
+		mixer_pad = 1;
+	} else {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
+		mixer_pad = 2;
+	}
+	mctl->media_entity.name = pcm->name;
+	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
+	ret =  media_device_register_entity(mctl->media_dev,
+					    &mctl->media_entity);
+	if (ret)
+		goto free_mctl;
+
+	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
+						  MAJOR(pcm_dev->devt),
+						  MINOR(pcm_dev->devt));
+	if (!mctl->intf_devnode) {
+		ret = -ENOMEM;
+		goto unregister_entity;
+	}
+	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+						 &mctl->intf_devnode->intf,
+						 MEDIA_LNK_FL_ENABLED);
+	if (!mctl->intf_link) {
+		ret = -ENOMEM;
+		goto devnode_remove;
+	}
+
+	/* create link between mixer and audio */
+	media_device_for_each_entity(entity, mdev) {
+		switch (entity->function) {
+		case MEDIA_ENT_F_AUDIO_MIXER:
+			ret = media_create_pad_link(entity, mixer_pad,
+						    &mctl->media_entity, 0,
+						    MEDIA_LNK_FL_ENABLED);
+			if (ret)
+				goto remove_intf_link;
+			break;
+		}
+	}
+
+	subs->media_ctl = mctl;
+	return 0;
+
+remove_intf_link:
+	media_remove_intf_link(mctl->intf_link);
+devnode_remove:
+	media_devnode_remove(mctl->intf_devnode);
+unregister_entity:
+	media_device_unregister_entity(&mctl->media_entity);
+free_mctl:
+	kfree(mctl);
+	return ret;
+}
+
+void media_snd_stream_delete(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl && mctl->media_dev) {
+		struct media_device *mdev;
+
+		mdev = mctl->media_dev;
+		if (mdev && media_devnode_is_registered(mdev->devnode)) {
+			media_devnode_remove(mctl->intf_devnode);
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		subs->media_ctl = NULL;
+	}
+}
+
+int media_snd_start_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl)
+		return media_snd_enable_source(mctl);
+	return 0;
+}
+
+void media_snd_stop_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl)
+		media_snd_disable_source(mctl);
+}
+
+static int media_snd_mixer_init(struct snd_usb_audio *chip)
+{
+	struct device *ctl_dev = &chip->card->ctl_dev;
+	struct media_intf_devnode *ctl_intf;
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+	struct media_mixer_ctl *mctl;
+	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
+	int ret;
+
+	if (!mdev)
+		return -ENODEV;
+
+	ctl_intf = chip->ctl_intf_media_devnode;
+	if (!ctl_intf) {
+		ctl_intf = media_devnode_create(mdev, intf_type, 0,
+						MAJOR(ctl_dev->devt),
+						MINOR(ctl_dev->devt));
+		if (!ctl_intf)
+			return -ENOMEM;
+		chip->ctl_intf_media_devnode = ctl_intf;
+	}
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+
+		if (mixer->media_mixer_ctl)
+			continue;
+
+		/* allocate media_mixer_ctl */
+		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+		if (!mctl)
+			return -ENOMEM;
+
+		mctl->media_dev = mdev;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
+		mctl->media_entity.name = chip->card->mixername;
+		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
+		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
+		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
+		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
+				  mctl->media_pad);
+		ret =  media_device_register_entity(mctl->media_dev,
+						    &mctl->media_entity);
+		if (ret) {
+			kfree(mctl);
+			return ret;
+		}
+
+		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+							 &ctl_intf->intf,
+							 MEDIA_LNK_FL_ENABLED);
+		if (!mctl->intf_link) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+			kfree(mctl);
+			return -ENOMEM;
+		}
+		mctl->intf_devnode = ctl_intf;
+		mixer->media_mixer_ctl = mctl;
+	}
+	return 0;
+}
+
+static void media_snd_mixer_delete(struct snd_usb_audio *chip)
+{
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+
+	if (!mdev)
+		return;
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+		struct media_mixer_ctl *mctl;
+
+		mctl = mixer->media_mixer_ctl;
+		if (!mixer->media_mixer_ctl)
+			continue;
+
+		if (media_devnode_is_registered(mdev->devnode)) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		mixer->media_mixer_ctl = NULL;
+	}
+	if (media_devnode_is_registered(mdev->devnode))
+		media_devnode_remove(chip->ctl_intf_media_devnode);
+	chip->ctl_intf_media_devnode = NULL;
+}
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+	if (!mdev)
+		return -ENOMEM;
+
+	if (!media_devnode_is_registered(mdev->devnode)) {
+		/* register media_device */
+		ret = media_device_register(mdev);
+		if (ret) {
+			media_device_delete(mdev, KBUILD_MODNAME);
+			dev_err(&usbdev->dev,
+				"Couldn't register media device. Error: %d\n",
+				ret);
+			return ret;
+		}
+	}
+
+	/* save media device - avoid lookups */
+	chip->media_dev = mdev;
+
+	/* Create media entities for mixer and control dev */
+	ret = media_snd_mixer_init(chip);
+	if (ret) {
+		dev_err(&usbdev->dev,
+			"Couldn't create media mixer entities. Error: %d\n",
+			ret);
+
+		/* clear saved media_dev */
+		chip->media_dev = NULL;
+
+		return ret;
+	}
+	return 0;
+}
+
+void media_snd_device_delete(struct snd_usb_audio *chip)
+{
+	struct media_device *mdev = chip->media_dev;
+	struct snd_usb_stream *stream;
+
+	/* release resources */
+	list_for_each_entry(stream, &chip->pcm_list, list) {
+		media_snd_stream_delete(&stream->substream[0]);
+		media_snd_stream_delete(&stream->substream[1]);
+	}
+
+	media_snd_mixer_delete(chip);
+
+	if (mdev) {
+		media_device_delete(mdev, KBUILD_MODNAME);
+		chip->media_dev = NULL;
+	}
+}
diff --git a/sound/usb/media.h b/sound/usb/media.h
new file mode 100644
index 0000000..d204375
--- /dev/null
+++ b/sound/usb/media.h
@@ -0,0 +1,73 @@
+/*
+ * media.h - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+#ifndef __MEDIA_H
+
+#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/media-dev-allocator.h>
+#include <sound/asound.h>
+
+struct media_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad;
+	struct media_pipeline media_pipe;
+};
+
+/*
+ * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
+ * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
+ * to AUDIO Source
+ */
+#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
+
+struct media_mixer_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
+	struct media_pipeline media_pipe;
+};
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			    struct usb_interface *iface);
+void media_snd_device_delete(struct snd_usb_audio *chip);
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			  int stream);
+void media_snd_stream_delete(struct snd_usb_substream *subs);
+int media_snd_start_pipeline(struct snd_usb_substream *subs);
+void media_snd_stop_pipeline(struct snd_usb_substream *subs);
+#else
+static inline int media_snd_device_create(struct snd_usb_audio *chip,
+					  struct usb_interface *iface)
+						{ return 0; }
+static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
+static inline int media_snd_stream_init(struct snd_usb_substream *subs,
+					struct snd_pcm *pcm, int stream)
+						{ return 0; }
+static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
+static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
+					{ return 0; }
+static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
+#endif
+#endif /* __MEDIA_H */
diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
index 3417ef3..f378944 100644
--- a/sound/usb/mixer.h
+++ b/sound/usb/mixer.h
@@ -3,6 +3,8 @@
 
 #include <sound/info.h>
 
+struct media_mixer_ctl;
+
 struct usb_mixer_interface {
 	struct snd_usb_audio *chip;
 	struct usb_host_interface *hostif;
@@ -22,6 +24,7 @@ struct usb_mixer_interface {
 	struct urb *rc_urb;
 	struct usb_ctrlrequest *rc_setup_packet;
 	u8 rc_buffer[6];
+	struct media_mixer_ctl *media_mixer_ctl;
 };
 
 #define MAX_CHANNELS	16	/* max logical channels */
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 44d178e..0e4e0640 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -35,6 +35,7 @@
 #include "pcm.h"
 #include "clock.h"
 #include "power.h"
+#include "media.h"
 
 #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
 #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
@@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 	struct audioformat *fmt;
 	int ret;
 
+	ret = media_snd_start_pipeline(subs);
+	if (ret)
+		return ret;
+
 	ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
 					       params_buffer_bytes(hw_params));
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 
 	subs->pcm_format = params_format(hw_params);
 	subs->period_bytes = params_period_bytes(hw_params);
@@ -734,22 +739,27 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		dev_dbg(&subs->dev->dev,
 			"cannot set format: format = %#x, rate = %d, channels = %d\n",
 			   subs->pcm_format, subs->cur_rate, subs->channels);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto err_ret;
 	}
 
 	ret = snd_usb_lock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 	ret = set_format(subs, fmt);
 	snd_usb_unlock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 
 	subs->interface = fmt->iface;
 	subs->altset_idx = fmt->altset_idx;
 	subs->need_setup_ep = true;
 
 	return 0;
+
+err_ret:
+	media_snd_stop_pipeline(subs);
+	return ret;
 }
 
 /*
@@ -761,6 +771,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
 {
 	struct snd_usb_substream *subs = substream->runtime->private_data;
 
+	media_snd_stop_pipeline(subs);
 	subs->cur_audiofmt = NULL;
 	subs->cur_rate = 0;
 	subs->period_bytes = 0;
@@ -1221,6 +1232,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
 	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct snd_usb_substream *subs = &as->substream[direction];
+	int ret;
 
 	subs->interface = -1;
 	subs->altset_idx = 0;
@@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
 	subs->dsd_dop.channel = 0;
 	subs->dsd_dop.marker = 1;
 
-	return setup_hw_info(runtime, subs);
+	ret = setup_hw_info(runtime, subs);
+	if (ret == 0)
+		ret = media_snd_stream_init(subs, as->pcm, direction);
+	if (ret)
+		snd_usb_autosuspend(subs->stream->chip);
+	return ret;
 }
 
 static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
@@ -1243,6 +1260,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
 	struct snd_usb_substream *subs = &as->substream[direction];
 
 	stop_endpoints(subs, true);
+	media_snd_stop_pipeline(subs);
 
 	if (subs->interface >= 0 &&
 	    !snd_usb_lock_shutdown(subs->stream->chip)) {
diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
index 8a59d47..164e249 100644
--- a/sound/usb/quirks-table.h
+++ b/sound/usb/quirks-table.h
@@ -2886,6 +2886,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
 		.product_name = pname, \
 		.ifnum = QUIRK_ANY_INTERFACE, \
 		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
+		.media_device = 1, \
 	} \
 }
 
diff --git a/sound/usb/stream.c b/sound/usb/stream.c
index 8e9548bc..6fe7f21 100644
--- a/sound/usb/stream.c
+++ b/sound/usb/stream.c
@@ -36,6 +36,7 @@
 #include "format.h"
 #include "clock.h"
 #include "stream.h"
+#include "media.h"
 
 /*
  * free a substream
@@ -52,6 +53,7 @@ static void free_substream(struct snd_usb_substream *subs)
 		kfree(fp);
 	}
 	kfree(subs->rate_list.list);
+	media_snd_stream_delete(subs);
 }
 
 
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index 4d5c89a..89b6853 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -30,6 +30,9 @@
  *
  */
 
+struct media_device;
+struct media_intf_devnode;
+
 struct snd_usb_audio {
 	int index;
 	struct usb_device *dev;
@@ -61,6 +64,8 @@ struct snd_usb_audio {
 	bool autoclock;			/* from the 'autoclock' module param */
 
 	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
+	struct media_device *media_dev;
+	struct media_intf_devnode *ctl_intf_media_devnode;
 };
 
 #define usb_audio_err(chip, fmt, args...) \
@@ -111,6 +116,7 @@ struct snd_usb_audio_quirk {
 	const char *product_name;
 	int16_t ifnum;
 	uint16_t type;
+	bool media_device;
 	const void *data;
 };
 
-- 
2.7.4

Re: [PATCH v4 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Sorry for the duplicates. Replying to catch bounced emails on the original
patch emails from git send-email

On 11/16/2016 07:29 AM, Shuah Khan wrote:
> Change ALSA driver to use Media Controller API to share media resources
> with DVB, and V4L2 drivers on a AU0828 media device.
> 
> Media Controller specific initialization is done after sound card is
> registered.  ALSA creates Media interface and entity function graph
> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
> Changes to patch 0003 since the last version in 4.7-rc1:
> - Included fixes to bugs found during testing. 
> - Updated to use the Media Allocator API.
> 
>  sound/usb/Kconfig        |   4 +
>  sound/usb/Makefile       |   2 +
>  sound/usb/card.c         |  14 +++
>  sound/usb/card.h         |   3 +
>  sound/usb/media.c        | 314 +++++++++++++++++++++++++++++++++++++++++++++++
>  sound/usb/media.h        |  73 +++++++++++
>  sound/usb/mixer.h        |   3 +
>  sound/usb/pcm.c          |  28 ++++-
>  sound/usb/quirks-table.h |   1 +
>  sound/usb/stream.c       |   2 +
>  sound/usb/usbaudio.h     |   6 +
>  11 files changed, 445 insertions(+), 5 deletions(-)
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
> 
> diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
> index a452ad7..d14bf41 100644
> --- a/sound/usb/Kconfig
> +++ b/sound/usb/Kconfig
> @@ -15,6 +15,7 @@ config SND_USB_AUDIO
>  	select SND_RAWMIDI
>  	select SND_PCM
>  	select BITREVERSE
> +	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
>  	help
>  	  Say Y here to include support for USB audio and USB MIDI
>  	  devices.
> @@ -22,6 +23,9 @@ config SND_USB_AUDIO
>  	  To compile this driver as a module, choose M here: the module
>  	  will be called snd-usb-audio.
>  
> +config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +	bool
> +
>  config SND_USB_UA101
>  	tristate "Edirol UA-101/UA-1000 driver"
>  	select SND_PCM
> diff --git a/sound/usb/Makefile b/sound/usb/Makefile
> index 2d2d122..8dca3c4 100644
> --- a/sound/usb/Makefile
> +++ b/sound/usb/Makefile
> @@ -15,6 +15,8 @@ snd-usb-audio-objs := 	card.o \
>  			quirks.o \
>  			stream.o
>  
> +snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
> +
>  snd-usbmidi-lib-objs := midi.o
>  
>  # Toplevel Module Dependency
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 9e5276d..2750e56 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -66,6 +66,7 @@
>  #include "format.h"
>  #include "power.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
>  MODULE_DESCRIPTION("USB Audio");
> @@ -615,6 +616,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>  	if (err < 0)
>  		goto __error;
>  
> +	if (quirk && quirk->media_device) {
> +		/* don't want to fail when media_snd_device_create() fails */
> +		media_snd_device_create(chip, intf);
> +	}
> +
>  	usb_chip[chip->index] = chip;
>  	chip->num_interfaces++;
>  	usb_set_intfdata(intf, chip);
> @@ -671,6 +677,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>  		list_for_each(p, &chip->midi_list) {
>  			snd_usbmidi_disconnect(p);
>  		}
> +		/*
> +		 * Nice to check quirk && quirk->media_device and
> +		 * then call media_snd_device_delete(). Don't have
> +		 * access to quirk here. media_snd_device_delete()
> +		 * acceses mixer_list
> +		 */
> +		media_snd_device_delete(chip);
> +
>  		/* release mixer resources */
>  		list_for_each_entry(mixer, &chip->mixer_list, list) {
>  			snd_usb_mixer_disconnect(mixer);
> diff --git a/sound/usb/card.h b/sound/usb/card.h
> index 111b0f0..6ef8e88 100644
> --- a/sound/usb/card.h
> +++ b/sound/usb/card.h
> @@ -105,6 +105,8 @@ struct snd_usb_endpoint {
>  	struct list_head list;
>  };
>  
> +struct media_ctl;
> +
>  struct snd_usb_substream {
>  	struct snd_usb_stream *stream;
>  	struct usb_device *dev;
> @@ -156,6 +158,7 @@ struct snd_usb_substream {
>  	} dsd_dop;
>  
>  	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
> +	struct media_ctl *media_ctl;
>  };
>  
>  struct snd_usb_stream {
> diff --git a/sound/usb/media.c b/sound/usb/media.c
> new file mode 100644
> index 0000000..ccdc4d9
> --- /dev/null
> +++ b/sound/usb/media.c
> @@ -0,0 +1,314 @@
> +/*
> + * media.c - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +
> +#include <linux/init.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include <sound/pcm.h>
> +#include <sound/core.h>
> +
> +#include "usbaudio.h"
> +#include "card.h"
> +#include "mixer.h"
> +#include "media.h"
> +
> +static int media_snd_enable_source(struct media_ctl *mctl)
> +{
> +	if (mctl && mctl->media_dev->enable_source)
> +		return mctl->media_dev->enable_source(&mctl->media_entity,
> +						      &mctl->media_pipe);
> +	return 0;
> +}
> +
> +static void media_snd_disable_source(struct media_ctl *mctl)
> +{
> +	if (mctl && mctl->media_dev->disable_source)
> +		mctl->media_dev->disable_source(&mctl->media_entity);
> +}
> +
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			int stream)
> +{
> +	struct media_device *mdev;
> +	struct media_ctl *mctl;
> +	struct device *pcm_dev = &pcm->streams[stream].dev;
> +	u32 intf_type;
> +	int ret = 0;
> +	u16 mixer_pad;
> +	struct media_entity *entity;
> +
> +	mdev = subs->stream->chip->media_dev;
> +	if (!mdev)
> +		return 0;
> +
> +	if (subs->media_ctl)
> +		return 0;
> +
> +	/* allocate media_ctl */
> +	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +	if (!mctl)
> +		return -ENOMEM;
> +
> +	mctl->media_dev = mdev;
> +	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
> +		mixer_pad = 1;
> +	} else {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
> +		mixer_pad = 2;
> +	}
> +	mctl->media_entity.name = pcm->name;
> +	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
> +	ret =  media_device_register_entity(mctl->media_dev,
> +					    &mctl->media_entity);
> +	if (ret)
> +		goto free_mctl;
> +
> +	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
> +						  MAJOR(pcm_dev->devt),
> +						  MINOR(pcm_dev->devt));
> +	if (!mctl->intf_devnode) {
> +		ret = -ENOMEM;
> +		goto unregister_entity;
> +	}
> +	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +						 &mctl->intf_devnode->intf,
> +						 MEDIA_LNK_FL_ENABLED);
> +	if (!mctl->intf_link) {
> +		ret = -ENOMEM;
> +		goto devnode_remove;
> +	}
> +
> +	/* create link between mixer and audio */
> +	media_device_for_each_entity(entity, mdev) {
> +		switch (entity->function) {
> +		case MEDIA_ENT_F_AUDIO_MIXER:
> +			ret = media_create_pad_link(entity, mixer_pad,
> +						    &mctl->media_entity, 0,
> +						    MEDIA_LNK_FL_ENABLED);
> +			if (ret)
> +				goto remove_intf_link;
> +			break;
> +		}
> +	}
> +
> +	subs->media_ctl = mctl;
> +	return 0;
> +
> +remove_intf_link:
> +	media_remove_intf_link(mctl->intf_link);
> +devnode_remove:
> +	media_devnode_remove(mctl->intf_devnode);
> +unregister_entity:
> +	media_device_unregister_entity(&mctl->media_entity);
> +free_mctl:
> +	kfree(mctl);
> +	return ret;
> +}
> +
> +void media_snd_stream_delete(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl && mctl->media_dev) {
> +		struct media_device *mdev;
> +
> +		mdev = mctl->media_dev;
> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
> +			media_devnode_remove(mctl->intf_devnode);
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		subs->media_ctl = NULL;
> +	}
> +}
> +
> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		return media_snd_enable_source(mctl);
> +	return 0;
> +}
> +
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		media_snd_disable_source(mctl);
> +}
> +
> +static int media_snd_mixer_init(struct snd_usb_audio *chip)
> +{
> +	struct device *ctl_dev = &chip->card->ctl_dev;
> +	struct media_intf_devnode *ctl_intf;
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +	struct media_mixer_ctl *mctl;
> +	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
> +	int ret;
> +
> +	if (!mdev)
> +		return -ENODEV;
> +
> +	ctl_intf = chip->ctl_intf_media_devnode;
> +	if (!ctl_intf) {
> +		ctl_intf = media_devnode_create(mdev, intf_type, 0,
> +						MAJOR(ctl_dev->devt),
> +						MINOR(ctl_dev->devt));
> +		if (!ctl_intf)
> +			return -ENOMEM;
> +		chip->ctl_intf_media_devnode = ctl_intf;
> +	}
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +
> +		if (mixer->media_mixer_ctl)
> +			continue;
> +
> +		/* allocate media_mixer_ctl */
> +		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +		if (!mctl)
> +			return -ENOMEM;
> +
> +		mctl->media_dev = mdev;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
> +		mctl->media_entity.name = chip->card->mixername;
> +		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
> +		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
> +		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
> +				  mctl->media_pad);
> +		ret =  media_device_register_entity(mctl->media_dev,
> +						    &mctl->media_entity);
> +		if (ret) {
> +			kfree(mctl);
> +			return ret;
> +		}
> +
> +		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +							 &ctl_intf->intf,
> +							 MEDIA_LNK_FL_ENABLED);
> +		if (!mctl->intf_link) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +			kfree(mctl);
> +			return -ENOMEM;
> +		}
> +		mctl->intf_devnode = ctl_intf;
> +		mixer->media_mixer_ctl = mctl;
> +	}
> +	return 0;
> +}
> +
> +static void media_snd_mixer_delete(struct snd_usb_audio *chip)
> +{
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +
> +	if (!mdev)
> +		return;
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +		struct media_mixer_ctl *mctl;
> +
> +		mctl = mixer->media_mixer_ctl;
> +		if (!mixer->media_mixer_ctl)
> +			continue;
> +
> +		if (media_devnode_is_registered(mdev->devnode)) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		mixer->media_mixer_ctl = NULL;
> +	}
> +	if (media_devnode_is_registered(mdev->devnode))
> +		media_devnode_remove(chip->ctl_intf_media_devnode);
> +	chip->ctl_intf_media_devnode = NULL;
> +}
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			struct usb_interface *iface)
> +{
> +	struct media_device *mdev;
> +	struct usb_device *usbdev = interface_to_usbdev(iface);
> +	int ret;
> +
> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
> +	if (!mdev)
> +		return -ENOMEM;
> +
> +	if (!media_devnode_is_registered(mdev->devnode)) {
> +		/* register media_device */
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			media_device_delete(mdev, KBUILD_MODNAME);
> +			dev_err(&usbdev->dev,
> +				"Couldn't register media device. Error: %d\n",
> +				ret);
> +			return ret;
> +		}
> +	}
> +
> +	/* save media device - avoid lookups */
> +	chip->media_dev = mdev;
> +
> +	/* Create media entities for mixer and control dev */
> +	ret = media_snd_mixer_init(chip);
> +	if (ret) {
> +		dev_err(&usbdev->dev,
> +			"Couldn't create media mixer entities. Error: %d\n",
> +			ret);
> +
> +		/* clear saved media_dev */
> +		chip->media_dev = NULL;
> +
> +		return ret;
> +	}
> +	return 0;
> +}
> +
> +void media_snd_device_delete(struct snd_usb_audio *chip)
> +{
> +	struct media_device *mdev = chip->media_dev;
> +	struct snd_usb_stream *stream;
> +
> +	/* release resources */
> +	list_for_each_entry(stream, &chip->pcm_list, list) {
> +		media_snd_stream_delete(&stream->substream[0]);
> +		media_snd_stream_delete(&stream->substream[1]);
> +	}
> +
> +	media_snd_mixer_delete(chip);
> +
> +	if (mdev) {
> +		media_device_delete(mdev, KBUILD_MODNAME);
> +		chip->media_dev = NULL;
> +	}
> +}
> diff --git a/sound/usb/media.h b/sound/usb/media.h
> new file mode 100644
> index 0000000..d204375
> --- /dev/null
> +++ b/sound/usb/media.h
> @@ -0,0 +1,73 @@
> +/*
> + * media.h - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +#ifndef __MEDIA_H
> +
> +#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +
> +#include <media/media-device.h>
> +#include <media/media-entity.h>
> +#include <media/media-dev-allocator.h>
> +#include <sound/asound.h>
> +
> +struct media_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad;
> +	struct media_pipeline media_pipe;
> +};
> +
> +/*
> + * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
> + * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
> + * to AUDIO Source
> + */
> +#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
> +
> +struct media_mixer_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
> +	struct media_pipeline media_pipe;
> +};
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			    struct usb_interface *iface);
> +void media_snd_device_delete(struct snd_usb_audio *chip);
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			  int stream);
> +void media_snd_stream_delete(struct snd_usb_substream *subs);
> +int media_snd_start_pipeline(struct snd_usb_substream *subs);
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs);
> +#else
> +static inline int media_snd_device_create(struct snd_usb_audio *chip,
> +					  struct usb_interface *iface)
> +						{ return 0; }
> +static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
> +static inline int media_snd_stream_init(struct snd_usb_substream *subs,
> +					struct snd_pcm *pcm, int stream)
> +						{ return 0; }
> +static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
> +static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +					{ return 0; }
> +static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
> +#endif
> +#endif /* __MEDIA_H */
> diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
> index 3417ef3..f378944 100644
> --- a/sound/usb/mixer.h
> +++ b/sound/usb/mixer.h
> @@ -3,6 +3,8 @@
>  
>  #include <sound/info.h>
>  
> +struct media_mixer_ctl;
> +
>  struct usb_mixer_interface {
>  	struct snd_usb_audio *chip;
>  	struct usb_host_interface *hostif;
> @@ -22,6 +24,7 @@ struct usb_mixer_interface {
>  	struct urb *rc_urb;
>  	struct usb_ctrlrequest *rc_setup_packet;
>  	u8 rc_buffer[6];
> +	struct media_mixer_ctl *media_mixer_ctl;
>  };
>  
>  #define MAX_CHANNELS	16	/* max logical channels */
> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
> index 44d178e..0e4e0640 100644
> --- a/sound/usb/pcm.c
> +++ b/sound/usb/pcm.c
> @@ -35,6 +35,7 @@
>  #include "pcm.h"
>  #include "clock.h"
>  #include "power.h"
> +#include "media.h"
>  
>  #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
>  #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
> @@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  	struct audioformat *fmt;
>  	int ret;
>  
> +	ret = media_snd_start_pipeline(subs);
> +	if (ret)
> +		return ret;
> +
>  	ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
>  					       params_buffer_bytes(hw_params));
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->pcm_format = params_format(hw_params);
>  	subs->period_bytes = params_period_bytes(hw_params);
> @@ -734,22 +739,27 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  		dev_dbg(&subs->dev->dev,
>  			"cannot set format: format = %#x, rate = %d, channels = %d\n",
>  			   subs->pcm_format, subs->cur_rate, subs->channels);
> -		return -EINVAL;
> +		ret = -EINVAL;
> +		goto err_ret;
>  	}
>  
>  	ret = snd_usb_lock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  	ret = set_format(subs, fmt);
>  	snd_usb_unlock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->interface = fmt->iface;
>  	subs->altset_idx = fmt->altset_idx;
>  	subs->need_setup_ep = true;
>  
>  	return 0;
> +
> +err_ret:
> +	media_snd_stop_pipeline(subs);
> +	return ret;
>  }
>  
>  /*
> @@ -761,6 +771,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
>  {
>  	struct snd_usb_substream *subs = substream->runtime->private_data;
>  
> +	media_snd_stop_pipeline(subs);
>  	subs->cur_audiofmt = NULL;
>  	subs->cur_rate = 0;
>  	subs->period_bytes = 0;
> @@ -1221,6 +1232,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
>  	struct snd_pcm_runtime *runtime = substream->runtime;
>  	struct snd_usb_substream *subs = &as->substream[direction];
> +	int ret;
>  
>  	subs->interface = -1;
>  	subs->altset_idx = 0;
> @@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	subs->dsd_dop.channel = 0;
>  	subs->dsd_dop.marker = 1;
>  
> -	return setup_hw_info(runtime, subs);
> +	ret = setup_hw_info(runtime, subs);
> +	if (ret == 0)
> +		ret = media_snd_stream_init(subs, as->pcm, direction);
> +	if (ret)
> +		snd_usb_autosuspend(subs->stream->chip);
> +	return ret;
>  }
>  
>  static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
> @@ -1243,6 +1260,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_substream *subs = &as->substream[direction];
>  
>  	stop_endpoints(subs, true);
> +	media_snd_stop_pipeline(subs);
>  
>  	if (subs->interface >= 0 &&
>  	    !snd_usb_lock_shutdown(subs->stream->chip)) {
> diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
> index 8a59d47..164e249 100644
> --- a/sound/usb/quirks-table.h
> +++ b/sound/usb/quirks-table.h
> @@ -2886,6 +2886,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
>  		.product_name = pname, \
>  		.ifnum = QUIRK_ANY_INTERFACE, \
>  		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
> +		.media_device = 1, \
>  	} \
>  }
>  
> diff --git a/sound/usb/stream.c b/sound/usb/stream.c
> index 8e9548bc..6fe7f21 100644
> --- a/sound/usb/stream.c
> +++ b/sound/usb/stream.c
> @@ -36,6 +36,7 @@
>  #include "format.h"
>  #include "clock.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  /*
>   * free a substream
> @@ -52,6 +53,7 @@ static void free_substream(struct snd_usb_substream *subs)
>  		kfree(fp);
>  	}
>  	kfree(subs->rate_list.list);
> +	media_snd_stream_delete(subs);
>  }
>  
>  
> diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
> index 4d5c89a..89b6853 100644
> --- a/sound/usb/usbaudio.h
> +++ b/sound/usb/usbaudio.h
> @@ -30,6 +30,9 @@
>   *
>   */
>  
> +struct media_device;
> +struct media_intf_devnode;
> +
>  struct snd_usb_audio {
>  	int index;
>  	struct usb_device *dev;
> @@ -61,6 +64,8 @@ struct snd_usb_audio {
>  	bool autoclock;			/* from the 'autoclock' module param */
>  
>  	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
> +	struct media_device *media_dev;
> +	struct media_intf_devnode *ctl_intf_media_devnode;
>  };
>  
>  #define usb_audio_err(chip, fmt, args...) \
> @@ -111,6 +116,7 @@ struct snd_usb_audio_quirk {
>  	const char *product_name;
>  	int16_t ifnum;
>  	uint16_t type;
> +	bool media_device;
>  	const void *data;
>  };
>  
> 

Re: [PATCH v4 3/3] sound/usb: Use Media Controller API to share media resources

[Mauro Carvalho Chehab]

Em Wed, 16 Nov 2016 07:29:11 -0700
Shuah Khan <shuahkh@osg.samsung.com> escreveu:

> Change ALSA driver to use Media Controller API to share media resources
> with DVB, and V4L2 drivers on a AU0828 media device.
> 
> Media Controller specific initialization is done after sound card is
> registered.  ALSA creates Media interface and entity function graph
> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
> Changes to patch 0003 since the last version in 4.7-rc1:
> - Included fixes to bugs found during testing. 
> - Updated to use the Media Allocator API.
> 
>  sound/usb/Kconfig        |   4 +
>  sound/usb/Makefile       |   2 +
>  sound/usb/card.c         |  14 +++
>  sound/usb/card.h         |   3 +
>  sound/usb/media.c        | 314 +++++++++++++++++++++++++++++++++++++++++++++++
>  sound/usb/media.h        |  73 +++++++++++
>  sound/usb/mixer.h        |   3 +
>  sound/usb/pcm.c          |  28 ++++-
>  sound/usb/quirks-table.h |   1 +
>  sound/usb/stream.c       |   2 +
>  sound/usb/usbaudio.h     |   6 +
>  11 files changed, 445 insertions(+), 5 deletions(-)
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
> 
> diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
> index a452ad7..d14bf41 100644
> --- a/sound/usb/Kconfig
> +++ b/sound/usb/Kconfig
> @@ -15,6 +15,7 @@ config SND_USB_AUDIO
>  	select SND_RAWMIDI
>  	select SND_PCM
>  	select BITREVERSE
> +	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
>  	help
>  	  Say Y here to include support for USB audio and USB MIDI
>  	  devices.
> @@ -22,6 +23,9 @@ config SND_USB_AUDIO
>  	  To compile this driver as a module, choose M here: the module
>  	  will be called snd-usb-audio.
>  
> +config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +	bool
> +
>  config SND_USB_UA101
>  	tristate "Edirol UA-101/UA-1000 driver"
>  	select SND_PCM
> diff --git a/sound/usb/Makefile b/sound/usb/Makefile
> index 2d2d122..8dca3c4 100644
> --- a/sound/usb/Makefile
> +++ b/sound/usb/Makefile
> @@ -15,6 +15,8 @@ snd-usb-audio-objs := 	card.o \
>  			quirks.o \
>  			stream.o
>  
> +snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
> +
>  snd-usbmidi-lib-objs := midi.o
>  
>  # Toplevel Module Dependency
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 9e5276d..2750e56 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -66,6 +66,7 @@
>  #include "format.h"
>  #include "power.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
>  MODULE_DESCRIPTION("USB Audio");
> @@ -615,6 +616,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>  	if (err < 0)
>  		goto __error;
>  
> +	if (quirk && quirk->media_device) {
> +		/* don't want to fail when media_snd_device_create() fails */
> +		media_snd_device_create(chip, intf);
> +	}
> +
>  	usb_chip[chip->index] = chip;
>  	chip->num_interfaces++;
>  	usb_set_intfdata(intf, chip);
> @@ -671,6 +677,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>  		list_for_each(p, &chip->midi_list) {
>  			snd_usbmidi_disconnect(p);
>  		}
> +		/*
> +		 * Nice to check quirk && quirk->media_device and
> +		 * then call media_snd_device_delete(). Don't have
> +		 * access to quirk here. media_snd_device_delete()
> +		 * acceses mixer_list
> +		 */
> +		media_snd_device_delete(chip);
> +
>  		/* release mixer resources */
>  		list_for_each_entry(mixer, &chip->mixer_list, list) {
>  			snd_usb_mixer_disconnect(mixer);
> diff --git a/sound/usb/card.h b/sound/usb/card.h
> index 111b0f0..6ef8e88 100644
> --- a/sound/usb/card.h
> +++ b/sound/usb/card.h
> @@ -105,6 +105,8 @@ struct snd_usb_endpoint {
>  	struct list_head list;
>  };
>  
> +struct media_ctl;
> +
>  struct snd_usb_substream {
>  	struct snd_usb_stream *stream;
>  	struct usb_device *dev;
> @@ -156,6 +158,7 @@ struct snd_usb_substream {
>  	} dsd_dop;
>  
>  	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
> +	struct media_ctl *media_ctl;
>  };
>  
>  struct snd_usb_stream {
> diff --git a/sound/usb/media.c b/sound/usb/media.c
> new file mode 100644
> index 0000000..ccdc4d9
> --- /dev/null
> +++ b/sound/usb/media.c
> @@ -0,0 +1,314 @@
> +/*
> + * media.c - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +
> +#include <linux/init.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include <sound/pcm.h>
> +#include <sound/core.h>
> +
> +#include "usbaudio.h"
> +#include "card.h"
> +#include "mixer.h"
> +#include "media.h"
> +
> +static int media_snd_enable_source(struct media_ctl *mctl)
> +{
> +	if (mctl && mctl->media_dev->enable_source)
> +		return mctl->media_dev->enable_source(&mctl->media_entity,
> +						      &mctl->media_pipe);

What happens if the module implementing enable_source is
removed while running this code? I guess you need to
protect it.

> +	return 0;
> +}
> +
> +static void media_snd_disable_source(struct media_ctl *mctl)
> +{
> +	if (mctl && mctl->media_dev->disable_source)
> +		mctl->media_dev->disable_source(&mctl->media_entity);

Same here.

> +}
> +
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			int stream)
> +{
> +	struct media_device *mdev;
> +	struct media_ctl *mctl;
> +	struct device *pcm_dev = &pcm->streams[stream].dev;
> +	u32 intf_type;
> +	int ret = 0;
> +	u16 mixer_pad;
> +	struct media_entity *entity;
> +
> +	mdev = subs->stream->chip->media_dev;
> +	if (!mdev)
> +		return 0;
> +
> +	if (subs->media_ctl)
> +		return 0;
> +
> +	/* allocate media_ctl */
> +	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +	if (!mctl)
> +		return -ENOMEM;
> +
> +	mctl->media_dev = mdev;
> +	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
> +		mixer_pad = 1;
> +	} else {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
> +		mixer_pad = 2;
> +	}
> +	mctl->media_entity.name = pcm->name;
> +	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
> +	ret =  media_device_register_entity(mctl->media_dev,
> +					    &mctl->media_entity);
> +	if (ret)
> +		goto free_mctl;
> +
> +	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
> +						  MAJOR(pcm_dev->devt),
> +						  MINOR(pcm_dev->devt));
> +	if (!mctl->intf_devnode) {
> +		ret = -ENOMEM;
> +		goto unregister_entity;
> +	}
> +	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +						 &mctl->intf_devnode->intf,
> +						 MEDIA_LNK_FL_ENABLED);
> +	if (!mctl->intf_link) {
> +		ret = -ENOMEM;
> +		goto devnode_remove;
> +	}
> +
> +	/* create link between mixer and audio */
> +	media_device_for_each_entity(entity, mdev) {
> +		switch (entity->function) {
> +		case MEDIA_ENT_F_AUDIO_MIXER:
> +			ret = media_create_pad_link(entity, mixer_pad,
> +						    &mctl->media_entity, 0,
> +						    MEDIA_LNK_FL_ENABLED);
> +			if (ret)
> +				goto remove_intf_link;
> +			break;
> +		}
> +	}
> +
> +	subs->media_ctl = mctl;
> +	return 0;
> +
> +remove_intf_link:
> +	media_remove_intf_link(mctl->intf_link);
> +devnode_remove:
> +	media_devnode_remove(mctl->intf_devnode);
> +unregister_entity:
> +	media_device_unregister_entity(&mctl->media_entity);
> +free_mctl:
> +	kfree(mctl);
> +	return ret;
> +}
> +
> +void media_snd_stream_delete(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl && mctl->media_dev) {
> +		struct media_device *mdev;
> +
> +		mdev = mctl->media_dev;
> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
> +			media_devnode_remove(mctl->intf_devnode);
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		subs->media_ctl = NULL;
> +	}
> +}
> +
> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		return media_snd_enable_source(mctl);
> +	return 0;
> +}
> +
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		media_snd_disable_source(mctl);
> +}
> +
> +static int media_snd_mixer_init(struct snd_usb_audio *chip)
> +{
> +	struct device *ctl_dev = &chip->card->ctl_dev;
> +	struct media_intf_devnode *ctl_intf;
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +	struct media_mixer_ctl *mctl;
> +	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
> +	int ret;
> +
> +	if (!mdev)
> +		return -ENODEV;
> +
> +	ctl_intf = chip->ctl_intf_media_devnode;
> +	if (!ctl_intf) {
> +		ctl_intf = media_devnode_create(mdev, intf_type, 0,
> +						MAJOR(ctl_dev->devt),
> +						MINOR(ctl_dev->devt));
> +		if (!ctl_intf)
> +			return -ENOMEM;
> +		chip->ctl_intf_media_devnode = ctl_intf;
> +	}
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +
> +		if (mixer->media_mixer_ctl)
> +			continue;
> +
> +		/* allocate media_mixer_ctl */
> +		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +		if (!mctl)
> +			return -ENOMEM;
> +
> +		mctl->media_dev = mdev;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
> +		mctl->media_entity.name = chip->card->mixername;
> +		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
> +		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
> +		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
> +				  mctl->media_pad);
> +		ret =  media_device_register_entity(mctl->media_dev,
> +						    &mctl->media_entity);
> +		if (ret) {
> +			kfree(mctl);
> +			return ret;
> +		}
> +
> +		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +							 &ctl_intf->intf,
> +							 MEDIA_LNK_FL_ENABLED);
> +		if (!mctl->intf_link) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +			kfree(mctl);
> +			return -ENOMEM;
> +		}
> +		mctl->intf_devnode = ctl_intf;
> +		mixer->media_mixer_ctl = mctl;
> +	}
> +	return 0;
> +}
> +
> +static void media_snd_mixer_delete(struct snd_usb_audio *chip)
> +{
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +
> +	if (!mdev)
> +		return;
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +		struct media_mixer_ctl *mctl;
> +
> +		mctl = mixer->media_mixer_ctl;
> +		if (!mixer->media_mixer_ctl)
> +			continue;
> +
> +		if (media_devnode_is_registered(mdev->devnode)) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		mixer->media_mixer_ctl = NULL;
> +	}
> +	if (media_devnode_is_registered(mdev->devnode))
> +		media_devnode_remove(chip->ctl_intf_media_devnode);
> +	chip->ctl_intf_media_devnode = NULL;
> +}
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			struct usb_interface *iface)
> +{
> +	struct media_device *mdev;
> +	struct usb_device *usbdev = interface_to_usbdev(iface);
> +	int ret;
> +
> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
> +	if (!mdev)
> +		return -ENOMEM;
> +
> +	if (!media_devnode_is_registered(mdev->devnode)) {
> +		/* register media_device */
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			media_device_delete(mdev, KBUILD_MODNAME);
> +			dev_err(&usbdev->dev,
> +				"Couldn't register media device. Error: %d\n",
> +				ret);
> +			return ret;
> +		}
> +	}
> +
> +	/* save media device - avoid lookups */
> +	chip->media_dev = mdev;
> +
> +	/* Create media entities for mixer and control dev */
> +	ret = media_snd_mixer_init(chip);
> +	if (ret) {
> +		dev_err(&usbdev->dev,
> +			"Couldn't create media mixer entities. Error: %d\n",
> +			ret);
> +
> +		/* clear saved media_dev */
> +		chip->media_dev = NULL;
> +
> +		return ret;
> +	}
> +	return 0;
> +}
> +
> +void media_snd_device_delete(struct snd_usb_audio *chip)
> +{
> +	struct media_device *mdev = chip->media_dev;
> +	struct snd_usb_stream *stream;
> +
> +	/* release resources */
> +	list_for_each_entry(stream, &chip->pcm_list, list) {
> +		media_snd_stream_delete(&stream->substream[0]);
> +		media_snd_stream_delete(&stream->substream[1]);
> +	}
> +
> +	media_snd_mixer_delete(chip);
> +
> +	if (mdev) {
> +		media_device_delete(mdev, KBUILD_MODNAME);
> +		chip->media_dev = NULL;
> +	}
> +}
> diff --git a/sound/usb/media.h b/sound/usb/media.h
> new file mode 100644
> index 0000000..d204375
> --- /dev/null
> +++ b/sound/usb/media.h
> @@ -0,0 +1,73 @@
> +/*
> + * media.h - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +#ifndef __MEDIA_H
> +
> +#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +
> +#include <media/media-device.h>
> +#include <media/media-entity.h>
> +#include <media/media-dev-allocator.h>
> +#include <sound/asound.h>
> +
> +struct media_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad;
> +	struct media_pipeline media_pipe;
> +};
> +
> +/*
> + * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
> + * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
> + * to AUDIO Source
> + */
> +#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
> +
> +struct media_mixer_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
> +	struct media_pipeline media_pipe;
> +};
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			    struct usb_interface *iface);
> +void media_snd_device_delete(struct snd_usb_audio *chip);
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			  int stream);
> +void media_snd_stream_delete(struct snd_usb_substream *subs);
> +int media_snd_start_pipeline(struct snd_usb_substream *subs);
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs);
> +#else
> +static inline int media_snd_device_create(struct snd_usb_audio *chip,
> +					  struct usb_interface *iface)
> +						{ return 0; }
> +static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
> +static inline int media_snd_stream_init(struct snd_usb_substream *subs,
> +					struct snd_pcm *pcm, int stream)
> +						{ return 0; }
> +static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
> +static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +					{ return 0; }
> +static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
> +#endif
> +#endif /* __MEDIA_H */
> diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
> index 3417ef3..f378944 100644
> --- a/sound/usb/mixer.h
> +++ b/sound/usb/mixer.h
> @@ -3,6 +3,8 @@
>  
>  #include <sound/info.h>
>  
> +struct media_mixer_ctl;
> +
>  struct usb_mixer_interface {
>  	struct snd_usb_audio *chip;
>  	struct usb_host_interface *hostif;
> @@ -22,6 +24,7 @@ struct usb_mixer_interface {
>  	struct urb *rc_urb;
>  	struct usb_ctrlrequest *rc_setup_packet;
>  	u8 rc_buffer[6];
> +	struct media_mixer_ctl *media_mixer_ctl;
>  };
>  
>  #define MAX_CHANNELS	16	/* max logical channels */
> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
> index 44d178e..0e4e0640 100644
> --- a/sound/usb/pcm.c
> +++ b/sound/usb/pcm.c
> @@ -35,6 +35,7 @@
>  #include "pcm.h"
>  #include "clock.h"
>  #include "power.h"
> +#include "media.h"
>  
>  #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
>  #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
> @@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  	struct audioformat *fmt;
>  	int ret;
>  
> +	ret = media_snd_start_pipeline(subs);
> +	if (ret)
> +		return ret;
> +
>  	ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
>  					       params_buffer_bytes(hw_params));
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->pcm_format = params_format(hw_params);
>  	subs->period_bytes = params_period_bytes(hw_params);
> @@ -734,22 +739,27 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  		dev_dbg(&subs->dev->dev,
>  			"cannot set format: format = %#x, rate = %d, channels = %d\n",
>  			   subs->pcm_format, subs->cur_rate, subs->channels);
> -		return -EINVAL;
> +		ret = -EINVAL;
> +		goto err_ret;
>  	}
>  
>  	ret = snd_usb_lock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  	ret = set_format(subs, fmt);
>  	snd_usb_unlock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->interface = fmt->iface;
>  	subs->altset_idx = fmt->altset_idx;
>  	subs->need_setup_ep = true;
>  
>  	return 0;
> +
> +err_ret:
> +	media_snd_stop_pipeline(subs);
> +	return ret;
>  }
>  
>  /*
> @@ -761,6 +771,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
>  {
>  	struct snd_usb_substream *subs = substream->runtime->private_data;
>  
> +	media_snd_stop_pipeline(subs);
>  	subs->cur_audiofmt = NULL;
>  	subs->cur_rate = 0;
>  	subs->period_bytes = 0;
> @@ -1221,6 +1232,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
>  	struct snd_pcm_runtime *runtime = substream->runtime;
>  	struct snd_usb_substream *subs = &as->substream[direction];
> +	int ret;
>  
>  	subs->interface = -1;
>  	subs->altset_idx = 0;
> @@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	subs->dsd_dop.channel = 0;
>  	subs->dsd_dop.marker = 1;
>  
> -	return setup_hw_info(runtime, subs);
> +	ret = setup_hw_info(runtime, subs);
> +	if (ret == 0)
> +		ret = media_snd_stream_init(subs, as->pcm, direction);
> +	if (ret)
> +		snd_usb_autosuspend(subs->stream->chip);
> +	return ret;
>  }
>  
>  static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
> @@ -1243,6 +1260,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_substream *subs = &as->substream[direction];
>  
>  	stop_endpoints(subs, true);
> +	media_snd_stop_pipeline(subs);
>  
>  	if (subs->interface >= 0 &&
>  	    !snd_usb_lock_shutdown(subs->stream->chip)) {
> diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
> index 8a59d47..164e249 100644
> --- a/sound/usb/quirks-table.h
> +++ b/sound/usb/quirks-table.h
> @@ -2886,6 +2886,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
>  		.product_name = pname, \
>  		.ifnum = QUIRK_ANY_INTERFACE, \
>  		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
> +		.media_device = 1, \
>  	} \
>  }
>  
> diff --git a/sound/usb/stream.c b/sound/usb/stream.c
> index 8e9548bc..6fe7f21 100644
> --- a/sound/usb/stream.c
> +++ b/sound/usb/stream.c
> @@ -36,6 +36,7 @@
>  #include "format.h"
>  #include "clock.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  /*
>   * free a substream
> @@ -52,6 +53,7 @@ static void free_substream(struct snd_usb_substream *subs)
>  		kfree(fp);
>  	}
>  	kfree(subs->rate_list.list);
> +	media_snd_stream_delete(subs);
>  }
>  
>  
> diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
> index 4d5c89a..89b6853 100644
> --- a/sound/usb/usbaudio.h
> +++ b/sound/usb/usbaudio.h
> @@ -30,6 +30,9 @@
>   *
>   */
>  
> +struct media_device;
> +struct media_intf_devnode;
> +
>  struct snd_usb_audio {
>  	int index;
>  	struct usb_device *dev;
> @@ -61,6 +64,8 @@ struct snd_usb_audio {
>  	bool autoclock;			/* from the 'autoclock' module param */
>  
>  	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
> +	struct media_device *media_dev;
> +	struct media_intf_devnode *ctl_intf_media_devnode;
>  };
>  
>  #define usb_audio_err(chip, fmt, args...) \
> @@ -111,6 +116,7 @@ struct snd_usb_audio_quirk {
>  	const char *product_name;
>  	int16_t ifnum;
>  	uint16_t type;
> +	bool media_device;
>  	const void *data;
>  };
>  



Thanks,
Mauro

Re: [PATCH v4 0/3] Media Device Allocator API

[Shuah Khan]

Sorry for the duplicates. Replying to catch bounced emails on the original
patch emails from git send-email

On 11/16/2016 07:29 AM, Shuah Khan wrote:
> Media Device Allocator API to allows multiple drivers share a media device.
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Patches 0001 and 0002 are rebased to 4.9-rc4. Patch 0003 for snd-usb-audio
> is a rebase of the patch that was tested with the original Media Device
> Allocator patch series.
> 
> snd-usb-audio patch includes the fixes found during 4.7-rc1 time in the
> original snd-usb-audio patch.
> 
> Changes to patch 0001 since v3:
> - Fixed undefined reference to `__media_device_usb_init compile error when
>   CONFIG_USB is disabled.
> - Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
>   module that owns the media_device. The fix bumps the reference count for
>   the owner when second driver comes along to share the media_device. If
>   au0828 owns the media_device, then snd_usb_audio will bump the refcount
>   for au0828, so it won't get deleted and vice versa.
> 
> Changes to patch 0002 since v2:
> - Updated media_device_delete() to pass in module name.
> 
> Changes to patch 0003 since the last version in 4.7-rc1:
> - Included fixes to bugs found during testing. 
> - Updated to use the Media Allocator API.
> 
> This patch series has been tested with au0828 and snd-usb-audio drivers.
> Ran bind and unbind loop tests on each driver with mc_nextgen_test and
> media_device_test app loop tests while checking lsmod and dmesg.
> 
> Please refer to tools/testing/selftests/media_tests/regression_test.txt
> for testing done on this series.
> 
> Shuah Khan (3):
>   media: Media Device Allocator API
>   media: change au0828 to use Media Device Allocator API
>   sound/usb: Use Media Controller API to share media resources
> 
>  drivers/media/Makefile                 |   3 +-
>  drivers/media/media-dev-allocator.c    | 146 +++++++++++++++
>  drivers/media/usb/au0828/au0828-core.c |  12 +-
>  drivers/media/usb/au0828/au0828.h      |   1 +
>  include/media/media-dev-allocator.h    |  87 +++++++++
>  sound/usb/Kconfig                      |   4 +
>  sound/usb/Makefile                     |   2 +
>  sound/usb/card.c                       |  14 ++
>  sound/usb/card.h                       |   3 +
>  sound/usb/media.c                      | 314 +++++++++++++++++++++++++++++++++
>  sound/usb/media.h                      |  73 ++++++++
>  sound/usb/mixer.h                      |   3 +
>  sound/usb/pcm.c                        |  28 ++-
>  sound/usb/quirks-table.h               |   1 +
>  sound/usb/stream.c                     |   2 +
>  sound/usb/usbaudio.h                   |   6 +
>  16 files changed, 685 insertions(+), 14 deletions(-)
>  create mode 100644 drivers/media/media-dev-allocator.c
>  create mode 100644 include/media/media-dev-allocator.h
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
>