* [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS
@ 2022-09-07 14:02 Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 1/4] media: v4l: move helper functions for fractions from uvc to v4l2-common Michael Grzeschik
` (4 more replies)
0 siblings, 5 replies; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:02 UTC (permalink / raw)
To: linux-usb
Cc: linux-media, balbi, laurent.pinchart, paul.elder, kernel, nicolas,
kieran.bingham
This series improves the uvc video gadget by parsing the configfs
entries. With the configfs data, the driver now is able to negotiate the
format with the usb host in the kernel and also exports the supported
frames/formats/intervals via the v4l2 VIDIOC interface.
The uvc userspace stack is also under development. One example is an generic
v4l2uvcsink gstreamer elemnt, which is currently under discussion. [1]
[1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1304
With the libusbgx library [1] used by the gadget-tool [2] it is now also
possible to fully describe the configfs layout of the uvc gadget with scheme
files.
[2] https://github.com/linux-usb-gadgets/libusbgx/pull/61/commits/53231c76f9d512f59fdc23b65cd5c46b7fb09eb4
[3] https://github.com/linux-usb-gadgets/gt/tree/master/examples/systemd
The bigger picture of these patches is to provide a more versatile interface to
the uvc gadget. The goal is to simply start a uvc-gadget with the following
commands:
$ gt load uvc.scheme
$ gst-launch v4l2src ! v4l2uvcsink
--
v1: https://lore.kernel.org/linux-usb/20210530222239.8793-1-m.grzeschik@pengutronix.de/
v2: https://lore.kernel.org/linux-usb/20211117004432.3763306-1-m.grzeschik@pengutronix.de/
v3: https://lore.kernel.org/linux-usb/20211117122435.2409362-1-m.grzeschik@pengutronix.de/
v4: https://lore.kernel.org/linux-usb/20211205225803.268492-1-m.grzeschik@pengutronix.de/
v5: https://lore.kernel.org/linux-usb/20211209084322.2662616-1-m.grzeschik@pengutronix.de/
v6: https://lore.kernel.org/linux-usb/20220105115527.3592860-1-m.grzeschik@pengutronix.de/
v7: https://lore.kernel.org/linux-usb/20220608105748.139922-1-m.grzeschik@pengutronix.de/
Regards,
Michael
Michael Grzeschik (4):
media: v4l: move helper functions for fractions from uvc to v4l2-common
media: uvcvideo: move uvc_format_desc to common header
usb: gadget: uvc: add VIDIOC function
usb: gadget: uvc: add format/frame handling code
drivers/media/usb/uvc/uvc_ctrl.c | 1 +
drivers/media/usb/uvc/uvc_driver.c | 290 +----------------
drivers/media/usb/uvc/uvc_v4l2.c | 14 +-
drivers/media/usb/uvc/uvcvideo.h | 147 ---------
drivers/media/v4l2-core/v4l2-common.c | 86 +++++
drivers/usb/gadget/function/f_uvc.c | 270 +++++++++++++++-
drivers/usb/gadget/function/uvc.h | 39 ++-
drivers/usb/gadget/function/uvc_queue.c | 3 +-
drivers/usb/gadget/function/uvc_v4l2.c | 413 +++++++++++++++++++++---
drivers/usb/gadget/function/uvc_video.c | 71 +++-
include/media/v4l2-common.h | 4 +
include/media/v4l2-uvc.h | 359 ++++++++++++++++++++
12 files changed, 1196 insertions(+), 501 deletions(-)
create mode 100644 include/media/v4l2-uvc.h
--
2.30.2
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v8 1/4] media: v4l: move helper functions for fractions from uvc to v4l2-common
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
@ 2022-09-07 14:02 ` Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 2/4] media: uvcvideo: move uvc_format_desc to common header Michael Grzeschik
` (3 subsequent siblings)
4 siblings, 0 replies; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:02 UTC (permalink / raw)
To: linux-usb
Cc: linux-media, balbi, laurent.pinchart, paul.elder, kernel, nicolas,
kieran.bingham
The functions uvc_simplify_fraction and uvc_fraction_to_interval are
generic helpers which are also useful for other v4l2 drivers. This patch
moves them to v4l2-common.
Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
---
v1 -> v7: -
v7 -> v8: - ported all style fixes and broken links from latest version on rebase
drivers/media/usb/uvc/uvc_driver.c | 84 --------------------------
drivers/media/usb/uvc/uvc_v4l2.c | 14 ++---
drivers/media/usb/uvc/uvcvideo.h | 3 -
drivers/media/v4l2-core/v4l2-common.c | 86 +++++++++++++++++++++++++++
include/media/v4l2-common.h | 4 ++
5 files changed, 97 insertions(+), 94 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 9c05776f11d1f0..0f14dee4b6d794 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -329,90 +329,6 @@ static enum v4l2_ycbcr_encoding uvc_ycbcr_enc(const u8 matrix_coefficients)
return V4L2_YCBCR_ENC_DEFAULT; /* Reserved */
}
-/*
- * Simplify a fraction using a simple continued fraction decomposition. The
- * idea here is to convert fractions such as 333333/10000000 to 1/30 using
- * 32 bit arithmetic only. The algorithm is not perfect and relies upon two
- * arbitrary parameters to remove non-significative terms from the simple
- * continued fraction decomposition. Using 8 and 333 for n_terms and threshold
- * respectively seems to give nice results.
- */
-void uvc_simplify_fraction(u32 *numerator, u32 *denominator,
- unsigned int n_terms, unsigned int threshold)
-{
- u32 *an;
- u32 x, y, r;
- unsigned int i, n;
-
- an = kmalloc_array(n_terms, sizeof(*an), GFP_KERNEL);
- if (an == NULL)
- return;
-
- /*
- * Convert the fraction to a simple continued fraction. See
- * https://en.wikipedia.org/wiki/Continued_fraction
- * Stop if the current term is bigger than or equal to the given
- * threshold.
- */
- x = *numerator;
- y = *denominator;
-
- for (n = 0; n < n_terms && y != 0; ++n) {
- an[n] = x / y;
- if (an[n] >= threshold) {
- if (n < 2)
- n++;
- break;
- }
-
- r = x - an[n] * y;
- x = y;
- y = r;
- }
-
- /* Expand the simple continued fraction back to an integer fraction. */
- x = 0;
- y = 1;
-
- for (i = n; i > 0; --i) {
- r = y;
- y = an[i-1] * y + x;
- x = r;
- }
-
- *numerator = y;
- *denominator = x;
- kfree(an);
-}
-
-/*
- * Convert a fraction to a frame interval in 100ns multiples. The idea here is
- * to compute numerator / denominator * 10000000 using 32 bit fixed point
- * arithmetic only.
- */
-u32 uvc_fraction_to_interval(u32 numerator, u32 denominator)
-{
- u32 multiplier;
-
- /* Saturate the result if the operation would overflow. */
- if (denominator == 0 ||
- numerator/denominator >= ((u32)-1)/10000000)
- return (u32)-1;
-
- /*
- * Divide both the denominator and the multiplier by two until
- * numerator * multiplier doesn't overflow. If anyone knows a better
- * algorithm please let me know.
- */
- multiplier = 10000000;
- while (numerator > ((u32)-1)/multiplier) {
- multiplier /= 2;
- denominator /= 2;
- }
-
- return denominator ? numerator * multiplier / denominator : 0;
-}
-
/* ------------------------------------------------------------------------
* Terminal and unit management
*/
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index 4cc3fa6b8c9812..f4d4c33b6dfbd7 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -386,7 +386,7 @@ static int uvc_v4l2_get_streamparm(struct uvc_streaming *stream,
mutex_unlock(&stream->mutex);
denominator = 10000000;
- uvc_simplify_fraction(&numerator, &denominator, 8, 333);
+ v4l2_simplify_fraction(&numerator, &denominator, 8, 333);
memset(parm, 0, sizeof(*parm));
parm->type = stream->type;
@@ -427,7 +427,7 @@ static int uvc_v4l2_set_streamparm(struct uvc_streaming *stream,
else
timeperframe = parm->parm.output.timeperframe;
- interval = uvc_fraction_to_interval(timeperframe.numerator,
+ interval = v4l2_fraction_to_interval(timeperframe.numerator,
timeperframe.denominator);
uvc_dbg(stream->dev, FORMAT, "Setting frame interval to %u/%u (%u)\n",
timeperframe.numerator, timeperframe.denominator, interval);
@@ -481,7 +481,7 @@ static int uvc_v4l2_set_streamparm(struct uvc_streaming *stream,
/* Return the actual frame period. */
timeperframe.numerator = probe.dwFrameInterval;
timeperframe.denominator = 10000000;
- uvc_simplify_fraction(&timeperframe.numerator,
+ v4l2_simplify_fraction(&timeperframe.numerator,
&timeperframe.denominator, 8, 333);
if (parm->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
@@ -1275,7 +1275,7 @@ static int uvc_ioctl_enum_frameintervals(struct file *file, void *fh,
fival->discrete.numerator =
frame->dwFrameInterval[index];
fival->discrete.denominator = 10000000;
- uvc_simplify_fraction(&fival->discrete.numerator,
+ v4l2_simplify_fraction(&fival->discrete.numerator,
&fival->discrete.denominator, 8, 333);
} else {
fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
@@ -1285,11 +1285,11 @@ static int uvc_ioctl_enum_frameintervals(struct file *file, void *fh,
fival->stepwise.max.denominator = 10000000;
fival->stepwise.step.numerator = frame->dwFrameInterval[2];
fival->stepwise.step.denominator = 10000000;
- uvc_simplify_fraction(&fival->stepwise.min.numerator,
+ v4l2_simplify_fraction(&fival->stepwise.min.numerator,
&fival->stepwise.min.denominator, 8, 333);
- uvc_simplify_fraction(&fival->stepwise.max.numerator,
+ v4l2_simplify_fraction(&fival->stepwise.max.numerator,
&fival->stepwise.max.denominator, 8, 333);
- uvc_simplify_fraction(&fival->stepwise.step.numerator,
+ v4l2_simplify_fraction(&fival->stepwise.step.numerator,
&fival->stepwise.step.denominator, 8, 333);
}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 24c911aeebce56..ff710bdd38b3fd 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -911,9 +911,6 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
struct uvc_xu_control_query *xqry);
/* Utility functions */
-void uvc_simplify_fraction(u32 *numerator, u32 *denominator,
- unsigned int n_terms, unsigned int threshold);
-u32 uvc_fraction_to_interval(u32 numerator, u32 denominator);
struct usb_host_endpoint *uvc_find_endpoint(struct usb_host_interface *alts,
u8 epaddr);
u16 uvc_endpoint_max_bpi(struct usb_device *dev, struct usb_host_endpoint *ep);
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index e0fbe6ba4b6c49..40f56e044640d7 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -484,3 +484,89 @@ s64 v4l2_get_link_freq(struct v4l2_ctrl_handler *handler, unsigned int mul,
return freq > 0 ? freq : -EINVAL;
}
EXPORT_SYMBOL_GPL(v4l2_get_link_freq);
+
+/*
+ * Simplify a fraction using a simple continued fraction decomposition. The
+ * idea here is to convert fractions such as 333333/10000000 to 1/30 using
+ * 32 bit arithmetic only. The algorithm is not perfect and relies upon two
+ * arbitrary parameters to remove non-significative terms from the simple
+ * continued fraction decomposition. Using 8 and 333 for n_terms and threshold
+ * respectively seems to give nice results.
+ */
+void v4l2_simplify_fraction(u32 *numerator, u32 *denominator,
+ unsigned int n_terms, unsigned int threshold)
+{
+ u32 *an;
+ u32 x, y, r;
+ unsigned int i, n;
+
+ an = kmalloc_array(n_terms, sizeof(*an), GFP_KERNEL);
+ if (an == NULL)
+ return;
+
+ /*
+ * Convert the fraction to a simple continued fraction. See
+ * https://en.wikipedia.org/wiki/Continued_fraction
+ * Stop if the current term is bigger than or equal to the given
+ * threshold.
+ */
+ x = *numerator;
+ y = *denominator;
+
+ for (n = 0; n < n_terms && y != 0; ++n) {
+ an[n] = x / y;
+ if (an[n] >= threshold) {
+ if (n < 2)
+ n++;
+ break;
+ }
+
+ r = x - an[n] * y;
+ x = y;
+ y = r;
+ }
+
+ /* Expand the simple continued fraction back to an integer fraction. */
+ x = 0;
+ y = 1;
+
+ for (i = n; i > 0; --i) {
+ r = y;
+ y = an[i-1] * y + x;
+ x = r;
+ }
+
+ *numerator = y;
+ *denominator = x;
+ kfree(an);
+}
+EXPORT_SYMBOL_GPL(v4l2_simplify_fraction);
+
+/*
+ * Convert a fraction to a frame interval in 100ns multiples. The idea here is
+ * to compute numerator / denominator * 10000000 using 32 bit fixed point
+ * arithmetic only.
+ */
+u32 v4l2_fraction_to_interval(u32 numerator, u32 denominator)
+{
+ u32 multiplier;
+
+ /* Saturate the result if the operation would overflow. */
+ if (denominator == 0 ||
+ numerator/denominator >= ((u32)-1)/10000000)
+ return (u32)-1;
+
+ /*
+ * Divide both the denominator and the multiplier by two until
+ * numerator * multiplier doesn't overflow. If anyone knows a better
+ * algorithm please let me know.
+ */
+ multiplier = 10000000;
+ while (numerator > ((u32)-1)/multiplier) {
+ multiplier /= 2;
+ denominator /= 2;
+ }
+
+ return denominator ? numerator * multiplier / denominator : 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_fraction_to_interval);
diff --git a/include/media/v4l2-common.h b/include/media/v4l2-common.h
index b708d63995f458..725ff91b26e063 100644
--- a/include/media/v4l2-common.h
+++ b/include/media/v4l2-common.h
@@ -540,6 +540,10 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, u32 pixelformat,
s64 v4l2_get_link_freq(struct v4l2_ctrl_handler *handler, unsigned int mul,
unsigned int div);
+void v4l2_simplify_fraction(u32 *numerator, u32 *denominator,
+ unsigned int n_terms, unsigned int threshold);
+u32 v4l2_fraction_to_interval(u32 numerator, u32 denominator);
+
static inline u64 v4l2_buffer_get_timestamp(const struct v4l2_buffer *buf)
{
/*
--
2.30.2
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v8 2/4] media: uvcvideo: move uvc_format_desc to common header
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 1/4] media: v4l: move helper functions for fractions from uvc to v4l2-common Michael Grzeschik
@ 2022-09-07 14:02 ` Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 3/4] usb: gadget: uvc: add VIDIOC function Michael Grzeschik
` (2 subsequent siblings)
4 siblings, 0 replies; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:02 UTC (permalink / raw)
To: linux-usb
Cc: linux-media, balbi, laurent.pinchart, paul.elder, kernel, nicolas,
kieran.bingham
The uvc_format_desc, GUID defines and the uvc_format_by_guid helper is
also useful for the uvc gadget stack. This patch moves them to a common
header.
Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
---
v1 -> v7: -
v7 -> v8: - fixed style comments while rebasing
drivers/media/usb/uvc/uvc_ctrl.c | 1 +
drivers/media/usb/uvc/uvc_driver.c | 206 +----------------
drivers/media/usb/uvc/uvcvideo.h | 144 ------------
include/media/v4l2-uvc.h | 359 +++++++++++++++++++++++++++++
4 files changed, 361 insertions(+), 349 deletions(-)
create mode 100644 include/media/v4l2-uvc.h
diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index 8c208db9600b46..b8a00a51679f6d 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -18,6 +18,7 @@
#include <linux/workqueue.h>
#include <linux/atomic.h>
#include <media/v4l2-ctrls.h>
+#include <media/v4l2-uvc.h>
#include "uvcvideo.h"
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 0f14dee4b6d794..2891bc9d319280 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -20,6 +20,7 @@
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
+#include <media/v4l2-uvc.h>
#include "uvcvideo.h"
@@ -34,198 +35,6 @@ static unsigned int uvc_quirks_param = -1;
unsigned int uvc_dbg_param;
unsigned int uvc_timeout_param = UVC_CTRL_STREAMING_TIMEOUT;
-/* ------------------------------------------------------------------------
- * Video formats
- */
-
-static struct uvc_format_desc uvc_fmts[] = {
- {
- .name = "YUV 4:2:2 (YUYV)",
- .guid = UVC_GUID_FORMAT_YUY2,
- .fcc = V4L2_PIX_FMT_YUYV,
- },
- {
- .name = "YUV 4:2:2 (YUYV)",
- .guid = UVC_GUID_FORMAT_YUY2_ISIGHT,
- .fcc = V4L2_PIX_FMT_YUYV,
- },
- {
- .name = "YUV 4:2:0 (NV12)",
- .guid = UVC_GUID_FORMAT_NV12,
- .fcc = V4L2_PIX_FMT_NV12,
- },
- {
- .name = "MJPEG",
- .guid = UVC_GUID_FORMAT_MJPEG,
- .fcc = V4L2_PIX_FMT_MJPEG,
- },
- {
- .name = "YVU 4:2:0 (YV12)",
- .guid = UVC_GUID_FORMAT_YV12,
- .fcc = V4L2_PIX_FMT_YVU420,
- },
- {
- .name = "YUV 4:2:0 (I420)",
- .guid = UVC_GUID_FORMAT_I420,
- .fcc = V4L2_PIX_FMT_YUV420,
- },
- {
- .name = "YUV 4:2:0 (M420)",
- .guid = UVC_GUID_FORMAT_M420,
- .fcc = V4L2_PIX_FMT_M420,
- },
- {
- .name = "YUV 4:2:2 (UYVY)",
- .guid = UVC_GUID_FORMAT_UYVY,
- .fcc = V4L2_PIX_FMT_UYVY,
- },
- {
- .name = "Greyscale 8-bit (Y800)",
- .guid = UVC_GUID_FORMAT_Y800,
- .fcc = V4L2_PIX_FMT_GREY,
- },
- {
- .name = "Greyscale 8-bit (Y8 )",
- .guid = UVC_GUID_FORMAT_Y8,
- .fcc = V4L2_PIX_FMT_GREY,
- },
- {
- .name = "Greyscale 8-bit (D3DFMT_L8)",
- .guid = UVC_GUID_FORMAT_D3DFMT_L8,
- .fcc = V4L2_PIX_FMT_GREY,
- },
- {
- .name = "IR 8-bit (L8_IR)",
- .guid = UVC_GUID_FORMAT_KSMEDIA_L8_IR,
- .fcc = V4L2_PIX_FMT_GREY,
- },
- {
- .name = "Greyscale 10-bit (Y10 )",
- .guid = UVC_GUID_FORMAT_Y10,
- .fcc = V4L2_PIX_FMT_Y10,
- },
- {
- .name = "Greyscale 12-bit (Y12 )",
- .guid = UVC_GUID_FORMAT_Y12,
- .fcc = V4L2_PIX_FMT_Y12,
- },
- {
- .name = "Greyscale 16-bit (Y16 )",
- .guid = UVC_GUID_FORMAT_Y16,
- .fcc = V4L2_PIX_FMT_Y16,
- },
- {
- .name = "BGGR Bayer (BY8 )",
- .guid = UVC_GUID_FORMAT_BY8,
- .fcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
- .name = "BGGR Bayer (BA81)",
- .guid = UVC_GUID_FORMAT_BA81,
- .fcc = V4L2_PIX_FMT_SBGGR8,
- },
- {
- .name = "GBRG Bayer (GBRG)",
- .guid = UVC_GUID_FORMAT_GBRG,
- .fcc = V4L2_PIX_FMT_SGBRG8,
- },
- {
- .name = "GRBG Bayer (GRBG)",
- .guid = UVC_GUID_FORMAT_GRBG,
- .fcc = V4L2_PIX_FMT_SGRBG8,
- },
- {
- .name = "RGGB Bayer (RGGB)",
- .guid = UVC_GUID_FORMAT_RGGB,
- .fcc = V4L2_PIX_FMT_SRGGB8,
- },
- {
- .name = "RGB565",
- .guid = UVC_GUID_FORMAT_RGBP,
- .fcc = V4L2_PIX_FMT_RGB565,
- },
- {
- .name = "BGR 8:8:8 (BGR3)",
- .guid = UVC_GUID_FORMAT_BGR3,
- .fcc = V4L2_PIX_FMT_BGR24,
- },
- {
- .name = "H.264",
- .guid = UVC_GUID_FORMAT_H264,
- .fcc = V4L2_PIX_FMT_H264,
- },
- {
- .name = "H.265",
- .guid = UVC_GUID_FORMAT_H265,
- .fcc = V4L2_PIX_FMT_HEVC,
- },
- {
- .name = "Greyscale 8 L/R (Y8I)",
- .guid = UVC_GUID_FORMAT_Y8I,
- .fcc = V4L2_PIX_FMT_Y8I,
- },
- {
- .name = "Greyscale 12 L/R (Y12I)",
- .guid = UVC_GUID_FORMAT_Y12I,
- .fcc = V4L2_PIX_FMT_Y12I,
- },
- {
- .name = "Depth data 16-bit (Z16)",
- .guid = UVC_GUID_FORMAT_Z16,
- .fcc = V4L2_PIX_FMT_Z16,
- },
- {
- .name = "Bayer 10-bit (SRGGB10P)",
- .guid = UVC_GUID_FORMAT_RW10,
- .fcc = V4L2_PIX_FMT_SRGGB10P,
- },
- {
- .name = "Bayer 16-bit (SBGGR16)",
- .guid = UVC_GUID_FORMAT_BG16,
- .fcc = V4L2_PIX_FMT_SBGGR16,
- },
- {
- .name = "Bayer 16-bit (SGBRG16)",
- .guid = UVC_GUID_FORMAT_GB16,
- .fcc = V4L2_PIX_FMT_SGBRG16,
- },
- {
- .name = "Bayer 16-bit (SRGGB16)",
- .guid = UVC_GUID_FORMAT_RG16,
- .fcc = V4L2_PIX_FMT_SRGGB16,
- },
- {
- .name = "Bayer 16-bit (SGRBG16)",
- .guid = UVC_GUID_FORMAT_GR16,
- .fcc = V4L2_PIX_FMT_SGRBG16,
- },
- {
- .name = "Depth data 16-bit (Z16)",
- .guid = UVC_GUID_FORMAT_INVZ,
- .fcc = V4L2_PIX_FMT_Z16,
- },
- {
- .name = "Greyscale 10-bit (Y10 )",
- .guid = UVC_GUID_FORMAT_INVI,
- .fcc = V4L2_PIX_FMT_Y10,
- },
- {
- .name = "IR:Depth 26-bit (INZI)",
- .guid = UVC_GUID_FORMAT_INZI,
- .fcc = V4L2_PIX_FMT_INZI,
- },
- {
- .name = "4-bit Depth Confidence (Packed)",
- .guid = UVC_GUID_FORMAT_CNF4,
- .fcc = V4L2_PIX_FMT_CNF4,
- },
- {
- .name = "HEVC",
- .guid = UVC_GUID_FORMAT_HEVC,
- .fcc = V4L2_PIX_FMT_HEVC,
- },
-};
-
/* ------------------------------------------------------------------------
* Utility functions
*/
@@ -245,19 +54,6 @@ struct usb_host_endpoint *uvc_find_endpoint(struct usb_host_interface *alts,
return NULL;
}
-static struct uvc_format_desc *uvc_format_by_guid(const u8 guid[16])
-{
- unsigned int len = ARRAY_SIZE(uvc_fmts);
- unsigned int i;
-
- for (i = 0; i < len; ++i) {
- if (memcmp(guid, uvc_fmts[i].guid, 16) == 0)
- return &uvc_fmts[i];
- }
-
- return NULL;
-}
-
static enum v4l2_colorspace uvc_colorspace(const u8 primaries)
{
static const enum v4l2_colorspace colorprimaries[] = {
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index ff710bdd38b3fd..df93db259312e7 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -41,144 +41,6 @@
#define UVC_EXT_GPIO_UNIT 0x7ffe
#define UVC_EXT_GPIO_UNIT_ID 0x100
-/* ------------------------------------------------------------------------
- * GUIDs
- */
-#define UVC_GUID_UVC_CAMERA \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}
-#define UVC_GUID_UVC_OUTPUT \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02}
-#define UVC_GUID_UVC_MEDIA_TRANSPORT_INPUT \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03}
-#define UVC_GUID_UVC_PROCESSING \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}
-#define UVC_GUID_UVC_SELECTOR \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x02}
-#define UVC_GUID_EXT_GPIO_CONTROLLER \
- {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x03}
-
-#define UVC_GUID_FORMAT_MJPEG \
- { 'M', 'J', 'P', 'G', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_YUY2 \
- { 'Y', 'U', 'Y', '2', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_YUY2_ISIGHT \
- { 'Y', 'U', 'Y', '2', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0x00, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_NV12 \
- { 'N', 'V', '1', '2', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_YV12 \
- { 'Y', 'V', '1', '2', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_I420 \
- { 'I', '4', '2', '0', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_UYVY \
- { 'U', 'Y', 'V', 'Y', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y800 \
- { 'Y', '8', '0', '0', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y8 \
- { 'Y', '8', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y10 \
- { 'Y', '1', '0', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y12 \
- { 'Y', '1', '2', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y16 \
- { 'Y', '1', '6', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_BY8 \
- { 'B', 'Y', '8', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_BA81 \
- { 'B', 'A', '8', '1', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_GBRG \
- { 'G', 'B', 'R', 'G', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_GRBG \
- { 'G', 'R', 'B', 'G', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_RGGB \
- { 'R', 'G', 'G', 'B', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_BG16 \
- { 'B', 'G', '1', '6', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_GB16 \
- { 'G', 'B', '1', '6', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_RG16 \
- { 'R', 'G', '1', '6', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_GR16 \
- { 'G', 'R', '1', '6', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_RGBP \
- { 'R', 'G', 'B', 'P', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_BGR3 \
- { 0x7d, 0xeb, 0x36, 0xe4, 0x4f, 0x52, 0xce, 0x11, \
- 0x9f, 0x53, 0x00, 0x20, 0xaf, 0x0b, 0xa7, 0x70}
-#define UVC_GUID_FORMAT_M420 \
- { 'M', '4', '2', '0', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-
-#define UVC_GUID_FORMAT_H264 \
- { 'H', '2', '6', '4', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_H265 \
- { 'H', '2', '6', '5', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y8I \
- { 'Y', '8', 'I', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Y12I \
- { 'Y', '1', '2', 'I', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_Z16 \
- { 'Z', '1', '6', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_RW10 \
- { 'R', 'W', '1', '0', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_INVZ \
- { 'I', 'N', 'V', 'Z', 0x90, 0x2d, 0x58, 0x4a, \
- 0x92, 0x0b, 0x77, 0x3f, 0x1f, 0x2c, 0x55, 0x6b}
-#define UVC_GUID_FORMAT_INZI \
- { 'I', 'N', 'Z', 'I', 0x66, 0x1a, 0x42, 0xa2, \
- 0x90, 0x65, 0xd0, 0x18, 0x14, 0xa8, 0xef, 0x8a}
-#define UVC_GUID_FORMAT_INVI \
- { 'I', 'N', 'V', 'I', 0xdb, 0x57, 0x49, 0x5e, \
- 0x8e, 0x3f, 0xf4, 0x79, 0x53, 0x2b, 0x94, 0x6f}
-#define UVC_GUID_FORMAT_CNF4 \
- { 'C', ' ', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-
-#define UVC_GUID_FORMAT_D3DFMT_L8 \
- {0x32, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-#define UVC_GUID_FORMAT_KSMEDIA_L8_IR \
- {0x32, 0x00, 0x00, 0x00, 0x02, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-
-#define UVC_GUID_FORMAT_HEVC \
- { 'H', 'E', 'V', 'C', 0x00, 0x00, 0x10, 0x00, \
- 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
-
-
/* ------------------------------------------------------------------------
* Driver specific constants.
*/
@@ -283,12 +145,6 @@ struct uvc_control {
struct uvc_fh *handle; /* File handle that last changed the control. */
};
-struct uvc_format_desc {
- char *name;
- u8 guid[16];
- u32 fcc;
-};
-
/*
* The term 'entity' refers to both UVC units and UVC terminals.
*
diff --git a/include/media/v4l2-uvc.h b/include/media/v4l2-uvc.h
new file mode 100644
index 00000000000000..f83e31661333bb
--- /dev/null
+++ b/include/media/v4l2-uvc.h
@@ -0,0 +1,359 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * v4l2 uvc internal API header
+ *
+ * Some commonly needed functions for uvc drivers
+ */
+
+#ifndef __LINUX_V4L2_UVC_H
+#define __LINUX_V4L2_UVC_H
+
+/* ------------------------------------------------------------------------
+ * GUIDs
+ */
+#define UVC_GUID_UVC_CAMERA \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01}
+#define UVC_GUID_UVC_OUTPUT \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02}
+#define UVC_GUID_UVC_MEDIA_TRANSPORT_INPUT \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03}
+#define UVC_GUID_UVC_PROCESSING \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01}
+#define UVC_GUID_UVC_SELECTOR \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x02}
+#define UVC_GUID_EXT_GPIO_CONTROLLER \
+ {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x03}
+
+#define UVC_GUID_FORMAT_MJPEG \
+ { 'M', 'J', 'P', 'G', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_YUY2 \
+ { 'Y', 'U', 'Y', '2', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_YUY2_ISIGHT \
+ { 'Y', 'U', 'Y', '2', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0x00, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_NV12 \
+ { 'N', 'V', '1', '2', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_YV12 \
+ { 'Y', 'V', '1', '2', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_I420 \
+ { 'I', '4', '2', '0', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_UYVY \
+ { 'U', 'Y', 'V', 'Y', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y800 \
+ { 'Y', '8', '0', '0', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y8 \
+ { 'Y', '8', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y10 \
+ { 'Y', '1', '0', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y12 \
+ { 'Y', '1', '2', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y16 \
+ { 'Y', '1', '6', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_BY8 \
+ { 'B', 'Y', '8', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_BA81 \
+ { 'B', 'A', '8', '1', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_GBRG \
+ { 'G', 'B', 'R', 'G', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_GRBG \
+ { 'G', 'R', 'B', 'G', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_RGGB \
+ { 'R', 'G', 'G', 'B', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_BG16 \
+ { 'B', 'G', '1', '6', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_GB16 \
+ { 'G', 'B', '1', '6', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_RG16 \
+ { 'R', 'G', '1', '6', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_GR16 \
+ { 'G', 'R', '1', '6', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_RGBP \
+ { 'R', 'G', 'B', 'P', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_BGR3 \
+ { 0x7d, 0xeb, 0x36, 0xe4, 0x4f, 0x52, 0xce, 0x11, \
+ 0x9f, 0x53, 0x00, 0x20, 0xaf, 0x0b, 0xa7, 0x70}
+#define UVC_GUID_FORMAT_M420 \
+ { 'M', '4', '2', '0', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+
+#define UVC_GUID_FORMAT_H264 \
+ { 'H', '2', '6', '4', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_H265 \
+ { 'H', '2', '6', '5', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y8I \
+ { 'Y', '8', 'I', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Y12I \
+ { 'Y', '1', '2', 'I', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_Z16 \
+ { 'Z', '1', '6', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_RW10 \
+ { 'R', 'W', '1', '0', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_INVZ \
+ { 'I', 'N', 'V', 'Z', 0x90, 0x2d, 0x58, 0x4a, \
+ 0x92, 0x0b, 0x77, 0x3f, 0x1f, 0x2c, 0x55, 0x6b}
+#define UVC_GUID_FORMAT_INZI \
+ { 'I', 'N', 'Z', 'I', 0x66, 0x1a, 0x42, 0xa2, \
+ 0x90, 0x65, 0xd0, 0x18, 0x14, 0xa8, 0xef, 0x8a}
+#define UVC_GUID_FORMAT_INVI \
+ { 'I', 'N', 'V', 'I', 0xdb, 0x57, 0x49, 0x5e, \
+ 0x8e, 0x3f, 0xf4, 0x79, 0x53, 0x2b, 0x94, 0x6f}
+#define UVC_GUID_FORMAT_CNF4 \
+ { 'C', ' ', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+
+#define UVC_GUID_FORMAT_D3DFMT_L8 \
+ {0x32, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+#define UVC_GUID_FORMAT_KSMEDIA_L8_IR \
+ {0x32, 0x00, 0x00, 0x00, 0x02, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+
+#define UVC_GUID_FORMAT_HEVC \
+ { 'H', 'E', 'V', 'C', 0x00, 0x00, 0x10, 0x00, \
+ 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
+
+/* ------------------------------------------------------------------------
+ * Video formats
+ */
+
+struct uvc_format_desc {
+ char *name;
+ u8 guid[16];
+ u32 fcc;
+};
+
+static struct uvc_format_desc uvc_fmts[] = {
+ {
+ .name = "YUV 4:2:2 (YUYV)",
+ .guid = UVC_GUID_FORMAT_YUY2,
+ .fcc = V4L2_PIX_FMT_YUYV,
+ },
+ {
+ .name = "YUV 4:2:2 (YUYV)",
+ .guid = UVC_GUID_FORMAT_YUY2_ISIGHT,
+ .fcc = V4L2_PIX_FMT_YUYV,
+ },
+ {
+ .name = "YUV 4:2:0 (NV12)",
+ .guid = UVC_GUID_FORMAT_NV12,
+ .fcc = V4L2_PIX_FMT_NV12,
+ },
+ {
+ .name = "MJPEG",
+ .guid = UVC_GUID_FORMAT_MJPEG,
+ .fcc = V4L2_PIX_FMT_MJPEG,
+ },
+ {
+ .name = "YVU 4:2:0 (YV12)",
+ .guid = UVC_GUID_FORMAT_YV12,
+ .fcc = V4L2_PIX_FMT_YVU420,
+ },
+ {
+ .name = "YUV 4:2:0 (I420)",
+ .guid = UVC_GUID_FORMAT_I420,
+ .fcc = V4L2_PIX_FMT_YUV420,
+ },
+ {
+ .name = "YUV 4:2:0 (M420)",
+ .guid = UVC_GUID_FORMAT_M420,
+ .fcc = V4L2_PIX_FMT_M420,
+ },
+ {
+ .name = "YUV 4:2:2 (UYVY)",
+ .guid = UVC_GUID_FORMAT_UYVY,
+ .fcc = V4L2_PIX_FMT_UYVY,
+ },
+ {
+ .name = "Greyscale 8-bit (Y800)",
+ .guid = UVC_GUID_FORMAT_Y800,
+ .fcc = V4L2_PIX_FMT_GREY,
+ },
+ {
+ .name = "Greyscale 8-bit (Y8 )",
+ .guid = UVC_GUID_FORMAT_Y8,
+ .fcc = V4L2_PIX_FMT_GREY,
+ },
+ {
+ .name = "Greyscale 8-bit (D3DFMT_L8)",
+ .guid = UVC_GUID_FORMAT_D3DFMT_L8,
+ .fcc = V4L2_PIX_FMT_GREY,
+ },
+ {
+ .name = "IR 8-bit (L8_IR)",
+ .guid = UVC_GUID_FORMAT_KSMEDIA_L8_IR,
+ .fcc = V4L2_PIX_FMT_GREY,
+ },
+ {
+ .name = "Greyscale 10-bit (Y10 )",
+ .guid = UVC_GUID_FORMAT_Y10,
+ .fcc = V4L2_PIX_FMT_Y10,
+ },
+ {
+ .name = "Greyscale 12-bit (Y12 )",
+ .guid = UVC_GUID_FORMAT_Y12,
+ .fcc = V4L2_PIX_FMT_Y12,
+ },
+ {
+ .name = "Greyscale 16-bit (Y16 )",
+ .guid = UVC_GUID_FORMAT_Y16,
+ .fcc = V4L2_PIX_FMT_Y16,
+ },
+ {
+ .name = "BGGR Bayer (BY8 )",
+ .guid = UVC_GUID_FORMAT_BY8,
+ .fcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .name = "BGGR Bayer (BA81)",
+ .guid = UVC_GUID_FORMAT_BA81,
+ .fcc = V4L2_PIX_FMT_SBGGR8,
+ },
+ {
+ .name = "GBRG Bayer (GBRG)",
+ .guid = UVC_GUID_FORMAT_GBRG,
+ .fcc = V4L2_PIX_FMT_SGBRG8,
+ },
+ {
+ .name = "GRBG Bayer (GRBG)",
+ .guid = UVC_GUID_FORMAT_GRBG,
+ .fcc = V4L2_PIX_FMT_SGRBG8,
+ },
+ {
+ .name = "RGGB Bayer (RGGB)",
+ .guid = UVC_GUID_FORMAT_RGGB,
+ .fcc = V4L2_PIX_FMT_SRGGB8,
+ },
+ {
+ .name = "RGB565",
+ .guid = UVC_GUID_FORMAT_RGBP,
+ .fcc = V4L2_PIX_FMT_RGB565,
+ },
+ {
+ .name = "BGR 8:8:8 (BGR3)",
+ .guid = UVC_GUID_FORMAT_BGR3,
+ .fcc = V4L2_PIX_FMT_BGR24,
+ },
+ {
+ .name = "H.264",
+ .guid = UVC_GUID_FORMAT_H264,
+ .fcc = V4L2_PIX_FMT_H264,
+ },
+ {
+ .name = "H.265",
+ .guid = UVC_GUID_FORMAT_H265,
+ .fcc = V4L2_PIX_FMT_HEVC,
+ },
+ {
+ .name = "Greyscale 8 L/R (Y8I)",
+ .guid = UVC_GUID_FORMAT_Y8I,
+ .fcc = V4L2_PIX_FMT_Y8I,
+ },
+ {
+ .name = "Greyscale 12 L/R (Y12I)",
+ .guid = UVC_GUID_FORMAT_Y12I,
+ .fcc = V4L2_PIX_FMT_Y12I,
+ },
+ {
+ .name = "Depth data 16-bit (Z16)",
+ .guid = UVC_GUID_FORMAT_Z16,
+ .fcc = V4L2_PIX_FMT_Z16,
+ },
+ {
+ .name = "Bayer 10-bit (SRGGB10P)",
+ .guid = UVC_GUID_FORMAT_RW10,
+ .fcc = V4L2_PIX_FMT_SRGGB10P,
+ },
+ {
+ .name = "Bayer 16-bit (SBGGR16)",
+ .guid = UVC_GUID_FORMAT_BG16,
+ .fcc = V4L2_PIX_FMT_SBGGR16,
+ },
+ {
+ .name = "Bayer 16-bit (SGBRG16)",
+ .guid = UVC_GUID_FORMAT_GB16,
+ .fcc = V4L2_PIX_FMT_SGBRG16,
+ },
+ {
+ .name = "Bayer 16-bit (SRGGB16)",
+ .guid = UVC_GUID_FORMAT_RG16,
+ .fcc = V4L2_PIX_FMT_SRGGB16,
+ },
+ {
+ .name = "Bayer 16-bit (SGRBG16)",
+ .guid = UVC_GUID_FORMAT_GR16,
+ .fcc = V4L2_PIX_FMT_SGRBG16,
+ },
+ {
+ .name = "Depth data 16-bit (Z16)",
+ .guid = UVC_GUID_FORMAT_INVZ,
+ .fcc = V4L2_PIX_FMT_Z16,
+ },
+ {
+ .name = "Greyscale 10-bit (Y10 )",
+ .guid = UVC_GUID_FORMAT_INVI,
+ .fcc = V4L2_PIX_FMT_Y10,
+ },
+ {
+ .name = "IR:Depth 26-bit (INZI)",
+ .guid = UVC_GUID_FORMAT_INZI,
+ .fcc = V4L2_PIX_FMT_INZI,
+ },
+ {
+ .name = "4-bit Depth Confidence (Packed)",
+ .guid = UVC_GUID_FORMAT_CNF4,
+ .fcc = V4L2_PIX_FMT_CNF4,
+ },
+ {
+ .name = "HEVC",
+ .guid = UVC_GUID_FORMAT_HEVC,
+ .fcc = V4L2_PIX_FMT_HEVC,
+ },
+};
+
+static inline struct uvc_format_desc *uvc_format_by_guid(const u8 guid[16])
+{
+ unsigned int len = ARRAY_SIZE(uvc_fmts);
+ unsigned int i;
+
+ for (i = 0; i < len; ++i) {
+ if (memcmp(guid, uvc_fmts[i].guid, 16) == 0)
+ return &uvc_fmts[i];
+ }
+
+ return NULL;
+}
+
+#endif /* __LINUX_V4L2_UVC_H */
--
2.30.2
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v8 3/4] usb: gadget: uvc: add VIDIOC function
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 1/4] media: v4l: move helper functions for fractions from uvc to v4l2-common Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 2/4] media: uvcvideo: move uvc_format_desc to common header Michael Grzeschik
@ 2022-09-07 14:02 ` Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code Michael Grzeschik
2022-09-07 14:28 ` [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Greg KH
4 siblings, 0 replies; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:02 UTC (permalink / raw)
To: linux-usb
Cc: linux-media, balbi, laurent.pinchart, paul.elder, kernel, nicolas,
kieran.bingham
This patch adds support to the v4l2 VIDIOC for enum_format,
enum_framesizes, enum_frameintervals and try_fmt. The linked/active
configfs userspace setup is used in the v4l2 interface functions.
With the v4l2 vidiocontrols the userspace can use the v4l2 api to
negotiate and allocate the data. It doesn't have to bring its own
configfs parser.
Also it only needs to be extended with subscription for streamon,
streamoff, connect and disconnect for stream handling and become able to
serve the uvc device.
Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
---
v1 -> v2:
- fixed indentation of find_frame/format_by_index
- fixed function name find_frm_by_size to find_frame_by_size
- fixed indentation of _uvc_v4l2_try_fmt
- fixed indentation in uvc_v4l2_enum_frameintervals
- removed unneeded declaration of uvc_v4l2_get_bytesperline in uvc_v4l2.h
- checked return values on config_group_find_item, handling refcount
- fixed sizeof using variables instead of types
- removed unsused def_format variable
- wrting grp, hdr, fmt and frm in full
- added proper ival handling
- removed analyze_configfs function
- added linked list of frames to uvcg_format
- added functon find_frame_by_index
v2 -> v3:
- fixed usage of u_uvc.h
- removed unused variable i in _try_fmt
- made uvc_v4l2_get_bytesperline static
v3 -> v4:
- conditionally return current or all frames/formats/frameintervals on enum
- dropped setting format and frame with set_format
- combined try and set format function to one call
v4 -> v5:
- fixed uninitialized return values reported by kernel test robot
- added local video variable to uvc_v4l2_enum_frameintervals
v5 -> v6:
-
v6 -> v7:
- fixed unlocking in f_uvc uvc_alloc function
- add uvc_get_frame_size function for sizeimage calculation
- add fallback to frame.dw_max_video_frame_buffer_size
v7 -> v8:
- replaced uvc_v4l2_set_fmt with noop function, since userspace can not set the format
- renamed function postfix _fmt to _format
drivers/usb/gadget/function/f_uvc.c | 33 +++
drivers/usb/gadget/function/uvc.h | 20 +-
drivers/usb/gadget/function/uvc_queue.c | 3 +-
drivers/usb/gadget/function/uvc_v4l2.c | 347 +++++++++++++++++++++---
drivers/usb/gadget/function/uvc_video.c | 61 ++++-
5 files changed, 417 insertions(+), 47 deletions(-)
diff --git a/drivers/usb/gadget/function/f_uvc.c b/drivers/usb/gadget/function/f_uvc.c
index f4f6cf75930beb..a8dcf5f01f16de 100644
--- a/drivers/usb/gadget/function/f_uvc.c
+++ b/drivers/usb/gadget/function/f_uvc.c
@@ -327,6 +327,8 @@ uvc_function_set_alt(struct usb_function *f, unsigned interface, unsigned alt)
if (uvc->video.ep)
usb_ep_disable(uvc->video.ep);
+ uvc->streamon = 0;
+
memset(&v4l2_event, 0, sizeof(v4l2_event));
v4l2_event.type = UVC_EVENT_STREAMOFF;
v4l2_event_queue(&uvc->vdev, &v4l2_event);
@@ -350,6 +352,8 @@ uvc_function_set_alt(struct usb_function *f, unsigned interface, unsigned alt)
return ret;
usb_ep_enable(uvc->video.ep);
+ uvc->streamon = 1;
+
memset(&v4l2_event, 0, sizeof(v4l2_event));
v4l2_event.type = UVC_EVENT_STREAMON;
v4l2_event_queue(&uvc->vdev, &v4l2_event);
@@ -888,6 +892,7 @@ static void uvc_free(struct usb_function *f)
struct uvc_device *uvc = to_uvc(f);
struct f_uvc_opts *opts = container_of(f->fi, struct f_uvc_opts,
func_inst);
+ config_item_put(&uvc->header->item);
--opts->refcnt;
kfree(uvc);
}
@@ -941,6 +946,7 @@ static struct usb_function *uvc_alloc(struct usb_function_instance *fi)
struct uvc_device *uvc;
struct f_uvc_opts *opts;
struct uvc_descriptor_header **strm_cls;
+ struct config_item *streaming, *header, *h;
uvc = kzalloc(sizeof(*uvc), GFP_KERNEL);
if (uvc == NULL)
@@ -973,6 +979,33 @@ static struct usb_function *uvc_alloc(struct usb_function_instance *fi)
uvc->desc.fs_streaming = opts->fs_streaming;
uvc->desc.hs_streaming = opts->hs_streaming;
uvc->desc.ss_streaming = opts->ss_streaming;
+
+ streaming = config_group_find_item(&opts->func_inst.group, "streaming");
+ if (!streaming) {
+ config_item_put(streaming);
+ mutex_unlock(&opts->lock);
+ return ERR_PTR(-ENOMEM);
+ }
+ header = config_group_find_item(to_config_group(streaming), "header");
+ config_item_put(streaming);
+ if (!header) {
+ config_item_put(header);
+ mutex_unlock(&opts->lock);
+ return ERR_PTR(-ENOMEM);
+ }
+ h = config_group_find_item(to_config_group(header), "h");
+ config_item_put(header);
+ if (!h) {
+ config_item_put(h);
+ mutex_unlock(&opts->lock);
+ return ERR_PTR(-ENOMEM);
+ }
+ uvc->header = to_uvcg_streaming_header(h);
+ if (!uvc->header->linked) {
+ mutex_unlock(&opts->lock);
+ return ERR_PTR(-EBUSY);
+ }
+
++opts->refcnt;
mutex_unlock(&opts->lock);
diff --git a/drivers/usb/gadget/function/uvc.h b/drivers/usb/gadget/function/uvc.h
index 58e383afdd4406..8a71d0c4abbcd5 100644
--- a/drivers/usb/gadget/function/uvc.h
+++ b/drivers/usb/gadget/function/uvc.h
@@ -90,11 +90,10 @@ struct uvc_video {
struct work_struct pump;
/* Frame parameters */
- u8 bpp;
- u32 fcc;
- unsigned int width;
- unsigned int height;
- unsigned int imagesize;
+ struct uvcg_format *cur_format;
+ struct uvcg_frame *cur_frame;
+ unsigned int cur_ival;
+
struct mutex mutex; /* protects frame parameters */
unsigned int uvc_num_requests;
@@ -133,6 +132,8 @@ struct uvc_device {
bool func_connected;
wait_queue_head_t func_connected_queue;
+ struct uvcg_streaming_header *header;
+
/* Descriptors */
struct {
const struct uvc_descriptor_header * const *fs_control;
@@ -142,6 +143,8 @@ struct uvc_device {
const struct uvc_descriptor_header * const *ss_streaming;
} desc;
+ bool streamon;
+
unsigned int control_intf;
struct usb_ep *control_ep;
struct usb_request *control_req;
@@ -178,4 +181,11 @@ extern void uvc_endpoint_stream(struct uvc_device *dev);
extern void uvc_function_connect(struct uvc_device *uvc);
extern void uvc_function_disconnect(struct uvc_device *uvc);
+extern int uvc_get_frame_size(struct uvc_video *video);
+extern struct uvcg_format *find_format_by_index(struct uvc_device *uvc,
+ int index);
+extern struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
+ struct uvcg_format *uformat,
+ int index);
+
#endif /* _UVC_GADGET_H_ */
diff --git a/drivers/usb/gadget/function/uvc_queue.c b/drivers/usb/gadget/function/uvc_queue.c
index ec500ee499eed1..7ac6b54ee14e94 100644
--- a/drivers/usb/gadget/function/uvc_queue.c
+++ b/drivers/usb/gadget/function/uvc_queue.c
@@ -21,6 +21,7 @@
#include <media/videobuf2-vmalloc.h>
#include "uvc.h"
+#include "uvc_configfs.h"
/* ------------------------------------------------------------------------
* Video buffers queue management.
@@ -52,7 +53,7 @@ static int uvc_queue_setup(struct vb2_queue *vq,
*nplanes = 1;
- sizes[0] = video->imagesize;
+ sizes[0] = uvc_get_frame_size(video);
req_size = video->ep->maxpacket
* max_t(unsigned int, video->ep->maxburst, 1)
diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c
index 511f106f984375..3311e98a040537 100644
--- a/drivers/usb/gadget/function/uvc_v4l2.c
+++ b/drivers/usb/gadget/function/uvc_v4l2.c
@@ -18,12 +18,161 @@
#include <media/v4l2-dev.h>
#include <media/v4l2-event.h>
#include <media/v4l2-ioctl.h>
+#include <media/v4l2-uvc.h>
#include "f_uvc.h"
#include "uvc.h"
#include "uvc_queue.h"
#include "uvc_video.h"
#include "uvc_v4l2.h"
+#include "uvc_configfs.h"
+
+static struct uvc_format_desc *to_uvc_format(struct uvcg_format *uformat)
+{
+ char guid[16] = UVC_GUID_FORMAT_MJPEG;
+ struct uvc_format_desc *format;
+ struct uvcg_uncompressed *unc;
+
+ if (uformat->type == UVCG_UNCOMPRESSED) {
+ unc = to_uvcg_uncompressed(&uformat->group.cg_item);
+ if (!unc)
+ return ERR_PTR(-EINVAL);
+
+ memcpy(guid, unc->desc.guidFormat, sizeof(guid));
+ }
+
+ format = uvc_format_by_guid(guid);
+ if (!format)
+ return ERR_PTR(-EINVAL);
+
+ return format;
+}
+
+static int uvc_v4l2_get_bytesperline(struct uvcg_format *uformat,
+ struct uvcg_frame *uframe)
+{
+ struct uvcg_uncompressed *u;
+
+ if (uformat->type == UVCG_UNCOMPRESSED) {
+ u = to_uvcg_uncompressed(&uformat->group.cg_item);
+ if (!u)
+ return 0;
+
+ return u->desc.bBitsPerPixel * uframe->frame.w_width / 8;
+ }
+
+ return 0;
+}
+
+int uvc_get_frame_size(struct uvc_video *video)
+{
+ unsigned int bpl = uvc_v4l2_get_bytesperline(video->cur_format,
+ video->cur_frame);
+
+ return bpl ? bpl * video->cur_frame->frame.w_height :
+ video->cur_frame->frame.dw_max_video_frame_buffer_size;
+}
+
+struct uvcg_format *find_format_by_index(struct uvc_device *uvc, int index)
+{
+ struct uvcg_format_ptr *format;
+ struct uvcg_format *uformat = NULL;
+ int i = 1;
+
+ list_for_each_entry(format, &uvc->header->formats, entry) {
+ if (index == i) {
+ uformat = format->fmt;
+ break;
+ }
+ i++;
+ }
+
+ return uformat;
+}
+
+struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
+ struct uvcg_format *uformat,
+ int index)
+{
+ struct uvcg_format_ptr *format;
+ struct uvcg_frame_ptr *frame;
+ struct uvcg_frame *uframe = NULL;
+
+ list_for_each_entry(format, &uvc->header->formats, entry) {
+ if (format->fmt->type != uformat->type)
+ continue;
+ list_for_each_entry(frame, &format->fmt->frames, entry) {
+ if (index == frame->frm->frame.b_frame_index) {
+ uframe = frame->frm;
+ break;
+ }
+ }
+ }
+
+ return uframe;
+}
+
+static struct uvcg_format *find_format_by_pix(struct uvc_device *uvc,
+ u32 pixelformat)
+{
+ struct uvcg_format_ptr *format;
+ struct uvcg_format *uformat = NULL;
+
+ list_for_each_entry(format, &uvc->header->formats, entry) {
+ struct uvc_format_desc *fmtdesc = to_uvc_format(format->fmt);
+
+ if (fmtdesc->fcc == pixelformat) {
+ uformat = format->fmt;
+ break;
+ }
+ }
+
+ return uformat;
+}
+
+static struct uvcg_frame *find_closest_frame_by_size(struct uvc_device *uvc,
+ struct uvcg_format *uformat,
+ u16 rw, u16 rh)
+{
+ struct uvc_video *video = &uvc->video;
+ struct uvcg_format_ptr *format;
+ struct uvcg_frame_ptr *frame;
+ struct uvcg_frame *uframe = NULL;
+ unsigned int d, maxd;
+
+ /* Find the closest image size. The distance between image sizes is
+ * the size in pixels of the non-overlapping regions between the
+ * requested size and the frame-specified size.
+ */
+ maxd = (unsigned int)-1;
+
+ list_for_each_entry(format, &uvc->header->formats, entry) {
+ if (format->fmt->type != uformat->type)
+ continue;
+
+ list_for_each_entry(frame, &format->fmt->frames, entry) {
+ u16 w, h;
+
+ w = frame->frm->frame.w_width;
+ h = frame->frm->frame.w_height;
+
+ d = min(w, rw) * min(h, rh);
+ d = w*h + rw*rh - 2*d;
+ if (d < maxd) {
+ maxd = d;
+ uframe = frame->frm;
+ }
+
+ if (maxd == 0)
+ break;
+ }
+ }
+
+ if (!uframe)
+ uvcg_dbg(&video->uvc->func, "Unsupported size %ux%u\n", rw, rh);
+
+ return uframe;
+}
/* --------------------------------------------------------------------------
* Requests handling
@@ -50,16 +199,6 @@ uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
* V4L2 ioctls
*/
-struct uvc_format {
- u8 bpp;
- u32 fcc;
-};
-
-static struct uvc_format uvc_formats[] = {
- { 16, V4L2_PIX_FMT_YUYV },
- { 0, V4L2_PIX_FMT_MJPEG },
-};
-
static int
uvc_v4l2_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
{
@@ -80,14 +219,16 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
struct video_device *vdev = video_devdata(file);
struct uvc_device *uvc = video_get_drvdata(vdev);
struct uvc_video *video = &uvc->video;
+ struct uvc_format_desc *fmtdesc;
- fmt->fmt.pix.pixelformat = video->fcc;
- fmt->fmt.pix.width = video->width;
- fmt->fmt.pix.height = video->height;
+ fmtdesc = to_uvc_format(video->cur_format);
+
+ fmt->fmt.pix.pixelformat = fmtdesc->fcc;
+ fmt->fmt.pix.width = video->cur_frame->frame.w_width;
+ fmt->fmt.pix.height = video->cur_frame->frame.w_height;
fmt->fmt.pix.field = V4L2_FIELD_NONE;
fmt->fmt.pix.bytesperline = video->bpp * video->width / 8;
fmt->fmt.pix.sizeimage = video->imagesize;
- fmt->fmt.pix.colorspace = V4L2_COLORSPACE_SRGB;
fmt->fmt.pix.priv = 0;
return 0;
@@ -99,41 +240,171 @@ uvc_v4l2_set_format(struct file *file, void *fh, struct v4l2_format *fmt)
struct video_device *vdev = video_devdata(file);
struct uvc_device *uvc = video_get_drvdata(vdev);
struct uvc_video *video = &uvc->video;
- struct uvc_format *format;
- unsigned int imagesize;
- unsigned int bpl;
- unsigned int i;
-
- for (i = 0; i < ARRAY_SIZE(uvc_formats); ++i) {
- format = &uvc_formats[i];
- if (format->fcc == fmt->fmt.pix.pixelformat)
- break;
- }
- if (i == ARRAY_SIZE(uvc_formats)) {
- uvcg_info(&uvc->func, "Unsupported format 0x%08x.\n",
- fmt->fmt.pix.pixelformat);
+ if (fmt->type != video->queue.queue.type)
return -EINVAL;
- }
- bpl = format->bpp * fmt->fmt.pix.width / 8;
- imagesize = bpl ? bpl * fmt->fmt.pix.height : fmt->fmt.pix.sizeimage;
+ return 0;
+};
+
+static int
+uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct uvc_device *uvc = video_get_drvdata(vdev);
+ struct uvc_video *video = &uvc->video;
+ struct uvcg_format *uformat;
+ struct uvcg_frame *uframe;
+ u8 *fcc;
+
+ if (fmt->type != video->queue.queue.type)
+ return -EINVAL;
+
+ fcc = (u8 *)&fmt->fmt.pix.pixelformat;
+ uvcg_dbg(&uvc->func, "Trying format 0x%08x (%c%c%c%c): %ux%u\n",
+ fmt->fmt.pix.pixelformat,
+ fcc[0], fcc[1], fcc[2], fcc[3],
+ fmt->fmt.pix.width, fmt->fmt.pix.height);
+
+ uformat = find_format_by_pix(uvc, fmt->fmt.pix.pixelformat);
+ if (!uformat)
+ return -EINVAL;
- video->fcc = format->fcc;
- video->bpp = format->bpp;
- video->width = fmt->fmt.pix.width;
- video->height = fmt->fmt.pix.height;
- video->imagesize = imagesize;
+ uframe = find_closest_frame_by_size(uvc, uformat,
+ fmt->fmt.pix.width, fmt->fmt.pix.height);
+ if (!uframe)
+ return -EINVAL;
+ fmt->fmt.pix.width = uframe->frame.w_width;
+ fmt->fmt.pix.height = uframe->frame.w_height;
fmt->fmt.pix.field = V4L2_FIELD_NONE;
fmt->fmt.pix.bytesperline = bpl;
fmt->fmt.pix.sizeimage = imagesize;
- fmt->fmt.pix.colorspace = V4L2_COLORSPACE_SRGB;
fmt->fmt.pix.priv = 0;
return 0;
}
+static int
+uvc_v4l2_enum_frameintervals(struct file *file, void *fh,
+ struct v4l2_frmivalenum *fival)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct uvc_device *uvc = video_get_drvdata(vdev);
+ struct uvc_video *video = &uvc->video;
+ struct uvcg_format *uformat = NULL;
+ struct uvcg_frame *uframe = NULL;
+ struct uvcg_frame_ptr *frame;
+
+ uformat = find_format_by_pix(uvc, fival->pixel_format);
+ if (!uformat)
+ return -EINVAL;
+
+ list_for_each_entry(frame, &uformat->frames, entry) {
+ if (frame->frm->frame.w_width == fival->width &&
+ frame->frm->frame.w_height == fival->height) {
+ uframe = frame->frm;
+ break;
+ }
+ }
+ if (!uframe)
+ return -EINVAL;
+
+ if (uvc->streamon) {
+ if (fival->index >= 1)
+ return -EINVAL;
+
+ fival->discrete.numerator =
+ uframe->dw_frame_interval[video->cur_ival - 1];
+ } else {
+ if (fival->index >= uframe->frame.b_frame_interval_type)
+ return -EINVAL;
+
+ fival->discrete.numerator =
+ uframe->dw_frame_interval[fival->index];
+ }
+
+ /* TODO: handle V4L2_FRMIVAL_TYPE_STEPWISE */
+ fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+ fival->discrete.denominator = 10000000;
+ v4l2_simplify_fraction(&fival->discrete.numerator,
+ &fival->discrete.denominator, 8, 333);
+
+ return 0;
+}
+
+static int
+uvc_v4l2_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct uvc_device *uvc = video_get_drvdata(vdev);
+ struct uvc_video *video = &uvc->video;
+ struct uvcg_format *uformat = NULL;
+ struct uvcg_frame *uframe = NULL;
+
+ if (uvc->streamon) {
+ if (fsize->index >= 1)
+ return -EINVAL;
+
+ uformat = video->cur_format;
+ uframe = video->cur_frame;
+ } else {
+ uformat = find_format_by_pix(uvc, fsize->pixel_format);
+ if (!uformat)
+ return -EINVAL;
+
+ if (fsize->index >= uformat->num_frames)
+ return -EINVAL;
+
+ uframe = find_frame_by_index(uvc, uformat, fsize->index + 1);
+ if (!uframe)
+ return -EINVAL;
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+ fsize->discrete.width = uframe->frame.w_width;
+ fsize->discrete.height = uframe->frame.w_height;
+
+ return 0;
+}
+
+static int
+uvc_v4l2_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct uvc_device *uvc = video_get_drvdata(vdev);
+ struct uvc_video *video = &uvc->video;
+ struct uvc_format_desc *fmtdesc;
+ struct uvcg_format *uformat;
+
+ if (uvc->streamon) {
+ if (f->index >= 1)
+ return -EINVAL;
+
+ uformat = video->cur_format;
+ } else {
+ if (f->index >= uvc->header->num_fmt)
+ return -EINVAL;
+
+ uformat = find_format_by_index(uvc, f->index + 1);
+ if (!uformat)
+ return -EINVAL;
+
+ }
+
+ if (uformat->type != UVCG_UNCOMPRESSED)
+ f->flags |= V4L2_FMT_FLAG_COMPRESSED;
+
+ fmtdesc = to_uvc_format(uformat);
+ f->pixelformat = fmtdesc->fcc;
+
+ strscpy(f->description, fmtdesc->name, sizeof(f->description));
+ f->description[strlen(fmtdesc->name) - 1] = 0;
+
+ return 0;
+}
+
static int
uvc_v4l2_reqbufs(struct file *file, void *fh, struct v4l2_requestbuffers *b)
{
@@ -298,8 +569,12 @@ uvc_v4l2_ioctl_default(struct file *file, void *fh, bool valid_prio,
const struct v4l2_ioctl_ops uvc_v4l2_ioctl_ops = {
.vidioc_querycap = uvc_v4l2_querycap,
+ .vidioc_try_fmt_vid_out = uvc_v4l2_try_format,
.vidioc_g_fmt_vid_out = uvc_v4l2_get_format,
.vidioc_s_fmt_vid_out = uvc_v4l2_set_format,
+ .vidioc_enum_frameintervals = uvc_v4l2_enum_frameintervals,
+ .vidioc_enum_framesizes = uvc_v4l2_enum_framesizes,
+ .vidioc_enum_fmt_vid_out = uvc_v4l2_enum_fmt,
.vidioc_reqbufs = uvc_v4l2_reqbufs,
.vidioc_querybuf = uvc_v4l2_querybuf,
.vidioc_qbuf = uvc_v4l2_qbuf,
diff --git a/drivers/usb/gadget/function/uvc_video.c b/drivers/usb/gadget/function/uvc_video.c
index c00ce0e91f5d5c..37867c93073418 100644
--- a/drivers/usb/gadget/function/uvc_video.c
+++ b/drivers/usb/gadget/function/uvc_video.c
@@ -19,6 +19,7 @@
#include "uvc.h"
#include "uvc_queue.h"
#include "uvc_video.h"
+#include "uvc_configfs.h"
/* --------------------------------------------------------------------------
* Video codecs
@@ -490,21 +491,71 @@ int uvcg_video_enable(struct uvc_video *video, int enable)
return ret;
}
+static int uvc_frame_default(struct uvcg_format *uformat)
+{
+ struct uvcg_uncompressed *u;
+ struct uvcg_mjpeg *m;
+
+ switch (uformat->type) {
+ case UVCG_UNCOMPRESSED:
+ u = to_uvcg_uncompressed(&uformat->group.cg_item);
+ if (u)
+ return u->desc.bDefaultFrameIndex;
+ break;
+ case UVCG_MJPEG:
+ m = to_uvcg_mjpeg(&uformat->group.cg_item);
+ if (m)
+ return m->desc.bDefaultFrameIndex;
+ break;
+ }
+
+ return 0;
+}
+
+static int uvc_default_frame_interval(struct uvc_video *video)
+{
+ int i;
+
+ for (i = 0; i < video->cur_frame->frame.b_frame_interval_type; i++) {
+ if (video->cur_frame->frame.dw_default_frame_interval ==
+ video->cur_frame->dw_frame_interval[i]) {
+ video->cur_ival = i + 1;
+ return i + 1;
+ }
+ }
+
+ /* fallback */
+ return 1;
+}
+
/*
* Initialize the UVC video stream.
*/
int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
{
+ int iframe;
+
INIT_LIST_HEAD(&video->req_free);
spin_lock_init(&video->req_lock);
INIT_WORK(&video->pump, uvcg_video_pump);
+ if (list_empty(&uvc->header->formats))
+ return -EINVAL;
+
video->uvc = uvc;
- video->fcc = V4L2_PIX_FMT_YUYV;
- video->bpp = 16;
- video->width = 320;
- video->height = 240;
- video->imagesize = 320 * 240 * 2;
+ video->cur_format = find_format_by_index(uvc, 1);
+ if (!video->cur_format)
+ return -EINVAL;
+
+ iframe = uvc_frame_default(video->cur_format);
+ if (!iframe)
+ return -EINVAL;
+
+ video->cur_frame = find_frame_by_index(uvc, video->cur_format, iframe);
+ if (!video->cur_frame)
+ return -EINVAL;
+
+ video->cur_ival = uvc_default_frame_interval(video);
/* Initialize the video buffers queue. */
uvcg_queue_init(&video->queue, uvc->v4l2_dev.dev->parent,
--
2.30.2
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
` (2 preceding siblings ...)
2022-09-07 14:02 ` [PATCH v8 3/4] usb: gadget: uvc: add VIDIOC function Michael Grzeschik
@ 2022-09-07 14:02 ` Michael Grzeschik
2022-09-07 15:07 ` Laurent Pinchart
2022-09-07 14:28 ` [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Greg KH
4 siblings, 1 reply; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:02 UTC (permalink / raw)
To: linux-usb
Cc: linux-media, balbi, laurent.pinchart, paul.elder, kernel, nicolas,
kieran.bingham
The Hostside format selection is currently only done in userspace, as
the events for SET_CUR and GET_CUR are always moved to the application
layer. Since the v4l2 device parses the configfs data, the format
negotiation can be done in the kernel. This patch adds the functions to
set the current configuration while continuing to forward all unknown
events to the userspace level.
Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
---
v1 -> v2:
- fixed the commit message
- changed pr_debug to pr_err in events_process_data
- aligned many indentations
- simplified uvc_events_process_data
- fixed uvc_fill_streaming_control calls in uvcg_video_init
- added setup_subscribed to decide if userspace takes over on EOPNOTSUPP
- added data_subscribed to decide if userspace takes over on EOPNOTSUPP
- removed duplicate send_response
- wrting fmt and frm in full
v2 -> v3:
- added find_format_index to set the right probe
v3 -> v4:
- add function find_ival_index and use for cur_ival
- fix swapped frame and format in uvc_events_process_data on uvc_fill_streaming_control
- set proper resp.length on ep0 complete
- dropped setting cur_probe on set_format since function was removed
- added locking around getting correspondent cur_{frame,format,ival}
v4 -> v5:
- fixed sparse errors reported by kernel test robot
v5 -> v6:
- fixed the handling in uvc_function_ep0_complete after events_process_data
v6 -> v7:
- set dwMaxPayloadTransferSize unconditionally from streaming_maxpacket
- fixed check for interface with masking for 0xff
v7 -> v8:
-
drivers/usb/gadget/function/f_uvc.c | 237 +++++++++++++++++++++++-
drivers/usb/gadget/function/uvc.h | 19 ++
drivers/usb/gadget/function/uvc_v4l2.c | 66 ++++++-
drivers/usb/gadget/function/uvc_video.c | 12 +-
4 files changed, 322 insertions(+), 12 deletions(-)
diff --git a/drivers/usb/gadget/function/f_uvc.c b/drivers/usb/gadget/function/f_uvc.c
index a8dcf5f01f16de..fc3fabc47018cd 100644
--- a/drivers/usb/gadget/function/f_uvc.c
+++ b/drivers/usb/gadget/function/f_uvc.c
@@ -16,7 +16,6 @@
#include <linux/string.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
-#include <linux/usb/g_uvc.h>
#include <linux/usb/video.h>
#include <linux/vmalloc.h>
#include <linux/wait.h>
@@ -204,21 +203,228 @@ static const struct usb_descriptor_header * const uvc_ss_streaming[] = {
* Control requests
*/
+void uvc_fill_streaming_control(struct uvc_device *uvc,
+ struct uvc_streaming_control *ctrl,
+ int iframe, int iformat, unsigned int ival)
+{
+ struct f_uvc_opts *opts;
+ struct uvcg_format *uformat;
+ struct uvcg_frame *uframe;
+
+ /* Restrict the iformat, iframe and ival to valid values. Negative
+ * values for ifrmat and iframe will result in the maximum valid value
+ * being selected
+ */
+ iformat = clamp((unsigned int)iformat, 1U,
+ (unsigned int)uvc->header->num_fmt);
+ uformat = find_format_by_index(uvc, iformat);
+ if (!uformat)
+ return;
+
+ iframe = clamp((unsigned int)iframe, 1U,
+ (unsigned int)uformat->num_frames);
+ uframe = find_frame_by_index(uvc, uformat, iframe);
+ if (!uframe)
+ return;
+
+ ival = clamp((unsigned int)ival, 1U,
+ (unsigned int)uframe->frame.b_frame_interval_type);
+ if (!uframe->dw_frame_interval[ival - 1])
+ return;
+
+ opts = fi_to_f_uvc_opts(uvc->func.fi);
+
+ memset(ctrl, 0, sizeof(*ctrl));
+
+ ctrl->bmHint = 1;
+ ctrl->bFormatIndex = iformat;
+ ctrl->bFrameIndex = iframe;
+ ctrl->dwFrameInterval = uframe->dw_frame_interval[ival - 1];
+ ctrl->dwMaxVideoFrameSize =
+ uframe->frame.dw_max_video_frame_buffer_size;
+
+ ctrl->dwMaxPayloadTransferSize = opts->streaming_maxpacket;
+ ctrl->bmFramingInfo = 3;
+ ctrl->bPreferedVersion = 1;
+ ctrl->bMaxVersion = 1;
+}
+
+static int uvc_events_process_data(struct uvc_device *uvc,
+ struct usb_request *req)
+{
+ struct uvc_video *video = &uvc->video;
+ struct uvc_streaming_control *target;
+ struct uvc_streaming_control *ctrl;
+ struct uvcg_frame *uframe;
+ struct uvcg_format *uformat;
+
+ switch (video->control) {
+ case UVC_VS_PROBE_CONTROL:
+ pr_debug("setting probe control, length = %d\n", req->actual);
+ target = &video->probe;
+ break;
+
+ case UVC_VS_COMMIT_CONTROL:
+ pr_debug("setting commit control, length = %d\n", req->actual);
+ target = &video->commit;
+ break;
+
+ default:
+ pr_err("setting unknown control, length = %d\n", req->actual);
+ return -EOPNOTSUPP;
+ }
+
+ ctrl = (struct uvc_streaming_control *)req->buf;
+
+ uvc_fill_streaming_control(uvc, target, ctrl->bFrameIndex,
+ ctrl->bFormatIndex, ctrl->dwFrameInterval);
+
+ if (video->control == UVC_VS_COMMIT_CONTROL) {
+ uformat = find_format_by_index(uvc, target->bFormatIndex);
+ if (!uformat)
+ return -EINVAL;
+
+ uframe = find_frame_by_index(uvc, uformat, ctrl->bFrameIndex);
+ if (!uframe)
+ return -EINVAL;
+
+ spin_lock(&video->frame_lock);
+
+ video->cur_frame = uframe;
+ video->cur_format = uformat;
+ video->cur_ival = find_ival_index(uframe, ctrl->dwFrameInterval);
+
+ spin_unlock(&video->frame_lock);
+ }
+
+ return 0;
+}
+
+static void
+uvc_events_process_streaming(struct uvc_device *uvc, uint8_t req, uint8_t cs,
+ struct uvc_request_data *resp)
+{
+ struct uvc_streaming_control *ctrl;
+
+ pr_debug("streaming request (req %02x cs %02x)\n", req, cs);
+
+ if (cs != UVC_VS_PROBE_CONTROL && cs != UVC_VS_COMMIT_CONTROL)
+ return;
+
+ ctrl = (struct uvc_streaming_control *)&resp->data;
+ resp->length = sizeof(*ctrl);
+
+ switch (req) {
+ case UVC_SET_CUR:
+ uvc->video.control = cs;
+ resp->length = 34;
+ break;
+
+ case UVC_GET_CUR:
+ if (cs == UVC_VS_PROBE_CONTROL)
+ memcpy(ctrl, &uvc->video.probe, sizeof(*ctrl));
+ else
+ memcpy(ctrl, &uvc->video.commit, sizeof(*ctrl));
+ break;
+
+ case UVC_GET_MIN:
+ case UVC_GET_MAX:
+ case UVC_GET_DEF:
+ if (req == UVC_GET_MAX)
+ uvc_fill_streaming_control(uvc, ctrl, -1, -1, UINT_MAX);
+ else
+ uvc_fill_streaming_control(uvc, ctrl, 1, 1, 0);
+ break;
+
+ case UVC_GET_RES:
+ memset(ctrl, 0, sizeof(*ctrl));
+ break;
+
+ case UVC_GET_LEN:
+ resp->data[0] = 0x00;
+ resp->data[1] = 0x22;
+ resp->length = 2;
+ break;
+
+ case UVC_GET_INFO:
+ resp->data[0] = 0x03;
+ resp->length = 1;
+ break;
+ }
+}
+
+static int
+uvc_events_process_class(struct uvc_device *uvc,
+ const struct usb_ctrlrequest *ctrl,
+ struct uvc_request_data *resp)
+{
+ unsigned int interface = le16_to_cpu(ctrl->wIndex) & 0xff;
+
+ if ((ctrl->bRequestType & USB_RECIP_MASK) != USB_RECIP_INTERFACE)
+ return -EINVAL;
+
+ if (interface == uvc->control_intf)
+ return -EOPNOTSUPP;
+ else if (interface == uvc->streaming_intf)
+ uvc_events_process_streaming(uvc, ctrl->bRequest,
+ le16_to_cpu(ctrl->wValue) >> 8,
+ resp);
+
+ return 0;
+}
+
+static int
+uvc_events_process_setup(struct uvc_device *uvc,
+ const struct usb_ctrlrequest *ctrl,
+ struct uvc_request_data *resp)
+{
+ uvc->video.control = 0;
+
+ pr_debug("bRequestType %02x bRequest %02x wValue %04x wIndex %04x wLength %04x\n",
+ ctrl->bRequestType, ctrl->bRequest, ctrl->wValue,
+ ctrl->wIndex, ctrl->wLength);
+
+ switch (ctrl->bRequestType & USB_TYPE_MASK) {
+ case USB_TYPE_STANDARD:
+ return -EOPNOTSUPP;
+
+ case USB_TYPE_CLASS:
+ return uvc_events_process_class(uvc, ctrl, resp);
+
+ default:
+ break;
+ }
+
+ return 0;
+}
+
static void
uvc_function_ep0_complete(struct usb_ep *ep, struct usb_request *req)
{
struct uvc_device *uvc = req->context;
struct v4l2_event v4l2_event;
struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
+ int ret;
if (uvc->event_setup_out) {
uvc->event_setup_out = 0;
- memset(&v4l2_event, 0, sizeof(v4l2_event));
- v4l2_event.type = UVC_EVENT_DATA;
- uvc_event->data.length = req->actual;
- memcpy(&uvc_event->data.data, req->buf, req->actual);
- v4l2_event_queue(&uvc->vdev, &v4l2_event);
+ ret = uvc_events_process_data(uvc, req);
+ /* If we have a real error on process */
+ if (ret == -EINVAL) {
+ struct uvc_request_data resp;
+
+ memset(&resp, 0, sizeof(resp));
+ resp.length = -EL2HLT;
+
+ uvc_send_response(uvc, &resp);
+ } else if (ret == -EOPNOTSUPP && uvc->data_subscribed) {
+ memset(&v4l2_event, 0, sizeof(v4l2_event));
+ v4l2_event.type = UVC_EVENT_DATA;
+ uvc_event->data.length = req->actual;
+ memcpy(&uvc_event->data.data, req->buf, req->actual);
+ v4l2_event_queue(&uvc->vdev, &v4l2_event);
+ }
}
}
@@ -228,6 +434,8 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
struct uvc_device *uvc = to_uvc(f);
struct v4l2_event v4l2_event;
struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
+ struct uvc_request_data resp;
+ int ret = 0;
if ((ctrl->bRequestType & USB_TYPE_MASK) != USB_TYPE_CLASS) {
uvcg_info(f, "invalid request type\n");
@@ -245,6 +453,23 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
uvc->event_setup_out = !(ctrl->bRequestType & USB_DIR_IN);
uvc->event_length = le16_to_cpu(ctrl->wLength);
+ memset(&resp, 0, sizeof(resp));
+ resp.length = -EL2HLT;
+
+ ret = uvc_events_process_setup(uvc, ctrl, &resp);
+ /* If we have no error on process */
+ if (!ret)
+ return uvc_send_response(uvc, &resp);
+
+ /* If we have a real error on process */
+ if (ret != -EOPNOTSUPP)
+ return ret;
+
+ /* If we have -EOPNOTSUPP */
+ if (!uvc->setup_subscribed)
+ return uvc_send_response(uvc, &resp);
+
+ /* If we have setup subscribed */
memset(&v4l2_event, 0, sizeof(v4l2_event));
v4l2_event.type = UVC_EVENT_SETUP;
memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req));
diff --git a/drivers/usb/gadget/function/uvc.h b/drivers/usb/gadget/function/uvc.h
index 8a71d0c4abbcd5..cdfd46f3a2861c 100644
--- a/drivers/usb/gadget/function/uvc.h
+++ b/drivers/usb/gadget/function/uvc.h
@@ -13,6 +13,8 @@
#include <linux/mutex.h>
#include <linux/spinlock.h>
#include <linux/usb/composite.h>
+#include <linux/usb/g_uvc.h>
+#include <linux/usb/video.h>
#include <linux/videodev2.h>
#include <linux/wait.h>
@@ -95,6 +97,12 @@ struct uvc_video {
unsigned int cur_ival;
struct mutex mutex; /* protects frame parameters */
+ spinlock_t frame_lock;
+
+ struct uvc_streaming_control probe;
+ struct uvc_streaming_control commit;
+
+ int control;
unsigned int uvc_num_requests;
@@ -131,6 +139,8 @@ struct uvc_device {
struct uvc_video video;
bool func_connected;
wait_queue_head_t func_connected_queue;
+ bool setup_subscribed;
+ bool data_subscribed;
struct uvcg_streaming_header *header;
@@ -187,5 +197,14 @@ extern struct uvcg_format *find_format_by_index(struct uvc_device *uvc,
extern struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
struct uvcg_format *uformat,
int index);
+extern int find_format_index(struct uvc_device *uvc,
+ struct uvcg_format *uformat);
+extern int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval);
+extern void uvc_fill_streaming_control(struct uvc_device *uvc,
+ struct uvc_streaming_control *ctrl,
+ int iframe, int iformat,
+ unsigned int ival);
+extern int uvc_send_response(struct uvc_device *uvc,
+ struct uvc_request_data *data);
#endif /* _UVC_GADGET_H_ */
diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c
index 3311e98a040537..e4c4bebd2ee860 100644
--- a/drivers/usb/gadget/function/uvc_v4l2.c
+++ b/drivers/usb/gadget/function/uvc_v4l2.c
@@ -90,6 +90,33 @@ struct uvcg_format *find_format_by_index(struct uvc_device *uvc, int index)
return uformat;
}
+int find_format_index(struct uvc_device *uvc, struct uvcg_format *uformat)
+{
+ struct uvcg_format_ptr *format;
+ int i = 1;
+
+ list_for_each_entry(format, &uvc->header->formats, entry) {
+ if (uformat == format->fmt)
+ return i;
+ i++;
+ }
+
+ return 0;
+}
+
+int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval)
+{
+ int i;
+
+ for (i = 0; i < uframe->frame.b_frame_interval_type; i++) {
+ if (dwFrameInterval == uframe->dw_frame_interval[i])
+ return i + 1;
+ }
+
+ /* fallback */
+ return 1;
+}
+
struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
struct uvcg_format *uformat,
int index)
@@ -178,8 +205,7 @@ static struct uvcg_frame *find_closest_frame_by_size(struct uvc_device *uvc,
* Requests handling
*/
-static int
-uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
+int uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
{
struct usb_composite_dev *cdev = uvc->func.config->cdev;
struct usb_request *req = uvc->control_req;
@@ -221,6 +247,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
struct uvc_video *video = &uvc->video;
struct uvc_format_desc *fmtdesc;
+ spin_lock(&video->frame_lock);
+
fmtdesc = to_uvc_format(video->cur_format);
fmt->fmt.pix.pixelformat = fmtdesc->fcc;
@@ -231,6 +259,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
fmt->fmt.pix.sizeimage = video->imagesize;
fmt->fmt.pix.priv = 0;
+ spin_unlock(&video->frame_lock);
+
return 0;
}
@@ -255,6 +285,7 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
struct uvc_video *video = &uvc->video;
struct uvcg_format *uformat;
struct uvcg_frame *uframe;
+ int iformat;
u8 *fcc;
if (fmt->type != video->queue.queue.type)
@@ -270,6 +301,10 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
if (!uformat)
return -EINVAL;
+ iformat = find_format_index(uvc, uformat);
+ if (!iformat)
+ return -EINVAL;
+
uframe = find_closest_frame_by_size(uvc, uformat,
fmt->fmt.pix.width, fmt->fmt.pix.height);
if (!uframe)
@@ -314,8 +349,12 @@ uvc_v4l2_enum_frameintervals(struct file *file, void *fh,
if (fival->index >= 1)
return -EINVAL;
+ spin_lock(&video->frame_lock);
+
fival->discrete.numerator =
uframe->dw_frame_interval[video->cur_ival - 1];
+
+ spin_unlock(&video->frame_lock);
} else {
if (fival->index >= uframe->frame.b_frame_interval_type)
return -EINVAL;
@@ -347,8 +386,12 @@ uvc_v4l2_enum_framesizes(struct file *file, void *fh,
if (fsize->index >= 1)
return -EINVAL;
+ spin_lock(&video->frame_lock);
+
uformat = video->cur_format;
uframe = video->cur_frame;
+
+ spin_unlock(&video->frame_lock);
} else {
uformat = find_format_by_pix(uvc, fsize->pixel_format);
if (!uformat)
@@ -382,7 +425,11 @@ uvc_v4l2_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
if (f->index >= 1)
return -EINVAL;
+ spin_lock(&video->frame_lock);
+
uformat = video->cur_format;
+
+ spin_unlock(&video->frame_lock);
} else {
if (f->index >= uvc->header->num_fmt)
return -EINVAL;
@@ -506,14 +553,20 @@ uvc_v4l2_subscribe_event(struct v4l2_fh *fh,
if (sub->type < UVC_EVENT_FIRST || sub->type > UVC_EVENT_LAST)
return -EINVAL;
- if (sub->type == UVC_EVENT_SETUP && uvc->func_connected)
+ if (sub->type == UVC_EVENT_STREAMON && uvc->func_connected)
return -EBUSY;
ret = v4l2_event_subscribe(fh, sub, 2, NULL);
if (ret < 0)
return ret;
- if (sub->type == UVC_EVENT_SETUP) {
+ if (sub->type == UVC_EVENT_SETUP)
+ uvc->setup_subscribed = true;
+
+ if (sub->type == UVC_EVENT_DATA)
+ uvc->data_subscribed = true;
+
+ if (sub->type == UVC_EVENT_STREAMON) {
uvc->func_connected = true;
handle->is_uvc_app_handle = true;
uvc_function_connect(uvc);
@@ -543,7 +596,10 @@ uvc_v4l2_unsubscribe_event(struct v4l2_fh *fh,
if (ret < 0)
return ret;
- if (sub->type == UVC_EVENT_SETUP && handle->is_uvc_app_handle) {
+ if (sub->type == UVC_EVENT_SETUP)
+ uvc->setup_subscribed = false;
+
+ if (sub->type == UVC_EVENT_STREAMON && handle->is_uvc_app_handle) {
uvc_v4l2_disable(uvc);
handle->is_uvc_app_handle = false;
}
diff --git a/drivers/usb/gadget/function/uvc_video.c b/drivers/usb/gadget/function/uvc_video.c
index 37867c93073418..c20d832c4a0b8a 100644
--- a/drivers/usb/gadget/function/uvc_video.c
+++ b/drivers/usb/gadget/function/uvc_video.c
@@ -533,10 +533,11 @@ static int uvc_default_frame_interval(struct uvc_video *video)
*/
int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
{
- int iframe;
+ int iframe, iformat;
INIT_LIST_HEAD(&video->req_free);
spin_lock_init(&video->req_lock);
+ spin_lock_init(&video->frame_lock);
INIT_WORK(&video->pump, uvcg_video_pump);
if (list_empty(&uvc->header->formats))
@@ -547,6 +548,10 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
if (!video->cur_format)
return -EINVAL;
+ iformat = find_format_index(uvc, video->cur_format);
+ if (!iformat)
+ return -EINVAL;
+
iframe = uvc_frame_default(video->cur_format);
if (!iframe)
return -EINVAL;
@@ -557,6 +562,11 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
video->cur_ival = uvc_default_frame_interval(video);
+ uvc_fill_streaming_control(uvc, &video->probe, iframe, iformat,
+ video->cur_ival);
+ uvc_fill_streaming_control(uvc, &video->commit, iframe, iformat,
+ video->cur_ival);
+
/* Initialize the video buffers queue. */
uvcg_queue_init(&video->queue, uvc->v4l2_dev.dev->parent,
V4L2_BUF_TYPE_VIDEO_OUTPUT, &video->mutex);
--
2.30.2
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
` (3 preceding siblings ...)
2022-09-07 14:02 ` [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code Michael Grzeschik
@ 2022-09-07 14:28 ` Greg KH
2022-09-07 14:52 ` Michael Grzeschik
4 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2022-09-07 14:28 UTC (permalink / raw)
To: Michael Grzeschik
Cc: linux-usb, linux-media, balbi, laurent.pinchart, paul.elder,
kernel, nicolas, kieran.bingham
On Wed, Sep 07, 2022 at 04:02:50PM +0200, Michael Grzeschik wrote:
> This series improves the uvc video gadget by parsing the configfs
> entries. With the configfs data, the driver now is able to negotiate the
> format with the usb host in the kernel and also exports the supported
> frames/formats/intervals via the v4l2 VIDIOC interface.
>
> The uvc userspace stack is also under development. One example is an generic
> v4l2uvcsink gstreamer elemnt, which is currently under discussion. [1]
>
> [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1304
>
> With the libusbgx library [1] used by the gadget-tool [2] it is now also
> possible to fully describe the configfs layout of the uvc gadget with scheme
> files.
>
> [2] https://github.com/linux-usb-gadgets/libusbgx/pull/61/commits/53231c76f9d512f59fdc23b65cd5c46b7fb09eb4
>
> [3] https://github.com/linux-usb-gadgets/gt/tree/master/examples/systemd
>
> The bigger picture of these patches is to provide a more versatile interface to
> the uvc gadget. The goal is to simply start a uvc-gadget with the following
> commands:
>
> $ gt load uvc.scheme
> $ gst-launch v4l2src ! v4l2uvcsink
>
> --
>
> v1: https://lore.kernel.org/linux-usb/20210530222239.8793-1-m.grzeschik@pengutronix.de/
> v2: https://lore.kernel.org/linux-usb/20211117004432.3763306-1-m.grzeschik@pengutronix.de/
> v3: https://lore.kernel.org/linux-usb/20211117122435.2409362-1-m.grzeschik@pengutronix.de/
> v4: https://lore.kernel.org/linux-usb/20211205225803.268492-1-m.grzeschik@pengutronix.de/
> v5: https://lore.kernel.org/linux-usb/20211209084322.2662616-1-m.grzeschik@pengutronix.de/
> v6: https://lore.kernel.org/linux-usb/20220105115527.3592860-1-m.grzeschik@pengutronix.de/
> v7: https://lore.kernel.org/linux-usb/20220608105748.139922-1-m.grzeschik@pengutronix.de/
Please say what changed somewhere :(
Anyway, this patch series does not even build properly:
drivers/usb/gadget/function/uvc_v4l2.c: In function ‘uvc_v4l2_get_format’:
drivers/usb/gadget/function/uvc_v4l2.c:258:42: error: ‘struct uvc_video’ has no member named ‘bpp’
258 | fmt->fmt.pix.bytesperline = video->bpp * video->width / 8;
| ^~
drivers/usb/gadget/function/uvc_v4l2.c:258:55: error: ‘struct uvc_video’ has no member named ‘width’
258 | fmt->fmt.pix.bytesperline = video->bpp * video->width / 8;
| ^~
drivers/usb/gadget/function/uvc_v4l2.c:259:39: error: ‘struct uvc_video’ has no member named ‘imagesize’
259 | fmt->fmt.pix.sizeimage = video->imagesize;
| ^~
drivers/usb/gadget/function/uvc_v4l2.c: In function ‘uvc_v4l2_try_format’:
drivers/usb/gadget/function/uvc_v4l2.c:316:37: error: ‘bpl’ undeclared (first use in this function)
316 | fmt->fmt.pix.bytesperline = bpl;
| ^~~
drivers/usb/gadget/function/uvc_v4l2.c:316:37: note: each undeclared identifier is reported only once for each function it appears in
drivers/usb/gadget/function/uvc_v4l2.c:317:34: error: ‘imagesize’ undeclared (first use in this function); did you mean ‘page_size’?
317 | fmt->fmt.pix.sizeimage = imagesize;
| ^~~~~~~~~
| page_size
What did you test this with?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS
2022-09-07 14:28 ` [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Greg KH
@ 2022-09-07 14:52 ` Michael Grzeschik
2022-09-07 15:01 ` Greg KH
0 siblings, 1 reply; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 14:52 UTC (permalink / raw)
To: Greg KH
Cc: balbi, linux-usb, paul.elder, kieran.bingham, nicolas,
laurent.pinchart, kernel, linux-media
[-- Attachment #1: Type: text/plain, Size: 4105 bytes --]
On Wed, Sep 07, 2022 at 04:28:34PM +0200, Greg KH wrote:
>On Wed, Sep 07, 2022 at 04:02:50PM +0200, Michael Grzeschik wrote:
>> This series improves the uvc video gadget by parsing the configfs
>> entries. With the configfs data, the driver now is able to negotiate the
>> format with the usb host in the kernel and also exports the supported
>> frames/formats/intervals via the v4l2 VIDIOC interface.
>>
>> The uvc userspace stack is also under development. One example is an generic
>> v4l2uvcsink gstreamer elemnt, which is currently under discussion. [1]
>>
>> [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1304
>>
>> With the libusbgx library [1] used by the gadget-tool [2] it is now also
>> possible to fully describe the configfs layout of the uvc gadget with scheme
>> files.
>>
>> [2] https://github.com/linux-usb-gadgets/libusbgx/pull/61/commits/53231c76f9d512f59fdc23b65cd5c46b7fb09eb4
>>
>> [3] https://github.com/linux-usb-gadgets/gt/tree/master/examples/systemd
>>
>> The bigger picture of these patches is to provide a more versatile interface to
>> the uvc gadget. The goal is to simply start a uvc-gadget with the following
>> commands:
>>
>> $ gt load uvc.scheme
>> $ gst-launch v4l2src ! v4l2uvcsink
>>
>> --
>>
>> v1: https://lore.kernel.org/linux-usb/20210530222239.8793-1-m.grzeschik@pengutronix.de/
>> v2: https://lore.kernel.org/linux-usb/20211117004432.3763306-1-m.grzeschik@pengutronix.de/
>> v3: https://lore.kernel.org/linux-usb/20211117122435.2409362-1-m.grzeschik@pengutronix.de/
>> v4: https://lore.kernel.org/linux-usb/20211205225803.268492-1-m.grzeschik@pengutronix.de/
>> v5: https://lore.kernel.org/linux-usb/20211209084322.2662616-1-m.grzeschik@pengutronix.de/
>> v6: https://lore.kernel.org/linux-usb/20220105115527.3592860-1-m.grzeschik@pengutronix.de/
>> v7: https://lore.kernel.org/linux-usb/20220608105748.139922-1-m.grzeschik@pengutronix.de/
>
>Please say what changed somewhere :(
I addressed each patch individually.
>Anyway, this patch series does not even build properly:
>
>drivers/usb/gadget/function/uvc_v4l2.c: In function ‘uvc_v4l2_get_format’:
>drivers/usb/gadget/function/uvc_v4l2.c:258:42: error: ‘struct uvc_video’ has no member named ‘bpp’
> 258 | fmt->fmt.pix.bytesperline = video->bpp * video->width / 8;
> | ^~
>drivers/usb/gadget/function/uvc_v4l2.c:258:55: error: ‘struct uvc_video’ has no member named ‘width’
> 258 | fmt->fmt.pix.bytesperline = video->bpp * video->width / 8;
> | ^~
>drivers/usb/gadget/function/uvc_v4l2.c:259:39: error: ‘struct uvc_video’ has no member named ‘imagesize’
> 259 | fmt->fmt.pix.sizeimage = video->imagesize;
> | ^~
>drivers/usb/gadget/function/uvc_v4l2.c: In function ‘uvc_v4l2_try_format’:
>drivers/usb/gadget/function/uvc_v4l2.c:316:37: error: ‘bpl’ undeclared (first use in this function)
> 316 | fmt->fmt.pix.bytesperline = bpl;
> | ^~~
>drivers/usb/gadget/function/uvc_v4l2.c:316:37: note: each undeclared identifier is reported only once for each function it appears in
>drivers/usb/gadget/function/uvc_v4l2.c:317:34: error: ‘imagesize’ undeclared (first use in this function); did you mean ‘page_size’?
> 317 | fmt->fmt.pix.sizeimage = imagesize;
> | ^~~~~~~~~
> | page_size
>
>
>What did you test this with?
I rebased and tested another series than I send.
Sorry for that. I will RESEND the v8 with the proper content.
Thanks,
Michael
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS
2022-09-07 14:52 ` Michael Grzeschik
@ 2022-09-07 15:01 ` Greg KH
0 siblings, 0 replies; 12+ messages in thread
From: Greg KH @ 2022-09-07 15:01 UTC (permalink / raw)
To: Michael Grzeschik
Cc: balbi, linux-usb, paul.elder, kieran.bingham, nicolas,
laurent.pinchart, kernel, linux-media
On Wed, Sep 07, 2022 at 04:52:04PM +0200, Michael Grzeschik wrote:
> On Wed, Sep 07, 2022 at 04:28:34PM +0200, Greg KH wrote:
> > On Wed, Sep 07, 2022 at 04:02:50PM +0200, Michael Grzeschik wrote:
> > > This series improves the uvc video gadget by parsing the configfs
> > > entries. With the configfs data, the driver now is able to negotiate the
> > > format with the usb host in the kernel and also exports the supported
> > > frames/formats/intervals via the v4l2 VIDIOC interface.
> > >
> > > The uvc userspace stack is also under development. One example is an generic
> > > v4l2uvcsink gstreamer elemnt, which is currently under discussion. [1]
> > >
> > > [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1304
> > >
> > > With the libusbgx library [1] used by the gadget-tool [2] it is now also
> > > possible to fully describe the configfs layout of the uvc gadget with scheme
> > > files.
> > >
> > > [2] https://github.com/linux-usb-gadgets/libusbgx/pull/61/commits/53231c76f9d512f59fdc23b65cd5c46b7fb09eb4
> > >
> > > [3] https://github.com/linux-usb-gadgets/gt/tree/master/examples/systemd
> > >
> > > The bigger picture of these patches is to provide a more versatile interface to
> > > the uvc gadget. The goal is to simply start a uvc-gadget with the following
> > > commands:
> > >
> > > $ gt load uvc.scheme
> > > $ gst-launch v4l2src ! v4l2uvcsink
> > >
> > > --
> > >
> > > v1: https://lore.kernel.org/linux-usb/20210530222239.8793-1-m.grzeschik@pengutronix.de/
> > > v2: https://lore.kernel.org/linux-usb/20211117004432.3763306-1-m.grzeschik@pengutronix.de/
> > > v3: https://lore.kernel.org/linux-usb/20211117122435.2409362-1-m.grzeschik@pengutronix.de/
> > > v4: https://lore.kernel.org/linux-usb/20211205225803.268492-1-m.grzeschik@pengutronix.de/
> > > v5: https://lore.kernel.org/linux-usb/20211209084322.2662616-1-m.grzeschik@pengutronix.de/
> > > v6: https://lore.kernel.org/linux-usb/20220105115527.3592860-1-m.grzeschik@pengutronix.de/
> > > v7: https://lore.kernel.org/linux-usb/20220608105748.139922-1-m.grzeschik@pengutronix.de/
> >
> > Please say what changed somewhere :(
>
> I addressed each patch individually.
Ah, yes, my mistake, I see that in the individual patches, sorry for the
noise.
greg k-h
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code
2022-09-07 14:02 ` [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code Michael Grzeschik
@ 2022-09-07 15:07 ` Laurent Pinchart
2022-09-07 15:15 ` Michael Grzeschik
0 siblings, 1 reply; 12+ messages in thread
From: Laurent Pinchart @ 2022-09-07 15:07 UTC (permalink / raw)
To: Michael Grzeschik
Cc: linux-usb, linux-media, balbi, paul.elder, kernel, nicolas,
kieran.bingham
Hi Michael,
Thank you for the patch.
On Wed, Sep 07, 2022 at 04:02:54PM +0200, Michael Grzeschik wrote:
> The Hostside format selection is currently only done in userspace, as
> the events for SET_CUR and GET_CUR are always moved to the application
> layer. Since the v4l2 device parses the configfs data, the format
> negotiation can be done in the kernel. This patch adds the functions to
> set the current configuration while continuing to forward all unknown
> events to the userspace level.
Why do you think this is better done in kernel space, given that
userspace has to process the event anyway ? It's more complexity in the
kernel side, for little added value as far as I can see. It will also
make it more difficult to handle different UVC versions (in particular
UVC 1.5). I'd rather not go in this direction if there's no clear
benefit.
> Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
>
> ---
> v1 -> v2:
> - fixed the commit message
> - changed pr_debug to pr_err in events_process_data
> - aligned many indentations
> - simplified uvc_events_process_data
> - fixed uvc_fill_streaming_control calls in uvcg_video_init
> - added setup_subscribed to decide if userspace takes over on EOPNOTSUPP
> - added data_subscribed to decide if userspace takes over on EOPNOTSUPP
> - removed duplicate send_response
> - wrting fmt and frm in full
> v2 -> v3:
> - added find_format_index to set the right probe
> v3 -> v4:
> - add function find_ival_index and use for cur_ival
> - fix swapped frame and format in uvc_events_process_data on uvc_fill_streaming_control
> - set proper resp.length on ep0 complete
> - dropped setting cur_probe on set_format since function was removed
> - added locking around getting correspondent cur_{frame,format,ival}
> v4 -> v5:
> - fixed sparse errors reported by kernel test robot
> v5 -> v6:
> - fixed the handling in uvc_function_ep0_complete after events_process_data
> v6 -> v7:
> - set dwMaxPayloadTransferSize unconditionally from streaming_maxpacket
> - fixed check for interface with masking for 0xff
> v7 -> v8:
> -
>
> drivers/usb/gadget/function/f_uvc.c | 237 +++++++++++++++++++++++-
> drivers/usb/gadget/function/uvc.h | 19 ++
> drivers/usb/gadget/function/uvc_v4l2.c | 66 ++++++-
> drivers/usb/gadget/function/uvc_video.c | 12 +-
> 4 files changed, 322 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/usb/gadget/function/f_uvc.c b/drivers/usb/gadget/function/f_uvc.c
> index a8dcf5f01f16de..fc3fabc47018cd 100644
> --- a/drivers/usb/gadget/function/f_uvc.c
> +++ b/drivers/usb/gadget/function/f_uvc.c
> @@ -16,7 +16,6 @@
> #include <linux/string.h>
> #include <linux/usb/ch9.h>
> #include <linux/usb/gadget.h>
> -#include <linux/usb/g_uvc.h>
> #include <linux/usb/video.h>
> #include <linux/vmalloc.h>
> #include <linux/wait.h>
> @@ -204,21 +203,228 @@ static const struct usb_descriptor_header * const uvc_ss_streaming[] = {
> * Control requests
> */
>
> +void uvc_fill_streaming_control(struct uvc_device *uvc,
> + struct uvc_streaming_control *ctrl,
> + int iframe, int iformat, unsigned int ival)
> +{
> + struct f_uvc_opts *opts;
> + struct uvcg_format *uformat;
> + struct uvcg_frame *uframe;
> +
> + /* Restrict the iformat, iframe and ival to valid values. Negative
> + * values for ifrmat and iframe will result in the maximum valid value
> + * being selected
> + */
> + iformat = clamp((unsigned int)iformat, 1U,
> + (unsigned int)uvc->header->num_fmt);
> + uformat = find_format_by_index(uvc, iformat);
> + if (!uformat)
> + return;
> +
> + iframe = clamp((unsigned int)iframe, 1U,
> + (unsigned int)uformat->num_frames);
> + uframe = find_frame_by_index(uvc, uformat, iframe);
> + if (!uframe)
> + return;
> +
> + ival = clamp((unsigned int)ival, 1U,
> + (unsigned int)uframe->frame.b_frame_interval_type);
> + if (!uframe->dw_frame_interval[ival - 1])
> + return;
> +
> + opts = fi_to_f_uvc_opts(uvc->func.fi);
> +
> + memset(ctrl, 0, sizeof(*ctrl));
> +
> + ctrl->bmHint = 1;
> + ctrl->bFormatIndex = iformat;
> + ctrl->bFrameIndex = iframe;
> + ctrl->dwFrameInterval = uframe->dw_frame_interval[ival - 1];
> + ctrl->dwMaxVideoFrameSize =
> + uframe->frame.dw_max_video_frame_buffer_size;
> +
> + ctrl->dwMaxPayloadTransferSize = opts->streaming_maxpacket;
> + ctrl->bmFramingInfo = 3;
> + ctrl->bPreferedVersion = 1;
> + ctrl->bMaxVersion = 1;
> +}
> +
> +static int uvc_events_process_data(struct uvc_device *uvc,
> + struct usb_request *req)
> +{
> + struct uvc_video *video = &uvc->video;
> + struct uvc_streaming_control *target;
> + struct uvc_streaming_control *ctrl;
> + struct uvcg_frame *uframe;
> + struct uvcg_format *uformat;
> +
> + switch (video->control) {
> + case UVC_VS_PROBE_CONTROL:
> + pr_debug("setting probe control, length = %d\n", req->actual);
> + target = &video->probe;
> + break;
> +
> + case UVC_VS_COMMIT_CONTROL:
> + pr_debug("setting commit control, length = %d\n", req->actual);
> + target = &video->commit;
> + break;
> +
> + default:
> + pr_err("setting unknown control, length = %d\n", req->actual);
> + return -EOPNOTSUPP;
> + }
> +
> + ctrl = (struct uvc_streaming_control *)req->buf;
> +
> + uvc_fill_streaming_control(uvc, target, ctrl->bFrameIndex,
> + ctrl->bFormatIndex, ctrl->dwFrameInterval);
> +
> + if (video->control == UVC_VS_COMMIT_CONTROL) {
> + uformat = find_format_by_index(uvc, target->bFormatIndex);
> + if (!uformat)
> + return -EINVAL;
> +
> + uframe = find_frame_by_index(uvc, uformat, ctrl->bFrameIndex);
> + if (!uframe)
> + return -EINVAL;
> +
> + spin_lock(&video->frame_lock);
> +
> + video->cur_frame = uframe;
> + video->cur_format = uformat;
> + video->cur_ival = find_ival_index(uframe, ctrl->dwFrameInterval);
> +
> + spin_unlock(&video->frame_lock);
> + }
> +
> + return 0;
> +}
> +
> +static void
> +uvc_events_process_streaming(struct uvc_device *uvc, uint8_t req, uint8_t cs,
> + struct uvc_request_data *resp)
> +{
> + struct uvc_streaming_control *ctrl;
> +
> + pr_debug("streaming request (req %02x cs %02x)\n", req, cs);
> +
> + if (cs != UVC_VS_PROBE_CONTROL && cs != UVC_VS_COMMIT_CONTROL)
> + return;
> +
> + ctrl = (struct uvc_streaming_control *)&resp->data;
> + resp->length = sizeof(*ctrl);
> +
> + switch (req) {
> + case UVC_SET_CUR:
> + uvc->video.control = cs;
> + resp->length = 34;
> + break;
> +
> + case UVC_GET_CUR:
> + if (cs == UVC_VS_PROBE_CONTROL)
> + memcpy(ctrl, &uvc->video.probe, sizeof(*ctrl));
> + else
> + memcpy(ctrl, &uvc->video.commit, sizeof(*ctrl));
> + break;
> +
> + case UVC_GET_MIN:
> + case UVC_GET_MAX:
> + case UVC_GET_DEF:
> + if (req == UVC_GET_MAX)
> + uvc_fill_streaming_control(uvc, ctrl, -1, -1, UINT_MAX);
> + else
> + uvc_fill_streaming_control(uvc, ctrl, 1, 1, 0);
> + break;
> +
> + case UVC_GET_RES:
> + memset(ctrl, 0, sizeof(*ctrl));
> + break;
> +
> + case UVC_GET_LEN:
> + resp->data[0] = 0x00;
> + resp->data[1] = 0x22;
> + resp->length = 2;
> + break;
> +
> + case UVC_GET_INFO:
> + resp->data[0] = 0x03;
> + resp->length = 1;
> + break;
> + }
> +}
> +
> +static int
> +uvc_events_process_class(struct uvc_device *uvc,
> + const struct usb_ctrlrequest *ctrl,
> + struct uvc_request_data *resp)
> +{
> + unsigned int interface = le16_to_cpu(ctrl->wIndex) & 0xff;
> +
> + if ((ctrl->bRequestType & USB_RECIP_MASK) != USB_RECIP_INTERFACE)
> + return -EINVAL;
> +
> + if (interface == uvc->control_intf)
> + return -EOPNOTSUPP;
> + else if (interface == uvc->streaming_intf)
> + uvc_events_process_streaming(uvc, ctrl->bRequest,
> + le16_to_cpu(ctrl->wValue) >> 8,
> + resp);
> +
> + return 0;
> +}
> +
> +static int
> +uvc_events_process_setup(struct uvc_device *uvc,
> + const struct usb_ctrlrequest *ctrl,
> + struct uvc_request_data *resp)
> +{
> + uvc->video.control = 0;
> +
> + pr_debug("bRequestType %02x bRequest %02x wValue %04x wIndex %04x wLength %04x\n",
> + ctrl->bRequestType, ctrl->bRequest, ctrl->wValue,
> + ctrl->wIndex, ctrl->wLength);
> +
> + switch (ctrl->bRequestType & USB_TYPE_MASK) {
> + case USB_TYPE_STANDARD:
> + return -EOPNOTSUPP;
> +
> + case USB_TYPE_CLASS:
> + return uvc_events_process_class(uvc, ctrl, resp);
> +
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +
> static void
> uvc_function_ep0_complete(struct usb_ep *ep, struct usb_request *req)
> {
> struct uvc_device *uvc = req->context;
> struct v4l2_event v4l2_event;
> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
> + int ret;
>
> if (uvc->event_setup_out) {
> uvc->event_setup_out = 0;
>
> - memset(&v4l2_event, 0, sizeof(v4l2_event));
> - v4l2_event.type = UVC_EVENT_DATA;
> - uvc_event->data.length = req->actual;
> - memcpy(&uvc_event->data.data, req->buf, req->actual);
> - v4l2_event_queue(&uvc->vdev, &v4l2_event);
> + ret = uvc_events_process_data(uvc, req);
> + /* If we have a real error on process */
> + if (ret == -EINVAL) {
> + struct uvc_request_data resp;
> +
> + memset(&resp, 0, sizeof(resp));
> + resp.length = -EL2HLT;
> +
> + uvc_send_response(uvc, &resp);
> + } else if (ret == -EOPNOTSUPP && uvc->data_subscribed) {
> + memset(&v4l2_event, 0, sizeof(v4l2_event));
> + v4l2_event.type = UVC_EVENT_DATA;
> + uvc_event->data.length = req->actual;
> + memcpy(&uvc_event->data.data, req->buf, req->actual);
> + v4l2_event_queue(&uvc->vdev, &v4l2_event);
> + }
> }
> }
>
> @@ -228,6 +434,8 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
> struct uvc_device *uvc = to_uvc(f);
> struct v4l2_event v4l2_event;
> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
> + struct uvc_request_data resp;
> + int ret = 0;
>
> if ((ctrl->bRequestType & USB_TYPE_MASK) != USB_TYPE_CLASS) {
> uvcg_info(f, "invalid request type\n");
> @@ -245,6 +453,23 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
> uvc->event_setup_out = !(ctrl->bRequestType & USB_DIR_IN);
> uvc->event_length = le16_to_cpu(ctrl->wLength);
>
> + memset(&resp, 0, sizeof(resp));
> + resp.length = -EL2HLT;
> +
> + ret = uvc_events_process_setup(uvc, ctrl, &resp);
> + /* If we have no error on process */
> + if (!ret)
> + return uvc_send_response(uvc, &resp);
> +
> + /* If we have a real error on process */
> + if (ret != -EOPNOTSUPP)
> + return ret;
> +
> + /* If we have -EOPNOTSUPP */
> + if (!uvc->setup_subscribed)
> + return uvc_send_response(uvc, &resp);
> +
> + /* If we have setup subscribed */
> memset(&v4l2_event, 0, sizeof(v4l2_event));
> v4l2_event.type = UVC_EVENT_SETUP;
> memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req));
> diff --git a/drivers/usb/gadget/function/uvc.h b/drivers/usb/gadget/function/uvc.h
> index 8a71d0c4abbcd5..cdfd46f3a2861c 100644
> --- a/drivers/usb/gadget/function/uvc.h
> +++ b/drivers/usb/gadget/function/uvc.h
> @@ -13,6 +13,8 @@
> #include <linux/mutex.h>
> #include <linux/spinlock.h>
> #include <linux/usb/composite.h>
> +#include <linux/usb/g_uvc.h>
> +#include <linux/usb/video.h>
> #include <linux/videodev2.h>
> #include <linux/wait.h>
>
> @@ -95,6 +97,12 @@ struct uvc_video {
> unsigned int cur_ival;
>
> struct mutex mutex; /* protects frame parameters */
> + spinlock_t frame_lock;
> +
> + struct uvc_streaming_control probe;
> + struct uvc_streaming_control commit;
> +
> + int control;
>
> unsigned int uvc_num_requests;
>
> @@ -131,6 +139,8 @@ struct uvc_device {
> struct uvc_video video;
> bool func_connected;
> wait_queue_head_t func_connected_queue;
> + bool setup_subscribed;
> + bool data_subscribed;
>
> struct uvcg_streaming_header *header;
>
> @@ -187,5 +197,14 @@ extern struct uvcg_format *find_format_by_index(struct uvc_device *uvc,
> extern struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
> struct uvcg_format *uformat,
> int index);
> +extern int find_format_index(struct uvc_device *uvc,
> + struct uvcg_format *uformat);
> +extern int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval);
> +extern void uvc_fill_streaming_control(struct uvc_device *uvc,
> + struct uvc_streaming_control *ctrl,
> + int iframe, int iformat,
> + unsigned int ival);
> +extern int uvc_send_response(struct uvc_device *uvc,
> + struct uvc_request_data *data);
>
> #endif /* _UVC_GADGET_H_ */
> diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c
> index 3311e98a040537..e4c4bebd2ee860 100644
> --- a/drivers/usb/gadget/function/uvc_v4l2.c
> +++ b/drivers/usb/gadget/function/uvc_v4l2.c
> @@ -90,6 +90,33 @@ struct uvcg_format *find_format_by_index(struct uvc_device *uvc, int index)
> return uformat;
> }
>
> +int find_format_index(struct uvc_device *uvc, struct uvcg_format *uformat)
> +{
> + struct uvcg_format_ptr *format;
> + int i = 1;
> +
> + list_for_each_entry(format, &uvc->header->formats, entry) {
> + if (uformat == format->fmt)
> + return i;
> + i++;
> + }
> +
> + return 0;
> +}
> +
> +int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval)
> +{
> + int i;
> +
> + for (i = 0; i < uframe->frame.b_frame_interval_type; i++) {
> + if (dwFrameInterval == uframe->dw_frame_interval[i])
> + return i + 1;
> + }
> +
> + /* fallback */
> + return 1;
> +}
> +
> struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
> struct uvcg_format *uformat,
> int index)
> @@ -178,8 +205,7 @@ static struct uvcg_frame *find_closest_frame_by_size(struct uvc_device *uvc,
> * Requests handling
> */
>
> -static int
> -uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
> +int uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
> {
> struct usb_composite_dev *cdev = uvc->func.config->cdev;
> struct usb_request *req = uvc->control_req;
> @@ -221,6 +247,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
> struct uvc_video *video = &uvc->video;
> struct uvc_format_desc *fmtdesc;
>
> + spin_lock(&video->frame_lock);
> +
> fmtdesc = to_uvc_format(video->cur_format);
>
> fmt->fmt.pix.pixelformat = fmtdesc->fcc;
> @@ -231,6 +259,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
> fmt->fmt.pix.sizeimage = video->imagesize;
> fmt->fmt.pix.priv = 0;
>
> + spin_unlock(&video->frame_lock);
> +
> return 0;
> }
>
> @@ -255,6 +285,7 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
> struct uvc_video *video = &uvc->video;
> struct uvcg_format *uformat;
> struct uvcg_frame *uframe;
> + int iformat;
> u8 *fcc;
>
> if (fmt->type != video->queue.queue.type)
> @@ -270,6 +301,10 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
> if (!uformat)
> return -EINVAL;
>
> + iformat = find_format_index(uvc, uformat);
> + if (!iformat)
> + return -EINVAL;
> +
> uframe = find_closest_frame_by_size(uvc, uformat,
> fmt->fmt.pix.width, fmt->fmt.pix.height);
> if (!uframe)
> @@ -314,8 +349,12 @@ uvc_v4l2_enum_frameintervals(struct file *file, void *fh,
> if (fival->index >= 1)
> return -EINVAL;
>
> + spin_lock(&video->frame_lock);
> +
> fival->discrete.numerator =
> uframe->dw_frame_interval[video->cur_ival - 1];
> +
> + spin_unlock(&video->frame_lock);
> } else {
> if (fival->index >= uframe->frame.b_frame_interval_type)
> return -EINVAL;
> @@ -347,8 +386,12 @@ uvc_v4l2_enum_framesizes(struct file *file, void *fh,
> if (fsize->index >= 1)
> return -EINVAL;
>
> + spin_lock(&video->frame_lock);
> +
> uformat = video->cur_format;
> uframe = video->cur_frame;
> +
> + spin_unlock(&video->frame_lock);
> } else {
> uformat = find_format_by_pix(uvc, fsize->pixel_format);
> if (!uformat)
> @@ -382,7 +425,11 @@ uvc_v4l2_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
> if (f->index >= 1)
> return -EINVAL;
>
> + spin_lock(&video->frame_lock);
> +
> uformat = video->cur_format;
> +
> + spin_unlock(&video->frame_lock);
> } else {
> if (f->index >= uvc->header->num_fmt)
> return -EINVAL;
> @@ -506,14 +553,20 @@ uvc_v4l2_subscribe_event(struct v4l2_fh *fh,
> if (sub->type < UVC_EVENT_FIRST || sub->type > UVC_EVENT_LAST)
> return -EINVAL;
>
> - if (sub->type == UVC_EVENT_SETUP && uvc->func_connected)
> + if (sub->type == UVC_EVENT_STREAMON && uvc->func_connected)
> return -EBUSY;
>
> ret = v4l2_event_subscribe(fh, sub, 2, NULL);
> if (ret < 0)
> return ret;
>
> - if (sub->type == UVC_EVENT_SETUP) {
> + if (sub->type == UVC_EVENT_SETUP)
> + uvc->setup_subscribed = true;
> +
> + if (sub->type == UVC_EVENT_DATA)
> + uvc->data_subscribed = true;
> +
> + if (sub->type == UVC_EVENT_STREAMON) {
> uvc->func_connected = true;
> handle->is_uvc_app_handle = true;
> uvc_function_connect(uvc);
> @@ -543,7 +596,10 @@ uvc_v4l2_unsubscribe_event(struct v4l2_fh *fh,
> if (ret < 0)
> return ret;
>
> - if (sub->type == UVC_EVENT_SETUP && handle->is_uvc_app_handle) {
> + if (sub->type == UVC_EVENT_SETUP)
> + uvc->setup_subscribed = false;
> +
> + if (sub->type == UVC_EVENT_STREAMON && handle->is_uvc_app_handle) {
> uvc_v4l2_disable(uvc);
> handle->is_uvc_app_handle = false;
> }
> diff --git a/drivers/usb/gadget/function/uvc_video.c b/drivers/usb/gadget/function/uvc_video.c
> index 37867c93073418..c20d832c4a0b8a 100644
> --- a/drivers/usb/gadget/function/uvc_video.c
> +++ b/drivers/usb/gadget/function/uvc_video.c
> @@ -533,10 +533,11 @@ static int uvc_default_frame_interval(struct uvc_video *video)
> */
> int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
> {
> - int iframe;
> + int iframe, iformat;
>
> INIT_LIST_HEAD(&video->req_free);
> spin_lock_init(&video->req_lock);
> + spin_lock_init(&video->frame_lock);
> INIT_WORK(&video->pump, uvcg_video_pump);
>
> if (list_empty(&uvc->header->formats))
> @@ -547,6 +548,10 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
> if (!video->cur_format)
> return -EINVAL;
>
> + iformat = find_format_index(uvc, video->cur_format);
> + if (!iformat)
> + return -EINVAL;
> +
> iframe = uvc_frame_default(video->cur_format);
> if (!iframe)
> return -EINVAL;
> @@ -557,6 +562,11 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
>
> video->cur_ival = uvc_default_frame_interval(video);
>
> + uvc_fill_streaming_control(uvc, &video->probe, iframe, iformat,
> + video->cur_ival);
> + uvc_fill_streaming_control(uvc, &video->commit, iframe, iformat,
> + video->cur_ival);
> +
> /* Initialize the video buffers queue. */
> uvcg_queue_init(&video->queue, uvc->v4l2_dev.dev->parent,
> V4L2_BUF_TYPE_VIDEO_OUTPUT, &video->mutex);
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code
2022-09-07 15:07 ` Laurent Pinchart
@ 2022-09-07 15:15 ` Michael Grzeschik
2022-09-07 15:32 ` Laurent Pinchart
0 siblings, 1 reply; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 15:15 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-usb, linux-media, balbi, paul.elder, kernel, nicolas,
kieran.bingham
[-- Attachment #1: Type: text/plain, Size: 21230 bytes --]
Hi Laurent,
On Wed, Sep 07, 2022 at 06:07:34PM +0300, Laurent Pinchart wrote:
>On Wed, Sep 07, 2022 at 04:02:54PM +0200, Michael Grzeschik wrote:
>> The Hostside format selection is currently only done in userspace, as
>> the events for SET_CUR and GET_CUR are always moved to the application
>> layer. Since the v4l2 device parses the configfs data, the format
>> negotiation can be done in the kernel. This patch adds the functions to
>> set the current configuration while continuing to forward all unknown
>> events to the userspace level.
>
>Why do you think this is better done in kernel space, given that
>userspace has to process the event anyway ? It's more complexity in the
>kernel side, for little added value as far as I can see. It will also
>make it more difficult to handle different UVC versions (in particular
>UVC 1.5). I'd rather not go in this direction if there's no clear
>benefit.
The current case is to set configfs from userspace to ensure
the host sees only what we configured. Then the userspace has to parse
this configfs again, to be sure not to allow something on
SET_CUR/GET_CUR that is not valid from configfs configuration. Since the
configfs-Setup could be done from another application comming from
somewhere in the userspace this limit will always stay.
Since the kernel knows the configfs-setup it was given in the beginning
it can ensure that SET_CUR/GET_CUR will only handle valid setups.
When done in Kernel, we can also use simple v4l2 API calls like try_format to
ask the driver what the host side did configure. So we can use simple
abstracion-libs like gstreamer for this.
I thought I already mentioned this all in the Cover-Letter of this
series.
Michael
>> Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
>>
>> ---
>> v1 -> v2:
>> - fixed the commit message
>> - changed pr_debug to pr_err in events_process_data
>> - aligned many indentations
>> - simplified uvc_events_process_data
>> - fixed uvc_fill_streaming_control calls in uvcg_video_init
>> - added setup_subscribed to decide if userspace takes over on EOPNOTSUPP
>> - added data_subscribed to decide if userspace takes over on EOPNOTSUPP
>> - removed duplicate send_response
>> - wrting fmt and frm in full
>> v2 -> v3:
>> - added find_format_index to set the right probe
>> v3 -> v4:
>> - add function find_ival_index and use for cur_ival
>> - fix swapped frame and format in uvc_events_process_data on uvc_fill_streaming_control
>> - set proper resp.length on ep0 complete
>> - dropped setting cur_probe on set_format since function was removed
>> - added locking around getting correspondent cur_{frame,format,ival}
>> v4 -> v5:
>> - fixed sparse errors reported by kernel test robot
>> v5 -> v6:
>> - fixed the handling in uvc_function_ep0_complete after events_process_data
>> v6 -> v7:
>> - set dwMaxPayloadTransferSize unconditionally from streaming_maxpacket
>> - fixed check for interface with masking for 0xff
>> v7 -> v8:
>> -
>>
>> drivers/usb/gadget/function/f_uvc.c | 237 +++++++++++++++++++++++-
>> drivers/usb/gadget/function/uvc.h | 19 ++
>> drivers/usb/gadget/function/uvc_v4l2.c | 66 ++++++-
>> drivers/usb/gadget/function/uvc_video.c | 12 +-
>> 4 files changed, 322 insertions(+), 12 deletions(-)
>>
>> diff --git a/drivers/usb/gadget/function/f_uvc.c b/drivers/usb/gadget/function/f_uvc.c
>> index a8dcf5f01f16de..fc3fabc47018cd 100644
>> --- a/drivers/usb/gadget/function/f_uvc.c
>> +++ b/drivers/usb/gadget/function/f_uvc.c
>> @@ -16,7 +16,6 @@
>> #include <linux/string.h>
>> #include <linux/usb/ch9.h>
>> #include <linux/usb/gadget.h>
>> -#include <linux/usb/g_uvc.h>
>> #include <linux/usb/video.h>
>> #include <linux/vmalloc.h>
>> #include <linux/wait.h>
>> @@ -204,21 +203,228 @@ static const struct usb_descriptor_header * const uvc_ss_streaming[] = {
>> * Control requests
>> */
>>
>> +void uvc_fill_streaming_control(struct uvc_device *uvc,
>> + struct uvc_streaming_control *ctrl,
>> + int iframe, int iformat, unsigned int ival)
>> +{
>> + struct f_uvc_opts *opts;
>> + struct uvcg_format *uformat;
>> + struct uvcg_frame *uframe;
>> +
>> + /* Restrict the iformat, iframe and ival to valid values. Negative
>> + * values for ifrmat and iframe will result in the maximum valid value
>> + * being selected
>> + */
>> + iformat = clamp((unsigned int)iformat, 1U,
>> + (unsigned int)uvc->header->num_fmt);
>> + uformat = find_format_by_index(uvc, iformat);
>> + if (!uformat)
>> + return;
>> +
>> + iframe = clamp((unsigned int)iframe, 1U,
>> + (unsigned int)uformat->num_frames);
>> + uframe = find_frame_by_index(uvc, uformat, iframe);
>> + if (!uframe)
>> + return;
>> +
>> + ival = clamp((unsigned int)ival, 1U,
>> + (unsigned int)uframe->frame.b_frame_interval_type);
>> + if (!uframe->dw_frame_interval[ival - 1])
>> + return;
>> +
>> + opts = fi_to_f_uvc_opts(uvc->func.fi);
>> +
>> + memset(ctrl, 0, sizeof(*ctrl));
>> +
>> + ctrl->bmHint = 1;
>> + ctrl->bFormatIndex = iformat;
>> + ctrl->bFrameIndex = iframe;
>> + ctrl->dwFrameInterval = uframe->dw_frame_interval[ival - 1];
>> + ctrl->dwMaxVideoFrameSize =
>> + uframe->frame.dw_max_video_frame_buffer_size;
>> +
>> + ctrl->dwMaxPayloadTransferSize = opts->streaming_maxpacket;
>> + ctrl->bmFramingInfo = 3;
>> + ctrl->bPreferedVersion = 1;
>> + ctrl->bMaxVersion = 1;
>> +}
>> +
>> +static int uvc_events_process_data(struct uvc_device *uvc,
>> + struct usb_request *req)
>> +{
>> + struct uvc_video *video = &uvc->video;
>> + struct uvc_streaming_control *target;
>> + struct uvc_streaming_control *ctrl;
>> + struct uvcg_frame *uframe;
>> + struct uvcg_format *uformat;
>> +
>> + switch (video->control) {
>> + case UVC_VS_PROBE_CONTROL:
>> + pr_debug("setting probe control, length = %d\n", req->actual);
>> + target = &video->probe;
>> + break;
>> +
>> + case UVC_VS_COMMIT_CONTROL:
>> + pr_debug("setting commit control, length = %d\n", req->actual);
>> + target = &video->commit;
>> + break;
>> +
>> + default:
>> + pr_err("setting unknown control, length = %d\n", req->actual);
>> + return -EOPNOTSUPP;
>> + }
>> +
>> + ctrl = (struct uvc_streaming_control *)req->buf;
>> +
>> + uvc_fill_streaming_control(uvc, target, ctrl->bFrameIndex,
>> + ctrl->bFormatIndex, ctrl->dwFrameInterval);
>> +
>> + if (video->control == UVC_VS_COMMIT_CONTROL) {
>> + uformat = find_format_by_index(uvc, target->bFormatIndex);
>> + if (!uformat)
>> + return -EINVAL;
>> +
>> + uframe = find_frame_by_index(uvc, uformat, ctrl->bFrameIndex);
>> + if (!uframe)
>> + return -EINVAL;
>> +
>> + spin_lock(&video->frame_lock);
>> +
>> + video->cur_frame = uframe;
>> + video->cur_format = uformat;
>> + video->cur_ival = find_ival_index(uframe, ctrl->dwFrameInterval);
>> +
>> + spin_unlock(&video->frame_lock);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static void
>> +uvc_events_process_streaming(struct uvc_device *uvc, uint8_t req, uint8_t cs,
>> + struct uvc_request_data *resp)
>> +{
>> + struct uvc_streaming_control *ctrl;
>> +
>> + pr_debug("streaming request (req %02x cs %02x)\n", req, cs);
>> +
>> + if (cs != UVC_VS_PROBE_CONTROL && cs != UVC_VS_COMMIT_CONTROL)
>> + return;
>> +
>> + ctrl = (struct uvc_streaming_control *)&resp->data;
>> + resp->length = sizeof(*ctrl);
>> +
>> + switch (req) {
>> + case UVC_SET_CUR:
>> + uvc->video.control = cs;
>> + resp->length = 34;
>> + break;
>> +
>> + case UVC_GET_CUR:
>> + if (cs == UVC_VS_PROBE_CONTROL)
>> + memcpy(ctrl, &uvc->video.probe, sizeof(*ctrl));
>> + else
>> + memcpy(ctrl, &uvc->video.commit, sizeof(*ctrl));
>> + break;
>> +
>> + case UVC_GET_MIN:
>> + case UVC_GET_MAX:
>> + case UVC_GET_DEF:
>> + if (req == UVC_GET_MAX)
>> + uvc_fill_streaming_control(uvc, ctrl, -1, -1, UINT_MAX);
>> + else
>> + uvc_fill_streaming_control(uvc, ctrl, 1, 1, 0);
>> + break;
>> +
>> + case UVC_GET_RES:
>> + memset(ctrl, 0, sizeof(*ctrl));
>> + break;
>> +
>> + case UVC_GET_LEN:
>> + resp->data[0] = 0x00;
>> + resp->data[1] = 0x22;
>> + resp->length = 2;
>> + break;
>> +
>> + case UVC_GET_INFO:
>> + resp->data[0] = 0x03;
>> + resp->length = 1;
>> + break;
>> + }
>> +}
>> +
>> +static int
>> +uvc_events_process_class(struct uvc_device *uvc,
>> + const struct usb_ctrlrequest *ctrl,
>> + struct uvc_request_data *resp)
>> +{
>> + unsigned int interface = le16_to_cpu(ctrl->wIndex) & 0xff;
>> +
>> + if ((ctrl->bRequestType & USB_RECIP_MASK) != USB_RECIP_INTERFACE)
>> + return -EINVAL;
>> +
>> + if (interface == uvc->control_intf)
>> + return -EOPNOTSUPP;
>> + else if (interface == uvc->streaming_intf)
>> + uvc_events_process_streaming(uvc, ctrl->bRequest,
>> + le16_to_cpu(ctrl->wValue) >> 8,
>> + resp);
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +uvc_events_process_setup(struct uvc_device *uvc,
>> + const struct usb_ctrlrequest *ctrl,
>> + struct uvc_request_data *resp)
>> +{
>> + uvc->video.control = 0;
>> +
>> + pr_debug("bRequestType %02x bRequest %02x wValue %04x wIndex %04x wLength %04x\n",
>> + ctrl->bRequestType, ctrl->bRequest, ctrl->wValue,
>> + ctrl->wIndex, ctrl->wLength);
>> +
>> + switch (ctrl->bRequestType & USB_TYPE_MASK) {
>> + case USB_TYPE_STANDARD:
>> + return -EOPNOTSUPP;
>> +
>> + case USB_TYPE_CLASS:
>> + return uvc_events_process_class(uvc, ctrl, resp);
>> +
>> + default:
>> + break;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static void
>> uvc_function_ep0_complete(struct usb_ep *ep, struct usb_request *req)
>> {
>> struct uvc_device *uvc = req->context;
>> struct v4l2_event v4l2_event;
>> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
>> + int ret;
>>
>> if (uvc->event_setup_out) {
>> uvc->event_setup_out = 0;
>>
>> - memset(&v4l2_event, 0, sizeof(v4l2_event));
>> - v4l2_event.type = UVC_EVENT_DATA;
>> - uvc_event->data.length = req->actual;
>> - memcpy(&uvc_event->data.data, req->buf, req->actual);
>> - v4l2_event_queue(&uvc->vdev, &v4l2_event);
>> + ret = uvc_events_process_data(uvc, req);
>> + /* If we have a real error on process */
>> + if (ret == -EINVAL) {
>> + struct uvc_request_data resp;
>> +
>> + memset(&resp, 0, sizeof(resp));
>> + resp.length = -EL2HLT;
>> +
>> + uvc_send_response(uvc, &resp);
>> + } else if (ret == -EOPNOTSUPP && uvc->data_subscribed) {
>> + memset(&v4l2_event, 0, sizeof(v4l2_event));
>> + v4l2_event.type = UVC_EVENT_DATA;
>> + uvc_event->data.length = req->actual;
>> + memcpy(&uvc_event->data.data, req->buf, req->actual);
>> + v4l2_event_queue(&uvc->vdev, &v4l2_event);
>> + }
>> }
>> }
>>
>> @@ -228,6 +434,8 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
>> struct uvc_device *uvc = to_uvc(f);
>> struct v4l2_event v4l2_event;
>> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
>> + struct uvc_request_data resp;
>> + int ret = 0;
>>
>> if ((ctrl->bRequestType & USB_TYPE_MASK) != USB_TYPE_CLASS) {
>> uvcg_info(f, "invalid request type\n");
>> @@ -245,6 +453,23 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
>> uvc->event_setup_out = !(ctrl->bRequestType & USB_DIR_IN);
>> uvc->event_length = le16_to_cpu(ctrl->wLength);
>>
>> + memset(&resp, 0, sizeof(resp));
>> + resp.length = -EL2HLT;
>> +
>> + ret = uvc_events_process_setup(uvc, ctrl, &resp);
>> + /* If we have no error on process */
>> + if (!ret)
>> + return uvc_send_response(uvc, &resp);
>> +
>> + /* If we have a real error on process */
>> + if (ret != -EOPNOTSUPP)
>> + return ret;
>> +
>> + /* If we have -EOPNOTSUPP */
>> + if (!uvc->setup_subscribed)
>> + return uvc_send_response(uvc, &resp);
>> +
>> + /* If we have setup subscribed */
>> memset(&v4l2_event, 0, sizeof(v4l2_event));
>> v4l2_event.type = UVC_EVENT_SETUP;
>> memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req));
>> diff --git a/drivers/usb/gadget/function/uvc.h b/drivers/usb/gadget/function/uvc.h
>> index 8a71d0c4abbcd5..cdfd46f3a2861c 100644
>> --- a/drivers/usb/gadget/function/uvc.h
>> +++ b/drivers/usb/gadget/function/uvc.h
>> @@ -13,6 +13,8 @@
>> #include <linux/mutex.h>
>> #include <linux/spinlock.h>
>> #include <linux/usb/composite.h>
>> +#include <linux/usb/g_uvc.h>
>> +#include <linux/usb/video.h>
>> #include <linux/videodev2.h>
>> #include <linux/wait.h>
>>
>> @@ -95,6 +97,12 @@ struct uvc_video {
>> unsigned int cur_ival;
>>
>> struct mutex mutex; /* protects frame parameters */
>> + spinlock_t frame_lock;
>> +
>> + struct uvc_streaming_control probe;
>> + struct uvc_streaming_control commit;
>> +
>> + int control;
>>
>> unsigned int uvc_num_requests;
>>
>> @@ -131,6 +139,8 @@ struct uvc_device {
>> struct uvc_video video;
>> bool func_connected;
>> wait_queue_head_t func_connected_queue;
>> + bool setup_subscribed;
>> + bool data_subscribed;
>>
>> struct uvcg_streaming_header *header;
>>
>> @@ -187,5 +197,14 @@ extern struct uvcg_format *find_format_by_index(struct uvc_device *uvc,
>> extern struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
>> struct uvcg_format *uformat,
>> int index);
>> +extern int find_format_index(struct uvc_device *uvc,
>> + struct uvcg_format *uformat);
>> +extern int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval);
>> +extern void uvc_fill_streaming_control(struct uvc_device *uvc,
>> + struct uvc_streaming_control *ctrl,
>> + int iframe, int iformat,
>> + unsigned int ival);
>> +extern int uvc_send_response(struct uvc_device *uvc,
>> + struct uvc_request_data *data);
>>
>> #endif /* _UVC_GADGET_H_ */
>> diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c
>> index 3311e98a040537..e4c4bebd2ee860 100644
>> --- a/drivers/usb/gadget/function/uvc_v4l2.c
>> +++ b/drivers/usb/gadget/function/uvc_v4l2.c
>> @@ -90,6 +90,33 @@ struct uvcg_format *find_format_by_index(struct uvc_device *uvc, int index)
>> return uformat;
>> }
>>
>> +int find_format_index(struct uvc_device *uvc, struct uvcg_format *uformat)
>> +{
>> + struct uvcg_format_ptr *format;
>> + int i = 1;
>> +
>> + list_for_each_entry(format, &uvc->header->formats, entry) {
>> + if (uformat == format->fmt)
>> + return i;
>> + i++;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < uframe->frame.b_frame_interval_type; i++) {
>> + if (dwFrameInterval == uframe->dw_frame_interval[i])
>> + return i + 1;
>> + }
>> +
>> + /* fallback */
>> + return 1;
>> +}
>> +
>> struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
>> struct uvcg_format *uformat,
>> int index)
>> @@ -178,8 +205,7 @@ static struct uvcg_frame *find_closest_frame_by_size(struct uvc_device *uvc,
>> * Requests handling
>> */
>>
>> -static int
>> -uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
>> +int uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
>> {
>> struct usb_composite_dev *cdev = uvc->func.config->cdev;
>> struct usb_request *req = uvc->control_req;
>> @@ -221,6 +247,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
>> struct uvc_video *video = &uvc->video;
>> struct uvc_format_desc *fmtdesc;
>>
>> + spin_lock(&video->frame_lock);
>> +
>> fmtdesc = to_uvc_format(video->cur_format);
>>
>> fmt->fmt.pix.pixelformat = fmtdesc->fcc;
>> @@ -231,6 +259,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
>> fmt->fmt.pix.sizeimage = video->imagesize;
>> fmt->fmt.pix.priv = 0;
>>
>> + spin_unlock(&video->frame_lock);
>> +
>> return 0;
>> }
>>
>> @@ -255,6 +285,7 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
>> struct uvc_video *video = &uvc->video;
>> struct uvcg_format *uformat;
>> struct uvcg_frame *uframe;
>> + int iformat;
>> u8 *fcc;
>>
>> if (fmt->type != video->queue.queue.type)
>> @@ -270,6 +301,10 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
>> if (!uformat)
>> return -EINVAL;
>>
>> + iformat = find_format_index(uvc, uformat);
>> + if (!iformat)
>> + return -EINVAL;
>> +
>> uframe = find_closest_frame_by_size(uvc, uformat,
>> fmt->fmt.pix.width, fmt->fmt.pix.height);
>> if (!uframe)
>> @@ -314,8 +349,12 @@ uvc_v4l2_enum_frameintervals(struct file *file, void *fh,
>> if (fival->index >= 1)
>> return -EINVAL;
>>
>> + spin_lock(&video->frame_lock);
>> +
>> fival->discrete.numerator =
>> uframe->dw_frame_interval[video->cur_ival - 1];
>> +
>> + spin_unlock(&video->frame_lock);
>> } else {
>> if (fival->index >= uframe->frame.b_frame_interval_type)
>> return -EINVAL;
>> @@ -347,8 +386,12 @@ uvc_v4l2_enum_framesizes(struct file *file, void *fh,
>> if (fsize->index >= 1)
>> return -EINVAL;
>>
>> + spin_lock(&video->frame_lock);
>> +
>> uformat = video->cur_format;
>> uframe = video->cur_frame;
>> +
>> + spin_unlock(&video->frame_lock);
>> } else {
>> uformat = find_format_by_pix(uvc, fsize->pixel_format);
>> if (!uformat)
>> @@ -382,7 +425,11 @@ uvc_v4l2_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
>> if (f->index >= 1)
>> return -EINVAL;
>>
>> + spin_lock(&video->frame_lock);
>> +
>> uformat = video->cur_format;
>> +
>> + spin_unlock(&video->frame_lock);
>> } else {
>> if (f->index >= uvc->header->num_fmt)
>> return -EINVAL;
>> @@ -506,14 +553,20 @@ uvc_v4l2_subscribe_event(struct v4l2_fh *fh,
>> if (sub->type < UVC_EVENT_FIRST || sub->type > UVC_EVENT_LAST)
>> return -EINVAL;
>>
>> - if (sub->type == UVC_EVENT_SETUP && uvc->func_connected)
>> + if (sub->type == UVC_EVENT_STREAMON && uvc->func_connected)
>> return -EBUSY;
>>
>> ret = v4l2_event_subscribe(fh, sub, 2, NULL);
>> if (ret < 0)
>> return ret;
>>
>> - if (sub->type == UVC_EVENT_SETUP) {
>> + if (sub->type == UVC_EVENT_SETUP)
>> + uvc->setup_subscribed = true;
>> +
>> + if (sub->type == UVC_EVENT_DATA)
>> + uvc->data_subscribed = true;
>> +
>> + if (sub->type == UVC_EVENT_STREAMON) {
>> uvc->func_connected = true;
>> handle->is_uvc_app_handle = true;
>> uvc_function_connect(uvc);
>> @@ -543,7 +596,10 @@ uvc_v4l2_unsubscribe_event(struct v4l2_fh *fh,
>> if (ret < 0)
>> return ret;
>>
>> - if (sub->type == UVC_EVENT_SETUP && handle->is_uvc_app_handle) {
>> + if (sub->type == UVC_EVENT_SETUP)
>> + uvc->setup_subscribed = false;
>> +
>> + if (sub->type == UVC_EVENT_STREAMON && handle->is_uvc_app_handle) {
>> uvc_v4l2_disable(uvc);
>> handle->is_uvc_app_handle = false;
>> }
>> diff --git a/drivers/usb/gadget/function/uvc_video.c b/drivers/usb/gadget/function/uvc_video.c
>> index 37867c93073418..c20d832c4a0b8a 100644
>> --- a/drivers/usb/gadget/function/uvc_video.c
>> +++ b/drivers/usb/gadget/function/uvc_video.c
>> @@ -533,10 +533,11 @@ static int uvc_default_frame_interval(struct uvc_video *video)
>> */
>> int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
>> {
>> - int iframe;
>> + int iframe, iformat;
>>
>> INIT_LIST_HEAD(&video->req_free);
>> spin_lock_init(&video->req_lock);
>> + spin_lock_init(&video->frame_lock);
>> INIT_WORK(&video->pump, uvcg_video_pump);
>>
>> if (list_empty(&uvc->header->formats))
>> @@ -547,6 +548,10 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
>> if (!video->cur_format)
>> return -EINVAL;
>>
>> + iformat = find_format_index(uvc, video->cur_format);
>> + if (!iformat)
>> + return -EINVAL;
>> +
>> iframe = uvc_frame_default(video->cur_format);
>> if (!iframe)
>> return -EINVAL;
>> @@ -557,6 +562,11 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
>>
>> video->cur_ival = uvc_default_frame_interval(video);
>>
>> + uvc_fill_streaming_control(uvc, &video->probe, iframe, iformat,
>> + video->cur_ival);
>> + uvc_fill_streaming_control(uvc, &video->commit, iframe, iformat,
>> + video->cur_ival);
>> +
>> /* Initialize the video buffers queue. */
>> uvcg_queue_init(&video->queue, uvc->v4l2_dev.dev->parent,
>> V4L2_BUF_TYPE_VIDEO_OUTPUT, &video->mutex);
>
>--
>Regards,
>
>Laurent Pinchart
>
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code
2022-09-07 15:15 ` Michael Grzeschik
@ 2022-09-07 15:32 ` Laurent Pinchart
2022-09-07 15:47 ` Michael Grzeschik
0 siblings, 1 reply; 12+ messages in thread
From: Laurent Pinchart @ 2022-09-07 15:32 UTC (permalink / raw)
To: Michael Grzeschik
Cc: linux-usb, linux-media, balbi, paul.elder, kernel, nicolas,
kieran.bingham
Hi Michael,
On Wed, Sep 07, 2022 at 05:15:16PM +0200, Michael Grzeschik wrote:
> On Wed, Sep 07, 2022 at 06:07:34PM +0300, Laurent Pinchart wrote:
> > On Wed, Sep 07, 2022 at 04:02:54PM +0200, Michael Grzeschik wrote:
> >> The Hostside format selection is currently only done in userspace, as
> >> the events for SET_CUR and GET_CUR are always moved to the application
> >> layer. Since the v4l2 device parses the configfs data, the format
> >> negotiation can be done in the kernel. This patch adds the functions to
> >> set the current configuration while continuing to forward all unknown
> >> events to the userspace level.
> >
> > Why do you think this is better done in kernel space, given that
> > userspace has to process the event anyway ? It's more complexity in the
> > kernel side, for little added value as far as I can see. It will also
> > make it more difficult to handle different UVC versions (in particular
> > UVC 1.5). I'd rather not go in this direction if there's no clear
> > benefit.
>
> The current case is to set configfs from userspace to ensure
> the host sees only what we configured. Then the userspace has to parse
> this configfs again, to be sure not to allow something on
> SET_CUR/GET_CUR that is not valid from configfs configuration. Since the
> configfs-Setup could be done from another application comming from
> somewhere in the userspace this limit will always stay.
That really depends on the architecture of the userspace stack, when the
same userspace application configures the gadget and handles the runtime
operations, it won't have to parse configfs. I'd also argue that in
practical use cases, the application will likely need to know the list
of formats and resolutions that are exposed by the gadget in order to
prepare for supporting them properly (for instance, allocating buffers
large enough to store the largest resolution is common when you want to
lower stream start delays if your system is not memory-constrained).
> Since the kernel knows the configfs-setup it was given in the beginning
> it can ensure that SET_CUR/GET_CUR will only handle valid setups.
>
> When done in Kernel, we can also use simple v4l2 API calls like try_format to
> ask the driver what the host side did configure. So we can use simple
> abstracion-libs like gstreamer for this.
This will cause trouble when extending support to UVC 1.5 as the
complexity will grow on the kernel side. Furthermore, by handling the
video probe and commit control in kernel space, you'd removing the
possibility for userspace to decide on how to handle the bHint flags, or
how to negociate dwFrameInterval key frame rate, compression quality, or
the additional fields specific to UVC 1.5. This effectively hardcodes
one particular policy in the driver, and that shouldn't be the role of
the kernel.
> I thought I already mentioned this all in the Cover-Letter of this
> series.
Sorry, I missed that.
> >> Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de>
> >>
> >> ---
> >> v1 -> v2:
> >> - fixed the commit message
> >> - changed pr_debug to pr_err in events_process_data
> >> - aligned many indentations
> >> - simplified uvc_events_process_data
> >> - fixed uvc_fill_streaming_control calls in uvcg_video_init
> >> - added setup_subscribed to decide if userspace takes over on EOPNOTSUPP
> >> - added data_subscribed to decide if userspace takes over on EOPNOTSUPP
> >> - removed duplicate send_response
> >> - wrting fmt and frm in full
> >> v2 -> v3:
> >> - added find_format_index to set the right probe
> >> v3 -> v4:
> >> - add function find_ival_index and use for cur_ival
> >> - fix swapped frame and format in uvc_events_process_data on uvc_fill_streaming_control
> >> - set proper resp.length on ep0 complete
> >> - dropped setting cur_probe on set_format since function was removed
> >> - added locking around getting correspondent cur_{frame,format,ival}
> >> v4 -> v5:
> >> - fixed sparse errors reported by kernel test robot
> >> v5 -> v6:
> >> - fixed the handling in uvc_function_ep0_complete after events_process_data
> >> v6 -> v7:
> >> - set dwMaxPayloadTransferSize unconditionally from streaming_maxpacket
> >> - fixed check for interface with masking for 0xff
> >> v7 -> v8:
> >> -
> >>
> >> drivers/usb/gadget/function/f_uvc.c | 237 +++++++++++++++++++++++-
> >> drivers/usb/gadget/function/uvc.h | 19 ++
> >> drivers/usb/gadget/function/uvc_v4l2.c | 66 ++++++-
> >> drivers/usb/gadget/function/uvc_video.c | 12 +-
> >> 4 files changed, 322 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/drivers/usb/gadget/function/f_uvc.c b/drivers/usb/gadget/function/f_uvc.c
> >> index a8dcf5f01f16de..fc3fabc47018cd 100644
> >> --- a/drivers/usb/gadget/function/f_uvc.c
> >> +++ b/drivers/usb/gadget/function/f_uvc.c
> >> @@ -16,7 +16,6 @@
> >> #include <linux/string.h>
> >> #include <linux/usb/ch9.h>
> >> #include <linux/usb/gadget.h>
> >> -#include <linux/usb/g_uvc.h>
> >> #include <linux/usb/video.h>
> >> #include <linux/vmalloc.h>
> >> #include <linux/wait.h>
> >> @@ -204,21 +203,228 @@ static const struct usb_descriptor_header * const uvc_ss_streaming[] = {
> >> * Control requests
> >> */
> >>
> >> +void uvc_fill_streaming_control(struct uvc_device *uvc,
> >> + struct uvc_streaming_control *ctrl,
> >> + int iframe, int iformat, unsigned int ival)
> >> +{
> >> + struct f_uvc_opts *opts;
> >> + struct uvcg_format *uformat;
> >> + struct uvcg_frame *uframe;
> >> +
> >> + /* Restrict the iformat, iframe and ival to valid values. Negative
> >> + * values for ifrmat and iframe will result in the maximum valid value
> >> + * being selected
> >> + */
> >> + iformat = clamp((unsigned int)iformat, 1U,
> >> + (unsigned int)uvc->header->num_fmt);
> >> + uformat = find_format_by_index(uvc, iformat);
> >> + if (!uformat)
> >> + return;
> >> +
> >> + iframe = clamp((unsigned int)iframe, 1U,
> >> + (unsigned int)uformat->num_frames);
> >> + uframe = find_frame_by_index(uvc, uformat, iframe);
> >> + if (!uframe)
> >> + return;
> >> +
> >> + ival = clamp((unsigned int)ival, 1U,
> >> + (unsigned int)uframe->frame.b_frame_interval_type);
> >> + if (!uframe->dw_frame_interval[ival - 1])
> >> + return;
> >> +
> >> + opts = fi_to_f_uvc_opts(uvc->func.fi);
> >> +
> >> + memset(ctrl, 0, sizeof(*ctrl));
> >> +
> >> + ctrl->bmHint = 1;
> >> + ctrl->bFormatIndex = iformat;
> >> + ctrl->bFrameIndex = iframe;
> >> + ctrl->dwFrameInterval = uframe->dw_frame_interval[ival - 1];
> >> + ctrl->dwMaxVideoFrameSize =
> >> + uframe->frame.dw_max_video_frame_buffer_size;
> >> +
> >> + ctrl->dwMaxPayloadTransferSize = opts->streaming_maxpacket;
> >> + ctrl->bmFramingInfo = 3;
> >> + ctrl->bPreferedVersion = 1;
> >> + ctrl->bMaxVersion = 1;
> >> +}
> >> +
> >> +static int uvc_events_process_data(struct uvc_device *uvc,
> >> + struct usb_request *req)
> >> +{
> >> + struct uvc_video *video = &uvc->video;
> >> + struct uvc_streaming_control *target;
> >> + struct uvc_streaming_control *ctrl;
> >> + struct uvcg_frame *uframe;
> >> + struct uvcg_format *uformat;
> >> +
> >> + switch (video->control) {
> >> + case UVC_VS_PROBE_CONTROL:
> >> + pr_debug("setting probe control, length = %d\n", req->actual);
> >> + target = &video->probe;
> >> + break;
> >> +
> >> + case UVC_VS_COMMIT_CONTROL:
> >> + pr_debug("setting commit control, length = %d\n", req->actual);
> >> + target = &video->commit;
> >> + break;
> >> +
> >> + default:
> >> + pr_err("setting unknown control, length = %d\n", req->actual);
> >> + return -EOPNOTSUPP;
> >> + }
> >> +
> >> + ctrl = (struct uvc_streaming_control *)req->buf;
> >> +
> >> + uvc_fill_streaming_control(uvc, target, ctrl->bFrameIndex,
> >> + ctrl->bFormatIndex, ctrl->dwFrameInterval);
> >> +
> >> + if (video->control == UVC_VS_COMMIT_CONTROL) {
> >> + uformat = find_format_by_index(uvc, target->bFormatIndex);
> >> + if (!uformat)
> >> + return -EINVAL;
> >> +
> >> + uframe = find_frame_by_index(uvc, uformat, ctrl->bFrameIndex);
> >> + if (!uframe)
> >> + return -EINVAL;
> >> +
> >> + spin_lock(&video->frame_lock);
> >> +
> >> + video->cur_frame = uframe;
> >> + video->cur_format = uformat;
> >> + video->cur_ival = find_ival_index(uframe, ctrl->dwFrameInterval);
> >> +
> >> + spin_unlock(&video->frame_lock);
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static void
> >> +uvc_events_process_streaming(struct uvc_device *uvc, uint8_t req, uint8_t cs,
> >> + struct uvc_request_data *resp)
> >> +{
> >> + struct uvc_streaming_control *ctrl;
> >> +
> >> + pr_debug("streaming request (req %02x cs %02x)\n", req, cs);
> >> +
> >> + if (cs != UVC_VS_PROBE_CONTROL && cs != UVC_VS_COMMIT_CONTROL)
> >> + return;
> >> +
> >> + ctrl = (struct uvc_streaming_control *)&resp->data;
> >> + resp->length = sizeof(*ctrl);
> >> +
> >> + switch (req) {
> >> + case UVC_SET_CUR:
> >> + uvc->video.control = cs;
> >> + resp->length = 34;
> >> + break;
> >> +
> >> + case UVC_GET_CUR:
> >> + if (cs == UVC_VS_PROBE_CONTROL)
> >> + memcpy(ctrl, &uvc->video.probe, sizeof(*ctrl));
> >> + else
> >> + memcpy(ctrl, &uvc->video.commit, sizeof(*ctrl));
> >> + break;
> >> +
> >> + case UVC_GET_MIN:
> >> + case UVC_GET_MAX:
> >> + case UVC_GET_DEF:
> >> + if (req == UVC_GET_MAX)
> >> + uvc_fill_streaming_control(uvc, ctrl, -1, -1, UINT_MAX);
> >> + else
> >> + uvc_fill_streaming_control(uvc, ctrl, 1, 1, 0);
> >> + break;
> >> +
> >> + case UVC_GET_RES:
> >> + memset(ctrl, 0, sizeof(*ctrl));
> >> + break;
> >> +
> >> + case UVC_GET_LEN:
> >> + resp->data[0] = 0x00;
> >> + resp->data[1] = 0x22;
> >> + resp->length = 2;
> >> + break;
> >> +
> >> + case UVC_GET_INFO:
> >> + resp->data[0] = 0x03;
> >> + resp->length = 1;
> >> + break;
> >> + }
> >> +}
> >> +
> >> +static int
> >> +uvc_events_process_class(struct uvc_device *uvc,
> >> + const struct usb_ctrlrequest *ctrl,
> >> + struct uvc_request_data *resp)
> >> +{
> >> + unsigned int interface = le16_to_cpu(ctrl->wIndex) & 0xff;
> >> +
> >> + if ((ctrl->bRequestType & USB_RECIP_MASK) != USB_RECIP_INTERFACE)
> >> + return -EINVAL;
> >> +
> >> + if (interface == uvc->control_intf)
> >> + return -EOPNOTSUPP;
> >> + else if (interface == uvc->streaming_intf)
> >> + uvc_events_process_streaming(uvc, ctrl->bRequest,
> >> + le16_to_cpu(ctrl->wValue) >> 8,
> >> + resp);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int
> >> +uvc_events_process_setup(struct uvc_device *uvc,
> >> + const struct usb_ctrlrequest *ctrl,
> >> + struct uvc_request_data *resp)
> >> +{
> >> + uvc->video.control = 0;
> >> +
> >> + pr_debug("bRequestType %02x bRequest %02x wValue %04x wIndex %04x wLength %04x\n",
> >> + ctrl->bRequestType, ctrl->bRequest, ctrl->wValue,
> >> + ctrl->wIndex, ctrl->wLength);
> >> +
> >> + switch (ctrl->bRequestType & USB_TYPE_MASK) {
> >> + case USB_TYPE_STANDARD:
> >> + return -EOPNOTSUPP;
> >> +
> >> + case USB_TYPE_CLASS:
> >> + return uvc_events_process_class(uvc, ctrl, resp);
> >> +
> >> + default:
> >> + break;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> static void
> >> uvc_function_ep0_complete(struct usb_ep *ep, struct usb_request *req)
> >> {
> >> struct uvc_device *uvc = req->context;
> >> struct v4l2_event v4l2_event;
> >> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
> >> + int ret;
> >>
> >> if (uvc->event_setup_out) {
> >> uvc->event_setup_out = 0;
> >>
> >> - memset(&v4l2_event, 0, sizeof(v4l2_event));
> >> - v4l2_event.type = UVC_EVENT_DATA;
> >> - uvc_event->data.length = req->actual;
> >> - memcpy(&uvc_event->data.data, req->buf, req->actual);
> >> - v4l2_event_queue(&uvc->vdev, &v4l2_event);
> >> + ret = uvc_events_process_data(uvc, req);
> >> + /* If we have a real error on process */
> >> + if (ret == -EINVAL) {
> >> + struct uvc_request_data resp;
> >> +
> >> + memset(&resp, 0, sizeof(resp));
> >> + resp.length = -EL2HLT;
> >> +
> >> + uvc_send_response(uvc, &resp);
> >> + } else if (ret == -EOPNOTSUPP && uvc->data_subscribed) {
> >> + memset(&v4l2_event, 0, sizeof(v4l2_event));
> >> + v4l2_event.type = UVC_EVENT_DATA;
> >> + uvc_event->data.length = req->actual;
> >> + memcpy(&uvc_event->data.data, req->buf, req->actual);
> >> + v4l2_event_queue(&uvc->vdev, &v4l2_event);
> >> + }
> >> }
> >> }
> >>
> >> @@ -228,6 +434,8 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
> >> struct uvc_device *uvc = to_uvc(f);
> >> struct v4l2_event v4l2_event;
> >> struct uvc_event *uvc_event = (void *)&v4l2_event.u.data;
> >> + struct uvc_request_data resp;
> >> + int ret = 0;
> >>
> >> if ((ctrl->bRequestType & USB_TYPE_MASK) != USB_TYPE_CLASS) {
> >> uvcg_info(f, "invalid request type\n");
> >> @@ -245,6 +453,23 @@ uvc_function_setup(struct usb_function *f, const struct usb_ctrlrequest *ctrl)
> >> uvc->event_setup_out = !(ctrl->bRequestType & USB_DIR_IN);
> >> uvc->event_length = le16_to_cpu(ctrl->wLength);
> >>
> >> + memset(&resp, 0, sizeof(resp));
> >> + resp.length = -EL2HLT;
> >> +
> >> + ret = uvc_events_process_setup(uvc, ctrl, &resp);
> >> + /* If we have no error on process */
> >> + if (!ret)
> >> + return uvc_send_response(uvc, &resp);
> >> +
> >> + /* If we have a real error on process */
> >> + if (ret != -EOPNOTSUPP)
> >> + return ret;
> >> +
> >> + /* If we have -EOPNOTSUPP */
> >> + if (!uvc->setup_subscribed)
> >> + return uvc_send_response(uvc, &resp);
> >> +
> >> + /* If we have setup subscribed */
> >> memset(&v4l2_event, 0, sizeof(v4l2_event));
> >> v4l2_event.type = UVC_EVENT_SETUP;
> >> memcpy(&uvc_event->req, ctrl, sizeof(uvc_event->req));
> >> diff --git a/drivers/usb/gadget/function/uvc.h b/drivers/usb/gadget/function/uvc.h
> >> index 8a71d0c4abbcd5..cdfd46f3a2861c 100644
> >> --- a/drivers/usb/gadget/function/uvc.h
> >> +++ b/drivers/usb/gadget/function/uvc.h
> >> @@ -13,6 +13,8 @@
> >> #include <linux/mutex.h>
> >> #include <linux/spinlock.h>
> >> #include <linux/usb/composite.h>
> >> +#include <linux/usb/g_uvc.h>
> >> +#include <linux/usb/video.h>
> >> #include <linux/videodev2.h>
> >> #include <linux/wait.h>
> >>
> >> @@ -95,6 +97,12 @@ struct uvc_video {
> >> unsigned int cur_ival;
> >>
> >> struct mutex mutex; /* protects frame parameters */
> >> + spinlock_t frame_lock;
> >> +
> >> + struct uvc_streaming_control probe;
> >> + struct uvc_streaming_control commit;
> >> +
> >> + int control;
> >>
> >> unsigned int uvc_num_requests;
> >>
> >> @@ -131,6 +139,8 @@ struct uvc_device {
> >> struct uvc_video video;
> >> bool func_connected;
> >> wait_queue_head_t func_connected_queue;
> >> + bool setup_subscribed;
> >> + bool data_subscribed;
> >>
> >> struct uvcg_streaming_header *header;
> >>
> >> @@ -187,5 +197,14 @@ extern struct uvcg_format *find_format_by_index(struct uvc_device *uvc,
> >> extern struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
> >> struct uvcg_format *uformat,
> >> int index);
> >> +extern int find_format_index(struct uvc_device *uvc,
> >> + struct uvcg_format *uformat);
> >> +extern int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval);
> >> +extern void uvc_fill_streaming_control(struct uvc_device *uvc,
> >> + struct uvc_streaming_control *ctrl,
> >> + int iframe, int iformat,
> >> + unsigned int ival);
> >> +extern int uvc_send_response(struct uvc_device *uvc,
> >> + struct uvc_request_data *data);
> >>
> >> #endif /* _UVC_GADGET_H_ */
> >> diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c
> >> index 3311e98a040537..e4c4bebd2ee860 100644
> >> --- a/drivers/usb/gadget/function/uvc_v4l2.c
> >> +++ b/drivers/usb/gadget/function/uvc_v4l2.c
> >> @@ -90,6 +90,33 @@ struct uvcg_format *find_format_by_index(struct uvc_device *uvc, int index)
> >> return uformat;
> >> }
> >>
> >> +int find_format_index(struct uvc_device *uvc, struct uvcg_format *uformat)
> >> +{
> >> + struct uvcg_format_ptr *format;
> >> + int i = 1;
> >> +
> >> + list_for_each_entry(format, &uvc->header->formats, entry) {
> >> + if (uformat == format->fmt)
> >> + return i;
> >> + i++;
> >> + }
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +int find_ival_index(struct uvcg_frame *uframe, int dwFrameInterval)
> >> +{
> >> + int i;
> >> +
> >> + for (i = 0; i < uframe->frame.b_frame_interval_type; i++) {
> >> + if (dwFrameInterval == uframe->dw_frame_interval[i])
> >> + return i + 1;
> >> + }
> >> +
> >> + /* fallback */
> >> + return 1;
> >> +}
> >> +
> >> struct uvcg_frame *find_frame_by_index(struct uvc_device *uvc,
> >> struct uvcg_format *uformat,
> >> int index)
> >> @@ -178,8 +205,7 @@ static struct uvcg_frame *find_closest_frame_by_size(struct uvc_device *uvc,
> >> * Requests handling
> >> */
> >>
> >> -static int
> >> -uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
> >> +int uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data)
> >> {
> >> struct usb_composite_dev *cdev = uvc->func.config->cdev;
> >> struct usb_request *req = uvc->control_req;
> >> @@ -221,6 +247,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
> >> struct uvc_video *video = &uvc->video;
> >> struct uvc_format_desc *fmtdesc;
> >>
> >> + spin_lock(&video->frame_lock);
> >> +
> >> fmtdesc = to_uvc_format(video->cur_format);
> >>
> >> fmt->fmt.pix.pixelformat = fmtdesc->fcc;
> >> @@ -231,6 +259,8 @@ uvc_v4l2_get_format(struct file *file, void *fh, struct v4l2_format *fmt)
> >> fmt->fmt.pix.sizeimage = video->imagesize;
> >> fmt->fmt.pix.priv = 0;
> >>
> >> + spin_unlock(&video->frame_lock);
> >> +
> >> return 0;
> >> }
> >>
> >> @@ -255,6 +285,7 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
> >> struct uvc_video *video = &uvc->video;
> >> struct uvcg_format *uformat;
> >> struct uvcg_frame *uframe;
> >> + int iformat;
> >> u8 *fcc;
> >>
> >> if (fmt->type != video->queue.queue.type)
> >> @@ -270,6 +301,10 @@ uvc_v4l2_try_format(struct file *file, void *fh, struct v4l2_format *fmt)
> >> if (!uformat)
> >> return -EINVAL;
> >>
> >> + iformat = find_format_index(uvc, uformat);
> >> + if (!iformat)
> >> + return -EINVAL;
> >> +
> >> uframe = find_closest_frame_by_size(uvc, uformat,
> >> fmt->fmt.pix.width, fmt->fmt.pix.height);
> >> if (!uframe)
> >> @@ -314,8 +349,12 @@ uvc_v4l2_enum_frameintervals(struct file *file, void *fh,
> >> if (fival->index >= 1)
> >> return -EINVAL;
> >>
> >> + spin_lock(&video->frame_lock);
> >> +
> >> fival->discrete.numerator =
> >> uframe->dw_frame_interval[video->cur_ival - 1];
> >> +
> >> + spin_unlock(&video->frame_lock);
> >> } else {
> >> if (fival->index >= uframe->frame.b_frame_interval_type)
> >> return -EINVAL;
> >> @@ -347,8 +386,12 @@ uvc_v4l2_enum_framesizes(struct file *file, void *fh,
> >> if (fsize->index >= 1)
> >> return -EINVAL;
> >>
> >> + spin_lock(&video->frame_lock);
> >> +
> >> uformat = video->cur_format;
> >> uframe = video->cur_frame;
> >> +
> >> + spin_unlock(&video->frame_lock);
> >> } else {
> >> uformat = find_format_by_pix(uvc, fsize->pixel_format);
> >> if (!uformat)
> >> @@ -382,7 +425,11 @@ uvc_v4l2_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
> >> if (f->index >= 1)
> >> return -EINVAL;
> >>
> >> + spin_lock(&video->frame_lock);
> >> +
> >> uformat = video->cur_format;
> >> +
> >> + spin_unlock(&video->frame_lock);
> >> } else {
> >> if (f->index >= uvc->header->num_fmt)
> >> return -EINVAL;
> >> @@ -506,14 +553,20 @@ uvc_v4l2_subscribe_event(struct v4l2_fh *fh,
> >> if (sub->type < UVC_EVENT_FIRST || sub->type > UVC_EVENT_LAST)
> >> return -EINVAL;
> >>
> >> - if (sub->type == UVC_EVENT_SETUP && uvc->func_connected)
> >> + if (sub->type == UVC_EVENT_STREAMON && uvc->func_connected)
> >> return -EBUSY;
> >>
> >> ret = v4l2_event_subscribe(fh, sub, 2, NULL);
> >> if (ret < 0)
> >> return ret;
> >>
> >> - if (sub->type == UVC_EVENT_SETUP) {
> >> + if (sub->type == UVC_EVENT_SETUP)
> >> + uvc->setup_subscribed = true;
> >> +
> >> + if (sub->type == UVC_EVENT_DATA)
> >> + uvc->data_subscribed = true;
> >> +
> >> + if (sub->type == UVC_EVENT_STREAMON) {
> >> uvc->func_connected = true;
> >> handle->is_uvc_app_handle = true;
> >> uvc_function_connect(uvc);
> >> @@ -543,7 +596,10 @@ uvc_v4l2_unsubscribe_event(struct v4l2_fh *fh,
> >> if (ret < 0)
> >> return ret;
> >>
> >> - if (sub->type == UVC_EVENT_SETUP && handle->is_uvc_app_handle) {
> >> + if (sub->type == UVC_EVENT_SETUP)
> >> + uvc->setup_subscribed = false;
> >> +
> >> + if (sub->type == UVC_EVENT_STREAMON && handle->is_uvc_app_handle) {
> >> uvc_v4l2_disable(uvc);
> >> handle->is_uvc_app_handle = false;
> >> }
> >> diff --git a/drivers/usb/gadget/function/uvc_video.c b/drivers/usb/gadget/function/uvc_video.c
> >> index 37867c93073418..c20d832c4a0b8a 100644
> >> --- a/drivers/usb/gadget/function/uvc_video.c
> >> +++ b/drivers/usb/gadget/function/uvc_video.c
> >> @@ -533,10 +533,11 @@ static int uvc_default_frame_interval(struct uvc_video *video)
> >> */
> >> int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
> >> {
> >> - int iframe;
> >> + int iframe, iformat;
> >>
> >> INIT_LIST_HEAD(&video->req_free);
> >> spin_lock_init(&video->req_lock);
> >> + spin_lock_init(&video->frame_lock);
> >> INIT_WORK(&video->pump, uvcg_video_pump);
> >>
> >> if (list_empty(&uvc->header->formats))
> >> @@ -547,6 +548,10 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
> >> if (!video->cur_format)
> >> return -EINVAL;
> >>
> >> + iformat = find_format_index(uvc, video->cur_format);
> >> + if (!iformat)
> >> + return -EINVAL;
> >> +
> >> iframe = uvc_frame_default(video->cur_format);
> >> if (!iframe)
> >> return -EINVAL;
> >> @@ -557,6 +562,11 @@ int uvcg_video_init(struct uvc_video *video, struct uvc_device *uvc)
> >>
> >> video->cur_ival = uvc_default_frame_interval(video);
> >>
> >> + uvc_fill_streaming_control(uvc, &video->probe, iframe, iformat,
> >> + video->cur_ival);
> >> + uvc_fill_streaming_control(uvc, &video->commit, iframe, iformat,
> >> + video->cur_ival);
> >> +
> >> /* Initialize the video buffers queue. */
> >> uvcg_queue_init(&video->queue, uvc->v4l2_dev.dev->parent,
> >> V4L2_BUF_TYPE_VIDEO_OUTPUT, &video->mutex);
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code
2022-09-07 15:32 ` Laurent Pinchart
@ 2022-09-07 15:47 ` Michael Grzeschik
0 siblings, 0 replies; 12+ messages in thread
From: Michael Grzeschik @ 2022-09-07 15:47 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-usb, linux-media, balbi, paul.elder, kernel, nicolas,
kieran.bingham
[-- Attachment #1: Type: text/plain, Size: 3983 bytes --]
On Wed, Sep 07, 2022 at 06:32:09PM +0300, Laurent Pinchart wrote:
>Hi Michael,
>
>On Wed, Sep 07, 2022 at 05:15:16PM +0200, Michael Grzeschik wrote:
>> On Wed, Sep 07, 2022 at 06:07:34PM +0300, Laurent Pinchart wrote:
>> > On Wed, Sep 07, 2022 at 04:02:54PM +0200, Michael Grzeschik wrote:
>> >> The Hostside format selection is currently only done in userspace, as
>> >> the events for SET_CUR and GET_CUR are always moved to the application
>> >> layer. Since the v4l2 device parses the configfs data, the format
>> >> negotiation can be done in the kernel. This patch adds the functions to
>> >> set the current configuration while continuing to forward all unknown
>> >> events to the userspace level.
>> >
>> > Why do you think this is better done in kernel space, given that
>> > userspace has to process the event anyway ? It's more complexity in the
>> > kernel side, for little added value as far as I can see. It will also
>> > make it more difficult to handle different UVC versions (in particular
>> > UVC 1.5). I'd rather not go in this direction if there's no clear
>> > benefit.
>>
>> The current case is to set configfs from userspace to ensure
>> the host sees only what we configured. Then the userspace has to parse
>> this configfs again, to be sure not to allow something on
>> SET_CUR/GET_CUR that is not valid from configfs configuration. Since the
>> configfs-Setup could be done from another application comming from
>> somewhere in the userspace this limit will always stay.
>
>That really depends on the architecture of the userspace stack, when the
>same userspace application configures the gadget and handles the runtime
>operations, it won't have to parse configfs. I'd also argue that in
>practical use cases, the application will likely need to know the list
>of formats and resolutions that are exposed by the gadget in order to
>prepare for supporting them properly (for instance, allocating buffers
>large enough to store the largest resolution is common when you want to
>lower stream start delays if your system is not memory-constrained).
The userspace can for sure can do resolution and format parsing that
is exposed by the gadget. Even more standardized, now with the simple
v4l2 API, without having to parse the whole configfs again.
>> Since the kernel knows the configfs-setup it was given in the beginning
>> it can ensure that SET_CUR/GET_CUR will only handle valid setups.
>>
>> When done in Kernel, we can also use simple v4l2 API calls like try_format to
>> ask the driver what the host side did configure. So we can use simple
>> abstracion-libs like gstreamer for this.
>
>This will cause trouble when extending support to UVC 1.5 as the
>complexity will grow on the kernel side. Furthermore, by handling the
>video probe and commit control in kernel space, you'd removing the
>possibility for userspace to decide on how to handle the bHint flags, or
>how to negociate dwFrameInterval key frame rate, compression quality, or
>the additional fields specific to UVC 1.5. This effectively hardcodes
>one particular policy in the driver, and that shouldn't be the role of
>the kernel.
When you look closely, you will see that for later user cases, I decided
the userspace appliaction will still be able to subscribe for DATA and
SETUP.
It will only fall back to the kernel side of operation if there is
nobody that is able to operate. In those cases, it will take over. I
tested this with your uvc-gadget appliaction. Since it subscribes more
than STREAMON and STREAMOFF, it is still working like before with this
patchstack applied.
Regards,
Michael
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-09-07 15:47 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-09-07 14:02 [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 1/4] media: v4l: move helper functions for fractions from uvc to v4l2-common Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 2/4] media: uvcvideo: move uvc_format_desc to common header Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 3/4] usb: gadget: uvc: add VIDIOC function Michael Grzeschik
2022-09-07 14:02 ` [PATCH v8 4/4] usb: gadget: uvc: add format/frame handling code Michael Grzeschik
2022-09-07 15:07 ` Laurent Pinchart
2022-09-07 15:15 ` Michael Grzeschik
2022-09-07 15:32 ` Laurent Pinchart
2022-09-07 15:47 ` Michael Grzeschik
2022-09-07 14:28 ` [PATCH v8 0/4] usb: gadget: uvc: use configfs entries for negotiation and v4l2 VIDIOCS Greg KH
2022-09-07 14:52 ` Michael Grzeschik
2022-09-07 15:01 ` Greg KH
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).