Re: [RFC PATCH V0 0/7] media: platform: Add support for Face Detection (FD) on mt8183 SoC

[Jerry-ch Chen <Jerry-ch.Chen@mediatek.com>]

Hi Hans, Tomasz,

On Thu, 2019-03-14 at 16:40 +0800, Hans Verkuil wrote:
> Hi Jerry-ch Chen,
> 
> On 2/20/19 8:48 AM, Jerry-ch Chen wrote:
> > Hello,
> > 
> > This is the first version of the RFC patch series adding Face Detection
> > (FD) driver on Mediatek mt8183 SoC, which will be used in camera features
> > on CrOS application. It belongs to the first Mediatek's camera driver
> > series based on V4L2 and media controller framework. I posted the main part
> > of the FD driver as RFC to discuss first and would like some review
> > comments on the overall structure of the driver.
> > 
> > Face Detection (FD) unit provide hardware accelerated face detection
> > feature. It can detect different sizes of faces in a given image.
> > Furthermore, it has the capability to detect the faces of Rotation-in-Plane
> > from -180 to +180 degrees and Rotation-off-Plane from -90 to +90 degrees.
> > 
> > The driver is implemented with V4L2 and media controller framework. We have
> > the following entities describing the FD path.
> 
> Just a high-level comment before you post the next version of this series:
> 
> Please compile the latest version of v4l2-compliance (part of
> git://linuxtv.org/v4l-utils.git) and run it against your driver:
> 
> v4l2-compliance -m /dev/mediaX
> 
> Whenever you post a new version of this series, please do a 'git pull' of
> the v4l-utils repo, recompile and retest with v4l2-compliance and post the
> test results in the cover letter.
> 
> Obviously, there should be no FAILs and probably no warnings.
> 
> I suspect that streaming (e.g. adding the -s10 option to v4l2-compliance)
> probably won't work since v4l2-compliance doesn't know about the meta data
> formats.
> 
> Regards,
> 
> 	Hans
> 

Thanks for comments,
I am reworking FD driver based on general comments of P1 and DIP driver.
After that, I will upload the RFC V1 patch with the results of
v4l2-compliance in the cover-letter.

Best Regards,

	Jerry

> > 
> > 1. Meta input (output video device): connects to FD sub device. It accepts
> >    the input parameter buffer from userspace. The metadata interface used
> >    currently is only a temporary solution to kick off driver development
> >    and is not ready for reviewed yet.
> > 
> > 2. RAW (output video device): connects to FD sub device. It accepts input
> >    image buffer from userspace.
> > 
> > 3. FD (sub device): connects to Meta output. When processing an image,
> >    FD hardware only returns the statistics of detected faces so it needs
> >    only one capture video devices to return the streaming data to the user.
> > 
> > 4. Meta output (capture video device): Return the result of detected faces
> >    as metadata output.
> > 
> >    The overall file structure of the FD driver is as following:
> > 
> > * mtk_fd-dev-ctx-core.c: Implements common software flow of FD driver.
> > * mtk_fd-v4l2.c: Static FD contexts configuration.
> > * mtk_fd.c: Controls the hardware flow.
> > * mtk_fd-dev.c: Implements context-independent flow.
> > * mtk_fd-ctrl.c: Handles the HW ctrl request from userspace.
> > * mtk_fd-smem-drv.c: Provides the shared memory management required
> > operation. We reserved a memory region for the co-processor and FD to
> > exchange the hardware configuration data.
> > * mtk_fd-v4l2-util.c: Implements V4L2 and vb2 ops.
> > 
> > Jerry-ch Chen (7):
> >   dt-bindings: mt8183: Add binding for FD shared memory
> >   dts: arm64: mt8183: Add FD shared memory node
> >   dt-bindings: mt8183: Added FD-SMEM dt-bindings
> >   dt-bindings: mt8183: Added FD dt-bindings
> >   dts: arm64: mt8183: Add FD nodes
> >   media: platform: Add Mediatek FD driver KConfig
> >   platform: mtk-isp: Add Mediatek FD driver
> > 
> >  .../devicetree/bindings/media/mediatek,fd_smem.txt |   28 +
> >  .../bindings/media/mediatek,mt8183-fd.txt          |   30 +
> >  .../mediatek,reserve-memory-fd_smem.txt            |   44 +
> >  arch/arm64/boot/dts/mediatek/mt8183.dtsi           |   28 +
> >  drivers/media/platform/Kconfig                     |    2 +
> >  drivers/media/platform/mtk-isp/Kconfig             |   10 +
> >  drivers/media/platform/mtk-isp/Makefile            |   16 +
> >  drivers/media/platform/mtk-isp/fd/Makefile         |   38 +
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-core.h    |  157 +++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-ctx.h     |  299 ++++++
> >  .../platform/mtk-isp/fd/mtk_fd-dev-ctx-core.c      |  917 +++++++++++++++++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-dev.c     |  355 +++++++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-dev.h     |  198 ++++
> >  .../media/platform/mtk-isp/fd/mtk_fd-smem-drv.c    |  452 +++++++++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-smem.h    |   25 +
> >  .../media/platform/mtk-isp/fd/mtk_fd-v4l2-util.c   | 1046 ++++++++++++++++++++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-v4l2.c    |  115 +++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd-v4l2.h    |   36 +
> >  drivers/media/platform/mtk-isp/fd/mtk_fd.c         |  730 ++++++++++++++
> >  drivers/media/platform/mtk-isp/fd/mtk_fd.h         |  127 +++
> >  20 files changed, 4653 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,fd_smem.txt
> >  create mode 100644 Documentation/devicetree/bindings/media/mediatek,mt8183-fd.txt
> >  create mode 100644 Documentation/devicetree/bindings/reserved-memory/mediatek,reserve-memory-fd_smem.txt
> >  create mode 100644 drivers/media/platform/mtk-isp/Kconfig
> >  create mode 100644 drivers/media/platform/mtk-isp/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/Makefile
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-core.h
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-ctx.h
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-dev-ctx-core.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-dev.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-dev.h
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-smem-drv.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-smem.h
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-v4l2-util.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-v4l2.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd-v4l2.h
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd.c
> >  create mode 100644 drivers/media/platform/mtk-isp/fd/mtk_fd.h
> > 
>