RFC: V4L2 driver for Qualcomm MSM camera.

["Shuzhen Wang" <shuzhenw@codeaurora.org>]

Hello, 

This is the architecture overview we put together for Qualcomm MSM camera
support in linux-media tree. Your comments are very much appreciated!


Introduction
============

This is the video4linux driver for Qualcomm MSM (Mobile Station Modem)
camera.

The driver supports video and camera functionality on Qualcomm MSM SoC.
It supports camera sensors provided by OEMs with parallel/MIPI
interfaces, and operates in continuous streaming, snapshot, or video
recording modes.

Hardware description
====================

MSM camera hardware contains the following components:
1. One or more camera sensors controlled via I2C bus, and data outputs
on AXI or MIPI bus.
2. Video Front End (VFE) core is an image signal processing hardware
pipeline that takes the sensor output, does the necessary processing,
and outputs to a buffer for V4L2 application. VFE is clocked by PCLK
(pixel clock) from the sensor.

Software description
====================

The driver encapsulates the low-level hardware management and aligns
the interface to V4L2 driver framework. There is also a user space
camera service daemon which handles events from driver and changes
settings accordingly.

During boot-up, the sensor is probed for and if the probe succeeds
/dev/video0 and /dev/msm_camera/config0 device nodes are created. The
/dev/video0 node is used for video buffer allocation in the kernel and for
receiving V4L2 ioctls for controlling the camera hardware (VFE, sensors).
The /dev/msm_camera/config0 node is used for sending commands and other
statistics available from the hardware to the camera service daemon. Note
that if more than one camera sensor is detected, there will be /dev/video1
and /dev/msm_camera/config1 device nodes created as well.

Design
======

For MSM camera IC, significant portion of image processing and optimization
codes are proprietary, so they cannot sit in kernel space. This plays an
important role when making design decisions.

Our design is to have a light-weighted kernel driver, and put the
proprietary code in a user space daemon. The daemon polls on events
from /dev/msm_camera/config0 device in the form of v4l2_event. The events
could either be asynchronous (generated by the hardware), or synchronous
(control command from the application). Based on the events it receives,
the daemon will send appropriate control commands to the hardware.

   +-------------+        +----------------+
   | Application |        | Service Daemon |
   +-------------+        +----------------+     User Space
 ..........................................................
   +--------------------------------------+      Kernel Space
   |                V4L2                  |
   +--------------------------------------+
   +---------+     +--------+     +-------+
   | VFE/ISP |     | Sensor |     | Flash |
   +---------+     +--------+     +-------+

Power Management
================

None at this point.

SMP/multi-core
==============

Mutex is used for synchronization of threads accessing the driver
simultaneously. Between hardware interrupt handler and threads,
we use spinlock to make sure locking is done properly.

Security
========

None.

Interface
=========

The driver uses V4L2 API for user kernel interaction. Refer to
http://v4l2spec.bytesex.org/spec-single/v4l2.html.

Between camera service daemon and the driver, we have customized IOCTL
commands associated with /dev/msm_camera/config0 node, that controls MSM
camera hardware. The list of IOCTLs are (declarations can be found in
include/media/msm_camera.h):

MSM_CAM_IOCTL_GET_SENSOR_INFO
        Get the basic information such as name, flash support for the
        detected sensor.
MSM_CAM_IOCTL_SENSOR_IO_CFG
        Get or set sensor configurations: fps, line_pf, pixels_pl,
        exposure and gain, etc. The setting is stored in sensor_cfg_data
        structure.
MSM_CAM_IOCTL_CONFIG_VFE
        Change settings of different components of VFE hardware.
MSM_CAM_IOCTL_CTRL_CMD_DONE
        Notify the driver that the ctrl command is finished.
MSM_CAM_IOCTL_RELEASE_STATS_BUFFER
        Notify the driver that the service daemon has done processing the
        stats buffer, and is returning it to the driver.
MSM_CAM_IOCTL_AXI_CONFIG
        Configure AXI bus parameters (frame buffer addresses, offsets) to
        the VFE hardware.

Driver parameters
=================

None.

Config options
==============

MSM_CAMERA in drivers/media/video/msm/Kconfig file

Dependencies
============

PMIC, I2C, Clock, GPIO

User space utilities
====================

A daemon process in the user space called mm-qcamera-daemon is polling
on events from the driver. This daemon is required in order for the V4L2
client application to run, and it's started by the init script.

Other
=====

To do
=====

1. Eliminate creation of /dev/msm_camera/config0 by routing private
ioctls through /dev/video0.
2. Create sub devices for sensor and VFE.



--
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.
--