Re: [PATCH v3 0/3] media: pci: intel: ivsc: Add driver of Intel Visual Sensing Controller(IVSC)

[Sakari Ailus <sakari.ailus@linux.intel.com>]

Hi Wentong,

On Mon, Mar 27, 2023 at 02:23:05PM +0800, Wentong Wu wrote:
> Intel Visual Sensing Controller (IVSC), codenamed "Clover Falls", is a
> companion chip designed to provide secure and low power vision capability
> to IA platforms. IVSC is available in existing commercial platforms from
> multiple OEMs.
> 
> The primary use case of IVSC is to bring in context awareness. IVSC
> interfaces directly with the platform main camera sensor via a CSI-2 link
> and processes the image data with the embedded AI engine. The detected
> events are sent over I2C to ISH (Intel Sensor Hub) for additional data
> fusion from multiple sensors. The fusion results are used to implement
> advanced use cases like:
>  - Face detection to unlock screen
>  - Detect user presence to manage backlight setting or waking up system
> 
> Since the Image Processing Unit(IPU) used on the host processor needs to
> configure the CSI-2 link in normal camera usages, the CSI-2 link and
> camera sensor can only be used in mutually-exclusive ways by host IPU and
> IVSC. By default the IVSC owns the CSI-2 link and camera sensor. The IPU
> driver can take ownership of the CSI-2 link and camera sensor using
> interfaces exported via v4l2 sub-device.
> 
> Switching ownership requires an interface with two different hardware
> modules inside IVSC. The software interface to these modules is via Intel
> MEI (The Intel Management Engine) commands. These two hardware modules
> have two different MEI UUIDs to enumerate. These hardware modules are:
>  - ACE (Algorithm Context Engine): This module is for algorithm computing
> when IVSC owns camera sensor. Also ACE module controls camera sensor's
> ownership. This hardware module is used to set ownership of camera sensor.
>  - CSI (Camera Serial Interface): This module is used to route camera
> sensor data either to IVSC or to host for IPU driver and application.
> 
> IVSC also provides a privacy mode. When privacy mode is turned on,
> camera sensor can't be used. This means that both ACE and host IPU can't
> get image data. And when this mode is turned on, users are informed via
> v4l2 control API.
> 
> In summary, to acquire ownership of camera by IPU driver, first ACE
> module needs to be informed of ownership and then to setup MIPI CSI-2
> link for the camera sensor and IPU.
> 
> Implementation:
> There are two different drivers to handle ACE and CSI hardware modules
> inside IVSC.
>  - ivsc_csi: MEI client driver to send commands and receive notifications
> from CSI module.
>  - ivsc_ace: MEI client driver to send commands and get status from ACE
> module.
> Interface is exposed via v4l2 sub-devcie APIs to acquire and release
> camera sensor and CSI-2 link.

Thanks for the update.

Could you elaborate the decision of keeping the csi_bridge entirely
separate from the cio2_bridge (to be turned to ipu_bridge first)? Both are
doing essentially the same and using the same data structures, aren't they?

-- 
Kind regards,

Sakari Ailus