From: bingbu.cao@intel.com
To: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com,
laurent.pinchart@ideasonboard.com,
andriy.shevchenko@linux.intel.com, hdegoede@redhat.com
Cc: ilpo.jarvinen@linux.intel.com, claus.stovgaard@gmail.com,
tomi.valkeinen@ideasonboard.com, tfiga@chromium.org,
senozhatsky@chromium.org, andreaskleist@gmail.com,
bingbu.cao@intel.com, bingbu.cao@linux.intel.com,
tian.shu.qiu@intel.com, hongju.wang@intel.com
Subject: [PATCH v3 15/17] Documentation: add documentation of Intel IPU6 driver and hardware overview
Date: Thu, 11 Jan 2024 14:55:29 +0800 [thread overview]
Message-ID: <20240111065531.2418836-16-bingbu.cao@intel.com> (raw)
In-Reply-To: <20240111065531.2418836-1-bingbu.cao@intel.com>
From: Bingbu Cao <bingbu.cao@intel.com>
Add a documentation for an overview of IPU6 hardware and describe the main
the components of IPU6 driver.
Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
---
.../driver-api/media/drivers/index.rst | 1 +
.../driver-api/media/drivers/ipu6.rst | 205 ++++++++++++++++++
2 files changed, 206 insertions(+)
create mode 100644 Documentation/driver-api/media/drivers/ipu6.rst
diff --git a/Documentation/driver-api/media/drivers/index.rst b/Documentation/driver-api/media/drivers/index.rst
index c4123a16b5f9..7f6f3dcd5c90 100644
--- a/Documentation/driver-api/media/drivers/index.rst
+++ b/Documentation/driver-api/media/drivers/index.rst
@@ -26,6 +26,7 @@ Video4Linux (V4L) drivers
vimc-devel
zoran
ccs/ccs
+ ipu6
Digital TV drivers
diff --git a/Documentation/driver-api/media/drivers/ipu6.rst b/Documentation/driver-api/media/drivers/ipu6.rst
new file mode 100644
index 000000000000..b6357155c13b
--- /dev/null
+++ b/Documentation/driver-api/media/drivers/ipu6.rst
@@ -0,0 +1,205 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================
+Intel IPU6 Driver
+==================
+
+Author: Bingbu Cao <bingbu.cao@intel.com>
+
+Overview
+=========
+
+Intel IPU6 is the sixth generation of Intel Image Processing Unit used in some
+Intel Chipsets such as Tiger Lake, Jasper Lake, Alder Lake, Raptor Lake and
+Meteor Lake. IPU6 consists of two major systems: Input System (IS) and
+Processing System (PS). IPU6 are visible on the PCI bus as a single device,
+it can be found by ``lspci``:
+
+``0000:00:05.0 Multimedia controller: Intel Corporation Device xxxx (rev xx)``
+
+IPU6 has a 16 MB BAR in PCI configuration Space for MMIO registers which is
+visible for driver.
+
+Buttress
+=========
+
+The IPU6 is connecting to the system fabric with ``Buttress`` which is enabling
+host driver to control the IPU6, it also allows IPU6 access the system memory to
+store and load frame pixel streams and any other metadata.
+
+``Buttress`` mainly manages several system functionalities - power management,
+interrupt handling, firmware authentication and global timer sync.
+
+IS and PS Power flow
+---------------------------
+
+IPU6 driver initialize the IS and PS power up or down request by setting the
+Buttress frequency control register for IS and PS -
+``IPU6_BUTTRESS_REG_IS_FREQ_CTL`` and ``IPU6_BUTTRESS_REG_PS_FREQ_CTL`` in
+function:
+
+.. c:function:: int ipu6_buttress_power(..., bool on)
+
+Buttress forwards the request to Punit, after Punit execute the power up flow,
+buttress indicates driver that IS or PS is powered up by updating the power
+status registers.
+
+.. Note:: IS power up needs take place prior to PS power up, IS power down needs
+ take place after PS power down due to hardware limitation.
+
+
+Interrupt
+------------
+
+IPU6 interrupt can be generated as MSI or INTA, interrupt will be triggered
+when IS, PS, Buttress event or error happen, driver can get the interrupt
+cause by reading the interrupt status register ``BUTTRESS_REG_ISR_STATUS``,
+driver firstly clear the irq status and then call specific IS or PS irq handler.
+
+.. c:function:: irqreturn_t ipu6_buttress_isr(int irq, ...)
+
+Security and firmware authentication
+-------------------------------------
+To address the IPU6 firmware security concerns, the IPU6 firmware needs to
+undergo an authentication process before it is allowed to executed on the IPU6
+internal processors. Driver will work with Converged Security Engine (CSE) to
+complete authentication process. CSE is responsible of authenticating the
+IPU6 firmware, the authenticated firmware binary is copied into an isolated
+memory region. Firmware authentication process is implemented by CSE following
+an IPC handshake with driver. There are some Buttress registers used by CSE and
+driver to communicate with each other as IPC messages.
+
+.. c:function:: int ipu6_buttress_authenticate(...)
+
+Global timer sync
+------------------
+IPU driver initiates a Hammock Harbor synchronization flow each time it starts
+camera operation. IPU will synchronizes an internal counter in the Buttress
+with a copy of SoC time, this counter keeps the updated time until camera
+operation is stopped. Driver can use this time counter to calibrate the
+timestamp based on the timestamp in response event from firmware.
+
+.. c:function:: int ipu6_buttress_start_tsc_sync(...)
+
+
+DMA and MMU
+============
+
+IPU6 has its own scalar processor where the firmware run at, it has
+an internal 32-bits virtual address space. IPU6 has MMU address translation
+hardware to allow that scalar process access the internal memory and external
+system memory through IPU6 virtual address. The address translation is
+based on two levels of page lookup tables stored in system memory which are
+maintained by IPU6 driver. IPU6 driver sets the level-1 page table base address
+to MMU register and allow MMU to lookup the page table.
+
+IPU6 driver exports its own DMA operations. Driver will update the page table
+entries for each DMA operation and invalidate the MMU TLB after each unmap and
+free.
+
+.. code-block:: none
+
+ const struct dma_map_ops ipu6_dma_ops = {
+ .alloc = ipu6_dma_alloc,
+ .free = ipu6_dma_free,
+ .mmap = ipu6_dma_mmap,
+ .map_sg = ipu6_dma_map_sg,
+ .unmap_sg = ipu6_dma_unmap_sg,
+ ...
+ };
+
+.. Note:: IPU6 MMU works behind IOMMU, so for each IPU6 DMA ops, driver will
+ call generic PCI DMA ops to ask IOMMU to do the additional mapping
+ if VT-d enabled.
+
+
+Firmware file format
+=====================
+
+IPU6 release the firmware in Code Partition Directory (CPD) file format. The
+CPD firmware contains a CPD header, several CPD entries and CPD components.
+CPD component includes 3 entries - manifest, metadata and module data. Manifest
+and metadata are defined by CSE and used by CSE for authentication. Module data
+is defined by IPU6 which holds the binary data of firmware called package
+directory. IPU6 driver (``ipu6-cpd.c``) parses and validates the CPD firmware
+file and get the package directory binary data of IPU6 firmware, copy it to
+specific DMA buffer and sets its base address to Buttress ``FW_SOURCE_BASE``
+register, CSE will do authentication for this firmware binary.
+
+
+Syscom interface
+================
+
+IPU6 driver communicates with firmware via syscom ABI. Syscom is an
+inter-processor communication mechanism between IPU scalar processor and CPU.
+There are a number of resources shared between firmware and software.
+A system memory region where the message queues reside, firmware can access the
+memory region via IPU MMU. Syscom queues are FIFO fixed depth queues with
+configurable elements ``token`` (message). There is also a common IPU MMIO
+registers where the queue read and write indices reside. Software and firmware
+work as producer and consumer of tokens in queue, and update the write and read
+indices separately when sending or receiving each message.
+
+IPU6 driver must prepare and configure the number of input and output queues,
+configure the count of tokens per queue and the size of per token before
+initiate and start the communication with firmware, firmware and software must
+use same configurations. IPU6 Buttress has a number of firmware boot parameter
+registers which can be used to store the address of configuration and initiate
+the Syscom state, then driver can request firmware to start and run via setting
+the scalar processor control status register.
+
+
+Input System
+==============
+
+IPU6 input system consists of MIPI D-PHY and several CSI receiver controllers,
+it can capture image pixel data from camera sensors or other MIPI CSI output
+devices.
+
+D-PHYs and CSI-2 ports lane mapping
+-----------------------------------
+
+IPU6 integrates different D-PHY IPs on different SoCs, on Tiger Lake and Alder
+Lake, IPU6 integrates MCD10 D-PHY, IPU6SE on Jasper Lake integrates JSL D-PHY
+and IPU6EP on Meteor Lake integrates a Synopsys DWC D-PHY. There is an adaption
+layer between D-PHY and CSI receiver controller which includes port
+configuration, PHY wrapper or private test interfaces for D-PHY. There are 3
+D-PHY drivers ``ipu6-isys-mcd-phy.c``, ``ipu6-isys-jsl-phy.c`` and
+``ipu6-isys-dwc-phy.c`` program the above 3 D-PHYs in IPU6.
+
+Different IPU6 version has different D-PHY lanes mappings, On Tiger Lake, there
+are 12 data lanes and 8 clock lanes, IPU6 support maximum 8 CSI-2 ports, see
+the ppi mmapping in ``ipu6-isys-mcd-phy.c`` for more information. On Jasper Lake
+and Alder Lake, D-PHY has 8 data lanes and 4 clock lanes, IPU6 support maximum 4
+CSI-2 ports. For Meteor Lake, D-PHY has 12 data lanes and 6 clock lanes, IPU6
+support maximum 6 CSI-2 ports.
+
+.. Note:: Each adjacent CSI ports work as a pair and share the data lanes.
+ For example, for CSI port 0 and 1, CSI port 0 support maximum 4
+ data lanes, CSI port 1 support maximum 2 data lanes, CSI port 0
+ with 2 data lanes can work together with CSI port 1 with 2 data lanes.
+ If trying to use CSI port 0 with 4 lanes, CSI port 1 will not be
+ available as the 4 data lanes are shared by CSI port 0 and 1. Same
+ scenario is also applied for CSI port 2/3, 4/5 and 7/8.
+
+IS firmware ABIs
+----------------
+
+IPU6 firmware define a series of ABIs to software. In general, software firstly
+prepare the stream configuration ``struct ipu6_fw_isys_stream_cfg_data_abi``
+and send the configuration to firmware via sending ``STREAM_OPEN`` command.
+Stream configuration includes input pins and output pins, input pin
+``struct ipu6_fw_isys_input_pin_info_abi`` defines the resolution and data type
+of input source, output pin ``struct ipu6_fw_isys_output_pin_info_abi``
+defines the output resolution, stride and frame format, etc. Once driver get the
+interrupt from firmware that indicates stream open successfully, driver will
+send the ``STREAM_START`` and ``STREAM_CAPTURE`` command to request firmware to
+start capturing image frames. ``STREAM_CAPTURE`` command queues the buffers to
+firmware with ``struct ipu6_fw_isys_frame_buff_set``, software then wait the
+interrupt and response from firmware, ``PIN_DATA_READY`` means data ready
+on specific output pin and then software return the buffers to user.
+
+.. Note:: See :ref:`Examples<ipu6_isys_capture_examples>` about how to do
+ capture by IPU6 IS driver.
+
+
--
2.42.0
next prev parent reply other threads:[~2024-01-11 6:47 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-11 6:55 [PATCH v3 00/15] Intel IPU6 and IPU6 input system drivers bingbu.cao
2024-01-11 6:55 ` [PATCH v3 01/17] media: intel/ipu6: add Intel IPU6 PCI device driver bingbu.cao
2024-01-15 13:36 ` Hans de Goede
2024-01-15 13:45 ` Sakari Ailus
2024-01-11 6:55 ` [PATCH v3 02/17] media: intel/ipu6: add IPU auxiliary devices bingbu.cao
2024-01-11 6:55 ` [PATCH v3 03/17] media: intel/ipu6: add IPU6 buttress interface driver bingbu.cao
2024-01-11 6:55 ` [PATCH v3 04/17] media: intel/ipu6: CPD parsing for get firmware components bingbu.cao
2024-01-11 6:55 ` [PATCH v3 05/17] media: intel/ipu6: add IPU6 DMA mapping API and MMU table bingbu.cao
2024-01-11 6:55 ` [PATCH v3 06/17] media: intel/ipu6: add syscom interfaces between firmware and driver bingbu.cao
2024-01-11 6:55 ` [PATCH v3 07/17] media: intel/ipu6: input system ABI " bingbu.cao
2024-01-15 12:17 ` Sakari Ailus
2024-01-11 6:55 ` [PATCH v3 08/17] media: intel/ipu6: add IPU6 CSI2 receiver v4l2 sub-device bingbu.cao
2024-01-11 6:55 ` [PATCH v3 09/17] media: intel/ipu6: add the CSI2 DPHY implementation bingbu.cao
2024-01-11 6:55 ` [PATCH v3 10/17] media: intel/ipu6: add input system driver bingbu.cao
2024-01-11 14:05 ` Sakari Ailus
2024-02-07 9:36 ` Andreas Helbech Kleist
2024-02-12 18:31 ` Sakari Ailus
2024-02-15 6:43 ` Andreas Helbech Kleist
2024-02-15 8:06 ` Sakari Ailus
2024-01-11 6:55 ` [PATCH v3 11/17] media: intel/ipu6: input system video capture nodes bingbu.cao
2024-03-08 9:58 ` Andreas Helbech Kleist
2024-01-11 6:55 ` [PATCH v3 12/17] media: add Kconfig and Makefile for IPU6 bingbu.cao
2024-01-11 6:55 ` [PATCH v3 13/17] MAINTAINERS: add maintainers for Intel IPU6 input system driver bingbu.cao
2024-01-11 6:55 ` [PATCH v3 14/17] Documentation: add Intel IPU6 ISYS driver admin-guide doc bingbu.cao
2024-01-11 6:55 ` bingbu.cao [this message]
2024-01-11 6:55 ` [PATCH v3 16/17] media: ipu6/isys: support line-based metadata capture support bingbu.cao
2024-01-11 12:52 ` Sakari Ailus
2024-01-11 21:20 ` kernel test robot
2024-01-12 3:31 ` kernel test robot
2024-01-11 6:55 ` [PATCH v3 17/17] media: ipu6/isys: support new v4l2 subdev state APIs bingbu.cao
2024-01-11 12:37 ` Sakari Ailus
2024-01-16 16:12 ` [PATCH v3 00/15] Intel IPU6 and IPU6 input system drivers Hans de Goede
2024-01-16 16:18 ` Laurent Pinchart
2024-01-16 16:38 ` Hans de Goede
2024-01-16 16:42 ` Laurent Pinchart
2024-01-16 16:43 ` Hans de Goede
2024-01-16 16:54 ` Sakari Ailus
2024-01-16 16:57 ` Hans de Goede
2024-01-16 17:48 ` Laurent Pinchart
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240111065531.2418836-16-bingbu.cao@intel.com \
--to=bingbu.cao@intel.com \
--cc=andreaskleist@gmail.com \
--cc=andriy.shevchenko@linux.intel.com \
--cc=bingbu.cao@linux.intel.com \
--cc=claus.stovgaard@gmail.com \
--cc=hdegoede@redhat.com \
--cc=hongju.wang@intel.com \
--cc=ilpo.jarvinen@linux.intel.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-media@vger.kernel.org \
--cc=sakari.ailus@linux.intel.com \
--cc=senozhatsky@chromium.org \
--cc=tfiga@chromium.org \
--cc=tian.shu.qiu@intel.com \
--cc=tomi.valkeinen@ideasonboard.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox